apache_beam.utils.timestamp module

Timestamp utilities.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.utils.timestamp.Timestamp(seconds: int | float = 0, subseconds: int | float = 0, precision: int = 6, *, micros: int | float | None = None)[source]

Bases: object

Represents a Unix second timestamp with configurable subsecond precision.

Can be treated in common timestamp arithmetic operations as a numeric type.

Internally stores the timestamp as an int of floored seconds since the epoch plus a non-negative int subsecond value interpreted with the specified precision where (0 <= subseconds < 10**precision).

Integer storage is necessary since floating point values lose precision when storing values, especially after arithmetic operations (for example, 10000000 % 0.1 evaluates to 0.0999999994448885).

precision is the number of decimal digits used to represent the fraction of a second (e.g. 3 for millis, 6 for micros, 9 for nanos). Defaults to microseconds. If seconds is a float, the fractional part will be captured up to precision digits.

Lossy conversion operations will throw an error unless allow_lossy_conversion=True is specified (e.g. see to_utc_datetime).

MICROS_PRECISION = 6

NANOS_PRECISION = 9

static of(seconds: int | float | Timestamp) → Timestamp[source]

Return the Timestamp for the given number of seconds.

If the input is already a Timestamp, the input itself will be returned.

Parameters:: seconds – Number of seconds as int, float, long, or Timestamp.
Returns:: Corresponding Timestamp object.

static now() → Timestamp[source]

classmethod from_utc_datetime(dt: datetime) → Timestamp[source]

Create a Timestamp instance from a datetime.datetime object.

Parameters:: dt – A datetime.datetime object in UTC (offset-aware).

classmethod from_rfc3339(rfc3339: str) → Timestamp[source]

Create a Timestamp instance from an RFC 3339 compliant string.

Fractional seconds up to microseconds produce a microsecond-precision Timestamp; a longer fraction (up to nanoseconds) produces a Timestamp whose precision matches the number of fractional digits.

Note

All timezones are implicitly converted to UTC.

Parameters:: rfc3339 – String in RFC 3339 form.

seconds() → int[source]: Returns the timestamp in seconds.

subseconds() → int[source]

Returns the fraction of a second, in units of 10**-precision seconds.

Always non-negative and less than 10**precision

precision() → int[source]: Returns the precision of this Timestamp.

property micros: int: Returns the total number of microseconds since the epoch.

property nanos: int: Returns the total number of nanoseconds since the epoch.

to_precision(precision: int, allow_lossy_conversion: bool = False) → Timestamp[source]

Returns this Timestamp converted to the given precision.

Increasing precision is always lossless. Decreasing precision raises ValueError if this timestamp has a non-zero component below the target precision, unless allow_lossy_conversion is True, in which case the timestamp is truncated (floored) to the target precision.

predecessor() → Timestamp[source]: Returns the largest timestamp smaller than self, at this precision.

successor() → Timestamp[source]: Returns the smallest timestamp larger than self, at this precision.

to_utc_datetime(has_tz: bool = False, allow_lossy_conversion: bool = False) → datetime[source]

Returns a datetime.datetime object of UTC for this Timestamp.

Note that this method returns a datetime.datetime object without a timezone info by default, as builtin datetime.datetime.utcnow method. If this is used as part of the processed data, one should set has_tz=True to avoid offset due to default timezone mismatch.

Parameters:

has_tz – whether the timezone info is attached, default to False.
allow_lossy_conversion – must be set to True to convert a timestamp with precision above microseconds, since datetime.datetime only supports microsecond resolution; the result is truncated (floored) to whole microseconds.

Returns:

a datetime.datetime object of UTC for this Timestamp.

Raises:

ValueError – if this timestamp has precision above microseconds and allow_lossy_conversion is not True.

to_rfc3339() → str[source]: Returns an RFC 3339 string for this Timestamp.

to_proto() → Timestamp[source]: Returns the google.protobuf.timestamp_pb2 representation.

static from_proto(timestamp_proto: Timestamp) → Timestamp[source]

Creates a Timestamp from a google.protobuf.timestamp_pb2.

The returned Timestamp has microsecond precision if the proto’s nanos are microsecond-aligned, and nanosecond precision otherwise.

class apache_beam.utils.timestamp.Duration(seconds: int | float = 0, micros: int | float = 0)[source]

Bases: object

Represents a second duration with microsecond granularity.

Can be treated in common arithmetic operations as a numeric type.

Internally stores a time interval as an int of microseconds. This strategy is necessary since floating point values lose precision when storing values, especially after arithmetic operations (for example, 10000000 % 0.1 evaluates to 0.0999999994448885).

static of(seconds: int | float | Duration) → Duration[source]

Return the Duration for the given number of seconds since Unix epoch.

If the input is already a Duration, the input itself will be returned.

Parameters:: seconds – Number of seconds as int, float or Duration.
Returns:: Corresponding Duration object.

to_proto() → Duration[source]: Returns the google.protobuf.duration_pb2 representation.

static from_proto(duration_proto: Duration) → Duration[source]

Creates a Duration from a google.protobuf.duration_pb2.

Note that the google has a sub-second resolution of nanoseconds whereas this class has a resolution of microsends. This class will truncate the nanosecond resolution down to the microsecond.