apache_beam.runners.direct.watermark_manager module

Manages watermarks of PCollections and AppliedPTransforms.

class apache_beam.runners.direct.watermark_manager.WatermarkManager(clock, root_transforms, value_to_consumers, transform_keyed_states)

Bases: future.types.newobject.newobject

For internal use only; no backwards-compatibility guarantees.

Tracks and updates watermarks for all AppliedPTransforms.

WATERMARK_POS_INF = Timestamp(9223372036854.775807)

WATERMARK_NEG_INF = Timestamp(-9223372036854.775808)

get_watermarks(applied_ptransform)

Gets the input and output watermarks for an AppliedPTransform.

If the applied_ptransform has not processed any elements, return a watermark with minimum value.

Parameters:applied_ptransform – AppliedPTransform to get the watermarks for. Returns:A snapshot (TransformWatermarks) of the input watermark and output watermark for the provided transform.

update_watermarks(completed_committed_bundle, applied_ptransform, completed_timers, outputs, unprocessed_bundles, keyed_earliest_holds, side_inputs_container)

extract_all_timers()

Extracts fired timers for all transforms and reports if there are any timers set.