apache_beam.runners.direct.watermark_manager module

Manages watermarks of PCollections and AppliedPTransforms.

class apache_beam.runners.direct.watermark_manager.WatermarkManager(clock, root_transforms, value_to_consumers, transform_keyed_states)[source]

Bases: future.types.newobject.newobject

For internal use only; no backwards-compatibility guarantees.

Tracks and updates watermarks for all AppliedPTransforms.

WATERMARK_POS_INF = Timestamp(9223372036854.775807)
WATERMARK_NEG_INF = Timestamp(-9223372036854.775808)
get_watermarks(applied_ptransform)[source]

Gets the input and output watermarks for an AppliedPTransform.

If the applied_ptransform has not processed any elements, return a watermark with minimum value.

Parameters:applied_ptransform – AppliedPTransform to get the watermarks for.
Returns:A snapshot (TransformWatermarks) of the input watermark and output watermark for the provided transform.
update_watermarks(completed_committed_bundle, applied_ptransform, completed_timers, outputs, unprocessed_bundles, keyed_earliest_holds, side_inputs_container)[source]
extract_all_timers()[source]

Extracts fired timers for all transforms and reports if there are any timers set.