apache_beam.runners.direct.watermark_manager module

Manages watermarks of PCollections and AppliedPTransforms.

class apache_beam.runners.direct.watermark_manager.WatermarkManager(clock, root_transforms, value_to_consumers, transform_keyed_states)

Bases: object

For internal use only; no backwards-compatibility guarantees.

Tracks and updates watermarks for all AppliedPTransforms.

WATERMARK_POS_INF = Timestamp(9223372036854.775000)

WATERMARK_NEG_INF = Timestamp(-9223372036854.775000)

get_watermarks(applied_ptransform: AppliedPTransform) → _TransformWatermarks

Gets the input and output watermarks for an AppliedPTransform.

If the applied_ptransform has not processed any elements, return a watermark with minimum value.

Parameters:

applied_ptransform – AppliedPTransform to get the watermarks for.

Returns:

A snapshot (TransformWatermarks) of the input watermark and output watermark for the provided transform.

update_watermarks(completed_committed_bundle: _Bundle, applied_ptransform: AppliedPTransform, completed_timers, outputs, unprocessed_bundles, keyed_earliest_holds, side_inputs_container)

extract_all_timers() → Tuple[List[Tuple[AppliedPTransform, List[TimerFiring]]], bool]

Extracts fired timers for all transforms and reports if there are any timers set.