Package org.apache.beam.sdk.transforms

Annotation Interface DoFn.NewWatermarkEstimator

Enclosing class:
DoFn<InputT extends @Nullable Object,OutputT extends @Nullable Object>
@Documented @Retention(RUNTIME) @Target(METHOD) public static @interface DoFn.NewWatermarkEstimator
Annotation for the method that creates a new WatermarkEstimator for the watermark state of a splittable DoFn.

Signature: MyWatermarkEstimator newWatermarkEstimator(<optional arguments>);

If the return type is a subtype of TimestampObservingWatermarkEstimator then the timestamp of each element output from this DoFn is provided to the watermark estimator.

This method must satisfy the following constraints:

  • The return type must be a subtype of WatermarkEstimator<WatermarkEstimatorStateT>. It is suggested to use as narrow of a return type definition as possible (for example prefer to use a square type over a shape type as a square is a type of a shape).
  • If one of its arguments is tagged with the DoFn.WatermarkEstimatorState annotation, then it will be passed the current watermark estimator state; the argument must be of type WatermarkEstimatorStateT.
  • If one of its arguments is tagged with the DoFn.Element annotation, then it will be passed the current element being processed; the argument must be of type InputT. Note that automatic conversion of Rows and DoFn.FieldAccess parameters are currently unsupported.
  • If one of its arguments is tagged with the DoFn.Restriction annotation, then it will be passed the current restriction being processed; the argument must be of type RestrictionT.
  • If one of its arguments is tagged with the DoFn.Timestamp annotation, then it will be passed the timestamp of the current element being processed; the argument must be of type Instant.
  • If one of its arguments is a subtype of BoundedWindow, then it will be passed the window of the current element. When applied by ParDo the subtype of BoundedWindow must match the type of windows on the input PCollection. If the window is not accessed a runner may perform additional optimizations.
  • If one of its arguments is of type PaneInfo, then it will be passed information about the current triggering pane.
  • If one of the parameters is of type PipelineOptions, then it will be passed the options for the current pipeline.