org.apache.beam.sdk.transforms

Annotation Type DoFn.ProcessElement

@Documented
 @Retention(value=RUNTIME)
 @Target(value=METHOD)
public static @interface DoFn.ProcessElement

Annotation for the method to use for processing elements. A subclass of DoFn must have a method with this annotation.

The signature of this method must satisfy the following constraints:

• If one of its arguments is a RestrictionTracker, then it is a splittable DoFn subject to the separate requirements described below. Items below are assuming this is not a splittable DoFn.
• If one of its arguments is tagged with the DoFn.Element annotation, then it will be passed the current element being processed; the argument type must match the input type of this DoFn.
• If one of its arguments is tagged with the DoFn.Timestamp annotation, then it will be passed the timestamp of the current element being processed; the argument must be of type Instant.
• If one of its arguments is a subtype of BoundedWindow, then it will be passed the window of the current element. When applied by ParDo the subtype of BoundedWindow must match the type of windows on the input PCollection. If the window is not accessed a runner may perform additional optimizations.
• If one of its arguments is of type PaneInfo, then it will be passed information about the current triggering pane.
• If one of the parameters is of type PipelineOptions, then it will be passed the options for the current pipeline.
• If one of the parameters is of type DoFn.OutputReceiver, then it will be passed an output receiver for outputting elements to the default output.
• If one of the parameters is of type DoFn.MultiOutputReceiver, then it will be passed an output receiver for outputting to multiple tagged outputs.
• If one of the parameters is of type DoFn.BundleFinalizer, then it will be passed a mechanism to register a callback that will be invoked after the runner successfully commits the output of this bundle. See Apache Beam Portability API: How to Finalize Bundles for further details.
• It must return void.

Splittable DoFn's

A DoFn is splittable if its DoFn.ProcessElement method has a parameter whose type is of RestrictionTracker. This is an advanced feature and an overwhelming majority of users will never need to write a splittable DoFn.

Not all runners support Splittable DoFn. See the capability matrix.

See the proposal for an overview of the involved concepts (splittable DoFn, restriction, restriction tracker).

If a DoFn is splittable, the following constraints must be respected:

• It must define a DoFn.GetInitialRestriction method.
• It may define a DoFn.GetSize method.
• It may define a DoFn.SplitRestriction method.
• It may define a DoFn.NewTracker method returning a subtype of RestrictionTracker<R> where R is the restriction type returned by DoFn.GetInitialRestriction. This method is optional in case the restriction type returned by DoFn.GetInitialRestriction implements HasDefaultTracker.
• It may define a DoFn.GetRestrictionCoder method.
• The type of restrictions used by all of these methods must be the same.
• Its DoFn.ProcessElement method may return a DoFn.ProcessContinuation to indicate whether there is more work to be done for the current element.
• Its DoFn.ProcessElement method must not use any extra context parameters, such as BoundedWindow.
• The DoFn itself may be annotated with DoFn.BoundedPerElement or DoFn.UnboundedPerElement, but not both at the same time. If it's not annotated with either of these, it's assumed to be DoFn.BoundedPerElement if its DoFn.ProcessElement method returns void and DoFn.UnboundedPerElement if it returns a DoFn.ProcessContinuation.

A non-splittable DoFn must not define any of these methods.