@Documented @Retention(value=RUNTIME) @Target(value=METHOD) public static @interface DoFn.ProcessElement
DoFnmust have a method with this annotation.
The signature of this method must satisfy the following constraints:
RestrictionTracker, then it is a splittable
DoFnsubject to the separate requirements described below. Items below are assuming this is not a splittable
DoFn.Elementannotation, then it will be passed the current element being processed; the argument type must match the input type of this DoFn.
DoFn.Timestampannotation, then it will be passed the timestamp of the current element being processed; the argument must be of type
BoundedWindow, then it will be passed the window of the current element. When applied by
ParDothe subtype of
BoundedWindowmust match the type of windows on the input
PCollection. If the window is not accessed a runner may perform additional optimizations.
PaneInfo, then it will be passed information about the current triggering pane.
PipelineOptions, then it will be passed the options for the current pipeline.
DoFn.OutputReceiver, then it will be passed an output receiver for outputting elements to the default output.
DoFn.MultiOutputReceiver, then it will be passed an output receiver for outputting to multiple tagged outputs.
DoFn.BundleFinalizer, then it will be passed a mechanism to register a callback that will be invoked after the runner successfully commits the output of this bundle. See Apache Beam Portability API: How to Finalize Bundles for further details.
DoFn is splittable if its
DoFn.ProcessElement method has a parameter
whose type is of
RestrictionTracker. This is an advanced feature and an overwhelming
majority of users will never need to write a splittable
Not all runners support Splittable DoFn. See the capability matrix.
See the proposal for an overview of the involved concepts (splittable DoFn, restriction, restriction tracker).
DoFn is splittable, the following constraints must be respected:
DoFn.NewTrackermethod returning a subtype of
Ris the restriction type returned by
DoFn.GetInitialRestriction. This method is optional in case the restriction type returned by
DoFn.ProcessElementmethod may return a
DoFn.ProcessContinuationto indicate whether there is more work to be done for the current element.
DoFn.ProcessElementmethod must not use any extra context parameters, such as
DoFnitself may be annotated with
DoFn.UnboundedPerElement, but not both at the same time. If it's not annotated with either of these, it's assumed to be
DoFn.UnboundedPerElementif it returns a
DoFn must not define any of these methods.