@Documented
@Retention(value=RUNTIME)
@Target(value=METHOD)
public static @interface DoFn.ProcessElement
DoFn
must have
a method with this annotation.
The signature of this method must satisfy the following constraints:
RestrictionTracker
, then it is a splittable DoFn
subject to the
separate requirements described below. Items below are assuming this is not a splittable
DoFn
.
DoFn.Element
annotation, then it will be
passed the current element being processed; the argument type must match the input type
of this DoFn.
DoFn.Timestamp
annotation, then it will be
passed the timestamp of the current element being processed; the argument must be of type
Instant
.
BoundedWindow
, then it will be passed the
window of the current element. When applied by ParDo
the subtype of BoundedWindow
must match the type of windows on the input PCollection
. If the
window is not accessed a runner may perform additional optimizations.
PaneInfo
, then it will be passed information
about the current triggering pane.
PipelineOptions
, then it will be passed the
options for the current pipeline.
DoFn.OutputReceiver
, then it will be passed an
output receiver for outputting elements to the default output.
DoFn.MultiOutputReceiver
, then it will be passed
an output receiver for outputting to multiple tagged outputs.
DoFn.BundleFinalizer
, then it will be passed a
mechanism to register a callback that will be invoked after the runner successfully
commits the output of this bundle. See Apache Beam Portability API: How to
Finalize Bundles for further details.
void
.
A DoFn
is splittable if its DoFn.ProcessElement
method has a parameter
whose type is of RestrictionTracker
. This is an advanced feature and an overwhelming
majority of users will never need to write a splittable DoFn
.
Not all runners support Splittable DoFn. See the capability matrix.
See the proposal for an overview of the involved concepts (splittable DoFn, restriction, restriction tracker).
If a DoFn
is splittable, the following constraints must be respected:
DoFn.GetInitialRestriction
method.
DoFn.SplitRestriction
method.
DoFn.NewTracker
method returning a subtype of RestrictionTracker<R>
where R
is the restriction type returned by DoFn.GetInitialRestriction
. This method is optional in case the restriction type returned by
DoFn.GetInitialRestriction
implements HasDefaultTracker
.
DoFn.GetRestrictionCoder
method.
DoFn.ProcessElement
method may return a DoFn.ProcessContinuation
to
indicate whether there is more work to be done for the current element.
DoFn.ProcessElement
method must not use any extra context parameters, such
as BoundedWindow
.
DoFn
itself may be annotated with DoFn.BoundedPerElement
or DoFn.UnboundedPerElement
, but not both at the same time. If it's not annotated with either of
these, it's assumed to be DoFn.BoundedPerElement
if its DoFn.ProcessElement
method
returns void
and DoFn.UnboundedPerElement
if it returns a DoFn.ProcessContinuation
.
A non-splittable DoFn
must not define any of these methods.