org.apache.beam.sdk.transforms

Annotation Type DoFn.GetPartition

@Documented
 @Retention(value=RUNTIME)
 @Target(value=METHOD)
 @Experimental(value=SPLITTABLE_DO_FN)
public static @interface DoFn.GetPartition

Annotation for the method that returns the corresponding partition identifier for an element and restriction pair.

Signature: byte[] getPartitition(InputT element, RestrictionT restriction);

Returns an immutable representation of the partition identifier as a byte[].

By default, the partition identifier is represented as the encoded element and restriction pair and should only be provided if the splittable DoFn can only provide a size over a shared resource such as a message queue that potentially multiple element and restriction pairs are doing work on. The partition identifier is used by runners for various size calculations. Sizes reported with the same partition identifier represent a point in time reporting of the size for that partition. For example, a runner can compute a global size by summing all reported sizes over all unique partition identifiers while it can compute the size of a specific partition based upon the last reported value.

For example splittable DoFns which consume elements from:

• a globally shared resource such as a Pubsub queue should set this to "".
• a shared partitioned resource should use the partition identifier.
• a uniquely partitioned resource such as a file and offset range should not override this since the default element and restriction pair should suffice.