Package org.apache.beam.sdk.extensions.ordered

Class OrderedEventProcessor<EventT,EventKeyT,ResultT,StateT extends MutableState<EventT,ResultT>>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<EventKeyT,KV<Long,EventT>>>,OrderedEventProcessorResult<EventKeyT,ResultT,EventT>>

org.apache.beam.sdk.extensions.ordered.OrderedEventProcessor<EventT,EventKeyT,ResultT,StateT>

Type Parameters:

EventT - type of event

EventKeyT - type of event key

StateT - type of the state

All Implemented Interfaces:

Serializable, HasDisplayData

public abstract class OrderedEventProcessor<EventT,EventKeyT,ResultT,StateT extends MutableState<EventT,ResultT>>
extends PTransform<PCollection<KV<EventKeyT,KV<Long,EventT>>>,OrderedEventProcessorResult<EventKeyT,ResultT,EventT>>

Transform for processing ordered events. Events are grouped by the key and within each key they are applied according to the provided sequence. Events which arrive out of sequence are buffered and processed after all the missing events for a given key have arrived.

There are two sequencing modes - a sequence per key and a global sequence. See OrderedProcessingHandler for details on how to configure this transform.

Notice: the global sequence processing on the Dataflow Runner requires running under Runner V2. Refer to Dataflow documentation for details.

See Also:

• Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
static final String	GLOBAL_SEQUENCE_TRACKER

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor	Description
OrderedEventProcessor()

Method Summary

Modifier and Type	Method	Description
static <EventTypeT, EventKeyTypeT, ResultTypeT, StateTypeT extends MutableState<EventTypeT, ResultTypeT>> OrderedEventProcessor<EventTypeT,EventKeyTypeT,ResultTypeT,StateTypeT>	create(OrderedProcessingHandler<EventTypeT,EventKeyTypeT,StateTypeT,ResultTypeT> handler)	Create the transform.
OrderedEventProcessorResult<EventKeyT,ResultT,EventT>	expand(PCollection<KV<EventKeyT,KV<Long,EventT>>> input)	Override this method to specify how this PTransform should be expanded on the given InputT.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

GLOBAL_SEQUENCE_TRACKER

public static final String GLOBAL_SEQUENCE_TRACKER

See Also:

• Constant Field Values

Constructor Details

OrderedEventProcessor

public OrderedEventProcessor()

Method Details

create

public static <EventTypeT,
EventKeyTypeT,
ResultTypeT,
StateTypeT extends MutableState<EventTypeT,
ResultTypeT>>
OrderedEventProcessor<EventTypeT,EventKeyTypeT,ResultTypeT,StateTypeT> create(OrderedProcessingHandler<EventTypeT,EventKeyTypeT,StateTypeT,ResultTypeT> handler)

Create the transform.

Type Parameters:

EventTypeT - type of event

EventKeyTypeT - type of event key

ResultTypeT - type of the result object

StateTypeT - type of the state to store

Parameters:

handler - provides the configuration of this transform

Returns:

the transform

expand

public OrderedEventProcessorResult<EventKeyT,ResultT,EventT> expand(PCollection<KV<EventKeyT,KV<Long,EventT>>> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<KV<EventKeyT,KV<Long,EventT>>>,OrderedEventProcessorResult<EventKeyT,ResultT,EventT>>