org.apache.beam.sdk.extensions.ordered

Class OrderedProcessingHandler.OrderedProcessingGlobalSequenceHandler<EventT,KeyT,StateT extends MutableState<EventT,?>,ResultT>

java.lang.Object

org.apache.beam.sdk.extensions.ordered.OrderedProcessingHandler<EventT,KeyT,StateT,ResultT>

org.apache.beam.sdk.extensions.ordered.OrderedProcessingHandler.OrderedProcessingGlobalSequenceHandler<EventT,KeyT,StateT,ResultT>

Type Parameters:

EventT - type of events to be processed

KeyT - type of keys which will be used to group the events

StateT - type of internal State which will be used for processing

ResultT - type of the result of the processing which will be output

All Implemented Interfaces:

java.io.Serializable

Enclosing class:

OrderedProcessingHandler<EventT,KeyT,StateT extends MutableState<EventT,?>,ResultT>

public abstract static class OrderedProcessingHandler.OrderedProcessingGlobalSequenceHandler<EventT,KeyT,StateT extends MutableState<EventT,?>,ResultT>
extends OrderedProcessingHandler<EventT,KeyT,StateT,ResultT>

Parent class for Ordered Processing configuration handlers to handle processing of the events where global sequence is used.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.extensions.ordered.OrderedProcessingHandler

OrderedProcessingHandler.OrderedProcessingGlobalSequenceHandler<EventT,KeyT,StateT extends MutableState<EventT,?>,ResultT>

Field Summary

Fields inherited from class org.apache.beam.sdk.extensions.ordered.OrderedProcessingHandler

DEFAULT_MAX_ELEMENTS_TO_OUTPUT

Constructor Summary

Constructors

Constructor and Description
OrderedProcessingGlobalSequenceHandler(java.lang.Class<EventT> eventTClass, java.lang.Class<KeyT> keyTClass, java.lang.Class<StateT> stateTClass, java.lang.Class<ResultT> resultTClass)

Method Summary

Modifier and Type	Method and Description
Duration	getContiguousSequenceRangeReevaluationFrequency() How frequently the combiner should reevaluate the maximum range? This parameter only affects the behaviour of streaming pipelines.
Combine.GloballyAsSingletonView<TimestampedValue<KV<KeyT,KV<java.lang.Long,EventT>>>,ContiguousSequenceRange>	getGlobalSequenceCombiner() Provide the global sequence combiner.
int	getMaxElementCountToTriggerContinuousSequenceRangeReevaluation() Number of new elements to trigger the re-evaluation.

Methods inherited from class org.apache.beam.sdk.extensions.ordered.OrderedProcessingHandler

getEventCoder, getEventExaminer, getKeyCoder, getMaxOutputElementsPerBundle, getResultCoder, getStateCoder, getStatusUpdateFrequency, isProduceStatusUpdateOnEveryEvent, setMaxOutputElementsPerBundle, setProduceStatusUpdateOnEveryEvent, setStatusUpdateFrequency

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OrderedProcessingGlobalSequenceHandler

public OrderedProcessingGlobalSequenceHandler(java.lang.Class<EventT> eventTClass,
                                              java.lang.Class<KeyT> keyTClass,
                                              java.lang.Class<StateT> stateTClass,
                                              java.lang.Class<ResultT> resultTClass)

Method Detail

getGlobalSequenceCombiner

public Combine.GloballyAsSingletonView<TimestampedValue<KV<KeyT,KV<java.lang.Long,EventT>>>,ContiguousSequenceRange> getGlobalSequenceCombiner()

Provide the global sequence combiner. Default is to use DefaultSequenceCombiner.

Returns:

combiner

getContiguousSequenceRangeReevaluationFrequency

public Duration getContiguousSequenceRangeReevaluationFrequency()

How frequently the combiner should reevaluate the maximum range? This parameter only affects the behaviour of streaming pipelines.

This parameter is used together with getMaxElementCountToTriggerContinuousSequenceRangeReevaluation(). The re-evaluation will occur as soon as the number of new elements exceeds the threshold or the time exceeds the frequency.

Notice that some runners cache the output of side inputs and this parameter might not appear to have an effect unless the cache time-to-live is equal or less than this frequency. For Dataflow runner, see this Dataflow streaming pipeline option

Returns:

frequency of reevaluating the ContiguousSequenceRange. Default - every second.

See Also:

getMaxElementCountToTriggerContinuousSequenceRangeReevaluation()

getMaxElementCountToTriggerContinuousSequenceRangeReevaluation

public int getMaxElementCountToTriggerContinuousSequenceRangeReevaluation()

Number of new elements to trigger the re-evaluation.

See getContiguousSequenceRangeReevaluationFrequency() for additional details.

Returns:

batch size. Default - 1000.