Package org.apache.beam.runners.spark.stateful

Class SparkTimerInternals

java.lang.Object

org.apache.beam.runners.spark.stateful.SparkTimerInternals

All Implemented Interfaces:

org.apache.beam.runners.core.TimerInternals

public class SparkTimerInternals
extends Object
implements org.apache.beam.runners.core.TimerInternals

An implementation of TimerInternals for the SparkRunner.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.runners.core.TimerInternals

org.apache.beam.runners.core.TimerInternals.TimerData, org.apache.beam.runners.core.TimerInternals.TimerDataCoder, org.apache.beam.runners.core.TimerInternals.TimerDataCoderV2

Method Summary

Modifier and Type	Method	Description
void	addTimers(Iterator<org.apache.beam.runners.core.TimerInternals.TimerData> timers)
void	advanceWatermark()	Advances the watermark.
Instant	currentInputWatermarkTime()
@Nullable Instant	currentOutputWatermarkTime()
Instant	currentProcessingTime()
@Nullable Instant	currentSynchronizedProcessingTime()
void	deleteTimer(org.apache.beam.runners.core.StateNamespace namespace, String timerId, String timerFamilyId)
void	deleteTimer(org.apache.beam.runners.core.StateNamespace namespace, String timerId, String timerFamilyId, TimeDomain timeDomain)
void	deleteTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)
static Iterator<org.apache.beam.runners.core.TimerInternals.TimerData>	deserializeTimers(Collection<byte[]> serTimers, org.apache.beam.runners.core.TimerInternals.TimerDataCoderV2 timerDataCoder)
static SparkTimerInternals	forStreamFromSources(List<Integer> sourceIds, Map<Integer,GlobalWatermarkHolder.SparkWatermarks> watermarks)	Build the TimerInternals according to the feeding streams.
@Nullable org.apache.beam.runners.core.TimerInternals.TimerData	getNextProcessingTimer()	Finds the latest timer in TimeDomain.PROCESSING_TIME domain that has expired based on the current processing time.
Collection<org.apache.beam.runners.core.TimerInternals.TimerData>	getTimers()
static SparkTimerInternals	global(Map<Integer,GlobalWatermarkHolder.SparkWatermarks> watermarks)	Build a global TimerInternals for all feeding streams.
boolean	hasNextProcessingTimer()	Checks if there are any expired timers in the TimeDomain.PROCESSING_TIME domain.
static Collection<byte[]>	serializeTimers(Collection<org.apache.beam.runners.core.TimerInternals.TimerData> timers, org.apache.beam.runners.core.TimerInternals.TimerDataCoderV2 timerDataCoder)
void	setTimer(org.apache.beam.runners.core.StateNamespace namespace, String timerId, String timerFamilyId, Instant target, Instant outputTimestamp, TimeDomain timeDomain)
void	setTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

forStreamFromSources

public static SparkTimerInternals forStreamFromSources(List<Integer> sourceIds,
 Map<Integer,GlobalWatermarkHolder.SparkWatermarks> watermarks)

Build the TimerInternals according to the feeding streams.

global

public static SparkTimerInternals global(Map<Integer,GlobalWatermarkHolder.SparkWatermarks> watermarks)

Build a global TimerInternals for all feeding streams.

getTimers

public Collection<org.apache.beam.runners.core.TimerInternals.TimerData> getTimers()

addTimers

public void addTimers(Iterator<org.apache.beam.runners.core.TimerInternals.TimerData> timers)

setTimer

public void setTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)

Specified by:

setTimer in interface org.apache.beam.runners.core.TimerInternals

deleteTimer

public void deleteTimer(org.apache.beam.runners.core.StateNamespace namespace,
 String timerId,
 String timerFamilyId,
 TimeDomain timeDomain)

Specified by:

deleteTimer in interface org.apache.beam.runners.core.TimerInternals

deleteTimer

public void deleteTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)

Specified by:

deleteTimer in interface org.apache.beam.runners.core.TimerInternals

currentProcessingTime

public Instant currentProcessingTime()

Specified by:

currentProcessingTime in interface org.apache.beam.runners.core.TimerInternals

currentSynchronizedProcessingTime

public @Nullable Instant currentSynchronizedProcessingTime()

Specified by:

currentSynchronizedProcessingTime in interface org.apache.beam.runners.core.TimerInternals

currentInputWatermarkTime

public Instant currentInputWatermarkTime()

Specified by:

currentInputWatermarkTime in interface org.apache.beam.runners.core.TimerInternals

advanceWatermark

public void advanceWatermark()

Advances the watermark.

currentOutputWatermarkTime

public @Nullable Instant currentOutputWatermarkTime()

Specified by:

currentOutputWatermarkTime in interface org.apache.beam.runners.core.TimerInternals

setTimer

public void setTimer(org.apache.beam.runners.core.StateNamespace namespace,
 String timerId,
 String timerFamilyId,
 Instant target,
 Instant outputTimestamp,
 TimeDomain timeDomain)

Specified by:

setTimer in interface org.apache.beam.runners.core.TimerInternals

deleteTimer

public void deleteTimer(org.apache.beam.runners.core.StateNamespace namespace,
 String timerId,
 String timerFamilyId)

Specified by:

deleteTimer in interface org.apache.beam.runners.core.TimerInternals

serializeTimers

public static Collection<byte[]> serializeTimers(Collection<org.apache.beam.runners.core.TimerInternals.TimerData> timers,
 org.apache.beam.runners.core.TimerInternals.TimerDataCoderV2 timerDataCoder)

deserializeTimers

public static Iterator<org.apache.beam.runners.core.TimerInternals.TimerData> deserializeTimers(Collection<byte[]> serTimers,
 org.apache.beam.runners.core.TimerInternals.TimerDataCoderV2 timerDataCoder)

hasNextProcessingTimer

public boolean hasNextProcessingTimer()

Checks if there are any expired timers in the TimeDomain.PROCESSING_TIME domain.

A timer is considered expired when its timestamp is less than the current processing time.

Returns:

true if at least one expired processing timer exists, false otherwise.

getNextProcessingTimer

public @Nullable org.apache.beam.runners.core.TimerInternals.TimerData getNextProcessingTimer()

Finds the latest timer in TimeDomain.PROCESSING_TIME domain that has expired based on the current processing time.

A timer is considered expired when its timestamp is less than the current processing time. If multiple expired timers exist, the one with the latest timestamp will be returned.

Returns:

The expired processing timer with the latest timestamp if one exists, or null if no processing timers are ready to fire.

toString

public String toString()

Overrides:

toString in class Object