org.apache.beam.sdk.transforms.splittabledofn

Class OffsetRangeTracker

java.lang.Object

org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker

All Implemented Interfaces:

RestrictionTracker<OffsetRange>

public class OffsetRangeTracker
extends java.lang.Object
implements RestrictionTracker<OffsetRange>

A RestrictionTracker for claiming offsets in an OffsetRange in a monotonically increasing fashion.

Constructor Summary

Constructors

Constructor and Description
OffsetRangeTracker(OffsetRange range)

Method Summary

Modifier and Type	Method and Description
void	checkDone() Called by the runner after DoFn.ProcessElement returns.
OffsetRange	checkpoint() Signals that the current DoFn.ProcessElement call should terminate as soon as possible.
OffsetRange	currentRestriction() Returns a restriction accurately describing the full range of work the current DoFn.ProcessElement call will do, including already completed work.
void	markDone() Marks that there are no more offsets to be claimed in the range.
boolean	tryClaim(long i) Attempts to claim the given offset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OffsetRangeTracker

public OffsetRangeTracker(OffsetRange range)

Method Detail

currentRestriction

public OffsetRange currentRestriction()

Description copied from interface: RestrictionTracker

Returns a restriction accurately describing the full range of work the current DoFn.ProcessElement call will do, including already completed work.

Specified by:

currentRestriction in interface RestrictionTracker<OffsetRange>

checkpoint

public OffsetRange checkpoint()

Description copied from interface: RestrictionTracker

Signals that the current DoFn.ProcessElement call should terminate as soon as possible. Modifies RestrictionTracker.currentRestriction(). Returns a restriction representing the rest of the work: the old value of RestrictionTracker.currentRestriction() is equivalent to the new value and the return value of this method combined. Must be called at most once on a given object.

Specified by:

checkpoint in interface RestrictionTracker<OffsetRange>

tryClaim

public boolean tryClaim(long i)

Attempts to claim the given offset.

Must be larger than the last successfully claimed offset.

Returns:

true if the offset was successfully claimed, false if it is outside the current OffsetRange of this tracker (in that case this operation is a no-op).

markDone

public void markDone()

Marks that there are no more offsets to be claimed in the range.

E.g., a DoFn reading a file and claiming the offset of each record in the file might call this if it hits EOF - even though the last attempted claim was before the end of the range, there are no more offsets to claim.

checkDone

public void checkDone()
               throws java.lang.IllegalStateException

Description copied from interface: RestrictionTracker

Called by the runner after DoFn.ProcessElement returns.

Must throw an exception with an informative error message, if there is still any unclaimed work remaining in the restriction.

Specified by:

checkDone in interface RestrictionTracker<OffsetRange>

Throws:

java.lang.IllegalStateException