org.apache.beam.sdk.transforms.splittabledofn

Class RestrictionTracker<RestrictionT,PositionT>

java.lang.Object

org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<RestrictionT,PositionT>

Direct Known Subclasses:

OffsetRangeTracker

public abstract class RestrictionTracker<RestrictionT,PositionT>
extends java.lang.Object

Manages concurrent access to the restriction and keeps track of its claimed part for a splittable DoFn.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	RestrictionTracker.ClaimObserver<PositionT> Internal interface allowing a runner to observe the calls to tryClaim(PositionT).

Constructor Summary

Constructors

Constructor and Description
RestrictionTracker()

Method Summary

Modifier and Type	Method and Description
abstract void	checkDone() Called by the runner after DoFn.ProcessElement returns.
abstract RestrictionT	checkpoint() Signals that the current DoFn.ProcessElement call should terminate as soon as possible: after this method returns, the tracker MUST refuse all future claim calls, and checkDone() MUST succeed.
abstract RestrictionT	currentRestriction() Returns a restriction accurately describing the full range of work the current DoFn.ProcessElement call will do, including already completed work.
void	setClaimObserver(RestrictionTracker.ClaimObserver<PositionT> claimObserver) Sets a RestrictionTracker.ClaimObserver to be invoked on every call to tryClaim(PositionT).
boolean	tryClaim(PositionT position) Attempts to claim the block of work in the current restriction identified by the given position.
protected abstract boolean	tryClaimImpl(PositionT position) Tracker-specific implementation of tryClaim(PositionT).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RestrictionTracker

public RestrictionTracker()

Method Detail

setClaimObserver

@Internal
public void setClaimObserver(RestrictionTracker.ClaimObserver<PositionT> claimObserver)

Sets a RestrictionTracker.ClaimObserver to be invoked on every call to tryClaim(PositionT). Internal: intended only for runner authors.

tryClaim

public final boolean tryClaim(PositionT position)

Attempts to claim the block of work in the current restriction identified by the given position.

If this succeeds, the DoFn MUST execute the entire block of work. If this fails:

• DoFn.ProcessElement MUST return DoFn.ProcessContinuation#stop without performing any additional work or emitting output (note that emitting output or performing work from DoFn.ProcessElement is also not allowed before the first call to this method).
• checkDone() MUST succeed.

Under the hood, calls tryClaimImpl(PositionT) and notifies RestrictionTracker.ClaimObserver of the result.

tryClaimImpl

@Internal
protected abstract boolean tryClaimImpl(PositionT position)

Tracker-specific implementation of tryClaim(PositionT).

currentRestriction

public abstract RestrictionT currentRestriction()

Returns a restriction accurately describing the full range of work the current DoFn.ProcessElement call will do, including already completed work.

checkpoint

public abstract RestrictionT checkpoint()

Signals that the current DoFn.ProcessElement call should terminate as soon as possible: after this method returns, the tracker MUST refuse all future claim calls, and checkDone() MUST succeed.

Modifies currentRestriction(). Returns a restriction representing the rest of the work: the old value of currentRestriction() is equivalent to the new value and the return value of this method combined.

Must be called at most once on a given object. Must not be called before the first successful tryClaim(PositionT) call.

checkDone

public abstract void checkDone()
                        throws java.lang.IllegalStateException

Called by the runner after DoFn.ProcessElement returns.

Must throw an exception with an informative error message, if there is still any unclaimed work remaining in the restriction.

Throws:

java.lang.IllegalStateException