Package org.apache.beam.runners.fnexecution.control

Class SdkHarnessClient.BundleProcessor

java.lang.Object

org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor

Enclosing class:

SdkHarnessClient

public class SdkHarnessClient.BundleProcessor
extends Object

A processor capable of creating bundles for some registered BeamFnApi.ProcessBundleDescriptor.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	SdkHarnessClient.BundleProcessor.ActiveBundle	An active bundle for a particular BeamFnApi.ProcessBundleDescriptor.

Method Summary

Modifier and Type	Method	Description
SdkHarnessClient.BundleProcessor.ActiveBundle	newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers, Map<KV<String,String>,RemoteOutputReceiver<org.apache.beam.sdk.util.construction.Timer<?>>> timerReceivers, StateRequestHandler stateRequestHandler, BundleProgressHandler progressHandler, BundleFinalizationHandler finalizationHandler, BundleCheckpointHandler checkpointHandler)	Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.
SdkHarnessClient.BundleProcessor.ActiveBundle	newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers, Map<KV<String,String>,RemoteOutputReceiver<org.apache.beam.sdk.util.construction.Timer<?>>> timerReceivers, StateRequestHandler stateRequestHandler, BundleProgressHandler progressHandler, BundleSplitHandler splitHandler, BundleCheckpointHandler checkpointHandler, BundleFinalizationHandler finalizationHandler)	Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.
SdkHarnessClient.BundleProcessor.ActiveBundle	newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers, BundleProgressHandler progressHandler)	Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.
SdkHarnessClient.BundleProcessor.ActiveBundle	newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers, StateRequestHandler stateRequestHandler, BundleProgressHandler progressHandler)	Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

newBundle

public SdkHarnessClient.BundleProcessor.ActiveBundle newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers,
 BundleProgressHandler progressHandler)

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

The input channels for the returned SdkHarnessClient.BundleProcessor.ActiveBundle are derived from the instructions in the BeamFnApi.ProcessBundleDescriptor.

NOTE: It is important to SdkHarnessClient.close() each bundle after all elements are emitted.

 try (ActiveBundle bundle = SdkHarnessClient.newBundle(...)) {
   FnDataReceiver<InputT> inputReceiver =
       (FnDataReceiver) bundle.getInputReceivers().get(mainPCollectionId);
   // send all main input elements ...
 }
 

An exception during SdkHarnessClient.close() will be thrown if the bundle requests finalization or attempts to checkpoint by returning a BeamFnApi.DelayedBundleApplication.

newBundle

public SdkHarnessClient.BundleProcessor.ActiveBundle newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers,
 StateRequestHandler stateRequestHandler,
 BundleProgressHandler progressHandler)

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

The input channels for the returned SdkHarnessClient.BundleProcessor.ActiveBundle are derived from the instructions in the BeamFnApi.ProcessBundleDescriptor.

NOTE: It is important to SdkHarnessClient.close() each bundle after all elements are emitted.

 try (ActiveBundle bundle = SdkHarnessClient.newBundle(...)) {
   FnDataReceiver<InputT> inputReceiver =
       (FnDataReceiver) bundle.getInputReceivers().get(mainPCollectionId);
   // send all main input elements ...
 }
 

An exception during SdkHarnessClient.close() will be thrown if the bundle requests finalization or attempts to checkpoint by returning a BeamFnApi.DelayedBundleApplication.

newBundle

public SdkHarnessClient.BundleProcessor.ActiveBundle newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers,
 Map<KV<String,String>,RemoteOutputReceiver<org.apache.beam.sdk.util.construction.Timer<?>>> timerReceivers,
 StateRequestHandler stateRequestHandler,
 BundleProgressHandler progressHandler,
 BundleFinalizationHandler finalizationHandler,
 BundleCheckpointHandler checkpointHandler)

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

The input channels for the returned SdkHarnessClient.BundleProcessor.ActiveBundle are derived from the instructions in the BeamFnApi.ProcessBundleDescriptor.

NOTE: It is important to SdkHarnessClient.close() each bundle after all elements are emitted.

 try (ActiveBundle bundle = SdkHarnessClient.newBundle(...)) {
   FnDataReceiver<InputT> inputReceiver =
       (FnDataReceiver) bundle.getInputReceivers().get(mainPCollectionId);
   // send all main input elements ...
 }
 

An exception during SdkHarnessClient.close() will be thrown if the bundle requests finalization if BundleFinalizationHandler is null or attempts to checkpoint by returning a BeamFnApi.DelayedBundleApplication .

newBundle

public SdkHarnessClient.BundleProcessor.ActiveBundle newBundle(Map<String,RemoteOutputReceiver<?>> outputReceivers,
 Map<KV<String,String>,RemoteOutputReceiver<org.apache.beam.sdk.util.construction.Timer<?>>> timerReceivers,
 StateRequestHandler stateRequestHandler,
 BundleProgressHandler progressHandler,
 BundleSplitHandler splitHandler,
 BundleCheckpointHandler checkpointHandler,
 BundleFinalizationHandler finalizationHandler)

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

The input channels for the returned SdkHarnessClient.BundleProcessor.ActiveBundle are derived from the instructions in the BeamFnApi.ProcessBundleDescriptor.

NOTE: It is important to SdkHarnessClient.close() each bundle after all elements are emitted.

 try (ActiveBundle bundle = SdkHarnessClient.newBundle(...)) {
   FnDataReceiver<InputT> inputReceiver =
       (FnDataReceiver) bundle.getInputReceivers().get(mainPCollectionId);
   // send all elements ...
 }