Package org.apache.beam.runners.fnexecution.data

Interface FnDataService

All Known Implementing Classes:

GrpcDataService

public interface FnDataService

The FnDataService is able to forward inbound elements to a consumer and is also a consumer of outbound elements. Callers can register themselves as consumers for inbound elements or can get a handle for a consumer for outbound elements.

Method Summary

Modifier and Type	Method	Description
BeamFnDataOutboundAggregator	createOutboundAggregator(Supplier<String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes)	Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
void	registerReceiver(String instructionId, CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)	Registers a receiver for the provided instruction id.
void	unregisterReceiver(String instructionId)	Receivers are only expected to be unregistered when bundle processing has completed successfully.

Method Details

registerReceiver

void registerReceiver(String instructionId,
 CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)

Registers a receiver for the provided instruction id.

The receiver is not required to be thread safe.

Receivers for successfully processed bundles must be unregistered. See unregisterReceiver(java.lang.String) for details.

Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

unregisterReceiver

void unregisterReceiver(String instructionId)

Receivers are only expected to be unregistered when bundle processing has completed successfully.

It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

createOutboundAggregator

BeamFnDataOutboundAggregator createOutboundAggregator(Supplier<String> processBundleRequestIdSupplier,
 boolean collectElementsIfNoFlushes)

Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

Closing the returned aggregator signals the end of the streams.

The returned aggregator is not thread safe.