Package org.apache.beam.sdk.fn.data

Class BeamFnDataOutboundAggregator

java.lang.Object

org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator

@NotThreadSafe
public class BeamFnDataOutboundAggregator
extends Object

An outbound data buffering aggregator with size-based buffer and time-based buffer if corresponding options are set.

The default size-based buffer threshold can be overridden by specifying the experiment data_buffer_size_limit=<bytes>

The default time-based buffer threshold can be overridden by specifying the experiment data_buffer_time_limit_ms=<milliseconds>

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DATA_BUFFER_SIZE_LIMIT
static final String	DATA_BUFFER_TIME_LIMIT_MS
static final int	DEFAULT_BUFFER_LIMIT_BYTES
static final long	DEFAULT_BUFFER_LIMIT_TIME_MS

Constructor Summary

Constructors

Constructor	Description
BeamFnDataOutboundAggregator(PipelineOptions options, Supplier<String> processBundleRequestIdSupplier, org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver, boolean collectElementsIfNoFlushes)

Method Summary

Modifier and Type	Method	Description
void	discard()
<T> FnDataReceiver<T>	registerOutputDataLocation(String pTransformId, Coder<T> coder)	Register the outbound data logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound data.
<T> FnDataReceiver<T>	registerOutputTimersLocation(String pTransformId, String timerFamilyId, Coder<T> coder)	Register the outbound timers logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound timers data.
void	sendElements(org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements elements)
org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements	sendOrCollectBufferedDataAndFinishOutboundStreams()	Closes the streams for all registered outbound endpoints.
void	start()	Starts the flushing daemon thread if data_buffer_time_limit_ms is set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DATA_BUFFER_SIZE_LIMIT

public static final String DATA_BUFFER_SIZE_LIMIT

See Also:

• Constant Field Values

DEFAULT_BUFFER_LIMIT_BYTES

public static final int DEFAULT_BUFFER_LIMIT_BYTES

See Also:

• Constant Field Values

DATA_BUFFER_TIME_LIMIT_MS

public static final String DATA_BUFFER_TIME_LIMIT_MS

See Also:

• Constant Field Values

DEFAULT_BUFFER_LIMIT_TIME_MS

public static final long DEFAULT_BUFFER_LIMIT_TIME_MS

See Also:

• Constant Field Values

Constructor Details

BeamFnDataOutboundAggregator

public BeamFnDataOutboundAggregator(PipelineOptions options,
 Supplier<String> processBundleRequestIdSupplier,
 org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver,
 boolean collectElementsIfNoFlushes)

Method Details

start

public void start()

Starts the flushing daemon thread if data_buffer_time_limit_ms is set.

registerOutputDataLocation

public <T> FnDataReceiver<T> registerOutputDataLocation(String pTransformId,
 Coder<T> coder)

Register the outbound data logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound data.

registerOutputTimersLocation

public <T> FnDataReceiver<T> registerOutputTimersLocation(String pTransformId,
 String timerFamilyId,
 Coder<T> coder)

Register the outbound timers logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound timers data.

sendOrCollectBufferedDataAndFinishOutboundStreams

public org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements sendOrCollectBufferedDataAndFinishOutboundStreams()

Closes the streams for all registered outbound endpoints. Should be called at the end of each bundle. Returns the buffered Elements if the BeamFnDataOutboundAggregator started with collectElementsIfNoFlushes=true, and there was no previous flush in this bundle, otherwise returns null.

sendElements

public void sendElements(org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements elements)

discard

public void discard()