org.apache.beam.sdk.fn.data

Interface BeamFnDataBufferingOutboundObserver<T>

All Superinterfaces:

java.lang.AutoCloseable, CloseableFnDataReceiver<T>, FnDataReceiver<T>

All Known Implementing Classes:

BeamFnDataSizeBasedBufferingOutboundObserver, BeamFnDataTimeBasedBufferingOutboundObserver

public interface BeamFnDataBufferingOutboundObserver<T>
extends CloseableFnDataReceiver<T>

A buffering outbound FnDataReceiver for the Beam Fn Data API.

Encodes individually consumed elements with the provided Coder producing a single BeamFnApi.Elements message when the buffer threshold is surpassed.

The default size-based buffer threshold can be overridden by specifying the experiment data_buffer_size_limit=<bytes>

The default time-based buffer threshold can be overridden by specifying the experiment data_buffer_time_limit_ms=<milliseconds>

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BEAM_FN_API_DATA_BUFFER_LIMIT Deprecated. Use DATA_BUFFER_SIZE_LIMIT instead.
static java.lang.String	BEAM_FN_API_DATA_BUFFER_SIZE_LIMIT Deprecated. Use DATA_BUFFER_SIZE_LIMIT instead.
static java.lang.String	BEAM_FN_API_DATA_BUFFER_TIME_LIMIT Deprecated. Use DATA_BUFFER_TIME_LIMIT_MS instead.
static java.lang.String	DATA_BUFFER_SIZE_LIMIT
static java.lang.String	DATA_BUFFER_TIME_LIMIT_MS
static int	DEFAULT_BUFFER_LIMIT_BYTES
static long	DEFAULT_BUFFER_LIMIT_TIME_MS

Method Summary

Modifier and Type	Method and Description
static <T> BeamFnDataSizeBasedBufferingOutboundObserver<T>	forLocation(PipelineOptions options, LogicalEndpoint endpoint, Coder<T> coder, org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver)
static int	getSizeLimit(PipelineOptions options)
static long	getTimeLimit(PipelineOptions options)

Methods inherited from interface org.apache.beam.sdk.fn.data.CloseableFnDataReceiver

close, flush

Methods inherited from interface org.apache.beam.sdk.fn.data.FnDataReceiver

accept

Field Detail

BEAM_FN_API_DATA_BUFFER_LIMIT

@Deprecated
static final java.lang.String BEAM_FN_API_DATA_BUFFER_LIMIT

Deprecated. Use DATA_BUFFER_SIZE_LIMIT instead.

See Also:

Constant Field Values

BEAM_FN_API_DATA_BUFFER_SIZE_LIMIT

@Deprecated
static final java.lang.String BEAM_FN_API_DATA_BUFFER_SIZE_LIMIT

Deprecated. Use DATA_BUFFER_SIZE_LIMIT instead.

See Also:

Constant Field Values

DATA_BUFFER_SIZE_LIMIT

static final java.lang.String DATA_BUFFER_SIZE_LIMIT

See Also:

Constant Field Values

DEFAULT_BUFFER_LIMIT_BYTES

static final int DEFAULT_BUFFER_LIMIT_BYTES

See Also:

Constant Field Values

BEAM_FN_API_DATA_BUFFER_TIME_LIMIT

@Deprecated
static final java.lang.String BEAM_FN_API_DATA_BUFFER_TIME_LIMIT

Deprecated. Use DATA_BUFFER_TIME_LIMIT_MS instead.

See Also:

Constant Field Values

DATA_BUFFER_TIME_LIMIT_MS

static final java.lang.String DATA_BUFFER_TIME_LIMIT_MS

See Also:

Constant Field Values

DEFAULT_BUFFER_LIMIT_TIME_MS

static final long DEFAULT_BUFFER_LIMIT_TIME_MS

See Also:

Constant Field Values

Method Detail

forLocation

static <T> BeamFnDataSizeBasedBufferingOutboundObserver<T> forLocation(PipelineOptions options,
                                                                       LogicalEndpoint endpoint,
                                                                       Coder<T> coder,
                                                                       org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver)

getSizeLimit

static int getSizeLimit(PipelineOptions options)

getTimeLimit

static long getTimeLimit(PipelineOptions options)