org.apache.beam.runners.fnexecution.data

Class GrpcDataService

java.lang.Object

org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.BeamFnDataImplBase

org.apache.beam.runners.fnexecution.data.GrpcDataService

All Implemented Interfaces:

java.lang.AutoCloseable, org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.AsyncService, FnDataService, FnService, org.apache.beam.vendor.grpc.v1p60p1.io.grpc.BindableService

public class GrpcDataService
extends org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.BeamFnDataImplBase
implements FnService, FnDataService

A FnDataService implemented via gRPC.

This service allows for multiple clients to transmit BeamFnApi.Elements messages.

This service transmits all outgoing BeamFnApi.Elements messages to the first client that connects.

Constructor Summary

Constructors

Constructor and Description
GrpcDataService() Deprecated. This constructor is for migrating Dataflow purpose only.

Method Summary

Modifier and Type	Method and Description
void	close() .
static GrpcDataService	create(PipelineOptions options, java.util.concurrent.ExecutorService executor, OutboundObserverFactory outboundObserverFactory)
BeamFnDataOutboundAggregator	createOutboundAggregator(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes) Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>	data(org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundElementObserver)
void	registerReceiver(java.lang.String instructionId, CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer) Registers a receiver for the provided instruction id.
void	unregisterReceiver(java.lang.String instructionId) Receivers are only expected to be unregistered when bundle processing has completed successfully.

Methods inherited from class org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.BeamFnDataImplBase

bindService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.vendor.grpc.v1p60p1.io.grpc.BindableService

bindService

Constructor Detail

GrpcDataService

@Deprecated
public GrpcDataService()

Deprecated. This constructor is for migrating Dataflow purpose only.

Method Detail

create

public static GrpcDataService create(PipelineOptions options,
                                     java.util.concurrent.ExecutorService executor,
                                     OutboundObserverFactory outboundObserverFactory)

data

public org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> data(org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundElementObserver)

Specified by:

data in interface org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.AsyncService

close

public void close()
           throws java.lang.Exception

Description copied from interface: FnService

.

There should be no more calls to any service method by the time a call to FnService.close() begins. Specifically, this means that a Server that this service is bound to should have completed a call to the Server.shutdown() method, and all future incoming calls will be rejected.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface FnService

Throws:

java.lang.Exception

registerReceiver

public void registerReceiver(java.lang.String instructionId,
                             CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)

Description copied from interface: FnDataService

Registers a receiver for the provided instruction id.

The receiver is not required to be thread safe.

Receivers for successfully processed bundles must be unregistered. See FnDataService.unregisterReceiver(java.lang.String) for details.

Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

Specified by:

registerReceiver in interface FnDataService

unregisterReceiver

public void unregisterReceiver(java.lang.String instructionId)

Description copied from interface: FnDataService

Receivers are only expected to be unregistered when bundle processing has completed successfully.

It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

Specified by:

unregisterReceiver in interface FnDataService

createOutboundAggregator

public BeamFnDataOutboundAggregator createOutboundAggregator(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                                             boolean collectElementsIfNoFlushes)

Description copied from interface: FnDataService

Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

Closing the returned aggregator signals the end of the streams.

The returned aggregator is not thread safe.

Specified by:

createOutboundAggregator in interface FnDataService