Package org.apache.beam.sdk.fn.data

Class BeamFnDataGrpcMultiplexer

java.lang.Object

org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer

All Implemented Interfaces:

AutoCloseable

public class BeamFnDataGrpcMultiplexer
extends Object
implements AutoCloseable

A gRPC multiplexer for a specific Endpoints.ApiServiceDescriptor.

Multiplexes data for inbound consumers based upon their instructionId.

Multiplexing inbound and outbound streams is as thread safe as the consumers of those streams. For inbound streams, this is as thread safe as the inbound observers. For outbound streams, this is as thread safe as the underlying stream observer.

TODO: Add support for multiplexing over multiple outbound observers by stickying the output location with a specific outbound observer.

Constructor Summary

Constructors

Constructor	Description
BeamFnDataGrpcMultiplexer(@Nullable org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, OutboundObserverFactory outboundObserverFactory, OutboundObserverFactory.BasicFactory<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements,org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> baseOutboundObserverFactory)

Method Summary

Modifier and Type	Method	Description
void	close()
org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>	getInboundObserver()
org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>	getOutboundObserver()
void	poisonInstructionId(String instructionId)	Poisons an instruction id.
void	registerConsumer(String instructionId, CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)	Registers a consumer for the specified instruction id.
String	toString()
void	unregisterConsumer(String instructionId)	Unregisters a previously registered consumer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

BeamFnDataGrpcMultiplexer

public BeamFnDataGrpcMultiplexer(@Nullable org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
 OutboundObserverFactory outboundObserverFactory,
 OutboundObserverFactory.BasicFactory<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements,org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> baseOutboundObserverFactory)

Method Details

toString

public String toString()

Overrides:

toString in class Object

getInboundObserver

public org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> getInboundObserver()

getOutboundObserver

public org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> getOutboundObserver()

registerConsumer

public void registerConsumer(String instructionId,
 CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)

Registers a consumer for the specified instruction id.

The BeamFnDataGrpcMultiplexer partitions BeamFnApi.Elements with multiple instruction ids ensuring that the receiver will only see BeamFnApi.Elements with a single instruction id.

The caller must either unregister the consumer when all messages have been processed or poison the instruction if messages for the instruction should be dropped.

unregisterConsumer

public void unregisterConsumer(String instructionId)

Unregisters a previously registered consumer.

poisonInstructionId

public void poisonInstructionId(String instructionId)

Poisons an instruction id.

Any records for the instruction on the inbound observer will be dropped for the next POISONED_INSTRUCTION_ID_CACHE_TIMEOUT.

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception