org.apache.beam.sdk.fn.data

Class BeamFnDataGrpcMultiplexer

java.lang.Object

org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer

All Implemented Interfaces:

java.lang.AutoCloseable

public class BeamFnDataGrpcMultiplexer
extends java.lang.Object
implements java.lang.AutoCloseable

A gRPC multiplexer for a specific Endpoints.ApiServiceDescriptor.

Multiplexes data for inbound consumers based upon their instructionId.

Multiplexing inbound and outbound streams is as thread safe as the consumers of those streams. For inbound streams, this is as thread safe as the inbound observers. For outbound streams, this is as thread safe as the underlying stream observer.

TODO: Add support for multiplexing over multiple outbound observers by stickying the output location with a specific outbound observer.

Constructor Summary

Constructors

Constructor and Description
BeamFnDataGrpcMultiplexer(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, OutboundObserverFactory outboundObserverFactory, OutboundObserverFactory.BasicFactory<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements,org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> baseOutboundObserverFactory)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>	getInboundObserver()
org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>	getOutboundObserver()
void	poisonInstructionId(java.lang.String instructionId) Poisons an instruction id.
void	registerConsumer(java.lang.String instructionId, CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver) Registers a consumer for the specified instruction id.
java.lang.String	toString()
void	unregisterConsumer(java.lang.String instructionId) Unregisters a previously registered consumer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BeamFnDataGrpcMultiplexer

public BeamFnDataGrpcMultiplexer(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                 OutboundObserverFactory outboundObserverFactory,
                                 OutboundObserverFactory.BasicFactory<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements,org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> baseOutboundObserverFactory)

Method Detail

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getInboundObserver

public org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> getInboundObserver()

getOutboundObserver

public org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> getOutboundObserver()

registerConsumer

public void registerConsumer(java.lang.String instructionId,
                             CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)

Registers a consumer for the specified instruction id.

The BeamFnDataGrpcMultiplexer partitions BeamFnApi.Elements with multiple instruction ids ensuring that the receiver will only see BeamFnApi.Elements with a single instruction id.

The caller must either unregister the consumer when all messages have been processed or poison the instruction if messages for the instruction should be dropped.

unregisterConsumer

public void unregisterConsumer(java.lang.String instructionId)

Unregisters a previously registered consumer.

poisonInstructionId

public void poisonInstructionId(java.lang.String instructionId)

Poisons an instruction id.

Any records for the instruction on the inbound observer will be dropped for the next POISONED_INSTRUCTION_ID_CACHE_TIMEOUT.

close

public void close()
           throws java.lang.Exception

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.lang.Exception