Package org.apache.beam.sdk.fn.data

Class BeamFnDataInboundObserver

java.lang.Object
org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver
All Implemented Interfaces:
AutoCloseable, CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>, FnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>
public class BeamFnDataInboundObserver extends Object implements CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>
Decodes BeamFnApi.Elements partitioning them using the provided DataEndpoints and TimerEndpoints.

Note that this receiver uses a queue to buffer and pass elements from one thread to be processed by the thread which invokes awaitCompletion().

Closing the receiver will unblock any upstream producer and downstream consumer exceptionally.

  • Method Details

    • forConsumers

      public static BeamFnDataInboundObserver forConsumers(List<DataEndpoint<?>> dataEndpoints, List<TimerEndpoint<?>> timerEndpoints)
      Creates a receiver that is able to consume elements multiplexing on to the provided set of endpoints.

    • accept

      public void accept(org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements elements) throws Exception
      Specified by:
      accept in interface FnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>
      Throws:
      Exception

    • flush

      public void flush() throws Exception
      Description copied from interface: CloseableFnDataReceiver
      Eagerly flushes any data that is buffered in this channel.
      Specified by:
      flush in interface CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>
      Throws:
      Exception

    • close

      public void close() throws Exception
      Description copied from interface: CloseableFnDataReceiver
      .

      Does nothing if this CloseableFnDataReceiver is already closed.

      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements>
      Throws:
      Exception

    • isConsumingReceivedData

      public boolean isConsumingReceivedData()

    • awaitCompletion

      public void awaitCompletion() throws Exception
      Uses the callers thread to process all elements received until we receive the end of the stream from the upstream producer for all endpoints specified.

      Erroneous elements passed from the producer will be visible to the caller of this method.

      Throws:
      Exception

    • multiplexElements

      public boolean multiplexElements(Iterator<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements.Data> dataElements, Iterator<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements.Timers> timerElements) throws Exception
      Dispatches the data and timers from the elements to corresponding receivers. Returns true if all the endpoints are done after elements dispatching.
      Throws:
      Exception

    • reset

      public void reset()
      Enables this receiver to be used again for another bundle.

    • getUnfinishedEndpoints

      public List<String> getUnfinishedEndpoints()
      Get all unfinished data and timers endpoints represented as [transform_id]:data and [transform_id]:timers:[timer_family_id].