Package org.apache.beam.sdk.io.sparkreceiver

Class WrappedSupervisor

java.lang.Object
org.apache.spark.streaming.receiver.ReceiverSupervisor
org.apache.beam.sdk.io.sparkreceiver.WrappedSupervisor
All Implemented Interfaces:
org.apache.spark.internal.Logging
public class WrappedSupervisor extends org.apache.spark.streaming.receiver.ReceiverSupervisor
Wrapper class for ReceiverSupervisor that doesn't use Spark Environment.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.spark.streaming.receiver.ReceiverSupervisor

    org.apache.spark.streaming.receiver.ReceiverSupervisor.ReceiverState$

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Constructor Summary

    Constructors
    Constructor
    Description
    WrappedSupervisor(org.apache.spark.streaming.receiver.Receiver<?> receiver, org.apache.spark.SparkConf conf, SerializableFunction<Object[],Void> storeFn)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.streaming.receiver.BlockGenerator
    createBlockGenerator(org.apache.spark.streaming.receiver.BlockGeneratorListener blockGeneratorListener)
     
    long
    getCurrentRateLimit()
     
    boolean
    isReceiverStopped()
     
    void
    logInfo(scala.Function0<String> msg)
     
    boolean
    onReceiverStart()
     
    void
    pushArrayBuffer(scala.collection.mutable.ArrayBuffer<?> arrayBuffer, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
     
    void
    pushBytes(ByteBuffer byteBuffer, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
     
    void
    pushIterator(scala.collection.Iterator<?> iterator, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
     
    void
    pushSingle(Object o)
     
    void
    reportError(String s, Throwable throwable)
     

    Methods inherited from class org.apache.spark.streaming.receiver.ReceiverSupervisor

    awaitTermination, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isReceiverStarted, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logName, logTrace, logTrace, logWarning, logWarning, onReceiverStop, onStart, onStop, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, receiverState, ReceiverState, receiverState_$eq, restartReceiver, restartReceiver, restartReceiver$default$2, start, startReceiver, stop, stoppingError, stoppingError_$eq, stopReceiver, streamId

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • WrappedSupervisor

      public WrappedSupervisor(org.apache.spark.streaming.receiver.Receiver<?> receiver, org.apache.spark.SparkConf conf, SerializableFunction<Object[],Void> storeFn)

  • Method Details

    • pushSingle

      public void pushSingle(Object o)
      Specified by:
      pushSingle in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • pushBytes

      public void pushBytes(ByteBuffer byteBuffer, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
      Specified by:
      pushBytes in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • pushIterator

      public void pushIterator(scala.collection.Iterator<?> iterator, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
      Specified by:
      pushIterator in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • pushArrayBuffer

      public void pushArrayBuffer(scala.collection.mutable.ArrayBuffer<?> arrayBuffer, scala.Option<Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)
      Specified by:
      pushArrayBuffer in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • createBlockGenerator

      public org.apache.spark.streaming.receiver.BlockGenerator createBlockGenerator(org.apache.spark.streaming.receiver.BlockGeneratorListener blockGeneratorListener)
      Specified by:
      createBlockGenerator in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • reportError

      public void reportError(String s, Throwable throwable)
      Specified by:
      reportError in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • onReceiverStart

      public boolean onReceiverStart()
      Specified by:
      onReceiverStart in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • getCurrentRateLimit

      public long getCurrentRateLimit()
      Overrides:
      getCurrentRateLimit in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • isReceiverStopped

      public boolean isReceiverStopped()
      Overrides:
      isReceiverStopped in class org.apache.spark.streaming.receiver.ReceiverSupervisor

    • logInfo

      public void logInfo(scala.Function0<String> msg)
      Specified by:
      logInfo in interface org.apache.spark.internal.Logging
      Overrides:
      logInfo in class org.apache.spark.streaming.receiver.ReceiverSupervisor