Package org.apache.beam.runners.spark.structuredstreaming.metrics

Class MetricsAccumulator

java.lang.Object
org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>
org.apache.beam.runners.spark.structuredstreaming.metrics.MetricsAccumulator
All Implemented Interfaces:
Serializable, scala.Serializable
public class MetricsAccumulator extends org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>
AccumulatorV2 for Beam metrics captured in MetricsContainerStepMap.
See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    MetricsAccumulator()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    add(org.apache.beam.runners.core.metrics.MetricsContainerStepMap other)
     
    static void
    clear()
     
    MetricsAccumulator
    copy()
     
    static MetricsAccumulator
    getInstance(org.apache.spark.sql.SparkSession session)
    Get the MetricsAccumulator on this driver.
    boolean
    isZero()
     
    void
    merge(org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap> other)
     
    void
    reset()
     
    org.apache.beam.runners.core.metrics.MetricsContainerStepMap
    value()
     

    Methods inherited from class org.apache.spark.util.AccumulatorV2

    copyAndReset, countFailedValues, id, isAtDriverSide, isRegistered, metadata, metadata_$eq, name, register, register$default$2, register$default$3, toInfo, toString, withBufferSerialized, writeReplace

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • MetricsAccumulator

      public MetricsAccumulator()

  • Method Details

    • isZero

      public boolean isZero()
      Specified by:
      isZero in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • copy

      public MetricsAccumulator copy()
      Specified by:
      copy in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • reset

      public void reset()
      Specified by:
      reset in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • add

      public void add(org.apache.beam.runners.core.metrics.MetricsContainerStepMap other)
      Specified by:
      add in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • merge

      public void merge(org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap> other)
      Specified by:
      merge in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • value

      public org.apache.beam.runners.core.metrics.MetricsContainerStepMap value()
      Specified by:
      value in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

    • getInstance

      public static MetricsAccumulator getInstance(org.apache.spark.sql.SparkSession session)
      Get the MetricsAccumulator on this driver. If there's no such accumulator yet, it will be created and registered using the provided SparkSession.

    • clear

      public static void clear()