Class MetricsAccumulator
java.lang.Object
org.apache.beam.runners.spark.metrics.MetricsAccumulator
For resilience,
Accumulators are required to be wrapped in a Singleton.- See Also:
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic classSpark Listener which checkpointsMetricsContainerStepMapvalues for fault-tolerance. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic voidclear()static voidinit(SparkPipelineOptions opts, org.apache.spark.api.java.JavaSparkContext jsc) Init metrics accumulator if it has not been initiated.static voidinit(SparkPipelineOptions opts, org.apache.spark.api.java.JavaSparkContext jsc, boolean useCheckpoint) Init metrics accumulator if it has not been initiated.
-
Constructor Details
-
MetricsAccumulator
public MetricsAccumulator()
-
-
Method Details
-
init
Init metrics accumulator if it has not been initiated. This method is idempotent. -
init
public static void init(SparkPipelineOptions opts, org.apache.spark.api.java.JavaSparkContext jsc, boolean useCheckpoint) Init metrics accumulator if it has not been initiated. This method is idempotent. WithuseCheckpointset, the value is recovered from the metrics checkpoint under the checkpoint directory ofopts, andMetricsAccumulator.AccumulatorCheckpointingSparkListenerwrites it back there. The DStream streaming path is the one using that checkpoint. -
getInstance
-
clear
public static void clear()
-