org.apache.beam.runners.spark.aggregators

Class AggregatorsAccumulator

java.lang.Object

org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator

public class AggregatorsAccumulator
extends java.lang.Object

For resilience, Accumulators are required to be wrapped in a Singleton.

See Also:

accumulatorsV2

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AggregatorsAccumulator.AccumulatorCheckpointingSparkListener Spark Listener which checkpoints NamedAggregators values for fault-tolerance.

Constructor Summary

Constructors

Constructor and Description
AggregatorsAccumulator()

Method Summary

Modifier and Type	Method and Description
static void	clear()
static NamedAggregatorsAccumulator	getInstance()
static void	init(SparkPipelineOptions opts, org.apache.spark.api.java.JavaSparkContext jsc) Init aggregators accumulator if it has not been initiated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AggregatorsAccumulator

public AggregatorsAccumulator()

Method Detail

init

public static void init(SparkPipelineOptions opts,
                        org.apache.spark.api.java.JavaSparkContext jsc)

Init aggregators accumulator if it has not been initiated. This method is idempotent.

getInstance

public static NamedAggregatorsAccumulator getInstance()

clear

public static void clear()