Package org.apache.beam.runners.spark.translation

Interface SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT extends SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT>>

Enclosing class:

SparkCombineFn<InputT,ValueT,AccumT,OutputT>

public static interface SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT extends SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT>>

Accumulator of WindowedValues holding values for different windows.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static enum	SparkCombineFn.WindowedAccumulator.Type	Type of the accumulator.

Method Summary

Modifier and Type	Method	Description
void	add(WindowedValue<InputT> value, SparkCombineFn<InputT,ValueT,AccumT,?> context)	Add value with unexploded windows into the accumulator.
static <InputT, ValueT, AccumT> SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?>	create(SparkCombineFn<InputT,ValueT,AccumT,?> context, org.apache.spark.api.java.function.Function<InputT,ValueT> toValue, WindowingStrategy<?,?> windowingStrategy, Comparator<BoundedWindow> windowComparator)	Create concrete accumulator for given type.
static <InputT, ValueT, AccumT> SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?>	create(org.apache.spark.api.java.function.Function<InputT,ValueT> toValue, SparkCombineFn.WindowedAccumulator.Type type, Iterable<WindowedValue<AccumT>> values, Comparator<BoundedWindow> windowComparator)	Create concrete accumulator for given type.
static <InputT, ValueT, AccumT> SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?>	create(org.apache.spark.api.java.function.Function<InputT,ValueT> toValue, SparkCombineFn.WindowedAccumulator.Type type, Comparator<BoundedWindow> windowComparator)
Collection<WindowedValue<AccumT>>	extractOutput()	Extract output.
boolean	isEmpty()	Check if this accumulator is empty.
void	merge(ImplT other, SparkCombineFn<?,?,AccumT,?> context)	Merge other accumulator into this one.

Method Details

isEmpty

boolean isEmpty()

Check if this accumulator is empty.

Returns:

true if this accumulator is empty

add

void add(WindowedValue<InputT> value,
 SparkCombineFn<InputT,ValueT,AccumT,?> context)
  throws Exception

Add value with unexploded windows into the accumulator.

Throws:

Exception

merge

void merge(ImplT other,
 SparkCombineFn<?,?,AccumT,?> context)
    throws Exception

Merge other accumulator into this one.

Parameters:

other - the other accumulator to merge

Throws:

Exception

extractOutput

Collection<WindowedValue<AccumT>> extractOutput()

Extract output.

create

static <InputT,
ValueT,
AccumT>
SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?> create(SparkCombineFn<InputT,ValueT,AccumT,?> context,
 org.apache.spark.api.java.function.Function<InputT,ValueT> toValue,
 WindowingStrategy<?,?> windowingStrategy,
 Comparator<BoundedWindow> windowComparator)

Create concrete accumulator for given type.

create

static <InputT,
ValueT,
AccumT>
SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?> create(org.apache.spark.api.java.function.Function<InputT,ValueT> toValue,
 SparkCombineFn.WindowedAccumulator.Type type,
 Comparator<BoundedWindow> windowComparator)

create

static <InputT,
ValueT,
AccumT>
SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,?> create(org.apache.spark.api.java.function.Function<InputT,ValueT> toValue,
 SparkCombineFn.WindowedAccumulator.Type type,
 Iterable<WindowedValue<AccumT>> values,
 Comparator<BoundedWindow> windowComparator)

Create concrete accumulator for given type.