Package org.apache.beam.runners.flink.translation.functions

Class AbstractFlinkCombineRunner<K,InputT,AccumT,OutputT,W extends BoundedWindow>

java.lang.Object

org.apache.beam.runners.flink.translation.functions.AbstractFlinkCombineRunner<K,InputT,AccumT,OutputT,W>

Direct Known Subclasses:

HashingFlinkCombineRunner, SingleWindowFlinkCombineRunner, SortingFlinkCombineRunner

public abstract class AbstractFlinkCombineRunner<K,InputT,AccumT,OutputT,W extends BoundedWindow>
extends Object

Abstract base for runners that execute a Combine.PerKey. This unifies processing of merging/non-merging and partial/final combines.

The input to combine(FlinkCombiner, WindowingStrategy, SideInputReader, PipelineOptions, Iterable, Collector) are elements of the same key but * for different windows.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AbstractFlinkCombineRunner.CompleteFlinkCombiner<K,InputT,AccumT,OutputT>	A straight wrapper of CombineFnBase.GlobalCombineFn that takes in InputT and emits OutputT.
static class	AbstractFlinkCombineRunner.FinalFlinkCombiner<K,AccumT,OutputT>	A final combiner that takes in AccumT and produces OutputT.
static interface	AbstractFlinkCombineRunner.FlinkCombiner<K,InputT,AccumT,OutputT>	Adapter interface that allows using a CombineFnBase.GlobalCombineFn to either produce the AccumT as output or to combine several accumulators into an OutputT.
static class	AbstractFlinkCombineRunner.PartialFlinkCombiner<K,InputT,AccumT>	A partial combiner that takes in InputT and produces AccumT.

Constructor Summary

Constructors

Constructor	Description
AbstractFlinkCombineRunner()

Method Summary

Modifier and Type	Method	Description
abstract void	combine(AbstractFlinkCombineRunner.FlinkCombiner<K,InputT,AccumT,OutputT> flinkCombiner, WindowingStrategy<Object,W> windowingStrategy, org.apache.beam.runners.core.SideInputReader sideInputReader, PipelineOptions options, Iterable<WindowedValue<KV<K,InputT>>> elements, org.apache.flink.util.Collector<WindowedValue<KV<K,OutputT>>> out)	Consumes WindowedValues and produces combined output to the given output.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AbstractFlinkCombineRunner

public AbstractFlinkCombineRunner()

Method Details

combine

public abstract void combine(AbstractFlinkCombineRunner.FlinkCombiner<K,InputT,AccumT,OutputT> flinkCombiner,
 WindowingStrategy<Object,W> windowingStrategy,
 org.apache.beam.runners.core.SideInputReader sideInputReader,
 PipelineOptions options,
 Iterable<WindowedValue<KV<K,InputT>>> elements,
 org.apache.flink.util.Collector<WindowedValue<KV<K,OutputT>>> out)
                      throws Exception

Consumes WindowedValues and produces combined output to the given output.

Throws:

Exception