Package org.apache.beam.runners.flink.translation.functions
package org.apache.beam.runners.flink.translation.functions
Internal implementation of the Beam runner for Apache Flink.
-
ClassDescriptionAbstractFlinkCombineRunner<K,
InputT, AccumT, OutputT, W extends BoundedWindow> Abstract base for runners that execute aCombine.PerKey
.AbstractFlinkCombineRunner.CompleteFlinkCombiner<K,InputT, AccumT, OutputT> AbstractFlinkCombineRunner.FinalFlinkCombiner<K,AccumT, OutputT> A final combiner that takes inAccumT
and producesOutputT
.AbstractFlinkCombineRunner.FlinkCombiner<K,InputT, AccumT, OutputT> Adapter interface that allows using aCombineFnBase.GlobalCombineFn
to either produce theAccumT
as output or to combine several accumulators into anOutputT
.AbstractFlinkCombineRunner.PartialFlinkCombiner<K,InputT, AccumT> A partial combiner that takes inInputT
and producesAccumT
.FlinkAssignWindows<T,W extends BoundedWindow> FlinkFlatMapFunction
for implementingWindow.Assign
.FlinkDoFnFunction<InputT,OutputT> Encapsulates aDoFn
inside a FlinkRichMapPartitionFunction
.Singleton class that contains oneExecutableStageContext.Factory
per job.FlinkExecutableStageFunction<InputT>Flink operator that passes its input DataSet through an SDK-executedExecutableStage
.A Flink function that demultiplexes output from aFlinkExecutableStageFunction
.ExplodeWindowedValue
that belongs to multiple windows into multiple "single window"values
, so we can safely group elements by (K, W) tuples.A map function that outputs the input element without any change.FlinkMergingNonShuffleReduceFunction<K,InputT, AccumT, OutputT, W extends BoundedWindow> Special version ofFlinkReduceFunction
that supports merging windows.AFlatMapFunction
function that filters out those elements that don't belong in this output.FlinkNonMergingReduceFunction<K,InputT> Reduce function for non-merging GBK implementation.AStepContext
for Flink Batch Runner execution.FlinkPartialReduceFunction<K,InputT, AccumT, W extends BoundedWindow> This is the first step for executing aCombine.PerKey
on Flink.FlinkReduceFunction<K,AccumT, OutputT, W extends BoundedWindow> This is the second part for executing aCombine.PerKey
on Flink, the second part isFlinkReduceFunction
.ASideInputReader
for the Flink Batch Runner.FlinkStatefulDoFnFunction<K,V, OutputT> ARichGroupReduceFunction
for statefulParDo
in Flink Batch Runner.HashingFlinkCombineRunner<K,InputT, AccumT, OutputT, W extends BoundedWindow> A Flink combine runner that builds a map of merged windows and produces output after seeing all input.Source function which sends a single global impulse to a downstream operator.SideInputInitializer<ViewT>BroadcastVariableInitializer
that initializes the broadcast input as aMap
from window to side input.SingleWindowFlinkCombineRunner<K,InputT, AccumT, OutputT, W extends BoundedWindow> A Flink combine runner takes elements pre-grouped by window and produces output after seeing all input.SortingFlinkCombineRunner<K,InputT, AccumT, OutputT, W extends BoundedWindow> A Flink combine runner that first sorts the elements by window and then does one pass that merges windows and outputs results.