Package org.apache.beam.runners.flink.translation.functions

Class FlinkExecutableStageFunction<InputT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction<InputT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.GroupReduceFunction<WindowedValue<InputT>,RawUnionValue>, org.apache.flink.api.common.functions.MapPartitionFunction<WindowedValue<InputT>,RawUnionValue>, org.apache.flink.api.common.functions.RichFunction

public class FlinkExecutableStageFunction<InputT>
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.api.common.functions.MapPartitionFunction<WindowedValue<InputT>,RawUnionValue>, org.apache.flink.api.common.functions.GroupReduceFunction<WindowedValue<InputT>,RawUnionValue>

Flink operator that passes its input DataSet through an SDK-executed ExecutableStage.

The output of this operation is a multiplexed DataSet whose elements are tagged with a union coder. The coder's tags are determined by the output coder map. The resulting data set should be further processed by a FlinkExecutableStagePruningFunction.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
FlinkExecutableStageFunction(String stepName, PipelineOptions pipelineOptions, org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload, JobInfo jobInfo, Map<String,Integer> outputMap, FlinkExecutableStageContextFactory contextFactory, Coder windowCoder, Coder<WindowedValue<InputT>> inputCoder)

Method Summary

Modifier and Type	Method	Description
void	close()
void	mapPartition(Iterable<WindowedValue<InputT>> iterable, org.apache.flink.util.Collector<RawUnionValue> collector)	For non-stateful processing via a simple MapPartitionFunction.
void	open(org.apache.flink.configuration.Configuration parameters)
void	reduce(Iterable<WindowedValue<InputT>> iterable, org.apache.flink.util.Collector<RawUnionValue> collector)	For stateful and timer processing via a GroupReduceFunction.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FlinkExecutableStageFunction

public FlinkExecutableStageFunction(String stepName,
 PipelineOptions pipelineOptions,
 org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload,
 JobInfo jobInfo,
 Map<String,Integer> outputMap,
 FlinkExecutableStageContextFactory contextFactory,
 Coder windowCoder,
 Coder<WindowedValue<InputT>> inputCoder)

Method Details

open

public void open(org.apache.flink.configuration.Configuration parameters)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

mapPartition

public void mapPartition(Iterable<WindowedValue<InputT>> iterable,
 org.apache.flink.util.Collector<RawUnionValue> collector)
                  throws Exception

For non-stateful processing via a simple MapPartitionFunction.

Specified by:

mapPartition in interface org.apache.flink.api.common.functions.MapPartitionFunction<WindowedValue<InputT>,RawUnionValue>

Throws:

Exception

reduce

public void reduce(Iterable<WindowedValue<InputT>> iterable,
 org.apache.flink.util.Collector<RawUnionValue> collector)
            throws Exception

For stateful and timer processing via a GroupReduceFunction.

Specified by:

reduce in interface org.apache.flink.api.common.functions.GroupReduceFunction<WindowedValue<InputT>,RawUnionValue>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception