Package org.apache.beam.runners.flink.translation.functions

Class FlinkDoFnFunction<InputT,OutputT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction<InputT,OutputT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.FlatMapFunction<WindowedValue<InputT>,WindowedValue<RawUnionValue>>, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

public class FlinkDoFnFunction<InputT,OutputT>
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.api.common.functions.FlatMapFunction<WindowedValue<InputT>,WindowedValue<RawUnionValue>>

Encapsulates a DoFn inside a Flink RichMapPartitionFunction.

We get a mapping from TupleTag to output index and must tag all outputs with the output number. Afterwards a filter will filter out those elements that are not to be in a specific output.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor

Description

FlinkDoFnFunction(DoFn<InputT,OutputT> doFn, String stepName, WindowingStrategy<?,?> windowingStrategy, Map<PCollectionView<?>,WindowingStrategy<?,?>> sideInputs, PipelineOptions options, Map<TupleTag<?>,Integer> outputMap, TupleTag<OutputT> mainOutputTag, Coder<InputT> inputCoder, Map<TupleTag<?>,Coder<?>> outputCoderMap, DoFnSchemaInformation doFnSchemaInformation, Map<String,PCollectionView<?>> sideInputMapping)

Method Summary

Modifier and Type	Method	Description
void	close()
void	flatMap(WindowedValue<InputT> value, org.apache.flink.util.Collector<WindowedValue<RawUnionValue>> out)
void	open(org.apache.flink.configuration.Configuration parameters)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FlinkDoFnFunction

public FlinkDoFnFunction(DoFn<InputT,OutputT> doFn,
 String stepName,
 WindowingStrategy<?,?> windowingStrategy,
 Map<PCollectionView<?>,WindowingStrategy<?,?>> sideInputs,
 PipelineOptions options,
 Map<TupleTag<?>,Integer> outputMap,
 TupleTag<OutputT> mainOutputTag,
 Coder<InputT> inputCoder,
 Map<TupleTag<?>,Coder<?>> outputCoderMap,
 DoFnSchemaInformation doFnSchemaInformation,
 Map<String,PCollectionView<?>> sideInputMapping)

Method Details

flatMap

public void flatMap(WindowedValue<InputT> value,
 org.apache.flink.util.Collector<WindowedValue<RawUnionValue>> out)

Specified by:

flatMap in interface org.apache.flink.api.common.functions.FlatMapFunction<InputT,OutputT>

open

public void open(org.apache.flink.configuration.Configuration parameters)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception