Package org.apache.beam.runners.flink

Interface FlinkPortablePipelineTranslator<T extends FlinkPortablePipelineTranslator.TranslationContext>

All Known Implementing Classes:

FlinkBatchPortablePipelineTranslator, FlinkStreamingPortablePipelineTranslator

public interface FlinkPortablePipelineTranslator<T extends FlinkPortablePipelineTranslator.TranslationContext>

Interface for portable Flink translators. This allows for a uniform invocation pattern for pipeline translation between streaming and portable runners.

Pipeline translators will generally provide a mechanism to produce the translation contexts that they use for pipeline translation. Post translation, the translation context should contain a pipeline plan that has not yet been executed.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	FlinkPortablePipelineTranslator.Executor	A handle used to execute a translated pipeline.
static interface	FlinkPortablePipelineTranslator.TranslationContext	The context used for pipeline translation.

Method Summary

Modifier and Type	Method	Description
T	createTranslationContext(JobInfo jobInfo, FlinkPipelineOptions pipelineOptions, @Nullable String confDir, List<String> filesToStage)
Set<String>	knownUrns()
default org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline	prepareForTranslation(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)
FlinkPortablePipelineTranslator.Executor	translate(T context, org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)	Translates the given pipeline.

Method Details

createTranslationContext

T createTranslationContext(JobInfo jobInfo,
 FlinkPipelineOptions pipelineOptions,
 @Nullable String confDir,
 List<String> filesToStage)

knownUrns

Set<String> knownUrns()

prepareForTranslation

default org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline prepareForTranslation(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)

translate

FlinkPortablePipelineTranslator.Executor translate(T context,
 org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)

Translates the given pipeline.