org.apache.beam.runners.flink

Interface FlinkPortablePipelineTranslator<T extends FlinkPortablePipelineTranslator.TranslationContext>

All Known Implementing Classes:

FlinkBatchPortablePipelineTranslator, FlinkStreamingPortablePipelineTranslator

public interface FlinkPortablePipelineTranslator<T extends FlinkPortablePipelineTranslator.TranslationContext>

Interface for portable Flink translators. This allows for a uniform invocation pattern for pipeline translation between streaming and portable runners.

Pipeline translators will generally provide a mechanism to produce the translation contexts that they use for pipeline translation. Post translation, the translation context should contain a pipeline plan that has not yet been executed.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	FlinkPortablePipelineTranslator.Executor A handle used to execute a translated pipeline.
static interface	FlinkPortablePipelineTranslator.TranslationContext The context used for pipeline translation.

Method Summary

Modifier and Type	Method and Description
T	createTranslationContext(JobInfo jobInfo, FlinkPipelineOptions pipelineOptions, @Nullable java.lang.String confDir, java.util.List<java.lang.String> filesToStage)
java.util.Set<java.lang.String>	knownUrns()
default org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline	prepareForTranslation(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)
FlinkPortablePipelineTranslator.Executor	translate(T context, org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline) Translates the given pipeline.

Method Detail

createTranslationContext

T createTranslationContext(JobInfo jobInfo,
                           FlinkPipelineOptions pipelineOptions,
                           @Nullable java.lang.String confDir,
                           java.util.List<java.lang.String> filesToStage)

knownUrns

java.util.Set<java.lang.String> knownUrns()

prepareForTranslation

default org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline prepareForTranslation(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)

translate

FlinkPortablePipelineTranslator.Executor translate(T context,
                                                   org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline)

Translates the given pipeline.