Package org.apache.beam.runners.dataflow

Class DataflowPipelineTranslator

java.lang.Object

org.apache.beam.runners.dataflow.DataflowPipelineTranslator

public class DataflowPipelineTranslator
extends Object

DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DataflowPipelineTranslator.JobSpecification	The result of a job translation.

Method Summary

Modifier and Type	Method	Description
static DataflowPipelineTranslator	fromOptions(DataflowPipelineOptions options)	Constructs a translator from the provided options.
<TransformT extends PTransform> TransformTranslator<TransformT>	getTransformTranslator(Class<TransformT> transformClass)	Returns the TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
static String	jobToString(Job job)	Renders a Job as a string.
static <TransformT extends PTransform> void	registerTransformTranslator(Class<TransformT> transformClass, TransformTranslator<? extends TransformT> transformTranslator)	Records that instances of the specified PTransform class should be translated by default by the corresponding TransformTranslator.
String	toString()
DataflowPipelineTranslator.JobSpecification	translate(Pipeline pipeline, org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipelineProto, org.apache.beam.sdk.util.construction.SdkComponents sdkComponents, DataflowRunner runner, List<DataflowPackage> packages)	Translates a Pipeline into a JobSpecification.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

fromOptions

public static DataflowPipelineTranslator fromOptions(DataflowPipelineOptions options)

Constructs a translator from the provided options.

Parameters:

options - Properties that configure the translator.

Returns:

The newly created translator.

translate

public DataflowPipelineTranslator.JobSpecification translate(Pipeline pipeline,
 org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipelineProto,
 org.apache.beam.sdk.util.construction.SdkComponents sdkComponents,
 DataflowRunner runner,
 List<DataflowPackage> packages)

Translates a Pipeline into a JobSpecification.

jobToString

public static String jobToString(Job job)

Renders a Job as a string.

registerTransformTranslator

public static <TransformT extends PTransform> void registerTransformTranslator(Class<TransformT> transformClass,
 TransformTranslator<? extends TransformT> transformTranslator)

Records that instances of the specified PTransform class should be translated by default by the corresponding TransformTranslator.

getTransformTranslator

public <TransformT extends PTransform>
TransformTranslator<TransformT> getTransformTranslator(Class<TransformT> transformClass)

Returns the TransformTranslator to use for instances of the specified PTransform class, or null if none registered.

toString

public String toString()

Overrides:

toString in class Object