org.apache.beam.runners.spark.structuredstreaming.translation.batch

Class PipelineTranslatorBatch

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.structuredstreaming.translation.PipelineTranslator

org.apache.beam.runners.spark.structuredstreaming.translation.batch.PipelineTranslatorBatch

All Implemented Interfaces:

Pipeline.PipelineVisitor

public class PipelineTranslatorBatch
extends PipelineTranslator

PipelineTranslator for executing a Pipeline in Spark in batch mode. This contains only the components specific to batch: registry of batch TransformTranslator and registry lookup code.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Pipeline.PipelineVisitor.CompositeBehavior, Pipeline.PipelineVisitor.Defaults

Field Summary

Fields inherited from class org.apache.beam.runners.spark.structuredstreaming.translation.PipelineTranslator

translationContext

Constructor Summary

Constructors

Constructor and Description
PipelineTranslatorBatch(SparkStructuredStreamingPipelineOptions options)

Method Summary

Modifier and Type	Method and Description
protected TransformTranslator<?>	getTransformTranslator(org.apache.beam.sdk.runners.TransformHierarchy.Node node) Returns a translator for the given node, if it is possible, otherwise null.

Methods inherited from class org.apache.beam.runners.spark.structuredstreaming.translation.PipelineTranslator

detectTranslationMode, enterCompositeTransform, getTranslationContext, leaveCompositeTransform, prepareFilesToStageForRemoteClusterExecution, replaceTransforms, translate, visitPrimitiveTransform

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

enterPipeline, getPipeline, leavePipeline, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PipelineTranslatorBatch

public PipelineTranslatorBatch(SparkStructuredStreamingPipelineOptions options)

Method Detail

getTransformTranslator

protected TransformTranslator<?> getTransformTranslator(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Returns a translator for the given node, if it is possible, otherwise null.

Specified by:

getTransformTranslator in class PipelineTranslator