org.apache.beam.runners.spark

Class SparkNativePipelineVisitor

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.SparkRunner.Evaluator

org.apache.beam.runners.spark.SparkNativePipelineVisitor

All Implemented Interfaces:

Pipeline.PipelineVisitor

public class SparkNativePipelineVisitor
extends SparkRunner.Evaluator

Pipeline visitor for translating a Beam pipeline into equivalent Spark operations. Used for debugging purposes using SparkRunnerDebugger.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Pipeline.PipelineVisitor.CompositeBehavior, Pipeline.PipelineVisitor.Defaults

Field Summary

Fields inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

ctxt, translator

Method Summary

Modifier and Type	Method and Description
Pipeline.PipelineVisitor.CompositeBehavior	enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node) Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.

Methods inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

isBoundedCollection, shouldDefer, translate, visitPrimitiveTransform

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

leaveCompositeTransform, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

enterCompositeTransform

public Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Description copied from interface: Pipeline.PipelineVisitor

Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.

The return value controls whether or not child transforms are visited.

Specified by:

enterCompositeTransform in interface Pipeline.PipelineVisitor

Overrides:

enterCompositeTransform in class SparkRunner.Evaluator