org.apache.beam.runners.spark.SparkNativePipelineVisitor

public class SparkNativePipelineVisitor extends SparkRunner.Evaluator

Pipeline visitor for translating a Beam pipeline into equivalent Spark operations. Used for debugging purposes using SparkRunnerDebugger.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor
Pipeline.PipelineVisitor.CompositeBehavior, Pipeline.PipelineVisitor.Defaults
Field Summary

Fields inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator
ctxt, translator
Method Summary

Modifier and Type

Method

Description

Pipeline.PipelineVisitor.CompositeBehavior

enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.

Methods inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator
isBoundedCollection, shouldDefer, translate, visitPrimitiveTransform

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults
enterPipeline, getPipeline, leaveCompositeTransform, leavePipeline, visitValue

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Class SparkNativePipelineVisitor

Nested Class Summary