org.apache.beam.runners.spark

Class SparkRunner.Evaluator

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.SparkRunner.Evaluator

All Implemented Interfaces:

Pipeline.PipelineVisitor

Direct Known Subclasses:

SparkNativePipelineVisitor

Enclosing class:

SparkRunner

public static class SparkRunner.Evaluator
extends Pipeline.PipelineVisitor.Defaults

Evaluator on the pipeline.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Pipeline.PipelineVisitor.CompositeBehavior, Pipeline.PipelineVisitor.Defaults

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.beam.runners.spark.translation.EvaluationContext	ctxt
protected org.apache.beam.runners.spark.translation.SparkPipelineTranslator	translator

Constructor Summary

Constructors

Constructor and Description
Evaluator(org.apache.beam.runners.spark.translation.SparkPipelineTranslator translator, org.apache.beam.runners.spark.translation.EvaluationContext ctxt)

Method Summary

Modifier and Type	Method and Description
Pipeline.PipelineVisitor.CompositeBehavior	enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node) Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.
protected PCollection.IsBounded	isBoundedCollection(java.util.Collection<PValue> pValues)
protected boolean	shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
protected <TransformT extends PTransform<? super PInput,POutput>> org.apache.beam.runners.spark.translation.TransformEvaluator<TransformT>	translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node, TransformT transform, java.lang.Class<TransformT> transformClass) Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.
void	visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node) Called for each primitive transform after all of its topological predecessors and inputs have been visited.

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

leaveCompositeTransform, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ctxt

protected final org.apache.beam.runners.spark.translation.EvaluationContext ctxt

translator

protected final org.apache.beam.runners.spark.translation.SparkPipelineTranslator translator

Constructor Detail

Evaluator

public Evaluator(org.apache.beam.runners.spark.translation.SparkPipelineTranslator translator,
                 org.apache.beam.runners.spark.translation.EvaluationContext ctxt)

Method Detail

enterCompositeTransform

public Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Description copied from interface: Pipeline.PipelineVisitor

Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.

The return value controls whether or not child transforms are visited.

Specified by:

enterCompositeTransform in interface Pipeline.PipelineVisitor

Overrides:

enterCompositeTransform in class Pipeline.PipelineVisitor.Defaults

shouldDefer

protected boolean shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

visitPrimitiveTransform

public void visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Description copied from interface: Pipeline.PipelineVisitor

Called for each primitive transform after all of its topological predecessors and inputs have been visited.

Specified by:

visitPrimitiveTransform in interface Pipeline.PipelineVisitor

Overrides:

visitPrimitiveTransform in class Pipeline.PipelineVisitor.Defaults

translate

protected <TransformT extends PTransform<? super PInput,POutput>> org.apache.beam.runners.spark.translation.TransformEvaluator<TransformT> translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node,
                                                                                                                                                     TransformT transform,
                                                                                                                                                     java.lang.Class<TransformT> transformClass)

Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.

isBoundedCollection

protected PCollection.IsBounded isBoundedCollection(java.util.Collection<PValue> pValues)