apache_beam.runners.direct.transform_evaluator module

An evaluator of a specific application of a transform.

class apache_beam.runners.direct.transform_evaluator.TransformEvaluatorRegistry(evaluation_context)

Bases: future.types.newobject.newobject

For internal use only; no backwards-compatibility guarantees.

Creates instances of TransformEvaluator for the application of a transform.

get_evaluator(applied_ptransform, input_committed_bundle, side_inputs)

Returns a TransformEvaluator suitable for processing given inputs.

get_root_bundle_provider(applied_ptransform)

should_execute_serially(applied_ptransform)

Returns True if this applied_ptransform should run one bundle at a time.

Some TransformEvaluators use a global state object to keep track of their global execution state. For example evaluator for _GroupByKeyOnly uses this state as an in memory dictionary to buffer keys.

Serially executed evaluators will act as syncing point in the graph and execution will not move forward until they receive all of their inputs. Once they receive all of their input, they will release the combined output. Their output may consist of multiple bundles as they may divide their output into pieces before releasing.

Parameters:applied_ptransform – Transform to be used for execution. Returns:True if executor should execute applied_ptransform serially.

class apache_beam.runners.direct.transform_evaluator.RootBundleProvider(evaluation_context, applied_ptransform)

Bases: future.types.newobject.newobject

Provides bundles for the initial execution of a root transform.

get_root_bundles()

class apache_beam.runners.direct.transform_evaluator.DefaultRootBundleProvider(evaluation_context, applied_ptransform)

Bases: apache_beam.runners.direct.transform_evaluator.RootBundleProvider

Provides an empty bundle by default for root transforms.

get_root_bundles()