See: Description
| Interface | Description |
|---|---|
| PipelineTranslator.TranslationState |
Shared, mutable state during the translation of a pipeline and omitted afterwards.
|
| PipelineTranslator.UnresolvedTranslation<InT,T> |
Unresolved translation, allowing to optimize the generated Spark DAG.
|
| Class | Description |
|---|---|
| EvaluationContext |
The
EvaluationContext is the result of a pipeline translation and can be used to evaluate / run the pipeline. |
| PipelineTranslator |
The pipeline translator translates a Beam
Pipeline into a Spark correspondence, that can
then be evaluated. |
| SparkSessionFactory | |
| SparkSessionFactory.SparkKryoRegistrator |
KryoRegistrator for Spark to serialize broadcast variables used for side-inputs. |
| TransformTranslator<InT extends PInput,OutT extends POutput,TransformT extends PTransform<InT,OutT>> |
A
TransformTranslator provides the capability to translate a specific primitive or
composite PTransform into its Spark correspondence. |