See: Description
| Interface | Description |
|---|---|
| SparkContextOptions |
A custom
PipelineOptions to work with properties related to JavaSparkContext. |
| SparkPipelineOptions |
Spark runner
PipelineOptions handles Spark execution-related configurations,
such as the master address, batch-interval, and other user-related knobs. |
| TestSparkPipelineOptions |
A
SparkPipelineOptions for tests. |
| Class | Description |
|---|---|
| SparkContextOptions.EmptyListenersList |
Returns an empty list, top avoid handling null.
|
| SparkNativePipelineVisitor |
Pipeline visitor for translating a Beam pipeline into equivalent Spark operations.
|
| SparkPipelineOptions.TmpCheckpointDirFactory |
Returns the default checkpoint directory of /tmp/${job.name}.
|
| SparkPipelineResult |
Represents a Spark pipeline execution result.
|
| SparkRunner |
The SparkRunner translate operations defined on a pipeline to a representation executable by
Spark, and then submitting the job to Spark to be executed.
|
| SparkRunner.Evaluator |
Evaluator on the pipeline.
|
| SparkRunnerDebugger |
Pipeline runner which translates a Beam pipeline into equivalent Spark operations, without
running them.
|
| SparkRunnerDebugger.DebugSparkPipelineResult |
PipelineResult of running a
Pipeline using SparkRunnerDebugger
Use SparkRunnerDebugger.DebugSparkPipelineResult.getDebugString() to get a String representation of the Pipeline
translated into Spark native operations. |
| SparkRunnerRegistrar | |
| SparkRunnerRegistrar.Options |
Registers the
SparkPipelineOptions. |
| SparkRunnerRegistrar.Runner |
Registers the
SparkRunner. |
| TestSparkPipelineOptions.DefaultStopPipelineWatermarkFactory |
A factory to provide the default watermark to stop a pipeline that reads
from an unbounded source.
|
| TestSparkRunner |
The SparkRunner translate operations defined on a pipeline to a representation executable
by Spark, and then submitting the job to Spark to be executed.
|