See: Description
| Interface | Description |
|---|---|
| TestDataflowPipelineOptions |
A set of options used to configure the
TestPipeline. |
| TransformTranslator<TransformT extends PTransform> |
A
TransformTranslator knows how to translate a particular subclass of PTransform
for the Cloud Dataflow service. |
| TransformTranslator.StepTranslationContext |
The interface for a
TransformTranslator to build a Dataflow step. |
| TransformTranslator.TranslationContext |
The interface provided to registered callbacks for interacting with the
DataflowRunner,
including reading and writing the values of PCollections and side inputs. |
| Class | Description |
|---|---|
| BatchStatefulParDoOverrides |
PTransformOverrideFactories that expands to correctly implement
stateful ParDo using window-unaware BatchViewOverrides.GroupByKeyAndSortValuesOnly to linearize
processing per key. |
| BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT> |
A key-preserving
DoFn that explodes an iterable that has been grouped by key and
window. |
| CreateDataflowView<ElemT,ViewT> |
A
DataflowRunner marker class for creating a PCollectionView. |
| DataflowClient |
Wrapper around the generated
Dataflow client to provide common functionality. |
| DataflowPipelineJob |
A DataflowPipelineJob represents a job submitted to Dataflow using
DataflowRunner. |
| DataflowPipelineRegistrar | |
| DataflowPipelineRegistrar.Options |
Register the
DataflowPipelineOptions. |
| DataflowPipelineRegistrar.Runner |
Register the
DataflowRunner. |
| DataflowPipelineTranslator |
DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud
Dataflow Service API Jobs. |
| DataflowPipelineTranslator.JobSpecification |
The result of a job translation.
|
| DataflowRunner |
A
PipelineRunner that executes the operations in the pipeline by first translating them
to the Dataflow representation using the DataflowPipelineTranslator and then submitting
them to a Dataflow service for execution. |
| DataflowRunner.StreamingPCollectionViewWriterFn<T> |
A marker
DoFn for writing the contents of a PCollection to a streaming PCollectionView backend implementation. |
| DataflowRunnerHooks |
An instance of this class can be passed to the
DataflowRunner to add user defined hooks
to be invoked at various times during pipeline execution. |
| DataflowRunnerInfo |
Populates versioning and other information for
DataflowRunner. |
| GroupIntoBatchesOverride | |
| PrimitiveParDoSingleFactory<InputT,OutputT> |
A
PTransformOverrideFactory that produces PrimitiveParDoSingleFactory.ParDoSingle instances from ParDo.SingleOutput instances. |
| PrimitiveParDoSingleFactory.ParDoSingle<InputT,OutputT> |
A single-output primitive
ParDo. |
| PrimitiveParDoSingleFactory.PayloadTranslator |
A translator for
PrimitiveParDoSingleFactory.ParDoSingle. |
| PrimitiveParDoSingleFactory.Registrar | |
| TestDataflowRunner |
TestDataflowRunner is a pipeline runner that wraps a DataflowRunner when running
tests against the TestPipeline. |
| Exception | Description |
|---|---|
| DataflowJobAlreadyExistsException |
An exception that is thrown if the unique job name constraint of the Dataflow service is broken
because an existing job with the same job name is currently active.
|
| DataflowJobAlreadyUpdatedException |
An exception that is thrown if the existing job has already been updated within the Dataflow
service and is no longer able to be updated.
|
| DataflowJobException |
A
RuntimeException that contains information about a DataflowPipelineJob. |
| DataflowServiceException |
Signals there was an error retrieving information about a job from the Cloud Dataflow Service.
|