apache_beam.runners.dataflow.ptransform_overrides module

Ptransform overrides for DataflowRunner.

class apache_beam.runners.dataflow.ptransform_overrides.CreatePTransformOverride[source]

Bases: apache_beam.pipeline.PTransformOverride

A PTransformOverride for Create in streaming mode.

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]
class apache_beam.runners.dataflow.ptransform_overrides.ReadPTransformOverride[source]

Bases: apache_beam.pipeline.PTransformOverride

A PTransformOverride for Read(BoundedSource)

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]
class apache_beam.runners.dataflow.ptransform_overrides.JrhReadPTransformOverride[source]

Bases: apache_beam.pipeline.PTransformOverride

A PTransformOverride for Read(BoundedSource)

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]
class apache_beam.runners.dataflow.ptransform_overrides.CombineValuesPTransformOverride[source]

Bases: apache_beam.pipeline.PTransformOverride

A PTransformOverride for CombineValues.

The DataflowRunner expects that the CombineValues PTransform acts as a primitive. So this override replaces the CombineValues with a primitive.

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]
class apache_beam.runners.dataflow.ptransform_overrides.NativeReadPTransformOverride[source]

Bases: apache_beam.pipeline.PTransformOverride

A PTransformOverride for Read using native sources.

The DataflowRunner expects that the Read PTransform using native sources act as a primitive. So this override replaces the Read with a primitive.

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]
class apache_beam.runners.dataflow.ptransform_overrides.WriteToBigQueryPTransformOverride(pipeline, options)[source]

Bases: apache_beam.pipeline.PTransformOverride

matches(applied_ptransform)[source]
get_replacement_transform(ptransform)[source]