apache_beam.yaml.yaml_provider module¶
This module defines Providers usable from yaml, which is a specification for where to find and how to invoke services that vend implementations of various PTransforms.
-
class
apache_beam.yaml.yaml_provider.
Provider
[source]¶ Bases:
object
Maps transform types names and args to concrete PTransform instances.
-
provided_transforms
() → Iterable[str][source]¶ Returns a list of transform type names this provider can handle.
-
requires_inputs
(typ: str, args: Mapping[str, Any]) → bool[source]¶ Returns whether this transform requires inputs.
Specifically, if this returns True and inputs are not provided than an error will be thrown.
This is best-effort, primarily for better and earlier error messages.
-
create_transform
(typ: str, args: Mapping[str, Any], yaml_create_transform: Callable[[Mapping[str, Any], Iterable[apache_beam.pvalue.PCollection]], apache_beam.transforms.ptransform.PTransform]) → apache_beam.transforms.ptransform.PTransform[source]¶ Creates a PTransform instance for the given transform type and arguments.
-
-
class
apache_beam.yaml.yaml_provider.
ExternalProvider
(urns, service)[source]¶ Bases:
apache_beam.yaml.yaml_provider.Provider
A Provider implemented via the cross language transform service.
-
apache_beam.yaml.yaml_provider.
maven_jar
(urns, *, artifact_id, group_id, version, repository='https://repo.maven.apache.org/maven2', classifier=None, appendix=None)[source]¶
-
apache_beam.yaml.yaml_provider.
beam_jar
(urns, *, gradle_target, appendix=None, version='2.51.0', artifact_id=None)[source]¶
-
class
apache_beam.yaml.yaml_provider.
InlineProvider
(transform_factories, no_input_transforms=())[source]¶
-
class
apache_beam.yaml.yaml_provider.
MetaInlineProvider
(transform_factories, no_input_transforms=())[source]¶
-
class
apache_beam.yaml.yaml_provider.
PypiExpansionService
(packages, base_python='/home/runner/work/beam/beam/beam/sdks/python/target/.tox/py38-docs/bin/python')[source]¶ Bases:
object
Expands transforms by fully qualified name in a virtual environment with the given dependencies.
-
VENV_CACHE
= '/home/runner/.apache_beam/cache/venvs'¶
-
-
class
apache_beam.yaml.yaml_provider.
RenamingProvider
(transforms, mappings, underlying_provider)[source]¶ Bases:
apache_beam.yaml.yaml_provider.Provider
-
create_transform
(typ: str, args: Mapping[str, Any], yaml_create_transform: Callable[[Mapping[str, Any], Iterable[apache_beam.pvalue.PCollection]], apache_beam.transforms.ptransform.PTransform]) → apache_beam.transforms.ptransform.PTransform[source]¶ Creates a PTransform instance for the given transform type and arguments.
-