apache_beam.yaml.yaml_provider module

This module defines Providers usable from yaml, which is a specification for where to find and how to invoke services that vend implementations of various PTransforms.

class apache_beam.yaml.yaml_provider.Provider[source]

Bases: object

Maps transform types names and args to concrete PTransform instances.

available() → bool[source]

Returns whether this provider is available to use in this environment.

provided_transforms() → Iterable[str][source]

Returns a list of transform type names this provider can handle.

create_transform(typ: str, args: Mapping[str, Any]) → apache_beam.transforms.ptransform.PTransform[source]

Creates a PTransform instance for the given transform type and arguments.

class apache_beam.yaml.yaml_provider.ExternalProvider(urns, service)[source]

Bases: apache_beam.yaml.yaml_provider.Provider

A Provider implemented via the cross language transform service.

provided_transforms()[source]
create_transform(type, args)[source]
create_external_transform(urn, args)[source]
static provider_from_spec(spec)[source]
class apache_beam.yaml.yaml_provider.RemoteProvider(urns, service)[source]

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()[source]
class apache_beam.yaml.yaml_provider.ExternalJavaProvider(urns, jar_provider)[source]

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()[source]
class apache_beam.yaml.yaml_provider.ExternalPythonProvider(urns, packages)[source]

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()[source]
create_external_transform(urn, args)[source]
apache_beam.yaml.yaml_provider.fix_pycallable()[source]
class apache_beam.yaml.yaml_provider.InlineProvider(transform_factories)[source]

Bases: apache_beam.yaml.yaml_provider.Provider

available()[source]
provided_transforms()[source]
create_transform(type, args)[source]
to_json()[source]
apache_beam.yaml.yaml_provider.create_builtin_provider()[source]
class apache_beam.yaml.yaml_provider.PypiExpansionService(packages, base_python='/usr/local/google/home/dannymccormick/website_update_dir/python_doc/beam/sdks/python/target/.tox/py38-docs/bin/python')[source]

Bases: object

Expands transforms by fully qualified name in a virtual environment with the given dependencies.

VENV_CACHE = '/usr/local/google/home/dannymccormick/.apache_beam/cache/venvs'
apache_beam.yaml.yaml_provider.parse_providers(provider_specs)[source]
apache_beam.yaml.yaml_provider.merge_providers(*provider_sets)[source]
apache_beam.yaml.yaml_provider.standard_providers()[source]