apache_beam.yaml.yaml_provider module

This module defines Providers usable from yaml, which is a specification for where to find and how to invoke services that vend implementations of various PTransforms.

class apache_beam.yaml.yaml_provider.Provider

Bases: object

Maps transform types names and args to concrete PTransform instances.

available() → bool

Returns whether this provider is available to use in this environment.

provided_transforms() → Iterable[str]

Returns a list of transform type names this provider can handle.

create_transform(typ: str, args: Mapping[str, Any]) → apache_beam.transforms.ptransform.PTransform

Creates a PTransform instance for the given transform type and arguments.

class apache_beam.yaml.yaml_provider.ExternalProvider(urns, service)

Bases: apache_beam.yaml.yaml_provider.Provider

A Provider implemented via the cross language transform service.

provided_transforms()

create_transform(type, args)

create_external_transform(urn, args)

static provider_from_spec(spec)

class apache_beam.yaml.yaml_provider.RemoteProvider(urns, service)

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()

class apache_beam.yaml.yaml_provider.ExternalJavaProvider(urns, jar_provider)

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()

class apache_beam.yaml.yaml_provider.ExternalPythonProvider(urns, packages)

Bases: apache_beam.yaml.yaml_provider.ExternalProvider

available()

create_external_transform(urn, args)

apache_beam.yaml.yaml_provider.fix_pycallable()

class apache_beam.yaml.yaml_provider.InlineProvider(transform_factories)

Bases: apache_beam.yaml.yaml_provider.Provider

available()

provided_transforms()

create_transform(type, args)

to_json()

apache_beam.yaml.yaml_provider.create_builtin_provider()

class apache_beam.yaml.yaml_provider.PypiExpansionService(packages, base_python='/usr/local/google/home/jrmccluskey/website_update_dir/python_doc/beam/sdks/python/target/.tox/py38-docs/bin/python')

Bases: object

Expands transforms by fully qualified name in a virtual environment with the given dependencies.

VENV_CACHE = '/usr/local/google/home/jrmccluskey/.apache_beam/cache/venvs'

apache_beam.yaml.yaml_provider.parse_providers(provider_specs)

apache_beam.yaml.yaml_provider.merge_providers(*provider_sets)

apache_beam.yaml.yaml_provider.standard_providers()