apache_beam.runners.pipeline_context module

Utility class for serializing pipelines via the runner API.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.runners.pipeline_context.PortableObject(*args, **kwargs)

Bases: Protocol

to_runner_api(_PortableObject__context: PipelineContext) → Any

classmethod from_runner_api(_PortableObject__proto: Any, _PortableObject__context: PipelineContext) → Any

class apache_beam.runners.pipeline_context.PipelineContext(proto: Components | ProcessBundleDescriptor | None = None, component_id_map: ComponentIdMap | None = None, default_environment: Environment | None = None, use_fake_coders: bool = False, iterable_state_read: Callable[[bytes, CoderImpl], Iterable] | None = None, iterable_state_write: Callable[[Iterable, CoderImpl], bytes] | None = None, namespace: str = 'ref', requirements: Iterable[str] = ())

Bases: object

For internal use only; no backwards-compatibility guarantees.

Used for accessing and constructing the referenced objects of a Pipeline.

add_requirement(requirement: str) → None

requirements() → FrozenSet[str]

coder_id_from_element_type(element_type: Any, requires_deterministic_key_coder: str | None = None) → str

deterministic_coder(coder: Coder, msg: str) → Coder

element_type_from_coder_id(coder_id: str) → Any

static from_runner_api(proto: Components) → PipelineContext

to_runner_api() → Components

default_environment_id() → str

get_environment_id_for_resource_hints(hints: Dict[str, bytes]) → str

Returns an environment id that has necessary resource hints.