apache_beam.runners.pipeline_context module¶

Utility class for serializing pipelines via the runner API.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.runners.pipeline_context.PortableObject(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

to_runner_api(_PortableObject__context: apache_beam.runners.pipeline_context.PipelineContext) → Any[source]¶

classmethod from_runner_api(_PortableObject__proto: Any, _PortableObject__context: apache_beam.runners.pipeline_context.PipelineContext) → Any[source]¶

class apache_beam.runners.pipeline_context.PipelineContext(proto: Union[org.apache.beam.model.pipeline.v1.beam_runner_api_pb2.Components, org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2.ProcessBundleDescriptor, None] = None, component_id_map: Optional[apache_beam.pipeline.ComponentIdMap] = None, default_environment: Optional[apache_beam.transforms.environments.Environment] = None, use_fake_coders: bool = False, iterable_state_read: Optional[Callable[[bytes, CoderImpl], Iterable[T_co]]] = None, iterable_state_write: Optional[Callable[[Iterable[T_co], CoderImpl], bytes]] = None, namespace: str = 'ref', requirements: Iterable[str] = ())[source]¶

Bases: object

For internal use only; no backwards-compatibility guarantees.

Used for accessing and constructing the referenced objects of a Pipeline.

add_requirement(requirement: str) → None[source]¶

requirements() → FrozenSet[str][source]¶

coder_id_from_element_type(element_type: Any, requires_deterministic_key_coder: Optional[str] = None) → str[source]¶

deterministic_coder(coder: apache_beam.coders.coders.Coder, msg: str) → apache_beam.coders.coders.Coder[source]¶

element_type_from_coder_id(coder_id: str) → Any[source]¶

static from_runner_api(proto: org.apache.beam.model.pipeline.v1.beam_runner_api_pb2.Components) → apache_beam.runners.pipeline_context.PipelineContext[source]¶

to_runner_api() → org.apache.beam.model.pipeline.v1.beam_runner_api_pb2.Components[source]¶

default_environment_id() → str[source]¶

get_environment_id_for_resource_hints(hints: Dict[str, bytes]) → str[source]¶: Returns an environment id that has necessary resource hints.