apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc module

Client and server classes corresponding to protobuf-defined services.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnControlStub(channel)

Bases: object

Control Plane API

Progress reporting and splitting still need further vetting. Also, this may change with the addition of new types of instructions/responses related to metrics.

An API that describes the work that a SDK harness is meant to do. Stable

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnControlServicer

Bases: object

Control Plane API

Progress reporting and splitting still need further vetting. Also, this may change with the addition of new types of instructions/responses related to metrics.

An API that describes the work that a SDK harness is meant to do. Stable

Control(request_iterator, context)

Instructions sent by the runner to the SDK requesting different types of work.

Header metadata has the specified keys pairs: - “worker_id”: the id of the sdk

GetProcessBundleDescriptor(request, context)

Used to get the full process bundle descriptors for bundles one is asked to process.

Header metadata has the specified keys pairs: - “worker_id”: the id of the sdk

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnControlServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnControl

Bases: object

Control Plane API

Progress reporting and splitting still need further vetting. Also, this may change with the addition of new types of instructions/responses related to metrics.

An API that describes the work that a SDK harness is meant to do. Stable

static Control(request_iterator, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

static GetProcessBundleDescriptor(request, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnDataStub(channel)

Bases: object

Stable

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnDataServicer

Bases: object

Stable

Data(request_iterator, context)

Used to send data between harnesses. Sdks default to using an unnamed data stream (without “data_stream_id” header value) for bundles unless the runner requests another named stream to be used for a bundle. SDKs can advertise that they support named data streams with the capability beam:protocol:named_data_streams:v1.

Header metadata has the specified keys pairs: - “worker_id”: value is the id of the sdk - “data_stream_id”: value is the id of the data stream, distinguishing it from other data streams from the same sdk. This field should only be populated if requested in a received ProcessBundleRequest from the runner.

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnDataServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnData

Bases: object

Stable

static Data(request_iterator, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnStateStub(channel)

Bases: object

An API for the SDK harness to access state stored by the runner on behalf of the SDK. State is scoped to the currently processing bundle and can be used for user state, side inputs, and runner-managed references. Stable

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnStateServicer

Bases: object

An API for the SDK harness to access state stored by the runner on behalf of the SDK. State is scoped to the currently processing bundle and can be used for user state, side inputs, and runner-managed references. Stable

State(request_iterator, context)

Used to get/append/clear state stored by the runner on behalf of the SDK.

Header metadata has the specified keys pairs: - “worker_id”: the id of the sdk

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnStateServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnState

Bases: object

An API for the SDK harness to access state stored by the runner on behalf of the SDK. State is scoped to the currently processing bundle and can be used for user state, side inputs, and runner-managed references. Stable

static State(request_iterator, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnLoggingStub(channel)

Bases: object

Stable

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnLoggingServicer

Bases: object

Stable

Logging(request_iterator, context)

Allows for the SDK to emit log entries which the runner can associate with the active job.

Header metadata has the specified keys pairs: - “worker_id”: the id of the sdk

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnLoggingServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnLogging

Bases: object

Stable

static Logging(request_iterator, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnExternalWorkerPoolStub(channel)

Bases: object

An API for the runner to manage external SDK worker processes. Allows the runner to start and stop SDK worker instances, typically used for containerized or process-based execution environments.

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnExternalWorkerPoolServicer

Bases: object

An API for the runner to manage external SDK worker processes. Allows the runner to start and stop SDK worker instances, typically used for containerized or process-based execution environments.

StartWorker(request, context)

Start the SDK worker with the given ID.

StopWorker(request, context)

Stop the SDK worker.

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnExternalWorkerPoolServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnExternalWorkerPool

Bases: object

An API for the runner to manage external SDK worker processes. Allows the runner to start and stop SDK worker instances, typically used for containerized or process-based execution environments.

static StartWorker(request, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

static StopWorker(request, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnWorkerStatusStub(channel)

Bases: object

API for SDKs to report debug-related statuses to runner during pipeline execution.

Constructor.

Parameters:

channel – A grpc.Channel.

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnWorkerStatusServicer

Bases: object

API for SDKs to report debug-related statuses to runner during pipeline execution.

WorkerStatus(request_iterator, context)

Header metadata has the specified keys pairs: - “worker_id”: the id of the sdk

apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.add_BeamFnWorkerStatusServicer_to_server(servicer, server)

class apache_beam.portability.api.org.apache.beam.model.fn_execution.v1.beam_fn_api_pb2_grpc.BeamFnWorkerStatus

Bases: object

API for SDKs to report debug-related statuses to runner during pipeline execution.

static WorkerStatus(request_iterator, target, options=(), channel_credentials=None, call_credentials=None, insecure=False, compression=None, wait_for_ready=None, timeout=None, metadata=None)