apache_beam.runners.interactive.messaging.interactive_environment_inspector module

Messaging mechanism to inspect the interactive environment.

A singleton instance is accessible from interactive_environment.current_env().inspector.

class apache_beam.runners.interactive.messaging.interactive_environment_inspector.InteractiveEnvironmentInspector(ignore_synthetic=True)[source]

Bases: object

Inspector that converts information of the current interactive environment including pipelines and pcollections into JSON data suitable for messaging with applications within/outside the Python kernel.

The usage is always that the application side reads the inspectables or list_inspectables first then communicates back to the kernel and get_val for usage on the kernel side.


Lists pipelines and pcollections assigned to variables as inspectables.


Returns a dictionary of all inspectable pipelines. The keys are stringified id of pipeline instances.

This includes user defined pipeline assigned to variables and anonymous pipelines with inspectable PCollections. If a user defined pipeline is not within the returned dict, it can be considered out of scope, and all resources and memory states related to it should be released.


Retrieves the in memory object itself by identifier.

The retrieved object could be a pipeline or a pcollection. If the identifier is not recognized, return None. The identifier can refer to an anonymous pipeline and the object will still be retrieved.

get_pcoll_data(identifier, include_window_info=False)[source]

Retrieves the json formatted PCollection data.

If no PCollection value can be retieved from the given identifier, an empty json string will be returned.

get_cluster_master_url(identifier: str) → str[source]

Returns the master_url corresponding to the obfuscated identifier.


Inspects current interactive environment to track metadata and values of pipelines and pcollections.

Each pipeline and pcollections tracked is given a unique identifier.


Inspects current interactive environment to track all pipelines assigned to variables. The keys are pipeline objects and values are pipeline names.

apache_beam.runners.interactive.messaging.interactive_environment_inspector.meta(name, val)[source]

Generates meta data for the given name and value.


Synthesizes a pipeline name for the given pipeline object.