apache_beam.runners.interactive.messaging.interactive_environment_inspector module
Messaging mechanism to inspect the interactive environment.
A singleton instance is accessible from interactive_environment.current_env().inspector.
- class apache_beam.runners.interactive.messaging.interactive_environment_inspector.InteractiveEnvironmentInspector(ignore_synthetic=True)[source]
Bases:
object
Inspector that converts information of the current interactive environment including pipelines and pcollections into JSON data suitable for messaging with applications within/outside the Python kernel.
The usage is always that the application side reads the inspectables or list_inspectables first then communicates back to the kernel and get_val for usage on the kernel side.
- property inspectables
Lists pipelines and pcollections assigned to variables as inspectables.
- property inspectable_pipelines
Returns a dictionary of all inspectable pipelines. The keys are stringified id of pipeline instances.
This includes user defined pipeline assigned to variables and anonymous pipelines with inspectable PCollections. If a user defined pipeline is not within the returned dict, it can be considered out of scope, and all resources and memory states related to it should be released.
- list_inspectables(**kwargs)
- get_val(identifier)[source]
Retrieves the in memory object itself by identifier.
The retrieved object could be a pipeline or a pcollection. If the identifier is not recognized, return None. The identifier can refer to an anonymous pipeline and the object will still be retrieved.
- get_pcoll_data(identifier, include_window_info=False)[source]
Retrieves the json formatted PCollection data.
If no PCollection value can be retieved from the given identifier, an empty json string will be returned.
- list_clusters(**kwargs)
- apache_beam.runners.interactive.messaging.interactive_environment_inspector.inspect(ignore_synthetic=True)[source]
Inspects current interactive environment to track metadata and values of pipelines and pcollections.
Each pipeline and pcollections tracked is given a unique identifier.
- apache_beam.runners.interactive.messaging.interactive_environment_inspector.inspect_pipelines()[source]
Inspects current interactive environment to track all pipelines assigned to variables. The keys are pipeline objects and values are pipeline names.