apache_beam.runners.interactive.display_manager module

Manages displaying pipeline graph and execution status on the frontend.

This module is experimental. No backwards-compatibility guarantees.

class apache_beam.runners.interactive.display_manager.DisplayManager(pipeline_info, pipeline_proto, caches_used, cache_manager, referenced_pcollections, required_transforms)

Bases: object

Manages displaying pipeline graph and execution status on the frontend.

Constructor of DisplayManager.

Parameters:

• pipeline_info – (interactive_runner.PipelineInfo)
• pipeline_proto – (Pipeline proto)
• caches_used – (set of str) A set of PCollection IDs of those whose cached results are used in the execution.
• cache_manager – (interactive_runner.CacheManager) DisplayManager fetches the latest status of pipeline execution by querying cache_manager.
• referenced_pcollections – (dict from str to PCollection proto) PCollection ID mapped to PCollection referenced during pipeline execution.
• required_transforms – (dict from str to PTransform proto) Mapping from transform ID to transforms that leads to visible results.

start_periodic_update()

Start a thread that periodically updates the display.

stop_periodic_update()

Stop periodically updating the display.

update_display(force=False)

Updates display on the frontend.

Retrieves the latest execution status by querying CacheManager and updates display on the fronend. The assumption is that there is only one pipeline in a cell, because it clears up everything in the cell output every update cycle.

Parameters:force – (bool) whether to force updating when no stats change happens.