apache_beam.runners.interactive.utils module

Utilities to be used in Interactive Beam.

apache_beam.runners.interactive.utils.to_element_list(reader, coder, include_window_info, n=None, include_time_events=False)

Returns an iterator that properly decodes the elements from the reader.

apache_beam.runners.interactive.utils.elements_to_df(elements, include_window_info=False, element_type=None)

Parses the given elements into a Dataframe.

If the elements are a list of WindowedValues, then it will break out the elements into their own DataFrame and return it. If include_window_info is True, then it will concatenate the windowing information onto the elements DataFrame.

apache_beam.runners.interactive.utils.register_ipython_log_handler()

Adds the IPython handler to a dummy parent logger (named ‘apache_beam.runners.interactive’) of all interactive modules’ loggers so that if is_in_notebook, logging displays the logs as HTML in frontends.

class apache_beam.runners.interactive.utils.IPythonLogHandler(level=0)

Bases: logging.Handler

A logging handler to display logs as HTML in IPython backed frontends.

Initializes the instance - basically setting the formatter to None and the filter list to empty.

log_template = '\n <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">\n <div class="alert alert-{level}">{msg}</div>'

logging_to_alert_level_map = {0: 'light', 10: 'dark', 20: 'info', 30: 'warning', 40: 'danger', 50: 'danger'}

emit(record)

apache_beam.runners.interactive.utils.obfuscate(*inputs)

Obfuscates any inputs into a hexadecimal string.

class apache_beam.runners.interactive.utils.ProgressIndicator(enter_text, exit_text)

Bases: object

An indicator visualizing code execution in progress.

spinner_template = '\n <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">\n <div id="{id}" class="spinner-border text-info" role="status">\n </div>'

spinner_removal_template = '\n $("#{id}").remove();'

apache_beam.runners.interactive.utils.progress_indicated(func)

A decorator using a unique progress indicator as a context manager to execute the given function within.

apache_beam.runners.interactive.utils.as_json(func)

A decorator convert python objects returned by callables to json string.

The decorated function should always return an object parsable by json.dumps. If the object is not parsable, the str() of original object is returned instead.

apache_beam.runners.interactive.utils.deferred_df_to_pcollection(df)