apache_beam.transforms.display module

DisplayData, its classes, interfaces and methods.

The classes in this module allow users and transform developers to define static display data to be displayed when a pipeline runs. PTransform s, DoFn s and other pipeline components are subclasses of the HasDisplayData mixin. To add static display data to a component, you can override the HasDisplayData.display_data() method.

Available classes:

• HasDisplayData - Components that inherit from this class can have static display data shown in the UI.

• DisplayDataItem - This class represents static display data elements.

• DisplayData - Internal class that is used to create display data and communicate it to the API.

class apache_beam.transforms.display.HasDisplayData

Bases: object

Basic mixin for elements that contain display data.

It implements only the display_data method and a _get_display_data_namespace method.

display_data() → dict

Returns the display data associated to a pipeline component.

It should be reimplemented in pipeline components that wish to have static display data.

Returns:

A dictionary containing key:value pairs. The value might be an integer, float or string value; a DisplayDataItem for values that have more data (e.g. short value, label, url); or a HasDisplayData instance that has more display data that should be picked up. For example:

{
  'key1': 'string_value',
  'key2': 1234,
  'key3': 3.14159265,
  'key4': DisplayDataItem('apache.org', url='http://apache.org'),
  'key5': subComponent
}

Return type:

Dict[str, Any]

class apache_beam.transforms.display.DisplayDataItem(value, url=None, label=None, namespace=None, key=None, shortValue=None)

Bases: object

A DisplayDataItem represents a unit of static display data.

Each item is identified by a key and the namespace of the component the display item belongs to.

typeDict = {<class 'bool'>: 'BOOLEAN', <class 'datetime.datetime'>: 'TIMESTAMP', <class 'datetime.timedelta'>: 'DURATION', <class 'float'>: 'FLOAT', <class 'int'>: 'INTEGER', <class 'str'>: 'STRING'}

drop_if_none() → DisplayDataItem

The item should be dropped if its value is None.

Returns:

Returns self.

drop_if_default(default) → DisplayDataItem

The item should be dropped if its value is equal to its default.

Returns:

Returns self.

should_drop() → bool

Return True if the item should be dropped, or False if it should not be dropped. This depends on the drop_if_none, and drop_if_default calls.

Returns:

True or False; depending on whether the item should be dropped or kept.

is_valid() → None

Checks that all the necessary fields of the DisplayDataItem are filled in. It checks that neither key, namespace, value or type are None.

Raises:

ValueError – If the item does not have a key, namespace, value or type.

get_dict() → dict

Returns the internal-API dictionary representing the DisplayDataItem.

Returns:

A dictionary. The internal-API dictionary representing the DisplayDataItem.

Return type:

Dict[str, Any]

Raises:

ValueError – if the item is not valid.

class apache_beam.transforms.display.DisplayData(namespace: str, display_data_dict: dict)

Bases: object

Static display data associated with a pipeline component.

to_proto() → List[DisplayData]

Returns a List of Beam proto representation of Display data.

classmethod create_from_options(pipeline_options)

Creates DisplayData from a PipelineOptions instance.

When creating DisplayData, this method will convert the value of any item of a non-supported type to its string representation. The normal create_from() method rejects those items.

Returns:

A DisplayData instance with populated items.

Return type:

DisplayData

Raises:

ValueError – If the has_display_data argument is not an instance of HasDisplayData.

classmethod create_from(has_display_data)

Creates DisplayData from a HasDisplayData instance.

Returns:

A DisplayData instance with populated items.

Return type:

DisplayData

Raises:

ValueError – If the has_display_data argument is not an instance of HasDisplayData.