apache_beam.transforms.display module

DisplayData, its classes, interfaces and methods.

The classes in this module allow users and transform developers to define static display data to be displayed when a pipeline runs. PTransform s, DoFn s and other pipeline components are subclasses of the HasDisplayData mixin. To add static display data to a component, you can override the HasDisplayData.display_data() method.

Available classes:

• HasDisplayData - Components that inherit from this class can have static display data shown in the UI.
• DisplayDataItem - This class represents static display data elements.
• DisplayData - Internal class that is used to create display data and communicate it to the API.

class apache_beam.transforms.display.HasDisplayData

Bases: object

Basic mixin for elements that contain display data.

It implements only the display_data method and a _namespace method.

display_data()

Returns the display data associated to a pipeline component.

It should be reimplemented in pipeline components that wish to have static display data.

Returns:A dictionary containing key:value pairs. The value might be an integer, float or string value; a DisplayDataItem for values that have more data (e.g. short value, label, url); or a HasDisplayData instance that has more display data that should be picked up. For example:

{
  'key1': 'string_value',
  'key2': 1234,
  'key3': 3.14159265,
  'key4': DisplayDataItem('apache.org', url='http://apache.org'),
  'key5': subComponent
}

Return type:Dict[str, Any]

class apache_beam.transforms.display.DisplayData(namespace, display_data_dict)

Bases: object

Static display data associated with a pipeline component.

classmethod create_from_options(pipeline_options)

Creates DisplayData from a PipelineOptions instance.

When creating DisplayData, this method will convert the value of any item of a non-supported type to its string representation. The normal create_from() method rejects those items.

Returns:A DisplayData instance with populated items. Return type:DisplayData Raises:ValueError – If the has_display_data argument is not an instance of HasDisplayData.

classmethod create_from(has_display_data)

Creates DisplayData from a HasDisplayData instance.

Returns:A DisplayData instance with populated items. Return type:DisplayData Raises:ValueError – If the has_display_data argument is not an instance of HasDisplayData.

class apache_beam.transforms.display.DisplayDataItem(value, url=None, label=None, namespace=None, key=None, shortValue=None)

Bases: object

A DisplayDataItem represents a unit of static display data.

Each item is identified by a key and the namespace of the component the display item belongs to.

typeDict = {<type 'float'>: 'FLOAT', <type 'datetime.timedelta'>: 'DURATION', <type 'str'>: 'STRING', <type 'int'>: 'INTEGER', <type 'bool'>: 'BOOLEAN', <type 'unicode'>: 'STRING', <type 'datetime.datetime'>: 'TIMESTAMP'}

drop_if_none()

The item should be dropped if its value is None.

Returns:Returns self.

drop_if_default(default)

The item should be dropped if its value is equal to its default.

Returns:Returns self.

should_drop()

Return True if the item should be dropped, or False if it should not be dropped. This depends on the drop_if_none, and drop_if_default calls.

Returns:True or False; depending on whether the item should be dropped or kept.

is_valid()

Checks that all the necessary fields of the DisplayDataItem are filled in. It checks that neither key, namespace, value or type are None.

Raises:ValueError – If the item does not have a key, namespace, value or type.

get_dict()

Returns the internal-API dictionary representing the DisplayDataItem.

Returns:A dictionary. The internal-API dictionary representing the DisplayDataItem. Return type:Dict[str, Any] Raises:ValueError – if the item is not valid.