apache_beam.transforms.display module

DisplayData, its classes, interfaces and methods.

The classes in this module allow users and transform developers to define static display data to be displayed when a pipeline runs. PTransform s, DoFn s and other pipeline components are subclasses of the HasDisplayData mixin. To add static display data to a component, you can override the HasDisplayData.display_data() method.

Available classes:

  • HasDisplayData - Components that inherit from this class can have static display data shown in the UI.
  • DisplayDataItem - This class represents static display data elements.
  • DisplayData - Internal class that is used to create display data and communicate it to the API.
class apache_beam.transforms.display.HasDisplayData[source]

Bases: future.types.newobject.newobject

Basic mixin for elements that contain display data.

It implements only the display_data method and a _namespace method.

display_data()[source]

Returns the display data associated to a pipeline component.

It should be reimplemented in pipeline components that wish to have static display data.

Returns:A dictionary containing key:value pairs. The value might be an integer, float or string value; a DisplayDataItem for values that have more data (e.g. short value, label, url); or a HasDisplayData instance that has more display data that should be picked up. For example:
{
  'key1': 'string_value',
  'key2': 1234,
  'key3': 3.14159265,
  'key4': DisplayDataItem('apache.org', url='http://apache.org'),
  'key5': subComponent
}
Return type:Dict[str, Any]
class apache_beam.transforms.display.DisplayData(namespace, display_data_dict)[source]

Bases: future.types.newobject.newobject

Static display data associated with a pipeline component.

classmethod create_from_options(pipeline_options)[source]

Creates DisplayData from a PipelineOptions instance.

When creating DisplayData, this method will convert the value of any item of a non-supported type to its string representation. The normal create_from() method rejects those items.

Returns:A DisplayData instance with populated items.
Return type:DisplayData
Raises:ValueError – If the has_display_data argument is not an instance of HasDisplayData.
classmethod create_from(has_display_data)[source]

Creates DisplayData from a HasDisplayData instance.

Returns:A DisplayData instance with populated items.
Return type:DisplayData
Raises:ValueError – If the has_display_data argument is not an instance of HasDisplayData.
class apache_beam.transforms.display.DisplayDataItem(value, url=None, label=None, namespace=None, key=None, shortValue=None)[source]

Bases: future.types.newobject.newobject

A DisplayDataItem represents a unit of static display data.

Each item is identified by a key and the namespace of the component the display item belongs to.

typeDict = {<type 'int'>: 'INTEGER', <type 'datetime.datetime'>: 'TIMESTAMP', <type 'bool'>: 'BOOLEAN', <type 'datetime.timedelta'>: 'DURATION', <type 'str'>: 'STRING', <type 'float'>: 'FLOAT', <type 'unicode'>: 'STRING'}
drop_if_none()[source]

The item should be dropped if its value is None.

Returns:Returns self.
drop_if_default(default)[source]

The item should be dropped if its value is equal to its default.

Returns:Returns self.
should_drop()[source]

Return True if the item should be dropped, or False if it should not be dropped. This depends on the drop_if_none, and drop_if_default calls.

Returns:True or False; depending on whether the item should be dropped or kept.
is_valid()[source]

Checks that all the necessary fields of the DisplayDataItem are filled in. It checks that neither key, namespace, value or type are None.

Raises:ValueError – If the item does not have a key, namespace, value or type.
get_dict()[source]

Returns the internal-API dictionary representing the DisplayDataItem.

Returns:A dictionary. The internal-API dictionary representing the DisplayDataItem.
Return type:Dict[str, Any]
Raises:ValueError – if the item is not valid.