apache_beam.transforms.enrichment_handlers.bigtable module

class apache_beam.transforms.enrichment_handlers.bigtable.ExceptionLevel[source]

Bases: enum.Enum

ExceptionLevel defines the exception level options to either log a warning, or raise an exception, or do nothing when a BigTable query returns an empty row.

Members:
  • RAISE: Raise the exception.
  • WARN: Log a warning for exception without raising it.
  • QUIET: Neither log nor raise the exception.
RAISE = 0
WARN = 1
QUIET = 2
class apache_beam.transforms.enrichment_handlers.bigtable.BigTableEnrichmentHandler(project_id: str, instance_id: str, table_id: str, row_key: str, row_filter: Optional[google.cloud.bigtable.row_filters.RowFilter] = <google.cloud.bigtable.row_filters.CellsColumnLimitFilter object>, *, app_profile_id: str = None, encoding: str = 'utf-8', exception_level: apache_beam.transforms.enrichment_handlers.bigtable.ExceptionLevel = <ExceptionLevel.WARN: 1>, include_timestamp: bool = False)[source]

Bases: apache_beam.transforms.enrichment.EnrichmentSourceHandler

BigTableEnrichmentHandler is a handler for apache_beam.transforms.enrichment.Enrichment transform to interact with GCP BigTable.

Parameters:
  • project_id (str) – GCP project-id of the BigTable cluster.
  • instance_id (str) – GCP instance-id of the BigTable cluster.
  • table_id (str) – GCP table-id of the BigTable.
  • row_key (str) – unique row-key field name from the input beam.Row object to use as row_key for BigTable querying.
  • row_filter – a :class:`google.cloud.bigtable.row_filters.RowFilter` to filter data read with read_row(). Defaults to CellsColumnLimitFilter(1).
  • app_profile_id (str) – App profile ID to use for BigTable. See https://cloud.google.com/bigtable/docs/app-profiles for more details.
  • encoding (str) – encoding type to convert the string to bytes and vice-versa from BigTable. Default is utf-8.
  • exception_level – a enum.Enum value from apache_beam.transforms.enrichment_handlers.bigtable.ExceptionLevel to set the level when an empty row is returned from the BigTable query. Defaults to ExceptionLevel.WARN.
  • include_timestamp (bool) – If enabled, the timestamp associated with the value is returned as (value, timestamp) for each row_key. Defaults to False - only the latest value without the timestamp is returned.