apache_beam.transforms.enrichment_handlers.bigtable module

class apache_beam.transforms.enrichment_handlers.bigtable.BigTableEnrichmentHandler(project_id: str, instance_id: str, table_id: str, row_key: str = '', row_filter: Optional[google.cloud.bigtable.row_filters.RowFilter] = <google.cloud.bigtable.row_filters.CellsColumnLimitFilter object>, *, app_profile_id: str = None, encoding: str = 'utf-8', row_key_fn: Optional[Callable[[apache_beam.pvalue.Row], bytes]] = None, exception_level: apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel = <ExceptionLevel.WARN: 1>, include_timestamp: bool = False)[source]

Bases: apache_beam.transforms.enrichment.EnrichmentSourceHandler

A handler for apache_beam.transforms.enrichment.Enrichment transform to interact with GCP BigTable.

Parameters:
  • project_id (str) – GCP project-id of the BigTable cluster.
  • instance_id (str) – GCP instance-id of the BigTable cluster.
  • table_id (str) – GCP table-id of the BigTable.
  • row_key (str) – unique row-key field name from the input beam.Row object to use as row_key for BigTable querying.
  • row_filter – a :class:`google.cloud.bigtable.row_filters.RowFilter` to filter data read with read_row(). Defaults to CellsColumnLimitFilter(1).
  • app_profile_id (str) – App profile ID to use for BigTable. See https://cloud.google.com/bigtable/docs/app-profiles for more details.
  • encoding (str) – encoding type to convert the string to bytes and vice-versa from BigTable. Default is utf-8.
  • row_key_fn – a lambda function that returns a string row key from the input row. It is used to build/extract the row key for Bigtable.
  • exception_level – a enum.Enum value from apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel to set the level when an empty row is returned from the BigTable query. Defaults to ExceptionLevel.WARN.
  • include_timestamp (bool) – If enabled, the timestamp associated with the value is returned as (value, timestamp) for each row_key. Defaults to False - only the latest value without the timestamp is returned.
get_cache_key(request: apache_beam.pvalue.Row) → str[source]

Returns a string formatted with row key since it is unique to a request made to Bigtable.