apache_beam.transforms.enrichment_handlers.bigtable module

class apache_beam.transforms.enrichment_handlers.bigtable.BigTableEnrichmentHandler(project_id: str, instance_id: str, table_id: str, row_key: str = '', row_filter: ~google.cloud.bigtable.row_filters.RowFilter | None = <google.cloud.bigtable.row_filters.CellsColumnLimitFilter object>, *, app_profile_id: str | None = None, encoding: str = 'utf-8', row_key_fn: ~typing.Callable[[~apache_beam.pvalue.Row], bytes] | None = None, exception_level: ~apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel = ExceptionLevel.WARN, include_timestamp: bool = False)[source]

Bases: EnrichmentSourceHandler[Row, Row]

A handler for apache_beam.transforms.enrichment.Enrichment transform to interact with GCP BigTable.

Parameters:

  • project_id (str) – GCP project-id of the BigTable cluster.

  • instance_id (str) – GCP instance-id of the BigTable cluster.

  • table_id (str) – GCP table-id of the BigTable.

  • row_key (str) – unique row-key field name from the input beam.Row object to use as row_key for BigTable querying.

  • row_filter – a :class:`google.cloud.bigtable.row_filters.RowFilter` to filter data read with read_row(). Defaults to CellsColumnLimitFilter(1).

  • app_profile_id (str) – App profile ID to use for BigTable. See https://cloud.google.com/bigtable/docs/app-profiles for more details.

  • encoding (str) – encoding type to convert the string to bytes and vice-versa from BigTable. Default is utf-8.

  • row_key_fn – a lambda function that returns a string row key from the input row. It is used to build/extract the row key for Bigtable.

  • exception_level – a enum.Enum value from apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel to set the level when an empty row is returned from the BigTable query. Defaults to ExceptionLevel.WARN.

  • include_timestamp (bool) – If enabled, the timestamp associated with the value is returned as (value, timestamp) for each row_key. Defaults to False - only the latest value without the timestamp is returned.

get_cache_key(request: Row) → str[source]

Returns a string formatted with row key since it is unique to a request made to Bigtable.