apache_beam.transforms.enrichment_handlers.bigtable module¶
-
class
apache_beam.transforms.enrichment_handlers.bigtable.
BigTableEnrichmentHandler
(project_id: str, instance_id: str, table_id: str, row_key: str = '', row_filter: Optional[google.cloud.bigtable.row_filters.RowFilter] = <google.cloud.bigtable.row_filters.CellsColumnLimitFilter object>, *, app_profile_id: str = None, encoding: str = 'utf-8', row_key_fn: Optional[Callable[[apache_beam.pvalue.Row], bytes]] = None, exception_level: apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel = <ExceptionLevel.WARN: 1>, include_timestamp: bool = False)[source]¶ Bases:
apache_beam.transforms.enrichment.EnrichmentSourceHandler
A handler for
apache_beam.transforms.enrichment.Enrichment
transform to interact with GCP BigTable.Parameters: - project_id (str) – GCP project-id of the BigTable cluster.
- instance_id (str) – GCP instance-id of the BigTable cluster.
- table_id (str) – GCP table-id of the BigTable.
- row_key (str) – unique row-key field name from the input beam.Row object to use as row_key for BigTable querying.
- row_filter – a
:class:`google.cloud.bigtable.row_filters.RowFilter`
to filter data read withread_row()
. Defaults to CellsColumnLimitFilter(1). - app_profile_id (str) – App profile ID to use for BigTable. See https://cloud.google.com/bigtable/docs/app-profiles for more details.
- encoding (str) – encoding type to convert the string to bytes and vice-versa from BigTable. Default is utf-8.
- row_key_fn – a lambda function that returns a string row key from the input row. It is used to build/extract the row key for Bigtable.
- exception_level – a enum.Enum value from
apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel
to set the level when an empty row is returned from the BigTable query. Defaults toExceptionLevel.WARN
. - include_timestamp (bool) – If enabled, the timestamp associated with the value is returned as (value, timestamp) for each row_key. Defaults to False - only the latest value without the timestamp is returned.