apache_beam.yaml.yaml_enrichment module

apache_beam.yaml.yaml_enrichment.enrichment_transform(enrichment_handler: str, handler_config: dict[str, Any], timeout: float | None = 30)[source]

The Enrichment transform allows one to dynamically enhance elements in a pipeline by performing key-value lookups against external services like APIs or databases.

Example using BigTable:

- type: Enrichment
  config:
    enrichment_handler: 'BigTable'
    handler_config:
      project_id: 'apache-beam-testing'
      instance_id: 'beam-test'
      table_id: 'bigtable-enrichment-test'
      row_key: 'product_id'
    timeout: 30

For more information on Enrichment, see the [Beam docs]( https://beam.apache.org/documentation/transforms/python/elementwise/enrichment/).

Parameters:

enrichment_handler (str) – Specifies the source from where data needs to be extracted into the pipeline for enriching data. One of “BigQuery”, “BigTable”, “FeastFeatureStore” or “VertexAIFeatureStore”.
handler_config (str) –
Specifies the parameters for the respective enrichment_handler in a YAML/JSON format. To see the full set of handler_config parameters, see their corresponding doc pages:
- [BigQueryEnrichmentHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.enrichment_handlers.bigquery.html#apache_beam.transforms.enrichment_handlers.bigquery.BigQueryEnrichmentHandler)
- [BigTableEnrichmentHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.enrichment_handlers.bigtable.html#apache_beam.transforms.enrichment_handlers.bigtable.BigTableEnrichmentHandler)
- [FeastFeatureStoreEnrichmentHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.enrichment_handlers.feast_feature_store.html#apache_beam.transforms.enrichment_handlers.feast_feature_store.FeastFeatureStoreEnrichmentHandler)
- [VertexAIFeatureStoreEnrichmentHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.enrichment_handlers.vertex_ai_feature_store.html#apache_beam.transforms.enrichment_handlers.vertex_ai_feature_store.VertexAIFeatureStoreEnrichmentHandler)
timeout (float) – Timeout for source requests in seconds. Defaults to 30 seconds.