apache_beam.ml.rag.ingestion package

Vector storage ingestion components for RAG pipelines. This module provides components for storing vectors in RAG pipelines.

Submodules

• apache_beam.ml.rag.ingestion.alloydb module
  • AlloyDBLanguageConnectorConfig
    • AlloyDBLanguageConnectorConfig.database_name
    • AlloyDBLanguageConnectorConfig.instance_name
    • AlloyDBLanguageConnectorConfig.ip_type
    • AlloyDBLanguageConnectorConfig.enable_iam_auth
    • AlloyDBLanguageConnectorConfig.target_principal
    • AlloyDBLanguageConnectorConfig.delegates
    • AlloyDBLanguageConnectorConfig.admin_service_endpoint
    • AlloyDBLanguageConnectorConfig.quota_project
    • AlloyDBLanguageConnectorConfig.database_name
    • AlloyDBLanguageConnectorConfig.instance_name
    • AlloyDBLanguageConnectorConfig.ip_type
    • AlloyDBLanguageConnectorConfig.enable_iam_auth
    • AlloyDBLanguageConnectorConfig.target_principal
    • AlloyDBLanguageConnectorConfig.delegates
    • AlloyDBLanguageConnectorConfig.admin_service_endpoint
    • AlloyDBLanguageConnectorConfig.quota_project
    • AlloyDBLanguageConnectorConfig.to_jdbc_url()
  • AlloyDBConnectionConfig
    • AlloyDBConnectionConfig.jdbc_url
    • AlloyDBConnectionConfig.username
    • AlloyDBConnectionConfig.password
    • AlloyDBConnectionConfig.connection_properties
    • AlloyDBConnectionConfig.connection_init_sqls
    • AlloyDBConnectionConfig.autosharding
    • AlloyDBConnectionConfig.max_connections
    • AlloyDBConnectionConfig.write_batch_size
    • AlloyDBConnectionConfig.additional_jdbc_args
    • AlloyDBConnectionConfig.jdbc_url
    • AlloyDBConnectionConfig.username
    • AlloyDBConnectionConfig.password
    • AlloyDBConnectionConfig.connection_properties
    • AlloyDBConnectionConfig.connection_init_sqls
    • AlloyDBConnectionConfig.autosharding
    • AlloyDBConnectionConfig.max_connections
    • AlloyDBConnectionConfig.write_batch_size
    • AlloyDBConnectionConfig.additional_jdbc_args
    • AlloyDBConnectionConfig.with_language_connector()
  • ConflictResolution
    • ConflictResolution.on_conflict_fields
    • ConflictResolution.action
    • ConflictResolution.update_fields
    • ConflictResolution.on_conflict_fields
    • ConflictResolution.action
    • ConflictResolution.update_fields
    • ConflictResolution.maybe_set_default_update_fields()
    • ConflictResolution.get_conflict_clause()
  • chunk_embedding_fn()
  • chunk_content_fn()
  • chunk_metadata_fn()
  • ColumnSpec
    • ColumnSpec.column_name
    • ColumnSpec.python_type
    • ColumnSpec.value_fn
    • ColumnSpec.sql_typecast
    • ColumnSpec.column_name
    • ColumnSpec.python_type
    • ColumnSpec.value_fn
    • ColumnSpec.sql_typecast
    • ColumnSpec.placeholder
    • ColumnSpec.text()
    • ColumnSpec.integer()
    • ColumnSpec.float()
    • ColumnSpec.vector()
    • ColumnSpec.jsonb()
  • chunk_id_fn()
  • ColumnSpecsBuilder
    • ColumnSpecsBuilder.with_defaults()
    • ColumnSpecsBuilder.with_id_spec()
    • ColumnSpecsBuilder.with_content_spec()
    • ColumnSpecsBuilder.with_metadata_spec()
    • ColumnSpecsBuilder.with_embedding_spec()
    • ColumnSpecsBuilder.add_metadata_field()
    • ColumnSpecsBuilder.add_custom_column_spec()
    • ColumnSpecsBuilder.build()
  • AlloyDBVectorWriterConfig
    • AlloyDBVectorWriterConfig.create_write_transform()
• apache_beam.ml.rag.ingestion.base module
  • VectorDatabaseWriteConfig
    • VectorDatabaseWriteConfig.create_write_transform()
  • VectorDatabaseWriteTransform
    • VectorDatabaseWriteTransform.expand()
• apache_beam.ml.rag.ingestion.bigquery module
  • SchemaConfig
    • SchemaConfig.schema
    • SchemaConfig.chunk_to_dict_fn
    • SchemaConfig.schema
    • SchemaConfig.chunk_to_dict_fn
  • BigQueryVectorWriterConfig
    • BigQueryVectorWriterConfig.create_write_transform()