apache_beam.ml.rag.ingestion package
Vector storage ingestion components for RAG pipelines. This module provides components for storing vectors in RAG pipelines.
Submodules
- apache_beam.ml.rag.ingestion.alloydb module
AlloyDBLanguageConnectorConfigAlloyDBLanguageConnectorConfig.usernameAlloyDBLanguageConnectorConfig.passwordAlloyDBLanguageConnectorConfig.database_nameAlloyDBLanguageConnectorConfig.instance_nameAlloyDBLanguageConnectorConfig.ip_typeAlloyDBLanguageConnectorConfig.enable_iam_authAlloyDBLanguageConnectorConfig.target_principalAlloyDBLanguageConnectorConfig.delegatesAlloyDBLanguageConnectorConfig.admin_service_endpointAlloyDBLanguageConnectorConfig.quota_projectAlloyDBLanguageConnectorConfig.connection_propertiesAlloyDBLanguageConnectorConfig.additional_propertiesAlloyDBLanguageConnectorConfig.usernameAlloyDBLanguageConnectorConfig.passwordAlloyDBLanguageConnectorConfig.database_nameAlloyDBLanguageConnectorConfig.instance_nameAlloyDBLanguageConnectorConfig.ip_typeAlloyDBLanguageConnectorConfig.enable_iam_authAlloyDBLanguageConnectorConfig.target_principalAlloyDBLanguageConnectorConfig.delegatesAlloyDBLanguageConnectorConfig.admin_service_endpointAlloyDBLanguageConnectorConfig.quota_projectAlloyDBLanguageConnectorConfig.connection_propertiesAlloyDBLanguageConnectorConfig.additional_propertiesAlloyDBLanguageConnectorConfig.to_jdbc_url()AlloyDBLanguageConnectorConfig.to_connection_config()AlloyDBLanguageConnectorConfig.additional_jdbc_args()
AlloyDBVectorWriterConfig
- apache_beam.ml.rag.ingestion.base module
- apache_beam.ml.rag.ingestion.bigquery module
- apache_beam.ml.rag.ingestion.cloudsql module
LanguageConnectorConfigLanguageConnectorConfig.usernameLanguageConnectorConfig.passwordLanguageConnectorConfig.database_nameLanguageConnectorConfig.instance_nameLanguageConnectorConfig.ip_typeLanguageConnectorConfig.enable_iam_authLanguageConnectorConfig.target_principalLanguageConnectorConfig.delegatesLanguageConnectorConfig.admin_service_endpointLanguageConnectorConfig.quota_projectLanguageConnectorConfig.connection_propertiesLanguageConnectorConfig.additional_propertiesLanguageConnectorConfig.usernameLanguageConnectorConfig.passwordLanguageConnectorConfig.database_nameLanguageConnectorConfig.instance_nameLanguageConnectorConfig.ip_typesLanguageConnectorConfig.enable_iam_authLanguageConnectorConfig.target_principalLanguageConnectorConfig.delegatesLanguageConnectorConfig.quota_projectLanguageConnectorConfig.connection_propertiesLanguageConnectorConfig.additional_propertiesLanguageConnectorConfig.to_connection_config()LanguageConnectorConfig.additional_jdbc_args()
CloudSQLPostgresVectorWriterConfigCloudSQLMySQLVectorWriterConfig
- apache_beam.ml.rag.ingestion.jdbc_common module
ConnectionConfigConnectionConfig.jdbc_urlConnectionConfig.usernameConnectionConfig.passwordConnectionConfig.connection_propertiesConnectionConfig.connection_init_sqlsConnectionConfig.additional_jdbc_argsConnectionConfig.jdbc_urlConnectionConfig.usernameConnectionConfig.passwordConnectionConfig.connection_propertiesConnectionConfig.connection_init_sqlsConnectionConfig.additional_jdbc_args
WriteConfig
- apache_beam.ml.rag.ingestion.milvus_search module
- apache_beam.ml.rag.ingestion.mysql module
- apache_beam.ml.rag.ingestion.mysql_common module
- apache_beam.ml.rag.ingestion.postgres module
- apache_beam.ml.rag.ingestion.postgres_common module
chunk_embedding_fn()ColumnSpecColumnSpecsBuilderColumnSpecsBuilder.with_defaults()ColumnSpecsBuilder.with_id_spec()ColumnSpecsBuilder.with_content_spec()ColumnSpecsBuilder.with_metadata_spec()ColumnSpecsBuilder.with_embedding_spec()ColumnSpecsBuilder.with_sparse_embedding_spec()ColumnSpecsBuilder.add_metadata_field()ColumnSpecsBuilder.add_custom_column_spec()ColumnSpecsBuilder.build()
ConflictResolution
- apache_beam.ml.rag.ingestion.qdrant module
QdrantConnectionParametersQdrantConnectionParameters.locationQdrantConnectionParameters.urlQdrantConnectionParameters.portQdrantConnectionParameters.grpc_portQdrantConnectionParameters.prefer_grpcQdrantConnectionParameters.httpsQdrantConnectionParameters.api_keyQdrantConnectionParameters.prefixQdrantConnectionParameters.timeoutQdrantConnectionParameters.hostQdrantConnectionParameters.pathQdrantConnectionParameters.kwargsQdrantConnectionParameters.for_cloud()QdrantConnectionParameters.for_host()QdrantConnectionParameters.for_url()QdrantConnectionParameters.local()QdrantConnectionParameters.in_memory()
QdrantWriteConfigQdrantWriteConfig.connection_paramsQdrantWriteConfig.collection_nameQdrantWriteConfig.timeoutQdrantWriteConfig.batch_sizeQdrantWriteConfig.max_batch_byte_sizeQdrantWriteConfig.kwargsQdrantWriteConfig.dense_embedding_keyQdrantWriteConfig.sparse_embedding_keyQdrantWriteConfig.create_write_transform()QdrantWriteConfig.create_converter()
- apache_beam.ml.rag.ingestion.spanner module
SpannerColumnSpecSpannerColumnSpecsBuilderSpannerColumnSpecsBuilder.with_defaults()SpannerColumnSpecsBuilder.with_id_spec()SpannerColumnSpecsBuilder.with_embedding_spec()SpannerColumnSpecsBuilder.with_content_spec()SpannerColumnSpecsBuilder.with_metadata_spec()SpannerColumnSpecsBuilder.add_metadata_field()SpannerColumnSpecsBuilder.add_column()SpannerColumnSpecsBuilder.build()
SpannerVectorWriterConfig
- apache_beam.ml.rag.ingestion.test_utils module