apache_beam.ml.rag package

Apache Beam RAG (Retrieval Augmented Generation) components. This package provides components for building RAG pipelines in Apache Beam, including: - Chunking - Embedding generation - Vector storage - Vector search enrichment

Subpackages

• apache_beam.ml.rag.chunking package
  • Submodules
    • apache_beam.ml.rag.chunking.base module
      • ChunkingTransformProvider
    • apache_beam.ml.rag.chunking.langchain module
      • LangChainChunker
• apache_beam.ml.rag.embeddings package
  • Submodules
    • apache_beam.ml.rag.embeddings.base module
      • create_rag_adapter()
    • apache_beam.ml.rag.embeddings.huggingface module
      • HuggingfaceTextEmbeddings
    • apache_beam.ml.rag.embeddings.vertex_ai module
      • VertexAITextEmbeddings
• apache_beam.ml.rag.enrichment package
  • Submodules
    • apache_beam.ml.rag.enrichment.bigquery_vector_search module
      • BigQueryVectorSearchParameters
      • BigQueryVectorSearchEnrichmentHandler
      • join_fn()
• apache_beam.ml.rag.ingestion package
  • Submodules
    • apache_beam.ml.rag.ingestion.alloydb module
      • AlloyDBLanguageConnectorConfig
      • AlloyDBVectorWriterConfig
    • apache_beam.ml.rag.ingestion.base module
      • VectorDatabaseWriteConfig
      • VectorDatabaseWriteTransform
    • apache_beam.ml.rag.ingestion.bigquery module
      • SchemaConfig
      • BigQueryVectorWriterConfig
    • apache_beam.ml.rag.ingestion.cloudsql module
      • LanguageConnectorConfig
      • CloudSQLPostgresVectorWriterConfig
    • apache_beam.ml.rag.ingestion.jdbc_common module
      • ConnectionConfig
      • WriteConfig
    • apache_beam.ml.rag.ingestion.postgres module
      • PostgresVectorWriterConfig
    • apache_beam.ml.rag.ingestion.postgres_common module
      • chunk_embedding_fn()
      • ColumnSpec
      • ColumnSpecsBuilder
      • ConflictResolution
    • apache_beam.ml.rag.ingestion.test_utils module
      • TestRow
      • row_to_chunk()
      • ChunkTestUtils
      • HashingFn
      • generate_expected_hash()
      • key_on_id()

Submodules

• apache_beam.ml.rag.types module
  • Content
    • Content.text
  • Embedding
    • Embedding.dense_embedding
    • Embedding.sparse_embedding
  • Chunk
    • Chunk.content
    • Chunk.id
    • Chunk.index
    • Chunk.metadata
    • Chunk.embedding