apache_beam.ml.rag.ingestion.qdrant module

class apache_beam.ml.rag.ingestion.qdrant.QdrantConnectionParameters(location: str | None = None, url: str | None = None, port: int | None = 6333, grpc_port: int = 6334, prefer_grpc: bool = False, https: bool | None = None, api_key: str | None = None, prefix: str | None = None, timeout: int | None = None, host: str | None = None, path: str | None = None, kwargs: dict[str, ~typing.Any] = <factory>)

Bases: object

Configuration parameters for connecting to Qdrant service.

Either location, url, host, or path must be provided to establish a connection.

Parameters:

• location – If str - use it as a url parameter. If None - use default values for host and port.

• url – either host or str of “<scheme>//<host>:<port>/<prefix>”. Default: None

• port – Port of the REST API interface. Default: 6333

• grpc_port – Port of the gRPC interface. Default: 6334

• prefer_grpc – If true - use gPRC interface whenever possible.

• https – If true - use HTTPS(SSL) protocol. Default: None

• api_key – API key for authentication in Qdrant Cloud. Default: None

• prefix – If not None - add prefix to the REST URL path. Example: service/v1 will result in http://localhost:6333/service/v1/{qdrant-endpoint} for REST API. Default: None

• timeout – Timeout for REST and gRPC API requests. Default: 5 seconds for REST and unlimited for gRPC

• host – Host name of Qdrant service. If url and host are None, set to ‘localhost’. Default: None

• path – Persistence path for QdrantLocal. Default: None

• **kwargs – Additional arguments passed directly into client initialization

location: str | None = None

url: str | None = None

port: int | None = 6333

grpc_port: int = 6334

prefer_grpc: bool = False

https: bool | None = None

api_key: str | None = None

prefix: str | None = None

timeout: int | None = None

host: str | None = None

path: str | None = None

kwargs: dict[str, Any]

classmethod for_cloud(url: str, api_key: str, *, prefer_grpc: bool = False, timeout: int | None = None, **kwargs: Any) → QdrantConnectionParameters

Connect to Qdrant Cloud. Requires the cluster URL and an API key.

classmethod for_host(host: str, port: int = 6333, *, grpc_port: int = 6334, prefer_grpc: bool = False, https: bool = False, api_key: str | None = None, timeout: int | None = None, **kwargs: Any) → QdrantConnectionParameters

Connect to a self-hosted Qdrant instance by host and port.

classmethod for_url(url: str, *, api_key: str | None = None, prefer_grpc: bool = False, timeout: int | None = None, **kwargs: Any) → QdrantConnectionParameters

Connect using a full URL like ‘https://my-qdrant.example.com:6333’.

classmethod local(path: str) → QdrantConnectionParameters

Use an embedded Qdrant instance persisted to the given path.

classmethod in_memory() → QdrantConnectionParameters

Use an embedded in-memory Qdrant instance. Useful for tests.

class apache_beam.ml.rag.ingestion.qdrant.QdrantWriteConfig(connection_params: ~apache_beam.ml.rag.ingestion.qdrant.QdrantConnectionParameters, collection_name: str, timeout: int | None = None, batch_size: int = 1000, max_batch_byte_size: int = 4194304, kwargs: dict[str, ~typing.Any] = <factory>, dense_embedding_key: str = 'dense', sparse_embedding_key: str = 'sparse')

Bases: VectorDatabaseWriteConfig

Configuration for writing to Qdrant vector database.

This class defines the parameters needed to write data to a qdrant collection, including collection targeting, batching behavior, and operation timeouts.

Parameters:

• connection_params – QdrantConnectionParameters with connection settings.

• collection_name – Name of the Qdrant collection to write to.

• timeout – Optional timeout for write operations in seconds. Default is None.

• batch_size – Number of points to write in each batch. Default is 1000.

• kwargs – Additional keyword arguments to pass to the client’s upsert method.

• dense_embedding_key – name for the dense vector in the qdrant collection.

• sparse_embedding_key – name for the sparse vector in the qdrant collection.

connection_params: QdrantConnectionParameters

collection_name: str

timeout: int | None = None

batch_size: int = 1000

max_batch_byte_size: int = 4194304

kwargs: dict[str, Any]

dense_embedding_key: str = 'dense'

sparse_embedding_key: str = 'sparse'

create_write_transform() → PTransform[EmbeddableItem, Any]

create_converter() → Callable[[EmbeddableItem], models.PointStruct]