apache_beam.ml.rag.ingestion.jdbc_common module

class apache_beam.ml.rag.ingestion.jdbc_common.ConnectionConfig(jdbc_url: str, username: str, password: str, connection_properties: dict[str, str] | None = None, connection_init_sqls: list[str] | None = None, additional_jdbc_args: dict[str, ~typing.Any] = <factory>)

Bases: object

Configuration for connecting to a JDBC database.

Provides connection details and options for connecting to a database instance.

jdbc_url

JDBC URL for the database instance. Example: ‘jdbc:postgresql://host:port/database’

Type:

str

username

Database username.

Type:

str

password

Database password.

Type:

str

connection_properties

Optional JDBC connection properties dict. Example: {‘ssl’: ‘true’}

Type:

dict[str, str] | None

connection_init_sqls

Optional list of SQL statements to execute when connection is established.

Type:

list[str] | None

additional_jdbc_args

Additional arguments that will be passed to WriteToJdbc. These may include ‘driver_jars’, ‘expansion_service’, ‘classpath’, etc. See full set of args at WriteToJdbc

Type:

dict[str, Any]

Example

>>> config = AlloyDBConnectionConfig(
...     jdbc_url='jdbc:postgresql://localhost:5432/mydb',
...     username='user',
...     password='pass',
...     connection_properties={'ssl': 'true'},
...     max_connections=10
... )

jdbc_url: str

username: str

password: str

connection_properties: dict[str, str] | None = None

connection_init_sqls: list[str] | None = None

additional_jdbc_args: dict[str, Any]

class apache_beam.ml.rag.ingestion.jdbc_common.WriteConfig(autosharding: bool | None = None, max_connections: int | None = None, write_batch_size: int | None = None)

Bases: object

Configuration writing to JDBC database.

Modifies the write behavior when writing via JdbcIO.

autosharding

Enable automatic re-sharding of bundles to scale the number of shards with workers.

Type:

bool | None

max_connections

Optional number of connections in the pool. Use negative for no limit.

Type:

int | None

write_batch_size

Optional write batch size for bulk operations.

Type:

int | None

autosharding: bool | None = None

max_connections: int | None = None

write_batch_size: int | None = None