apache_beam.ml.rag.ingestion.jdbc_common module
- class apache_beam.ml.rag.ingestion.jdbc_common.ConnectionConfig(jdbc_url: str, username: str, password: str, connection_properties: ~typing.Dict[str, str] | None = None, connection_init_sqls: ~typing.List[str] | None = None, additional_jdbc_args: ~typing.Dict[str, ~typing.Any] = <factory>)[source]
Bases:
object
Configuration for connecting to a JDBC database.
Provides connection details and options for connecting to a database instance.
- jdbc_url
JDBC URL for the database instance. Example: ‘jdbc:postgresql://host:port/database’
- Type:
- connection_properties
Optional JDBC connection properties dict. Example: {‘ssl’: ‘true’}
- connection_init_sqls
Optional list of SQL statements to execute when connection is established.
- Type:
List[str] | None
- additional_jdbc_args
Additional arguments that will be passed to WriteToJdbc. These may include ‘driver_jars’, ‘expansion_service’, ‘classpath’, etc. See full set of args at
WriteToJdbc
- Type:
Dict[str, Any]
Example
>>> config = AlloyDBConnectionConfig( ... jdbc_url='jdbc:postgresql://localhost:5432/mydb', ... username='user', ... password='pass', ... connection_properties={'ssl': 'true'}, ... max_connections=10 ... )
- class apache_beam.ml.rag.ingestion.jdbc_common.WriteConfig(autosharding: bool | None = None, max_connections: int | None = None, write_batch_size: int | None = None)[source]
Bases:
object
Configuration writing to JDBC database.
Modifies the write behavior when writing via JdbcIO.
- autosharding
Enable automatic re-sharding of bundles to scale the number of shards with workers.
- Type:
bool | None