Package org.apache.beam.sdk.io.jdbc

Class JdbcIO.WriteWithResults<T,V extends JdbcWriteResult>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PCollection<V>>

org.apache.beam.sdk.io.jdbc.JdbcIO.WriteWithResults<T,V>

All Implemented Interfaces:

Serializable, HasDisplayData

Enclosing class:

JdbcIO

public abstract static class JdbcIO.WriteWithResults<T,V extends JdbcWriteResult>
extends PTransform<PCollection<T>,PCollection<V>>

A PTransform to write to a JDBC datasource. Executes statements one by one.

The INSERT, UPDATE, and DELETE commands sometimes have an optional RETURNING clause that supports obtaining data from modified rows while they are being manipulated. Output PCollection of this transform is a collection of such returning results mapped by JdbcIO.RowMapper.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor	Description
WriteWithResults()

Method Summary

Modifier and Type	Method	Description
PCollection<V>	expand(PCollection<T> input)	Override this method to specify how this PTransform should be expanded on the given InputT.
JdbcIO.WriteWithResults<T,V>	withAutoSharding()	If true, enables using a dynamically determined number of shards to write.
JdbcIO.WriteWithResults<T,V>	withBatchSize(long batchSize)
JdbcIO.WriteWithResults<T,V>	withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)
JdbcIO.WriteWithResults<T,V>	withDataSourceProviderFn(SerializableFunction<Void,DataSource> dataSourceProviderFn)
JdbcIO.WriteWithResults<T,V>	withMaxBatchBufferingDuration(long maxBatchBufferingDuration)
JdbcIO.WriteWithResults<T,V>	withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)
JdbcIO.WriteWithResults<T,V>	withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)	When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.
JdbcIO.WriteWithResults<T,V>	withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)	When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements.
JdbcIO.WriteWithResults<T,V>	withRowMapper(JdbcIO.RowMapper<V> rowMapper)
JdbcIO.WriteWithResults<T,V>	withStatement(String statement)
JdbcIO.WriteWithResults<T,V>	withStatement(ValueProvider<String> statement)
JdbcIO.WriteWithResults<T,V>	withTable(String table)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

WriteWithResults

public WriteWithResults()

Method Details

withBatchSize

public JdbcIO.WriteWithResults<T,V> withBatchSize(long batchSize)

withMaxBatchBufferingDuration

public JdbcIO.WriteWithResults<T,V> withMaxBatchBufferingDuration(long maxBatchBufferingDuration)

withDataSourceConfiguration

public JdbcIO.WriteWithResults<T,V> withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)

withDataSourceProviderFn

public JdbcIO.WriteWithResults<T,V> withDataSourceProviderFn(SerializableFunction<Void,DataSource> dataSourceProviderFn)

withStatement

public JdbcIO.WriteWithResults<T,V> withStatement(String statement)

withStatement

public JdbcIO.WriteWithResults<T,V> withStatement(ValueProvider<String> statement)

withPreparedStatementSetter

public JdbcIO.WriteWithResults<T,V> withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)

withAutoSharding

public JdbcIO.WriteWithResults<T,V> withAutoSharding()

If true, enables using a dynamically determined number of shards to write.

withRetryStrategy

public JdbcIO.WriteWithResults<T,V> withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements. If JdbcIO.RetryStrategy.apply(SQLException) returns true, then JdbcIO.Write retries the statements.

withRetryConfiguration

public JdbcIO.WriteWithResults<T,V> withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.

Usage of RetryConfiguration -

 pipeline.apply(JdbcIO.<T>write())
    .withReturningResults(...)
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, Duration.standardSeconds(5), Duration.standardSeconds(1))

 

maxDuration and initialDuration are Nullable

 pipeline.apply(JdbcIO.<T>write())
    .withReturningResults(...)
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, null, null)

 

withTable

public JdbcIO.WriteWithResults<T,V> withTable(String table)

withRowMapper

public JdbcIO.WriteWithResults<T,V> withRowMapper(JdbcIO.RowMapper<V> rowMapper)

expand

public PCollection<V> expand(PCollection<T> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<T>,PCollection<V extends JdbcWriteResult>>