Package org.apache.beam.sdk.io.jdbc

Class JdbcIO.WriteVoid<T>

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PCollection<Void>>
org.apache.beam.sdk.io.jdbc.JdbcIO.WriteVoid<T>
All Implemented Interfaces:
Serializable, HasDisplayData
Enclosing class:
JdbcIO
public abstract static class JdbcIO.WriteVoid<T> extends PTransform<PCollection<T>,PCollection<Void>>
A PTransform to write to a JDBC datasource. Executes statements in a batch, and returns a trivial result.
See Also:

  • Constructor Details

    • WriteVoid

      public WriteVoid()

  • Method Details

    • withAutoSharding

      public JdbcIO.WriteVoid<T> withAutoSharding()
      If true, enables using a dynamically determined number of shards to write.

    • withDataSourceConfiguration

      public JdbcIO.WriteVoid<T> withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)

    • withDataSourceProviderFn

      public JdbcIO.WriteVoid<T> withDataSourceProviderFn(SerializableFunction<Void,DataSource> dataSourceProviderFn)

    • withStatement

      public JdbcIO.WriteVoid<T> withStatement(String statement)

    • withStatement

      public JdbcIO.WriteVoid<T> withStatement(ValueProvider<String> statement)

    • withPreparedStatementSetter

      public JdbcIO.WriteVoid<T> withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)

    • withBatchSize

      public JdbcIO.WriteVoid<T> withBatchSize(long batchSize)
      Provide a maximum size in number of SQL statement for the batch. Default is 1000. The pipeline will either commit a batch when this maximum is reached or its maximum buffering time has been reached. See withMaxBatchBufferingDuration(long)
      Parameters:
      batchSize - maximum batch size in number of statements

    • withMaxBatchBufferingDuration

      public JdbcIO.WriteVoid<T> withMaxBatchBufferingDuration(long maxBatchBufferingDuration)
      Provide maximum buffering time to batch elements before committing SQL statement. Default is 200 The pipeline will either commit a batch when this maximum buffering time has been reached or the maximum amount of elements has been collected. See withBatchSize(long)
      Parameters:
      maxBatchBufferingDuration - maximum time in milliseconds before batch is committed

    • withRetryStrategy

      public JdbcIO.WriteVoid<T> withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)
      When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements. If JdbcIO.RetryStrategy.apply(SQLException) returns true, then JdbcIO.Write retries the statements.

    • withRetryConfiguration

      public JdbcIO.WriteVoid<T> withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)
      When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.

      Usage of RetryConfiguration - 

      
 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, Duration.standardSeconds(5), Duration.standardSeconds(1))
      maxDuration and initialDuration are Nullable 
      
 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, null, null)

    • withTable

      public JdbcIO.WriteVoid<T> withTable(String table)

    • expand

      public PCollection<Void> expand(PCollection<T> input)
      Description copied from class: PTransform
      Override this method to specify how this PTransform should be expanded on the given InputT.

      NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      Specified by:
      expand in class PTransform<PCollection<T>,PCollection<Void>>