Package org.apache.beam.sdk.io.clickhouse

Class ClickHouseIO.Write<T>

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PDone>
org.apache.beam.sdk.io.clickhouse.ClickHouseIO.Write<T>
All Implemented Interfaces:
Serializable, HasDisplayData
Enclosing class:
ClickHouseIO
public abstract static class ClickHouseIO.Write<T> extends PTransform<PCollection<T>,PDone>
A PTransform to write to ClickHouse.
See Also:

  • Constructor Details

    • Write

      public Write()

  • Method Details

    • jdbcUrl

      public abstract String jdbcUrl()

    • table

      public abstract String table()

    • properties

      public abstract Properties properties()

    • maxInsertBlockSize

      public abstract long maxInsertBlockSize()

    • maxRetries

      public abstract int maxRetries()

    • maxCumulativeBackoff

      public abstract Duration maxCumulativeBackoff()

    • initialBackoff

      public abstract Duration initialBackoff()

    • tableSchema

      public abstract @Nullable TableSchema tableSchema()

    • insertDistributedSync

      public abstract @Nullable Boolean insertDistributedSync()

    • insertQuorum

      public abstract @Nullable Long insertQuorum()

    • insertDeduplicate

      public abstract @Nullable Boolean insertDeduplicate()

    • expand

      public PDone expand(PCollection<T> input)
      Description copied from class: PTransform
      Override this method to specify how this PTransform should be expanded on the given InputT.

      NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      Specified by:
      expand in class PTransform<PCollection<T>,PDone>

    • withMaxInsertBlockSize

      public ClickHouseIO.Write<T> withMaxInsertBlockSize(long value)
      The maximum block size for insertion, if we control the creation of blocks for insertion.
      Parameters:
      value - number of rows
      Returns:
      a PTransform writing data to ClickHouse
      See Also:

    • withInsertDistributedSync

      public ClickHouseIO.Write<T> withInsertDistributedSync(@Nullable Boolean value)
      If setting is enabled, insert query into distributed waits until data will be sent to all nodes in cluster.
      Parameters:
      value - true to enable, null for server default
      Returns:
      a PTransform writing data to ClickHouse

    • withInsertQuorum

      public ClickHouseIO.Write<T> withInsertQuorum(@Nullable Long value)
      For INSERT queries in the replicated table, wait writing for the specified number of replicas and linearize the addition of the data. 0 - disabled.

      This setting is disabled in default server settings.

      Parameters:
      value - number of replicas, 0 for disabling, null for server default
      Returns:
      a PTransform writing data to ClickHouse
      See Also:

    • withInsertDeduplicate

      public ClickHouseIO.Write<T> withInsertDeduplicate(Boolean value)
      For INSERT queries in the replicated table, specifies that deduplication of inserting blocks should be performed.

      Enabled by default. Shouldn't be disabled unless your input has duplicate blocks, and you don't want to deduplicate them.

      Parameters:
      value - true to enable, null for server default
      Returns:
      a PTransform writing data to ClickHouse

    • withMaxRetries

      public ClickHouseIO.Write<T> withMaxRetries(int value)
      Maximum number of retries per insert.

      See FluentBackoff.withMaxRetries(int).

      Parameters:
      value - maximum number of retries
      Returns:
      a PTransform writing data to ClickHouse

    • withMaxCumulativeBackoff

      public ClickHouseIO.Write<T> withMaxCumulativeBackoff(Duration value)
      Limits total time spent in backoff.

      See FluentBackoff.withMaxCumulativeBackoff(org.joda.time.Duration).

      Parameters:
      value - maximum duration
      Returns:
      a PTransform writing data to ClickHouse

    • withInitialBackoff

      public ClickHouseIO.Write<T> withInitialBackoff(Duration value)
      Set initial backoff duration.

      See FluentBackoff.withInitialBackoff(org.joda.time.Duration).

      Parameters:
      value - initial duration
      Returns:
      a PTransform writing data to ClickHouse

    • withTableSchema

      public ClickHouseIO.Write<T> withTableSchema(@Nullable TableSchema tableSchema)
      Set TableSchema. If not set, then TableSchema will be fetched from clickhouse server itself
      Parameters:
      tableSchema - schema of Table in which rows are going to be inserted
      Returns:
      a PTransform writing data to ClickHouse