Package org.apache.beam.sdk.io.gcp.spanner.changestreams.dao

Class PartitionMetadataAdminDao

java.lang.Object
org.apache.beam.sdk.io.gcp.spanner.changestreams.dao.PartitionMetadataAdminDao
public class PartitionMetadataAdminDao extends Object
Data access object for creating and dropping the partition metadata table.

The partition metadata table will be used to keep the state of a partition as the Connector is performing change stream queries.

  • Field Details

    • COLUMN_PARTITION_TOKEN

      public static final String COLUMN_PARTITION_TOKEN
      Metadata table column name for the partition token.
      See Also:

    • COLUMN_PARENT_TOKENS

      public static final String COLUMN_PARENT_TOKENS
      Metadata table column name for parent partition tokens.
      See Also:

    • COLUMN_START_TIMESTAMP

      public static final String COLUMN_START_TIMESTAMP
      Metadata table column name for the timestamp to start the change stream query of the partition.
      See Also:

    • COLUMN_END_TIMESTAMP

      public static final String COLUMN_END_TIMESTAMP
      Metadata table column name for the timestamp to end the change stream query of the partition.
      See Also:

    • COLUMN_HEARTBEAT_MILLIS

      public static final String COLUMN_HEARTBEAT_MILLIS
      Metadata table column name for the change stream query heartbeat interval in millis.
      See Also:

    • COLUMN_STATE

      public static final String COLUMN_STATE
      Metadata table column name for the state that the partition is currently in. Possible states can be seen in PartitionMetadata.State.
      See Also:

    • COLUMN_WATERMARK

      public static final String COLUMN_WATERMARK
      Metadata table column name for the current watermark of the partition.
      See Also:

    • COLUMN_CREATED_AT

      public static final String COLUMN_CREATED_AT
      Metadata table column name for the timestamp at which the partition row was first created.
      See Also:

    • COLUMN_SCHEDULED_AT

      public static final String COLUMN_SCHEDULED_AT
      Metadata table column name for the timestamp at which the partition was scheduled by the DetectNewPartitionsDoFn SDF.
      See Also:

    • COLUMN_RUNNING_AT

      public static final String COLUMN_RUNNING_AT
      Metadata table column name for the timestamp at which the partition was marked as running by the ReadChangeStreamPartitionDoFn SDF.
      See Also:

    • COLUMN_FINISHED_AT

      public static final String COLUMN_FINISHED_AT
      Metadata table column name for the timestamp at which the partition was marked as finished by the ReadChangeStreamPartitionDoFn SDF.
      See Also:

  • Method Details

    • createPartitionMetadataTable

      public void createPartitionMetadataTable()
      Creates the metadata table in the given instance, database configuration, with the constructor specified table name. The operation is intended to complete in TIMEOUT_MINUTES minutes and specifies a TTL of partition rows after they are marked as FINISHED as TTL_AFTER_PARTITION_FINISHED_DAYS days.

    • deletePartitionMetadataTable

      public void deletePartitionMetadataTable(List<String> indexes)
      Drops the metadata table. This operation should complete in TIMEOUT_MINUTES minutes.