@Internal public class MetadataTableDao extends java.lang.Object
Metadata table is shared across many beam jobs. Each beam job uses a specific prefix to identify itself which is used as the row prefix.
Constructor and Description |
---|
MetadataTableDao(com.google.cloud.bigtable.data.v2.BigtableDataClient dataClient,
java.lang.String tableId,
com.google.protobuf.ByteString changeStreamNamePrefix) |
Modifier and Type | Method and Description |
---|---|
com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange |
convertNewPartitionRowKeyToPartition(com.google.protobuf.ByteString rowKey)
Convert new partition row key to partition to process metadata read from Bigtable.
|
com.google.protobuf.ByteString |
convertPartitionToNewPartitionRowKey(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
Convert partition to a New Partition row key to query for partitions ready to be streamed as
the result of splits and merges.
|
com.google.protobuf.ByteString |
convertPartitionToStreamPartitionRowKey(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
Convert partition to a Stream Partition row key to query for metadata of partitions that are
currently being streamed.
|
com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange |
convertStreamPartitionRowKeyToPartition(com.google.protobuf.ByteString rowKey)
Convert stream partition row key to partition to process metadata read from Bigtable.
|
void |
deleteRowKey(com.google.protobuf.ByteString rowKey)
Delete the row.
|
void |
deleteStreamPartitionRow(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
Delete the row key represented by the partition.
|
com.google.protobuf.ByteString |
getChangeStreamNamePrefix() |
boolean |
lockPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition,
java.lang.String uuid)
Lock the partition in the metadata table for the DoFn streaming it.
|
java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.lang.Long> |
readDetectNewPartitionMissingPartitions()
Read and deserialize missing partition and how long they have been missing from the metadata
table.
|
com.google.api.gax.rpc.ServerStream<com.google.cloud.bigtable.data.v2.models.Row> |
readFromMdTableStreamPartitionsWithWatermark() |
com.google.api.gax.rpc.ServerStream<com.google.cloud.bigtable.data.v2.models.Row> |
readNewPartitions() |
void |
updateWatermark(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition,
Instant watermark,
com.google.cloud.bigtable.data.v2.models.ChangeStreamContinuationToken currentToken)
Update the metadata for the row key represented by the partition.
|
void |
writeDetectNewPartitionMissingPartitions(java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.lang.Long> missingPartitionDurations)
Write to metadata table serialized missing partitions and how long they have been missing.
|
void |
writeDetectNewPartitionVersion()
Set the version number for DetectNewPartition.
|
void |
writeNewPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange newPartition,
com.google.cloud.bigtable.data.v2.models.ChangeStreamContinuationToken changeStreamContinuationToken,
com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange parentPartition,
Instant lowWatermark)
After a split or merge from a close stream, write the new partition's information to the
metadata table.
|
public MetadataTableDao(com.google.cloud.bigtable.data.v2.BigtableDataClient dataClient, java.lang.String tableId, com.google.protobuf.ByteString changeStreamNamePrefix)
public com.google.protobuf.ByteString getChangeStreamNamePrefix()
public com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange convertStreamPartitionRowKeyToPartition(com.google.protobuf.ByteString rowKey) throws com.google.protobuf.InvalidProtocolBufferException
RowKey should be directly from Cloud Bigtable and not altered in any way.
rowKey
- row key from Cloud Bigtablecom.google.protobuf.InvalidProtocolBufferException
- if conversion from rowKey to partition failspublic com.google.protobuf.ByteString convertPartitionToStreamPartitionRowKey(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
partition
- convert to row keypublic com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange convertNewPartitionRowKeyToPartition(com.google.protobuf.ByteString rowKey) throws com.google.protobuf.InvalidProtocolBufferException
RowKey should be directly from Cloud Bigtable and not altered in any way.
rowKey
- row key from Cloud Bigtablecom.google.protobuf.InvalidProtocolBufferException
- if conversion from rowKey to partition failspublic com.google.protobuf.ByteString convertPartitionToNewPartitionRowKey(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
partition
- convert to row keypublic com.google.api.gax.rpc.ServerStream<com.google.cloud.bigtable.data.v2.models.Row> readNewPartitions()
public void writeNewPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange newPartition, com.google.cloud.bigtable.data.v2.models.ChangeStreamContinuationToken changeStreamContinuationToken, com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange parentPartition, Instant lowWatermark)
newPartition
- the new partitionchangeStreamContinuationToken
- the token that can be used to pick up from where the
parent left offparentPartition
- the parent that stopped and split or mergedlowWatermark
- the low watermark of the parent streampublic com.google.api.gax.rpc.ServerStream<com.google.cloud.bigtable.data.v2.models.Row> readFromMdTableStreamPartitionsWithWatermark()
public void updateWatermark(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition, Instant watermark, @Nullable com.google.cloud.bigtable.data.v2.models.ChangeStreamContinuationToken currentToken)
partition
- forms the row key of the row to updatewatermark
- watermark value to set for the cellcurrentToken
- continuation token to set for the cellpublic void deleteStreamPartitionRow(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition)
partition
- forms the row key of the row to deletepublic void deleteRowKey(com.google.protobuf.ByteString rowKey)
rowKey
- row key of the row to deletepublic boolean lockPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition, java.lang.String uuid)
partition
- form the row key in the metadata table to lockuuid
- id of the DoFnpublic void writeDetectNewPartitionVersion()
public java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.lang.Long> readDetectNewPartitionMissingPartitions()
public void writeDetectNewPartitionMissingPartitions(java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.lang.Long> missingPartitionDurations)
missingPartitionDurations
- missing partitions and duration.