@Internal public class PartitionReconciler extends java.lang.Object
Example of race condition:
To reconcile this, we identify partitions that haven't been streamed for at least 5 minutes. This is probably an indication that there were some races of CloseStream merge messages.
Constructor and Description |
---|
PartitionReconciler(MetadataTableDao metadataTableDao) |
Modifier and Type | Method and Description |
---|---|
void |
addMissingPartitions(java.util.List<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange> missingPartitions)
Capture partitions that are not currently being streamed.
|
void |
addNewPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition,
com.google.protobuf.ByteString rowKey)
Capture NewPartition row that's waiting to be created.
|
java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.util.Set<com.google.protobuf.ByteString>> |
getPartitionsToReconcile()
Match partitions that have been missing for a while and need to be reconciled with NewPartition
row key.
|
public PartitionReconciler(MetadataTableDao metadataTableDao)
public void addMissingPartitions(java.util.List<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange> missingPartitions)
Combine existing missing partitions and current (newly added) missing partitions. If missing partitions have been missing for more than allotted time, it will be reconciled.
It is possible that a missing partition's boundary can change frequently, such that it can take a long time to realize a partition is truly missing. For example, if [C, D) is missing, but there are a lot of splits and merges around [C, D), we may see that sometimes [B,D) is missing, or at other times [C-E) is missing due to split and merge activities of [B-C) and [D-E), while [C-D) is truly missing. The moving boundaries would reset the timer leading to slower reconciliation of the missing partition.
missingPartitions
- partitions not being streamed.public void addNewPartition(com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange partition, com.google.protobuf.ByteString rowKey)
partition
- new partitions waiting to be created.rowKey
- the full row key of the new partition.public java.util.HashMap<com.google.cloud.bigtable.data.v2.models.Range.ByteStringRange,java.util.Set<com.google.protobuf.ByteString>> getPartitionsToReconcile()