org.apache.beam.sdk.io.aws2.kinesis

Interface KinesisPartitioner<T>

All Superinterfaces:

java.io.Serializable

All Known Subinterfaces:

KinesisPartitioner.ExplicitPartitioner<T>

public interface KinesisPartitioner<T>
extends java.io.Serializable

Kinesis interface for custom partitioner.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	KinesisPartitioner.ExplicitPartitioner<T> An explicit partitioner that always returns a Nonnull explicit hash key.

Field Summary

Fields

Modifier and Type	Field and Description
static java.math.BigInteger	MAX_HASH_KEY
static java.math.BigInteger	MIN_HASH_KEY

Method Summary

Modifier and Type	Method and Description
static <T> KinesisPartitioner<T>	explicitRandomPartitioner(int shards) Explicit hash key partitioner that randomly returns one of x precalculated hash keys.
default java.lang.String	getExplicitHashKey(T record) Optional hash value (128-bit integer) to determine explicitly the shard a record is assigned to based on the hash key range of each shard.
java.lang.String	getPartitionKey(T record) Determines which shard in the stream the record is assigned to.

Field Detail

MIN_HASH_KEY

static final java.math.BigInteger MIN_HASH_KEY

MAX_HASH_KEY

static final java.math.BigInteger MAX_HASH_KEY

Method Detail

getPartitionKey

@Nonnull
java.lang.String getPartitionKey(T record)

Determines which shard in the stream the record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard.

getExplicitHashKey

@Nullable
default java.lang.String getExplicitHashKey(T record)

Optional hash value (128-bit integer) to determine explicitly the shard a record is assigned to based on the hash key range of each shard. The explicit hash key overrides the partition key hash.

explicitRandomPartitioner

static <T> KinesisPartitioner<T> explicitRandomPartitioner(int shards)

Explicit hash key partitioner that randomly returns one of x precalculated hash keys. Hash keys are derived by equally dividing the 128-bit hash universe, assuming that hash ranges of shards are also equally sized.

Note: This simple approach is likely not applicable anymore after resharding a stream. In that case it is recommended to use the ListShards API to retrieve the actual hash key range of each shard and partition based on that.

See Also:

ListShards API