Class CustomTimestampPolicyWithLimitedDelay<K,V>

java.lang.Object
org.apache.beam.sdk.io.kafka.TimestampPolicy<K,V>
org.apache.beam.sdk.io.kafka.CustomTimestampPolicyWithLimitedDelay<K,V>

public class CustomTimestampPolicyWithLimitedDelay<K,V> extends TimestampPolicy<K,V>
A policy for custom record timestamps where timestamps within a partition are expected to be roughly monotonically increasing with a cap on out of order event delays (say 1 minute). The watermark at any time is '(Min(now(), Max(event timestamp so far)) - max delay)'. However, watermark is never set in future and capped to 'now - max delay'. In addition, watermark advanced to 'now - max delay' when a partition is idle.

By default the idle advance requires at least one record to have been read, so a partition which is caught up but has delivered nothing since the job started holds its watermark at BoundedWindow.TIMESTAMP_MIN_VALUE until its first record arrives. See CustomTimestampPolicyWithLimitedDelay(SerializableFunction, Duration, Optional, boolean) to opt out of that.