Package org.apache.beam.sdk.transforms

Class Latest

java.lang.Object
org.apache.beam.sdk.transforms.Latest
public class Latest extends Object
PTransform and Combine.CombineFn for computing the latest element in a PCollection.

Example: compute the latest value for each session: 


 PCollection<Long> input = ...;
 PCollection<Long> sessioned = input
    .apply(Window.<Long>into(Sessions.withGapDuration(Duration.standardMinutes(5)));
 PCollection<Long> latestValues = sessioned.apply(Latest.<Long>globally());

combineFn() can also be used manually, in combination with state and with the Combine transform.

For elements with the same timestamp, the element chosen for output is arbitrary.

  • Method Details

    • combineFn

      public static <T> Combine.CombineFn<TimestampedValue<T>,?,T> combineFn()
      Returns a Combine.CombineFn that selects the latest element among its inputs.

    • globally

      public static <T> PTransform<PCollection<T>,PCollection<T>> globally()
      Returns a PTransform that takes as input a PCollection<T> and returns a PCollection<T> whose contents is the latest element according to its event time, or null if there are no elements.
      Type Parameters:
      T - The type of the elements being combined.

    • perKey

      public static <K, V> PTransform<PCollection<KV<K,V>>,PCollection<KV<K,V>>> perKey()
      Returns a PTransform that takes as input a PCollection<KV<K, V>> and returns a PCollection<KV<K, V>> whose contents is the latest element per-key according to its event time.
      Type Parameters:
      K - The key type of the elements being combined.
      V - The value type of the elements being combined.