org.apache.beam.sdk.transforms

Class Latest

java.lang.Object

org.apache.beam.sdk.transforms.Latest

public class Latest
extends java.lang.Object

PTransform and Combine.CombineFn for computing the latest element in a PCollection.

Example: compute the latest value for each session:

 PCollection<Long> input = ...;
 PCollection<Long> sessioned = input
    .apply(Window.<Long>into(Sessions.withGapDuration(Duration.standardMinutes(5)));
 PCollection<Long> latestValues = sessioned.apply(Latest.<Long>globally());
 

combineFn() can also be used manually, in combination with state and with the Combine transform.

For elements with the same timestamp, the element chosen for output is arbitrary.

Method Summary

Modifier and Type	Method and Description
static <T> Combine.CombineFn<TimestampedValue<T>,?,T>	combineFn() Returns a Combine.CombineFn that selects the latest element among its inputs.
static <T> PTransform<PCollection<T>,PCollection<T>>	globally() Returns a PTransform that takes as input a PCollection<T> and returns a PCollection<T> whose contents is the latest element according to its event time, or null if there are no elements.
static <K,V> PTransform<PCollection<KV<K,V>>,PCollection<KV<K,V>>>	perKey() Returns a PTransform that takes as input a PCollection<KV<K, V>> and returns a PCollection<KV<K, V>> whose contents is the latest element per-key according to its event time.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

combineFn

public static <T> Combine.CombineFn<TimestampedValue<T>,?,T> combineFn()

Returns a Combine.CombineFn that selects the latest element among its inputs.

globally

public static <T> PTransform<PCollection<T>,PCollection<T>> globally()

Returns a PTransform that takes as input a PCollection<T> and returns a PCollection<T> whose contents is the latest element according to its event time, or null if there are no elements.

Type Parameters:

T - The type of the elements being combined.

perKey

public static <K,V> PTransform<PCollection<KV<K,V>>,PCollection<KV<K,V>>> perKey()

Returns a PTransform that takes as input a PCollection<KV<K, V>> and returns a PCollection<KV<K, V>> whose contents is the latest element per-key according to its event time.

Type Parameters:

K - The key type of the elements being combined.

V - The value type of the elements being combined.