org.apache.beam.sdk.testing

Class WindowFnTestUtils

java.lang.Object

org.apache.beam.sdk.testing.WindowFnTestUtils

public class WindowFnTestUtils
extends java.lang.Object

A utility class for testing WindowFns.

Constructor Summary

Constructors

Constructor and Description
WindowFnTestUtils()

Method Summary

Modifier and Type	Method and Description
static <T,W extends BoundedWindow> java.util.Collection<W>	assignedWindows(WindowFn<T,W> windowFn, long timestamp) runs WindowFn.assignWindows(WindowFn.AssignContext).
static <T,W extends BoundedWindow> java.util.Collection<W>	assignedWindowsWithValue(WindowFn<T,W> windowFn, TimestampedValue<T> timestampedValue) runs WindowFn.assignWindows(WindowFn.AssignContext).
static <T,W extends BoundedWindow> java.util.Map<W,java.util.Set<java.lang.String>>	runWindowFn(WindowFn<T,W> windowFn, java.util.List<java.lang.Long> timestamps) Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.
static <T,W extends BoundedWindow> java.util.Map<W,java.util.Set<java.lang.String>>	runWindowFnWithValue(WindowFn<T,W> windowFn, java.util.List<TimestampedValue<T>> timestampedValues) Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.
static java.util.Set<java.lang.String>	set(long... timestamps) Creates a Set of elements to be used as expected output in runWindowFn(org.apache.beam.sdk.transforms.windowing.WindowFn<T, W>, java.util.List<java.lang.Long>).
static <T,W extends BoundedWindow> void	validateGetOutputTimestamp(WindowFn<T,W> windowFn, long timestamp) Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.
static <T,W extends IntervalWindow> void	validateGetOutputTimestamps(WindowFn<T,W> windowFn, TimestampCombiner timestampCombiner, java.util.List<java.util.List<java.lang.Long>> timestampsPerWindow) Verifies that later-ending merged windows from any of the timestamps hold up output of earlier-ending windows, using the provided WindowFn and TimestampCombiner.
static <T,W extends IntervalWindow> void	validateGetOutputTimestampsWithValue(WindowFn<T,W> windowFn, TimestampCombiner timestampCombiner, java.util.List<java.util.List<TimestampedValue<T>>> timestampValuesPerWindow) Verifies that later-ending merged windows from any of the timestampValues hold up output of earlier-ending windows, using the provided WindowFn and TimestampCombiner.
static <T,W extends BoundedWindow> void	validateGetOutputTimestampWithValue(WindowFn<T,W> windowFn, TimestampedValue<T> timestampedValue) Assigns the given timestampedValue to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.
static <T,W extends BoundedWindow> void	validateNonInterferingOutputTimes(WindowFn<T,W> windowFn, long timestamp) Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.
static <T,W extends BoundedWindow> void	validateNonInterferingOutputTimesWithValue(WindowFn<T,W> windowFn, TimestampedValue<T> timestampedValue) Assigns the given timestampedValue to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WindowFnTestUtils

public WindowFnTestUtils()

Method Detail

set

public static java.util.Set<java.lang.String> set(long... timestamps)

Creates a Set of elements to be used as expected output in runWindowFn(org.apache.beam.sdk.transforms.windowing.WindowFn<T, W>, java.util.List<java.lang.Long>).

runWindowFn

public static <T,W extends BoundedWindow> java.util.Map<W,java.util.Set<java.lang.String>> runWindowFn(WindowFn<T,W> windowFn,
                                                                                                       java.util.List<java.lang.Long> timestamps)
                                                                                                throws java.lang.Exception

Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.

Throws:

java.lang.Exception

runWindowFnWithValue

public static <T,W extends BoundedWindow> java.util.Map<W,java.util.Set<java.lang.String>> runWindowFnWithValue(WindowFn<T,W> windowFn,
                                                                                                                java.util.List<TimestampedValue<T>> timestampedValues)
                                                                                                         throws java.lang.Exception

Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows. This version allows to pass a list of TimestampedValue in case the values are used to assign windows.

Throws:

java.lang.Exception

assignedWindows

public static <T,W extends BoundedWindow> java.util.Collection<W> assignedWindows(WindowFn<T,W> windowFn,
                                                                                  long timestamp)
                                                                           throws java.lang.Exception

runs WindowFn.assignWindows(WindowFn.AssignContext).

Throws:

java.lang.Exception

assignedWindowsWithValue

public static <T,W extends BoundedWindow> java.util.Collection<W> assignedWindowsWithValue(WindowFn<T,W> windowFn,
                                                                                           TimestampedValue<T> timestampedValue)
                                                                                    throws java.lang.Exception

runs WindowFn.assignWindows(WindowFn.AssignContext). This version allows passing a TimestampedValue in case the value is needed to assign windows.

Throws:

java.lang.Exception

validateNonInterferingOutputTimes

public static <T,W extends BoundedWindow> void validateNonInterferingOutputTimes(WindowFn<T,W> windowFn,
                                                                                 long timestamp)
                                                                          throws java.lang.Exception

Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.

Throws:

java.lang.Exception

validateNonInterferingOutputTimesWithValue

public static <T,W extends BoundedWindow> void validateNonInterferingOutputTimesWithValue(WindowFn<T,W> windowFn,
                                                                                          TimestampedValue<T> timestampedValue)
                                                                                   throws java.lang.Exception

Assigns the given timestampedValue to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound. This version allows passing a TimestampedValue in case the value is needed to assign windows.

Throws:

java.lang.Exception

validateGetOutputTimestamp

public static <T,W extends BoundedWindow> void validateGetOutputTimestamp(WindowFn<T,W> windowFn,
                                                                          long timestamp)
                                                                   throws java.lang.Exception

Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.

This verifies that overlapping windows don't interfere at all. Depending on the windowFn this may be stricter than desired.

Throws:

java.lang.Exception

validateGetOutputTimestampWithValue

public static <T,W extends BoundedWindow> void validateGetOutputTimestampWithValue(WindowFn<T,W> windowFn,
                                                                                   TimestampedValue<T> timestampedValue)
                                                                            throws java.lang.Exception

Assigns the given timestampedValue to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.

This verifies that overlapping windows don't interfere at all. Depending on the windowFn this may be stricter than desired. This version allows passing a TimestampedValue in case the value is needed to assign windows.

Throws:

java.lang.Exception

validateGetOutputTimestamps

public static <T,W extends IntervalWindow> void validateGetOutputTimestamps(WindowFn<T,W> windowFn,
                                                                            TimestampCombiner timestampCombiner,
                                                                            java.util.List<java.util.List<java.lang.Long>> timestampsPerWindow)
                                                                     throws java.lang.Exception

Verifies that later-ending merged windows from any of the timestamps hold up output of earlier-ending windows, using the provided WindowFn and TimestampCombiner.

Given a list of lists of timestamps, where each list is expected to merge into a single window with end times in ascending order, assigns and merges windows for each list (as though each were a separate key/user session). Then combines each timestamp in the list according to the provided TimestampCombiner.

Verifies that a overlapping windows do not hold each other up via the watermark.

Throws:

java.lang.Exception

validateGetOutputTimestampsWithValue

public static <T,W extends IntervalWindow> void validateGetOutputTimestampsWithValue(WindowFn<T,W> windowFn,
                                                                                     TimestampCombiner timestampCombiner,
                                                                                     java.util.List<java.util.List<TimestampedValue<T>>> timestampValuesPerWindow)
                                                                              throws java.lang.Exception

Verifies that later-ending merged windows from any of the timestampValues hold up output of earlier-ending windows, using the provided WindowFn and TimestampCombiner.

Given a list of lists of timestampValues, where each list is expected to merge into a single window with end times in ascending order, assigns and merges windows for each list (as though each were a separate key/user session). Then combines each timestamp in the list according to the provided TimestampCombiner.

Verifies that a overlapping windows do not hold each other up via the watermark. This version allows passing TimestampedValue in case the value is needed to assign windows.

Throws:

java.lang.Exception