Beam SQL extensions: Joins

Supported JOIN types in Beam SQL:

Unsupported JOIN types in Beam SQL:

The scenarios of join can be categorized into 3 cases:

  1. Bounded input JOIN bounded input
  2. Unbounded input JOIN unbounded input
  3. Unbounded input JOIN bounded input

Bounded JOIN Bounded

Standard join implementation is used. All elements from one input are matched with all elements from another input. Due to the fact that both inputs are bounded, no windowing or triggering is involved.

Unbounded JOIN Unbounded

Standard join implementation is used. All elements from one input are matched with all elements from another input.

Windowing and Triggering

The following properties must be satisfied when joining unbounded inputs:

This means that inputs are joined per-window. That is, when the trigger fires (only once), then join is performed on all elements in the current window in both inputs. This allows to reason about what kind of output is going to be produced.

Note: similarly to GroupByKeys JOIN will update triggers using Trigger.continuationTrigger(). Other aspects of the inputs’ windowing strategies remain unchanged.

Unbounded JOIN Bounded

For this type of JOIN bounded input is treated as a side-input by the implementation. This means that window/trigger is inherited from upstreams.