org.apache.beam.sdk.transforms

Class Flatten

java.lang.Object

org.apache.beam.sdk.transforms.Flatten

public class Flatten
extends java.lang.Object

Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections. The name "Flatten" suggests taking a list of lists and flattening them into a single list.

Example of use:

 PCollection<String> pc1 = ...;
 PCollection<String> pc2 = ...;
 PCollection<String> pc3 = ...;
 PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
 PCollection<String> merged = pcs.apply(Flatten.<String>pCollections());
 

By default, the Coder of the output PCollection is the same as the Coder of the first PCollection in the input PCollectionList (if the PCollectionList is non-empty).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Flatten.Iterables<T> FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a PCollection<T> that contains all the elements from each iterable.
static class	Flatten.PCollections<T> A PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

Constructor Summary

Constructors

Constructor and Description
Flatten()

Method Summary

Modifier and Type	Method and Description
static <T> Flatten.Iterables<T>	iterables() Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.
static <T> Flatten.PCollections<T>	pCollections() Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Flatten

public Flatten()

Method Detail

pCollections

public static <T> Flatten.PCollections<T> pCollections()

Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

All inputs must have equal WindowFns. The output elements of Flatten<T> are in the same windows and have the same timestamps as their corresponding input elements. The output PCollection will have the same WindowFn as all of the inputs.

Type Parameters:

T - the type of the elements in the input and output PCollections.

iterables

public static <T> Flatten.Iterables<T> iterables()

Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

Example of use:

 PCollection<Iterable<Integer>> pcOfIterables = ...;
 PCollection<Integer> pc = pcOfIterables.apply(Flatten.<Integer>iterables());
 

By default, the output PCollection encodes its elements using the same Coder that the input uses for the elements in its Iterable.

Type Parameters:

T - the type of the elements of the input Iterable and the output PCollection