Flatten
Merges multiple PCollection
objects into a single logical PCollection
.
By default, the coder for the output PCollection
is the same as the coder
for the first PCollection
in the input PCollectionList
. However, the
input PCollection
objects can each use different coders, as long as
they all contain the same data type in your chosen language.
When using Flatten
to merge PCollection
objects that have a windowing
strategy applied, all of the PCollection
objects you want to merge must
use a compatible windowing strategy and window sizing. For example, all
the collections you’re merging must all use (hypothetically) identical
5-minute fixed windows or 4-minute sliding windows starting every 30 seconds.
If your pipeline attempts to use Flatten
to merge PCollection
objects
with incompatible windows, Beam generates an IllegalStateException
error
when your pipeline is constructed
See more information in the Beam Programming Guide.
Examples
Example: Apply a Flatten
transform to merge multiple PCollection
objects
The resulting collection now has all the elements: “Hello”, “World”, “Beam”, “Is”, and “Fun”.
Related transforms
Last updated on 2024/11/04
Have you found everything you were looking for?
Was it all useful and clear? Is there anything that you would like to change? Let us know!