Python transform catalog overview

Element-wise

TransformDescription
FilterGiven a predicate, filter out all elements that don't satisfy the predicate.
FlatMapApplies a function that returns a collection to every element in the input and outputs all resulting elements.
KeysExtracts the key from each element in a collection of key-value pairs.
KvSwapSwaps the key and value of each element in a collection of key-value pairs.
MapApplies a function to every element in the input and outputs the result.
ParDoThe most-general mechanism for applying a user-defined DoFn to every element in the input collection.
PartitionRoutes each input element to a specific output collection based on some partition function.
RegexFilters input string elements based on a regex. May also transform them based on the matching groups.
ReifyTransforms for converting between explicit and implicit form of various Beam values.
ToStringTransforms every element in an input collection a string.
WithTimestampsApplies a function to determine a timestamp to each element in the output collection, and updates the implicit timestamp associated with each input. Note that it is only safe to adjust timestamps forwards.
ValuesExtracts the value from each element in a collection of key-value pairs.

Aggregation

TransformDescription
ApproximateQuantilesNot available. See BEAM-6694 for updates.
ApproximateUniqueNot available. See BEAM-6693 for updates.
CoGroupByKeyTakes several keyed collections of elements and produces a collection where each element consists of a key and all values associated with that key.
CombineGloballyTransforms to combine elements.
CombineWithContextNot available.
CountCounts the number of elements within each aggregation.
DistinctProduces a collection containing distinct elements from the input collection.
GroupByKeyTakes a keyed collection of elements and produces a collection where each element consists of a key and all values associated with that key.
GroupIntoBatchesNot available. See BEAM-6696 for updates.
LatestNot available. See BEAM-6695 for updates.
MaxNot available.
MeanComputes the average within each aggregation.
MinNot available.
SampleRandomly select some number of elements from each aggregation.
SumNot available.
TopCompute the largest element(s) in each aggregation.

Other

TransformDescription
CreateCreates a collection from an in-memory list.
FlattenGiven multiple input collections, produces a single output collection containing all elements from all of the input collections.
PAssertNot available.
ReshuffleGiven an input collection, redistributes the elements between workers. This is most useful for adjusting parallelism or preventing coupled failures.
ViewNot available.
WindowIntoLogically divides up or groups the elements of a collection into finite windows according to a function.