apache_beam.yaml.yaml_combine module
This module defines the basic Combine operation.
- apache_beam.yaml.yaml_combine.normalize_combine(spec)[source]
Expands various shorthand specs for combine (which can otherwise be quite verbose for simple cases.) We do this here so that it doesn’t need to be done per language. The following are all equivalent:
dest: fn_type dest: value: dest fn: fn_type dest: value: dest fn: type: fn_type
- class apache_beam.yaml.yaml_combine.PyJsYamlCombine(group_by: Iterable[str], combine: Mapping[str, Mapping[str, Any]], language: str | None = None)[source]
Bases:
PTransform
Groups and combines records sharing common fields.
Built-in combine functions are sum, max, min, all, any, mean, count, group, concat but custom aggregation functions can be used as well.
See also the documentation on [YAML Aggregation](https://beam.apache.org/documentation/sdks/yaml-combine/).