org.apache.beam.sdk.schemas.transforms

Class Unnest.Inner<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PCollection<Row>>

org.apache.beam.sdk.schemas.transforms.Unnest.Inner<T>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

Unnest

public abstract static class Unnest.Inner<T>
extends PTransform<PCollection<T>,PCollection<Row>>

A PTransform that unnests nested row.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Inner()

Method Summary

Modifier and Type	Method and Description
PCollection<Row>	expand(PCollection<T> input) Override this method to specify how this PTransform should be expanded on the given InputT.
Unnest.Inner<T>	withFieldNameFunction(SerializableFunction<java.util.List<java.lang.String>,java.lang.String> fn) Sets a policy for naming deeply-nested fields.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Inner

public Inner()

Method Detail

withFieldNameFunction

public Unnest.Inner<T> withFieldNameFunction(SerializableFunction<java.util.List<java.lang.String>,java.lang.String> fn)

Sets a policy for naming deeply-nested fields.

This is needed to prevent name collisions when differently-nested fields have the same name. The default is to use the Unnest.CONCAT_FIELD_NAMES strategy that concatenates all names in the path to generate the unnested name. For example, an unnested name might be field1_field2_field3. In some cases the Unnest.KEEP_NESTED_NAME strategy can be used to keep only the most-deeply nested name. However if this results in conflicting names (e.g. if a schema has two subrows that each have the same schema this will happen), the pipeline will fail at construction time.

An example of using this function to customize the separator character:

 pc.apply(Unnest.<Type>create().withFieldNameFunction(l -> Strings.join("+", l)));
 

expand

public PCollection<Row> expand(PCollection<T> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<T>,PCollection<Row>>