org.apache.beam.sdk.schemas.transforms

Class Unnest

java.lang.Object

org.apache.beam.sdk.schemas.transforms.Unnest

@Experimental(value=SCHEMAS)
public class Unnest
extends java.lang.Object

A PTransform to unnest nested rows.

For example, consider a Row with the following nestedschema:

UserEvent Schema: userid: INT64 timestamp: DATETIME location: LatLong

LatLong Schema: latitude: DOUBLE longitude: DOUBLE

After unnesting, all of the rows will be converted to rows satisfying the following schema:

UserEvent Schema: userid: INT64 timestamp: DATETIME location.latitude: DOUBLE location.longitude: DOUBLE

By default nested names are concatenated to generated the unnested name, however Unnest.Inner.withFieldNameFunction(org.apache.beam.sdk.transforms.SerializableFunction<java.util.List<java.lang.String>, java.lang.String>) can be used to specify a custom naming policy.

Note that currently array and map values are not unnested.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Unnest.Inner<T> A PTransform that unnests nested row.

Field Summary

Fields

Modifier and Type	Field and Description
static SerializableFunction<java.util.List<java.lang.String>,java.lang.String>	CONCAT_FIELD_NAMES This is the default naming policy for naming fields.
static SerializableFunction<java.util.List<java.lang.String>,java.lang.String>	KEEP_NESTED_NAME This policy keeps the raw nested field name.

Constructor Summary

Constructors

Constructor and Description
Unnest()

Method Summary

Modifier and Type	Method and Description
static <T> Unnest.Inner<T>	create()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CONCAT_FIELD_NAMES

public static final SerializableFunction<java.util.List<java.lang.String>,java.lang.String> CONCAT_FIELD_NAMES

This is the default naming policy for naming fields. Every field name in the path to a given field is concated with _ characters.

KEEP_NESTED_NAME

public static final SerializableFunction<java.util.List<java.lang.String>,java.lang.String> KEEP_NESTED_NAME

This policy keeps the raw nested field name. If two differently-nested fields have the same name, unnesting will fail with this policy.

Constructor Detail

Unnest

public Unnest()

Method Detail

create

public static <T> Unnest.Inner<T> create()