org.apache.beam.sdk.schemas

Class FieldAccessDescriptor

java.lang.Object

org.apache.beam.sdk.schemas.FieldAccessDescriptor

All Implemented Interfaces:

java.io.Serializable

@Experimental(value=SCHEMAS)
public abstract class FieldAccessDescriptor
extends java.lang.Object
implements java.io.Serializable

Used inside of a DoFn to describe which fields in a schema type need to be accessed for processing.

This class always puts the selected fields in a deterministic order.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FieldAccessDescriptor.FieldDescriptor Description of a single field.

Constructor Summary

Constructors

Constructor and Description
FieldAccessDescriptor()

Method Summary

Modifier and Type	Method and Description
static FieldAccessDescriptor	create() Return an empty FieldAccessDescriptor.
java.util.List<java.lang.Integer>	fieldIdsAccessed() Return the field ids accessed.
abstract boolean	getAllFields() If true, all fields are being accessed.
abstract boolean	getFieldInsertionOrder()
abstract java.util.List<FieldAccessDescriptor.FieldDescriptor>	getFieldsAccessed()
abstract java.util.Map<FieldAccessDescriptor.FieldDescriptor,FieldAccessDescriptor>	getNestedFieldsAccessed()
java.util.Map<java.lang.Integer,FieldAccessDescriptor>	nestedFieldsById() Return the nested fields keyed by field ids.
FieldAccessDescriptor	resolve(Schema schema) Resolve the FieldAccessDescriptor against a schema.
static FieldAccessDescriptor	withAllFields()
static FieldAccessDescriptor	withFieldIds(java.lang.Integer... ids) Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldIds(java.lang.Iterable<java.lang.Integer> ids) Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldNames(java.lang.Iterable<java.lang.String> fieldNames) Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldNames(java.lang.String... names) Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFields(FieldAccessDescriptor.FieldDescriptor... fields) Returns a FieldAccessDescriptor that accesses the specified fields.
static FieldAccessDescriptor	withFields(java.lang.Iterable<FieldAccessDescriptor.FieldDescriptor> fields) Returns a FieldAccessDescriptor that accesses the specified fields.
FieldAccessDescriptor	withNestedField(FieldAccessDescriptor.FieldDescriptor field, FieldAccessDescriptor fieldAccess)
FieldAccessDescriptor	withNestedField(int nestedFieldId, FieldAccessDescriptor fieldAccess) Return a descriptor that access the specified nested field.
FieldAccessDescriptor	withNestedField(java.lang.String nestedFieldName, FieldAccessDescriptor fieldAccess) Return a descriptor that access the specified nested field.
FieldAccessDescriptor	withOrderByFieldInsertionOrder() By default, fields are sorted by name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FieldAccessDescriptor

public FieldAccessDescriptor()

Method Detail

getAllFields

public abstract boolean getAllFields()

If true, all fields are being accessed.

getFieldsAccessed

public abstract java.util.List<FieldAccessDescriptor.FieldDescriptor> getFieldsAccessed()

getNestedFieldsAccessed

public abstract java.util.Map<FieldAccessDescriptor.FieldDescriptor,FieldAccessDescriptor> getNestedFieldsAccessed()

getFieldInsertionOrder

public abstract boolean getFieldInsertionOrder()

withAllFields

public static FieldAccessDescriptor withAllFields()

withFieldNames

public static FieldAccessDescriptor withFieldNames(java.lang.String... names)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. The syntax for a field name allows specifying nested fields and wildcards, as specified in the file-level Javadoc. withNestedField can also be called to specify recursive field access.

withFieldNames

public static FieldAccessDescriptor withFieldNames(java.lang.Iterable<java.lang.String> fieldNames)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. The syntax for a field name allows specifying nested fields and wildcards, as specified in the file-level Javadoc. withNestedField can also be called to specify recursive field access.

withFieldIds

public static FieldAccessDescriptor withFieldIds(java.lang.Integer... ids)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. For finer-grained acccess to nested rows, call withNestedField and pass in a recursive FieldAccessDescriptor.

withFieldIds

public static FieldAccessDescriptor withFieldIds(java.lang.Iterable<java.lang.Integer> ids)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. For finer-grained acccess to nested rows, call withNestedField and pass in a recursive FieldAccessDescriptor.

withFields

public static FieldAccessDescriptor withFields(FieldAccessDescriptor.FieldDescriptor... fields)

Returns a FieldAccessDescriptor that accesses the specified fields.

withFields

public static FieldAccessDescriptor withFields(java.lang.Iterable<FieldAccessDescriptor.FieldDescriptor> fields)

Returns a FieldAccessDescriptor that accesses the specified fields.

create

public static FieldAccessDescriptor create()

Return an empty FieldAccessDescriptor.

withNestedField

public FieldAccessDescriptor withNestedField(int nestedFieldId,
                                             FieldAccessDescriptor fieldAccess)

Return a descriptor that access the specified nested field. The nested field must be of type Schema.TypeName.ROW, and the fieldAccess argument specifies what fields of the nested type will be accessed.

withNestedField

public FieldAccessDescriptor withNestedField(java.lang.String nestedFieldName,
                                             FieldAccessDescriptor fieldAccess)

Return a descriptor that access the specified nested field. The nested field must be of type Schema.TypeName.ROW, and the fieldAccess argument specifies what fields of the nested type will be accessed.

withNestedField

public FieldAccessDescriptor withNestedField(FieldAccessDescriptor.FieldDescriptor field,
                                             FieldAccessDescriptor fieldAccess)

withOrderByFieldInsertionOrder

public FieldAccessDescriptor withOrderByFieldInsertionOrder()

By default, fields are sorted by name. If this is set, they will instead be sorted by insertion order. All sorting happens in the resolve(Schema) method.

fieldIdsAccessed

public java.util.List<java.lang.Integer> fieldIdsAccessed()

Return the field ids accessed. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called. Iteration order is consistent with getFieldsAccessed().

nestedFieldsById

public java.util.Map<java.lang.Integer,FieldAccessDescriptor> nestedFieldsById()

Return the nested fields keyed by field ids. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called.

resolve

public FieldAccessDescriptor resolve(Schema schema)

Resolve the FieldAccessDescriptor against a schema.

Resolve will resolve all of the field names into field ids, validating that all field names specified in the descriptor exist in the actual schema.