Package org.apache.beam.sdk.schemas

Class FieldAccessDescriptor

java.lang.Object

org.apache.beam.sdk.schemas.FieldAccessDescriptor

All Implemented Interfaces:

Serializable

public abstract class FieldAccessDescriptor
extends Object
implements Serializable

Used inside of a DoFn to describe which fields in a schema type need to be accessed for processing.

This class always puts the selected fields in a deterministic order.

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FieldAccessDescriptor.FieldDescriptor	Description of a single field.

Constructor Summary

Constructors

Constructor	Description
FieldAccessDescriptor()

Method Summary

Modifier and Type	Method	Description
static FieldAccessDescriptor	create()	Return an empty FieldAccessDescriptor.
List<Integer>	fieldIdsAccessed()	Return the field ids accessed.
Set<String>	fieldNamesAccessed()	Return the field names accessed.
abstract boolean	getAllFields()	If true, all fields are being accessed.
abstract List<FieldAccessDescriptor.FieldDescriptor>	getFieldsAccessed()
abstract Map<FieldAccessDescriptor.FieldDescriptor,FieldAccessDescriptor>	getNestedFieldsAccessed()
Map<Integer,FieldAccessDescriptor>	nestedFieldsById()	Return the nested fields keyed by field ids.
Map<String,FieldAccessDescriptor>	nestedFieldsByName()	Return the nested fields keyed by field name.
boolean	referencesSingleField()	Returns true if this descriptor references only a single, non-wildcard field.
FieldAccessDescriptor	resolve(Schema schema)	Resolve the FieldAccessDescriptor against a schema.
final String	toString()
static FieldAccessDescriptor	union(Iterable<FieldAccessDescriptor> fieldAccessDescriptors)
static FieldAccessDescriptor	withAllFields()
static FieldAccessDescriptor	withFieldIds(Integer... ids)	Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldIds(Iterable<Integer> ids)	Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldIds(FieldAccessDescriptor baseDescriptor, Integer... fieldIds)	Return a descriptor that accesses the specified field ids as nested subfields of the baseDescriptor.
static FieldAccessDescriptor	withFieldIds(FieldAccessDescriptor baseDescriptor, Iterable<Integer> fieldIds)	Return a descriptor that accesses the specified field ids as nested subfields of the baseDescriptor.
FieldAccessDescriptor	withFieldNameAs(String fieldName, String fieldRename)	Add a field with a new name.
static FieldAccessDescriptor	withFieldNames(Iterable<String> fieldNames)	Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldNames(String... names)	Return a descriptor that access the specified fields.
static FieldAccessDescriptor	withFieldNames(FieldAccessDescriptor baseDescriptor, Iterable<String> fieldNames)	Return a descriptor that accesses the specified field names as nested subfields of the baseDescriptor.
static FieldAccessDescriptor	withFieldNames(FieldAccessDescriptor baseDescriptor, String... fieldNames)	Return a descriptor that accesses the specified field names as nested subfields of the baseDescriptor.
static FieldAccessDescriptor	withFieldNamesAs(Map<String,String> fieldNamesAs)	Return a descriptor that accesses the specified fields, renaming those fields.
static FieldAccessDescriptor	withFields(Iterable<FieldAccessDescriptor.FieldDescriptor> fields)	Returns a FieldAccessDescriptor that accesses the specified fields.
static FieldAccessDescriptor	withFields(FieldAccessDescriptor.FieldDescriptor... fields)	Returns a FieldAccessDescriptor that accesses the specified fields.
FieldAccessDescriptor	withNestedField(int nestedFieldId, FieldAccessDescriptor fieldAccess)	Return a descriptor that access the specified nested field.
FieldAccessDescriptor	withNestedField(String nestedFieldName, FieldAccessDescriptor fieldAccess)	Return a descriptor that access the specified nested field.
FieldAccessDescriptor	withNestedField(FieldAccessDescriptor.FieldDescriptor field, FieldAccessDescriptor fieldAccess)
FieldAccessDescriptor	withNestedFieldAs(String nestedFieldName, String nestedFieldRename, FieldAccessDescriptor fieldAccess)	Like withNestedField(int, org.apache.beam.sdk.schemas.FieldAccessDescriptor) along with a rename of the nested field.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

FieldAccessDescriptor

public FieldAccessDescriptor()

Method Details

getAllFields

public abstract boolean getAllFields()

If true, all fields are being accessed.

getFieldsAccessed

public abstract List<FieldAccessDescriptor.FieldDescriptor> getFieldsAccessed()

getNestedFieldsAccessed

public abstract Map<FieldAccessDescriptor.FieldDescriptor,FieldAccessDescriptor> getNestedFieldsAccessed()

withAllFields

public static FieldAccessDescriptor withAllFields()

withFieldNames

public static FieldAccessDescriptor withFieldNames(String... names)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. The syntax for a field name allows specifying nested fields and wildcards, as specified in the file-level Javadoc. withNestedField can also be called to specify recursive field access.

withFieldNames

public static FieldAccessDescriptor withFieldNames(Iterable<String> fieldNames)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. The syntax for a field name allows specifying nested fields and wildcards, as specified in the file-level Javadoc. withNestedField can also be called to specify recursive field access.

withFieldNamesAs

public static FieldAccessDescriptor withFieldNamesAs(Map<String,String> fieldNamesAs)

Return a descriptor that accesses the specified fields, renaming those fields.

withFieldNames

public static FieldAccessDescriptor withFieldNames(FieldAccessDescriptor baseDescriptor,
 String... fieldNames)

Return a descriptor that accesses the specified field names as nested subfields of the baseDescriptor.

This is only supported when baseDescriptor refers to a single field.

withFieldNames

public static FieldAccessDescriptor withFieldNames(FieldAccessDescriptor baseDescriptor,
 Iterable<String> fieldNames)

Return a descriptor that accesses the specified field names as nested subfields of the baseDescriptor.

This is only supported when baseDescriptor refers to a single field.

withFieldIds

public static FieldAccessDescriptor withFieldIds(FieldAccessDescriptor baseDescriptor,
 Integer... fieldIds)

Return a descriptor that accesses the specified field ids as nested subfields of the baseDescriptor.

This is only supported when baseDescriptor refers to a single field.

withFieldIds

public static FieldAccessDescriptor withFieldIds(FieldAccessDescriptor baseDescriptor,
 Iterable<Integer> fieldIds)

Return a descriptor that accesses the specified field ids as nested subfields of the baseDescriptor.

This is only supported when baseDescriptor refers to a single field.

withFieldIds

public static FieldAccessDescriptor withFieldIds(Integer... ids)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. For finer-grained acccess to nested rows, call withNestedField and pass in a recursive FieldAccessDescriptor.

withFieldIds

public static FieldAccessDescriptor withFieldIds(Iterable<Integer> ids)

Return a descriptor that access the specified fields.

By default, if the field is a nested row (or a container containing a row), all fields of said rows are accessed. For finer-grained acccess to nested rows, call withNestedField and pass in a recursive FieldAccessDescriptor.

withFields

public static FieldAccessDescriptor withFields(FieldAccessDescriptor.FieldDescriptor... fields)

Returns a FieldAccessDescriptor that accesses the specified fields.

withFields

public static FieldAccessDescriptor withFields(Iterable<FieldAccessDescriptor.FieldDescriptor> fields)

Returns a FieldAccessDescriptor that accesses the specified fields.

union

public static FieldAccessDescriptor union(Iterable<FieldAccessDescriptor> fieldAccessDescriptors)

create

public static FieldAccessDescriptor create()

Return an empty FieldAccessDescriptor.

withFieldNameAs

public FieldAccessDescriptor withFieldNameAs(String fieldName,
 String fieldRename)

Add a field with a new name. This is only valid if the fieldName references a single field (wildcards are not allowed here).

withNestedField

public FieldAccessDescriptor withNestedField(int nestedFieldId,
 FieldAccessDescriptor fieldAccess)

Return a descriptor that access the specified nested field. The nested field must be of type Schema.TypeName.ROW, and the fieldAccess argument specifies what fields of the nested type will be accessed.

withNestedField

public FieldAccessDescriptor withNestedField(String nestedFieldName,
 FieldAccessDescriptor fieldAccess)

Return a descriptor that access the specified nested field. The nested field must be of type Schema.TypeName.ROW, and the fieldAccess argument specifies what fields of the nested type will be accessed.

withNestedFieldAs

public FieldAccessDescriptor withNestedFieldAs(String nestedFieldName,
 String nestedFieldRename,
 FieldAccessDescriptor fieldAccess)

Like withNestedField(int, org.apache.beam.sdk.schemas.FieldAccessDescriptor) along with a rename of the nested field.

withNestedField

public FieldAccessDescriptor withNestedField(FieldAccessDescriptor.FieldDescriptor field,
 FieldAccessDescriptor fieldAccess)

fieldIdsAccessed

public List<Integer> fieldIdsAccessed()

Return the field ids accessed. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called. Iteration order is consistent with getFieldsAccessed().

fieldNamesAccessed

public Set<String> fieldNamesAccessed()

Return the field names accessed. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called.

nestedFieldsById

public Map<Integer,FieldAccessDescriptor> nestedFieldsById()

Return the nested fields keyed by field ids. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called.

nestedFieldsByName

public Map<String,FieldAccessDescriptor> nestedFieldsByName()

Return the nested fields keyed by field name. Should not be called until after resolve(org.apache.beam.sdk.schemas.Schema) is called.

referencesSingleField

public boolean referencesSingleField()

Returns true if this descriptor references only a single, non-wildcard field.

resolve

public FieldAccessDescriptor resolve(Schema schema)

Resolve the FieldAccessDescriptor against a schema.

Resolve will resolve all of the field names into field ids, validating that all field names specified in the descriptor exist in the actual schema.

toString

public final String toString()

Overrides:

toString in class Object