org.apache.beam.sdk.io.thrift

Class ThriftSchema

java.lang.Object

org.apache.beam.sdk.schemas.GetterBasedSchemaProvider

org.apache.beam.sdk.schemas.GetterBasedSchemaProviderV2

org.apache.beam.sdk.io.thrift.ThriftSchema

All Implemented Interfaces:

java.io.Serializable, SchemaProvider

public final class ThriftSchema
extends GetterBasedSchemaProviderV2

Schema provider for generated thrift types.

• Primitive type mapping is straight-forward (e.g. TType.I32 -> Schema.FieldType.INT32).
• TType.STRING gets mapped as either Schema.FieldType.STRING or Schema.FieldType.BYTES, depending on whether the FieldValueMetaData.isBinary() flag is set.
• TType.MAP becomes a beam map passing the key and value types recursively.
• TType.SET gets translated into a beam iterable, passing the corresponding element type.
• TType.LIST becomes an array of the corresponding element type.
• thrift enums are converted into beam enumeration types.
• thrift union types get mapped to beam one-of types.

The mapping logic relies on the available thrift metadata introspection and tries to make as few assumptions about the generated code as possible (i.e. does not rely on accessor naming convention, as the thrift compiler supports options such as "beans" or "fullcamel"/"nocamel".
However, the following strong assumptions are made by this class:

• All thrift generated classes implement TBase, except for enums which become java enums implementing TEnum.
• All TUnion types provide static factory methods for each of the supported field types, with the same name as the field itself and only one such method taking a single parameter exists.
• All non-union types have a corresponding java field with the same name for every field in the original thrift source file.

Thrift typedefs for container types (and possibly others) do not preserve the full type information. For this reason, this class allows for manual registration of such "lossy" typedefs with their corresponding beam types.

Note: Thrift encoding and decoding are not fully symmetrical, i.e. the isSet flag may not be preserved upon converting a thrift object to a beam row and back. On encoding, we extract all thrift values, no matter if the fields are set or not. On decoding, we set all non-null beam row values to the corresponding thrift fields, leaving the rest unset.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ThriftSchema.Customizer

Method Summary

Modifier and Type	Method and Description
static @NonNull ThriftSchema.Customizer	custom() Builds a schema provider that maps any thrift type to a Beam schema, allowing for custom thrift typedef entries (which cannot be resolved using the available metadata) to be manually registered with their corresponding beam types.
@NonNull java.util.List<FieldValueGetter>	fieldValueGetters(@NonNull TypeDescriptor<?> targetTypeDescriptor, @NonNull Schema schema) Delegates to the GetterBasedSchemaProvider.fieldValueGetters(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.
@NonNull java.util.List<FieldValueTypeInformation>	fieldValueTypeInformations(@NonNull TypeDescriptor<?> targetTypeDescriptor, @NonNull Schema schema) Delegates to the GetterBasedSchemaProvider.fieldValueTypeInformations(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.
static @NonNull SchemaProvider	provider() Schema provider that maps any thrift type to a Beam schema, assuming that any typedefs that might have been used in the thrift definitions will preserve all required metadata to infer the beam type (which is the case for any primitive typedefs and alike).
<T> @NonNull Schema	schemaFor(TypeDescriptor<T> typeDescriptor) Lookup a schema for the given type.
@NonNull SchemaUserTypeCreator	schemaTypeCreator(@NonNull TypeDescriptor<?> targetTypeDescriptor, @NonNull Schema schema) Delegates to the GetterBasedSchemaProvider.schemaTypeCreator(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.

Methods inherited from class org.apache.beam.sdk.schemas.GetterBasedSchemaProviderV2

fieldValueGetters, fieldValueTypeInformations, schemaTypeCreator

Methods inherited from class org.apache.beam.sdk.schemas.GetterBasedSchemaProvider

equals, fromRowFunction, hashCode, toRowFunction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Method Detail

provider

public static @NonNull SchemaProvider provider()

Schema provider that maps any thrift type to a Beam schema, assuming that any typedefs that might have been used in the thrift definitions will preserve all required metadata to infer the beam type (which is the case for any primitive typedefs and alike).

See Also:

for how to manually pass the beam type for container typedefs

custom

public static @NonNull ThriftSchema.Customizer custom()

Builds a schema provider that maps any thrift type to a Beam schema, allowing for custom thrift typedef entries (which cannot be resolved using the available metadata) to be manually registered with their corresponding beam types.

E.g. typedef set<string> StringSet will not carry the element type information and needs to be manually mapped here as .custom().withTypedef("StringSet", FieldType.iterable(FieldType.STRING)).provider().

schemaFor

public <T> @NonNull Schema schemaFor(TypeDescriptor<T> typeDescriptor)

Description copied from interface: SchemaProvider

Lookup a schema for the given type. If no schema exists, returns null.

fieldValueGetters

public @NonNull java.util.List<FieldValueGetter> fieldValueGetters(@NonNull TypeDescriptor<?> targetTypeDescriptor,
                                                                   @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Delegates to the GetterBasedSchemaProvider.fieldValueGetters(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.

Specified by:

fieldValueGetters in class GetterBasedSchemaProviderV2

fieldValueTypeInformations

public @NonNull java.util.List<FieldValueTypeInformation> fieldValueTypeInformations(@NonNull TypeDescriptor<?> targetTypeDescriptor,
                                                                                     @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Delegates to the GetterBasedSchemaProvider.fieldValueTypeInformations(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.

Specified by:

fieldValueTypeInformations in class GetterBasedSchemaProviderV2

schemaTypeCreator

public @NonNull SchemaUserTypeCreator schemaTypeCreator(@NonNull TypeDescriptor<?> targetTypeDescriptor,
                                                        @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Delegates to the GetterBasedSchemaProvider.schemaTypeCreator(Class, Schema) for backwards compatibility, override it if you want to use the richer type signature contained in the TypeDescriptor not subject to the type erasure.

Specified by:

schemaTypeCreator in class GetterBasedSchemaProviderV2