org.apache.beam.sdk.io.thrift

Class ThriftSchema

java.lang.Object

org.apache.beam.sdk.schemas.GetterBasedSchemaProvider

org.apache.beam.sdk.io.thrift.ThriftSchema

All Implemented Interfaces:

java.io.Serializable, SchemaProvider

@Experimental(value=SCHEMAS)
public final class ThriftSchema
extends GetterBasedSchemaProvider

Schema provider for generated thrift types.

• Primitive type mapping is straight-forward (e.g. TType.I32 -> Schema.FieldType.INT32).
• TType.STRING gets mapped as either Schema.FieldType.STRING or Schema.FieldType.BYTES, depending on whether the FieldValueMetaData.isBinary() flag is set.
• TType.MAP becomes a beam map passing the key and value types recursively.
• TType.SET gets translated into a beam iterable, passing the corresponding element type.
• TType.LIST becomes an array of the corresponding element type.
• thrift enums are converted into beam enumeration types.
• thrift union types get mapped to beam one-of types.

The mapping logic relies on the available thrift metadata introspection and tries to make as few assumptions about the generated code as possible (i.e. does not rely on accessor naming convention, as the thrift compiler supports options such as "beans" or "fullcamel"/"nocamel".
However, the following strong assumptions are made by this class:

• All thrift generated classes implement TBase, except for enums which become java enums implementing TEnum.
• All TUnion types provide static factory methods for each of the supported field types, with the same name as the field itself and only one such method taking a single parameter exists.
• All non-union types have a corresponding java field with the same name for every field in the original thrift source file.

Thrift typedefs for container types (and possibly others) do not preserve the full type information. For this reason, this class allows for manual registration of such "lossy" typedefs with their corresponding beam types.

Note: Thrift encoding and decoding are not fully symmetrical, i.e. the isSet flag may not be preserved upon converting a thrift object to a beam row and back. On encoding, we extract all thrift values, no matter if the fields are set or not. On decoding, we set all non-null beam row values to the corresponding thrift fields, leaving the rest unset.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ThriftSchema.Customizer

Method Summary

Modifier and Type	Method and Description
static @NonNull ThriftSchema.Customizer	custom() Builds a schema provider that maps any thrift type to a Beam schema, allowing for custom thrift typedef entries (which cannot be resolved using the available metadata) to be manually registered with their corresponding beam types.
@NonNull java.util.List<FieldValueGetter>	fieldValueGetters(@NonNull java.lang.Class<?> targetClass, @NonNull Schema schema) Implementing class should override to return FieldValueGetters.
@NonNull java.util.List<FieldValueTypeInformation>	fieldValueTypeInformations(@NonNull java.lang.Class<?> targetClass, @NonNull Schema schema) Implementing class should override to return a list of type-informations.
static @NonNull SchemaProvider	provider() Schema provider that maps any thrift type to a Beam schema, assuming that any typedefs that might have been used in the thrift definitions will preserve all required metadata to infer the beam type (which is the case for any primitive typedefs and alike).
<T> @NonNull Schema	schemaFor(TypeDescriptor<T> typeDescriptor) Lookup a schema for the given type.
@NonNull SchemaUserTypeCreator	schemaTypeCreator(@NonNull java.lang.Class<?> targetClass, @NonNull Schema schema) Implementing class should override to return a constructor.

Methods inherited from class org.apache.beam.sdk.schemas.GetterBasedSchemaProvider

equals, fromRowFunction, hashCode, toRowFunction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Method Detail

provider

public static @NonNull SchemaProvider provider()

Schema provider that maps any thrift type to a Beam schema, assuming that any typedefs that might have been used in the thrift definitions will preserve all required metadata to infer the beam type (which is the case for any primitive typedefs and alike).

See Also:

for how to manually pass the beam type for container typedefs

custom

public static @NonNull ThriftSchema.Customizer custom()

Builds a schema provider that maps any thrift type to a Beam schema, allowing for custom thrift typedef entries (which cannot be resolved using the available metadata) to be manually registered with their corresponding beam types.

E.g. typedef set<string> StringSet will not carry the element type information and needs to be manually mapped here as .custom().withTypedef("StringSet", FieldType.iterable(FieldType.STRING)).provider().

schemaFor

public <T> @NonNull Schema schemaFor(TypeDescriptor<T> typeDescriptor)

Description copied from interface: SchemaProvider

Lookup a schema for the given type. If no schema exists, returns null.

fieldValueGetters

public @NonNull java.util.List<FieldValueGetter> fieldValueGetters(@NonNull java.lang.Class<?> targetClass,
                                                                   @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Implementing class should override to return FieldValueGetters.

Specified by:

fieldValueGetters in class GetterBasedSchemaProvider

fieldValueTypeInformations

public @NonNull java.util.List<FieldValueTypeInformation> fieldValueTypeInformations(@NonNull java.lang.Class<?> targetClass,
                                                                                     @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Implementing class should override to return a list of type-informations.

Specified by:

fieldValueTypeInformations in class GetterBasedSchemaProvider

schemaTypeCreator

public @NonNull SchemaUserTypeCreator schemaTypeCreator(@NonNull java.lang.Class<?> targetClass,
                                                        @NonNull Schema schema)

Description copied from class: GetterBasedSchemaProvider

Implementing class should override to return a constructor.

Specified by:

schemaTypeCreator in class GetterBasedSchemaProvider