org.apache.beam.sdk.schemas

Class SchemaRegistry

java.lang.Object

org.apache.beam.sdk.schemas.SchemaRegistry

public class SchemaRegistry
extends java.lang.Object

A SchemaRegistry allows registering Schemas for a given Java Class or a TypeDescriptor.

Types registered in a pipeline's schema registry will automatically be discovered by any PCollection that uses SchemaCoder. This allows users to write pipelines in terms of their own Java types, yet still register schemas for these types.

TODO: Provide support for schemas registered via a ServiceLoader interface. This will allow optional modules to register schemas as well.

Method Summary

Modifier and Type	Method and Description
static SchemaRegistry	createDefault()
<T> SerializableFunction<Row,T>	getFromRowFunction(java.lang.Class<T> clazz) Retrieve the function that converts a Row object to the specified type.
<T> SerializableFunction<Row,T>	getFromRowFunction(TypeDescriptor<T> typeDescriptor) Retrieve the function that converts a Row object to the specified type.
<T> Schema	getSchema(java.lang.Class<T> clazz) Retrieve a Schema for a given Class type.
<T> Schema	getSchema(TypeDescriptor<T> typeDescriptor) Retrieve a Schema for a given TypeDescriptor type.
<T> SchemaCoder<T>	getSchemaCoder(java.lang.Class<T> clazz) Retrieve a SchemaCoder for a given Class type.
<T> SchemaCoder<T>	getSchemaCoder(TypeDescriptor<T> typeDescriptor) Retrieve a SchemaCoder for a given TypeDescriptor type.
<T> SchemaProvider	getSchemaProvider(java.lang.Class<T> clazz) Retrieve a registered SchemaProvider for a given Class.
<T> SchemaProvider	getSchemaProvider(TypeDescriptor<T> typeDescriptor) Retrieve a registered SchemaProvider for a given TypeDescriptor.
<T> SerializableFunction<T,Row>	getToRowFunction(java.lang.Class<T> clazz) Retrieve the function that converts an object of the specified type to a Row object.
<T> SerializableFunction<T,Row>	getToRowFunction(TypeDescriptor<T> typeDescriptor) Retrieve the function that converts an object of the specified type to a Row object.
<T> void	registerJavaBean(java.lang.Class<T> clazz) Register a JavaBean type for automatic schema inference.
<T> void	registerJavaBean(TypeDescriptor<T> typeDescriptor) Register a JavaBean type for automatic schema inference.
<T> void	registerPOJO(java.lang.Class<T> clazz) Register a POJO type for automatic schema inference.
<T> void	registerPOJO(TypeDescriptor<T> typeDescriptor) Register a POJO type for automatic schema inference.
<T> void	registerSchemaForClass(java.lang.Class<T> clazz, Schema schema, SerializableFunction<T,Row> toRow, SerializableFunction<Row,T> fromRow) Register a schema for a specific Class type.
<T> void	registerSchemaForType(TypeDescriptor<T> type, Schema schema, SerializableFunction<T,Row> toRow, SerializableFunction<Row,T> fromRow) Register a schema for a specific TypeDescriptor type.
<T> void	registerSchemaProvider(java.lang.Class<T> clazz, SchemaProvider schemaProvider) Register a SchemaProvider to be used for a specific type.
void	registerSchemaProvider(SchemaProvider schemaProvider) Register a SchemaProvider.
<T> void	registerSchemaProvider(TypeDescriptor<T> typeDescriptor, SchemaProvider schemaProvider) Register a SchemaProvider to be used for a specific type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createDefault

public static SchemaRegistry createDefault()

registerSchemaForClass

public <T> void registerSchemaForClass(java.lang.Class<T> clazz,
                                       Schema schema,
                                       SerializableFunction<T,Row> toRow,
                                       SerializableFunction<Row,T> fromRow)

Register a schema for a specific Class type.

registerSchemaForType

public <T> void registerSchemaForType(TypeDescriptor<T> type,
                                      Schema schema,
                                      SerializableFunction<T,Row> toRow,
                                      SerializableFunction<Row,T> fromRow)

Register a schema for a specific TypeDescriptor type.

registerSchemaProvider

public void registerSchemaProvider(SchemaProvider schemaProvider)

Register a SchemaProvider.

A SchemaProvider allows for deferred lookups of per-type schemas. This can be used when schemas are registered in an external service. The SchemaProvider will lookup the type in the external service and return the correct Schema.

registerSchemaProvider

public <T> void registerSchemaProvider(java.lang.Class<T> clazz,
                                       SchemaProvider schemaProvider)

Register a SchemaProvider to be used for a specific type. *

registerSchemaProvider

public <T> void registerSchemaProvider(TypeDescriptor<T> typeDescriptor,
                                       SchemaProvider schemaProvider)

Register a SchemaProvider to be used for a specific type. *

registerPOJO

public <T> void registerPOJO(java.lang.Class<T> clazz)

Register a POJO type for automatic schema inference.

Currently schema field names will match field names in the POJO, and all fields must be mutable (i.e. no final fields).

registerPOJO

public <T> void registerPOJO(TypeDescriptor<T> typeDescriptor)

Register a POJO type for automatic schema inference.

Currently schema field names will match field names in the POJO, and all fields must be mutable (i.e. no final fields). The Java object is expected to have implemented a correct .equals() and .hashCode methods The equals method must be completely determined by the schema fields. i.e. if the object has hidden fields that are not reflected in the schema but are compared in equals, then results will be incorrect.

registerJavaBean

public <T> void registerJavaBean(java.lang.Class<T> clazz)

Register a JavaBean type for automatic schema inference.

Currently schema field names will match getter names in the bean, and all getters must have matching setters. The Java object is expected to have implemented a correct .equals() and .hashCode methods The equals method must be completely determined by the schema fields. i.e. if the object has hidden fields that are not reflected in the schema but are compared in equals, then results will be incorrect.

registerJavaBean

public <T> void registerJavaBean(TypeDescriptor<T> typeDescriptor)

Register a JavaBean type for automatic schema inference.

Currently schema field names will match getter names in the bean, and all getters must have matching setters.

getSchema

public <T> Schema getSchema(java.lang.Class<T> clazz)
                     throws NoSuchSchemaException

Retrieve a Schema for a given Class type. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException

getSchema

public <T> Schema getSchema(TypeDescriptor<T> typeDescriptor)
                     throws NoSuchSchemaException

Retrieve a Schema for a given TypeDescriptor type. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException

getToRowFunction

public <T> SerializableFunction<T,Row> getToRowFunction(java.lang.Class<T> clazz)
                                                 throws NoSuchSchemaException

Retrieve the function that converts an object of the specified type to a Row object.

Throws:

NoSuchSchemaException

getToRowFunction

public <T> SerializableFunction<T,Row> getToRowFunction(TypeDescriptor<T> typeDescriptor)
                                                 throws NoSuchSchemaException

Retrieve the function that converts an object of the specified type to a Row object.

Throws:

NoSuchSchemaException

getFromRowFunction

public <T> SerializableFunction<Row,T> getFromRowFunction(java.lang.Class<T> clazz)
                                                   throws NoSuchSchemaException

Retrieve the function that converts a Row object to the specified type.

Throws:

NoSuchSchemaException

getFromRowFunction

public <T> SerializableFunction<Row,T> getFromRowFunction(TypeDescriptor<T> typeDescriptor)
                                                   throws NoSuchSchemaException

Retrieve the function that converts a Row object to the specified type.

Throws:

NoSuchSchemaException

getSchemaCoder

public <T> SchemaCoder<T> getSchemaCoder(java.lang.Class<T> clazz)
                                  throws NoSuchSchemaException

Retrieve a SchemaCoder for a given Class type. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException

getSchemaCoder

public <T> SchemaCoder<T> getSchemaCoder(TypeDescriptor<T> typeDescriptor)
                                  throws NoSuchSchemaException

Retrieve a SchemaCoder for a given TypeDescriptor type. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException

getSchemaProvider

public <T> SchemaProvider getSchemaProvider(TypeDescriptor<T> typeDescriptor)
                                     throws NoSuchSchemaException

Retrieve a registered SchemaProvider for a given TypeDescriptor. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException

getSchemaProvider

public <T> SchemaProvider getSchemaProvider(java.lang.Class<T> clazz)
                                     throws NoSuchSchemaException

Retrieve a registered SchemaProvider for a given Class. If no schema exists, throws NoSuchSchemaException.

Throws:

NoSuchSchemaException