org.apache.beam.sdk.schemas.transforms

Class TypedSchemaTransformProvider<ConfigT>

java.lang.Object

org.apache.beam.sdk.schemas.transforms.TypedSchemaTransformProvider<ConfigT>

All Implemented Interfaces:

SchemaTransformProvider

Direct Known Subclasses:

BigQueryDirectReadSchemaTransformProvider, BigQueryExportReadSchemaTransformProvider, BigQueryFileLoadsWriteSchemaTransformProvider, BigQueryStorageWriteApiSchemaTransformProvider, BigtableReadSchemaTransformProvider, BigtableWriteSchemaTransformProvider, CsvWriteTransformProvider, DebeziumReadSchemaTransformProvider, FileReadSchemaTransformProvider, FileWriteSchemaTransformProvider, FlattenTransformProvider, GenerateSequenceSchemaTransformProvider, IcebergReadSchemaTransformProvider, IcebergWriteSchemaTransformProvider, JavaExplodeTransformProvider, JavaFilterTransformProvider, JavaMapToFieldsTransformProvider, JdbcReadSchemaTransformProvider, JdbcWriteSchemaTransformProvider, JsonWriteTransformProvider, KafkaReadSchemaTransformProvider, KafkaWriteSchemaTransformProvider, LoggingTransformProvider, ManagedSchemaTransformProvider, PubsubLiteReadSchemaTransformProvider, PubsubLiteWriteSchemaTransformProvider, PubsubReadSchemaTransformProvider, PubsubWriteSchemaTransformProvider, SingleStoreSchemaTransformReadProvider, SingleStoreSchemaTransformWriteProvider, SpannerChangestreamsReadSchemaTransformProvider, SpannerWriteSchemaTransformProvider, TestSchemaTransformProvider, WindowIntoTransformProvider

@Internal
public abstract class TypedSchemaTransformProvider<ConfigT>
extends java.lang.Object
implements SchemaTransformProvider

Like SchemaTransformProvider except uses a configuration object instead of Schema and Row.

ConfigT should be available in the SchemaRegistry.

configurationSchema() produces a configuration Schema that is inferred from ConfigT using the SchemaRegistry. A Beam Row can still be used to produce a SchemaTransform using from(Row), as long as the Row fits the configuration Schema.

NOTE: The inferred field names in the configuration Schema and Row follow the snake_case naming convention.

Internal only: This interface is actively being worked on and it will likely change as we provide implementations for more standard Beam transforms. We provide no backwards compatibility guarantees and it should not be implemented outside of the Beam repository.

Constructor Summary

Constructors

Constructor and Description
TypedSchemaTransformProvider()

Method Summary

Modifier and Type	Method and Description
protected java.lang.Class<ConfigT>	configurationClass()
Schema	configurationSchema() Returns the expected schema of the configuration object.
java.util.Optional<java.util.List<java.lang.String>>	dependencies(Row configuration, PipelineOptions options) List the dependencies needed for this transform.
protected abstract SchemaTransform	from(ConfigT configuration) Produce a SchemaTransform from ConfigT.
SchemaTransform	from(Row configuration) Produces a SchemaTransform from a Row configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.schemas.transforms.SchemaTransformProvider

description, identifier, inputCollectionNames, outputCollectionNames

Constructor Detail

TypedSchemaTransformProvider

public TypedSchemaTransformProvider()

Method Detail

configurationClass

protected java.lang.Class<ConfigT> configurationClass()

from

protected abstract SchemaTransform from(ConfigT configuration)

Produce a SchemaTransform from ConfigT. Can throw a InvalidConfigurationException or a InvalidSchemaException.

configurationSchema

public final Schema configurationSchema()

Description copied from interface: SchemaTransformProvider

Returns the expected schema of the configuration object. Note this is distinct from the schema of the transform itself.

Specified by:

configurationSchema in interface SchemaTransformProvider

from

public final SchemaTransform from(Row configuration)

Produces a SchemaTransform from a Row configuration. Row fields are expected to have `snake_case` naming convention.

Specified by:

from in interface SchemaTransformProvider

dependencies

public final java.util.Optional<java.util.List<java.lang.String>> dependencies(Row configuration,
                                                                               PipelineOptions options)

Description copied from interface: SchemaTransformProvider

List the dependencies needed for this transform. Jars from classpath are used by default when Optional.empty() is returned.

Specified by:

dependencies in interface SchemaTransformProvider