org.apache.beam.sdk.extensions.avro.io

Class DynamicAvroDestinations<UserT,DestinationT,OutputT>

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations<UserT,DestinationT,OutputT>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

public abstract class DynamicAvroDestinations<UserT,DestinationT,OutputT>
extends FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

A specialization of FileBasedSink.DynamicDestinations for AvroIO. In addition to dynamic file destinations, this allows specifying other AVRO properties (schema, metadata, codec, datum writer) per destination.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicAvroDestinations()

Method Summary

Modifier and Type	Method and Description
CodecFactory	getCodec(DestinationT destination) Return an AVRO codec for a given destination.
AvroSink.DatumWriterFactory<OutputT>	getDatumWriterFactory(DestinationT destinationT) Return a AvroSink.DatumWriterFactory for a given destination.
java.util.Map<java.lang.String,java.lang.Object>	getMetadata(DestinationT destination) Return AVRO file metadata for a given destination.
abstract Schema	getSchema(DestinationT destination) Return an AVRO schema for a given destination.

Methods inherited from class org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations

formatRecord, getDefaultDestination, getDestination, getDestinationCoder, getFilenamePolicy, getSideInputs, populateDisplayData, sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicAvroDestinations

public DynamicAvroDestinations()

Method Detail

getSchema

public abstract Schema getSchema(DestinationT destination)

Return an AVRO schema for a given destination.

getMetadata

public java.util.Map<java.lang.String,java.lang.Object> getMetadata(DestinationT destination)

Return AVRO file metadata for a given destination.

getCodec

public CodecFactory getCodec(DestinationT destination)

Return an AVRO codec for a given destination.

getDatumWriterFactory

public AvroSink.DatumWriterFactory<OutputT> getDatumWriterFactory(DestinationT destinationT)

Return a AvroSink.DatumWriterFactory for a given destination. If provided, it will be used to created DatumWriter instances as required.