Package org.apache.beam.sdk.extensions.avro.io

Class DynamicAvroDestinations<UserT,DestinationT,OutputT>

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations<UserT,DestinationT,OutputT>

All Implemented Interfaces:

Serializable, HasDisplayData

public abstract class DynamicAvroDestinations<UserT,DestinationT,OutputT>
extends FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

A specialization of FileBasedSink.DynamicDestinations for AvroIO. In addition to dynamic file destinations, this allows specifying other AVRO properties (schema, metadata, codec, datum writer) per destination.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
DynamicAvroDestinations()

Method Summary

Modifier and Type	Method	Description
CodecFactory	getCodec(DestinationT destination)	Return an AVRO codec for a given destination.
@Nullable AvroSink.DatumWriterFactory<OutputT>	getDatumWriterFactory(DestinationT destinationT)	Return a AvroSink.DatumWriterFactory for a given destination.
Map<String,Object>	getMetadata(DestinationT destination)	Return AVRO file metadata for a given destination.
abstract Schema	getSchema(DestinationT destination)	Return an AVRO schema for a given destination.

Methods inherited from class org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations

formatRecord, getDefaultDestination, getDestination, getDestinationCoder, getFilenamePolicy, getSideInputs, populateDisplayData, sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DynamicAvroDestinations

public DynamicAvroDestinations()

Method Details

getSchema

public abstract Schema getSchema(DestinationT destination)

Return an AVRO schema for a given destination.

getMetadata

public Map<String,Object> getMetadata(DestinationT destination)

Return AVRO file metadata for a given destination.

getCodec

public CodecFactory getCodec(DestinationT destination)

Return an AVRO codec for a given destination.

getDatumWriterFactory

public @Nullable AvroSink.DatumWriterFactory<OutputT> getDatumWriterFactory(DestinationT destinationT)

Return a AvroSink.DatumWriterFactory for a given destination. If provided, it will be used to created DatumWriter instances as required.