Package org.apache.beam.sdk.io

Class TFRecordWriteSchemaTransformConfiguration

java.lang.Object

org.apache.beam.sdk.io.TFRecordWriteSchemaTransformConfiguration

@DefaultSchema(AutoValueSchema.class)
public abstract class TFRecordWriteSchemaTransformConfiguration
extends Object

Configuration for reading from TFRecord.

This class is meant to be used with TFRecordWriteSchemaTransformProvider.

Internal only: This class is actively being worked on, and it will likely change. We provide no backwards compatibility guarantees, and it should not be implemented outside the Beam repository.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TFRecordWriteSchemaTransformConfiguration.Builder

Constructor Summary

Constructors

Constructor	Description
TFRecordWriteSchemaTransformConfiguration()

Method Summary

Modifier and Type	Method	Description
static TFRecordWriteSchemaTransformConfiguration.Builder	builder()	Instantiates a TFRecordWriteSchemaTransformConfiguration.Builder instance.
abstract String	getCompression()
abstract @Nullable ErrorHandling	getErrorHandling()
abstract @Nullable String	getFilenameSuffix()
abstract @Nullable Integer	getMaxNumWritersPerBundle()
abstract @Nullable Boolean	getNoSpilling()
abstract int	getNumShards()
abstract String	getOutputPrefix()
abstract @Nullable String	getShardTemplate()
void	validate()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TFRecordWriteSchemaTransformConfiguration

public TFRecordWriteSchemaTransformConfiguration()

Method Details

validate

public void validate()

builder

public static TFRecordWriteSchemaTransformConfiguration.Builder builder()

Instantiates a TFRecordWriteSchemaTransformConfiguration.Builder instance.

getOutputPrefix

@SchemaFieldDescription("The directory to which files will be written.")
public abstract String getOutputPrefix()

getFilenameSuffix

@SchemaFieldDescription("The suffix of each file written, combined with prefix and shardTemplate.")
public abstract @Nullable String getFilenameSuffix()

getNumShards

@SchemaFieldDescription("The number of shards to use, or 0 for automatic.")
public abstract int getNumShards()

getShardTemplate

@SchemaFieldDescription("The shard template of each file written, combined with prefix and suffix.")
public abstract @Nullable String getShardTemplate()

getCompression

@SchemaFieldDescription("Option to indicate the output sink\'s compression type. Default is NONE.")
public abstract String getCompression()

getNoSpilling

@SchemaFieldDescription("Whether to skip the spilling of data caused by having maxNumWritersPerBundle.")
public abstract @Nullable Boolean getNoSpilling()

getMaxNumWritersPerBundle

@SchemaFieldDescription("Maximum number of writers created in a bundle before spilling to shuffle.")
public abstract @Nullable Integer getMaxNumWritersPerBundle()

getErrorHandling

@SchemaFieldDescription("This option specifies whether and where to output unwritable rows.")
public abstract @Nullable ErrorHandling getErrorHandling()