org.apache.beam.sdk.io

Class TFRecordWriteSchemaTransformConfiguration

java.lang.Object

org.apache.beam.sdk.io.TFRecordWriteSchemaTransformConfiguration

@DefaultSchema(value=AutoValueSchema.class)
public abstract class TFRecordWriteSchemaTransformConfiguration
extends java.lang.Object

Configuration for reading from TFRecord.

This class is meant to be used with TFRecordWriteSchemaTransformProvider.

Internal only: This class is actively being worked on, and it will likely change. We provide no backwards compatibility guarantees, and it should not be implemented outside the Beam repository.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TFRecordWriteSchemaTransformConfiguration.Builder

Constructor Summary

Constructors

Constructor and Description
TFRecordWriteSchemaTransformConfiguration()

Method Summary

Modifier and Type	Method and Description
static TFRecordWriteSchemaTransformConfiguration.Builder	builder() Instantiates a TFRecordWriteSchemaTransformConfiguration.Builder instance.
abstract java.lang.String	getCompression()
abstract @Nullable ErrorHandling	getErrorHandling()
abstract @Nullable java.lang.String	getFilenameSuffix()
abstract @Nullable java.lang.Integer	getMaxNumWritersPerBundle()
abstract @Nullable java.lang.Boolean	getNoSpilling()
abstract int	getNumShards()
abstract java.lang.String	getOutputPrefix()
abstract @Nullable java.lang.String	getShardTemplate()
void	validate()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TFRecordWriteSchemaTransformConfiguration

public TFRecordWriteSchemaTransformConfiguration()

Method Detail

validate

public void validate()

builder

public static TFRecordWriteSchemaTransformConfiguration.Builder builder()

Instantiates a TFRecordWriteSchemaTransformConfiguration.Builder instance.

getOutputPrefix

@SchemaFieldDescription(value="The directory to which files will be written.")
public abstract java.lang.String getOutputPrefix()

getFilenameSuffix

@SchemaFieldDescription(value="The suffix of each file written, combined with prefix and shardTemplate.")
public abstract @Nullable java.lang.String getFilenameSuffix()

getNumShards

@SchemaFieldDescription(value="The number of shards to use, or 0 for automatic.")
public abstract int getNumShards()

getShardTemplate

@SchemaFieldDescription(value="The shard template of each file written, combined with prefix and suffix.")
public abstract @Nullable java.lang.String getShardTemplate()

getCompression

@SchemaFieldDescription(value="Option to indicate the output sink\'s compression type. Default is NONE.")
public abstract java.lang.String getCompression()

getNoSpilling

@SchemaFieldDescription(value="Whether to skip the spilling of data caused by having maxNumWritersPerBundle.")
public abstract @Nullable java.lang.Boolean getNoSpilling()

getMaxNumWritersPerBundle

@SchemaFieldDescription(value="Maximum number of writers created in a bundle before spilling to shuffle.")
public abstract @Nullable java.lang.Integer getMaxNumWritersPerBundle()

getErrorHandling

@SchemaFieldDescription(value="This option specifies whether and where to output unwritable rows.")
public abstract @Nullable ErrorHandling getErrorHandling()