org.apache.beam.sdk.io

Class TFRecordIO

java.lang.Object

org.apache.beam.sdk.io.TFRecordIO

public class TFRecordIO
extends java.lang.Object

PTransforms for reading and writing TensorFlow TFRecord files.

For reading files, use read().

For simple cases of writing files, use write(). For more complex cases (such as ability to write windowed data or writing to multiple destinations) use sink() in combination with FileIO.write() or FileIO.writeDynamic().

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TFRecordIO.CompressionType Deprecated. Use Compression.
static class	TFRecordIO.Read Implementation of read().
static class	TFRecordIO.Sink A FileIO.Sink for use with FileIO.write() and FileIO.writeDynamic().
static class	TFRecordIO.Write Implementation of write().

Field Summary

Fields

Modifier and Type	Field and Description
static Coder<byte[]>	DEFAULT_BYTE_ARRAY_CODER The default coder, which returns each record of the input file as a byte array.

Method Summary

Modifier and Type	Method and Description
static TFRecordIO.Read	read() A PTransform that reads from a TFRecord file (or multiple TFRecord files matching a pattern) and returns a PCollection containing the decoding of each of the records of the TFRecord file(s) as a byte array.
static TFRecordIO.Sink	sink() Returns a FileIO.Sink for use with FileIO.write() and FileIO.writeDynamic().
static TFRecordIO.Write	write() A PTransform that writes a PCollection to TFRecord file (or multiple TFRecord files matching a sharding pattern), with each element of the input collection encoded into its own record.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_BYTE_ARRAY_CODER

public static final Coder<byte[]> DEFAULT_BYTE_ARRAY_CODER

The default coder, which returns each record of the input file as a byte array.

Method Detail

read

public static TFRecordIO.Read read()

A PTransform that reads from a TFRecord file (or multiple TFRecord files matching a pattern) and returns a PCollection containing the decoding of each of the records of the TFRecord file(s) as a byte array.

write

public static TFRecordIO.Write write()

A PTransform that writes a PCollection to TFRecord file (or multiple TFRecord files matching a sharding pattern), with each element of the input collection encoded into its own record.

sink

public static TFRecordIO.Sink sink()

Returns a FileIO.Sink for use with FileIO.write() and FileIO.writeDynamic().