org.apache.beam.sdk.io

Interface Summary
Interface	Description
CompressedSource.DecompressingChannelFactory	Factory interface for creating channels that decompress the content of an underlying channel.
FileBasedSink.WritableByteChannelFactory	Implementations create instances of `WritableByteChannel` used by `FileBasedSink` and related classes to allow decorating, or otherwise transforming, the raw data that would normally be written directly to the `WritableByteChannel` passed into `FileBasedSink.WritableByteChannelFactory.create(WritableByteChannel)`.
FileSystemRegistrar	A registrar that creates `FileSystem` instances from `PipelineOptions`.
UnboundedSource.CheckpointMark	A marker representing the progress and state of an `UnboundedSource.UnboundedReader`.

Class Summary
Class	Description
AvroIO	`PTransform`s for reading and writing Avro files.
AvroIO.Read<T>	Implementation of `AvroIO.read(java.lang.Class<T>)`.
AvroIO.Write<T>	Implementation of `AvroIO.write(java.lang.Class<T>)`.
AvroSource<T>	Do not use in pipelines directly: most users should use `AvroIO.Read`.
AvroSource.AvroReader<T>	A `BlockBasedSource.BlockBasedReader` for reading blocks from Avro files.
BlockBasedSource<T>	A `BlockBasedSource` is a `FileBasedSource` where a file consists of blocks of records.
BlockBasedSource.Block<T>	A `Block` represents a block of records that can be read.
BlockBasedSource.BlockBasedReader<T>	A `Reader` that reads records from a `BlockBasedSource`.
BoundedReadFromUnboundedSource<T>	`PTransform` that reads a bounded amount of data from an `UnboundedSource`, specified as one or both of a maximum number of elements or a maximum period of time to read.
BoundedSource<T>	A `Source` that reads a finite amount of input and, because of that, supports some additional operations.
BoundedSource.BoundedReader<T>	A `Reader` that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.
CompressedSource<T>	A Source that reads from compressed files.
CompressedSource.CompressedReader<T>	Reader for a `CompressedSource`.
CountingSource	Most users should use `GenerateSequence` instead.
CountingSource.CounterMark	The checkpoint for an unbounded `CountingSource` is simply the last value produced.
DefaultFilenamePolicy	A default `FileBasedSink.FilenamePolicy` for windowed and unwindowed files.
FileBasedSink<T>	Abstract class for file-based output.
FileBasedSink.FilenamePolicy	A naming policy for output files.
FileBasedSink.FilenamePolicy.Context	Context used for generating a name based on shard number, and num shards.
FileBasedSink.FilenamePolicy.WindowedContext	Context used for generating a name based on window, pane, shard number, and num shards.
FileBasedSink.FileResult	Result of a single bundle write.
FileBasedSink.FileResultCoder	A coder for `FileBasedSink.FileResult` objects.
FileBasedSink.WriteOperation<T>	Abstract operation that manages the process of writing to `FileBasedSink`.
FileBasedSink.Writer<T>	Abstract writer that writes a bundle to a `FileBasedSink`.
FileBasedSource<T>	A common base class for all file-based `Source`s.
FileBasedSource.FileBasedReader<T>	A `reader` that implements code common to readers of `FileBasedSource`s.
FileSystem<ResourceIdT extends ResourceId>	File system interface in Beam.
FileSystems	Clients facing `FileSystem` utility.
GenerateSequence	A `PTransform` that produces longs starting from the given value, and either up to the given limit or until `Long.MAX_VALUE` / until the given time elapses.
LocalFileSystemRegistrar	`AutoService` registrar for the `LocalFileSystem`.
LocalResources	Helper functions for producing a `ResourceId` that references a local file or directory.
OffsetBasedSource<T>	A `BoundedSource` that uses offsets to define starting and ending positions.
OffsetBasedSource.OffsetBasedReader<T>	A `Source.Reader` that implements code common to readers of all `OffsetBasedSource`s.
Read	A `PTransform` for reading from a `Source`.
Read.Bounded<T>	`PTransform` that reads from a `BoundedSource`.
Read.Builder	Helper class for building `Read` transforms.
Read.Unbounded<T>	`PTransform` that reads from a `UnboundedSource`.
ShardNameTemplate	Standard shard naming templates.
Source<T>	Base class for defining input formats and creating a `Source` for reading the input.
Source.Reader<T>	The interface that readers of custom input sources must implement.
TextIO	`PTransform`s for reading and writing text files.
TextIO.Read	Implementation of `TextIO.read()`.
TextIO.Write	Implementation of `TextIO.write()`.
TFRecordIO	`PTransform`s for reading and writing TensorFlow TFRecord files.
TFRecordIO.Read	Implementation of `TFRecordIO.read()`.
TFRecordIO.Write	Implementation of `TFRecordIO.write()`.
UnboundedSource<OutputT,CheckpointMarkT extends UnboundedSource.CheckpointMark>	A `Source` that reads an unbounded amount of input and, because of that, supports some additional operations such as checkpointing, watermarks, and record ids.
UnboundedSource.UnboundedReader<OutputT>	A `Reader` that reads an unbounded amount of input.
WriteFiles<T>	A `PTransform` that writes to a `FileBasedSink`.

Enum Summary
Enum	Description
CompressedSource.CompressionMode	Default compression types supported by the `CompressedSource`.
FileBasedSink.CompressionType	Directly supported file output compression types.
FileBasedSource.Mode	A given `FileBasedSource` represents a file resource of one of these types.
TextIO.CompressionType	Possible text file compression types.
TFRecordIO.CompressionType	Possible TFRecord file compression types.

Package org.apache.beam.sdk.io Description

Defines transforms for reading and writing common storage formats, including AvroIO, and TextIO.

The classes in this package provide Read transforms that create PCollections from existing storage:


 PCollection<TableRow> inputData = pipeline.apply(
     BigQueryIO.read().from("clouddataflow-readonly:samples.weather_stations"));

and Write transforms that persist PCollections to external storage:

 
 PCollection<Integer> numbers = ...;
 numbers.apply(TextIO.write().to("gs://my_bucket/path/to/numbers"));