ThriftIO (Apache Beam 2.54.0)

java.lang.Object
- org.apache.beam.sdk.io.thrift.ThriftIO

public class ThriftIO
extends java.lang.Object

PTransforms for reading and writing files containing Thrift encoded data.

Reading Thrift Files

For reading each file in a PCollection of FileIO.ReadableFile, use the readFiles(Class) transform.

For example:


 PCollection<FileIO.ReadableFile> files = pipeline
   .apply(FileIO.match().filepattern(options.getInputFilepattern())
   .apply(FileIO.readMatches());

 PCollection<ExampleType> examples = files.apply(ThriftIO.readFiles(ExampleType.class).withProtocol(thriftProto);

Writing Thrift Files

ThriftIO.Sink allows for a PCollection of TBase to be written to Thrift files. It can be used with the general-purpose FileIO transforms with FileIO.write/writeDynamic specifically.

For example:


 pipeline
   .apply(...) // PCollection<ExampleType>
   .apply(FileIO
     .<ExampleType>write()
     .via(ThriftIO.sink(thriftProto))
     .to("destination/path");

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ThriftIO.ReadFiles<T>` Implementation of `readFiles(java.lang.Class<T>)`.
`static class`	`ThriftIO.Sink<T extends org.apache.thrift.TBase<?,?>>` Implementation of `sink(org.apache.thrift.protocol.TProtocolFactory)`.
`protected static class`	`ThriftIO.ThriftWriter<T extends org.apache.thrift.TBase<?,?>>` Writer to write Thrift object to `OutputStream`.

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static <T> ThriftIO.ReadFiles<T>`	`readFiles(java.lang.Class<T> recordClass)` Reads each file in a `PCollection` of `FileIO.ReadableFile`, which allows more flexible usage.
`static <T extends org.apache.thrift.TBase<?,?>> ThriftIO.Sink<T>`	`sink(org.apache.thrift.protocol.TProtocolFactory factory)` Creates a `ThriftIO.Sink` for use with `FileIO.write()` and `FileIO.writeDynamic()`.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

readFiles
```
public static <T> ThriftIO.ReadFiles<T> readFiles(java.lang.Class<T> recordClass)
```
Reads each file in a PCollection of FileIO.ReadableFile, which allows more flexible usage.

sink

public static <T extends org.apache.thrift.TBase<?,?>> ThriftIO.Sink<T> sink(org.apache.thrift.protocol.TProtocolFactory factory)

Creates a ThriftIO.Sink for use with FileIO.write() and FileIO.writeDynamic().

Class ThriftIO