org.apache.beam.sdk.fn.stream

Class DataStreams

java.lang.Object

org.apache.beam.sdk.fn.stream.DataStreams

public class DataStreams
extends java.lang.Object

DataStreams.DataStreamDecoder treats multiple ByteStrings as a single input stream decoding values with the supplied iterator. outbound(OutputChunkConsumer) treats a single OutputStream as multiple ByteStrings.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DataStreams.BlockingQueueIterator<T> Allows for one or more writing threads to append values to this iterator while one reading thread reads values.
static class	DataStreams.DataStreamDecoder<T> An adapter which converts an InputStream to a PrefetchableIterator of T values using the specified Coder.
static class	DataStreams.ElementDelimitedOutputStream An adapter which wraps an DataStreams.OutputChunkConsumer as an OutputStream.
static interface	DataStreams.OutputChunkConsumer<T> A callback which is invoked whenever the outbound(org.apache.beam.sdk.fn.stream.DataStreams.OutputChunkConsumer<org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.ByteString>) OutputStream becomes full.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_OUTBOUND_BUFFER_LIMIT_BYTES

Constructor Summary

Constructors

Constructor and Description
DataStreams()

Method Summary

Modifier and Type	Method and Description
static DataStreams.ElementDelimitedOutputStream	outbound(DataStreams.OutputChunkConsumer<org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.ByteString> consumer) Converts a single element delimited OutputStream into multiple ByteStrings.
static DataStreams.ElementDelimitedOutputStream	outbound(DataStreams.OutputChunkConsumer<org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.ByteString> consumer, int maximumChunkSize) Converts a single element delimited OutputStream into multiple ByteStrings using the specified maximum chunk size.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_OUTBOUND_BUFFER_LIMIT_BYTES

public static final int DEFAULT_OUTBOUND_BUFFER_LIMIT_BYTES

See Also:

Constant Field Values

Constructor Detail

DataStreams

public DataStreams()

Method Detail

outbound

public static DataStreams.ElementDelimitedOutputStream outbound(DataStreams.OutputChunkConsumer<org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.ByteString> consumer)

Converts a single element delimited OutputStream into multiple ByteStrings.

Note that users must call DataStreams.ElementDelimitedOutputStream.delimitElement() after each element.

Note that this OutputStream follows the Beam Fn API specification for forcing values that encode producing zero bytes to produce exactly one byte.

outbound

public static DataStreams.ElementDelimitedOutputStream outbound(DataStreams.OutputChunkConsumer<org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.ByteString> consumer,
                                                                int maximumChunkSize)

Converts a single element delimited OutputStream into multiple ByteStrings using the specified maximum chunk size.

Note that users must call DataStreams.ElementDelimitedOutputStream.delimitElement() after each element.

Note that this OutputStream follows the Beam Fn API specification for forcing values that encode producing zero bytes to produce exactly one byte.