Package org.apache.beam.sdk.fn.stream

Class DataStreams.DataStreamDecoder<T>

java.lang.Object

org.apache.beam.sdk.fn.stream.DataStreams.DataStreamDecoder<T>

All Implemented Interfaces:

Iterator<T>, PrefetchableIterator<T>

Enclosing class:

DataStreams

public static class DataStreams.DataStreamDecoder<T>
extends Object
implements PrefetchableIterator<T>

An adapter which converts an InputStream to a PrefetchableIterator of T values using the specified Coder.

Note that this adapter follows the Beam Fn API specification for forcing values that decode consuming zero bytes to consuming exactly one byte.

Note that access to the underlying InputStream is lazy and will only be invoked on first access to next(), hasNext(), isReady(), and prefetch().

Note that isReady() and prefetch() rely on non-empty ByteStrings being returned via the underlying PrefetchableIterator otherwise the prefetch() will seemingly make zero progress yet will actually advance through the empty pages.

Constructor Summary

Constructors

Constructor	Description
DataStreamDecoder(Coder<T> coder, PrefetchableIterator<org.apache.beam.vendor.grpc.v1p69p0.com.google.protobuf.ByteString> inputStream)

Method Summary

Modifier and Type	Method	Description
WeightedList<T>	decodeFromChunkBoundaryToChunkBoundary()	Skips any remaining bytes in the current ByteString moving to the next ByteString in the underlying ByteString iterator and decoding elements till at the next boundary.
boolean	hasNext()
boolean	isReady()	Returns true if and only if Iterator.hasNext() and Iterator.next() will not require an expensive operation.
T	next()
void	prefetch()	If not PrefetchableIterator.isReady(), schedules the next expensive operation such that at some point in time in the future PrefetchableIterator.isReady() will return true.
void	remove()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Details

DataStreamDecoder

public DataStreamDecoder(Coder<T> coder,
 PrefetchableIterator<org.apache.beam.vendor.grpc.v1p69p0.com.google.protobuf.ByteString> inputStream)

Method Details

decodeFromChunkBoundaryToChunkBoundary

public WeightedList<T> decodeFromChunkBoundaryToChunkBoundary()

Skips any remaining bytes in the current ByteString moving to the next ByteString in the underlying ByteString iterator and decoding elements till at the next boundary.

isReady

public boolean isReady()

Description copied from interface: PrefetchableIterator

Returns true if and only if Iterator.hasNext() and Iterator.next() will not require an expensive operation.

Specified by:

isReady in interface PrefetchableIterator<T>

prefetch

public void prefetch()

Description copied from interface: PrefetchableIterator

If not PrefetchableIterator.isReady(), schedules the next expensive operation such that at some point in time in the future PrefetchableIterator.isReady() will return true.

Specified by:

prefetch in interface PrefetchableIterator<T>

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<T>

next

public T next()

Specified by:

next in interface Iterator<T>

remove

public void remove()

Specified by:

remove in interface Iterator<T>