@Experimental(value=SOURCE_SINK) public static class BigQueryStorageStreamSource.BigQueryStorageStreamReader<T> extends OffsetBasedSource.OffsetBasedReader<T>
Source.Reader which reads records from a stream.SPLIT_POINTS_UNKNOWN| Modifier and Type | Method and Description |
|---|---|
protected boolean |
advanceImpl()
Advances to the next record and returns
true, or returns false if there is no next
record. |
boolean |
allowsDynamicSplitting()
Whether this reader should allow dynamic splitting of the offset ranges.
|
void |
close()
Closes the reader.
|
T |
getCurrent()
Returns the value of the data item that was read by the last
Source.Reader.start() or Source.Reader.advance() call. |
protected long |
getCurrentOffset()
Returns the starting offset of the
current record,
which has been read by the last successful Source.Reader.start() or Source.Reader.advance() call. |
BigQueryStorageStreamSource<T> |
getCurrentSource()
Returns a
Source describing the same input that this Reader currently reads
(including items already read). |
protected boolean |
startImpl()
Initializes the
OffsetBasedSource.OffsetBasedReader and advances to the first record,
returning true if there is a record available to be read. |
advance, getFractionConsumed, getSplitPointsConsumed, getSplitPointsRemaining, isAtSplitPoint, isDone, isStarted, splitAtFraction, startgetCurrentTimestampprotected boolean startImpl()
throws java.io.IOException
OffsetBasedSource.OffsetBasedReaderOffsetBasedSource.OffsetBasedReader and advances to the first record,
returning true if there is a record available to be read. This method will be invoked
exactly once and may perform expensive setup operations that are needed to initialize the
reader.
This function is the OffsetBasedReader implementation of BoundedReader#start. The key difference is that the implementor can ignore the possibility
that it should no longer produce the first record, either because it has exceeded the
original endOffset assigned to the reader, or because a concurrent call to OffsetBasedSource.OffsetBasedReader.splitAtFraction(double) has changed the source to shrink the offset range being read.
startImpl in class OffsetBasedSource.OffsetBasedReader<T>java.io.IOExceptionBoundedReader#startprotected boolean advanceImpl()
throws java.io.IOException
OffsetBasedSource.OffsetBasedReadertrue, or returns false if there is no next
record.
This function is the OffsetBasedReader implementation of BoundedReader#advance. The key difference is that the implementor can ignore the possibility
that it should no longer produce the next record, either because it has exceeded the original
endOffset assigned to the reader, or because a concurrent call to OffsetBasedSource.OffsetBasedReader.splitAtFraction(double) has changed the source to shrink the offset range being read.
advanceImpl in class OffsetBasedSource.OffsetBasedReader<T>java.io.IOExceptionBoundedReader#advancepublic T getCurrent() throws java.util.NoSuchElementException
Source.ReaderSource.Reader.start() or Source.Reader.advance() call. The returned value must be effectively immutable and remain valid
indefinitely.
Multiple calls to this method without an intervening call to Source.Reader.advance() should
return the same result.
getCurrent in class Source.Reader<T>java.util.NoSuchElementException - if Source.Reader.start() was never called, or if the last
Source.Reader.start() or Source.Reader.advance() returned false.protected long getCurrentOffset()
throws java.util.NoSuchElementException
OffsetBasedSource.OffsetBasedReadercurrent record,
which has been read by the last successful Source.Reader.start() or Source.Reader.advance() call.
If no such call has been made yet, the return value is unspecified.
See RangeTracker for description of offset semantics.
getCurrentOffset in class OffsetBasedSource.OffsetBasedReader<T>java.util.NoSuchElementExceptionpublic void close()
Source.Readerclose in interface java.lang.AutoCloseableclose in class Source.Reader<T>public BigQueryStorageStreamSource<T> getCurrentSource()
BoundedSource.BoundedReaderSource describing the same input that this Reader currently reads
(including items already read).
Reader subclasses can use this method for convenience to access unchanging properties of the source being read. Alternatively, they can cache these properties in the constructor.
The framework will call this method in the course of dynamic work rebalancing, e.g. after
a successful BoundedSource.BoundedReader.splitAtFraction(double) call.
Remember that Source objects must always be immutable. However, the return value
of this function may be affected by dynamic work rebalancing, happening asynchronously via
BoundedSource.BoundedReader.splitAtFraction(double), meaning it can return a different Source object. However, the returned object itself will still itself be immutable. Callers
must take care not to rely on properties of the returned source that may be asynchronously
changed as a result of this process (e.g. do not cache an end offset when reading a file).
For convenience, subclasses should usually return the most concrete subclass of Source possible. In practice, the implementation of this method should nearly always be one
of the following:
BoundedSource.BoundedReader.getCurrentSource(): delegate to base class. In this case, it is almost always an error
for the subclass to maintain its own copy of the source.
public FooReader(FooSource<T> source) {
super(source);
}
public FooSource<T> getCurrentSource() {
return (FooSource<T>)super.getCurrentSource();
}
private final FooSource<T> source;
public FooReader(FooSource<T> source) {
this.source = source;
}
public FooSource<T> getCurrentSource() {
return source;
}
BoundedSource.BoundedReader that explicitly supports dynamic work rebalancing:
maintain a variable pointing to an immutable source object, and protect it with
synchronization.
private FooSource<T> source;
public FooReader(FooSource<T> source) {
this.source = source;
}
public synchronized FooSource<T> getCurrentSource() {
return source;
}
public synchronized FooSource<T> splitAtFraction(double fraction) {
...
FooSource<T> primary = ...;
FooSource<T> residual = ...;
this.source = primary;
return residual;
}
getCurrentSource in class OffsetBasedSource.OffsetBasedReader<T>public boolean allowsDynamicSplitting()
OffsetBasedSource.OffsetBasedReaderTrue by default. Override this to return false if the reader cannot support dynamic
splitting correctly. If this returns false, OffsetBasedSource.OffsetBasedReader.splitAtFraction(double) will
refuse all split requests.
allowsDynamicSplitting in class OffsetBasedSource.OffsetBasedReader<T>