public class ValuesSource<T> extends UnboundedSource<T,UnboundedSource.CheckpointMark>
Iterable.UnboundedSource.CheckpointMark, UnboundedSource.UnboundedReader<OutputT>Source.Reader<T>| Constructor and Description |
|---|
ValuesSource(java.lang.Iterable<T> values,
Coder<T> coder) |
| Modifier and Type | Method and Description |
|---|---|
UnboundedSource.UnboundedReader<T> |
createReader(PipelineOptions options,
UnboundedSource.CheckpointMark checkpointMark)
Create a new
UnboundedSource.UnboundedReader to read from this source, resuming from the given
checkpoint if present. |
Coder<UnboundedSource.CheckpointMark> |
getCheckpointMarkCoder()
Returns a
Coder for encoding and decoding the checkpoints for this source. |
Coder<T> |
getDefaultOutputCoder() |
java.util.List<? extends UnboundedSource<T,UnboundedSource.CheckpointMark>> |
split(int desiredNumSplits,
PipelineOptions options)
Returns a list of
UnboundedSource objects representing the instances of this source
that should be used when executing the workflow. |
requiresDedupinggetOutputCoder, populateDisplayData, validatepublic java.util.List<? extends UnboundedSource<T,UnboundedSource.CheckpointMark>> split(int desiredNumSplits, PipelineOptions options) throws java.lang.Exception
UnboundedSourceUnboundedSource objects representing the instances of this source
that should be used when executing the workflow. Each split should return a separate partition
of the input data.
For example, for a source reading from a growing directory of files, each split could correspond to a prefix of file names.
Some sources are not splittable, such as reading from a single TCP stream. In that case, only a single split should be returned.
Some data sources automatically partition their data among readers. For these types of
inputs, n identical replicas of the top-level source can be returned.
The size of the returned list should be as close to desiredNumSplits as possible,
but does not have to match exactly. A low number of splits will limit the amount of parallelism
in the source.
split in class UnboundedSource<T,UnboundedSource.CheckpointMark>java.lang.Exceptionpublic UnboundedSource.UnboundedReader<T> createReader(PipelineOptions options, @Nullable UnboundedSource.CheckpointMark checkpointMark)
UnboundedSourceUnboundedSource.UnboundedReader to read from this source, resuming from the given
checkpoint if present.createReader in class UnboundedSource<T,UnboundedSource.CheckpointMark>@Nullable public Coder<UnboundedSource.CheckpointMark> getCheckpointMarkCoder()
UnboundedSourceCoder for encoding and decoding the checkpoints for this source.getCheckpointMarkCoder in class UnboundedSource<T,UnboundedSource.CheckpointMark>public Coder<T> getDefaultOutputCoder()
getDefaultOutputCoder in class Source<T>