|Modifier and Type||Method and Description|
Create a new
Returns a list of
getDefaultOutputCoder, populateDisplayData, validate
public java.util.List<? extends UnboundedSource<com.google.cloud.pubsublite.proto.SequencedMessage,CheckpointMarkImpl>> split(int desiredNumSplits, PipelineOptions options) throws java.lang.Exception
UnboundedSourceobjects representing the instances of this source that should be used when executing the workflow. Each split should return a separate partition of the input data.
For example, for a source reading from a growing directory of files, each split could correspond to a prefix of file names.
Some sources are not splittable, such as reading from a single TCP stream. In that case, only a single split should be returned.
Some data sources automatically partition their data among readers. For these types of
n identical replicas of the top-level source can be returned.
The size of the returned list should be as close to
desiredNumSplits as possible,
but does not have to match exactly. A low number of splits will limit the amount of parallelism
in the source.
public UnboundedSource.UnboundedReader<com.google.cloud.pubsublite.proto.SequencedMessage> createReader(PipelineOptions options, @Nullable CheckpointMarkImpl checkpointMark) throws java.io.IOException
UnboundedSource.UnboundedReaderto read from this source, resuming from the given checkpoint if present.
public Coder<CheckpointMarkImpl> getCheckpointMarkCoder()
Coderfor encoding and decoding the checkpoints for this source.