public abstract class GenerateSequence extends PTransform<PBegin,PCollection<java.lang.Long>>
PTransform that produces longs starting from the given value, and either up to the
given limit or until Long.MAX_VALUE / until the given time elapses.
The bounded GenerateSequence is implemented based on OffsetBasedSource and
OffsetBasedSource.OffsetBasedReader, so it performs efficient initial splitting and it
supports dynamic work rebalancing.
To produce a bounded PCollection<Long>:
Pipeline p = ...
PCollection<Long> bounded = p.apply(GenerateSequence.from(0).to(1000));
To produce an unbounded PCollection<Long>, simply do not specify to(long),
calling withTimestampFn(SerializableFunction) to provide values with timestamps other
than Instant.now().
Pipeline p = ...
// To use processing time as the element timestamp.
PCollection<Long> unbounded = p.apply(GenerateSequence.from(0));
// Or, to use a provided function to set the element timestamp.
PCollection<Long> unboundedWithTimestamps =
p.apply(GenerateSequence.from(0).withTimestampFn(someFn));
In all cases, the sequence of numbers is generated in parallel, so there is no inherent
ordering between the generated values - it is only guaranteed that all values in the given range
will be present in the resulting PCollection.
| Modifier and Type | Class and Description |
|---|---|
static class |
GenerateSequence.External
Exposes GenerateSequence as an external transform for cross-language usage.
|
annotations, displayData, name, resourceHints| Constructor and Description |
|---|
GenerateSequence() |
| Modifier and Type | Method and Description |
|---|---|
PCollection<java.lang.Long> |
expand(PBegin input)
Override this method to specify how this
PTransform should be expanded on the given
InputT. |
static GenerateSequence |
from(long from)
Specifies the minimum number to generate (inclusive).
|
void |
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.
|
GenerateSequence |
to(long to)
Specifies the maximum number to generate (exclusive).
|
GenerateSequence |
withMaxReadTime(Duration maxReadTime)
Specifies to stop generating elements after the given time.
|
GenerateSequence |
withRate(long numElements,
Duration periodLength)
Specifies to generate at most a given number of elements per a given period.
|
GenerateSequence |
withTimestampFn(SerializableFunction<java.lang.Long,Instant> timestampFn)
Specifies the function to use to assign timestamps to the elements.
|
addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validatepublic static GenerateSequence from(long from)
public GenerateSequence to(long to)
public GenerateSequence withTimestampFn(SerializableFunction<java.lang.Long,Instant> timestampFn)
public GenerateSequence withRate(long numElements, Duration periodLength)
public GenerateSequence withMaxReadTime(Duration maxReadTime)
public PCollection<java.lang.Long> expand(PBegin input)
PTransformPTransform should be expanded on the given
InputT.
NOTE: This method should not be called directly. Instead apply the PTransform should
be applied to the InputT using the apply method.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
expand in class PTransform<PBegin,PCollection<java.lang.Long>>public void populateDisplayData(DisplayData.Builder builder)
PTransformpopulateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect
display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace,
but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace
of the subcomponent.
By default, does not register any display data. Implementors may override this method to provide their own display data.
populateDisplayData in interface HasDisplayDatapopulateDisplayData in class PTransform<PBegin,PCollection<java.lang.Long>>builder - The builder to populate with display data.HasDisplayData