Package org.apache.beam.sdk.io

Class CountingSource

java.lang.Object

org.apache.beam.sdk.io.CountingSource

public class CountingSource
extends Object

Most users should use GenerateSequence instead.

A source that produces longs. When used as a BoundedSource, CountingSource starts at 0 and counts up to a specified maximum. When used as an UnboundedSource, it counts up to Long.MAX_VALUE and then never produces more output. (In practice, this limit should never be reached.)

The bounded CountingSource is implemented based on OffsetBasedSource and OffsetBasedSource.OffsetBasedReader, so it performs efficient initial splitting and it supports dynamic work rebalancing.

To produce a bounded source, use createSourceForSubrange(long, long). To produce an unbounded source, use createUnboundedFrom(long).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CountingSource.CounterMark	The checkpoint for an unbounded CountingSource is simply the last value produced.
static class	CountingSource.CounterMarkCoder	A custom coder for CounterMark.

Method Summary

Modifier and Type	Method	Description
static UnboundedSource<Long,CountingSource.CounterMark>	unbounded()	Deprecated. use GenerateSequence instead
static UnboundedSource<Long,CountingSource.CounterMark>	unboundedWithTimestampFn(SerializableFunction<Long,Instant> timestampFn)	Deprecated. use GenerateSequence and call GenerateSequence.withTimestampFn(SerializableFunction) instead
static BoundedSource<Long>	upTo(long numElements)	Deprecated. use GenerateSequence instead

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

upTo

@Deprecated
public static BoundedSource<Long> upTo(long numElements)

Deprecated.

use GenerateSequence instead

Creates a BoundedSource that will produce the specified number of elements, from 0 to numElements - 1.

unbounded

@Deprecated
public static UnboundedSource<Long,CountingSource.CounterMark> unbounded()

Deprecated.

use GenerateSequence instead

Creates an UnboundedSource that will produce numbers starting from 0 up to Long.MAX_VALUE.

After Long.MAX_VALUE, the source never produces more output. (In practice, this limit should never be reached.)

Elements in the resulting PCollection<Long> will have timestamps corresponding to processing time at element generation, provided by Instant.now().

unboundedWithTimestampFn

@Deprecated
public static UnboundedSource<Long,CountingSource.CounterMark> unboundedWithTimestampFn(SerializableFunction<Long,Instant> timestampFn)

Deprecated.

use GenerateSequence and call GenerateSequence.withTimestampFn(SerializableFunction) instead

Creates an UnboundedSource that will produce numbers starting from 0 up to Long.MAX_VALUE, with element timestamps supplied by the specified function.

After Long.MAX_VALUE, the source never produces more output. (In practice, this limit should never be reached.)

Note that the timestamps produced by timestampFn may not decrease.