org.apache.beam.sdk.io.gcp.bigtable.changestreams.estimator

Class BytesThroughputEstimator<T>

java.lang.Object

org.apache.beam.sdk.io.gcp.bigtable.changestreams.estimator.BytesThroughputEstimator<T>

@Internal
public class BytesThroughputEstimator<T>
extends java.lang.Object

An estimator to provide an estimate on the byte throughput of the outputted elements.

This estimator will keep track of the bytes of reported records within a sliding window. The window consists of the configured number of seconds and each record's bytes will fall into exactly one given second bucket. When more than window size seconds have passed from the current time, the bytes reported for the seconds that fall outside of the window will not be considered anymore. The bytes of the records will be estimated using the configured Coder.

The estimator will sample only 2% of records to save the cost of doing sizeEstimation for every element

Constructor Summary

Constructors

Constructor and Description
BytesThroughputEstimator(SizeEstimator<T> sizeEstimator, @Nullable Instant lastRunTimestamp)
BytesThroughputEstimator(SizeEstimator<T> sizeEstimator, int sampleRate, Instant startTimestamp)

Method Summary

Modifier and Type	Method and Description
java.math.BigDecimal	get() Returns the estimated throughput bytes for this run.
void	update(Instant timeOfRecords, T element) Updates the estimator with the bytes of records if it is selected to be sampled.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BytesThroughputEstimator

public BytesThroughputEstimator(SizeEstimator<T> sizeEstimator,
                                @Nullable Instant lastRunTimestamp)

BytesThroughputEstimator

public BytesThroughputEstimator(SizeEstimator<T> sizeEstimator,
                                int sampleRate,
                                Instant startTimestamp)

Method Detail

update

public void update(Instant timeOfRecords,
                   T element)

Updates the estimator with the bytes of records if it is selected to be sampled.

Parameters:

timeOfRecords - the committed timestamp of the records

element - the element to estimate the byte size of

get

public java.math.BigDecimal get()

Returns the estimated throughput bytes for this run.