Package org.apache.beam.sdk.io.components.throttling

Class ReactiveThrottler

java.lang.Object

org.apache.beam.sdk.io.components.throttling.AdaptiveThrottler

org.apache.beam.sdk.io.components.throttling.ReactiveThrottler

public class ReactiveThrottler
extends AdaptiveThrottler

A wrapper around the AdaptiveThrottler that also handles logging and signaling throttling to the SDK harness using the provided namespace.

For usage, instantiate one instance of a ReactiveThrottler class for a PTransform. When making remote calls to a service, preface that call with the throttle() method to potentially pre-emptively throttle the request. This will throttle future calls based on the failure rate of preceding calls, with higher failure rates leading to longer periods of throttling to allow system recovery. capture the timestamp of the attempted request, then execute the request code. On a success, call successfulRequest(timestamp) to report the success to the throttler.

Field Summary

Fields inherited from class org.apache.beam.sdk.io.components.throttling.AdaptiveThrottler

MIN_REQUESTS

Constructor Summary

Constructors

Constructor	Description
ReactiveThrottler(long samplePeriodMs, long sampleUpdateMs, double overloadRatio, String namespace, int throttleDelaySecs)	Initializes the ReactiveThrottler.

Method Summary

Modifier and Type	Method	Description
void	throttle()	Stops request code from advancing while the underlying AdaptiveThrottler is signaling to preemptively throttle the request.

Methods inherited from class org.apache.beam.sdk.io.components.throttling.AdaptiveThrottler

successfulRequest, throttleRequest, throttlingProbability

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ReactiveThrottler

public ReactiveThrottler(long samplePeriodMs,
 long sampleUpdateMs,
 double overloadRatio,
 String namespace,
 int throttleDelaySecs)

Initializes the ReactiveThrottler.

Parameters:

samplePeriodMs - length of history to consider, in ms, to set throttling.

sampleUpdateMs - granularity of time buckets that we store data in, in ms.

overloadRatio - the target ratio between requests sent and successful requests.

namespace - the namespace to use for logging and signaling throttling is occurring.

throttleDelaySecs - the amount of time in seconds to wait after preemptively throttled requests.

Method Details

throttle

public void throttle()
              throws InterruptedException

Stops request code from advancing while the underlying AdaptiveThrottler is signaling to preemptively throttle the request. Automatically handles logging the throttling and signaling to the SDK harness that the request is being throttled. This should be called in any context where a call to a remote service is being contacted prior to the call being performed.

Throws:

InterruptedException