AdaptiveThrottler(window_ms, bucket_ms, overload_ratio)[source]¶
Implements adaptive throttling.
See https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg for a full discussion of the use case and algorithm applied.
- window_ms – int, length of history to consider, in ms, to set throttling.
- bucket_ms – int, granularity of time buckets that we store data in, in ms.
- overload_ratio – float, the target ratio between requests sent and successful requests. This is “K” in the formula in https://landing.google.com/sre/book/chapters/handling-overload.html.
Determines whether one RPC attempt should be throttled.
This should be called once each time the caller intends to send an RPC; if it returns true, drop or delay that request (calling this function again after the delay).
Parameters: now – int, time in ms since the epoch Returns: bool, True if the caller should throttle or delay the request.