apache_beam.utils.retry module

Retry decorators for calls raising exceptions.

For internal use only; no backwards-compatibility guarantees.

This module is used mostly to decorate all integration points where the code makes calls to remote services. Searching through the code base for @retry should find all such places. For this reason even places where retry is not needed right now use a @retry.no_retries decorator.

exception apache_beam.utils.retry.PermanentException

Bases: Exception

Base class for exceptions that should not be retried.

class apache_beam.utils.retry.FuzzedExponentialIntervals(initial_delay_secs, num_retries, factor=2, fuzz=0.5, max_delay_secs=3600, stop_after_secs=None)

Bases: object

Iterable for intervals that are exponentially spaced, with fuzzing.

On iteration, yields retry interval lengths, in seconds. Every iteration over this iterable will yield differently fuzzed interval lengths, as long as fuzz is nonzero.

Parameters:

• initial_delay_secs – The delay before the first retry, in seconds.
• num_retries – The total number of times to retry.
• factor – The exponential factor to use on subsequent retries. Default is 2 (doubling).
• fuzz – A value between 0 and 1, indicating the fraction of fuzz. For a given delay d, the fuzzed delay is randomly chosen between [(1 - fuzz) * d, d].
• max_delay_secs – Maximum delay (in seconds). After this limit is reached, further tries use max_delay_sec instead of exponentially increasing the time. Defaults to 1 hour.
• stop_after_secs – Places a limit on the sum of intervals returned (in seconds), such that the sum is <= stop_after_secs. Defaults to disabled (None). You may need to increase num_retries to effectively use this feature.

apache_beam.utils.retry.retry_on_server_errors_filter(exception)

Filter allowing retries on server errors and non-HttpErrors.

apache_beam.utils.retry.retry_on_server_errors_and_notfound_filter(exception)

apache_beam.utils.retry.retry_on_server_errors_and_timeout_filter(exception)

apache_beam.utils.retry.retry_on_server_errors_timeout_or_quota_issues_filter(exception)

Retry on server, timeout, 429, and some 403 errors.

403 errors from BigQuery include both non-transient (accessDenied, billingNotEnabled) and transient errors (rateLimitExceeded). Only retry transient errors.

apache_beam.utils.retry.retry_on_beam_io_error_filter(exception)

Filter allowing retries on Beam IO errors.

apache_beam.utils.retry.retry_if_valid_input_but_server_error_and_timeout_filter(exception)

class apache_beam.utils.retry.Clock

Bases: object

A simple clock implementing sleep().

sleep(value)

apache_beam.utils.retry.no_retries(fun)

A retry decorator for places where we do not want retries.

apache_beam.utils.retry.with_exponential_backoff(num_retries=7, initial_delay_secs=5.0, logger=<bound method Logger.warning of <Logger apache_beam.utils.retry (WARNING)>>, retry_filter=<function retry_on_server_errors_filter>, clock=<apache_beam.utils.retry.Clock object>, fuzz=True, factor=2, max_delay_secs=3600, stop_after_secs=None)

Decorator with arguments that control the retry logic.

Parameters:

• num_retries – The total number of times to retry.
• initial_delay_secs – The delay before the first retry, in seconds.
• logger – A callable used to report an exception. Must have the same signature as functions in the standard logging module. The default is _LOGGER.warning.
• retry_filter – A callable getting the exception raised and returning True if the retry should happen. For instance we do not want to retry on 404 Http errors most of the time. The default value will return true for server errors (HTTP status code >= 500) and non Http errors.
• clock – A clock object implementing a sleep method. The default clock will use time.sleep().
• fuzz – True if the delay should be fuzzed (default). During testing False can be used so that the delays are not randomized.
• factor – The exponential factor to use on subsequent retries. Default is 2 (doubling).
• max_delay_secs – Maximum delay (in seconds). After this limit is reached, further tries use max_delay_sec instead of exponentially increasing the time. Defaults to 1 hour.
• stop_after_secs – Places a limit on the sum of delays between retries, such that the sum is <= stop_after_secs. Retries will stop after the limit is reached. Defaults to disabled (None). You may need to increase num_retries to effectively use this feature.

Returns:

As per Python decorators with arguments pattern returns a decorator for the function which in turn will return the wrapped (decorated) function.

The decorator is intended to be used on callables that make HTTP or RPC requests that can temporarily timeout or have transient errors. For instance the make_http_request() call below will be retried 16 times with exponential backoff and fuzzing of the delay interval (default settings).

from apache_beam.utils import retry # … @retry.with_exponential_backoff() make_http_request(args)