apache_beam.io.gcp.datastore.v1new.rampup_throttling_fn module

class apache_beam.io.gcp.datastore.v1new.rampup_throttling_fn.RampupThrottlingFn(num_workers, *unused_args, **unused_kwargs)[source]

Bases: apache_beam.transforms.core.DoFn

A DoFn that throttles ramp-up following an exponential function.

An implementation of a client-side throttler that enforces a gradual ramp-up, broadly in line with Datastore best practices. See also https://cloud.google.com/datastore/docs/best-practices#ramping_up_traffic.

Initializes a ramp-up throttler transform.

Parameters:num_workers – A hint for the expected number of workers, used to derive the local rate limit.
process(element, **kwargs)[source]