apache_beam.runners.interactive.options.capture_limiters module

Module to condition how Interactive Beam stops capturing data.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.runners.interactive.options.capture_limiters.Limiter[source]

Bases: object

Limits an aspect of the caching layer.

is_triggered()[source]

Returns True if the limiter has triggered, and caching should stop.

class apache_beam.runners.interactive.options.capture_limiters.ElementLimiter[source]

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

A Limiter that limits reading from cache based on some property of an element.

update(e)[source]

Update the internal state based on some property of an element.

This is executed on every element that is read from cache.

class apache_beam.runners.interactive.options.capture_limiters.SizeLimiter(size_limit)[source]

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

Limits the cache size to a specified byte limit.

is_triggered()[source]
class apache_beam.runners.interactive.options.capture_limiters.DurationLimiter(duration_limit)[source]

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

Limits the duration of the capture.

is_triggered()[source]
class apache_beam.runners.interactive.options.capture_limiters.CountLimiter(max_count)[source]

Bases: apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Limits by counting the number of elements seen.

update(e)[source]
is_triggered()[source]
class apache_beam.runners.interactive.options.capture_limiters.ProcessingTimeLimiter(max_duration_secs)[source]

Bases: apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Limits by how long the ProcessingTime passed in the element stream.

Reads all elements from the timespan [start, start + duration).

This measures the duration from the first element in the stream. Each subsequent element has a delta “advance_duration” that moves the internal clock forward. This triggers when the duration from the internal clock and the start exceeds the given duration.

Initialize the ProcessingTimeLimiter.

update(e)[source]
is_triggered()[source]