apache_beam.runners.interactive.options.capture_limiters module

Module to condition how Interactive Beam stops capturing data.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.runners.interactive.options.capture_limiters.Limiter

Bases: object

Limits an aspect of the caching layer.

is_triggered() → bool

Returns True if the limiter has triggered, and caching should stop.

class apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Bases: Limiter

A Limiter that limits reading from cache based on some property of an element.

update(e: Any) → None

Update the internal state based on some property of an element.

This is executed on every element that is read from cache.

class apache_beam.runners.interactive.options.capture_limiters.SizeLimiter(size_limit: int)

Bases: Limiter

Limits the cache size to a specified byte limit.

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.DurationLimiter(duration_limit: timedelta)

Bases: Limiter

Limits the duration of the capture.

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.CountLimiter(max_count)

Bases: ElementLimiter

Limits by counting the number of elements seen.

update(e)

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.ProcessingTimeLimiter(max_duration_secs)

Bases: ElementLimiter

Limits by how long the ProcessingTime passed in the element stream.

Reads all elements from the timespan [start, start + duration).

This measures the duration from the first element in the stream. Each subsequent element has a delta “advance_duration” that moves the internal clock forward. This triggers when the duration from the internal clock and the start exceeds the given duration.

Initialize the ProcessingTimeLimiter.

update(e)

is_triggered()