apache_beam.runners.interactive.options.capture_limiters module

Module to condition how Interactive Beam stops capturing data.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.runners.interactive.options.capture_limiters.Limiter

Bases: object

Limits an aspect of the caching layer.

is_triggered()

Returns True if the limiter has triggered, and caching should stop.

class apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

A Limiter that limits reading from cache based on some property of an element.

update(e)

Update the internal state based on some property of an element.

This is executed on every element that is read from cache.

class apache_beam.runners.interactive.options.capture_limiters.SizeLimiter(size_limit)

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

Limits the cache size to a specified byte limit.

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.DurationLimiter(duration_limit)

Bases: apache_beam.runners.interactive.options.capture_limiters.Limiter

Limits the duration of the capture.

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.CountLimiter(max_count)

Bases: apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Limits by counting the number of elements seen.

update(e)

is_triggered()

class apache_beam.runners.interactive.options.capture_limiters.ProcessingTimeLimiter(max_duration_secs)

Bases: apache_beam.runners.interactive.options.capture_limiters.ElementLimiter

Limits by how long the ProcessingTime passed in the element stream.

Reads all elements from the timespan [start, start + duration).

This measures the duration from the first element in the stream. Each subsequent element has a delta “advance_duration” that moves the internal clock forward. This triggers when the duration from the internal clock and the start exceeds the given duration.

Initialize the ProcessingTimeLimiter.

update(e)

is_triggered()