apache_beam.ml.anomaly.univariate.quantile module

Trackers for calculating quantiles in windowed fashion.

This module defines different types of quantile trackers that operate on windows of data. It includes:

• SimpleSlidingQuantileTracker: Calculates quantile using numpy in a sliding window.

• BufferedLandmarkQuantileTracker: Sortedlist based quantile tracker in landmark window mode.

• BufferedSlidingQuantileTracker: Sortedlist based quantile tracker in sliding window mode.

class apache_beam.ml.anomaly.univariate.quantile.QuantileTracker(q)

Bases: BaseTracker

Abstract base class for quantile trackers.

Currently, it does not add any specific functionality but provides a type hierarchy for quantile trackers.

class apache_beam.ml.anomaly.univariate.quantile.SimpleSlidingQuantileTracker(*args, **kwargs)

Bases: WindowedTracker, QuantileTracker

Sliding window quantile tracker using NumPy.

This tracker uses NumPy’s nanquantile function to calculate the specified quantile of the values currently in the sliding window. It’s a simple, non-incremental approach.

Parameters:

• window_size – The size of the sliding window.

• q – The quantile to calculate, a float between 0 and 1 (inclusive).

get()

Calculates and returns the specified quantile of the current sliding window.

Returns:

The specified quantile of the values in the current sliding window.

Returns NaN if the window is empty.

Return type:

float

SimpleSlidingQuantileTracker__spec_type = 'SimpleSlidingQuantileTracker'

classmethod from_spec(spec: Spec, _run_init: bool = True) → Self | type[Self]

Generate a Specifiable subclass object based on a spec.

Parameters:

• spec – the specification of a Specifiable subclass object

• _run_init – whether to call __init__ or not for the initial instantiation

Returns:

the Specifiable subclass object

Return type:

Self

run_original_init() → None

Execute the original __init__ method with its saved arguments.

For instances of the Specifiable class, initialization is deferred (lazy initialization). This function forces the execution of the original __init__ method using the arguments captured during the object’s initial instantiation.

classmethod spec_type()

to_spec() → Spec

Generate a spec from a Specifiable subclass object.

Returns:

The specification of the instance.

Return type:

Spec

classmethod unspecifiable()

class apache_beam.ml.anomaly.univariate.quantile.BufferedQuantileTracker(window_mode, q, **kwargs)

Bases: WindowedTracker, QuantileTracker

Abstract base class for buffered quantile trackers.

Warning

Buffered quantile trackers are NOT truly incremental in the sense that they don’t update the quantile in constant time per new data point. They maintain a sorted list of all values in the window.

Parameters:

• window_mode – A WindowMode enum specifying whether the window is LANDMARK or SLIDING.

• q – The quantile to calculate, a float between 0 and 1 (inclusive).

• **kwargs – Keyword arguments passed to the parent class constructor.

push(x)

Pushes a new value, maintains the sorted list, and manages the window.

Parameters:

x – The new value to be pushed.

get()

Returns the current quantile value using the sorted list.

Calculates the quantile using linear interpolation on the sorted values.

Returns:

The calculated quantile value. Returns NaN if the window is empty.

Return type:

float

class apache_beam.ml.anomaly.univariate.quantile.SecondaryBufferedQuantileTracker(*args, **kwargs)

Bases: WindowedTracker, QuantileTracker

A secondary quantile tracker that shares its data with a master tracker.

This tracker acts as a read-only view of the master tracker’s data, providing quantile calculations without maintaining its own independent buffer. It relies on the master’s sorted items for quantile estimations.

Parameters:

• master – The BufferedQuantileTracker instance to share data with.

• q – A list of quantiles to track.

push(x)

Does nothing, as this is a secondary tracker.

get()

Returns the calculated quantiles based on the master tracker’s buffer.

Returns:

A list of calculated quantiles.

SecondaryBufferedQuantileTracker__spec_type = 'SecondaryBufferedQuantileTracker'

classmethod from_spec(spec: Spec, _run_init: bool = True) → Self | type[Self]

Generate a Specifiable subclass object based on a spec.

Parameters:

• spec – the specification of a Specifiable subclass object

• _run_init – whether to call __init__ or not for the initial instantiation

Returns:

the Specifiable subclass object

Return type:

Self

run_original_init() → None

Execute the original __init__ method with its saved arguments.

For instances of the Specifiable class, initialization is deferred (lazy initialization). This function forces the execution of the original __init__ method using the arguments captured during the object’s initial instantiation.

classmethod spec_type()

to_spec() → Spec

Generate a spec from a Specifiable subclass object.

Returns:

The specification of the instance.

Return type:

Spec

classmethod unspecifiable()

class apache_beam.ml.anomaly.univariate.quantile.BufferedLandmarkQuantileTracker(*args, **kwargs)

Bases: BufferedQuantileTracker

Landmark quantile tracker using a sorted list for quantile calculation.

Warning

Landmark quantile trackers have unbounded memory consumption as they store all pushed values in a sorted list. Avoid using in production for long-running streams.

Parameters:

q – The quantile to calculate, a float between 0 and 1 (inclusive).

BufferedLandmarkQuantileTracker__spec_type = 'BufferedLandmarkQuantileTracker'

classmethod from_spec(spec: Spec, _run_init: bool = True) → Self | type[Self]

Generate a Specifiable subclass object based on a spec.

Parameters:

• spec – the specification of a Specifiable subclass object

• _run_init – whether to call __init__ or not for the initial instantiation

Returns:

the Specifiable subclass object

Return type:

Self

run_original_init() → None

Execute the original __init__ method with its saved arguments.

For instances of the Specifiable class, initialization is deferred (lazy initialization). This function forces the execution of the original __init__ method using the arguments captured during the object’s initial instantiation.

classmethod spec_type()

to_spec() → Spec

Generate a spec from a Specifiable subclass object.

Returns:

The specification of the instance.

Return type:

Spec

classmethod unspecifiable()

class apache_beam.ml.anomaly.univariate.quantile.BufferedSlidingQuantileTracker(*args, **kwargs)

Bases: BufferedQuantileTracker

Sliding window quantile tracker using a sorted list for quantile calculation.

Warning

Maintains a sorted list of values within the sliding window to calculate the specified quantile. Memory consumption is bounded by the window size but can still be significant for large windows.

Parameters:

• window_size – The size of the sliding window.

• q – The quantile to calculate, a float between 0 and 1 (inclusive).

BufferedSlidingQuantileTracker__spec_type = 'BufferedSlidingQuantileTracker'

classmethod from_spec(spec: Spec, _run_init: bool = True) → Self | type[Self]

Generate a Specifiable subclass object based on a spec.

Parameters:

• spec – the specification of a Specifiable subclass object

• _run_init – whether to call __init__ or not for the initial instantiation

Returns:

the Specifiable subclass object

Return type:

Self

run_original_init() → None

Execute the original __init__ method with its saved arguments.

For instances of the Specifiable class, initialization is deferred (lazy initialization). This function forces the execution of the original __init__ method using the arguments captured during the object’s initial instantiation.

classmethod spec_type()

to_spec() → Spec

Generate a spec from a Specifiable subclass object.

Returns:

The specification of the instance.

Return type:

Spec

classmethod unspecifiable()