apache_beam.ml.anomaly.detectors.robust_zscore module

class apache_beam.ml.anomaly.detectors.robust_zscore.RobustZScore(*args, **kwargs)

Bases: AnomalyDetector

Robust Z-Score anomaly detector.

This class implements an detection algorithm based on Robust Z-Score (also known as Modified Z-Score), which is a robust alternative to the traditional Z-score [1]. It uses the median and Median Absolute Deviation (MAD) to compute a score that is less sensitive to outliers.

The score is calculated as: |0.6745 * (value - median) / MAD|

Important

In the streaming setting, we use the online version of median and MAD in the calculation. Therefore, the score computed here does not exactly match its batch counterpart.

This implementation is adapted from the implementation within PySAD [2]: https://github.com/selimfirat/pysad/blob/master/pysad/models/median_absolute_deviation.py

The batch version can be seen at PyOD [3]: https://github.com/yzhao062/pyod/blob/master/pyod/models/mad.py

Parameters:

• mad_tracker – Optional MadTracker instance. If None, a default MadTracker is created.

• threshold_criterion – threshold_criterion: Optional ThresholdFn to apply on the score. Defaults to FixedThreshold(3) due to the commonly used 3-sigma rule.

• **kwargs – Additional keyword arguments.

[1]

Hoaglin, David C.. (2013). Volume 16: How to Detect and Handle Outliers.

[2]

Yilmaz, Selim & Kozat, Suleyman. (2020). PySAD: A Streaming Anomaly Detection Framework in Python. 10.48550/arXiv.2009.02572.

[3]

Zhao, Y., Nasrullah, Z. and Li, Z.. (2019). PyOD: A Python Toolbox for Scalable Outlier Detection. Journal of machine learning research (JMLR), 20(96), pp.1-7.

SCALE_FACTOR = 0.6745

learn_one(x: Row) → None

Updates the MadTracker with a new data point.

Parameters:

x – A beam.Row containing a single numerical value.

score_one(x: Row) → float | None

Scores a data point using the Robust Z-Score.

Parameters:

x – A beam.Row containing a single numerical value.

Returns:

The Robust Z-Score.

Return type:

float | None

RobustZScore__spec_type = 'RobustZScore'

classmethod from_spec(spec: Spec, _run_init: bool = True) → Self | type[Self]

Generate a Specifiable subclass object based on a spec.

Parameters:

• spec – the specification of a Specifiable subclass object

• _run_init – whether to call __init__ or not for the initial instantiation

Returns:

the Specifiable subclass object

Return type:

Self

run_original_init() → None

Execute the original __init__ method with its saved arguments.

For instances of the Specifiable class, initialization is deferred (lazy initialization). This function forces the execution of the original __init__ method using the arguments captured during the object’s initial instantiation.

classmethod spec_type()

to_spec() → Spec

Generate a spec from a Specifiable subclass object.

Returns:

The specification of the instance.

Return type:

Spec

classmethod unspecifiable()