apache_beam.io.range_trackers module¶

iobase.RangeTracker implementations provided with Apache Beam.

class apache_beam.io.range_trackers.OffsetRangeTracker(start, end)[source]¶

Bases: apache_beam.io.iobase.RangeTracker

A ‘RangeTracker’ for non-negative positions of type ‘long’.

OFFSET_INFINITY = inf¶

start_position()[source]¶

stop_position()[source]¶

last_record_start¶

last_attempted_record_start¶

Return current value of last_attempted_record_start.

last_attempted_record_start records a valid position that tried to be claimed by calling try_claim(). This value is only updated by try_claim() no matter try_claim() returns True or False.

try_claim(record_start)[source]¶

set_current_position(record_start)[source]¶

try_split(split_offset)[source]¶

fraction_consumed()[source]¶

position_to_fraction(pos, start, stop)[source]¶

position_at_fraction(fraction)[source]¶

split_points()[source]¶

set_split_points_unclaimed_callback(callback)[source]¶

class apache_beam.io.range_trackers.OrderedPositionRangeTracker(start_position=None, stop_position=None)[source]¶

Bases: apache_beam.io.iobase.RangeTracker

An abstract base class for range trackers whose positions are comparable.

Subclasses only need to implement the mapping from position ranges to and from the closed interval [0, 1].

UNSTARTED = <object object>¶

start_position()[source]¶

stop_position()[source]¶

try_claim(position)[source]¶

position_at_fraction(fraction)[source]¶

try_split(position)[source]¶

fraction_consumed()[source]¶

fraction_to_position(fraction, start, end)[source]¶: Converts a fraction between 0 and 1 to a position between start and end.

position_to_fraction(position, start, end)[source]¶: Returns the fraction of keys in the range [start, end) that are less than the given key.

class apache_beam.io.range_trackers.UnsplittableRangeTracker(range_tracker)[source]¶

Bases: apache_beam.io.iobase.RangeTracker

A RangeTracker that always ignores split requests.

This can be used to make a given RangeTracker object unsplittable by ignoring all calls to try_split(). All other calls will be delegated to the given RangeTracker.

Initializes UnsplittableRangeTracker.

Parameters:	range_tracker (RangeTracker) – a `RangeTracker` to which all method calls except calls to `try_split()` will be delegated.

start_position()[source]¶

stop_position()[source]¶

position_at_fraction(fraction)[source]¶

try_claim(position)[source]¶

try_split(position)[source]¶

set_current_position(position)[source]¶

fraction_consumed()[source]¶

split_points()[source]¶

set_split_points_unclaimed_callback(callback)[source]¶

class apache_beam.io.range_trackers.LexicographicKeyRangeTracker(start_position=None, stop_position=None)[source]¶

Bases: apache_beam.io.range_trackers.OrderedPositionRangeTracker

A range tracker that tracks progress through a lexicographically ordered keyspace of strings.

classmethod fraction_to_position(fraction: float, start: Union[bytes, str] = None, end: Union[bytes, str] = None) → Union[bytes, str][source]¶: Linearly interpolates a key that is lexicographically fraction of the way between start and end.

classmethod position_to_fraction(key: Union[bytes, str] = None, start: Union[bytes, str] = None, end: Union[bytes, str] = None) → float[source]¶: Returns the fraction of keys in the range [start, end) that are less than the given key.