apache_beam.io.range_trackers module

iobase.RangeTracker implementations provided with Apache Beam.

class apache_beam.io.range_trackers.OffsetRangeTracker(start, end)[source]

Bases: apache_beam.io.iobase.RangeTracker

A ‘RangeTracker’ for non-negative positions of type ‘long’.

OFFSET_INFINITY = inf
start_position()[source]
stop_position()[source]
last_record_start
last_attempted_record_start

Return current value of last_attempted_record_start.

last_attempted_record_start records a valid position that tried to be claimed by calling try_claim(). This value is only updated by try_claim() no matter try_claim() returns True or False.

try_claim(record_start)[source]
set_current_position(record_start)[source]
try_split(split_offset)[source]
fraction_consumed()[source]
position_to_fraction(pos, start, stop)[source]
position_at_fraction(fraction)[source]
split_points()[source]
set_split_points_unclaimed_callback(callback)[source]
class apache_beam.io.range_trackers.OrderedPositionRangeTracker(start_position=None, stop_position=None)[source]

Bases: apache_beam.io.iobase.RangeTracker

An abstract base class for range trackers whose positions are comparable.

Subclasses only need to implement the mapping from position ranges to and from the closed interval [0, 1].

UNSTARTED = <object object>
start_position()[source]
stop_position()[source]
try_claim(position)[source]
position_at_fraction(fraction)[source]
try_split(position)[source]
fraction_consumed()[source]
fraction_to_position(fraction, start, end)[source]

Converts a fraction between 0 and 1 to a position between start and end.

position_to_fraction(position, start, end)[source]

Returns the fraction of keys in the range [start, end) that are less than the given key.

class apache_beam.io.range_trackers.UnsplittableRangeTracker(range_tracker)[source]

Bases: apache_beam.io.iobase.RangeTracker

A RangeTracker that always ignores split requests.

This can be used to make a given RangeTracker object unsplittable by ignoring all calls to try_split(). All other calls will be delegated to the given RangeTracker.

Initializes UnsplittableRangeTracker.

Parameters:range_tracker (RangeTracker) – a RangeTracker to which all method calls expect calls to try_split() will be delegated.
start_position()[source]
stop_position()[source]
position_at_fraction(fraction)[source]
try_claim(position)[source]
try_split(position)[source]
set_current_position(position)[source]
fraction_consumed()[source]
split_points()[source]
set_split_points_unclaimed_callback(callback)[source]
class apache_beam.io.range_trackers.LexicographicKeyRangeTracker(start_position=None, stop_position=None)[source]

Bases: apache_beam.io.range_trackers.OrderedPositionRangeTracker

A range tracker that tracks progress through a lexicographically ordered keyspace of strings.

classmethod fraction_to_position(fraction: float, start: Union[bytes, str] = None, end: Union[bytes, str] = None) → Union[bytes, str][source]

Linearly interpolates a key that is lexicographically fraction of the way between start and end.

classmethod position_to_fraction(key: Union[bytes, str] = None, start: Union[bytes, str] = None, end: Union[bytes, str] = None) → float[source]

Returns the fraction of keys in the range [start, end) that are less than the given key.