apache_beam.io.range_trackers module

iobase.RangeTracker implementations provided with Apache Beam.

class apache_beam.io.range_trackers.OffsetRangeTracker(start, end)

Bases: RangeTracker

A ‘RangeTracker’ for non-negative positions of type ‘long’.

OFFSET_INFINITY = inf

start_position()

stop_position()

property last_record_start

property last_attempted_record_start

Return current value of last_attempted_record_start.

last_attempted_record_start records a valid position that tried to be claimed by calling try_claim(). This value is only updated by try_claim() no matter try_claim() returns True or False.

try_claim(record_start)

set_current_position(record_start)

try_split(split_offset)

fraction_consumed()

position_to_fraction(pos, start, stop)

position_at_fraction(fraction)

split_points()

set_split_points_unclaimed_callback(callback)

class apache_beam.io.range_trackers.LexicographicKeyRangeTracker(start_position=None, stop_position=None)

Bases: OrderedPositionRangeTracker

A range tracker that tracks progress through a lexicographically ordered keyspace of strings.

classmethod fraction_to_position(fraction: float, start: bytes | str | None = None, end: bytes | str | None = None) → bytes | str

Linearly interpolates a key that is lexicographically fraction of the way between start and end.

classmethod position_to_fraction(key: bytes | str | None = None, start: bytes | str | None = None, end: bytes | str | None = None) → float

Returns the fraction of keys in the range [start, end) that are less than the given key.

class apache_beam.io.range_trackers.OrderedPositionRangeTracker(start_position=None, stop_position=None)

Bases: RangeTracker

An abstract base class for range trackers whose positions are comparable.

Subclasses only need to implement the mapping from position ranges to and from the closed interval [0, 1].

UNSTARTED = <object object>

start_position()

stop_position()

try_claim(position)

position_at_fraction(fraction)

try_split(position)

fraction_consumed()

fraction_to_position(fraction, start, end)

Converts a fraction between 0 and 1 to a position between start and end.

position_to_fraction(position, start, end)

Returns the fraction of keys in the range [start, end) that are less than the given key.

class apache_beam.io.range_trackers.UnsplittableRangeTracker(range_tracker)

Bases: RangeTracker

A RangeTracker that always ignores split requests.

This can be used to make a given RangeTracker object unsplittable by ignoring all calls to try_split(). All other calls will be delegated to the given RangeTracker.

Initializes UnsplittableRangeTracker.

Parameters:

range_tracker (RangeTracker) – a RangeTracker to which all method calls except calls to try_split() will be delegated.

start_position()

stop_position()

position_at_fraction(fraction)

try_claim(position)

try_split(position)

set_current_position(position)

fraction_consumed()

split_points()

set_split_points_unclaimed_callback(callback)