apache_beam.io.range_trackers module

iobase.RangeTracker implementations provided with Dataflow SDK.

class apache_beam.io.range_trackers.OffsetRangeTracker(start, end)[source]

Bases: apache_beam.io.iobase.RangeTracker

A ‘RangeTracker’ for non-negative positions of type ‘long’.

OFFSET_INFINITY = inf
start_position()[source]
stop_position()[source]
last_record_start
try_claim(record_start)[source]
set_current_position(record_start)[source]
try_split(split_offset)[source]
fraction_consumed()[source]
position_at_fraction(fraction)[source]
split_points()[source]
set_split_points_unclaimed_callback(callback)[source]
class apache_beam.io.range_trackers.OrderedPositionRangeTracker(start_position=None, stop_position=None)[source]

Bases: apache_beam.io.iobase.RangeTracker

An abstract base class for range trackers whose positions are comparable.

Subclasses only need to implement the mapping from position ranges to and from the closed interval [0, 1].

UNSTARTED = <object object>
start_position()[source]
stop_position()[source]
try_claim(position)[source]
position_at_fraction(fraction)[source]
try_split(position)[source]
fraction_consumed()[source]
position_to_fraction(pos, start, end)[source]

Converts a position pos betweeen start and end (inclusive) to a fraction between 0 and 1.

fraction_to_position(fraction, start, end)[source]

Converts a fraction between 0 and 1 to a position between start and end.

class apache_beam.io.range_trackers.UnsplittableRangeTracker(range_tracker)[source]

Bases: apache_beam.io.iobase.RangeTracker

A RangeTracker that always ignores split requests.

This can be used to make a given RangeTracker object unsplittable by ignoring all calls to try_split(). All other calls will be delegated to the given RangeTracker.

Initializes UnsplittableRangeTracker.

Parameters:range_tracker (RangeTracker) – a RangeTracker to which all method calls expect calls to try_split() will be delegated.
start_position()[source]
stop_position()[source]
position_at_fraction(fraction)[source]
try_claim(position)[source]
try_split(position)[source]
set_current_position(position)[source]
fraction_consumed()[source]
split_points()[source]
set_split_points_unclaimed_callback(callback)[source]
class apache_beam.io.range_trackers.LexicographicKeyRangeTracker(start_position=None, stop_position=None)[source]

Bases: apache_beam.io.range_trackers.OrderedPositionRangeTracker

A range tracker that tracks progress through a lexicographically ordered keyspace of strings.

classmethod fraction_to_position(fraction, start=None, end=None)[source]

Linearly interpolates a key that is lexicographically fraction of the way between start and end.

classmethod position_to_fraction(key, start=None, end=None)[source]

Returns the fraction of keys in the range [start, end) that are less than the given key.