apache_beam.io.range_trackers module

iobase.RangeTracker implementations provided with Dataflow SDK.

class apache_beam.io.range_trackers.OffsetRangeTracker(start, end)

Bases: apache_beam.io.iobase.RangeTracker

A ‘RangeTracker’ for non-negative positions of type ‘long’.

OFFSET_INFINITY = inf

start_position()

stop_position()

last_record_start

try_claim(record_start)

set_current_position(record_start)

try_split(split_offset)

fraction_consumed()

position_at_fraction(fraction)

split_points()

set_split_points_unclaimed_callback(callback)

class apache_beam.io.range_trackers.OrderedPositionRangeTracker(start_position=None, stop_position=None)

Bases: apache_beam.io.iobase.RangeTracker

An abstract base class for range trackers whose positions are comparable.

Subclasses only need to implement the mapping from position ranges to and from the closed interval [0, 1].

UNSTARTED = <object object>

start_position()

stop_position()

try_claim(position)

position_at_fraction(fraction)

try_split(position)

fraction_consumed()

position_to_fraction(pos, start, end)

Converts a position pos betweeen start and end (inclusive) to a fraction between 0 and 1.

fraction_to_position(fraction, start, end)

Converts a fraction between 0 and 1 to a position between start and end.

class apache_beam.io.range_trackers.UnsplittableRangeTracker(range_tracker)

Bases: apache_beam.io.iobase.RangeTracker

A RangeTracker that always ignores split requests.

This can be used to make a given RangeTracker object unsplittable by ignoring all calls to try_split(). All other calls will be delegated to the given RangeTracker.

Initializes UnsplittableRangeTracker.

Parameters:range_tracker (RangeTracker) – a RangeTracker to which all method calls expect calls to try_split() will be delegated.

start_position()

stop_position()

position_at_fraction(fraction)

try_claim(position)

try_split(position)

set_current_position(position)

fraction_consumed()

split_points()

set_split_points_unclaimed_callback(callback)

class apache_beam.io.range_trackers.LexicographicKeyRangeTracker(start_position=None, stop_position=None)

Bases: apache_beam.io.range_trackers.OrderedPositionRangeTracker

A range tracker that tracks progress through a lexicographically ordered keyspace of strings.

classmethod fraction_to_position(fraction, start=None, end=None)

Linearly interpolates a key that is lexicographically fraction of the way between start and end.

classmethod position_to_fraction(key, start=None, end=None)

Returns the fraction of keys in the range [start, end) that are less than the given key.