apache_beam.internal.util module

Utility functions used throughout the package.

For internal use only. No backwards compatibility guarantees.

class apache_beam.internal.util.ArgumentPlaceholder

Bases: future.types.newobject.newobject

For internal use only; no backwards-compatibility guarantees.

A place holder object replacing PValues in argument lists.

A Fn object can take any number of “side inputs”, which are PValues that will be evaluated during pipeline execution and will be provided to the function at the moment of its execution as positional or keyword arguments.

This is used only internally and should never be used by user code. A custom Fn object by the time it executes will have such values replaced with real computed values.

apache_beam.internal.util.remove_objects_from_args(args, kwargs, pvalue_classes)

For internal use only; no backwards-compatibility guarantees.

Replaces all objects of a given type in args/kwargs with a placeholder.

Parameters:

• args – A list of positional arguments.
• kwargs – A dictionary of keyword arguments.
• pvalue_classes – A tuple of class objects representing the types of the arguments that must be replaced with a placeholder value (instance of ArgumentPlaceholder)

Returns:

A 3-tuple containing a modified list of positional arguments, a modified dictionary of keyword arguments, and a list of all objects replaced with a placeholder value.

apache_beam.internal.util.insert_values_in_args(args, kwargs, values)

For internal use only; no backwards-compatibility guarantees.

Replaces all placeholders in args/kwargs with actual values.

Parameters:

• args – A list of positional arguments.
• kwargs – A dictionary of keyword arguments.
• values – A list of values that will be used to replace placeholder values.

Returns:

A 2-tuple containing a modified list of positional arguments, and a modified dictionary of keyword arguments.

apache_beam.internal.util.run_using_threadpool(fn_to_execute, inputs, pool_size)

For internal use only; no backwards-compatibility guarantees.

Runs the given function on given inputs using a thread pool.

Parameters:

• fn_to_execute – Function to execute
• inputs – Inputs on which given function will be executed in parallel.
• pool_size – Size of thread pool.

Returns:

Results retrieved after executing the given function on given inputs.