apache_beam.utils.profiler module

A profiler context manager based on cProfile.Profile and guppy.hpy objects.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.utils.profiler.Profile(profile_id, profile_location=None, log_results=False, file_copy_fn=None, time_prefix='%Y-%m-%d_%H_%M_%S-', enable_cpu_profiling=False, enable_memory_profiling=False)[source]

Bases: object

cProfile and Heapy wrapper context for saving and logging profiler results.

Creates a Profile object.

Parameters:
  • profile_id – Unique id of the profiling session.
  • profile_location – The file location where the profiling results will be stored.
  • log_results – Log the result to console if true.
  • file_copy_fn – Lambda function for copying files.
  • time_prefix – Format of the timestamp prefix in profiling result files.
  • enable_cpu_profiling – CPU profiler will be enabled during the profiling session.
  • enable_memory_profiling – Memory profiler will be enabled during the profiling session, the profiler only records the newly allocated objects in this session.
SORTBY = 'cumulative'
profile_output = None
stats = None
static default_file_copy_fn(src, dest)[source]
static factory_from_options(options)[source]