apache_beam.utils.profiler module

A profiler context manager based on cProfile.Profile and guppy.hpy objects.

For internal use only; no backwards-compatibility guarantees.

class apache_beam.utils.profiler.Profile(profile_id: str, profile_location: str | None = None, log_results: bool = False, file_copy_fn: Callable[[str, str], None] | None = None, time_prefix: str = '%Y-%m-%d_%H_%M_%S-', enable_cpu_profiling: bool = False, enable_memory_profiling: bool = False)[source]

Bases: object

cProfile and Heapy wrapper context for saving and logging profiler results.

Creates a Profile object.

Parameters:

  • profile_id – Unique id of the profiling session.

  • profile_location – The file location where the profiling results will be stored.

  • log_results – Log the result to console if true.

  • file_copy_fn – Lambda function for copying files.

  • time_prefix – Format of the timestamp prefix in profiling result files.

  • enable_cpu_profiling – CPU profiler will be enabled during the profiling session.

  • enable_memory_profiling – Memory profiler will be enabled during the profiling session, the profiler only records the newly allocated objects in this session.

SORTBY = 'cumulative'
profile_output: str
stats: Stats
static default_file_copy_fn(src, dest)[source]
static factory_from_options(options) → Callable[[...], Profile] | None[source]