apache_beam.tools.coders_microbenchmark module

A microbenchmark for measuring performance of coders.

This runs a sequence of encode-decode operations on random inputs to collect performance of various coders.

To evaluate coders performance we approximate the behavior how the coders are used in PCollections: we encode and decode a list of elements. An element can be a string, a list of integers, a windowed value, or any other object we want a coder to process.

Run as:
python -m apache_beam.tools.coders_microbenchmark
apache_beam.tools.coders_microbenchmark.coder_benchmark_factory(coder, generate_fn)[source]

Creates a benchmark that encodes and decodes a list of elements.

Parameters:
  • coder – coder to use to encode an element.
  • generate_fn – a callable that generates an element.
apache_beam.tools.coders_microbenchmark.small_int()[source]
apache_beam.tools.coders_microbenchmark.large_int()[source]
apache_beam.tools.coders_microbenchmark.random_string(length)[source]
apache_beam.tools.coders_microbenchmark.small_string()[source]
apache_beam.tools.coders_microbenchmark.large_string()[source]
apache_beam.tools.coders_microbenchmark.list_int(size)[source]
apache_beam.tools.coders_microbenchmark.dict_int_int(size)[source]
apache_beam.tools.coders_microbenchmark.small_list()[source]
apache_beam.tools.coders_microbenchmark.large_list()[source]
apache_beam.tools.coders_microbenchmark.small_tuple()[source]
apache_beam.tools.coders_microbenchmark.large_tuple()[source]
apache_beam.tools.coders_microbenchmark.small_dict()[source]
apache_beam.tools.coders_microbenchmark.large_dict()[source]
apache_beam.tools.coders_microbenchmark.large_iterable()[source]
apache_beam.tools.coders_microbenchmark.globally_windowed_value()[source]
apache_beam.tools.coders_microbenchmark.random_windowed_value(num_windows)[source]
apache_beam.tools.coders_microbenchmark.wv_with_one_window()[source]
apache_beam.tools.coders_microbenchmark.wv_with_multiple_windows()[source]
apache_beam.tools.coders_microbenchmark.run_coder_benchmarks(num_runs, input_size, seed, verbose, filter_regex='.*')[source]