apache_beam.tools.coders_microbenchmark module

A microbenchmark for measuring performance of coders.

This runs a sequence of encode-decode operations on random inputs to collect performance of various coders.

To evaluate coders performance we approximate the behavior how the coders are used in PCollections: we encode and decode a list of elements. An element can be a string, a list of integers, a windowed value, or any other object we want a coder to process.

Run as:

python -m apache_beam.tools.coders_microbenchmark

apache_beam.tools.coders_microbenchmark.coder_benchmark_factory(coder, generate_fn)

Creates a benchmark that encodes and decodes a list of elements.

Parameters:

• coder – coder to use to encode an element.
• generate_fn – a callable that generates an element.

apache_beam.tools.coders_microbenchmark.small_int()

apache_beam.tools.coders_microbenchmark.large_int()

apache_beam.tools.coders_microbenchmark.random_string(length)

apache_beam.tools.coders_microbenchmark.small_string()

apache_beam.tools.coders_microbenchmark.large_string()

apache_beam.tools.coders_microbenchmark.list_int(size)

apache_beam.tools.coders_microbenchmark.dict_int_int(size)

apache_beam.tools.coders_microbenchmark.small_list()

apache_beam.tools.coders_microbenchmark.large_list()

apache_beam.tools.coders_microbenchmark.small_tuple()

apache_beam.tools.coders_microbenchmark.large_tuple()

apache_beam.tools.coders_microbenchmark.small_dict()

apache_beam.tools.coders_microbenchmark.large_dict()

apache_beam.tools.coders_microbenchmark.large_iterable()

apache_beam.tools.coders_microbenchmark.random_message_with_map(size)

apache_beam.tools.coders_microbenchmark.small_message_with_map()

apache_beam.tools.coders_microbenchmark.large_message_with_map()

apache_beam.tools.coders_microbenchmark.globally_windowed_value()

apache_beam.tools.coders_microbenchmark.random_windowed_value(num_windows)

apache_beam.tools.coders_microbenchmark.wv_with_one_window()

apache_beam.tools.coders_microbenchmark.wv_with_multiple_windows()

apache_beam.tools.coders_microbenchmark.run_coder_benchmarks(num_runs, input_size, seed, verbose, filter_regex='.*')