apache_beam.testing.benchmarks.nexmark.queries.query10 module

Query 10, ‘Log to sharded files’ (Not in original suite.)

Every window_size_sec, save all events from the last period into 2*max_workers log files.

class apache_beam.testing.benchmarks.nexmark.queries.query10.OutputFile(max_timestamp, shard, index, timing, filename)[source]

Bases: object

apache_beam.testing.benchmarks.nexmark.queries.query10.open_writable_gcs_file(options, filename)[source]
apache_beam.testing.benchmarks.nexmark.queries.query10.output_file_for(window, shard, pane)[source]
Returns:an OutputFile object constructed with pane, window and shard.
apache_beam.testing.benchmarks.nexmark.queries.query10.index_path_for(window)[source]
Returns:
path to the index file containing all shard names or None if no output_path
is set
apache_beam.testing.benchmarks.nexmark.queries.query10.load(events, pipeline_options, metadata=None)[source]
class apache_beam.testing.benchmarks.nexmark.queries.query10.ShardEventsDoFn(*unused_args, **unused_kwargs)[source]

Bases: apache_beam.transforms.core.DoFn

process(element)[source]
class apache_beam.testing.benchmarks.nexmark.queries.query10.WriteEventDoFn(*unused_args, **unused_kwargs)[source]

Bases: apache_beam.transforms.core.DoFn

process(element, pipeline_options, window=WindowParam, pane_info=PaneInfoParam)[source]
class apache_beam.testing.benchmarks.nexmark.queries.query10.WriteIndexDoFn(*unused_args, **unused_kwargs)[source]

Bases: apache_beam.transforms.core.DoFn

process(element, pipeline_options, window=WindowParam)[source]