apache_beam.testing.benchmarks.tpcds.tpcds_launcher module

TPC-DS launcher for Apache Beam Python using cross-language SqlTransform and PrismRunner.

Example usage via Gradle:
./gradlew :sdks:python:apache_beam:testing:benchmarks:tpcds:run

-Ptpcds.args=”–queries=supported –data_size=10MB”

apache_beam.testing.benchmarks.tpcds.tpcds_launcher.check_and_normalize_data_size(data_size: str) → str[source]

Validates data_size against SUPPORTED_DATA_SIZES and normalizes ‘10M’ -> ‘10MB’, etc.

class apache_beam.testing.benchmarks.tpcds.tpcds_launcher.TpcdsQueryResult(query_name: str, job_name: str, data_size: str, dialect: str, is_successful: bool, status: str, start_time: str, end_time: str, elapsed_sec: float, row_count: int = 0, error_message: str = '')[source]

Bases: object

query_name: str
job_name: str
data_size: str
dialect: str
is_successful: bool
status: str
start_time: str
end_time: str
elapsed_sec: float
row_count: int = 0
error_message: str = ''
apache_beam.testing.benchmarks.tpcds.tpcds_launcher.build_table_pcollections(pipeline: Pipeline, table_names: Sequence[str], data_directory: str = 'gs://beam-tpcds/datasets/text', data_size: str = '10MB') → Dict[str, PCollection][source]

Creates schema-typed input PCollections from TPC-DS .dat files.

apache_beam.testing.benchmarks.tpcds.tpcds_launcher.run_single_query(query_name: str, query_sql: str, pipeline_args: Sequence[str], data_directory: str = 'gs://beam-tpcds/datasets/text', data_size: str = '10MB', results_directory: str | None = None, skip_planner_timeouts: bool = False) → TpcdsQueryResult[source]

Executes a single TPC-DS query via SqlTransform.

apache_beam.testing.benchmarks.tpcds.tpcds_launcher.format_summary_table(results: Sequence[TpcdsQueryResult]) → str[source]

Formats the TPC-DS execution summary table.

apache_beam.testing.benchmarks.tpcds.tpcds_launcher.main(argv: Sequence[str] | None = None) → List[TpcdsQueryResult][source]
apache_beam.testing.benchmarks.tpcds.tpcds_launcher.verify_no_regressions(results: Sequence[TpcdsQueryResult], known_failing: Dict[str, str] | None = None) → None[source]

Verifies that query results match SUPPORTED_QUERIES and KNOWN_FAILING_QUERIES.

Raises RuntimeError if:
  • A query in SUPPORTED_QUERIES fails, or

  • A query in KNOWN_FAILING_QUERIES fails with an unexpected error message, or

  • A query is unclassified (in neither SUPPORTED_QUERIES nor KNOWN_FAILING_QUERIES).

Queries in KNOWN_FAILING_QUERIES that succeed are tolerated with a warning.