apache_beam.testing.benchmarks.tpcds.tpcds_launcher module
TPC-DS launcher for Apache Beam Python using cross-language SqlTransform and PrismRunner.
- Example usage via Gradle:
- ./gradlew :sdks:python:apache_beam:testing:benchmarks:tpcds:run
-Ptpcds.args=”–queries=supported –data_size=10MB”
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.check_and_normalize_data_size(data_size: str) str[source]
Validates data_size against SUPPORTED_DATA_SIZES and normalizes ‘10M’ -> ‘10MB’, etc.
- class apache_beam.testing.benchmarks.tpcds.tpcds_launcher.TpcdsQueryResult(query_name: str, job_name: str, data_size: str, dialect: str, is_successful: bool, status: str, start_time: str, end_time: str, elapsed_sec: float, row_count: int = 0, error_message: str = '')[source]
Bases:
object
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.build_table_pcollections(pipeline: Pipeline, table_names: Sequence[str], data_directory: str = 'gs://beam-tpcds/datasets/text', data_size: str = '10MB') Dict[str, PCollection][source]
Creates schema-typed input PCollections from TPC-DS .dat files.
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.run_single_query(query_name: str, query_sql: str, pipeline_args: Sequence[str], data_directory: str = 'gs://beam-tpcds/datasets/text', data_size: str = '10MB', results_directory: str | None = None, skip_planner_timeouts: bool = False) TpcdsQueryResult[source]
Executes a single TPC-DS query via SqlTransform.
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.format_summary_table(results: Sequence[TpcdsQueryResult]) str[source]
Formats the TPC-DS execution summary table.
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.main(argv: Sequence[str] | None = None) List[TpcdsQueryResult][source]
- apache_beam.testing.benchmarks.tpcds.tpcds_launcher.verify_no_regressions(results: Sequence[TpcdsQueryResult], known_failing: Dict[str, str] | None = None) None[source]
Verifies that query results match SUPPORTED_QUERIES and KNOWN_FAILING_QUERIES.
- Raises RuntimeError if:
A query in SUPPORTED_QUERIES fails, or
A query in KNOWN_FAILING_QUERIES fails with an unexpected error message, or
A query is unclassified (in neither SUPPORTED_QUERIES nor KNOWN_FAILING_QUERIES).
Queries in KNOWN_FAILING_QUERIES that succeed are tolerated with a warning.