apache_beam.runners.dataflow package

The DataflowRunner executes pipelines on Google Cloud Dataflow.

Anything in this package not imported here is an internal implementation detail with no backwards-compatibility guarantees.

Submodules

• apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline module
  • metric_matchers()
  • UserMetricsDoFn
    • UserMetricsDoFn.start_bundle()
    • UserMetricsDoFn.process()
    • UserMetricsDoFn.finish_bundle()
  • apply_and_run()
• apache_beam.runners.dataflow.dataflow_exercise_streaming_metrics_pipeline module
  • StreamingUserMetricsDoFn
    • StreamingUserMetricsDoFn.start_bundle()
    • StreamingUserMetricsDoFn.process()
    • StreamingUserMetricsDoFn.finish_bundle()
  • run()
• apache_beam.runners.dataflow.dataflow_job_service module
  • DataflowBeamJob
    • DataflowBeamJob.cancel()
  • run()
• apache_beam.runners.dataflow.dataflow_metrics module
  • DataflowMetrics
    • DataflowMetrics.all_metrics()
    • DataflowMetrics.query()
  • main()
• apache_beam.runners.dataflow.dataflow_runner module
  • DataflowRunner
    • DataflowRunner.NativeReadPTransformOverride
      • DataflowRunner.NativeReadPTransformOverride.get_replacement_transform()
      • DataflowRunner.NativeReadPTransformOverride.matches()
    • DataflowRunner.default_pickle_library_override()
    • DataflowRunner.is_fnapi_compatible()
    • DataflowRunner.poll_for_job_completion()
    • DataflowRunner.side_input_visitor()
    • DataflowRunner.flatten_input_visitor()
    • DataflowRunner.combinefn_visitor()
    • DataflowRunner.run_pipeline()
    • DataflowRunner.get_default_gcp_region()
• apache_beam.runners.dataflow.ptransform_overrides module
  • NativeReadPTransformOverride
    • NativeReadPTransformOverride.matches()
    • NativeReadPTransformOverride.get_replacement_transform()
• apache_beam.runners.dataflow.test_dataflow_runner module
  • TestDataflowRunner
    • TestDataflowRunner.run_pipeline()
    • TestDataflowRunner.build_console_url()
    • TestDataflowRunner.wait_until_in_state()