DataflowRunner implementation of MetricResults. It is in charge of responding to queries of current metrics by going to the dataflow service.
DataflowMetrics(dataflow_client=None, job_result=None, job_graph=None)[source]¶
Implementation of MetricResults class for the Dataflow runner.
Initialize the Dataflow metrics object.
- dataflow_client – apiclient.DataflowApplicationClient to interact with the dataflow service.
- job_result – DataflowPipelineResult with the state and id information of the job.
- job_graph – apiclient.Job instance to be able to translate between internal step names (e.g. “s2”), and user step names (e.g. “split”).
Print the metric results for the dataflow –job_id and –project.
Instead of running an entire pipeline which takes several minutes, use this main method to display MetricResults for a specific –job_id and –project which takes only a few seconds.