Package org.apache.beam.runners.flink.metrics

Class FlinkMetricContainer

java.lang.Object
org.apache.beam.runners.flink.metrics.FlinkMetricContainer
public class FlinkMetricContainer extends Object
Helper class for holding a MetricsContainerImpl and forwarding Beam metrics to Flink accumulators and metrics.

Using accumulators can be turned off because it is memory and network intensive. The accumulator results are only meaningful in batch applications or testing streaming applications which have a defined end. They are not essential during execution because metrics will also be reported using the configured metrics reporter.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    ACCUMULATOR_NAME
     
    protected final org.apache.beam.runners.core.metrics.MetricsContainerStepMap
    metricsContainers
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    FlinkMetricContainer(org.apache.flink.api.common.functions.RuntimeContext runtimeContext)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.flink.metrics.MetricGroup
    getMetricGroup()
     
    org.apache.beam.runners.core.metrics.MetricsContainerImpl
    getMetricsContainer(String stepName)
     
    void
    registerMetricsForPipelineResult()
    This should be called at the end of the Flink job and sets up an accumulator to push the metrics to the PipelineResult.
    void
    updateMetrics(String stepName, List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo> monitoringInfos)
    Update this container with metrics from the passed MetricsApi.MonitoringInfos, and send updates along to Flink's internal metrics framework.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • ACCUMULATOR_NAME

      public static final String ACCUMULATOR_NAME
      See Also:

    • metricsContainers

      protected final org.apache.beam.runners.core.metrics.MetricsContainerStepMap metricsContainers

  • Constructor Details

    • FlinkMetricContainer

      public FlinkMetricContainer(org.apache.flink.api.common.functions.RuntimeContext runtimeContext)

  • Method Details

    • registerMetricsForPipelineResult

      public void registerMetricsForPipelineResult()
      This should be called at the end of the Flink job and sets up an accumulator to push the metrics to the PipelineResult. This should not be called beforehand, to avoid the overhead which accumulators cause at runtime.

    • getMetricGroup

      public org.apache.flink.metrics.MetricGroup getMetricGroup()

    • getMetricsContainer

      public org.apache.beam.runners.core.metrics.MetricsContainerImpl getMetricsContainer(String stepName)

    • updateMetrics

      public void updateMetrics(String stepName, List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo> monitoringInfos)
      Update this container with metrics from the passed MetricsApi.MonitoringInfos, and send updates along to Flink's internal metrics framework.