org.apache.beam.runners.flink.metrics

Class FlinkMetricContainer

java.lang.Object

org.apache.beam.runners.flink.metrics.FlinkMetricContainer

public class FlinkMetricContainer
extends java.lang.Object

Helper class for holding a MetricsContainerImpl and forwarding Beam metrics to Flink accumulators and metrics.

Using accumulators can be turned off because it is memory and network intensive. The accumulator results are only meaningful in batch applications or testing streaming applications which have a defined end. They are not essential during execution because metrics will also be reported using the configured metrics reporter.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	org.apache.beam.runners.flink.metrics.FlinkMetricContainerBase.FlinkDistributionGauge Flink Gauge for DistributionResult.
static class	org.apache.beam.runners.flink.metrics.FlinkMetricContainerBase.FlinkGauge Flink Gauge for GaugeResult.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ACCUMULATOR_NAME
protected org.apache.beam.runners.core.metrics.MetricsContainerStepMap	metricsContainers

Constructor Summary

Constructors

Constructor and Description
FlinkMetricContainer(org.apache.flink.api.common.functions.RuntimeContext runtimeContext)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.metrics.MetricGroup	getMetricGroup()
org.apache.beam.runners.core.metrics.MetricsContainerImpl	getMetricsContainer(java.lang.String stepName)
void	registerMetricsForPipelineResult() This should be called at the end of the Flink job and sets up an accumulator to push the metrics to the PipelineResult.
void	updateMetrics(java.lang.String stepName, java.util.List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo> monitoringInfos) Update this container with metrics from the passed MetricsApi.MonitoringInfos, and send updates along to Flink's internal metrics framework.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ACCUMULATOR_NAME

public static final java.lang.String ACCUMULATOR_NAME

See Also:

Constant Field Values

metricsContainers

protected final org.apache.beam.runners.core.metrics.MetricsContainerStepMap metricsContainers

Constructor Detail

FlinkMetricContainer

public FlinkMetricContainer(org.apache.flink.api.common.functions.RuntimeContext runtimeContext)

Method Detail

registerMetricsForPipelineResult

public void registerMetricsForPipelineResult()

This should be called at the end of the Flink job and sets up an accumulator to push the metrics to the PipelineResult. This should not be called beforehand, to avoid the overhead which accumulators cause at runtime.

getMetricGroup

public org.apache.flink.metrics.MetricGroup getMetricGroup()

getMetricsContainer

public org.apache.beam.runners.core.metrics.MetricsContainerImpl getMetricsContainer(java.lang.String stepName)

updateMetrics

public void updateMetrics(java.lang.String stepName,
                          java.util.List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo> monitoringInfos)

Update this container with metrics from the passed MetricsApi.MonitoringInfos, and send updates along to Flink's internal metrics framework.