org.apache.beam.sdk.metrics

Class Metrics

java.lang.Object

org.apache.beam.sdk.metrics.Metrics

public class Metrics
extends java.lang.Object

The Metrics is a utility class for producing various kinds of metrics for reporting properties of an executing pipeline.

Metrics are created by calling one of the static methods in this class. Each metric is associated with a namespace and a name. The namespace allows grouping related metrics together based on the definition while also disambiguating common names based on where they are defined.

Reported metrics are implicitly scoped to the transform within the pipeline that reported them. This allows reporting the same metric name in multiple places and identifying the value each transform reported, as well as aggregating the metric across

It is runner-dependent whether Metrics are accessible during pipeline execution or only after jobs have completed.

Example:

 class SomeDoFn extends DoFn<String, String> {
   private Counter counter = Metrics.counter(SomeDoFn.class, "my-counter");

   @ProcessElement
   public void processElement(ProcessContext c) {
     counter.inc();
     Metrics.counter(SomeDoFn.class, "my-counter2").inc();
   }
 }

See MetricResults (available from the PipelineResults interface) for an example off how to query metrics.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	THROTTLE_TIME_COUNTER_NAME
static java.lang.String	THROTTLE_TIME_NAMESPACE

Method Summary

Modifier and Type	Method and Description
static Counter	counter(java.lang.Class<?> namespace, java.lang.String name) Create a metric that can be incremented and decremented, and is aggregated by taking the sum.
static Counter	counter(java.lang.String namespace, java.lang.String name) Create a metric that can be incremented and decremented, and is aggregated by taking the sum.
static Distribution	distribution(java.lang.Class<?> namespace, java.lang.String name) Create a metric that records various statistics about the distribution of reported values.
static Distribution	distribution(java.lang.String namespace, java.lang.String name) Create a metric that records various statistics about the distribution of reported values.
static Gauge	gauge(java.lang.Class<?> namespace, java.lang.String name) Create a metric that can have its new value set, and is aggregated by taking the last reported value.
static Gauge	gauge(java.lang.String namespace, java.lang.String name) Create a metric that can have its new value set, and is aggregated by taking the last reported value.
static void	setDefaultPipelineOptions(PipelineOptions options) Initialize metrics flags if not already done so.
static StringSet	stringSet(java.lang.Class<?> namespace, java.lang.String name) Create a metric that accumulates and reports set of unique string values.
static StringSet	stringSet(java.lang.String namespace, java.lang.String name) Create a metric that accumulates and reports set of unique string values.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

THROTTLE_TIME_NAMESPACE

public static final java.lang.String THROTTLE_TIME_NAMESPACE

See Also:

Constant Field Values

THROTTLE_TIME_COUNTER_NAME

public static final java.lang.String THROTTLE_TIME_COUNTER_NAME

See Also:

Constant Field Values

Method Detail

setDefaultPipelineOptions

@Internal
public static void setDefaultPipelineOptions(PipelineOptions options)

Initialize metrics flags if not already done so.

Should be called by worker at worker harness initialization. Should not be called by user code (and it does not have an effect as the initialization completed before).

counter

public static Counter counter(java.lang.String namespace,
                              java.lang.String name)

Create a metric that can be incremented and decremented, and is aggregated by taking the sum.

counter

public static Counter counter(java.lang.Class<?> namespace,
                              java.lang.String name)

Create a metric that can be incremented and decremented, and is aggregated by taking the sum.

distribution

public static Distribution distribution(java.lang.String namespace,
                                        java.lang.String name)

Create a metric that records various statistics about the distribution of reported values.

distribution

public static Distribution distribution(java.lang.Class<?> namespace,
                                        java.lang.String name)

Create a metric that records various statistics about the distribution of reported values.

gauge

public static Gauge gauge(java.lang.String namespace,
                          java.lang.String name)

Create a metric that can have its new value set, and is aggregated by taking the last reported value.

gauge

public static Gauge gauge(java.lang.Class<?> namespace,
                          java.lang.String name)

Create a metric that can have its new value set, and is aggregated by taking the last reported value.

stringSet

public static StringSet stringSet(java.lang.String namespace,
                                  java.lang.String name)

Create a metric that accumulates and reports set of unique string values.

stringSet

public static StringSet stringSet(java.lang.Class<?> namespace,
                                  java.lang.String name)

Create a metric that accumulates and reports set of unique string values.