apache_beam.metrics.monitoring_infos module

apache_beam.metrics.monitoring_infos.extract_counter_value(monitoring_info_proto)[source]

Returns the counter value of the monitoring info.

apache_beam.metrics.monitoring_infos.extract_gauge_value(monitoring_info_proto)[source]

Returns a tuple containing (timestamp, value)

apache_beam.metrics.monitoring_infos.extract_distribution(monitoring_info_proto)[source]

Returns a tuple of (count, sum, min, max).

Parameters:

proto – The monitoring info for the distribution.

apache_beam.metrics.monitoring_infos.extract_string_set_value(monitoring_info_proto)[source]
apache_beam.metrics.monitoring_infos.create_labels(ptransform=None, namespace=None, name=None, pcollection=None)[source]

Create the label dictionary based on the provided values.

Parameters:
  • ptransform – The ptransform id used as a label.

  • pcollection – The pcollection id used as a label.

apache_beam.metrics.monitoring_infos.int64_user_counter(namespace, name, metric, ptransform=None) MonitoringInfo[source]

Return the counter monitoring info for the specifed URN, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • metric – The payload field to use in the monitoring info or an int value.

  • ptransform – The ptransform id used as a label.

apache_beam.metrics.monitoring_infos.int64_counter(urn, metric, ptransform=None, pcollection=None, labels=None) MonitoringInfo[source]

Return the counter monitoring info for the specifed URN, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • metric – The payload field to use in the monitoring info or an int value.

  • ptransform – The ptransform id used as a label.

  • pcollection – The pcollection id used as a label.

apache_beam.metrics.monitoring_infos.int64_user_distribution(namespace, name, metric, ptransform=None) MonitoringInfo[source]

Return the distribution monitoring info for the URN, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • metric – The DistributionData for the metric.

  • ptransform – The ptransform id used as a label.

apache_beam.metrics.monitoring_infos.int64_distribution(urn, metric, ptransform=None, pcollection=None) MonitoringInfo[source]

Return a distribution monitoring info for the URN, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • metric – The DistributionData for the metric.

  • ptransform – The ptransform id used as a label.

  • pcollection – The pcollection id used as a label.

apache_beam.metrics.monitoring_infos.int64_user_gauge(namespace, name, metric, ptransform=None) MonitoringInfo[source]

Return the gauge monitoring info for the URN, metric and labels.

Parameters:
  • namespace – User-defined namespace of gauge metric.

  • name – Name of gauge metric.

  • metric – The GaugeData containing the metrics.

  • ptransform – The ptransform id used as a label.

apache_beam.metrics.monitoring_infos.int64_gauge(urn, metric, ptransform=None) MonitoringInfo[source]

Return the gauge monitoring info for the URN, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • metric – An int representing the value. The current time will be used for the timestamp.

  • ptransform – The ptransform id used as a label.

apache_beam.metrics.monitoring_infos.user_set_string(namespace, name, metric, ptransform=None)[source]

Return the string set monitoring info for the URN, metric and labels.

Parameters:
  • namespace – User-defined namespace of StringSet.

  • name – Name of StringSet.

  • metric – The StringSetData representing the metrics.

  • ptransform – The ptransform id used as a label.

apache_beam.metrics.monitoring_infos.create_monitoring_info(urn, type_urn, payload, labels=None) MonitoringInfo[source]

Return the gauge monitoring info for the URN, type, metric and labels.

Parameters:
  • urn – The URN of the monitoring info/metric.

  • type_urn – The URN of the type of the monitoring info/metric. i.e. beam:metrics:sum_int_64, beam:metrics:latest_int_64.

  • payload – The payload field to use in the monitoring info.

  • labels – The label dictionary to use in the MonitoringInfo.

apache_beam.metrics.monitoring_infos.is_counter(monitoring_info_proto)[source]

Returns true if the monitoring info is a coutner metric.

apache_beam.metrics.monitoring_infos.is_gauge(monitoring_info_proto)[source]

Returns true if the monitoring info is a gauge metric.

apache_beam.metrics.monitoring_infos.is_distribution(monitoring_info_proto)[source]

Returns true if the monitoring info is a distrbution metric.

apache_beam.metrics.monitoring_infos.is_string_set(monitoring_info_proto)[source]

Returns true if the monitoring info is a StringSet metric.

apache_beam.metrics.monitoring_infos.is_user_monitoring_info(monitoring_info_proto)[source]

Returns true if the monitoring info is a user metric.

apache_beam.metrics.monitoring_infos.extract_metric_result_map_value(monitoring_info_proto) None | int | DistributionResult | GaugeResult | set[source]

Returns the relevant GaugeResult, DistributionResult or int value for counter metric, set for StringSet metric.

These are the proper format for use in the MetricResult.query() result.

apache_beam.metrics.monitoring_infos.parse_namespace_and_name(monitoring_info_proto)[source]

Returns the (namespace, name) tuple of the URN in the monitoring info.

apache_beam.metrics.monitoring_infos.get_step_name(monitoring_info_proto)[source]

Returns a step name for the given monitoring info or None if step name cannot be specified.

apache_beam.metrics.monitoring_infos.to_key(monitoring_info_proto: MonitoringInfo) FrozenSet[Hashable][source]

Returns a key based on the URN and labels.

This is useful in maps to prevent reporting the same MonitoringInfo twice.

apache_beam.metrics.monitoring_infos.sum_payload_combiner(payload_a, payload_b)[source]
apache_beam.metrics.monitoring_infos.distribution_payload_combiner(payload_a, payload_b)[source]
apache_beam.metrics.monitoring_infos.consolidate(metrics, key=<function to_key>)[source]