Package org.apache.beam.sdk.io.gcp.bigquery

Class BigQuerySinkMetrics

java.lang.Object
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySinkMetrics
public class BigQuerySinkMetrics extends Object
Helper class to create perworker metrics for BigQuery Sink stages.

In general metrics be in the namespace 'BigQuerySink' and have their name formatted as:

'{baseName}*{metricLabelKey1}:{metricLabelVal1};...{metricLabelKeyN}:{metricLabelValN};'

  • Field Details

  • Constructor Details

    • BigQuerySinkMetrics

      public BigQuerySinkMetrics()

  • Method Details

    • appendRowsRowStatusCounter

      public static Counter appendRowsRowStatusCounter(org.apache.beam.sdk.io.gcp.bigquery.BigQuerySinkMetrics.RowStatus rowStatus, String rpcStatus, String tableId)
      Parameters:
      rowStatus - Status of these BigQuery rows.
      rpcStatus - rpcStatus
      tableId - Table pertaining to the write method. Only included in the metric key if 'supportsMetricsDeletion' is enabled.
      Returns:
      Metric that tracks the status of BigQuery rows after making an AppendRows RPC call.

    • throttledTimeCounter

      public static Counter throttledTimeCounter(BigQuerySinkMetrics.RpcMethod method)
      Parameters:
      method - StorageWriteAPI write method.
      Returns:
      Counter that tracks throttled time due to RPC retries.

    • throwableToGRPCCodeString

      public static String throwableToGRPCCodeString(@Nullable Throwable t)
      Converts a Throwable to a gRPC Status code.
      Parameters:
      t - Throwable.
      Returns:
      gRPC status code string or 'UNKNOWN' if 't' is null or does not map to a gRPC error.

    • reportSuccessfulRpcMetrics

      public static void reportSuccessfulRpcMetrics(@Nullable org.apache.beam.sdk.io.gcp.bigquery.RetryManager.Operation.Context<?> c, BigQuerySinkMetrics.RpcMethod method)
      Records RpcRequests counter and RpcLatency histogram for this RPC call. If 'SupportMetricsDeletion' is enabled, RpcRequests counter will have tableId label set to UNKNOWN. RpcRequets counter will have RpcStatus label set to OK.
      Parameters:
      c - Context of successful RPC call.
      method - StorageWriteAPI method associated with this metric.

    • reportSuccessfulRpcMetrics

      public static void reportSuccessfulRpcMetrics(@Nullable org.apache.beam.sdk.io.gcp.bigquery.RetryManager.Operation.Context<?> c, BigQuerySinkMetrics.RpcMethod method, String tableId)
      Records RpcRequests counter and RpcLatency histogram for this RPC call. RpcRequets counter will have RpcStatus label set to OK.
      Parameters:
      c - Context of successful RPC call.
      method - StorageWriteAPI method associated with this metric.
      tableId - Table pertaining to the write method. Only included in the metric key if 'supportsMetricsDeletion' is enabled.

    • reportFailedRPCMetrics

      public static void reportFailedRPCMetrics(@Nullable org.apache.beam.sdk.io.gcp.bigquery.RetryManager.Operation.Context<?> c, BigQuerySinkMetrics.RpcMethod method)
      Records RpcRequests counter and RpcLatency histogram for this RPC call. If 'SupportMetricsDeletion' is enabled, RpcRequests counter will have tableId label set to UNKNOWN. RpcRequets counter will have a RpcStatus label set from the gRPC error.
      Parameters:
      c - Context of successful RPC call.
      method - StorageWriteAPI method associated with this metric.

    • reportFailedRPCMetrics

      public static void reportFailedRPCMetrics(@Nullable org.apache.beam.sdk.io.gcp.bigquery.RetryManager.Operation.Context<?> c, BigQuerySinkMetrics.RpcMethod method, String tableId)
      Records RpcRequests counter and RpcLatency histogram for this RPC call. RpcRequets counter will have a RpcStatus label set from the gRPC error.
      Parameters:
      c - Context of successful RPC call.
      method - StorageWriteAPI method associated with this metric.
      tableId - Table pertaining to the write method. Only included in the metric key if 'supportsMetricsDeletion' is enabled.

    • setSupportStreamingInsertsMetrics

      public static void setSupportStreamingInsertsMetrics(boolean supportStreamingInsertsMetrics)

    • setSupportMetricsDeletion

      public static void setSupportMetricsDeletion(boolean supportMetricsDeletion)