Package org.apache.beam.sdk.io.kafka

Class KafkaMetrics.KafkaMetricsImpl

java.lang.Object

org.apache.beam.sdk.io.kafka.KafkaMetrics.KafkaMetricsImpl

All Implemented Interfaces:

KafkaMetrics

Enclosing interface:

KafkaMetrics

public abstract static class KafkaMetrics.KafkaMetricsImpl
extends Object
implements KafkaMetrics

Metrics of a batch of RPCs. Member variables are thread safe; however, this class does not have atomicity across member variables.

Expected usage: A number of threads record metrics in an instance of this class with the member methods. Afterwards, a single thread should call updateStreamingInsertsMetrics which will export all counters metrics and RPC latency distribution metrics to the underlying perWorkerMetrics container. Afterwards, metrics should not be written/read from this object.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.io.kafka.KafkaMetrics

KafkaMetrics.KafkaMetricsImpl, KafkaMetrics.NoOpKafkaMetrics

Constructor Summary

Constructors

Constructor	Description
KafkaMetricsImpl()

Method Summary

Modifier and Type	Method	Description
static KafkaMetrics.KafkaMetricsImpl	create()
void	flushBufferedMetrics()	Export all metrics recorded in this instance to the underlying perWorkerMetrics containers.
void	updateBacklogBytes(String topicName, int partitionId, long backlog)	This is for tracking backlog bytes to be added to the Metric Container at a later time.
void	updateSuccessfulRpcMetrics(String topic, Duration elapsedTime)	Record the rpc status and latency of a successful Kafka poll RPC call.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

KafkaMetricsImpl

public KafkaMetricsImpl()

Method Details

create

public static KafkaMetrics.KafkaMetricsImpl create()

updateSuccessfulRpcMetrics

public void updateSuccessfulRpcMetrics(String topic,
 Duration elapsedTime)

Record the rpc status and latency of a successful Kafka poll RPC call.

TODO(naireenhussain): It's possible that `isWritable().get()` is called before it's set to false in another thread, allowing an extraneous measurement to slip in, so perTopicRpcLatencies() isn't necessarily thread safe. One way to address this would be to add synchronized blocks to ensure that there is only one thread ever reading/modifying the perTopicRpcLatencies() map.

Specified by:

updateSuccessfulRpcMetrics in interface KafkaMetrics

updateBacklogBytes

public void updateBacklogBytes(String topicName,
 int partitionId,
 long backlog)

This is for tracking backlog bytes to be added to the Metric Container at a later time.

Specified by:

updateBacklogBytes in interface KafkaMetrics

Parameters:

topicName - topicName

partitionId - partitionId

backlog - backlog for the specific partitionID of topicName

flushBufferedMetrics

public void flushBufferedMetrics()

Export all metrics recorded in this instance to the underlying perWorkerMetrics containers. This function will only report metrics once per instance. Subsequent calls to this function will no-op.

Specified by:

flushBufferedMetrics in interface KafkaMetrics