Package org.apache.beam.sdk.io.gcp.firestore

Class RpcQosOptions

java.lang.Object

org.apache.beam.sdk.io.gcp.firestore.RpcQosOptions

All Implemented Interfaces:

Serializable, HasDisplayData

@Immutable
@ThreadSafe
public final class RpcQosOptions
extends Object
implements Serializable, HasDisplayData

Quality of Service manager options for Firestore RPCs.

Every RPC which is sent to Cloud Firestore is subject to QoS considerations. Successful, failed, attempted requests are all tracked and directly drive the determination of when to attempt an RPC.

Configuration of options can be accomplished by passing an instances of RpcQosOptions to the withRpcQosOptions method of each Builder available in FirestoreV1.

A new instance of RpcQosOptions.Builder can be created via newBuilder(). A default instance of RpcQosOptions can be created via defaultOptions().

See Also:

• FirestoreV1
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
• Standard limits
• Designing for scale
• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	RpcQosOptions.Builder	Mutable Builder class for creating instances of RpcQosOptions.

Method Summary

Modifier and Type	Method	Description
static RpcQosOptions	defaultOptions()	Factory method to return a new instance of RpcQosOptions with all default values.
boolean	equals(@Nullable Object o)
int	getBatchInitialCount()	The initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.
long	getBatchMaxBytes()	The maximum number of bytes to include in a batch.
int	getBatchMaxCount()	The maximum number of writes to include in a batch.
Duration	getBatchTargetLatency()	Target latency for batch requests.
int	getHintMaxNumWorkers()	A hint to the QoS system for the intended max number of workers for a pipeline.
Duration	getInitialBackoff()	The initial backoff duration to be used before retrying a request for the first time.
int	getMaxAttempts()	The maximum number of times a request will be attempted for a complete successful result.
double	getOverloadRatio()	The target ratio between requests sent and successful requests.
Duration	getSamplePeriod()	The length of time sampled request data will be retained.
Duration	getSamplePeriodBucketSize()	The size of buckets within the specified samplePeriod.
Duration	getThrottleDuration()	The amount of time an attempt will be throttled if deemed necessary based on previous success rate.
int	hashCode()
boolean	isShouldReportDiagnosticMetrics()	Whether additional diagnostic metrics should be reported for a Transform.
static RpcQosOptions.Builder	newBuilder()	Factory method to return a new instance of RpcQosOptions.Builder with all values set to their initial default values.
void	populateDisplayData(DisplayData.Builder builder)	Register display data for the given transform or component.
RpcQosOptions.Builder	toBuilder()	Create a new RpcQosOptions.Builder initialized with the values from this instance.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Method Details

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from interface: HasDisplayData

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

Specified by:

populateDisplayData in interface HasDisplayData

Parameters:

builder - The builder to populate with display data.

See Also:

• HasDisplayData

getMaxAttempts

public int getMaxAttempts()

The maximum number of times a request will be attempted for a complete successful result.

For a stream based response, the full stream read must complete within the specified number of attempts. Restarting a stream will count as a new attempt.

Default Value: 5

See Also:

• RpcQosOptions.Builder.withMaxAttempts(int)

getInitialBackoff

public Duration getInitialBackoff()

The initial backoff duration to be used before retrying a request for the first time.

Default Value: 5 sec

See Also:

• RpcQosOptions.Builder.withInitialBackoff(Duration)

getSamplePeriod

public Duration getSamplePeriod()

The length of time sampled request data will be retained.

Default Value: 2 min

See Also:

• RpcQosOptions.Builder.withSamplePeriod(Duration)

getSamplePeriodBucketSize

public Duration getSamplePeriodBucketSize()

The size of buckets within the specified samplePeriod.

Default Value: 10 sec

See Also:

• RpcQosOptions.Builder.withSamplePeriodBucketSize(Duration)

getOverloadRatio

public double getOverloadRatio()

The target ratio between requests sent and successful requests. This is "K" in the formula in SRE Book - Handling Overload

Default Value: 1.05

See Also:

• RpcQosOptions.Builder.withOverloadRatio(double)

getThrottleDuration

public Duration getThrottleDuration()

The amount of time an attempt will be throttled if deemed necessary based on previous success rate.

Default value: 5 sec

See Also:

• RpcQosOptions.Builder.withThrottleDuration(Duration)

getBatchInitialCount

public int getBatchInitialCount()

The initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.

Default Value: 20

See Also:

• RpcQosOptions.Builder.withBatchInitialCount(int)

getBatchMaxCount

public int getBatchMaxCount()

The maximum number of writes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of writes per request may be lower if we reach batchMaxBytes.

Default Value: 500

See Also:

• RpcQosOptions.Builder.withBatchMaxCount(int)

getBatchMaxBytes

public long getBatchMaxBytes()

The maximum number of bytes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of bytes per request may be lower if batchMaxCount is reached first.

Default Value: 9.5 MiB

See Also:

• RpcQosOptions.Builder.withBatchMaxBytes(long)

getBatchTargetLatency

public Duration getBatchTargetLatency()

Target latency for batch requests. It aims for a target response time per RPC: the response times for previous RPCs and the number of writes contained in them, calculates a rolling average time-per-write, and chooses the number of writes for future requests to hit the target time.

Default Value: 5 sec

See Also:

• RpcQosOptions.Builder.withBatchTargetLatency(Duration)

getHintMaxNumWorkers

public int getHintMaxNumWorkers()

A hint to the QoS system for the intended max number of workers for a pipeline. The provided value can be used to try to scale calculations to values appropriate for the pipeline as a whole.

If you are running your pipeline on Cloud Dataflow this parameter should be set to the same value as maxNumWorkers

Default Value: 500

See Also:

• RpcQosOptions.Builder.withBatchMaxCount(int)

isShouldReportDiagnosticMetrics

public boolean isShouldReportDiagnosticMetrics()

Whether additional diagnostic metrics should be reported for a Transform.

If true, additional detailed diagnostic metrics will be reported for the RPC QoS subsystem.

This parameter should be used with care as it will output ~40 additional custom metrics which will count toward any possible pipeline metrics limits (For example Dataflow Custom Metrics limits and Cloud Monitoring custom metrics quota)

Default Value: false

See Also:

• RpcQosOptions.Builder.withReportDiagnosticMetrics()

toBuilder

public RpcQosOptions.Builder toBuilder()

Create a new RpcQosOptions.Builder initialized with the values from this instance.

Returns:

a new RpcQosOptions.Builder initialized with the values from this instance.

equals

public boolean equals(@Nullable Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object

defaultOptions

public static RpcQosOptions defaultOptions()

Factory method to return a new instance of RpcQosOptions with all default values.

Returns:

New instance of RpcQosOptions with all default values

See Also:

• newBuilder()

newBuilder

public static RpcQosOptions.Builder newBuilder()

Factory method to return a new instance of RpcQosOptions.Builder with all values set to their initial default values.

Returns:

New instance of RpcQosOptions.Builder with all values set to their initial default values

See Also:

• defaultOptions()