org.apache.beam.sdk.io.gcp.firestore

Class RpcQosOptions.Builder

java.lang.Object

org.apache.beam.sdk.io.gcp.firestore.RpcQosOptions.Builder

Enclosing class:

RpcQosOptions

public static final class RpcQosOptions.Builder
extends java.lang.Object

Mutable Builder class for creating instances of RpcQosOptions.

A new instance of RpcQosOptions.Builder can be created via RpcQosOptions.newBuilder().

NOTE: All with methods in this class function as set rather than copy with new value

Method Summary

Modifier and Type	Method and Description
RpcQosOptions	build() Create a new instance of RpcQosOptions from the current builder state.
RpcQosOptions.Builder	withBatchInitialCount(int batchInitialCount) Configure the initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.
RpcQosOptions.Builder	withBatchMaxBytes(long batchMaxBytes) Configure the maximum number of bytes to include in a batch.
RpcQosOptions.Builder	withBatchMaxCount(int batchMaxCount) Configure the maximum number of writes to include in a batch.
RpcQosOptions.Builder	withBatchTargetLatency(Duration batchTargetLatency) Target latency for batch requests.
RpcQosOptions.Builder	withHintMaxNumWorkers(int hintMaxNumWorkers) Provide a hint to the QoS system for the intended max number of workers for a pipeline.
RpcQosOptions.Builder	withInitialBackoff(Duration initialBackoff) Configure the initial backoff duration to be used before retrying a request for the first time.
RpcQosOptions.Builder	withMaxAttempts(int maxAttempts) Configure the maximum number of times a request will be attempted for a complete successful result.
RpcQosOptions.Builder	withOverloadRatio(double overloadRatio) The target ratio between requests sent and successful requests.
RpcQosOptions.Builder	withReportDiagnosticMetrics() Whether additional diagnostic metrics should be reported for a Transform.
RpcQosOptions.Builder	withSamplePeriod(Duration samplePeriod) Configure the length of time sampled request data will be retained.
RpcQosOptions.Builder	withSamplePeriodBucketSize(Duration samplePeriodBucketSize) Configure the size of buckets within the specified samplePeriod.
RpcQosOptions.Builder	withThrottleDuration(Duration throttleDuration) Configure the amount of time an attempt will be throttled if deemed necessary based on previous success rate.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

withMaxAttempts

public RpcQosOptions.Builder withMaxAttempts(int maxAttempts)

Configure the maximum number of times a request will be attempted for a complete successful result.

For a stream based response, the full stream read must complete within the specified number of attempts. Restarting a stream will count as a new attempt.

Default Value: 5

Parameters:

maxAttempts - an int in the range 1 <= maxAttempts <= 5

Returns:

this builder

See Also:

RpcQosOptions.getMaxAttempts()

withInitialBackoff

public RpcQosOptions.Builder withInitialBackoff(Duration initialBackoff)

Configure the initial backoff duration to be used before retrying a request for the first time.

Default Value: 5 sec

Parameters:

initialBackoff - a Duration in the range 5 sec <= initialBackoff <= 2 min

Returns:

this builder

See Also:

RpcQosOptions.getInitialBackoff()

withSamplePeriod

public RpcQosOptions.Builder withSamplePeriod(Duration samplePeriod)

Configure the length of time sampled request data will be retained.

Default Value: 2 min

Parameters:

samplePeriod - a Duration in the range 2 min <= samplePeriod <= 20 min

Returns:

this builder

See Also:

RpcQosOptions.getSamplePeriod()

withSamplePeriodBucketSize

public RpcQosOptions.Builder withSamplePeriodBucketSize(Duration samplePeriodBucketSize)

Configure the size of buckets within the specified samplePeriod.

Default Value: 10 sec

Parameters:

samplePeriodBucketSize - a Duration in the range 10 sec <= samplePeriodBucketSize <= 20 min

Returns:

this builder

See Also:

RpcQosOptions.getSamplePeriodBucketSize()

withOverloadRatio

public RpcQosOptions.Builder withOverloadRatio(double overloadRatio)

The target ratio between requests sent and successful requests. This is "K" in the formula in SRE Book - Handling Overload

Default Value: 1.05

Parameters:

overloadRatio - the target ratio between requests sent and successful requests. A double in the range 1.0 <= overloadRatio <= 1.5

Returns:

this builder

See Also:

RpcQosOptions.getOverloadRatio()

withThrottleDuration

public RpcQosOptions.Builder withThrottleDuration(Duration throttleDuration)

Configure the amount of time an attempt will be throttled if deemed necessary based on previous success rate.

Default value: 5 sec

Parameters:

throttleDuration - a Duration in the range 5 sec <= throttleDuration <= 1 min

Returns:

this builder

See Also:

RpcQosOptions.getThrottleDuration()

withBatchInitialCount

public RpcQosOptions.Builder withBatchInitialCount(int batchInitialCount)

Configure the initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.

Default Value: 20

Parameters:

batchInitialCount - an int in the range 1 <= batchInitialCount <= 500

Returns:

this builder

See Also:

RpcQosOptions.getBatchInitialCount()

withBatchMaxCount

public RpcQosOptions.Builder withBatchMaxCount(int batchMaxCount)

Configure the maximum number of writes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of writes per request may be lower if batchMaxBytes is reached first.

Default Value: 500

Parameters:

batchMaxCount - an int in the range 1 <= batchMaxCount <= 500

Returns:

this builder

See Also:

RpcQosOptions.getBatchMaxCount()

withBatchMaxBytes

public RpcQosOptions.Builder withBatchMaxBytes(long batchMaxBytes)

Configure the maximum number of bytes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of bytes per request may be lower if batchMaxCount is reached first.

Default Value: 9.5 MiB

Parameters:

batchMaxBytes - an int in the range 1 B <= batchMaxBytes <= 9.5 MiB

Returns:

this builder

See Also:

RpcQosOptions.getBatchMaxBytes()

withBatchTargetLatency

public RpcQosOptions.Builder withBatchTargetLatency(Duration batchTargetLatency)

Target latency for batch requests. It aims for a target response time per RPC: the response times for previous RPCs and the number of writes contained in them, calculates a rolling average time-per-write, and chooses the number of writes for future requests to hit the target time.

Default Value: 5 sec

Parameters:

batchTargetLatency - a Duration in the range 5 sec <= batchTargetLatency <= 2 min

Returns:

this builder

See Also:

RpcQosOptions.getBatchTargetLatency()

withHintMaxNumWorkers

public RpcQosOptions.Builder withHintMaxNumWorkers(int hintMaxNumWorkers)

Provide a hint to the QoS system for the intended max number of workers for a pipeline. The provided value can be used to try to scale calculations to values appropriate for the pipeline as a whole.

If you are running your pipeline on Cloud Dataflow this parameter should be set to the same value as maxNumWorkers

Default Value: 500

Parameters:

hintMaxNumWorkers - an int in the range 1 <= hintMaxNumWorkers <= Integer.MAX_VALUE

Returns:

this builder

See Also:

RpcQosOptions.getHintMaxNumWorkers()

withReportDiagnosticMetrics

public RpcQosOptions.Builder withReportDiagnosticMetrics()

Whether additional diagnostic metrics should be reported for a Transform.

If invoked on this builder, additional detailed diagnostic metrics will be reported for the RPC QoS subsystem.

This parameter should be used with care as it will output ~40 additional custom metrics which will count toward any possible pipeline metrics limits (For example Dataflow Custom Metrics limits and Cloud Monitoring custom metrics quota)

Returns:

this builder

See Also:

RpcQosOptions.isShouldReportDiagnosticMetrics()

build

public RpcQosOptions build()

Create a new instance of RpcQosOptions from the current builder state.

All provided values will be validated for non-nullness and that each value falls within its allowed range.

Returns:

a new instance of RpcQosOptions from the current builder state.

Throws:

java.lang.NullPointerException - if any nullable value is null

java.lang.IllegalArgumentException - if any provided value does not fall within its allowed range