Package org.apache.beam.sdk.io.gcp.firestore

Class RpcQosOptions.Builder

java.lang.Object
org.apache.beam.sdk.io.gcp.firestore.RpcQosOptions.Builder
Enclosing class:
RpcQosOptions
public static final class RpcQosOptions.Builder extends Object
Mutable Builder class for creating instances of RpcQosOptions.

A new instance of RpcQosOptions.Builder can be created via RpcQosOptions.newBuilder().

NOTE: All with methods in this class function as set rather than copy with new value

  • Method Details

    • withMaxAttempts

      public RpcQosOptions.Builder withMaxAttempts(int maxAttempts)
      Configure the maximum number of times a request will be attempted for a complete successful result.

      For a stream based response, the full stream read must complete within the specified number of attempts. Restarting a stream will count as a new attempt.

      Default Value: 5

      Parameters:
      maxAttempts - an int in the range 1 invalid input: '<'= maxAttempts invalid input: '<'= 5
      Returns:
      this builder
      See Also:

    • withInitialBackoff

      public RpcQosOptions.Builder withInitialBackoff(Duration initialBackoff)
      Configure the initial backoff duration to be used before retrying a request for the first time.

      Default Value: 5 sec

      Parameters:
      initialBackoff - a Duration in the range 5 sec invalid input: '<'= initialBackoff invalid input: '<'= 2 min
      Returns:
      this builder
      See Also:

    • withSamplePeriod

      public RpcQosOptions.Builder withSamplePeriod(Duration samplePeriod)
      Configure the length of time sampled request data will be retained.

      Default Value: 2 min

      Parameters:
      samplePeriod - a Duration in the range 2 min invalid input: '<'= samplePeriod invalid input: '<'= 20 min
      Returns:
      this builder
      See Also:

    • withSamplePeriodBucketSize

      public RpcQosOptions.Builder withSamplePeriodBucketSize(Duration samplePeriodBucketSize)
      Configure the size of buckets within the specified samplePeriod.

      Default Value: 10 sec

      Parameters:
      samplePeriodBucketSize - a Duration in the range 10 sec invalid input: '<'= samplePeriodBucketSize invalid input: '<'= 20 min
      Returns:
      this builder
      See Also:

    • withOverloadRatio

      public RpcQosOptions.Builder withOverloadRatio(double overloadRatio)
      The target ratio between requests sent and successful requests. This is "K" in the formula in SRE Book - Handling Overload

      Default Value: 1.05

      Parameters:
      overloadRatio - the target ratio between requests sent and successful requests. A double in the range 1.0 invalid input: '<'= overloadRatio invalid input: '<'= 1.5
      Returns:
      this builder
      See Also:

    • withThrottleDuration

      public RpcQosOptions.Builder withThrottleDuration(Duration throttleDuration)
      Configure the amount of time an attempt will be throttled if deemed necessary based on previous success rate.

      Default value: 5 sec

      Parameters:
      throttleDuration - a Duration in the range 5 sec invalid input: '<'= throttleDuration invalid input: '<'= 1 min
      Returns:
      this builder
      See Also:

    • withBatchInitialCount

      public RpcQosOptions.Builder withBatchInitialCount(int batchInitialCount)
      Configure the initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.

      Default Value: 20

      Parameters:
      batchInitialCount - an int in the range 1 invalid input: '<'= batchInitialCount invalid input: '<'= 500
      Returns:
      this builder
      See Also:

    • withBatchMaxCount

      public RpcQosOptions.Builder withBatchMaxCount(int batchMaxCount)
      Configure the maximum number of writes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of writes per request may be lower if batchMaxBytes is reached first.

      Default Value: 500

      Parameters:
      batchMaxCount - an int in the range 1 invalid input: '<'= batchMaxCount invalid input: '<'= 500
      Returns:
      this builder
      See Also:

    • withBatchMaxBytes

      public RpcQosOptions.Builder withBatchMaxBytes(long batchMaxBytes)
      Configure the maximum number of bytes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of bytes per request may be lower if batchMaxCount is reached first.

      Default Value: 9.5 MiB

      Parameters:
      batchMaxBytes - an int in the range 1 B invalid input: '<'= batchMaxBytes invalid input: '<'= 9.5 MiB
      Returns:
      this builder
      See Also:

    • withBatchTargetLatency

      public RpcQosOptions.Builder withBatchTargetLatency(Duration batchTargetLatency)
      Target latency for batch requests. It aims for a target response time per RPC: the response times for previous RPCs and the number of writes contained in them, calculates a rolling average time-per-write, and chooses the number of writes for future requests to hit the target time.

      Default Value: 5 sec

      Parameters:
      batchTargetLatency - a Duration in the range 5 sec invalid input: '<'= batchTargetLatency invalid input: '<'= 2 min
      Returns:
      this builder
      See Also:

    • withHintMaxNumWorkers

      public RpcQosOptions.Builder withHintMaxNumWorkers(int hintMaxNumWorkers)
      Provide a hint to the QoS system for the intended max number of workers for a pipeline. The provided value can be used to try to scale calculations to values appropriate for the pipeline as a whole.

      If you are running your pipeline on Cloud Dataflow this parameter should be set to the same value as maxNumWorkers

      Default Value: 500

      Parameters:
      hintMaxNumWorkers - an int in the range 1 invalid input: '<'= hintMaxNumWorkers invalid input: '<'= Integer.MAX_VALUE
      Returns:
      this builder
      See Also:

    • withReportDiagnosticMetrics

      public RpcQosOptions.Builder withReportDiagnosticMetrics()
      Whether additional diagnostic metrics should be reported for a Transform.

      If invoked on this builder, additional detailed diagnostic metrics will be reported for the RPC QoS subsystem.

      This parameter should be used with care as it will output ~40 additional custom metrics which will count toward any possible pipeline metrics limits (For example Dataflow Custom Metrics limits and Cloud Monitoring custom metrics quota)

      Returns:
      this builder
      See Also:

    • build

      public RpcQosOptions build()
      Create a new instance of RpcQosOptions from the current builder state.

      All provided values will be validated for non-nullness and that each value falls within its allowed range.

      Returns:
      a new instance of RpcQosOptions from the current builder state.
      Throws:
      NullPointerException - if any nullable value is null
      IllegalArgumentException - if any provided value does not fall within its allowed range