RequestResponseIO (Apache Beam 2.54.0)

java.lang.Object
- org.apache.beam.sdk.transforms.PTransform<PCollection<RequestT>,Result<ResponseT>>
- - org.apache.beam.io.requestresponse.RequestResponseIO<RequestT,ResponseT>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData
```
public class RequestResponseIO<RequestT,ResponseT>
extends PTransform<PCollection<RequestT>,Result<ResponseT>>
```
PTransform for reading from and writing to Web APIs.
RequestResponseIO is recommended for interacting with external systems that offer RPCs that execute relatively quickly and do not offer advance features to make RPC execution efficient.
For systems that offer features for more efficient reading, for example, tracking progress of RPCs, support for splitting RPCs (deduct two or more RPCs which when combined return the same result), consider using the Apache Beam's `Splittable DoFn` interface instead.
Basic Usage
RequestResponseIO minimally requires implementing the Caller interface:
```
 class MyCaller implements Caller<SomeRequest, SomeResponse> {
    public SomeResponse call(SomeRequest request) throws UserCodeExecutionException {
      // calls the API submitting SomeRequest payload and returning SomeResponse
    }
 }
```
Then provide RequestResponseIO's of(org.apache.beam.io.requestresponse.Caller<RequestT, ResponseT>, org.apache.beam.sdk.coders.Coder<ResponseT>) method your Caller implementation.
```
 Coder<SomeResponse> responseCoder = ...
 PCollection<SomeRequest> requests = ...
 Result result = requests.apply(RequestResponseIO.of(new MyCaller(), responseCoder));
 result.getResponses().apply( ... );
 result.getFailures().apply( ... );
 
```
See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static Duration`	`DEFAULT_TIMEOUT` The default `Duration` to wait until completion of user code.
`static java.util.Set<java.lang.Class<? extends UserCodeExecutionException>>`	`REPEATABLE_ERROR_TYPES` `Set` of `UserCodeExecutionException`s that warrant repeating.

Fields inherited from class org.apache.beam.sdk.transforms.PTransform
annotations, name, resourceHints

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`Result<ResponseT>`	`expand(PCollection<RequestT> input)` Override this method to specify how this `PTransform` should be expanded on the given `InputT`.
`static <RequestT,ResponseT> RequestResponseIO<RequestT,ResponseT>`	`of(Caller<RequestT,ResponseT> caller, Coder<ResponseT> responseTCoder)` Instantiates a `RequestResponseIO` with a `Caller` and a `ResponseT` `Coder`.
`static <RequestT,ResponseT,CallerSetupTeardownT extends Caller<RequestT,ResponseT> & SetupTeardown> RequestResponseIO<RequestT,ResponseT>`	`ofCallerAndSetupTeardown(CallerSetupTeardownT implementsCallerAndSetupTeardown, Coder<ResponseT> responseTCoder)` Instantiates a `RequestResponseIO` with a `ResponseT` `Coder` and an implementation of both the `Caller` and `SetupTeardown` interfaces.
`RequestResponseIO<RequestT,ResponseT>`	`withCache(Cache.Pair<RequestT,ResponseT> pair)` Configures `RequestResponseIO` for reading and writing `RequestT` and `ResponseT` pairs using a cache.
`RequestResponseIO<RequestT,ResponseT>`	`withCallShouldBackoff(CallShouldBackoff<ResponseT> value)` Overrides the private no-op implementation of `CallShouldBackoff` that determines whether the `DoFn` should hold `RequestT`s.
`RequestResponseIO<RequestT,ResponseT>`	`withMonitoringConfiguration(Monitoring value)`
`RequestResponseIO<RequestT,ResponseT>`	`withoutRepeater()` Turns off repeat invocations (default is on) of `SetupTeardown` and `Caller`, using the `Repeater`, in the setting of `REPEATABLE_ERROR_TYPES`.
`RequestResponseIO<RequestT,ResponseT>`	`withPreventiveThrottle(PTransform<PCollection<RequestT>,Result<RequestT>> throttle)` Configures `RequestResponseIO` with a `PTransform` that holds back `RequestT`s to prevent quota errors such as HTTP 429 or gRPC RESOURCE_EXHAUSTION errors.
`RequestResponseIO<RequestT,ResponseT>`	`withTimeout(Duration value)` Overrides the `DEFAULT_TIMEOUT` expected timeout of all user custom code.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform
addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - DEFAULT_TIMEOUT
```
public static final Duration DEFAULT_TIMEOUT
```
    The default Duration to wait until completion of user code. A UserCodeTimeoutException is thrown when Caller.call(RequestT), SetupTeardown.setup(), or SetupTeardown.teardown() exceed this timeout.
  - REPEATABLE_ERROR_TYPES
```
public static final java.util.Set<java.lang.Class<? extends UserCodeExecutionException>> REPEATABLE_ERROR_TYPES
```
    Set of UserCodeExecutionExceptions that warrant repeating. Not all errors should be repeat execution such as bad or unauthenticated requests. However, certain errors such as timeouts, remote system or quota exceed errors may not be related to the code but due to the remote system and thus warrant repeating.
- Method Detail
  - of
```
public static <RequestT,ResponseT> RequestResponseIO<RequestT,ResponseT> of(Caller<RequestT,ResponseT> caller,
                                                                            Coder<ResponseT> responseTCoder)
```
    Instantiates a RequestResponseIO with a Caller and a ResponseT Coder. Checks for the Caller's SerializableUtils.ensureSerializable(T) serializable errors.
  - ofCallerAndSetupTeardown
```
public static <RequestT,ResponseT,CallerSetupTeardownT extends Caller<RequestT,ResponseT> & SetupTeardown> RequestResponseIO<RequestT,ResponseT> ofCallerAndSetupTeardown(CallerSetupTeardownT implementsCallerAndSetupTeardown,
                                                                                                                                                                          Coder<ResponseT> responseTCoder)
```
    Instantiates a RequestResponseIO with a ResponseT Coder and an implementation of both the Caller and SetupTeardown interfaces. Checks SerializableUtils.ensureSerializable(T) serializable errors.
  - withTimeout
```
public RequestResponseIO<RequestT,ResponseT> withTimeout(Duration value)
```
    Overrides the DEFAULT_TIMEOUT expected timeout of all user custom code. If user custom code exceeds this timeout, then a UserCodeTimeoutException is thrown. User custom code may throw this exception prior to the configured timeout value on their own.
  - withoutRepeater
```
public RequestResponseIO<RequestT,ResponseT> withoutRepeater()
```
    Turns off repeat invocations (default is on) of SetupTeardown and Caller, using the Repeater, in the setting of REPEATABLE_ERROR_TYPES.
  - withCallShouldBackoff
```
public RequestResponseIO<RequestT,ResponseT> withCallShouldBackoff(CallShouldBackoff<ResponseT> value)
```
    Overrides the private no-op implementation of CallShouldBackoff that determines whether the DoFn should hold RequestTs. Without this configuration, RequestTs are never held; no-op implemented CallShouldBackoff.isTrue() always returns false.
  - withCache
```
public RequestResponseIO<RequestT,ResponseT> withCache(Cache.Pair<RequestT,ResponseT> pair)
```
    Configures RequestResponseIO for reading and writing RequestT and ResponseT pairs using a cache. RequestResponseIO, by default, does not interact with a cache.
```
When reading, the transform Flattens the ResponseT PCollection
 of successful pairs with that resulting from API calls of RequestTs of unsuccessful pairs.
 
```
```
When writing, the transform Flattens the ResponseT PCollection of
    successful pairs with that resulting from API calls of RequestTs.
 
```
  - withMonitoringConfiguration
```
public RequestResponseIO<RequestT,ResponseT> withMonitoringConfiguration(Monitoring value)
```
  - withPreventiveThrottle
```
public RequestResponseIO<RequestT,ResponseT> withPreventiveThrottle(PTransform<PCollection<RequestT>,Result<RequestT>> throttle)
```
    Configures RequestResponseIO with a PTransform that holds back RequestTs to prevent quota errors such as HTTP 429 or gRPC RESOURCE_EXHAUSTION errors.
  - expand
```
public Result<ResponseT> expand(PCollection<RequestT> input)
```
    Description copied from class: PTransform
    
    Override this method to specify how this PTransform should be expanded on the given InputT.
    NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.
    Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
    
    Specified by:
    
    expand in class PTransform<PCollection<RequestT>,Result<ResponseT>>

Class RequestResponseIO<RequestT,ResponseT>

Basic Usage

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

Method Summary

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

Methods inherited from class java.lang.Object

Field Detail

DEFAULT_TIMEOUT

REPEATABLE_ERROR_TYPES

Method Detail

of

ofCallerAndSetupTeardown

withTimeout

withoutRepeater

withCallShouldBackoff

withCache

withMonitoringConfiguration

withPreventiveThrottle

expand