org.apache.beam.sdk.io.aws2.sns

Class SnsIO

java.lang.Object

org.apache.beam.sdk.io.aws2.sns.SnsIO

@Experimental(value=SOURCE_SINK)
public final class SnsIO
extends java.lang.Object

IO to send notifications via SNS.

Writing to SNS

Example usage:

 PCollection<String> data = ...;
 data.apply(SnsIO.<String>write()
     .withTopicArn("topicArn")
     .withPublishRequestBuilder(msg -> PublishRequest.builder().message(msg)));
 

At a minimum you have to provide:

• SNS topic ARN you're going to publish to (optional, but required for most use cases)
• Request builder function to create SNS publish requests from your input

By default, the output PublishResponse contains only the SNS messageId, all other fields are null. If you need to include the full SdkHttpResponse and AwsResponseMetadata, you can call SnsIO.Write.withFullPublishResponse(). If you need the HTTP status code only but no headers, you can use SnsIO.Write.withFullPublishResponseWithoutHeaders().

Configuration of AWS clients

AWS clients for all AWS IOs can be configured using AwsOptions, e.g. --awsRegion=us-west-1. AwsOptions contain reasonable defaults based on default providers for Region and AwsCredentialsProvider.

If you require more advanced configuration, you may change the ClientBuilderFactory using AwsOptions.setClientBuilderFactory(Class).

Configuration for a specific IO can be overwritten using withClientConfiguration(), which also allows to configure the retry behavior for the respective IO.

Retries

Retries for failed requests can be configured using ClientConfiguration.Builder#retry(Consumer) and are handled by the AWS SDK unless there's a partial success (batch requests). The SDK uses a backoff strategy with equal jitter for computing the delay before the next retry.

Note: Once retries are exhausted the error is surfaced to the runner which may then opt to retry the current partition in entirety or abort if the max number of retries of the runner is reached.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SnsIO.RetryConfiguration Deprecated. Use RetryConfiguration instead to delegate retries to the AWS SDK.
protected static class	SnsIO.SnsWriteException Exception class for SNS write exceptions.
static class	SnsIO.Write<T> Implementation of write().
static class	SnsIO.WriteAsync<T> Deprecated. Please use write() to avoid the risk of data loss.

Constructor Summary

Constructors

Constructor and Description
SnsIO()

Method Summary

Modifier and Type	Method and Description
static <T> SnsIO.Write<T>	write()
static <T> SnsIO.WriteAsync<T>	writeAsync() Deprecated. Please use write() to avoid the risk of data loss.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnsIO

public SnsIO()

Method Detail

write

public static <T> SnsIO.Write<T> write()

writeAsync

@Deprecated
public static <T> SnsIO.WriteAsync<T> writeAsync()

Deprecated. Please use write() to avoid the risk of data loss.

See Also:

Issue #21366, BEAM-13203