org.apache.beam.sdk.io.aws.sqs

Class SqsIO

java.lang.Object

org.apache.beam.sdk.io.aws.sqs.SqsIO

@Experimental(value=SOURCE_SINK)
public class SqsIO
extends java.lang.Object

An unbounded source for Amazon Simple Queue Service (SQS).

Reading from an SQS queue

The SqsIO SqsIO.Read returns an unbounded PCollection of Message containing the received messages. Note: This source does not currently advance the watermark when no new messages are received.

To configure an SQS source, you have to provide the queueUrl to connect to. The following example illustrates how to configure the source:

 pipeline.apply(SqsIO.read().withQueueUrl(queueUrl))
 

Writing to an SQS queue

The following example illustrates how to use the sink:

 pipeline
   .apply(...) // returns PCollection<SendMessageRequest>
   .apply(SqsIO.write())
 

Additional Configuration

Additional configuration can be provided via AwsOptions from command line args or in code. For example, if you wanted to provide a secret access key via code:

 PipelineOptions pipelineOptions = PipelineOptionsFactory.fromArgs(args).withValidation().create();
 AwsOptions awsOptions = pipelineOptions.as(AwsOptions.class);
 BasicAWSCredentials awsCreds = new BasicAWSCredentials("accesskey", "secretkey");
 awsOptions.setAwsCredentialsProvider(new AWSStaticCredentialsProvider(awsCreds));
 Pipeline pipeline = Pipeline.create(options);
 

For more information on the available options see AwsOptions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SqsIO.Read A PTransform to read/receive messages from SQS.
static class	SqsIO.Write A PTransform to send messages to SQS.

Method Summary

Modifier and Type	Method and Description
static SqsIO.Read	read()
static SqsIO.Write	write()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

read

public static SqsIO.Read read()

write

public static SqsIO.Write write()