@Experimental(value=SOURCE_SINK) public class SqsIO extends java.lang.Object
The SqsIO
SqsIO.Read
returns an unbounded PCollection
of Message
containing the received messages. Note: This source
does not currently advance the watermark when no new messages are received.
To configure an SQS source, you have to provide the queueUrl to connect to. The following example illustrates how to configure the source:
pipeline.apply(SqsIO.read().withQueueUrl(queueUrl))
The following example illustrates how to use the sink:
pipeline
.apply(...) // returns PCollection<SendMessageRequest>
.apply(SqsIO.write())
Additional configuration can be provided via AwsOptions
from command line args or in
code. For example, if you wanted to provide a secret access key via code:
PipelineOptions pipelineOptions = PipelineOptionsFactory.fromArgs(args).withValidation().create();
AwsOptions awsOptions = pipelineOptions.as(AwsOptions.class);
BasicAWSCredentials awsCreds = new BasicAWSCredentials("accesskey", "secretkey");
awsOptions.setAwsCredentialsProvider(new AWSStaticCredentialsProvider(awsCreds));
Pipeline pipeline = Pipeline.create(options);
For more information on the available options see AwsOptions
.
Modifier and Type | Class and Description |
---|---|
static class |
SqsIO.Read
A
PTransform to read/receive messages from SQS. |
static class |
SqsIO.Write
A
PTransform to send messages to SQS. |
Modifier and Type | Method and Description |
---|---|
static SqsIO.Read |
read() |
static SqsIO.Write |
write() |
public static SqsIO.Read read()
public static SqsIO.Write write()