org.apache.beam.sdk.io.gcp.pubsub

Class PubsubClient

java.lang.Object

org.apache.beam.sdk.io.gcp.pubsub.PubsubClient

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

PubsubGrpcClient, PubsubJsonClient, PubsubTestClient

public abstract class PubsubClient
extends java.lang.Object
implements java.io.Closeable

An (abstract) helper class for talking to Pubsub via an underlying transport.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PubsubClient.OutgoingMessage A message to be sent to Pubsub.
static class	PubsubClient.ProjectPath Path representing a cloud project id.
static interface	PubsubClient.PubsubClientFactory Factory for creating clients.
static class	PubsubClient.SchemaPath Path representing a Pubsub schema.
static class	PubsubClient.SubscriptionPath Path representing a Pubsub subscription.
static class	PubsubClient.TopicPath Path representing a Pubsub topic.

Constructor Summary

Constructors

Constructor and Description
PubsubClient()

Method Summary

Modifier and Type	Method and Description
abstract int	ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription) Return the ack deadline, in seconds, for subscription.
abstract void	acknowledge(PubsubClient.SubscriptionPath subscription, java.util.List<java.lang.String> ackIds) Acknowldege messages from subscription with ackIds.
PubsubClient.SubscriptionPath	createRandomSubscription(PubsubClient.ProjectPath project, PubsubClient.TopicPath topic, int ackDeadlineSeconds) Create a random subscription for topic.
abstract void	createSchema(PubsubClient.SchemaPath schemaPath, java.lang.String schemaContent, com.google.pubsub.v1.Schema.Type type) Create Schema from Schema definition content.
abstract void	createSubscription(PubsubClient.TopicPath topic, PubsubClient.SubscriptionPath subscription, int ackDeadlineSeconds) Create subscription to topic.
abstract void	createTopic(PubsubClient.TopicPath topic) Create topic.
abstract void	createTopic(PubsubClient.TopicPath topic, PubsubClient.SchemaPath schema) Create {link TopicPath} with PubsubClient.SchemaPath.
abstract void	deleteSchema(PubsubClient.SchemaPath schemaPath) Delete PubsubClient.SchemaPath.
abstract void	deleteSubscription(PubsubClient.SubscriptionPath subscription) Delete subscription.
abstract void	deleteTopic(PubsubClient.TopicPath topic)
protected static long	extractTimestampAttribute(java.lang.String timestampAttribute, @Nullable java.util.Map<java.lang.String,java.lang.String> attributes) Return the timestamp (in ms since unix epoch) to use for a Pubsub message with timestampAttribute and attriutes.
abstract Schema	getSchema(PubsubClient.SchemaPath schemaPath) Return a Beam Schema from the Pub/Sub schema resource, if exists.
abstract PubsubClient.SchemaPath	getSchemaPath(PubsubClient.TopicPath topicPath) Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.
abstract boolean	isEOF() Return true if pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list.
abstract java.util.List<PubsubClient.SubscriptionPath>	listSubscriptions(PubsubClient.ProjectPath project, PubsubClient.TopicPath topic) Return a list of subscriptions for topic in project.
abstract java.util.List<PubsubClient.TopicPath>	listTopics(PubsubClient.ProjectPath project) Return a list of topics for project.
abstract void	modifyAckDeadline(PubsubClient.SubscriptionPath subscription, java.util.List<java.lang.String> ackIds, int deadlineSeconds) Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.
protected static java.lang.Long	parseTimestampAsMsSinceEpoch(java.lang.String timestamp) Return timestamp as ms-since-unix-epoch corresponding to timestamp.
static PubsubClient.ProjectPath	projectPathFromId(java.lang.String projectId)
static PubsubClient.ProjectPath	projectPathFromPath(java.lang.String path)
abstract int	publish(PubsubClient.TopicPath topic, java.util.List<PubsubClient.OutgoingMessage> outgoingMessages) Publish outgoingMessages to Pubsub topic.
abstract java.util.List<org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.IncomingMessage>	pull(long requestTimeMsSinceEpoch, PubsubClient.SubscriptionPath subscription, int batchSize, boolean returnImmediately) Request the next batch of up to batchSize messages from subscription.
static PubsubClient.SchemaPath	schemaPathFromId(java.lang.String projectId, java.lang.String schemaId)
static PubsubClient.SchemaPath	schemaPathFromPath(java.lang.String path)
static PubsubClient.SubscriptionPath	subscriptionPathFromName(java.lang.String projectId, java.lang.String subscriptionName)
static PubsubClient.SubscriptionPath	subscriptionPathFromPath(java.lang.String path)
static PubsubClient.TopicPath	topicPathFromName(java.lang.String projectId, java.lang.String topicName)
static PubsubClient.TopicPath	topicPathFromPath(java.lang.String path)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.io.Closeable

close

Constructor Detail

PubsubClient

public PubsubClient()

Method Detail

parseTimestampAsMsSinceEpoch

protected static java.lang.Long parseTimestampAsMsSinceEpoch(java.lang.String timestamp)

Return timestamp as ms-since-unix-epoch corresponding to timestamp. Throw IllegalArgumentException if timestamp cannot be recognized.

extractTimestampAttribute

protected static long extractTimestampAttribute(java.lang.String timestampAttribute,
                                                @Nullable java.util.Map<java.lang.String,java.lang.String> attributes)

Return the timestamp (in ms since unix epoch) to use for a Pubsub message with timestampAttribute and attriutes.

The message attributes must contain timestampAttribute, and the value of that attribute will be taken as the timestamp.

Throws:

java.lang.IllegalArgumentException - if the timestamp cannot be recognized as a ms-since-unix-epoch or RFC3339 time.

projectPathFromPath

public static PubsubClient.ProjectPath projectPathFromPath(java.lang.String path)

projectPathFromId

public static PubsubClient.ProjectPath projectPathFromId(java.lang.String projectId)

schemaPathFromPath

public static PubsubClient.SchemaPath schemaPathFromPath(java.lang.String path)

schemaPathFromId

public static PubsubClient.SchemaPath schemaPathFromId(java.lang.String projectId,
                                                       java.lang.String schemaId)

subscriptionPathFromPath

public static PubsubClient.SubscriptionPath subscriptionPathFromPath(java.lang.String path)

subscriptionPathFromName

public static PubsubClient.SubscriptionPath subscriptionPathFromName(java.lang.String projectId,
                                                                     java.lang.String subscriptionName)

topicPathFromPath

public static PubsubClient.TopicPath topicPathFromPath(java.lang.String path)

topicPathFromName

public static PubsubClient.TopicPath topicPathFromName(java.lang.String projectId,
                                                       java.lang.String topicName)

publish

public abstract int publish(PubsubClient.TopicPath topic,
                            java.util.List<PubsubClient.OutgoingMessage> outgoingMessages)
                     throws java.io.IOException

Publish outgoingMessages to Pubsub topic. Return number of messages published.

Throws:

java.io.IOException

pull

public abstract java.util.List<org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.IncomingMessage> pull(long requestTimeMsSinceEpoch,
                                                                                                    PubsubClient.SubscriptionPath subscription,
                                                                                                    int batchSize,
                                                                                                    boolean returnImmediately)
                                                                                             throws java.io.IOException

Request the next batch of up to batchSize messages from subscription. Return the received messages, or empty collection if none were available. Does not wait for messages to arrive if returnImmediately is true. Returned messages will record their request time as requestTimeMsSinceEpoch.

Throws:

java.io.IOException

acknowledge

public abstract void acknowledge(PubsubClient.SubscriptionPath subscription,
                                 java.util.List<java.lang.String> ackIds)
                          throws java.io.IOException

Acknowldege messages from subscription with ackIds.

Throws:

java.io.IOException

modifyAckDeadline

public abstract void modifyAckDeadline(PubsubClient.SubscriptionPath subscription,
                                       java.util.List<java.lang.String> ackIds,
                                       int deadlineSeconds)
                                throws java.io.IOException

Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.

Throws:

java.io.IOException

createTopic

public abstract void createTopic(PubsubClient.TopicPath topic)
                          throws java.io.IOException

Create topic.

Throws:

java.io.IOException

createTopic

public abstract void createTopic(PubsubClient.TopicPath topic,
                                 PubsubClient.SchemaPath schema)
                          throws java.io.IOException

Create {link TopicPath} with PubsubClient.SchemaPath.

Throws:

java.io.IOException

deleteTopic

public abstract void deleteTopic(PubsubClient.TopicPath topic)
                          throws java.io.IOException

Throws:

java.io.IOException

listTopics

public abstract java.util.List<PubsubClient.TopicPath> listTopics(PubsubClient.ProjectPath project)
                                                           throws java.io.IOException

Return a list of topics for project.

Throws:

java.io.IOException

createSubscription

public abstract void createSubscription(PubsubClient.TopicPath topic,
                                        PubsubClient.SubscriptionPath subscription,
                                        int ackDeadlineSeconds)
                                 throws java.io.IOException

Create subscription to topic.

Throws:

java.io.IOException

createRandomSubscription

public PubsubClient.SubscriptionPath createRandomSubscription(PubsubClient.ProjectPath project,
                                                              PubsubClient.TopicPath topic,
                                                              int ackDeadlineSeconds)
                                                       throws java.io.IOException

Create a random subscription for topic. Return the PubsubClient.SubscriptionPath. It is the responsibility of the caller to later delete the subscription.

Throws:

java.io.IOException

deleteSubscription

public abstract void deleteSubscription(PubsubClient.SubscriptionPath subscription)
                                 throws java.io.IOException

Delete subscription.

Throws:

java.io.IOException

listSubscriptions

public abstract java.util.List<PubsubClient.SubscriptionPath> listSubscriptions(PubsubClient.ProjectPath project,
                                                                                PubsubClient.TopicPath topic)
                                                                         throws java.io.IOException

Return a list of subscriptions for topic in project.

Throws:

java.io.IOException

ackDeadlineSeconds

public abstract int ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription)
                                throws java.io.IOException

Return the ack deadline, in seconds, for subscription.

Throws:

java.io.IOException

isEOF

public abstract boolean isEOF()

Return true if pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list. Actual clients will return false. Test clients may return true to signal that all expected messages have been pulled and the test may complete.

createSchema

public abstract void createSchema(PubsubClient.SchemaPath schemaPath,
                                  java.lang.String schemaContent,
                                  com.google.pubsub.v1.Schema.Type type)
                           throws java.io.IOException

Create Schema from Schema definition content.

Throws:

java.io.IOException

deleteSchema

public abstract void deleteSchema(PubsubClient.SchemaPath schemaPath)
                           throws java.io.IOException

Delete PubsubClient.SchemaPath.

Throws:

java.io.IOException

getSchemaPath

public abstract PubsubClient.SchemaPath getSchemaPath(PubsubClient.TopicPath topicPath)
                                               throws java.io.IOException

Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.

Throws:

java.io.IOException

getSchema

public abstract Schema getSchema(PubsubClient.SchemaPath schemaPath)
                          throws java.io.IOException

Return a Beam Schema from the Pub/Sub schema resource, if exists.

Throws:

java.io.IOException