Package org.apache.beam.sdk.io.gcp.pubsub

Class PubsubClient

java.lang.Object

org.apache.beam.sdk.io.gcp.pubsub.PubsubClient

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

PubsubGrpcClient, PubsubJsonClient, PubsubTestClient

public abstract class PubsubClient
extends Object
implements Closeable

An (abstract) helper class for talking to Pubsub via an underlying transport.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	PubsubClient.IncomingMessage	A message received from Pubsub.
static class	PubsubClient.OutgoingMessage	A message to be sent to Pubsub.
static class	PubsubClient.ProjectPath	Path representing a cloud project id.
static interface	PubsubClient.PubsubClientFactory	Factory for creating clients.
static class	PubsubClient.SchemaPath	Path representing a Pubsub schema.
static class	PubsubClient.SubscriptionPath	Path representing a Pubsub subscription.
static class	PubsubClient.TopicPath	Path representing a Pubsub topic.

Constructor Summary

Constructors

Constructor	Description
PubsubClient()

Method Summary

Modifier and Type	Method	Description
abstract int	ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription)	Return the ack deadline, in seconds, for subscription.
abstract void	acknowledge(PubsubClient.SubscriptionPath subscription, List<String> ackIds)	Acknowldege messages from subscription with ackIds.
PubsubClient.SubscriptionPath	createRandomSubscription(PubsubClient.ProjectPath project, PubsubClient.TopicPath topic, int ackDeadlineSeconds)	Create a random subscription for topic.
abstract void	createSchema(PubsubClient.SchemaPath schemaPath, String schemaContent, com.google.pubsub.v1.Schema.Type type)	Create Schema from Schema definition content.
abstract void	createSubscription(PubsubClient.TopicPath topic, PubsubClient.SubscriptionPath subscription, int ackDeadlineSeconds)	Create subscription to topic.
abstract void	createTopic(PubsubClient.TopicPath topic)	Create topic.
abstract void	createTopic(PubsubClient.TopicPath topic, PubsubClient.SchemaPath schema)	Create {link TopicPath} with PubsubClient.SchemaPath.
abstract void	deleteSchema(PubsubClient.SchemaPath schemaPath)	Delete PubsubClient.SchemaPath.
abstract void	deleteSubscription(PubsubClient.SubscriptionPath subscription)	Delete subscription.
abstract void	deleteTopic(PubsubClient.TopicPath topic)
protected static long	extractTimestampAttribute(String timestampAttribute, @Nullable Map<String,String> attributes)	Return the timestamp (in ms since unix epoch) to use for a Pubsub message with timestampAttribute and attriutes.
abstract Schema	getSchema(PubsubClient.SchemaPath schemaPath)	Return a Beam Schema from the Pub/Sub schema resource, if exists.
abstract PubsubClient.SchemaPath	getSchemaPath(PubsubClient.TopicPath topicPath)	Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.
abstract boolean	isEOF()	Return true if pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list.
abstract boolean	isTopicExists(PubsubClient.TopicPath topic)	Return true if topic exists.
abstract List<PubsubClient.SubscriptionPath>	listSubscriptions(PubsubClient.ProjectPath project, PubsubClient.TopicPath topic)	Return a list of subscriptions for topic in project.
abstract List<PubsubClient.TopicPath>	listTopics(PubsubClient.ProjectPath project)	Return a list of topics for project.
abstract void	modifyAckDeadline(PubsubClient.SubscriptionPath subscription, List<String> ackIds, int deadlineSeconds)	Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.
protected static Long	parseTimestampAsMsSinceEpoch(String timestamp)	Return timestamp as ms-since-unix-epoch corresponding to timestamp.
static PubsubClient.ProjectPath	projectPathFromId(String projectId)
static PubsubClient.ProjectPath	projectPathFromPath(String path)
abstract int	publish(PubsubClient.TopicPath topic, List<PubsubClient.OutgoingMessage> outgoingMessages)	Publish outgoingMessages to Pubsub topic.
abstract List<PubsubClient.IncomingMessage>	pull(long requestTimeMsSinceEpoch, PubsubClient.SubscriptionPath subscription, int batchSize, boolean returnImmediately)	Request the next batch of up to batchSize messages from subscription.
static PubsubClient.SchemaPath	schemaPathFromId(String projectId, String schemaId)
static PubsubClient.SchemaPath	schemaPathFromPath(String path)
static PubsubClient.SubscriptionPath	subscriptionPathFromName(String projectId, String subscriptionName)
static PubsubClient.SubscriptionPath	subscriptionPathFromPath(String path)
static PubsubClient.TopicPath	topicPathFromName(String projectId, String topicName)
static PubsubClient.TopicPath	topicPathFromPath(String path)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.io.Closeable

close

Constructor Details

PubsubClient

public PubsubClient()

Method Details

parseTimestampAsMsSinceEpoch

protected static Long parseTimestampAsMsSinceEpoch(String timestamp)

Return timestamp as ms-since-unix-epoch corresponding to timestamp. Throw IllegalArgumentException if timestamp cannot be recognized.

extractTimestampAttribute

protected static long extractTimestampAttribute(String timestampAttribute,
 @Nullable Map<String,String> attributes)

Return the timestamp (in ms since unix epoch) to use for a Pubsub message with timestampAttribute and attriutes.

The message attributes must contain timestampAttribute, and the value of that attribute will be taken as the timestamp.

Throws:

IllegalArgumentException - if the timestamp cannot be recognized as a ms-since-unix-epoch or RFC3339 time.

projectPathFromPath

public static PubsubClient.ProjectPath projectPathFromPath(String path)

projectPathFromId

public static PubsubClient.ProjectPath projectPathFromId(String projectId)

schemaPathFromPath

public static PubsubClient.SchemaPath schemaPathFromPath(String path)

schemaPathFromId

public static PubsubClient.SchemaPath schemaPathFromId(String projectId,
 String schemaId)

subscriptionPathFromPath

public static PubsubClient.SubscriptionPath subscriptionPathFromPath(String path)

subscriptionPathFromName

public static PubsubClient.SubscriptionPath subscriptionPathFromName(String projectId,
 String subscriptionName)

topicPathFromPath

public static PubsubClient.TopicPath topicPathFromPath(String path)

topicPathFromName

public static PubsubClient.TopicPath topicPathFromName(String projectId,
 String topicName)

publish

public abstract int publish(PubsubClient.TopicPath topic,
 List<PubsubClient.OutgoingMessage> outgoingMessages)
                     throws IOException

Publish outgoingMessages to Pubsub topic. Return number of messages published.

Throws:

IOException

pull

public abstract List<PubsubClient.IncomingMessage> pull(long requestTimeMsSinceEpoch,
 PubsubClient.SubscriptionPath subscription,
 int batchSize,
 boolean returnImmediately)
                                                 throws IOException

Request the next batch of up to batchSize messages from subscription. Return the received messages, or empty collection if none were available. Does not wait for messages to arrive if returnImmediately is true. Returned messages will record their request time as requestTimeMsSinceEpoch.

Throws:

IOException

acknowledge

public abstract void acknowledge(PubsubClient.SubscriptionPath subscription,
 List<String> ackIds)
                          throws IOException

Acknowldege messages from subscription with ackIds.

Throws:

IOException

modifyAckDeadline

public abstract void modifyAckDeadline(PubsubClient.SubscriptionPath subscription,
 List<String> ackIds,
 int deadlineSeconds)
                                throws IOException

Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.

Throws:

IOException

createTopic

public abstract void createTopic(PubsubClient.TopicPath topic)
                          throws IOException

Create topic.

Throws:

IOException

createTopic

public abstract void createTopic(PubsubClient.TopicPath topic,
 PubsubClient.SchemaPath schema)
                          throws IOException

Create {link TopicPath} with PubsubClient.SchemaPath.

Throws:

IOException

deleteTopic

public abstract void deleteTopic(PubsubClient.TopicPath topic)
                          throws IOException

Throws:

IOException

listTopics

public abstract List<PubsubClient.TopicPath> listTopics(PubsubClient.ProjectPath project)
                                                 throws IOException

Return a list of topics for project.

Throws:

IOException

isTopicExists

public abstract boolean isTopicExists(PubsubClient.TopicPath topic)
                               throws IOException

Return true if topic exists.

Throws:

IOException

createSubscription

public abstract void createSubscription(PubsubClient.TopicPath topic,
 PubsubClient.SubscriptionPath subscription,
 int ackDeadlineSeconds)
                                 throws IOException

Create subscription to topic.

Throws:

IOException

createRandomSubscription

public PubsubClient.SubscriptionPath createRandomSubscription(PubsubClient.ProjectPath project,
 PubsubClient.TopicPath topic,
 int ackDeadlineSeconds)
                                                       throws IOException

Create a random subscription for topic. Return the PubsubClient.SubscriptionPath. It is the responsibility of the caller to later delete the subscription.

Throws:

IOException

deleteSubscription

public abstract void deleteSubscription(PubsubClient.SubscriptionPath subscription)
                                 throws IOException

Delete subscription.

Throws:

IOException

listSubscriptions

public abstract List<PubsubClient.SubscriptionPath> listSubscriptions(PubsubClient.ProjectPath project,
 PubsubClient.TopicPath topic)
                                                               throws IOException

Return a list of subscriptions for topic in project.

Throws:

IOException

ackDeadlineSeconds

public abstract int ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription)
                                throws IOException

Return the ack deadline, in seconds, for subscription.

Throws:

IOException

isEOF

public abstract boolean isEOF()

Return true if pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list. Actual clients will return false. Test clients may return true to signal that all expected messages have been pulled and the test may complete.

createSchema

public abstract void createSchema(PubsubClient.SchemaPath schemaPath,
 String schemaContent,
 com.google.pubsub.v1.Schema.Type type)
                           throws IOException

Create Schema from Schema definition content.

Throws:

IOException

deleteSchema

public abstract void deleteSchema(PubsubClient.SchemaPath schemaPath)
                           throws IOException

Delete PubsubClient.SchemaPath.

Throws:

IOException

getSchemaPath

public abstract PubsubClient.SchemaPath getSchemaPath(PubsubClient.TopicPath topicPath)
                                               throws IOException

Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.

Throws:

IOException

getSchema

public abstract Schema getSchema(PubsubClient.SchemaPath schemaPath)
                          throws IOException

Return a Beam Schema from the Pub/Sub schema resource, if exists.

Throws:

IOException