public class PubsubUnboundedSource extends PTransform<PBegin,PCollection<PubsubMessage>>
PubsubIO#read
instead.
A PTransform which streams messages from Pubsub.
UnboundedSource
which receives messages in
batches and hands them out one at a time.
UnboundedSource.UnboundedReader
instances to execute concurrently and thus hide latency.
name
Constructor and Description |
---|
PubsubUnboundedSource(Clock clock,
PubsubClient.PubsubClientFactory pubsubFactory,
ValueProvider<PubsubClient.ProjectPath> project,
ValueProvider<PubsubClient.TopicPath> topic,
ValueProvider<PubsubClient.SubscriptionPath> subscription,
java.lang.String timestampAttribute,
java.lang.String idAttribute,
boolean needsAttributes)
Construct an unbounded source to consume from the Pubsub
subscription . |
PubsubUnboundedSource(PubsubClient.PubsubClientFactory pubsubFactory,
ValueProvider<PubsubClient.ProjectPath> project,
ValueProvider<PubsubClient.TopicPath> topic,
ValueProvider<PubsubClient.SubscriptionPath> subscription,
java.lang.String timestampAttribute,
java.lang.String idAttribute,
boolean needsAttributes)
Construct an unbounded source to consume from the Pubsub
subscription . |
PubsubUnboundedSource(PubsubClient.PubsubClientFactory pubsubFactory,
ValueProvider<PubsubClient.ProjectPath> project,
ValueProvider<PubsubClient.TopicPath> topic,
ValueProvider<PubsubClient.SubscriptionPath> subscription,
java.lang.String timestampAttribute,
java.lang.String idAttribute,
boolean needsAttributes,
boolean needsMessageId)
Construct an unbounded source to consume from the Pubsub
subscription . |
Modifier and Type | Method and Description |
---|---|
PCollection<PubsubMessage> |
expand(PBegin input)
Override this method to specify how this
PTransform should be expanded on the given
InputT . |
java.lang.String |
getIdAttribute()
Get the id attribute.
|
boolean |
getNeedsAttributes() |
boolean |
getNeedsMessageId() |
PubsubClient.ProjectPath |
getProject()
Get the project path.
|
PubsubClient.SubscriptionPath |
getSubscription()
Get the subscription being read from.
|
ValueProvider<PubsubClient.SubscriptionPath> |
getSubscriptionProvider()
Get the
ValueProvider for the subscription being read from. |
java.lang.String |
getTimestampAttribute()
Get the timestamp attribute.
|
PubsubClient.TopicPath |
getTopic()
Get the topic being read from.
|
ValueProvider<PubsubClient.TopicPath> |
getTopicProvider()
Get the
ValueProvider for the topic being read from. |
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate
public PubsubUnboundedSource(PubsubClient.PubsubClientFactory pubsubFactory, @Nullable ValueProvider<PubsubClient.ProjectPath> project, @Nullable ValueProvider<PubsubClient.TopicPath> topic, @Nullable ValueProvider<PubsubClient.SubscriptionPath> subscription, @Nullable java.lang.String timestampAttribute, @Nullable java.lang.String idAttribute, boolean needsAttributes)
subscription
.public PubsubUnboundedSource(Clock clock, PubsubClient.PubsubClientFactory pubsubFactory, @Nullable ValueProvider<PubsubClient.ProjectPath> project, @Nullable ValueProvider<PubsubClient.TopicPath> topic, @Nullable ValueProvider<PubsubClient.SubscriptionPath> subscription, @Nullable java.lang.String timestampAttribute, @Nullable java.lang.String idAttribute, boolean needsAttributes)
subscription
.public PubsubUnboundedSource(PubsubClient.PubsubClientFactory pubsubFactory, @Nullable ValueProvider<PubsubClient.ProjectPath> project, @Nullable ValueProvider<PubsubClient.TopicPath> topic, @Nullable ValueProvider<PubsubClient.SubscriptionPath> subscription, @Nullable java.lang.String timestampAttribute, @Nullable java.lang.String idAttribute, boolean needsAttributes, boolean needsMessageId)
subscription
.@Nullable public PubsubClient.ProjectPath getProject()
@Nullable public PubsubClient.TopicPath getTopic()
@Nullable public ValueProvider<PubsubClient.TopicPath> getTopicProvider()
ValueProvider
for the topic being read from.@Nullable public PubsubClient.SubscriptionPath getSubscription()
@Nullable public ValueProvider<PubsubClient.SubscriptionPath> getSubscriptionProvider()
ValueProvider
for the subscription being read from.@Nullable public java.lang.String getTimestampAttribute()
@Nullable public java.lang.String getIdAttribute()
public boolean getNeedsAttributes()
public boolean getNeedsMessageId()
public PCollection<PubsubMessage> expand(PBegin input)
PTransform
PTransform
should be expanded on the given
InputT
.
NOTE: This method should not be called directly. Instead apply the PTransform
should
be applied to the InputT
using the apply
method.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
expand
in class PTransform<PBegin,PCollection<PubsubMessage>>