Package org.apache.beam.sdk.io.pulsar

Class PulsarIO.Read<T>

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<T>>
org.apache.beam.sdk.io.pulsar.PulsarIO.Read<T>
All Implemented Interfaces:
Serializable, HasDisplayData
Enclosing class:
PulsarIO
public abstract static class PulsarIO.Read<T> extends PTransform<PBegin,PCollection<T>>
See Also:

  • Constructor Details

    • Read

      public Read()

  • Method Details

    • withAdminUrl

      public PulsarIO.Read<T> withAdminUrl(String url)
      Configure Pulsar admin url.

      Admin client is used to approximate backlogs. This setting is optional.

      Parameters:
      url - admin url. For example, "http://localhost:8080".

    • withClientUrl

      public PulsarIO.Read<T> withClientUrl(String url)
      Configure Pulsar client url. "pulsar://localhost:6650".
      Parameters:
      url - client url. For example,

    • withTopic

      public PulsarIO.Read<T> withTopic(String topic)

    • withStartTimestamp

      public PulsarIO.Read<T> withStartTimestamp(Long timestamp)

    • withEndTimestamp

      public PulsarIO.Read<T> withEndTimestamp(Long timestamp)

    • withEndMessageId

      public PulsarIO.Read<T> withEndMessageId(org.apache.pulsar.client.api.MessageId msgId)

    • withPublishTime

      public PulsarIO.Read<T> withPublishTime()
      Set elements timestamped by Message.getPublishTime(). It is the default.

    • withProcessingTime

      public PulsarIO.Read<T> withProcessingTime()
      Set elements timestamped to the moment it get processed.

    • withConsumerPollingTimeout

      public PulsarIO.Read<T> withConsumerPollingTimeout(long duration)
      Sets the timeout time in seconds for Pulsar consumer polling request. A lower timeout optimizes for latency. Increase the timeout if the consumer is not fetching any records. The default is 2 seconds.

    • withPulsarClient

      public PulsarIO.Read<T> withPulsarClient(SerializableFunction<String,org.apache.pulsar.client.api.PulsarClient> pulsarClientFn)

    • withPulsarAdmin

      public PulsarIO.Read<T> withPulsarAdmin(SerializableFunction<String,org.apache.pulsar.client.admin.PulsarAdmin> pulsarAdminFn)

    • expand

      public PCollection<T> expand(PBegin input)
      Description copied from class: PTransform
      Override this method to specify how this PTransform should be expanded on the given InputT.

      NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      Specified by:
      expand in class PTransform<PBegin,PCollection<T>>