Package org.apache.beam.sdk.io.cdap

Class CdapIO.Read<K,V>

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<KV<K,V>>>
org.apache.beam.sdk.io.cdap.CdapIO.Read<K,V>
All Implemented Interfaces:
Serializable, HasDisplayData
Enclosing class:
CdapIO
public abstract static class CdapIO.Read<K,V> extends PTransform<PBegin,PCollection<KV<K,V>>>
A PTransform to read from CDAP source.
See Also:

  • Constructor Details

    • Read

      public Read()

  • Method Details

    • withCdapPlugin

      public CdapIO.Read<K,V> withCdapPlugin(Plugin<K,V> plugin)
      Sets a CDAP Plugin.

    • withCdapPluginClass

      public CdapIO.Read<K,V> withCdapPluginClass(Class<?> cdapPluginClass)
      Sets a CDAP Plugin class.

    • withPluginConfig

      public CdapIO.Read<K,V> withPluginConfig(PluginConfig pluginConfig)
      Sets a PluginConfig.

    • withKeyClass

      public CdapIO.Read<K,V> withKeyClass(Class<K> keyClass)
      Sets a key class.

    • withValueClass

      public CdapIO.Read<K,V> withValueClass(Class<V> valueClass)
      Sets a value class.

    • withPullFrequencySec

      public CdapIO.Read<K,V> withPullFrequencySec(Long pullFrequencySec)
      Delay in seconds between polling for new records updates. Applicable only for streaming Cdap Plugins.

    • withStartPollTimeoutSec

      public CdapIO.Read<K,V> withStartPollTimeoutSec(Long startPollTimeoutSec)
      Delay in seconds before start polling. Applicable only for streaming Cdap Plugins.

    • withStartOffset

      public CdapIO.Read<K,V> withStartOffset(Long startOffset)
      Inclusive start offset from which the reading should be started. Applicable only for streaming Cdap Plugins.

    • expand

      public PCollection<KV<K,V>> expand(PBegin input)
      Description copied from class: PTransform
      Override this method to specify how this PTransform should be expanded on the given InputT.

      NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      Specified by:
      expand in class PTransform<PBegin,PCollection<KV<K,V>>>