apache_beam.io.gcp.pubsublite package

class apache_beam.io.gcp.pubsublite.ReadFromPubSubLite(subscription_path, min_bundle_timeout=None, deduplicate=None, expansion_service=None)[source]

Bases: apache_beam.transforms.ptransform.PTransform

A PTransform for reading from Pub/Sub Lite.

Produces a PCollection of google.cloud.pubsublite.SequencedMessage

Experimental; no backwards-compatibility guarantees.

Initializes ReadFromPubSubLite.

Parameters:
  • subscription_path – Pub/Sub Lite Subscription in the form projects/<project>/locations/<location>/subscriptions/<subscription>
  • min_bundle_timeout – The minimum wall time to pass before allowing bundle closure. Setting this to too small of a value will result in increased compute costs and lower throughput per byte. Immediate timeouts (0) may be useful for testing.
  • deduplicate – Whether to deduplicate messages based on the value of the ‘x-goog-pubsublite-dataflow-uuid’ attribute. Defaults to False.
expand(pvalue)[source]
class apache_beam.io.gcp.pubsublite.WriteToPubSubLite(topic_path, add_uuids=None, expansion_service=None)[source]

Bases: apache_beam.transforms.ptransform.PTransform

A PTransform for writing to Pub/Sub Lite.

Consumes a PCollection of google.cloud.pubsublite.PubSubMessage

Experimental; no backwards-compatibility guarantees.

Initializes WriteToPubSubLite.

Parameters:
  • topic_path – A Pub/Sub Lite Topic path.
  • add_uuids – Whether to add uuids to the ‘x-goog-pubsublite-dataflow-uuid’ uuid attribute. Defaults to False.
expand(pcoll)[source]