org.apache.beam.sdk.io.gcp.bigtable

Class BigtableIO.ReadChangeStream

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<KV<com.google.protobuf.ByteString,com.google.cloud.bigtable.data.v2.models.ChangeStreamMutation>>>

org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.ReadChangeStream

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

BigtableIO

public abstract static class BigtableIO.ReadChangeStream
extends PTransform<PBegin,PCollection<KV<com.google.protobuf.ByteString,com.google.cloud.bigtable.data.v2.models.ChangeStreamMutation>>>

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Constructor Summary

Constructors

Constructor and Description
ReadChangeStream()

Method Summary

Modifier and Type	Method and Description
PCollection<KV<com.google.protobuf.ByteString,com.google.cloud.bigtable.data.v2.models.ChangeStreamMutation>>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
BigtableIO.ReadChangeStream	withAppProfileId(java.lang.String appProfileId) Returns a new BigtableIO.ReadChangeStream that will stream from the cluster specified by app profile id.
BigtableIO.ReadChangeStream	withChangeStreamName(java.lang.String changeStreamName) Returns a new BigtableIO.ReadChangeStream that uses changeStreamName as prefix for the metadata table.
BigtableIO.ReadChangeStream	withCreateOrUpdateMetadataTable(boolean shouldCreate) Returns a new BigtableIO.ReadChangeStream that, if set to true, will create or update metadata table before launching pipeline.
BigtableIO.ReadChangeStream	withExistingPipelineOptions(BigtableIO.ExistingPipelineOptions existingPipelineOptions) Returns a new BigtableIO.ReadChangeStream that decides what to do if an existing pipeline exists with the same change stream name.
BigtableIO.ReadChangeStream	withInstanceId(java.lang.String instanceId) Returns a new BigtableIO.ReadChangeStream that will stream from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(java.lang.String) to be called to determine the project.
BigtableIO.ReadChangeStream	withMetadataTableAppProfileId(java.lang.String appProfileId) Returns a new BigtableIO.ReadChangeStream that will use the cluster specified by app profile id to store the metadata of the stream.
BigtableIO.ReadChangeStream	withMetadataTableInstanceId(java.lang.String instanceId) Returns a new BigtableIO.ReadChangeStream that will use the Cloud Bigtable instance indicated by given parameter to manage the metadata of the stream.
BigtableIO.ReadChangeStream	withMetadataTableProjectId(java.lang.String projectId) Returns a new BigtableIO.ReadChangeStream that will use the Cloud Bigtable project indicated by given parameter to manage the metadata of the stream.
BigtableIO.ReadChangeStream	withMetadataTableTableId(java.lang.String tableId) Returns a new BigtableIO.ReadChangeStream that will use specified table to store the metadata of the stream.
BigtableIO.ReadChangeStream	withProjectId(java.lang.String projectId) Returns a new BigtableIO.ReadChangeStream that will stream from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(java.lang.String) to be called to determine the instance.
BigtableIO.ReadChangeStream	withStartTime(Instant startTime) Returns a new BigtableIO.ReadChangeStream that will start streaming at the specified start time.
BigtableIO.ReadChangeStream	withTableId(java.lang.String tableId) Returns a new BigtableIO.ReadChangeStream that will stream from the specified table.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ReadChangeStream

public ReadChangeStream()

Method Detail

withProjectId

public BigtableIO.ReadChangeStream withProjectId(java.lang.String projectId)

Returns a new BigtableIO.ReadChangeStream that will stream from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(java.lang.String) to be called to determine the instance.

Does not modify this object.

withInstanceId

public BigtableIO.ReadChangeStream withInstanceId(java.lang.String instanceId)

Returns a new BigtableIO.ReadChangeStream that will stream from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(java.lang.String) to be called to determine the project.

Does not modify this object.

withTableId

public BigtableIO.ReadChangeStream withTableId(java.lang.String tableId)

Returns a new BigtableIO.ReadChangeStream that will stream from the specified table.

Does not modify this object.

withAppProfileId

public BigtableIO.ReadChangeStream withAppProfileId(java.lang.String appProfileId)

Returns a new BigtableIO.ReadChangeStream that will stream from the cluster specified by app profile id.

This must use single-cluster routing policy. If not setting a separate app profile for the metadata table with withMetadataTableAppProfileId(java.lang.String), this app profile also needs to enable allow single-row transactions.

Does not modify this object.

withStartTime

public BigtableIO.ReadChangeStream withStartTime(Instant startTime)

Returns a new BigtableIO.ReadChangeStream that will start streaming at the specified start time.

Does not modify this object.

withChangeStreamName

public BigtableIO.ReadChangeStream withChangeStreamName(java.lang.String changeStreamName)

Returns a new BigtableIO.ReadChangeStream that uses changeStreamName as prefix for the metadata table.

Does not modify this object.

withExistingPipelineOptions

public BigtableIO.ReadChangeStream withExistingPipelineOptions(BigtableIO.ExistingPipelineOptions existingPipelineOptions)

Returns a new BigtableIO.ReadChangeStream that decides what to do if an existing pipeline exists with the same change stream name.

Does not modify this object.

withMetadataTableProjectId

public BigtableIO.ReadChangeStream withMetadataTableProjectId(java.lang.String projectId)

Returns a new BigtableIO.ReadChangeStream that will use the Cloud Bigtable project indicated by given parameter to manage the metadata of the stream.

Optional: defaults to value from withProjectId

Does not modify this object.

withMetadataTableInstanceId

public BigtableIO.ReadChangeStream withMetadataTableInstanceId(java.lang.String instanceId)

Returns a new BigtableIO.ReadChangeStream that will use the Cloud Bigtable instance indicated by given parameter to manage the metadata of the stream.

Optional: defaults to value from withInstanceId

Does not modify this object.

withMetadataTableTableId

public BigtableIO.ReadChangeStream withMetadataTableTableId(java.lang.String tableId)

Returns a new BigtableIO.ReadChangeStream that will use specified table to store the metadata of the stream.

Optional: defaults to value from withTableId

Does not modify this object.

withMetadataTableAppProfileId

public BigtableIO.ReadChangeStream withMetadataTableAppProfileId(java.lang.String appProfileId)

Returns a new BigtableIO.ReadChangeStream that will use the cluster specified by app profile id to store the metadata of the stream.

Optional: defaults to value from withAppProfileId

This must use single-cluster routing policy with allow single-row transactions enabled.

Does not modify this object.

withCreateOrUpdateMetadataTable

public BigtableIO.ReadChangeStream withCreateOrUpdateMetadataTable(boolean shouldCreate)

Returns a new BigtableIO.ReadChangeStream that, if set to true, will create or update metadata table before launching pipeline. Otherwise, it is expected that a metadata table with correct schema exists.

Optional: defaults to true

Does not modify this object.

expand

public PCollection<KV<com.google.protobuf.ByteString,com.google.cloud.bigtable.data.v2.models.ChangeStreamMutation>> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<KV<com.google.protobuf.ByteString,com.google.cloud.bigtable.data.v2.models.ChangeStreamMutation>>>