org.apache.beam.sdk.io.gcp.bigtable

Class BigtableIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.Read

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

BigtableIO

public abstract static class BigtableIO.Read
extends PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

A PTransform that reads from Google Cloud Bigtable. See the class-level Javadoc on BigtableIO for more information.

See Also:

BigtableIO, Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor and Description
Read()

Method Summary

Modifier and Type	Method and Description
PCollection<com.google.bigtable.v2.Row>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
@Nullable BigtableOptions	getBigtableOptions() Deprecated. read options are configured directly on BigtableIO.read(). Use populateDisplayData(DisplayData.Builder) to view the current configurations.
@Nullable java.lang.String	getTableId() Returns the table being read from.
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
java.lang.String	toString()
void	validate(PipelineOptions options) Called before running the Pipeline to verify this transform is fully and correctly specified.
BigtableIO.Read	withAppProfileId(java.lang.String appProfileId) Returns a new BigtableIO.Read that will read using the specified app profile id.
BigtableIO.Read	withAppProfileId(ValueProvider<java.lang.String> appProfileId) Returns a new BigtableIO.Read that will read using the specified app profile id.
BigtableIO.Read	withAttemptTimeout(Duration timeout) Returns a new BigtableIO.Read with the attempt timeout.
BigtableIO.Read	withBigtableOptions(BigtableOptions.Builder optionsBuilder) Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.
BigtableIO.Read	withBigtableOptions(BigtableOptions options) Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.
BigtableIO.Read	withBigtableOptionsConfigurator(SerializableFunction<BigtableOptions.Builder,BigtableOptions.Builder> configurator) Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.
BigtableIO.Read	withEmulator(java.lang.String emulatorHost) Returns a new BigtableIO.Read that will use an official Bigtable emulator.
BigtableIO.Read	withInstanceId(java.lang.String instanceId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.
BigtableIO.Read	withInstanceId(ValueProvider<java.lang.String> instanceId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.
BigtableIO.Read	withKeyRange(ByteKeyRange keyRange) Returns a new BigtableIO.Read that will read only rows in the specified range.
BigtableIO.Read	withKeyRanges(java.util.List<ByteKeyRange> keyRanges) Returns a new BigtableIO.Read that will read only rows in the specified ranges.
BigtableIO.Read	withKeyRanges(ValueProvider<java.util.List<ByteKeyRange>> keyRanges) Returns a new BigtableIO.Read that will read only rows in the specified ranges.
BigtableIO.Read	withMaxBufferElementCount(@Nullable java.lang.Integer maxBufferElementCount) Returns a new BigtableIO.Read that will break up read requests into smaller batches.
BigtableIO.Read	withOperationTimeout(Duration timeout) Returns a new BigtableIO.Read with the operation timeout.
BigtableIO.Read	withoutValidation() Disables validation that the table being read from exists.
BigtableIO.Read	withProjectId(java.lang.String projectId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.
BigtableIO.Read	withProjectId(ValueProvider<java.lang.String> projectId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.
BigtableIO.Read	withRowFilter(com.google.bigtable.v2.RowFilter filter) Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.
BigtableIO.Read	withRowFilter(ValueProvider<com.google.bigtable.v2.RowFilter> filter) Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.
BigtableIO.Read	withTableId(java.lang.String tableId) Returns a new BigtableIO.Read that will read from the specified table.
BigtableIO.Read	withTableId(ValueProvider<java.lang.String> tableId) Returns a new BigtableIO.Read that will read from the specified table.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Read

public Read()

Method Detail

getTableId

public @Nullable java.lang.String getTableId()

Returns the table being read from.

getBigtableOptions

@Deprecated
public @Nullable BigtableOptions getBigtableOptions()

Deprecated. read options are configured directly on BigtableIO.read(). Use populateDisplayData(DisplayData.Builder) to view the current configurations.

Returns the Google Cloud Bigtable instance being read from, and other parameters.

withProjectId

public BigtableIO.Read withProjectId(ValueProvider<java.lang.String> projectId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.

Does not modify this object.

withProjectId

public BigtableIO.Read withProjectId(java.lang.String projectId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.

Does not modify this object.

withInstanceId

public BigtableIO.Read withInstanceId(ValueProvider<java.lang.String> instanceId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.

Does not modify this object.

withInstanceId

public BigtableIO.Read withInstanceId(java.lang.String instanceId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.

Does not modify this object.

withTableId

public BigtableIO.Read withTableId(ValueProvider<java.lang.String> tableId)

Returns a new BigtableIO.Read that will read from the specified table.

Does not modify this object.

withTableId

public BigtableIO.Read withTableId(java.lang.String tableId)

Returns a new BigtableIO.Read that will read from the specified table.

Does not modify this object.

withAppProfileId

public BigtableIO.Read withAppProfileId(ValueProvider<java.lang.String> appProfileId)

Returns a new BigtableIO.Read that will read using the specified app profile id.

Does not modify this object.

withAppProfileId

public BigtableIO.Read withAppProfileId(java.lang.String appProfileId)

Returns a new BigtableIO.Read that will read using the specified app profile id.

Does not modify this object.

withBigtableOptions

@Deprecated
public BigtableIO.Read withBigtableOptions(BigtableOptions options)

Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.

WARNING: Should be used only to specify additional parameters for connection to the Cloud Bigtable, instanceId and projectId should be provided over withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) respectively.

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>), and using any other specified customizations.

Does not modify this object.

withBigtableOptions

@Deprecated
public BigtableIO.Read withBigtableOptions(BigtableOptions.Builder optionsBuilder)

Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.

WARNING: Should be used only to specify additional parameters for connection to the Cloud Bigtable, instanceId and projectId should be provided over withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) respectively.

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by the given options, and using any other specified customizations.

Clones the given BigtableOptions builder so that any further changes will have no effect on the returned BigtableIO.Read.

Does not modify this object.

withBigtableOptionsConfigurator

@Deprecated
public BigtableIO.Read withBigtableOptionsConfigurator(SerializableFunction<BigtableOptions.Builder,BigtableOptions.Builder> configurator)

Deprecated. please set the configurations directly: BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId) and set credentials in PipelineOptions.

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance with customized options provided by given configurator.

WARNING: instanceId and projectId should not be provided here and should be provided over withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>).

Does not modify this object.

withRowFilter

public BigtableIO.Read withRowFilter(ValueProvider<com.google.bigtable.v2.RowFilter> filter)

Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.

Does not modify this object.

withRowFilter

public BigtableIO.Read withRowFilter(com.google.bigtable.v2.RowFilter filter)

Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.

Does not modify this object.

withMaxBufferElementCount

public BigtableIO.Read withMaxBufferElementCount(@Nullable java.lang.Integer maxBufferElementCount)

Returns a new BigtableIO.Read that will break up read requests into smaller batches. This function will switch the base BigtableIO.Reader class to using the SegmentReader. If null is passed, this behavior will be disabled and the stream reader will be used.

Does not modify this object.

When we have a builder, we initialize the value. When they call the method then we override the value

withKeyRange

public BigtableIO.Read withKeyRange(ByteKeyRange keyRange)

Returns a new BigtableIO.Read that will read only rows in the specified range.

Does not modify this object.

withKeyRanges

public BigtableIO.Read withKeyRanges(ValueProvider<java.util.List<ByteKeyRange>> keyRanges)

Returns a new BigtableIO.Read that will read only rows in the specified ranges. Ranges must not overlap.

Does not modify this object.

withKeyRanges

public BigtableIO.Read withKeyRanges(java.util.List<ByteKeyRange> keyRanges)

Returns a new BigtableIO.Read that will read only rows in the specified ranges. Ranges must not overlap.

Does not modify this object.

withoutValidation

public BigtableIO.Read withoutValidation()

Disables validation that the table being read from exists.

withEmulator

public BigtableIO.Read withEmulator(java.lang.String emulatorHost)

Returns a new BigtableIO.Read that will use an official Bigtable emulator.

This is used for testing.

withAttemptTimeout

public BigtableIO.Read withAttemptTimeout(Duration timeout)

Returns a new BigtableIO.Read with the attempt timeout. Attempt timeout controls the timeout for each remote call.

Does not modify this object.

withOperationTimeout

public BigtableIO.Read withOperationTimeout(Duration timeout)

Returns a new BigtableIO.Read with the operation timeout. Operation timeout has ultimate control over how long the logic should keep trying the remote call until it gives up completely.

Does not modify this object.

expand

public PCollection<com.google.bigtable.v2.Row> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

validate

public void validate(PipelineOptions options)

Description copied from class: PTransform

Called before running the Pipeline to verify this transform is fully and correctly specified.

By default, does nothing.

Overrides:

validate in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

toString

public final java.lang.String toString()

Overrides:

toString in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>