org.apache.beam.sdk.io.gcp.bigtable

Class BigtableIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.Read

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

BigtableIO

public abstract static class BigtableIO.Read
extends PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

A PTransform that reads from Google Cloud Bigtable. See the class-level Javadoc on BigtableIO for more information.

See Also:

BigtableIO, Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Constructor Summary

Constructors

Constructor and Description
Read()

Method Summary

Modifier and Type	Method and Description
PCollection<com.google.bigtable.v2.Row>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
@Nullable BigtableOptions	getBigtableOptions() Deprecated. will be replaced by bigtable options configurator.
@Nullable java.lang.String	getTableId() Returns the table being read from.
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
java.lang.String	toString()
void	validate(PipelineOptions options) Called before running the Pipeline to verify this transform is fully and correctly specified.
BigtableIO.Read	withBigtableOptions(BigtableOptions.Builder optionsBuilder) Deprecated. will be replaced by bigtable options configurator.
BigtableIO.Read	withBigtableOptions(BigtableOptions options) Deprecated. will be replaced by bigtable options configurator.
BigtableIO.Read	withBigtableOptionsConfigurator(SerializableFunction<BigtableOptions.Builder,BigtableOptions.Builder> configurator) Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance with customized options provided by given configurator.
BigtableIO.Read	withEmulator(java.lang.String emulatorHost) Returns a new BigtableIO.Read that will use an official Bigtable emulator.
BigtableIO.Read	withInstanceId(java.lang.String instanceId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.
BigtableIO.Read	withInstanceId(ValueProvider<java.lang.String> instanceId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.
BigtableIO.Read	withKeyRange(ByteKeyRange keyRange) Returns a new BigtableIO.Read that will read only rows in the specified range.
BigtableIO.Read	withKeyRanges(java.util.List<ByteKeyRange> keyRanges) Returns a new BigtableIO.Read that will read only rows in the specified ranges.
BigtableIO.Read	withKeyRanges(ValueProvider<java.util.List<ByteKeyRange>> keyRanges) Returns a new BigtableIO.Read that will read only rows in the specified ranges.
BigtableIO.Read	withMaxBufferElementCount(@Nullable java.lang.Integer maxBufferElementCount) Returns a new BigtableIO.Read that will break up read requests into smaller batches.
BigtableIO.Read	withoutValidation() Disables validation that the table being read from exists.
BigtableIO.Read	withProjectId(java.lang.String projectId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.
BigtableIO.Read	withProjectId(ValueProvider<java.lang.String> projectId) Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.
BigtableIO.Read	withRowFilter(com.google.bigtable.v2.RowFilter filter) Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.
BigtableIO.Read	withRowFilter(ValueProvider<com.google.bigtable.v2.RowFilter> filter) Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.
BigtableIO.Read	withTableId(java.lang.String tableId) Returns a new BigtableIO.Read that will read from the specified table.
BigtableIO.Read	withTableId(ValueProvider<java.lang.String> tableId) Returns a new BigtableIO.Read that will read from the specified table.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Read

public Read()

Method Detail

getTableId

public @Nullable java.lang.String getTableId()

Returns the table being read from.

getBigtableOptions

@Deprecated
public @Nullable BigtableOptions getBigtableOptions()

Deprecated. will be replaced by bigtable options configurator.

Returns the Google Cloud Bigtable instance being read from, and other parameters.

withProjectId

public BigtableIO.Read withProjectId(ValueProvider<java.lang.String> projectId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.

Does not modify this object.

withProjectId

public BigtableIO.Read withProjectId(java.lang.String projectId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable project indicated by given parameter, requires withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the instance.

Does not modify this object.

withInstanceId

public BigtableIO.Read withInstanceId(ValueProvider<java.lang.String> instanceId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.

Does not modify this object.

withInstanceId

public BigtableIO.Read withInstanceId(java.lang.String instanceId)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by given parameter, requires withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) to be called to determine the project.

Does not modify this object.

withTableId

public BigtableIO.Read withTableId(ValueProvider<java.lang.String> tableId)

Returns a new BigtableIO.Read that will read from the specified table.

Does not modify this object.

withTableId

public BigtableIO.Read withTableId(java.lang.String tableId)

Returns a new BigtableIO.Read that will read from the specified table.

Does not modify this object.

withBigtableOptions

@Deprecated
public BigtableIO.Read withBigtableOptions(BigtableOptions options)

Deprecated. will be replaced by bigtable options configurator.

WARNING: Should be used only to specify additional parameters for connection to the Cloud Bigtable, instanceId and projectId should be provided over withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) respectively.

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>), and using any other specified customizations.

Does not modify this object.

withBigtableOptions

@Deprecated
public BigtableIO.Read withBigtableOptions(BigtableOptions.Builder optionsBuilder)

Deprecated. will be replaced by bigtable options configurator.

WARNING: Should be used only to specify additional parameters for connection to the Cloud Bigtable, instanceId and projectId should be provided over withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) respectively.

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance indicated by the given options, and using any other specified customizations.

Clones the given BigtableOptions builder so that any further changes will have no effect on the returned BigtableIO.Read.

Does not modify this object.

withBigtableOptionsConfigurator

public BigtableIO.Read withBigtableOptionsConfigurator(SerializableFunction<BigtableOptions.Builder,BigtableOptions.Builder> configurator)

Returns a new BigtableIO.Read that will read from the Cloud Bigtable instance with customized options provided by given configurator.

WARNING: instanceId and projectId should not be provided here and should be provided over withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>) and withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String>).

Does not modify this object.

withRowFilter

public BigtableIO.Read withRowFilter(ValueProvider<com.google.bigtable.v2.RowFilter> filter)

Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.

Does not modify this object.

withRowFilter

public BigtableIO.Read withRowFilter(com.google.bigtable.v2.RowFilter filter)

Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.

Does not modify this object.

withMaxBufferElementCount

@Experimental(value=SOURCE_SINK)
public BigtableIO.Read withMaxBufferElementCount(@Nullable java.lang.Integer maxBufferElementCount)

Returns a new BigtableIO.Read that will break up read requests into smaller batches. This function will switch the base BigtableIO.Reader class to using the SegmentReader. If null is passed, this behavior will be disabled and the stream reader will be used.

Does not modify this object.

When we have a builder, we initialize the value. When they call the method then we override the value

withKeyRange

public BigtableIO.Read withKeyRange(ByteKeyRange keyRange)

Returns a new BigtableIO.Read that will read only rows in the specified range.

Does not modify this object.

withKeyRanges

public BigtableIO.Read withKeyRanges(ValueProvider<java.util.List<ByteKeyRange>> keyRanges)

Returns a new BigtableIO.Read that will read only rows in the specified ranges. Ranges must not overlap.

Does not modify this object.

withKeyRanges

public BigtableIO.Read withKeyRanges(java.util.List<ByteKeyRange> keyRanges)

Returns a new BigtableIO.Read that will read only rows in the specified ranges. Ranges must not overlap.

Does not modify this object.

withoutValidation

public BigtableIO.Read withoutValidation()

Disables validation that the table being read from exists.

withEmulator

public BigtableIO.Read withEmulator(java.lang.String emulatorHost)

Returns a new BigtableIO.Read that will use an official Bigtable emulator.

This is used for testing.

expand

public PCollection<com.google.bigtable.v2.Row> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

validate

public void validate(PipelineOptions options)

Description copied from class: PTransform

Called before running the Pipeline to verify this transform is fully and correctly specified.

By default, does nothing.

Overrides:

validate in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

toString

public final java.lang.String toString()

Overrides:

toString in class PTransform<PBegin,PCollection<com.google.bigtable.v2.Row>>