Package org.apache.beam.sdk.io.iceberg

Class IcebergIO.ReadRows

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<Row>>

org.apache.beam.sdk.io.iceberg.IcebergIO.ReadRows

All Implemented Interfaces:

Serializable, HasDisplayData

Enclosing class:

IcebergIO

public abstract static class IcebergIO.ReadRows
extends PTransform<PBegin,PCollection<Row>>

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	IcebergIO.ReadRows.StartingStrategy

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor	Description
ReadRows()

Method Summary

Modifier and Type	Method	Description
IcebergIO.ReadRows	dropping(@Nullable List<String> drop)
PCollection<Row>	expand(PBegin input)	Override this method to specify how this PTransform should be expanded on the given InputT.
IcebergIO.ReadRows	from(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)
IcebergIO.ReadRows	fromSnapshot(@Nullable Long fromSnapshot)
IcebergIO.ReadRows	fromTimestamp(@Nullable Long fromTimestamp)
IcebergIO.ReadRows	keeping(@Nullable List<String> keep)
IcebergIO.ReadRows	streaming(@Nullable Boolean streaming)
IcebergIO.ReadRows	toSnapshot(@Nullable Long toSnapshot)
IcebergIO.ReadRows	toTimestamp(@Nullable Long toTimestamp)
IcebergIO.ReadRows	withCdc()
IcebergIO.ReadRows	withFilter(@Nullable String filter)
IcebergIO.ReadRows	withMetadataColumns(@Nullable List<String> metadataColumns)	Appends top-level metadata columns to CDC output rows.
IcebergIO.ReadRows	withPollInterval(Duration pollInterval)
IcebergIO.ReadRows	withStartingStrategy(@Nullable IcebergIO.ReadRows.StartingStrategy strategy)
IcebergIO.ReadRows	withWatermarkColumn(@Nullable String watermarkColumn)
IcebergIO.ReadRows	withWatermarkColumnTimeUnit(@Nullable String timeUnit)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

ReadRows

public ReadRows()

Method Details

withCdc

public IcebergIO.ReadRows withCdc()

from

public IcebergIO.ReadRows from(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)

fromSnapshot

public IcebergIO.ReadRows fromSnapshot(@Nullable Long fromSnapshot)

toSnapshot

public IcebergIO.ReadRows toSnapshot(@Nullable Long toSnapshot)

fromTimestamp

public IcebergIO.ReadRows fromTimestamp(@Nullable Long fromTimestamp)

toTimestamp

public IcebergIO.ReadRows toTimestamp(@Nullable Long toTimestamp)

withPollInterval

public IcebergIO.ReadRows withPollInterval(Duration pollInterval)

streaming

public IcebergIO.ReadRows streaming(@Nullable Boolean streaming)

withStartingStrategy

public IcebergIO.ReadRows withStartingStrategy(@Nullable IcebergIO.ReadRows.StartingStrategy strategy)

keeping

public IcebergIO.ReadRows keeping(@Nullable List<String> keep)

dropping

public IcebergIO.ReadRows dropping(@Nullable List<String> drop)

withFilter

public IcebergIO.ReadRows withFilter(@Nullable String filter)

withWatermarkColumn

public IcebergIO.ReadRows withWatermarkColumn(@Nullable String watermarkColumn)

withWatermarkColumnTimeUnit

public IcebergIO.ReadRows withWatermarkColumnTimeUnit(@Nullable String timeUnit)

withMetadataColumns

public IcebergIO.ReadRows withMetadataColumns(@Nullable List<String> metadataColumns)

Appends top-level metadata columns to CDC output rows.

Supported values are _change_type, _commit_snapshot_id, _commit_snapshot_sequence_number, _row_id, and _last_updated_sequence_number. The row metadata columns are read from Iceberg data files and require a row-lineage table. The changelog metadata columns come from the emitted change kind and snapshot context and are appended when final Beam rows are emitted.

This option is only valid withCdc().

expand

public PCollection<Row> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<Row>>