Package org.apache.beam.sdk.io.iceberg.cdc
package org.apache.beam.sdk.io.iceberg.cdc
Iceberg CDC connectors.
-
ClassDescriptionRead-side helpers specific to the CDC source.Shuffle key for bidirectional CDC rows.Descriptor for a set of
SerializableChangelogTasks.DeleteReader<T>Reads aDataFileand returns records marked deleted by the givenDeleteFiles.Delete data already loaded by a planning/pushdown path for one task read.Supported top-level metadata columns for Beam Iceberg CDC reads.An Iceberg source that incrementally reads a table's changelogs, processing one snapshot at a time.APTransformthat processes batches ofChangelogScanTasks and routes them accordingly: Records from Uni-directional batches are directly emitted, as INSERT or DELETE kind Records from Bi-directional batches are compared against the Primary Key overlap range: if outside the overlap, emit directly as INSERT or DELETE kind if inside the overlap, key by (snapshot seq#, pk) and route to downstreamCoGroupByKeyand final resolution byResolveChangesAWindowFnthat assigns each element to a 1-millisecondIntervalWindowanchored at the element's event timestamp.