Package org.apache.beam.sdk.io.iceberg.cdc

Class IcebergCdcMetadataColumns

java.lang.Object
org.apache.beam.sdk.io.iceberg.cdc.IcebergCdcMetadataColumns
@Internal public final class IcebergCdcMetadataColumns extends Object
Supported top-level metadata columns for Beam Iceberg CDC reads.

The supported columns come from two sources:

  • Iceberg row metadata: _row_id and _last_updated_sequence_number. These are requested from the physical Iceberg reader and are only available for row-lineage tables (v3+).
  • Changelog context metadata: _change_type, _commit_snapshot_id, and _commit_snapshot_sequence_number. These are known from the changelog snapshot/task context and are appended when Beam output rows are built.

  • Field Details

    • CHANGE_TYPE

      public static final String CHANGE_TYPE

    • COMMIT_SNAPSHOT_SEQUENCE_NUMBER

      public static final String COMMIT_SNAPSHOT_SEQUENCE_NUMBER
      See Also:

    • COMMIT_SNAPSHOT_ID

      public static final String COMMIT_SNAPSHOT_ID

    • ROW_ID

      public static final String ROW_ID

    • LAST_UPDATED_SEQUENCE_NUMBER

      public static final String LAST_UPDATED_SEQUENCE_NUMBER

    • SUPPORTED_COLUMNS

      public static final org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.ImmutableList<String> SUPPORTED_COLUMNS

  • Constructor Details

    • IcebergCdcMetadataColumns

      public IcebergCdcMetadataColumns()

  • Method Details

    • isSupportedColumn

      public static boolean isSupportedColumn(String name)

    • isRowMetadataColumn

      public static boolean isRowMetadataColumn(String name)

    • beamField

      public static Schema.Field beamField(String name)

    • icebergRowMetadataField

      public static @Nullable org.apache.iceberg.types.Types.NestedField icebergRowMetadataField(String name)
      Returns the Iceberg reader field for row-sourced metadata, or null for commit metadata.