Package org.apache.beam.sdk.io.iceberg.cdc

Class DeleteReader<T>

java.lang.Object
org.apache.beam.sdk.io.iceberg.cdc.DeleteReader<T>
Direct Known Subclasses:
CdcReadUtils.GenericDeleteReader
public abstract class DeleteReader<T> extends Object
Reads a DataFile and returns records marked deleted by the given DeleteFiles.

This is mostly a copy of DeleteFilter, but flipping the logic to output deleted records instead of filtering them out.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    DeleteReader.PreloadedDeletes
    Delete data already loaded by a planning/pushdown path for one task read.

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    DeleteReader(String filePath, List<org.apache.iceberg.DeleteFile> deletes, org.apache.iceberg.Schema tableSchema, org.apache.iceberg.Schema expectedSchema, boolean needRowPosCol, DeleteReader.PreloadedDeletes preloadedDeletes)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected abstract org.apache.iceberg.StructLike
    asStructLike(T record)
     
    org.apache.iceberg.deletes.PositionDeleteIndex
    deletedRowPositions()
     
    protected abstract org.apache.iceberg.io.InputFile
    getInputFile(String location)
     
    protected org.apache.iceberg.io.InputFile
    loadInputFile(org.apache.iceberg.DeleteFile deleteFile)
     
    protected org.apache.iceberg.data.DeleteLoader
    newDeleteLoader()
     
    protected long
    pos(T record)
     
    org.apache.iceberg.io.CloseableIterable<T>
    read(org.apache.iceberg.io.CloseableIterable<T> records)
    Returns records that are deleted by either the position deletes or the equality deletes attached to this reader — i.e.
    org.apache.iceberg.Schema
    requiredSchema()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • DeleteReader

      protected DeleteReader(String filePath, List<org.apache.iceberg.DeleteFile> deletes, org.apache.iceberg.Schema tableSchema, org.apache.iceberg.Schema expectedSchema, boolean needRowPosCol, DeleteReader.PreloadedDeletes preloadedDeletes)

  • Method Details

    • requiredSchema

      public org.apache.iceberg.Schema requiredSchema()

    • asStructLike

      protected abstract org.apache.iceberg.StructLike asStructLike(T record)

    • getInputFile

      protected abstract org.apache.iceberg.io.InputFile getInputFile(String location)

    • loadInputFile

      protected org.apache.iceberg.io.InputFile loadInputFile(org.apache.iceberg.DeleteFile deleteFile)

    • pos

      protected long pos(T record)

    • newDeleteLoader

      protected org.apache.iceberg.data.DeleteLoader newDeleteLoader()

    • read

      public org.apache.iceberg.io.CloseableIterable<T> read(org.apache.iceberg.io.CloseableIterable<T> records)
      Returns records that are deleted by either the position deletes or the equality deletes attached to this reader — i.e. the union of the two delete predicates.

      Each delete-type predicate is built independently and defaults to "false" (no contribution to the union) when its side has no delete files. Both predicates are then OR-combined and applied in a single pass over records. This guarantees that:

      • A task with only position deletes emits all records whose position is in the index.
      • A task with only equality deletes emits all records matching any equality delete value.
      • A task with both emits the union of the two (without duplication).

    • deletedRowPositions

      public org.apache.iceberg.deletes.PositionDeleteIndex deletedRowPositions()