Class SourceRecordJson

java.lang.Object
org.apache.beam.io.debezium.SourceRecordJson

public class SourceRecordJson extends Object
This class can be used as a mapper for each SourceRecord retrieved.

What it does

It maps any SourceRecord retrieved from any supported

invalid reference
io.debezium.connector
to JSON

How it works

It will extract valuable fields from any given SourceRecord:

  • before - loadBefore()
  • after - loadAfter()
  • metadata - loadMetadata()
    • connector - Connector used
    • version - Connector version
    • name - Connector name
    • database - Database name
    • schema - Database Schema (for MySQL, the binlog file name)
    • table - Table name

Full metadata

When constructed with fullMetadata = true (see SourceRecordJsonMapper(boolean) and DebeziumIO.Read.withFullMetadata(boolean)), extra keys are appended to metadata after table:

  • op - The Debezium operation (c, u, d, r, t), if the record envelope has an op field
  • Every other field of the Debezium source block, in schema order, that is not already represented above (for example ts_ms, snapshot, MySQL file, pos, row, or PostgreSQL lsn, txId)

By default (fullMetadata = false) the output only contains the six keys above.

Usage Example

Map each SourceRecord to JSON

     DebeziumIO.read()
         .withFormatFunction(new SourceRecordJson.SourceRecordJsonMapper()
 
  • Constructor Details

    • SourceRecordJson

      public SourceRecordJson(@Nullable org.apache.kafka.connect.source.SourceRecord sourceRecord)
      Initializer.
      Parameters:
      sourceRecord - retrieved SourceRecord using a supported SourceConnector
    • SourceRecordJson

      public SourceRecordJson(@Nullable org.apache.kafka.connect.source.SourceRecord sourceRecord, boolean fullMetadata)
      Initializer.
      Parameters:
      sourceRecord - retrieved SourceRecord using a supported SourceConnector
      fullMetadata - whether to add op and the remaining Debezium source fields to metadata
  • Method Details

    • toJson

      public String toJson()
      Transforms the extracted data to a JSON string.
      Returns:
      JSON String