Package org.apache.beam.io.debezium
Class SourceRecordJson
java.lang.Object
org.apache.beam.io.debezium.SourceRecordJson
This class can be used as a mapper for each
SourceRecord retrieved.
What it does
It maps any SourceRecord retrieved from any supported
to JSON
invalid reference
io.debezium.connector
How it works
It will extract valuable fields from any given SourceRecord:
- before -
loadBefore() - after -
loadAfter() - metadata -
loadMetadata()- connector - Connector used
- version - Connector version
- name - Connector name
- database - Database name
- schema - Database Schema (for MySQL, the binlog file name)
- table - Table name
Full metadata
When constructed with fullMetadata = true (see SourceRecordJsonMapper(boolean) and DebeziumIO.Read.withFullMetadata(boolean)), extra keys are appended to metadata after
table:
- op - The Debezium operation (
c,u,d,r,t), if the record envelope has anopfield - Every other field of the Debezium
sourceblock, in schema order, that is not already represented above (for examplets_ms,snapshot, MySQLfile,pos,row, or PostgreSQLlsn,txId)
By default (fullMetadata = false) the output only contains the six keys above.
Usage Example
Map each SourceRecord to JSON
DebeziumIO.read()
.withFormatFunction(new SourceRecordJson.SourceRecordJsonMapper()
-
Nested Class Summary
Nested Classes -
Constructor Summary
ConstructorsConstructorDescriptionSourceRecordJson(@Nullable org.apache.kafka.connect.source.SourceRecord sourceRecord) Initializer.SourceRecordJson(@Nullable org.apache.kafka.connect.source.SourceRecord sourceRecord, boolean fullMetadata) Initializer. -
Method Summary
-
Constructor Details
-
SourceRecordJson
Initializer.- Parameters:
sourceRecord- retrieved SourceRecord using a supported SourceConnector
-
SourceRecordJson
public SourceRecordJson(@Nullable org.apache.kafka.connect.source.SourceRecord sourceRecord, boolean fullMetadata) Initializer.- Parameters:
sourceRecord- retrieved SourceRecord using a supported SourceConnectorfullMetadata- whether to addopand the remaining Debeziumsourcefields tometadata
-
-
Method Details
-
toJson
Transforms the extracted data to a JSON string.- Returns:
- JSON String
-