org.apache.beam.sdk.io.kafka

Class KafkaRecordCoder<K,V>

java.lang.Object

org.apache.beam.sdk.coders.Coder<T>

org.apache.beam.sdk.coders.StructuredCoder<KafkaRecord<K,V>>

org.apache.beam.sdk.io.kafka.KafkaRecordCoder<K,V>

All Implemented Interfaces:

java.io.Serializable

public class KafkaRecordCoder<K,V>
extends StructuredCoder<KafkaRecord<K,V>>

Coder for KafkaRecord.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.coders.Coder

Coder.Context, Coder.NonDeterministicException

Constructor Summary

Constructors

Constructor and Description
KafkaRecordCoder(Coder<K> keyCoder, Coder<V> valueCoder)

Method Summary

Modifier and Type	Method and Description
boolean	consistentWithEquals() Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
KafkaRecord<K,V>	decode(java.io.InputStream inStream) Decodes a value of type T from the given input stream in the given context.
KafkaRecord<K,V>	decode(java.io.InputStream inStream, Coder.Context context) Decodes a value of type T from the given input stream in the given context.
void	encode(KafkaRecord<K,V> value, java.io.OutputStream outStream) Encodes the given value of type T onto the given output stream.
void	encode(KafkaRecord<K,V> value, java.io.OutputStream outStream, Coder.Context context) Encodes the given value of type T onto the given output stream in the given context.
java.util.List<? extends Coder<?>>	getCoderArguments() If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature.
boolean	isRegisterByteSizeObserverCheap(KafkaRecord<K,V> value) Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
static <K,V> KafkaRecordCoder<K,V>	of(Coder<K> keyCoder, Coder<V> valueCoder)
java.lang.Object	structuralValue(KafkaRecord<K,V> value) Returns an object with an Object.equals() method that represents structural equality on the argument.
void	verifyDeterministic() Throw Coder.NonDeterministicException if the coding is not deterministic.

Methods inherited from class org.apache.beam.sdk.coders.StructuredCoder

equals, getComponents, getEncodedTypeDescriptor, hashCode, toString

Methods inherited from class org.apache.beam.sdk.coders.Coder

getEncodedElementByteSize, registerByteSizeObserver, verifyDeterministic, verifyDeterministic

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

KafkaRecordCoder

public KafkaRecordCoder(Coder<K> keyCoder,
                        Coder<V> valueCoder)

Method Detail

of

public static <K,V> KafkaRecordCoder<K,V> of(Coder<K> keyCoder,
                                             Coder<V> valueCoder)

encode

public void encode(KafkaRecord<K,V> value,
                   java.io.OutputStream outStream)
            throws CoderException,
                   java.io.IOException

Description copied from class: Coder

Encodes the given value of type T onto the given output stream.

Specified by:

encode in class Coder<KafkaRecord<K,V>>

Throws:

CoderException - if the value could not be encoded for some reason

java.io.IOException - if writing to the OutputStream fails for some reason

encode

public void encode(KafkaRecord<K,V> value,
                   java.io.OutputStream outStream,
                   Coder.Context context)
            throws CoderException,
                   java.io.IOException

Description copied from class: Coder

Encodes the given value of type T onto the given output stream in the given context.

Overrides:

encode in class Coder<KafkaRecord<K,V>>

Throws:

CoderException - if the value could not be encoded for some reason

java.io.IOException - if writing to the OutputStream fails for some reason

decode

public KafkaRecord<K,V> decode(java.io.InputStream inStream)
                        throws CoderException,
                               java.io.IOException

Description copied from class: Coder

Decodes a value of type T from the given input stream in the given context. Returns the decoded value.

Specified by:

decode in class Coder<KafkaRecord<K,V>>

Throws:

CoderException - if the value could not be decoded for some reason

java.io.IOException - if reading from the InputStream fails for some reason

decode

public KafkaRecord<K,V> decode(java.io.InputStream inStream,
                               Coder.Context context)
                        throws CoderException,
                               java.io.IOException

Description copied from class: Coder

Decodes a value of type T from the given input stream in the given context. Returns the decoded value.

Overrides:

decode in class Coder<KafkaRecord<K,V>>

Throws:

CoderException - if the value could not be decoded for some reason

java.io.IOException - if reading from the InputStream fails for some reason

getCoderArguments

public java.util.List<? extends Coder<?>> getCoderArguments()

Description copied from class: Coder

If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature. If this cannot be done, or this Coder does not encode/decode a parameterized type, returns the empty list.

Specified by:

getCoderArguments in class Coder<KafkaRecord<K,V>>

verifyDeterministic

public void verifyDeterministic()
                         throws Coder.NonDeterministicException

Description copied from class: Coder

Throw Coder.NonDeterministicException if the coding is not deterministic.

In order for a Coder to be considered deterministic, the following must be true:

• two values that compare as equal (via Object.equals() or Comparable.compareTo(), if supported) have the same encoding.
• the Coder always produces a canonical encoding, which is the same for an instance of an object even if produced on different computers at different times.

Specified by:

verifyDeterministic in class Coder<KafkaRecord<K,V>>

Throws:

Coder.NonDeterministicException - if this coder is not deterministic.

isRegisterByteSizeObserverCheap

public boolean isRegisterByteSizeObserverCheap(KafkaRecord<K,V> value)

Description copied from class: Coder

Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

Not intended to be called by user code, but instead by PipelineRunner implementations.

By default, returns false. The default Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) implementation invokes Coder.getEncodedElementByteSize(T) which requires re-encoding an element unless it is overridden. This is considered expensive.

Overrides:

isRegisterByteSizeObserverCheap in class Coder<KafkaRecord<K,V>>

structuralValue

public java.lang.Object structuralValue(KafkaRecord<K,V> value)

Description copied from class: Coder

Returns an object with an Object.equals() method that represents structural equality on the argument.

For any two values x and y of type T, if their encoded bytes are the same, then it must be the case that structuralValue(x).equals(@code structuralValue(y).

Most notably:

• The structural value for an array coder should perform a structural comparison of the contents of the arrays, rather than the default behavior of comparing according to object identity.
• The structural value for a coder accepting null should be a proper object with an equals() method, even if the input value is null.

See also Coder.consistentWithEquals().

By default, if this coder is Coder.consistentWithEquals(), and the value is not null, returns the provided object. Otherwise, encodes the value into a byte[], and returns an object that performs array equality on the encoded bytes.

Overrides:

structuralValue in class StructuredCoder<KafkaRecord<K,V>>

consistentWithEquals

public boolean consistentWithEquals()

Description copied from class: StructuredCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

Whenever the encoded bytes of two values are equal, then the original values are equal according to Objects.equals(). Note that this is well-defined for null.

This condition is most notably false for arrays. More generally, this condition is false whenever equals() compares object identity, rather than performing a semantic/structural comparison.

By default, returns false.

Overrides:

consistentWithEquals in class StructuredCoder<KafkaRecord<K,V>>

Returns:

false for StructuredCoder unless overridden.