org.apache.beam.sdk.coders

Class Coder<T>

java.lang.Object

org.apache.beam.sdk.coders.Coder<T>

Type Parameters:

T - the type of values being encoded and decoded

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

CustomCoder, StructuredCoder, ZstdCoder

public abstract class Coder<T>
extends java.lang.Object
implements java.io.Serializable

A Coder<T> defines how to encode and decode values of type T into byte streams.

Coder instances are serialized during job creation and deserialized before use. This will generally be performed by serializing the object via Java Serialization.

Coder classes for compound types are often composed from coder classes for types contains therein. The composition of Coder instances into a coder for the compound class is the subject of the CoderProvider type, which enables automatic generic composition of Coder classes within the CoderRegistry. See CoderProvider and CoderRegistry for more information about how coders are inferred.

All methods of a Coder are required to be thread safe.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Coder.Context Deprecated. To implement a coder, do not use any Coder.Context. Just implement only those abstract methods which do not accept a Coder.Context and leave the default implementations for methods accepting a Coder.Context.
static class	Coder.NonDeterministicException Exception thrown by verifyDeterministic() if the encoding is not deterministic, including details of why the encoding is not deterministic.

Constructor Summary

Constructors

Constructor and Description
Coder()

Method Summary

Modifier and Type	Method and Description
boolean	consistentWithEquals() Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
abstract T	decode(java.io.InputStream inStream) Decodes a value of type T from the given input stream in the given context.
T	decode(java.io.InputStream inStream, Coder.Context context) Deprecated. only implement and call decode(InputStream)
abstract void	encode(T value, java.io.OutputStream outStream) Encodes the given value of type T onto the given output stream.
void	encode(T value, java.io.OutputStream outStream, Coder.Context context) Deprecated. only implement and call encode(Object value, OutputStream)
abstract java.util.List<? extends Coder<?>>	getCoderArguments() If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature.
protected long	getEncodedElementByteSize(T value) Returns the size in bytes of the encoded value using this coder.
TypeDescriptor<T>	getEncodedTypeDescriptor() Returns the TypeDescriptor for the type encoded.
boolean	isRegisterByteSizeObserverCheap(T value) Returns whether registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
void	registerByteSizeObserver(T value, org.apache.beam.sdk.util.common.ElementByteSizeObserver observer) Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
java.lang.Object	structuralValue(T value) Returns an object with an Object.equals() method that represents structural equality on the argument.
abstract void	verifyDeterministic() Throw Coder.NonDeterministicException if the coding is not deterministic.
static void	verifyDeterministic(Coder<?> target, java.lang.String message, Coder<?>... coders) Verifies all of the provided coders are deterministic.
static void	verifyDeterministic(Coder<?> target, java.lang.String message, java.lang.Iterable<Coder<?>> coders) Verifies all of the provided coders are deterministic.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Coder

public Coder()

Method Detail

encode

public abstract void encode(T value,
                            java.io.OutputStream outStream)
                     throws CoderException,
                            java.io.IOException

Encodes the given value of type T onto the given output stream.

Throws:

java.io.IOException - if writing to the OutputStream fails for some reason

CoderException - if the value could not be encoded for some reason

encode

@Deprecated
 @Experimental(value=CODER_CONTEXT)
public void encode(T value,
                                                                   java.io.OutputStream outStream,
                                                                   Coder.Context context)
                                                            throws CoderException,
                                                                   java.io.IOException

Deprecated. only implement and call encode(Object value, OutputStream)

Encodes the given value of type T onto the given output stream in the given context.

Throws:

java.io.IOException - if writing to the OutputStream fails for some reason

CoderException - if the value could not be encoded for some reason

decode

public abstract T decode(java.io.InputStream inStream)
                  throws CoderException,
                         java.io.IOException

Decodes a value of type T from the given input stream in the given context. Returns the decoded value.

Throws:

java.io.IOException - if reading from the InputStream fails for some reason

CoderException - if the value could not be decoded for some reason

decode

@Deprecated
 @Experimental(value=CODER_CONTEXT)
public T decode(java.io.InputStream inStream,
                                                                Coder.Context context)
                                                         throws CoderException,
                                                                java.io.IOException

Deprecated. only implement and call decode(InputStream)

Decodes a value of type T from the given input stream in the given context. Returns the decoded value.

Throws:

java.io.IOException - if reading from the InputStream fails for some reason

CoderException - if the value could not be decoded for some reason

getCoderArguments

public abstract java.util.List<? extends Coder<?>> getCoderArguments()

If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature. If this cannot be done, or this Coder does not encode/decode a parameterized type, returns the empty list.

verifyDeterministic

public abstract void verifyDeterministic()
                                  throws Coder.NonDeterministicException

Throw Coder.NonDeterministicException if the coding is not deterministic.

In order for a Coder to be considered deterministic, the following must be true:

• two values that compare as equal (via Object.equals() or Comparable.compareTo(), if supported) have the same encoding.
• the Coder always produces a canonical encoding, which is the same for an instance of an object even if produced on different computers at different times.

Throws:

Coder.NonDeterministicException - if this coder is not deterministic.

verifyDeterministic

public static void verifyDeterministic(Coder<?> target,
                                       java.lang.String message,
                                       java.lang.Iterable<Coder<?>> coders)
                                throws Coder.NonDeterministicException

Verifies all of the provided coders are deterministic. If any are not, throws a Coder.NonDeterministicException for the target Coder.

Throws:

Coder.NonDeterministicException

verifyDeterministic

public static void verifyDeterministic(Coder<?> target,
                                       java.lang.String message,
                                       Coder<?>... coders)
                                throws Coder.NonDeterministicException

Verifies all of the provided coders are deterministic. If any are not, throws a Coder.NonDeterministicException for the target Coder.

Throws:

Coder.NonDeterministicException

consistentWithEquals

public boolean consistentWithEquals()

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

Whenever the encoded bytes of two values are equal, then the original values are equal according to Objects.equals(). Note that this is well-defined for null.

This condition is most notably false for arrays. More generally, this condition is false whenever equals() compares object identity, rather than performing a semantic/structural comparison.

By default, returns false.

structuralValue

public java.lang.Object structuralValue(T value)

Returns an object with an Object.equals() method that represents structural equality on the argument.

For any two values x and y of type T, if their encoded bytes are the same, then it must be the case that structuralValue(x).equals(structuralValue(y)).

Most notably:

• The structural value for an array coder should perform a structural comparison of the contents of the arrays, rather than the default behavior of comparing according to object identity.
• The structural value for a coder accepting null should be a proper object with an equals() method, even if the input value is null.

See also consistentWithEquals().

By default, if this coder is consistentWithEquals(), and the value is not null, returns the provided object. Otherwise, encodes the value into a byte[], and returns an object that performs array equality on the encoded bytes.

isRegisterByteSizeObserverCheap

public boolean isRegisterByteSizeObserverCheap(T value)

Returns whether registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

Not intended to be called by user code, but instead by PipelineRunner implementations.

By default, returns false. The default registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) implementation invokes getEncodedElementByteSize(T) which requires re-encoding an element unless it is overridden. This is considered expensive.

registerByteSizeObserver

public void registerByteSizeObserver(T value,
                                     org.apache.beam.sdk.util.common.ElementByteSizeObserver observer)
                              throws java.lang.Exception

Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.

Not intended to be called by user code, but instead by PipelineRunner implementations.

By default, this notifies observer about the byte size of the encoded value using this coder as returned by getEncodedElementByteSize(T).

Throws:

java.lang.Exception

getEncodedElementByteSize

protected long getEncodedElementByteSize(T value)
                                  throws java.lang.Exception

Returns the size in bytes of the encoded value using this coder.

Throws:

java.lang.Exception

getEncodedTypeDescriptor

@Experimental(value=CODER_TYPE_ENCODING)
public TypeDescriptor<T> getEncodedTypeDescriptor()

Returns the TypeDescriptor for the type encoded.