org.apache.beam.sdk.coders

Class IterableLikeCoder<T,IterableT extends java.lang.Iterable<T>>

java.lang.Object

org.apache.beam.sdk.coders.Coder<T>

org.apache.beam.sdk.coders.StructuredCoder<IterableT>

org.apache.beam.sdk.coders.IterableLikeCoder<T,IterableT>

Type Parameters:

T - the type of the elements of the Iterables being transcoded

IterableT - the type of the Iterables being transcoded

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

CollectionCoder, DequeCoder, IterableCoder, ListCoder, SetCoder

public abstract class IterableLikeCoder<T,IterableT extends java.lang.Iterable<T>>
extends StructuredCoder<IterableT>

An abstract base class with functionality for assembling a Coder for a class that implements Iterable.

To complete a subclass, implement the decodeToIterable(java.util.List<T>) method. This superclass will decode the elements in the input stream into a List and then pass them to that method to be converted into the appropriate iterable type. Note that this means the input iterables must fit into memory.

The format of this coder is as follows:

• If the input Iterable has a known and finite size, then the size is written to the output stream in big endian format, followed by all of the encoded elements.
• If the input Iterable is not known to have a finite size, then each element of the input is preceded by true encoded as a byte (indicating "more data") followed by the encoded element, and terminated by false encoded as a byte.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.coders.Coder

Coder.Context, Coder.NonDeterministicException

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	IterableLikeCoder(Coder<T> elementCoder, java.lang.String iterableName)

Method Summary

Modifier and Type	Method and Description
IterableT	decode(java.io.InputStream inStream) Decodes a value of type T from the given input stream in the given context.
protected abstract IterableT	decodeToIterable(java.util.List<T> decodedElements) Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.
protected IterableT	decodeToIterable(java.util.List<T> decodedElements, long terminatorValue, java.io.InputStream in) Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements with the InputStream at the position where this coder detected the end of the stream.
void	encode(IterableT iterable, java.io.OutputStream outStream) Encodes the given value of type T onto the given output stream.
java.util.List<? extends Coder<?>>	getCoderArguments() If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature.
Coder<T>	getElemCoder()
boolean	isRegisterByteSizeObserverCheap(IterableT iterable) Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
void	registerByteSizeObserver(IterableT iterable, org.apache.beam.sdk.util.common.ElementByteSizeObserver observer) Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
void	verifyDeterministic() Throw Coder.NonDeterministicException if the coding is not deterministic.

Methods inherited from class org.apache.beam.sdk.coders.StructuredCoder

equals, getComponents, hashCode, toString

Methods inherited from class org.apache.beam.sdk.coders.Coder

consistentWithEquals, decode, encode, getEncodedElementByteSize, getEncodedTypeDescriptor, structuralValue, verifyDeterministic, verifyDeterministic

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IterableLikeCoder

protected IterableLikeCoder(Coder<T> elementCoder,
                            java.lang.String iterableName)

Method Detail

getElemCoder

public Coder<T> getElemCoder()

decodeToIterable

protected abstract IterableT decodeToIterable(java.util.List<T> decodedElements)

Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.

Override decodeToIterable(List, long, InputStream) if you need access to the terminator value and the InputStream.

decodeToIterable

protected IterableT decodeToIterable(java.util.List<T> decodedElements,
                                     long terminatorValue,
                                     java.io.InputStream in)
                              throws java.io.IOException

Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements with the InputStream at the position where this coder detected the end of the stream.

Throws:

java.io.IOException

encode

public void encode(IterableT iterable,
                   java.io.OutputStream outStream)
            throws java.io.IOException,
                   CoderException

Description copied from class: Coder

Encodes the given value of type T onto the given output stream. Multiple elements can be encoded next to each other on the output stream, each coder should encode information to know how many bytes to read when decoding. A common approach is to prefix the encoding with the element's encoded length.

Specified by:

encode in class Coder<IterableT extends java.lang.Iterable<T>>

Throws:

java.io.IOException - if writing to the OutputStream fails for some reason

CoderException - if the value could not be encoded for some reason

decode

public IterableT decode(java.io.InputStream inStream)
                 throws java.io.IOException,
                        CoderException

Description copied from class: Coder

Decodes a value of type T from the given input stream in the given context. Returns the decoded value. Multiple elements can be encoded next to each other on the input stream, each coder should encode information to know how many bytes to read when decoding. A common approach is to prefix the encoding with the element's encoded length.

Specified by:

decode in class Coder<IterableT extends java.lang.Iterable<T>>

Throws:

java.io.IOException - if reading from the InputStream fails for some reason

CoderException - if the value could not be decoded for some reason

getCoderArguments

public java.util.List<? extends Coder<?>> getCoderArguments()

Description copied from class: Coder

If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters in the same order they appear within the parameterized type's type signature. If this cannot be done, or this Coder does not encode/decode a parameterized type, returns the empty list.

Specified by:

getCoderArguments in class Coder<IterableT extends java.lang.Iterable<T>>

verifyDeterministic

public void verifyDeterministic()
                         throws Coder.NonDeterministicException

Throw Coder.NonDeterministicException if the coding is not deterministic.

In order for a Coder to be considered deterministic, the following must be true:

• two values that compare as equal (via Object.equals() or Comparable.compareTo(), if supported) have the same encoding.
• the Coder always produces a canonical encoding, which is the same for an instance of an object even if produced on different computers at different times.

Specified by:

verifyDeterministic in class Coder<IterableT extends java.lang.Iterable<T>>

Throws:

NonDeterministicException - always. Encoding is not deterministic for the general Iterable case, as it depends upon the type of iterable. This may allow two objects to compare as equal while the encoding differs.

Coder.NonDeterministicException - if this coder is not deterministic.

isRegisterByteSizeObserverCheap

public boolean isRegisterByteSizeObserverCheap(IterableT iterable)

Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

Not intended to be called by user code, but instead by PipelineRunner implementations.

By default, returns false. The default Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver) implementation invokes Coder.getEncodedElementByteSize(T) which requires re-encoding an element unless it is overridden. This is considered expensive.

Overrides:

isRegisterByteSizeObserverCheap in class Coder<IterableT extends java.lang.Iterable<T>>

Returns:

true if the iterable is of a known class that supports lazy counting of byte size, since that requires minimal extra computation.

registerByteSizeObserver

public void registerByteSizeObserver(IterableT iterable,
                                     org.apache.beam.sdk.util.common.ElementByteSizeObserver observer)
                              throws java.lang.Exception

Description copied from class: Coder

Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.

Not intended to be called by user code, but instead by PipelineRunner implementations.

By default, this notifies observer about the byte size of the encoded value using this coder as returned by Coder.getEncodedElementByteSize(T).

Overrides:

registerByteSizeObserver in class Coder<IterableT extends java.lang.Iterable<T>>

Throws:

java.lang.Exception