Package org.apache.beam.sdk.coders

@DefaultAnnotation(org.checkerframework.checker.nullness.qual.NonNull.class) package org.apache.beam.sdk.coders

Defines Coders to specify how data is encoded to and decoded from byte strings.

During execution of a Pipeline, elements in a PCollection may need to be encoded into byte strings. This happens both at the beginning and end of a pipeline when data is read from and written to persistent storage and also during execution of a pipeline when elements are communicated between machines.

Exactly when PCollection elements are encoded during execution depends on which PipelineRunner is being used and how that runner chooses to execute the pipeline. As such, Beam requires that all PCollections have an appropriate Coder in case it becomes necessary. In many cases, the Coder can be inferred from the available Java type information and the Pipeline's CoderRegistry. It can be specified per PCollection via PCollection.setCoder(Coder) or per type using the DefaultCoder annotation.

This package provides a number of coders for common types like Integer, String, and List, as well as coders like

invalid reference

org.apache.beam.sdk.coders.AvroCoder

that can be used to encode many custom types.

Related Packages

Package	Description
org.apache.beam.sdk	Provides a simple, powerful model for building both batch and streaming parallel data processing Pipelines.

Class	Description
AtomicCoder<T>	A Coder that has no component Coders or other configuration.
BigDecimalCoder	A BigDecimalCoder encodes a BigDecimal as an integer scale encoded with VarIntCoder and a BigInteger encoded using BigIntegerCoder.
BigEndianIntegerCoder	A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.
BigEndianLongCoder	A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.
BigEndianShortCoder	A BigEndianShortCoder encodes Shorts in 2 bytes, big-endian.
BigIntegerCoder	A BigIntegerCoder encodes a BigInteger as a byte array containing the big endian two's-complement representation, encoded via ByteArrayCoder.
BitSetCoder	Coder for BitSet.
BooleanCoder	A Coder for Boolean.
ByteArrayCoder	A Coder for byte[].
ByteCoder	A ByteCoder encodes Byte values in 1 byte using Java serialization.
CannotProvideCoderException	The exception thrown when a CoderRegistry or CoderProvider cannot provide a Coder that has been requested.
CannotProvideCoderException.ReasonCode	Indicates the reason that Coder inference failed.
Coder<T>	A Coder<T> defines how to encode and decode values of type T into byte streams.
Coder.Context	Deprecated. To implement a coder, do not use any Coder.Context.
Coder.NonDeterministicException	Exception thrown by Coder.verifyDeterministic() if the encoding is not deterministic, including details of why the encoding is not deterministic.
CoderException	An Exception thrown if there is a problem encoding or decoding a value.
CoderProvider	A CoderProvider provides Coders.
CoderProviderRegistrar	Coder creators have the ability to automatically have their coders registered with this SDK by creating a ServiceLoader entry and a concrete implementation of this interface.
CoderProviders	Static utility methods for creating and working with CoderProviders.
CoderRegistry	A CoderRegistry allows creating a Coder for a given Java class or type descriptor.
CollectionCoder<T>	A CollectionCoder encodes Collections in the format of IterableLikeCoder.
CustomCoder<T>	An abstract base class that implements all methods of Coder except Coder.encode(T, java.io.OutputStream) and Coder.decode(java.io.InputStream).
DefaultCoder	The DefaultCoder annotation specifies a Coder class to handle encoding and decoding instances of the annotated class.
DefaultCoder.DefaultCoderProviderRegistrar	A CoderProviderRegistrar that registers a CoderProvider which can use the @DefaultCoder annotation to provide coder providers that creates Coders.
DefaultCoder.DefaultCoderProviderRegistrar.DefaultCoderProvider	A CoderProvider that uses the @DefaultCoder annotation to provide coder providers that create Coders.
DelegateCoder<T,IntermediateT>	A DelegateCoder<T, IntermediateT> wraps a Coder for IntermediateT and encodes/decodes values of type T by converting to/from IntermediateT and then encoding/decoding using the underlying Coder<IntermediateT>.
DelegateCoder.CodingFunction<InputT,OutputT>	A CodingFunction<InputT, OutputT> is a serializable function from InputT to OutputT that may throw any Exception.
DequeCoder<T extends @NonNull Object>	A Coder for Deque, using the format of IterableLikeCoder.
DoubleCoder	A DoubleCoder encodes Double values in 8 bytes using Java serialization.
DurationCoder	A Coder that encodes a joda Duration as a Long using the format of VarLongCoder.
FloatCoder	A FloatCoder encodes Float values in 4 bytes using Java serialization.
InstantCoder	A Coder for joda Instant that encodes it as a big endian Long shifted such that lexicographic ordering of the bytes corresponds to chronological order.
IterableCoder<T>	An IterableCoder encodes any Iterable in the format of IterableLikeCoder.
IterableLikeCoder<T,IterableT extends Iterable<T>>	An abstract base class with functionality for assembling a Coder for a class that implements Iterable.
KvCoder<K extends @Nullable Object,V extends @Nullable Object>	A KvCoder encodes KVs.
LengthPrefixCoder<T>	A Coder which is able to take any existing coder and wrap it such that it is only invoked in the outer context.
ListCoder<T>	A Coder for List, using the format of IterableLikeCoder.
MapCoder<K,V>	A Coder for Maps that encodes them according to provided coders for keys and values.
NullableCoder<T>	A NullableCoder encodes nullable values of type T using a nested Coder<T> that does not tolerate null values.
OptionalCoder<T>	A OptionalCoder encodes optional values of type T using a nested Coder<T>.
RowCoder	A sub-class of SchemaCoder that can only encode Row instances.
RowCoderGenerator	A utility for automatically generating a Coder for Row objects corresponding to a specific schema.
SerializableCoder<T extends Serializable>	A Coder for Java classes that implement Serializable.
SerializableCoder.SerializableCoderProviderRegistrar	A CoderProviderRegistrar which registers a CoderProvider which can handle serializable types.
SetCoder<T>	A SetCoder encodes any Set using the format of IterableLikeCoder.
ShardedKeyCoder<KeyT>	A Coder for ShardedKey, using a wrapped key Coder.
SnappyCoder<T>	Wraps an existing coder with Snappy compression.
SortedMapCoder<K extends Comparable<? super K>,V>	A Coder for Maps that encodes them according to provided coders for keys and values.
StringDelegateCoder<T>	A Coder that wraps a Coder<String> and encodes/decodes values via string representations.
StringUtf8Coder	A Coder that encodes Strings in UTF-8 encoding.
StructuralByteArray	A wrapper around a byte[] that uses structural, value-based equality rather than byte[]'s normal object identity.
StructuredCoder<T>	An abstract base class to implement a Coder that defines equality, hashing, and printing via the class name and recursively using StructuredCoder.getComponents().
TextualIntegerCoder	A Coder that encodes Integer Integers as the ASCII bytes of their textual, decimal, representation.
TimestampPrefixingWindowCoder<T extends BoundedWindow>	A TimestampPrefixingWindowCoder wraps arbitrary user custom window coder.
VarIntCoder	A Coder that encodes Integers using between 1 and 5 bytes.
VarLongCoder	A Coder that encodes Longs using between 1 and 10 bytes.
VoidCoder	A Coder for Void.
ZstdCoder<T>	Wraps an existing coder with Zstandard compression.