org.apache.beam.sdk.extensions.sorter

Class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>>

org.apache.beam.sdk.extensions.sorter.SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

public class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>
extends PTransform<PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>>

SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> takes a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>>> with elements consisting of a primary key and iterables over <secondary key, value> pairs, and returns a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>> of the same elements but with values sorted by a secondary key.

This transform ignores the primary key, there is no guarantee of any relationship between elements with different primary keys. The primary key is explicit here only because this transform is typically used on a result of a GroupByKey transform.

This transform sorts by lexicographic comparison of the byte representations of the secondary keys and may write secondary key-value pairs to disk. In order to retrieve the byte representations it requires the input PCollection to use a KvCoder for its input, an IterableCoder for its input values and a KvCoder for its secondary key-value pairs.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Method Summary

Modifier and Type	Method and Description
static <PrimaryKeyT,SecondaryKeyT,ValueT> SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>	create(BufferedExternalSorter.Options sorterOptions) Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.
PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>	expand(PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> input) Applies this PTransform on the given InputT, and returns its Output.
protected Coder<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>	getDefaultOutputCoder(PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> input) Returns the default Coder to use for the output of this single-output PTransform when applied to the given input.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

create

public static <PrimaryKeyT,SecondaryKeyT,ValueT> SortValues<PrimaryKeyT,SecondaryKeyT,ValueT> create(BufferedExternalSorter.Options sorterOptions)

Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.

Type Parameters:

PrimaryKeyT - the type of the primary keys of the input and output PCollections

SecondaryKeyT - the type of the secondary (sort) keys of the input and output PCollections

ValueT - the type of the values of the input and output PCollections

expand

public PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> expand(PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Description copied from class: PTransform

Applies this PTransform on the given InputT, and returns its Output.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>>

getDefaultOutputCoder

protected Coder<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> getDefaultOutputCoder(PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Description copied from class: PTransform

Returns the default Coder to use for the output of this single-output PTransform when applied to the given input.

By default, always throws.

Overrides:

getDefaultOutputCoder in class PTransform<PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,java.lang.Iterable<KV<SecondaryKeyT,ValueT>>>>>