Package org.apache.beam.sdk.extensions.sorter

Class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>

org.apache.beam.sdk.extensions.sorter.SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

All Implemented Interfaces:

Serializable, HasDisplayData

public class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>
extends PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>

SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> takes a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>>> with elements consisting of a primary key and iterables over <secondary key, value> pairs, and returns a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>> of the same elements but with values sorted by a secondary key.

This transform ignores the primary key, there is no guarantee of any relationship between elements with different primary keys. The primary key is explicit here only because this transform is typically used on a result of a GroupByKey transform.

This transform sorts by lexicographic comparison of the byte representations of the secondary keys and may write secondary key-value pairs to disk. In order to retrieve the byte representations it requires the input PCollection to use a KvCoder for its input, an IterableCoder for its input values and a KvCoder for its secondary key-value pairs.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method	Description
static <PrimaryKeyT, SecondaryKeyT, ValueT> SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>	create(BufferedExternalSorter.Options sorterOptions)	Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.
PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>	expand(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)	Override this method to specify how this PTransform should be expanded on the given InputT.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

create

public static <PrimaryKeyT,
SecondaryKeyT,
ValueT>
SortValues<PrimaryKeyT,SecondaryKeyT,ValueT> create(BufferedExternalSorter.Options sorterOptions)

Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.

Type Parameters:

PrimaryKeyT - the type of the primary keys of the input and output PCollections

SecondaryKeyT - the type of the secondary (sort) keys of the input and output PCollections

ValueT - the type of the values of the input and output PCollections

expand

public PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> expand(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>