Package org.apache.beam.sdk.values

Class PCollectionList<T>

java.lang.Object

org.apache.beam.sdk.values.PCollectionList<T>

Type Parameters:

T - the type of the elements of all the PCollections in this list

All Implemented Interfaces:

PInput, POutput

public class PCollectionList<T>
extends Object
implements PInput, POutput

A PCollectionList<T> is an immutable list of homogeneously typed PCollection<T>s. A PCollectionList is used, for instance, as the input to Flatten or the output of Partition.

PCollectionLists can be created and accessed like follows:

 PCollection<String> pc1 = ...;
 PCollection<String> pc2 = ...;
 PCollection<String> pc3 = ...;

 // Create a PCollectionList with three PCollections:
 PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);

 // Create an empty PCollectionList:
 Pipeline p = ...;
 PCollectionList<String> pcs2 = PCollectionList.<String>empty(p);

 // Get PCollections out of a PCollectionList, by index (origin 0):
 PCollection<String> pcX = pcs.get(1);
 PCollection<String> pcY = pcs.get(0);
 PCollection<String> pcZ = pcs.get(2);

 // Get a list of all PCollections in a PCollectionList:
 List<PCollection<String>> allPcs = pcs.getAll();
 

Method Summary

Modifier and Type	Method	Description
PCollectionList<T>	and(Iterable<PCollection<T>> pcs)	Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.
PCollectionList<T>	and(PCollection<T> pc)	Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.
<OutputT extends POutput> OutputT	apply(String name, PTransform<PCollectionList<T>,OutputT> t)	Applies the given PTransform to this input PCollectionList, using name to identify this specific application of the transform.
<OutputT extends POutput> OutputT	apply(PTransform<PCollectionList<T>,OutputT> t)	Like apply(String, PTransform) but defaulting to the name of the PTransform.
static <T> PCollectionList<T>	empty(Pipeline pipeline)	Returns an empty PCollectionList that is part of the given Pipeline.
boolean	equals(@Nullable Object other)
Map<TupleTag<?>,PValue>	expand()	Expands this PInput into a list of its component output PValues.
void	finishSpecifyingOutput(String transformName, PInput input, PTransform<?,?> transform)	As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.
PCollection<T>	get(int index)	Returns the PCollection at the given index (origin zero).
List<PCollection<T>>	getAll()	Returns an immutable List of all the PCollections in this PCollectionList.
Pipeline	getPipeline()	Returns the owning Pipeline of this PInput.
int	hashCode()
static <T> PCollectionList<T>	of(Iterable<PCollection<T>> pcs)	Returns a PCollectionList containing the given PCollections, in order.
static <T> PCollectionList<T>	of(PCollection<T> pc)	Returns a singleton PCollectionList containing the given PCollection.
int	size()	Returns the number of PCollections in this PCollectionList.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Method Details

empty

public static <T> PCollectionList<T> empty(Pipeline pipeline)

Returns an empty PCollectionList that is part of the given Pipeline.

Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on the result.

of

public static <T> PCollectionList<T> of(PCollection<T> pc)

Returns a singleton PCollectionList containing the given PCollection.

Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on the result.

of

public static <T> PCollectionList<T> of(Iterable<PCollection<T>> pcs)

Returns a PCollectionList containing the given PCollections, in order.

The argument list cannot be empty.

All the PCollections in the resulting PCollectionList must be part of the same Pipeline.

Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on the result.

and

public PCollectionList<T> and(PCollection<T> pc)

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.

All the PCollections in the resulting PCollectionList must be part of the same Pipeline.

and

public PCollectionList<T> and(Iterable<PCollection<T>> pcs)

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.

All the PCollections in the resulting PCollectionList must be part of the same Pipeline.

size

public int size()

Returns the number of PCollections in this PCollectionList.

get

public PCollection<T> get(int index)

Returns the PCollection at the given index (origin zero).

Throws:

IndexOutOfBoundsException - if the index is out of the range [0..size()-1].

getAll

public List<PCollection<T>> getAll()

Returns an immutable List of all the PCollections in this PCollectionList.

apply

public <OutputT extends POutput> OutputT apply(PTransform<PCollectionList<T>,OutputT> t)

Like apply(String, PTransform) but defaulting to the name of the PTransform.

apply

public <OutputT extends POutput> OutputT apply(String name,
 PTransform<PCollectionList<T>,OutputT> t)

Applies the given PTransform to this input PCollectionList, using name to identify this specific application of the transform. This name is used in various places, including the monitoring UI, logging, and to stably identify this application node in the job graph.

Returns:

the output of the applied PTransform

getPipeline

public Pipeline getPipeline()

Description copied from interface: PInput

Returns the owning Pipeline of this PInput.

Specified by:

getPipeline in interface PInput

Specified by:

getPipeline in interface POutput

expand

public Map<TupleTag<?>,PValue> expand()

Description copied from interface: PInput

Expands this PInput into a list of its component output PValues.

• A PValue expands to itself.
• A tuple or list of PValues (such as PCollectionTuple or PCollectionList) expands to its component PValue PValues.

Not intended to be invoked directly by user code.

Specified by:

expand in interface PInput

Specified by:

expand in interface POutput

finishSpecifyingOutput

public void finishSpecifyingOutput(String transformName,
 PInput input,
 PTransform<?,?> transform)

Description copied from interface: POutput

As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.

This includes ensuring that all PCollections have Coders specified or defaulted.

Automatically invoked whenever this POutput is output, after POutput.finishSpecifyingOutput(String, PInput, PTransform) has been called on each component PValue returned by POutput.expand().

Specified by:

finishSpecifyingOutput in interface POutput

equals

public boolean equals(@Nullable Object other)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object