T - the type of the elements of all the PCollections in this listpublic class PCollectionList<T> extends java.lang.Object implements PInput, POutput
PCollectionList<T> is an immutable list of homogeneously typed
PCollection<T>s. A PCollectionList is used, for instance, as
the input to Flatten or the output of Partition.
PCollectionLists can be created and accessed like follows:
PCollection<String> pc1 = ...;
PCollection<String> pc2 = ...;
PCollection<String> pc3 = ...;
// Create a PCollectionList with three PCollections:
PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
// Create an empty PCollectionList:
Pipeline p = ...;
PCollectionList<String> pcs2 = PCollectionList.<String>empty(p);
// Get PCollections out of a PCollectionList, by index (origin 0):
PCollection<String> pcX = pcs.get(1);
PCollection<String> pcY = pcs.get(0);
PCollection<String> pcZ = pcs.get(2);
// Get a list of all PCollections in a PCollectionList:
List<PCollection<String>> allPcs = pcs.getAll();
| Modifier and Type | Method and Description |
|---|---|
PCollectionList<T> |
and(java.lang.Iterable<PCollection<T>> pcs)
Returns a new
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollections appended to the end, in
order. |
PCollectionList<T> |
and(PCollection<T> pc)
Returns a new
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollection appended to the end. |
<OutputT extends POutput> |
apply(PTransform<PCollectionList<T>,OutputT> t)
Like
apply(String, PTransform) but defaulting to the name of the PTransform. |
<OutputT extends POutput> |
apply(java.lang.String name,
PTransform<PCollectionList<T>,OutputT> t)
Applies the given
PTransform to this input PCollectionList, using name
to identify this specific application of the transform. |
static <T> PCollectionList<T> |
empty(Pipeline pipeline)
Returns an empty
PCollectionList that is part of the given Pipeline. |
boolean |
equals(@Nullable java.lang.Object other) |
java.util.Map<TupleTag<?>,PValue> |
expand()
|
void |
finishSpecifyingOutput(java.lang.String transformName,
PInput input,
PTransform<?,?> transform)
As part of applying the producing
PTransform, finalizes this output to make it ready
for being used as an input and for running. |
PCollection<T> |
get(int index)
Returns the
PCollection at the given index (origin zero). |
java.util.List<PCollection<T>> |
getAll()
Returns an immutable List of all the
PCollections in this PCollectionList. |
Pipeline |
getPipeline()
|
int |
hashCode() |
static <T> PCollectionList<T> |
of(java.lang.Iterable<PCollection<T>> pcs)
Returns a
PCollectionList containing the given PCollections, in
order. |
static <T> PCollectionList<T> |
of(PCollection<T> pc)
Returns a singleton
PCollectionList containing the given PCollection. |
int |
size()
Returns the number of
PCollections in this PCollectionList. |
public static <T> PCollectionList<T> empty(Pipeline pipeline)
PCollectionList that is part of the given Pipeline.
Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on
the result.
public static <T> PCollectionList<T> of(PCollection<T> pc)
PCollectionList containing the given PCollection.
Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on
the result.
public static <T> PCollectionList<T> of(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList containing the given PCollections, in
order.
The argument list cannot be empty.
All the PCollections in the resulting PCollectionList must be
part of the same Pipeline.
Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>) on the result.
public PCollectionList<T> and(PCollection<T> pc)
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollection appended to the end.
All the PCollections in the resulting PCollectionList must be
part of the same Pipeline.
public PCollectionList<T> and(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollections appended to the end, in
order.
All the PCollections in the resulting PCollectionList must be
part of the same Pipeline.
public int size()
PCollections in this PCollectionList.public PCollection<T> get(int index)
PCollection at the given index (origin zero).java.lang.IndexOutOfBoundsException - if the index is out of the range [0..size()-1].public java.util.List<PCollection<T>> getAll()
PCollections in this PCollectionList.public <OutputT extends POutput> OutputT apply(PTransform<PCollectionList<T>,OutputT> t)
apply(String, PTransform) but defaulting to the name of the PTransform.public <OutputT extends POutput> OutputT apply(java.lang.String name, PTransform<PCollectionList<T>,OutputT> t)
PTransform to this input PCollectionList, using name
to identify this specific application of the transform. This name is used in various places,
including the monitoring UI, logging, and to stably identify this application node in the job
graph.PTransformpublic Pipeline getPipeline()
PInputgetPipeline in interface PInputgetPipeline in interface POutputpublic java.util.Map<TupleTag<?>,PValue> expand()
PInputPInput into a list of its component output PValues.
PValue expands to itself.
PValues (such as PCollectionTuple or PCollectionList) expands to its component PValue PValues.
Not intended to be invoked directly by user code.
public void finishSpecifyingOutput(java.lang.String transformName,
PInput input,
PTransform<?,?> transform)
POutputPTransform, finalizes this output to make it ready
for being used as an input and for running.
This includes ensuring that all PCollections have Coders specified or defaulted.
Automatically invoked whenever this POutput is output, after POutput.finishSpecifyingOutput(String, PInput, PTransform) has been called on each component
PValue returned by POutput.expand().
finishSpecifyingOutput in interface POutputpublic boolean equals(@Nullable java.lang.Object other)
equals in class java.lang.Objectpublic int hashCode()
hashCode in class java.lang.Object