T
- the type of the elements of all the PCollections
in this listpublic class PCollectionList<T> extends java.lang.Object implements PInput, POutput
PCollectionList<T>
is an immutable list of homogeneously typed
PCollection<T>s
. A PCollectionList
is used, for instance, as
the input to Flatten
or the output of Partition
.
PCollectionLists can be created and accessed like follows:
PCollection<String> pc1 = ...;
PCollection<String> pc2 = ...;
PCollection<String> pc3 = ...;
// Create a PCollectionList with three PCollections:
PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
// Create an empty PCollectionList:
Pipeline p = ...;
PCollectionList<String> pcs2 = PCollectionList.<String>empty(p);
// Get PCollections out of a PCollectionList, by index (origin 0):
PCollection<String> pcX = pcs.get(1);
PCollection<String> pcY = pcs.get(0);
PCollection<String> pcZ = pcs.get(2);
// Get a list of all PCollections in a PCollectionList:
List<PCollection<String>> allPcs = pcs.getAll();
Modifier and Type | Method and Description |
---|---|
PCollectionList<T> |
and(java.lang.Iterable<PCollection<T>> pcs)
Returns a new
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollections appended to the end, in
order. |
PCollectionList<T> |
and(PCollection<T> pc)
Returns a new
PCollectionList that has all the PCollections of this
PCollectionList plus the given PCollection appended to the end. |
<OutputT extends POutput> |
apply(PTransform<PCollectionList<T>,OutputT> t)
Like
apply(String, PTransform) but defaulting to the name of the PTransform . |
<OutputT extends POutput> |
apply(java.lang.String name,
PTransform<PCollectionList<T>,OutputT> t)
Applies the given
PTransform to this input PCollectionList , using name
to identify this specific application of the transform. |
static <T> PCollectionList<T> |
empty(Pipeline pipeline)
Returns an empty
PCollectionList that is part of the given Pipeline . |
boolean |
equals(@Nullable java.lang.Object other) |
java.util.Map<TupleTag<?>,PValue> |
expand()
|
void |
finishSpecifyingOutput(java.lang.String transformName,
PInput input,
PTransform<?,?> transform)
As part of applying the producing
PTransform , finalizes this output to make it ready
for being used as an input and for running. |
PCollection<T> |
get(int index)
Returns the
PCollection at the given index (origin zero). |
java.util.List<PCollection<T>> |
getAll()
Returns an immutable List of all the
PCollections in this PCollectionList . |
Pipeline |
getPipeline()
|
int |
hashCode() |
static <T> PCollectionList<T> |
of(java.lang.Iterable<PCollection<T>> pcs)
Returns a
PCollectionList containing the given PCollections , in
order. |
static <T> PCollectionList<T> |
of(PCollection<T> pc)
Returns a singleton
PCollectionList containing the given PCollection . |
int |
size()
Returns the number of
PCollections in this PCollectionList . |
public static <T> PCollectionList<T> empty(Pipeline pipeline)
PCollectionList
that is part of the given Pipeline
.
Longer PCollectionLists
can be created by calling and(org.apache.beam.sdk.values.PCollection<T>)
on
the result.
public static <T> PCollectionList<T> of(PCollection<T> pc)
PCollectionList
containing the given PCollection
.
Longer PCollectionLists
can be created by calling and(org.apache.beam.sdk.values.PCollection<T>)
on
the result.
public static <T> PCollectionList<T> of(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList
containing the given PCollections
, in
order.
The argument list cannot be empty.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
Longer PCollectionLists can be created by calling and(org.apache.beam.sdk.values.PCollection<T>)
on the result.
public PCollectionList<T> and(PCollection<T> pc)
PCollectionList
that has all the PCollections
of this
PCollectionList
plus the given PCollection
appended to the end.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
public PCollectionList<T> and(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList
that has all the PCollections
of this
PCollectionList
plus the given PCollections
appended to the end, in
order.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
public int size()
PCollections
in this PCollectionList
.public PCollection<T> get(int index)
PCollection
at the given index (origin zero).java.lang.IndexOutOfBoundsException
- if the index is out of the range [0..size()-1]
.public java.util.List<PCollection<T>> getAll()
PCollections
in this PCollectionList
.public <OutputT extends POutput> OutputT apply(PTransform<PCollectionList<T>,OutputT> t)
apply(String, PTransform)
but defaulting to the name of the PTransform
.public <OutputT extends POutput> OutputT apply(java.lang.String name, PTransform<PCollectionList<T>,OutputT> t)
PTransform
to this input PCollectionList
, using name
to identify this specific application of the transform. This name is used in various places,
including the monitoring UI, logging, and to stably identify this application node in the job
graph.PTransform
public Pipeline getPipeline()
PInput
getPipeline
in interface PInput
getPipeline
in interface POutput
public java.util.Map<TupleTag<?>,PValue> expand()
PInput
PInput
into a list of its component output PValues
.
PValue
expands to itself.
PValues
(such as PCollectionTuple
or PCollectionList
) expands to its component PValue PValues
.
Not intended to be invoked directly by user code.
public void finishSpecifyingOutput(java.lang.String transformName, PInput input, PTransform<?,?> transform)
POutput
PTransform
, finalizes this output to make it ready
for being used as an input and for running.
This includes ensuring that all PCollections
have Coders
specified or defaulted.
Automatically invoked whenever this POutput
is output, after POutput.finishSpecifyingOutput(String, PInput, PTransform)
has been called on each component
PValue
returned by POutput.expand()
.
finishSpecifyingOutput
in interface POutput
public boolean equals(@Nullable java.lang.Object other)
equals
in class java.lang.Object
public int hashCode()
hashCode
in class java.lang.Object