T
- the type of the elements of all the PCollections
in this listpublic class PCollectionList<T> extends java.lang.Object implements PInput, POutput
PCollectionList<T>
is an immutable list of homogeneously
typed PCollection<T>s
. A PCollectionList
is used, for
instance, as the input to
Flatten
or the output of Partition
.
PCollectionLists can be created and accessed like follows:
PCollection<String> pc1 = ...;
PCollection<String> pc2 = ...;
PCollection<String> pc3 = ...;
// Create a PCollectionList with three PCollections:
PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
// Create an empty PCollectionList:
Pipeline p = ...;
PCollectionList<String> pcs2 = PCollectionList.<String>empty(p);
// Get PCollections out of a PCollectionList, by index (origin 0):
PCollection<String> pcX = pcs.get(1);
PCollection<String> pcY = pcs.get(0);
PCollection<String> pcZ = pcs.get(2);
// Get a list of all PCollections in a PCollectionList:
List<PCollection<String>> allPcs = pcs.getAll();
Modifier and Type | Method and Description |
---|---|
PCollectionList<T> |
and(java.lang.Iterable<PCollection<T>> pcs)
Returns a new
PCollectionList that has all the PCollections of
this PCollectionList plus the given PCollections appended to the
end, in order. |
PCollectionList<T> |
and(PCollection<T> pc)
Returns a new
PCollectionList that has all the PCollections of
this PCollectionList plus the given PCollection appended to the end. |
<OutputT extends POutput> |
apply(PTransform<PCollectionList<T>,OutputT> t)
Like
apply(String, PTransform) but defaulting to the name
of the PTransform . |
<OutputT extends POutput> |
apply(java.lang.String name,
PTransform<PCollectionList<T>,OutputT> t)
Applies the given
PTransform to this input PCollectionList ,
using name to identify this specific application of the transform. |
static <T> PCollectionList<T> |
empty(Pipeline pipeline)
Returns an empty
PCollectionList that is part of the given Pipeline . |
boolean |
equals(java.lang.Object other) |
java.util.Map<TupleTag<?>,PValue> |
expand()
|
void |
finishSpecifyingOutput(java.lang.String transformName,
PInput input,
PTransform<?,?> transform)
As part of applying the producing
PTransform , finalizes this output to make it ready
for being used as an input and for running. |
PCollection<T> |
get(int index)
Returns the
PCollection at the given index (origin zero). |
java.util.List<PCollection<T>> |
getAll()
Returns an immutable List of all the
PCollections in this
PCollectionList . |
Pipeline |
getPipeline()
|
int |
hashCode() |
static <T> PCollectionList<T> |
of(java.lang.Iterable<PCollection<T>> pcs)
Returns a
PCollectionList containing the given PCollections ,
in order. |
static <T> PCollectionList<T> |
of(PCollection<T> pc)
Returns a singleton
PCollectionList containing the given PCollection . |
int |
size()
Returns the number of
PCollections in this PCollectionList . |
public static <T> PCollectionList<T> empty(Pipeline pipeline)
PCollectionList
that is part of the given Pipeline
.
Longer PCollectionLists
can be created by calling
and(org.apache.beam.sdk.values.PCollection<T>)
on the result.
public static <T> PCollectionList<T> of(PCollection<T> pc)
PCollectionList
containing the given PCollection
.
Longer PCollectionLists
can be created by calling
and(org.apache.beam.sdk.values.PCollection<T>)
on the result.
public static <T> PCollectionList<T> of(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList
containing the given PCollections
,
in order.
The argument list cannot be empty.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
Longer PCollectionLists can be created by calling
and(org.apache.beam.sdk.values.PCollection<T>)
on the result.
public PCollectionList<T> and(PCollection<T> pc)
PCollectionList
that has all the PCollections
of
this PCollectionList
plus the given PCollection
appended to the end.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
public PCollectionList<T> and(java.lang.Iterable<PCollection<T>> pcs)
PCollectionList
that has all the PCollections
of
this PCollectionList
plus the given PCollections
appended to the
end, in order.
All the PCollections
in the resulting PCollectionList
must be
part of the same Pipeline
.
public int size()
PCollections
in this PCollectionList
.public PCollection<T> get(int index)
PCollection
at the given index (origin zero).java.lang.IndexOutOfBoundsException
- if the index is out of the range
[0..size()-1]
.public java.util.List<PCollection<T>> getAll()
PCollections
in this
PCollectionList
.public <OutputT extends POutput> OutputT apply(PTransform<PCollectionList<T>,OutputT> t)
apply(String, PTransform)
but defaulting to the name
of the PTransform
.public <OutputT extends POutput> OutputT apply(java.lang.String name, PTransform<PCollectionList<T>,OutputT> t)
PTransform
to this input PCollectionList
,
using name
to identify this specific application of the transform.
This name is used in various places, including the monitoring UI, logging,
and to stably identify this application node in the job graph.PTransform
public Pipeline getPipeline()
PInput
getPipeline
in interface PInput
getPipeline
in interface POutput
public java.util.Map<TupleTag<?>,PValue> expand()
PInput
PInput
into a list of its component output
PValues
.
PValue
expands to itself.PValues
(such as
PCollectionTuple
or PCollectionList
)
expands to its component PValue PValues
.Not intended to be invoked directly by user code.
public void finishSpecifyingOutput(java.lang.String transformName, PInput input, PTransform<?,?> transform)
POutput
PTransform
, finalizes this output to make it ready
for being used as an input and for running.
This includes ensuring that all PCollections
have Coders
specified or defaulted.
Automatically invoked whenever this POutput
is output, after
POutput.finishSpecifyingOutput(String, PInput, PTransform)
has been called on each
component PValue
returned by POutput.expand()
.
finishSpecifyingOutput
in interface POutput
public boolean equals(java.lang.Object other)
equals
in class java.lang.Object
public int hashCode()
hashCode
in class java.lang.Object