Class PCollectionRowTuple
PCollectionRowTuple
is an immutable tuple of PCollections
,
"keyed" by a string tag. A PCollectionRowTuple
can be used as the input or output of a
PTransform
taking or producing multiple PCollection<Row>
inputs or outputs.
A PCollectionRowTuple
can be created and accessed like follows:
PCollection<Row> pc1 = ...;
PCollection<Row> pc2 = ...;
// Create tags for each of the PCollections to put in the PCollectionRowTuple:
String tag1 = "pc1";
String tag2 = "pc2";
String tag3 = "pc3";
// Create a PCollectionRowTuple with three PCollections:
PCollectionRowTuple pcs = PCollectionRowTuple.of(tag1, pc1).and(tag2, pc2).and(tag3, pc3);
// Create an empty PCollectionRowTuple:
Pipeline p = ...;
PCollectionRowTuple pcs2 = PCollectionRowTuple.empty(p);
// Get PCollections out of a PCollectionRowTuple, using the same tags that were used to put them in:
PCollection<Row> pcX = pcs.get(tag2);
PCollection<Row> pcY = pcs.get(tag1);
// Get a map of all PCollections in a PCollectionRowTuple:
Map<String, PCollection<Row>> allPcs = pcs.getAll();
-
Method Summary
Modifier and TypeMethodDescriptionand
(String tag, PCollection<Row> pc) Returns a newPCollectionRowTuple
that has eachPCollection
and tag of thisPCollectionRowTuple
plus the givenPCollection
associated with the given tag.<OutputT extends POutput>
OutputTapply
(String name, PTransform<? super PCollectionRowTuple, OutputT> t) Applies the givenPTransform
to this inputPCollectionRowTuple
, usingname
to identify this specific application of the transform.<OutputT extends POutput>
OutputTapply
(PTransform<? super PCollectionRowTuple, OutputT> t) Likeapply(String, PTransform)
but defaulting to the name of thePTransform
.static PCollectionRowTuple
Returns an emptyPCollectionRowTuple
that is part of the givenPipeline
.boolean
expand()
void
finishSpecifyingOutput
(String transformName, PInput input, PTransform<?, ?> transform) As part of applying the producingPTransform
, finalizes this output to make it ready for being used as an input and for running.getAll()
Returns an immutable Map from tag to correspondingPCollection
, for all the members of thisPCollectionRowTuple
.Likeget(String)
, but is a convenience method to get a single PCollection without providing a tag for that output.boolean
Returns whether thisPCollectionRowTuple
contains aPCollection
with the given tag.int
hashCode()
static PCollectionRowTuple
of
(String tag, PCollection<Row> pc) Returns a singletonPCollectionRowTuple
containing the givenPCollection
keyed by the given tag.static PCollectionRowTuple
of
(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2) A version ofof(String, PCollection)
that takes in two PCollections of the same type.static PCollectionRowTuple
of
(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3) A version ofof(String, PCollection)
that takes in three PCollections of the same type.static PCollectionRowTuple
of
(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4) A version ofof(String, PCollection)
that takes in four PCollections of the same type.static PCollectionRowTuple
of
(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4, String tag5, PCollection<Row> pc5) A version ofof(String, PCollection)
that takes in five PCollections of the same type.
-
Method Details
-
empty
Returns an emptyPCollectionRowTuple
that is part of the givenPipeline
.A
PCollectionRowTuple
containing additional elements can be created by callingand(java.lang.String, org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>)
on the result. -
of
Returns a singletonPCollectionRowTuple
containing the givenPCollection
keyed by the given tag.A
PCollectionRowTuple
containing additional elements can be created by callingand(java.lang.String, org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>)
on the result. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2) A version ofof(String, PCollection)
that takes in two PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3) A version ofof(String, PCollection)
that takes in three PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4) A version ofof(String, PCollection)
that takes in four PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4, String tag5, PCollection<Row> pc5) A version ofof(String, PCollection)
that takes in five PCollections of the same type. -
and
Returns a newPCollectionRowTuple
that has eachPCollection
and tag of thisPCollectionRowTuple
plus the givenPCollection
associated with the given tag.The given tag should not already be mapped to a
PCollection
in thisPCollectionRowTuple
.Each
PCollection
in the resultingPCollectionRowTuple
must be part of the samePipeline
. -
has
Returns whether thisPCollectionRowTuple
contains aPCollection
with the given tag. -
get
Returns thePCollection
associated with the givenString
in thisPCollectionRowTuple
. ThrowsIllegalArgumentException
if there is no suchPCollection
, i.e.,!has(tag)
. -
getSinglePCollection
Likeget(String)
, but is a convenience method to get a single PCollection without providing a tag for that output. Use only when there is a single collection in this tuple.Throws
IllegalStateException
if more than one output exists in thePCollectionRowTuple
. -
getAll
Returns an immutable Map from tag to correspondingPCollection
, for all the members of thisPCollectionRowTuple
. -
apply
Likeapply(String, PTransform)
but defaulting to the name of thePTransform
.- Returns:
- the output of the applied
PTransform
-
apply
public <OutputT extends POutput> OutputT apply(String name, PTransform<? super PCollectionRowTuple, OutputT> t) Applies the givenPTransform
to this inputPCollectionRowTuple
, usingname
to identify this specific application of the transform. This name is used in various places, including the monitoring UI, logging, and to stably identify this application node in the job graph.- Returns:
- the output of the applied
PTransform
-
getPipeline
Description copied from interface:PInput
- Specified by:
getPipeline
in interfacePInput
- Specified by:
getPipeline
in interfacePOutput
-
expand
Description copied from interface:PInput
Expands thisPInput
into a list of its component outputPValues
.- A
PValue
expands to itself. - A tuple or list of
PValues
(such asPCollectionTuple
orPCollectionList
) expands to its componentPValue PValues
.
Not intended to be invoked directly by user code.
- A
-
finishSpecifyingOutput
Description copied from interface:POutput
As part of applying the producingPTransform
, finalizes this output to make it ready for being used as an input and for running.This includes ensuring that all
PCollections
haveCoders
specified or defaulted.Automatically invoked whenever this
POutput
is output, afterPOutput.finishSpecifyingOutput(String, PInput, PTransform)
has been called on each componentPValue
returned byPOutput.expand()
.- Specified by:
finishSpecifyingOutput
in interfacePOutput
-
equals
-
hashCode
public int hashCode()
-