Class PCollectionRowTuple
PCollectionRowTuple is an immutable tuple of PCollections
,
"keyed" by a string tag. A PCollectionRowTuple can be used as the input or output of a
PTransform taking or producing multiple PCollection<Row> inputs or outputs.
A PCollectionRowTuple can be created and accessed like follows:
PCollection<Row> pc1 = ...;
PCollection<Row> pc2 = ...;
// Create tags for each of the PCollections to put in the PCollectionRowTuple:
String tag1 = "pc1";
String tag2 = "pc2";
String tag3 = "pc3";
// Create a PCollectionRowTuple with three PCollections:
PCollectionRowTuple pcs = PCollectionRowTuple.of(tag1, pc1).and(tag2, pc2).and(tag3, pc3);
// Create an empty PCollectionRowTuple:
Pipeline p = ...;
PCollectionRowTuple pcs2 = PCollectionRowTuple.empty(p);
// Get PCollections out of a PCollectionRowTuple, using the same tags that were used to put them in:
PCollection<Row> pcX = pcs.get(tag2);
PCollection<Row> pcY = pcs.get(tag1);
// Get a map of all PCollections in a PCollectionRowTuple:
Map<String, PCollection<Row>> allPcs = pcs.getAll();
-
Method Summary
Modifier and TypeMethodDescriptionand(String tag, PCollection<Row> pc) Returns a newPCollectionRowTuplethat has eachPCollectionand tag of thisPCollectionRowTupleplus the givenPCollectionassociated with the given tag.<OutputT extends POutput>
OutputTapply(String name, PTransform<? super PCollectionRowTuple, OutputT> t) Applies the givenPTransformto this inputPCollectionRowTuple, usingnameto identify this specific application of the transform.<OutputT extends POutput>
OutputTapply(PTransform<? super PCollectionRowTuple, OutputT> t) Likeapply(String, PTransform)but defaulting to the name of thePTransform.static PCollectionRowTupleReturns an emptyPCollectionRowTuplethat is part of the givenPipeline.booleanexpand()voidfinishSpecifyingOutput(String transformName, PInput input, PTransform<?, ?> transform) As part of applying the producingPTransform, finalizes this output to make it ready for being used as an input and for running.getAll()Returns an immutable Map from tag to correspondingPCollection, for all the members of thisPCollectionRowTuple.Likeget(String), but is a convenience method to get a single PCollection without providing a tag for that output.booleanReturns whether thisPCollectionRowTuplecontains aPCollectionwith the given tag.inthashCode()static PCollectionRowTupleof(String tag, PCollection<Row> pc) Returns a singletonPCollectionRowTuplecontaining the givenPCollectionkeyed by the given tag.static PCollectionRowTupleof(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2) A version ofof(String, PCollection)that takes in two PCollections of the same type.static PCollectionRowTupleof(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3) A version ofof(String, PCollection)that takes in three PCollections of the same type.static PCollectionRowTupleof(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4) A version ofof(String, PCollection)that takes in four PCollections of the same type.static PCollectionRowTupleof(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4, String tag5, PCollection<Row> pc5) A version ofof(String, PCollection)that takes in five PCollections of the same type.
-
Method Details
-
empty
Returns an emptyPCollectionRowTuplethat is part of the givenPipeline.A
PCollectionRowTuplecontaining additional elements can be created by callingand(java.lang.String, org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>)on the result. -
of
Returns a singletonPCollectionRowTuplecontaining the givenPCollectionkeyed by the given tag.A
PCollectionRowTuplecontaining additional elements can be created by callingand(java.lang.String, org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>)on the result. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2) A version ofof(String, PCollection)that takes in two PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3) A version ofof(String, PCollection)that takes in three PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4) A version ofof(String, PCollection)that takes in four PCollections of the same type. -
of
public static PCollectionRowTuple of(String tag1, PCollection<Row> pc1, String tag2, PCollection<Row> pc2, String tag3, PCollection<Row> pc3, String tag4, PCollection<Row> pc4, String tag5, PCollection<Row> pc5) A version ofof(String, PCollection)that takes in five PCollections of the same type. -
and
Returns a newPCollectionRowTuplethat has eachPCollectionand tag of thisPCollectionRowTupleplus the givenPCollectionassociated with the given tag.The given tag should not already be mapped to a
PCollectionin thisPCollectionRowTuple.Each
PCollectionin the resultingPCollectionRowTuplemust be part of the samePipeline. -
has
Returns whether thisPCollectionRowTuplecontains aPCollectionwith the given tag. -
get
Returns thePCollectionassociated with the givenStringin thisPCollectionRowTuple. ThrowsIllegalArgumentExceptionif there is no suchPCollection, i.e.,!has(tag). -
getSinglePCollection
Likeget(String), but is a convenience method to get a single PCollection without providing a tag for that output. Use only when there is a single collection in this tuple.Throws
IllegalStateExceptionif more than one output exists in thePCollectionRowTuple. -
getAll
Returns an immutable Map from tag to correspondingPCollection, for all the members of thisPCollectionRowTuple. -
apply
Likeapply(String, PTransform)but defaulting to the name of thePTransform.- Returns:
- the output of the applied
PTransform
-
apply
public <OutputT extends POutput> OutputT apply(String name, PTransform<? super PCollectionRowTuple, OutputT> t) Applies the givenPTransformto this inputPCollectionRowTuple, usingnameto identify this specific application of the transform. This name is used in various places, including the monitoring UI, logging, and to stably identify this application node in the job graph.- Returns:
- the output of the applied
PTransform
-
getPipeline
Description copied from interface:PInput- Specified by:
getPipelinein interfacePInput- Specified by:
getPipelinein interfacePOutput
-
expand
Description copied from interface:PInputExpands thisPInputinto a list of its component outputPValues.- A
PValueexpands to itself. - A tuple or list of
PValues(such asPCollectionTupleorPCollectionList) expands to its componentPValue PValues.
Not intended to be invoked directly by user code.
- A
-
finishSpecifyingOutput
Description copied from interface:POutputAs part of applying the producingPTransform, finalizes this output to make it ready for being used as an input and for running.This includes ensuring that all
PCollectionshaveCodersspecified or defaulted.Automatically invoked whenever this
POutputis output, afterPOutput.finishSpecifyingOutput(String, PInput, PTransform)has been called on each componentPValuereturned byPOutput.expand().- Specified by:
finishSpecifyingOutputin interfacePOutput
-
equals
-
hashCode
public int hashCode()
-