PCollection (Apache Beam 2.38.0)

java.lang.Object
- org.apache.beam.sdk.values.PValueBase
- - org.apache.beam.sdk.values.PCollection<T>

Type Parameters:

T - the type of the elements of this PCollection

All Implemented Interfaces:

PInput, POutput, PValue
```
public class PCollection<T>
extends PValueBase
implements PValue
```
A PCollection<T> is an immutable collection of values of type T. A PCollection can contain either a bounded or unbounded number of elements. Bounded and unbounded PCollections are produced as the output of PTransforms (including root PTransforms like Read and Create), and can be passed as the inputs of other PTransforms.
Some root transforms produce bounded PCollections and others produce unbounded ones. For example, GenerateSequence.from(long) with GenerateSequence.to(long) produces a fixed set of integers, so it produces a bounded PCollection. GenerateSequence.from(long) without a GenerateSequence.to(long) produces all integers as an infinite stream, so it produces an unbounded PCollection.
Each element in a PCollection has an associated timestamp. Readers assign timestamps to elements when they create PCollections, and other PTransforms propagate these timestamps from their input to their output. See the documentation on BoundedSource.BoundedReader and UnboundedSource.UnboundedReader for more information on how these readers produce timestamps and watermarks.
Additionally, a PCollection has an associated WindowFn and each element is assigned to a set of windows. By default, the windowing function is GlobalWindows and all elements are assigned into a single default window. This default can be overridden with the Window PTransform.
See the individual PTransform subclasses for specific information on how they propagate timestamps and windowing.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class PCollection.IsBounded
The enumeration of cases for whether a PCollection is bounded.

Nested Classes
Modifier and Type	Class and Description
`static class`	`PCollection.IsBounded` The enumeration of cases for whether a `PCollection` is bounded.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`<OutputT extends POutput> OutputT`	`apply(PTransform<? super PCollection<T>,OutputT> t)` of the `PTransform`.
`<OutputT extends POutput> OutputT`	`apply(java.lang.String name, PTransform<? super PCollection<T>,OutputT> t)` Applies the given `PTransform` to this input `PCollection`, using `name` to identify this specific application of the transform.
`static <T> PCollection<T>`	`createPrimitiveOutputInternal(Pipeline pipeline, WindowingStrategy<?,?> windowingStrategy, PCollection.IsBounded isBounded, @Nullable Coder<T> coder)` *For internal use only; no backwards-compatibility guarantees.*
`static <T> PCollection<T>`	`createPrimitiveOutputInternal(Pipeline pipeline, WindowingStrategy<?,?> windowingStrategy, PCollection.IsBounded isBounded, @Nullable Coder<T> coder, TupleTag<?> tag)` *For internal use only; no backwards-compatibility guarantees.*
`java.util.Map<TupleTag<?>,PValue>`	`expand()` Expands this `POutput` into a list of its component output `PValues`.
`void`	`finishSpecifying(PInput input, PTransform<?,?> transform)` After building, finalizes this `PValue` to make it ready for running.
`void`	`finishSpecifyingOutput(java.lang.String transformName, PInput input, PTransform<?,?> transform)` As part of applying the producing `PTransform`, finalizes this output to make it ready for being used as an input and for running.
`Coder<T>`	`getCoder()` Returns the `Coder` used by this `PCollection` to encode and decode the values stored in it.
`SerializableFunction<Row,T>`	`getFromRowFunction()` Returns the attached schema's fromRowFunction.
`java.lang.String`	`getName()` Returns the name of this `PCollection`.
`Schema`	`getSchema()` Returns the attached schema.
`SerializableFunction<T,Row>`	`getToRowFunction()` Returns the attached schema's toRowFunction.
`@Nullable TypeDescriptor<T>`	`getTypeDescriptor()` Returns a `TypeDescriptor<T>` with some reflective information about `T`, if possible.
`WindowingStrategy<?,?>`	`getWindowingStrategy()` Returns the `WindowingStrategy` of this `PCollection`.
`boolean`	`hasSchema()` Returns whether this `PCollection` has an attached schema.
`PCollection.IsBounded`	`isBounded()`
`PCollection<T>`	`setCoder(Coder<T> coder)` Sets the `Coder` used by this `PCollection` to encode and decode the values stored in it.
`PCollection<T>`	`setIsBoundedInternal(PCollection.IsBounded isBounded)` *For internal use only; no backwards-compatibility guarantees.*
`PCollection<T>`	`setName(java.lang.String name)` Sets the name of this `PCollection`.
`PCollection<T>`	`setRowSchema(Schema schema)` Sets a schema on this PCollection.
`PCollection<T>`	`setSchema(Schema schema, TypeDescriptor<T> typeDescriptor, SerializableFunction<T,Row> toRowFunction, SerializableFunction<Row,T> fromRowFunction)` Sets a `Schema` on this `PCollection`.
`PCollection<T>`	`setTypeDescriptor(TypeDescriptor<T> typeDescriptor)` Sets the `TypeDescriptor<T>` for this `PCollection<T>`.
`PCollection<T>`	`setWindowingStrategyInternal(WindowingStrategy<?,?> windowingStrategy)` *For internal use only; no backwards-compatibility guarantees.*

Methods inherited from class org.apache.beam.sdk.values.PValueBase
getKindString, getPipeline, toString

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.values.POutput
getPipeline

Methods inherited from interface org.apache.beam.sdk.values.PInput
getPipeline

- Method Detail
  - finishSpecifyingOutput
```
public void finishSpecifyingOutput(java.lang.String transformName,
                                   PInput input,
                                   PTransform<?,?> transform)
```
    Description copied from interface: POutput
    
    As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.
    This includes ensuring that all PCollections have Coders specified or defaulted.
    Automatically invoked whenever this POutput is output, after POutput.finishSpecifyingOutput(String, PInput, PTransform) has been called on each component PValue returned by POutput.expand().
    
    Specified by:
    
    finishSpecifyingOutput in interface POutput
    
    Overrides:
    
    finishSpecifyingOutput in class PValueBase
  - finishSpecifying
```
public void finishSpecifying(PInput input,
                             PTransform<?,?> transform)
```
    After building, finalizes this PValue to make it ready for running. Automatically invoked whenever the PValue is "used" (e.g., when apply() is called on it) and when the Pipeline is run (useful if this is a PValue with no consumers).
    
    Specified by:
    
    finishSpecifying in interface PValue
    
    Overrides:
    
    finishSpecifying in class PValueBase
    
    Parameters:
    
    input - the PInput the PTransform was applied to to produce this output
    
    transform - the PTransform that produced this PValue
  - getTypeDescriptor
```
public @Nullable TypeDescriptor<T> getTypeDescriptor()
```
    Returns a TypeDescriptor<T> with some reflective information about T, if possible. May return null if no information is available. Subclasses may override this to enable better Coder inference.
  - getName
```
public java.lang.String getName()
```
    Returns the name of this PCollection.
    By default, the name of a PCollection is based on the name of the PTransform that produces it. It can be specified explicitly by calling setName(java.lang.String).
    
    Specified by:
    
    getName in interface PValue
    
    Overrides:
    
    getName in class PValueBase
    
    Throws:
    
    java.lang.IllegalStateException - if the name hasn't been set yet
  - expand
```
public final java.util.Map<TupleTag<?>,PValue> expand()
```
    Description copied from interface: PValue
    Expands this POutput into a list of its component output PValues.
    - A PValue expands to itself.
    - A tuple or list of PValues (such as PCollectionTuple or PCollectionList) expands to its component PValue PValues.
    Not intended to be invoked directly by user code..
    Specified by:
    
    expand in interface PInput
    
    Specified by:
    
    expand in interface POutput
    
    Specified by:
    
    expand in interface PValue
  - setName
```
public PCollection<T> setName(java.lang.String name)
```
    Sets the name of this PCollection. Returns this.
    
    Overrides:
    
    setName in class PValueBase
    
    Throws:
    
    java.lang.IllegalStateException - if this PCollection has already been finalized and may no longer be set. Once apply(org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PCollection<T>, OutputT>) has been called, this will be the case.
  - getCoder
```
public Coder<T> getCoder()
```
    Returns the Coder used by this PCollection to encode and decode the values stored in it.
    
    Throws:
    
    java.lang.IllegalStateException - if the Coder hasn't been set, and couldn't be inferred.
  - setCoder
```
public PCollection<T> setCoder(Coder<T> coder)
```
    Sets the Coder used by this PCollection to encode and decode the values stored in it. Returns this.
    
    Throws:
    
    java.lang.IllegalStateException - if this PCollection has already been finalized and may no longer be set. Once apply(org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PCollection<T>, OutputT>) has been called, this will be the case.
  - setRowSchema
```
@Experimental(value=SCHEMAS)
public PCollection<T> setRowSchema(Schema schema)
```
    Sets a schema on this PCollection.
    Can only be called on a PCollection.
  - setSchema
```
@Experimental(value=SCHEMAS)
public PCollection<T> setSchema(Schema schema,
                                                             TypeDescriptor<T> typeDescriptor,
                                                             SerializableFunction<T,Row> toRowFunction,
                                                             SerializableFunction<Row,T> fromRowFunction)
```
    Sets a Schema on this PCollection.
  - hasSchema
```
@Experimental(value=SCHEMAS)
public boolean hasSchema()
```
    Returns whether this PCollection has an attached schema.
  - getSchema
```
@Experimental(value=SCHEMAS)
public Schema getSchema()
```
    Returns the attached schema.
  - getToRowFunction
```
@Experimental(value=SCHEMAS)
public SerializableFunction<T,Row> getToRowFunction()
```
    Returns the attached schema's toRowFunction.
  - getFromRowFunction
```
@Experimental(value=SCHEMAS)
public SerializableFunction<Row,T> getFromRowFunction()
```
    Returns the attached schema's fromRowFunction.
  - apply
```
public <OutputT extends POutput> OutputT apply(PTransform<? super PCollection<T>,OutputT> t)
```
    of the PTransform.
    
    Returns:
    
    the output of the applied PTransform
  - apply
```
public <OutputT extends POutput> OutputT apply(java.lang.String name,
                                               PTransform<? super PCollection<T>,OutputT> t)
```
    Applies the given PTransform to this input PCollection, using name to identify this specific application of the transform. This name is used in various places, including the monitoring UI, logging, and to stably identify this application node in the job graph.
    
    Returns:
    
    the output of the applied PTransform
  - getWindowingStrategy
```
public WindowingStrategy<?,?> getWindowingStrategy()
```
    Returns the WindowingStrategy of this PCollection.
  - isBounded
```
public PCollection.IsBounded isBounded()
```
  - setTypeDescriptor
```
public PCollection<T> setTypeDescriptor(TypeDescriptor<T> typeDescriptor)
```
    Sets the TypeDescriptor<T> for this PCollection<T>. This may allow the enclosing PCollectionTuple, PCollectionList, or PTransform<?, PCollection<T>>, etc., to provide more detailed reflective information.
  - setWindowingStrategyInternal
```
@Internal
public PCollection<T> setWindowingStrategyInternal(WindowingStrategy<?,?> windowingStrategy)
```
    For internal use only; no backwards-compatibility guarantees.
  - setIsBoundedInternal
```
@Internal
public PCollection<T> setIsBoundedInternal(PCollection.IsBounded isBounded)
```
    For internal use only; no backwards-compatibility guarantees.
  - createPrimitiveOutputInternal
```
@Internal
public static <T> PCollection<T> createPrimitiveOutputInternal(Pipeline pipeline,
                                                                         WindowingStrategy<?,?> windowingStrategy,
                                                                         PCollection.IsBounded isBounded,
                                                                         @Nullable Coder<T> coder)
```
    For internal use only; no backwards-compatibility guarantees.
  - createPrimitiveOutputInternal
```
@Internal
public static <T> PCollection<T> createPrimitiveOutputInternal(Pipeline pipeline,
                                                                         WindowingStrategy<?,?> windowingStrategy,
                                                                         PCollection.IsBounded isBounded,
                                                                         @Nullable Coder<T> coder,
                                                                         TupleTag<?> tag)
```
    For internal use only; no backwards-compatibility guarantees.

Class PCollection<T>

Nested Class Summary

Method Summary

Methods inherited from class org.apache.beam.sdk.values.PValueBase

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.beam.sdk.values.POutput

Methods inherited from interface org.apache.beam.sdk.values.PInput

Method Detail

finishSpecifyingOutput

finishSpecifying

getTypeDescriptor

getName

expand

setName

getCoder

setCoder

setRowSchema

setSchema

hasSchema

getSchema

getToRowFunction

getFromRowFunction

apply

apply

getWindowingStrategy

isBounded

setTypeDescriptor

setWindowingStrategyInternal

setIsBoundedInternal

createPrimitiveOutputInternal

createPrimitiveOutputInternal