Package org.apache.beam.sdk.io.gcp.bigquery

Class WriteResult

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.WriteResult

All Implemented Interfaces:

POutput

public final class WriteResult
extends Object
implements POutput

The result of a BigQueryIO.Write transform.

Method Summary

Modifier and Type	Method	Description
Map<TupleTag<?>,PValue>	expand()	Expands this POutput into a list of its component output PValues.
void	finishSpecifyingOutput(String transformName, PInput input, PTransform<?,?> transform)	As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.
PCollection<TableRow>	getFailedInserts()	Returns a PCollection containing the TableRows that didn't make it to BQ.
PCollection<BigQueryInsertError>	getFailedInsertsWithErr()	Returns a PCollection containing the BigQueryInsertErrors with detailed error information.
PCollection<BigQueryStorageApiInsertError>	getFailedStorageApiInserts()	Return any rows that persistently fail to insert when using a storage-api method.
Pipeline	getPipeline()	Returns the owning Pipeline of this POutput.
PCollection<TableRow>	getSuccessfulInserts()	Returns a PCollection containing the TableRows that were written to BQ via the streaming insert API.
PCollection<TableRow>	getSuccessfulStorageApiInserts()	Return all rows successfully inserted using one of the storage-api insert methods.
PCollection<TableDestination>	getSuccessfulTableLoads()	Returns a PCollection containing the TableDestinations that were successfully loaded using the batch load API.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

expand

public Map<TupleTag<?>,PValue> expand()

Description copied from interface: POutput

Expands this POutput into a list of its component output PValues.

• A PValue expands to itself.
• A tuple or list of PValues (such as PCollectionTuple or PCollectionList) expands to its component PValue PValues.

Not intended to be invoked directly by user code.

Specified by:

expand in interface POutput

getSuccessfulTableLoads

public PCollection<TableDestination> getSuccessfulTableLoads()

Returns a PCollection containing the TableDestinations that were successfully loaded using the batch load API.

getSuccessfulInserts

public PCollection<TableRow> getSuccessfulInserts()

Returns a PCollection containing the TableRows that were written to BQ via the streaming insert API.

getFailedInserts

public PCollection<TableRow> getFailedInserts()

Returns a PCollection containing the TableRows that didn't make it to BQ.

Only use this method if you haven't enabled BigQueryIO.Write.withExtendedErrorInfo(). Otherwise use getFailedInsertsWithErr()

getFailedInsertsWithErr

public PCollection<BigQueryInsertError> getFailedInsertsWithErr()

Returns a PCollection containing the BigQueryInsertErrors with detailed error information.

Only use this method if you have enabled BigQueryIO.Write.withExtendedErrorInfo(). Otherwise use getFailedInserts()

getFailedStorageApiInserts

public PCollection<BigQueryStorageApiInsertError> getFailedStorageApiInserts()

Return any rows that persistently fail to insert when using a storage-api method. For example: rows with values that do not match the BigQuery schema or rows that are too large to insert. This collection is in the global window.

getSuccessfulStorageApiInserts

public PCollection<TableRow> getSuccessfulStorageApiInserts()

Return all rows successfully inserted using one of the storage-api insert methods. Rows undergo a conversion process, so while these TableRow objects are logically the same as the rows in the initial PCollection, they may not be physically identical. This PCollection is in the global window.

getPipeline

public Pipeline getPipeline()

Description copied from interface: POutput

Returns the owning Pipeline of this POutput.

Specified by:

getPipeline in interface POutput

finishSpecifyingOutput

public void finishSpecifyingOutput(String transformName,
 PInput input,
 PTransform<?,?> transform)

Description copied from interface: POutput

As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.

This includes ensuring that all PCollections have Coders specified or defaulted.

Automatically invoked whenever this POutput is output, after POutput.finishSpecifyingOutput(String, PInput, PTransform) has been called on each component PValue returned by POutput.expand().

Specified by:

finishSpecifyingOutput in interface POutput