public class StorageApiLoads<DestinationT,ElementT> extends PTransform<PCollection<KV<DestinationT,ElementT>>,WriteResult>
PTransform
manages loads into BigQuery using the Storage API.annotations, displayData, name, resourceHints
Constructor and Description |
---|
StorageApiLoads(Coder<DestinationT> destinationCoder,
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations<ElementT,DestinationT> dynamicDestinations,
SerializableFunction<ElementT,RowMutationInformation> rowUpdateFn,
BigQueryIO.Write.CreateDisposition createDisposition,
java.lang.String kmsKey,
Duration triggeringFrequency,
BigQueryServices bqServices,
int numShards,
boolean allowInconsistentWrites,
boolean allowAutosharding,
boolean autoUpdateSchema,
boolean ignoreUnknownValues,
boolean propagateSuccessfulStorageApiWrites,
boolean usesCdc,
com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation defaultMissingValueInterpretation,
BadRecordRouter badRecordRouter,
ErrorHandler<BadRecord,?> badRecordErrorHandler) |
Modifier and Type | Method and Description |
---|---|
WriteResult |
expand(PCollection<KV<DestinationT,ElementT>> input)
Override this method to specify how this
PTransform should be expanded on the given
InputT . |
WriteResult |
expandInconsistent(PCollection<KV<DestinationT,ElementT>> input,
Coder<KV<DestinationT,StorageApiWritePayload>> successCoder) |
WriteResult |
expandTriggered(PCollection<KV<DestinationT,ElementT>> input,
Coder<KV<DestinationT,StorageApiWritePayload>> successCoder,
Coder<StorageApiWritePayload> payloadCoder) |
WriteResult |
expandUntriggered(PCollection<KV<DestinationT,ElementT>> input,
Coder<KV<DestinationT,StorageApiWritePayload>> successCoder) |
TupleTag<BigQueryStorageApiInsertError> |
getFailedRowsTag() |
boolean |
usesErrorHandler() |
addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate
public StorageApiLoads(Coder<DestinationT> destinationCoder, org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations<ElementT,DestinationT> dynamicDestinations, @Nullable SerializableFunction<ElementT,RowMutationInformation> rowUpdateFn, BigQueryIO.Write.CreateDisposition createDisposition, java.lang.String kmsKey, Duration triggeringFrequency, BigQueryServices bqServices, int numShards, boolean allowInconsistentWrites, boolean allowAutosharding, boolean autoUpdateSchema, boolean ignoreUnknownValues, boolean propagateSuccessfulStorageApiWrites, boolean usesCdc, com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation defaultMissingValueInterpretation, BadRecordRouter badRecordRouter, ErrorHandler<BadRecord,?> badRecordErrorHandler)
public TupleTag<BigQueryStorageApiInsertError> getFailedRowsTag()
public boolean usesErrorHandler()
public WriteResult expand(PCollection<KV<DestinationT,ElementT>> input)
PTransform
PTransform
should be expanded on the given
InputT
.
NOTE: This method should not be called directly. Instead apply the PTransform
should
be applied to the InputT
using the apply
method.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
expand
in class PTransform<PCollection<KV<DestinationT,ElementT>>,WriteResult>
public WriteResult expandInconsistent(PCollection<KV<DestinationT,ElementT>> input, Coder<KV<DestinationT,StorageApiWritePayload>> successCoder)
public WriteResult expandTriggered(PCollection<KV<DestinationT,ElementT>> input, Coder<KV<DestinationT,StorageApiWritePayload>> successCoder, Coder<StorageApiWritePayload> payloadCoder)
public WriteResult expandUntriggered(PCollection<KV<DestinationT,ElementT>> input, Coder<KV<DestinationT,StorageApiWritePayload>> successCoder)