org.apache.beam.sdk.io.gcp.bigquery

Interface BigQueryServices.WriteStreamService

All Superinterfaces:

java.lang.AutoCloseable

All Known Implementing Classes:

BigQueryServicesImpl.WriteStreamServiceImpl, FakeDatasetService

Enclosing interface:

BigQueryServices

public static interface BigQueryServices.WriteStreamService
extends java.lang.AutoCloseable

An interface to get, create and flush Cloud BigQuery STORAGE API write streams.

Method Summary

Modifier and Type	Method and Description
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse>	commitWriteStreams(java.lang.String tableUrn, java.lang.Iterable<java.lang.String> writeStreamNames) Commit write streams of type PENDING.
com.google.cloud.bigquery.storage.v1.WriteStream	createWriteStream(java.lang.String tableUrn, com.google.cloud.bigquery.storage.v1.WriteStream.Type type) Create a Write Stream for use with the Storage Write API.
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse>	finalizeWriteStream(java.lang.String streamName) Finalize a write stream.
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse>	flush(java.lang.String streamName, long flushOffset) Flush a given stream up to the given offset.
BigQueryServices.StreamAppendClient	getStreamAppendClient(java.lang.String streamName, com.google.protobuf.DescriptorProtos.DescriptorProto descriptor, boolean useConnectionPool, com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation) Create an append client for a given Storage API write stream.
@Nullable com.google.cloud.bigquery.storage.v1.WriteStream	getWriteStream(java.lang.String writeStream)

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

createWriteStream

com.google.cloud.bigquery.storage.v1.WriteStream createWriteStream(java.lang.String tableUrn,
                                                                   com.google.cloud.bigquery.storage.v1.WriteStream.Type type)
                                                            throws java.io.IOException,
                                                                   java.lang.InterruptedException

Create a Write Stream for use with the Storage Write API.

Throws:

java.io.IOException

java.lang.InterruptedException

getWriteStream

@Nullable com.google.cloud.bigquery.storage.v1.WriteStream getWriteStream(java.lang.String writeStream)

getStreamAppendClient

BigQueryServices.StreamAppendClient getStreamAppendClient(java.lang.String streamName,
                                                          com.google.protobuf.DescriptorProtos.DescriptorProto descriptor,
                                                          boolean useConnectionPool,
                                                          com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation)
                                                   throws java.lang.Exception

Create an append client for a given Storage API write stream. The stream must be created first.

Throws:

java.lang.Exception

flush

com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse> flush(java.lang.String streamName,
                                                                                            long flushOffset)
                                                                                     throws java.io.IOException,
                                                                                            java.lang.InterruptedException

Flush a given stream up to the given offset. The stream must have type BUFFERED.

Throws:

java.io.IOException

java.lang.InterruptedException

finalizeWriteStream

com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse> finalizeWriteStream(java.lang.String streamName)

Finalize a write stream. After finalization, no more records can be appended to the stream.

commitWriteStreams

com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse> commitWriteStreams(java.lang.String tableUrn,
                                                                                                                       java.lang.Iterable<java.lang.String> writeStreamNames)

Commit write streams of type PENDING. The streams must be finalized before committing.