Package org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryServicesImpl.WriteStreamServiceImpl

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl.WriteStreamServiceImpl

All Implemented Interfaces:

AutoCloseable, BigQueryServices.WriteStreamService

Enclosing class:

BigQueryServicesImpl

public static class BigQueryServicesImpl.WriteStreamServiceImpl
extends Object
implements BigQueryServices.WriteStreamService

Constructor Summary

Constructors

Constructor	Description
WriteStreamServiceImpl(BigQueryOptions bqOptions)

Method Summary

Modifier and Type	Method	Description
void	close()
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse>	commitWriteStreams(String tableUrn, Iterable<String> writeStreamNames)	Commit write streams of type PENDING.
com.google.cloud.bigquery.storage.v1.WriteStream	createWriteStream(String tableUrn, com.google.cloud.bigquery.storage.v1.WriteStream.Type type)	Create a Write Stream for use with the Storage Write API.
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse>	finalizeWriteStream(String streamName)	Finalize a write stream.
com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse>	flush(String streamName, long flushOffset)	Flush a given stream up to the given offset.
BigQueryServices.StreamAppendClient	getStreamAppendClient(String streamName, DescriptorProtos.DescriptorProto descriptor, boolean useConnectionPool, com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation)	Create an append client for a given Storage API write stream.
@Nullable com.google.cloud.bigquery.storage.v1.TableSchema	getWriteStreamSchema(String writeStream)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

WriteStreamServiceImpl

public WriteStreamServiceImpl(BigQueryOptions bqOptions)

Method Details

createWriteStream

public com.google.cloud.bigquery.storage.v1.WriteStream createWriteStream(String tableUrn,
 com.google.cloud.bigquery.storage.v1.WriteStream.Type type)
                                                                   throws IOException

Description copied from interface: BigQueryServices.WriteStreamService

Create a Write Stream for use with the Storage Write API.

Specified by:

createWriteStream in interface BigQueryServices.WriteStreamService

Throws:

IOException

getWriteStreamSchema

public @Nullable com.google.cloud.bigquery.storage.v1.TableSchema getWriteStreamSchema(String writeStream)

Specified by:

getWriteStreamSchema in interface BigQueryServices.WriteStreamService

getStreamAppendClient

public BigQueryServices.StreamAppendClient getStreamAppendClient(String streamName,
 DescriptorProtos.DescriptorProto descriptor,
 boolean useConnectionPool,
 com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation)
                                                          throws Exception

Description copied from interface: BigQueryServices.WriteStreamService

Create an append client for a given Storage API write stream. The stream must be created first.

Specified by:

getStreamAppendClient in interface BigQueryServices.WriteStreamService

Throws:

Exception

flush

public com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse> flush(String streamName,
 long flushOffset)
                                                                                            throws IOException,
InterruptedException

Description copied from interface: BigQueryServices.WriteStreamService

Flush a given stream up to the given offset. The stream must have type BUFFERED.

Specified by:

flush in interface BigQueryServices.WriteStreamService

Throws:

IOException

InterruptedException

finalizeWriteStream

public com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse> finalizeWriteStream(String streamName)

Description copied from interface: BigQueryServices.WriteStreamService

Finalize a write stream. After finalization, no more records can be appended to the stream.

Specified by:

finalizeWriteStream in interface BigQueryServices.WriteStreamService

commitWriteStreams

public com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse> commitWriteStreams(String tableUrn,
 Iterable<String> writeStreamNames)

Description copied from interface: BigQueryServices.WriteStreamService

Commit write streams of type PENDING. The streams must be finalized before committing.

Specified by:

commitWriteStreams in interface BigQueryServices.WriteStreamService

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception