Package org.apache.beam.sdk.io.gcp.bigquery

Interface BigQueryServices.WriteStreamService

All Superinterfaces:
AutoCloseable
All Known Implementing Classes:
BigQueryServicesImpl.WriteStreamServiceImpl, FakeDatasetService
Enclosing interface:
BigQueryServices
public static interface BigQueryServices.WriteStreamService extends AutoCloseable
An interface to get, create and flush Cloud BigQuery STORAGE API write streams.

  • Method Summary

    Modifier and Type
    Method
    Description
    com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse>
    commitWriteStreams(String tableUrn, Iterable<String> writeStreamNames)
    Commit write streams of type PENDING.
    com.google.cloud.bigquery.storage.v1.WriteStream
    createWriteStream(String tableUrn, com.google.cloud.bigquery.storage.v1.WriteStream.Type type)
    Create a Write Stream for use with the Storage Write API.
    com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse>
    finalizeWriteStream(String streamName)
    Finalize a write stream.
    com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse>
    flush(String streamName, long flushOffset)
    Flush a given stream up to the given offset.
    BigQueryServices.StreamAppendClient
    getStreamAppendClient(String streamName, DescriptorProtos.DescriptorProto descriptor, boolean useConnectionPool, com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation)
    Create an append client for a given Storage API write stream.
    @Nullable com.google.cloud.bigquery.storage.v1.TableSchema
    getWriteStreamSchema(String writeStream)
     

    Methods inherited from interface java.lang.AutoCloseable

    close

  • Method Details

    • createWriteStream

      com.google.cloud.bigquery.storage.v1.WriteStream createWriteStream(String tableUrn, com.google.cloud.bigquery.storage.v1.WriteStream.Type type) throws IOException, InterruptedException
      Create a Write Stream for use with the Storage Write API.
      Throws:
      IOException
      InterruptedException

    • getWriteStreamSchema

      @Nullable com.google.cloud.bigquery.storage.v1.TableSchema getWriteStreamSchema(String writeStream)

    • getStreamAppendClient

      BigQueryServices.StreamAppendClient getStreamAppendClient(String streamName, DescriptorProtos.DescriptorProto descriptor, boolean useConnectionPool, com.google.cloud.bigquery.storage.v1.AppendRowsRequest.MissingValueInterpretation missingValueInterpretation) throws Exception
      Create an append client for a given Storage API write stream. The stream must be created first.
      Throws:
      Exception

    • flush

      com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FlushRowsResponse> flush(String streamName, long flushOffset) throws IOException, InterruptedException
      Flush a given stream up to the given offset. The stream must have type BUFFERED.
      Throws:
      IOException
      InterruptedException

    • finalizeWriteStream

      com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse> finalizeWriteStream(String streamName)
      Finalize a write stream. After finalization, no more records can be appended to the stream.

    • commitWriteStreams

      com.google.api.core.ApiFuture<com.google.cloud.bigquery.storage.v1.BatchCommitWriteStreamsResponse> commitWriteStreams(String tableUrn, Iterable<String> writeStreamNames)
      Commit write streams of type PENDING. The streams must be finalized before committing.