org.apache.beam.sdk.io.gcp.testing

Class BigqueryClient

java.lang.Object

org.apache.beam.sdk.io.gcp.testing.BigqueryClient

@Internal
public class BigqueryClient
extends java.lang.Object

A wrapper class to call Bigquery API calls.

Example:

Get a new Bigquery client:

 [
    BigqueryClient client = BigqueryClient.getNewBigqueryClient(applicationName);
 ]

Execute a query with retries:

 [
    QueryResponse response = client.queryWithRetries(queryString, projectId);
 ]

Create a new dataset in one project:

 [
    client.createNewDataset(projectId, datasetId);
 ]

Delete a dataset in one project, included its all tables:

 [
    client.deleteDataset(projectId, datasetId);
 ]

Create a new table

 [
    client.createNewTable(projectId, datasetId, newTable)
 ]

Insert data into table

 [
    client.insertDataToTable(projectId, datasetId, tableName, rows)
 ]

Constructor Summary

Constructors

Constructor and Description
BigqueryClient(java.lang.String applicationName)

Method Summary

Modifier and Type	Method and Description
void	createNewDataset(java.lang.String projectId, java.lang.String datasetId) Creates a new dataset.
void	createNewDataset(java.lang.String projectId, java.lang.String datasetId, @Nullable java.lang.Long defaultTableExpirationMs) Creates a new dataset with defaultTableExpirationMs.
void	createNewDataset(java.lang.String projectId, java.lang.String datasetId, @Nullable java.lang.Long defaultTableExpirationMs, @Nullable java.lang.String location) Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).
void	createNewTable(java.lang.String projectId, java.lang.String datasetId, Table newTable)
void	deleteDataset(java.lang.String projectId, java.lang.String datasetId)
void	deleteTable(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableName)
static BigqueryClient	getClient(java.lang.String applicationName)
static Bigquery	getNewBigqueryClient(java.lang.String applicationName)
Table	getTableResource(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableId)
void	insertDataToTable(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableName, java.util.List<java.util.Map<java.lang.String,java.lang.Object>> rows) Inserts rows to a table using a BigQuery streaming write.
java.util.List<TableRow>	queryUnflattened(java.lang.String query, java.lang.String projectId, boolean typed, boolean useStandardSql) Performs a query without flattening results.
java.util.List<TableRow>	queryUnflattened(java.lang.String query, java.lang.String projectId, boolean typed, boolean useStandardSql, @Nullable java.lang.String location) Performs a query without flattening results.
QueryResponse	queryWithRetries(java.lang.String query, java.lang.String projectId)
QueryResponse	queryWithRetries(java.lang.String query, java.lang.String projectId, boolean typed)
QueryResponse	queryWithRetriesUsingStandardSql(java.lang.String query, java.lang.String projectId)
void	updateTableSchema(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableId, TableSchema newSchema)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigqueryClient

public BigqueryClient(java.lang.String applicationName)

Method Detail

getNewBigqueryClient

public static Bigquery getNewBigqueryClient(java.lang.String applicationName)

getClient

public static BigqueryClient getClient(java.lang.String applicationName)

queryWithRetries

@Nonnull
public QueryResponse queryWithRetries(java.lang.String query,
                                               java.lang.String projectId)
                                        throws java.io.IOException,
                                               java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

queryWithRetriesUsingStandardSql

@Nonnull
public QueryResponse queryWithRetriesUsingStandardSql(java.lang.String query,
                                                               java.lang.String projectId)
                                                        throws java.io.IOException,
                                                               java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

queryUnflattened

@Nonnull
public java.util.List<TableRow> queryUnflattened(java.lang.String query,
                                                          java.lang.String projectId,
                                                          boolean typed,
                                                          boolean useStandardSql)
                                                   throws java.io.IOException,
                                                          java.lang.InterruptedException

Performs a query without flattening results.

Throws:

java.io.IOException

java.lang.InterruptedException

queryUnflattened

@Nonnull
public java.util.List<TableRow> queryUnflattened(java.lang.String query,
                                                          java.lang.String projectId,
                                                          boolean typed,
                                                          boolean useStandardSql,
                                                          @Nullable java.lang.String location)
                                                   throws java.io.IOException,
                                                          java.lang.InterruptedException

Performs a query without flattening results. May choose a location (GCP region) to perform this operation in.

Throws:

java.io.IOException

java.lang.InterruptedException

queryWithRetries

@Nonnull
public QueryResponse queryWithRetries(java.lang.String query,
                                               java.lang.String projectId,
                                               boolean typed)
                                        throws java.io.IOException,
                                               java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

createNewDataset

public void createNewDataset(java.lang.String projectId,
                             java.lang.String datasetId)
                      throws java.io.IOException,
                             java.lang.InterruptedException

Creates a new dataset.

Throws:

java.io.IOException

java.lang.InterruptedException

createNewDataset

public void createNewDataset(java.lang.String projectId,
                             java.lang.String datasetId,
                             @Nullable java.lang.Long defaultTableExpirationMs)
                      throws java.io.IOException,
                             java.lang.InterruptedException

Creates a new dataset with defaultTableExpirationMs.

Throws:

java.io.IOException

java.lang.InterruptedException

createNewDataset

public void createNewDataset(java.lang.String projectId,
                             java.lang.String datasetId,
                             @Nullable java.lang.Long defaultTableExpirationMs,
                             @Nullable java.lang.String location)
                      throws java.io.IOException,
                             java.lang.InterruptedException

Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).

Throws:

java.io.IOException

java.lang.InterruptedException

deleteTable

public void deleteTable(java.lang.String projectId,
                        java.lang.String datasetId,
                        java.lang.String tableName)

deleteDataset

public void deleteDataset(java.lang.String projectId,
                          java.lang.String datasetId)

createNewTable

public void createNewTable(java.lang.String projectId,
                           java.lang.String datasetId,
                           Table newTable)
                    throws java.io.IOException,
                           java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

insertDataToTable

public void insertDataToTable(java.lang.String projectId,
                              java.lang.String datasetId,
                              java.lang.String tableName,
                              java.util.List<java.util.Map<java.lang.String,java.lang.Object>> rows)
                       throws java.io.IOException,
                              java.lang.InterruptedException

Inserts rows to a table using a BigQuery streaming write.

Throws:

java.io.IOException

java.lang.InterruptedException

getTableResource

public Table getTableResource(java.lang.String projectId,
                              java.lang.String datasetId,
                              java.lang.String tableId)
                       throws java.io.IOException,
                              java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

updateTableSchema

public void updateTableSchema(java.lang.String projectId,
                              java.lang.String datasetId,
                              java.lang.String tableId,
                              TableSchema newSchema)
                       throws java.io.IOException

Throws:

java.io.IOException