Package org.apache.beam.sdk.io.gcp.testing

Class BigqueryClient

java.lang.Object

org.apache.beam.sdk.io.gcp.testing.BigqueryClient

@Internal
public class BigqueryClient
extends Object

A wrapper class to call Bigquery API calls.

Example:

Get a new Bigquery client:

[
    BigqueryClient client = BigqueryClient.getNewBigqueryClient(applicationName);
 ]

Execute a query with retries:

[
    QueryResponse response = client.queryWithRetries(queryString, projectId);
 ]

Create a new dataset in one project:

[
    client.createNewDataset(projectId, datasetId);
 ]

Delete a dataset in one project, included its all tables:

[
    client.deleteDataset(projectId, datasetId);
 ]

Create a new table

[
    client.createNewTable(projectId, datasetId, newTable)
 ]

Insert data into table

[
    client.insertDataToTable(projectId, datasetId, tableName, rows)
 ]

Constructor Summary

Constructors

Constructor	Description
BigqueryClient(String applicationName)

Method Summary

Modifier and Type	Method	Description
void	createNewDataset(String projectId, String datasetId)	Creates a new dataset.
void	createNewDataset(String projectId, String datasetId, @Nullable Long defaultTableExpirationMs)	Creates a new dataset with defaultTableExpirationMs.
void	createNewDataset(String projectId, String datasetId, @Nullable Long defaultTableExpirationMs, @Nullable String location)	Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).
void	createNewTable(String projectId, String datasetId, Table newTable)
void	deleteDataset(String projectId, String datasetId)
void	deleteTable(String projectId, String datasetId, String tableName)
static BigqueryClient	getClient(String applicationName)
static Bigquery	getNewBigqueryClient(String applicationName)
Table	getTableResource(String projectId, String datasetId, String tableId)
void	insertDataToTable(String projectId, String datasetId, String tableName, List<Map<String,Object>> rows)	Inserts rows to a table using a BigQuery streaming write.
List<TableRow>	queryUnflattened(String query, String projectId, boolean typed, boolean useStandardSql)	Performs a query without flattening results.
List<TableRow>	queryUnflattened(String query, String projectId, boolean typed, boolean useStandardSql, @Nullable String location)	Performs a query without flattening results.
QueryResponse	queryWithRetries(String query, String projectId)
QueryResponse	queryWithRetries(String query, String projectId, boolean typed)
QueryResponse	queryWithRetriesUsingStandardSql(String query, String projectId)
void	updateTableSchema(String projectId, String datasetId, String tableId, TableSchema newSchema)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BigqueryClient

public BigqueryClient(String applicationName)

Method Details

getNewBigqueryClient

public static Bigquery getNewBigqueryClient(String applicationName)

getClient

public static BigqueryClient getClient(String applicationName)

queryWithRetries

@Nonnull
public QueryResponse queryWithRetries(String query,
 String projectId)
                               throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

queryWithRetriesUsingStandardSql

@Nonnull
public QueryResponse queryWithRetriesUsingStandardSql(String query,
 String projectId)
                                               throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

queryUnflattened

@Nonnull
public List<TableRow> queryUnflattened(String query,
 String projectId,
 boolean typed,
 boolean useStandardSql)
                                throws IOException,
InterruptedException

Performs a query without flattening results.

Throws:

IOException

InterruptedException

queryUnflattened

@Nonnull
public List<TableRow> queryUnflattened(String query,
 String projectId,
 boolean typed,
 boolean useStandardSql,
 @Nullable String location)
                                throws IOException,
InterruptedException

Performs a query without flattening results. May choose a location (GCP region) to perform this operation in.

Throws:

IOException

InterruptedException

queryWithRetries

@Nonnull
public QueryResponse queryWithRetries(String query,
 String projectId,
 boolean typed)
                               throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

createNewDataset

public void createNewDataset(String projectId,
 String datasetId)
                      throws IOException,
InterruptedException

Creates a new dataset.

Throws:

IOException

InterruptedException

createNewDataset

public void createNewDataset(String projectId,
 String datasetId,
 @Nullable Long defaultTableExpirationMs)
                      throws IOException,
InterruptedException

Creates a new dataset with defaultTableExpirationMs.

Throws:

IOException

InterruptedException

createNewDataset

public void createNewDataset(String projectId,
 String datasetId,
 @Nullable Long defaultTableExpirationMs,
 @Nullable String location)
                      throws IOException,
InterruptedException

Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).

Throws:

IOException

InterruptedException

deleteTable

public void deleteTable(String projectId,
 String datasetId,
 String tableName)

deleteDataset

public void deleteDataset(String projectId,
 String datasetId)

createNewTable

public void createNewTable(String projectId,
 String datasetId,
 Table newTable)
                    throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

insertDataToTable

public void insertDataToTable(String projectId,
 String datasetId,
 String tableName,
 List<Map<String,Object>> rows)
                       throws IOException,
InterruptedException

Inserts rows to a table using a BigQuery streaming write.

Throws:

IOException

InterruptedException

getTableResource

public Table getTableResource(String projectId,
 String datasetId,
 String tableId)
                       throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

updateTableSchema

public void updateTableSchema(String projectId,
 String datasetId,
 String tableId,
 TableSchema newSchema)
                       throws IOException

Throws:

IOException