Package org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryHelpers

java.lang.Object
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers
public class BigQueryHelpers extends Object
A set of helper functions and classes used by BigQueryIO.

  • Constructor Details

    • BigQueryHelpers

      public BigQueryHelpers()

  • Method Details

    • toTableSpec

      public static String toTableSpec(com.google.api.services.bigquery.model.TableReference ref)
      Returns a canonical string representation of the TableReference.

    • dataCatalogSegments

      public static List<String> dataCatalogSegments(com.google.api.services.bigquery.model.TableReference ref, BigQueryOptions options)

    • parseTableSpec

      public static com.google.api.services.bigquery.model.TableReference parseTableSpec(String tableSpec)
      Parse a table specification in the form "[project_id]:[dataset_id].[table_id]" or "[project_id].[dataset_id].[table_id]" or "[dataset_id].[table_id]".

      Lakehouse runtime catalog (BigLake metastore) tables are referenced with four parts, "[project_id].[catalog_id].[namespace_id].[table_id]" (or "[project_id]:[catalog_id].[namespace_id].[table_id]"); these parse to a composite "[catalog_id].[namespace_id]" dataset id, which is the form the BigQuery APIs accept for such tables. More generally, when a specification contains more than three segments, everything between the project id and the final (table) segment becomes the dataset id.

      If the project id is omitted, the default project id is used.

    • parseTableUrn

      public static com.google.api.services.bigquery.model.TableReference parseTableUrn(String tableUrn)

    • stripPartitionDecorator

      public static String stripPartitionDecorator(String tableSpec)
      Strip off any partition decorator information from a tablespec.

    • toJsonString

      public static @PolyNull String toJsonString(@PolyNull Object item)

    • fromJsonString

      public static <T> @PolyNull T fromJsonString(@PolyNull String json, Class<T> clazz)

    • getNumRows

      public static @Nullable BigInteger getNumRows(BigQueryOptions options, com.google.api.services.bigquery.model.TableReference tableRef) throws InterruptedException, IOException
      It returns the number of rows for a given table.
      Returns:
      The number of rows in the table or null if it cannot get any estimate.
      Throws:
      InterruptedException
      IOException

    • getTable

      public static @Nullable com.google.api.services.bigquery.model.Table getTable(BigQueryOptions options, com.google.api.services.bigquery.model.TableReference tableRef) throws InterruptedException, IOException
      Throws:
      InterruptedException
      IOException