org.apache.beam.sdk.io.gcp.bigquery

Enum BigQueryIO.Write.Method

java.lang.Object

java.lang.Enum<BigQueryIO.Write.Method>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.Write.Method

All Implemented Interfaces:

java.io.Serializable, java.lang.Comparable<BigQueryIO.Write.Method>

Enclosing class:

BigQueryIO.Write<T>

public static enum BigQueryIO.Write.Method
extends java.lang.Enum<BigQueryIO.Write.Method>

Determines the method used to insert data in BigQuery.

Enum Constant Summary

Enum Constants

Enum Constant and Description
DEFAULT The default behavior if no method is explicitly set.
FILE_LOADS Use BigQuery load jobs to insert data.
STORAGE_API_AT_LEAST_ONCE Use the new, Storage Write API without exactly once enabled.
STORAGE_WRITE_API Use the new, exactly-once Storage Write API.
STREAMING_INSERTS Use the BigQuery streaming insert API to insert data.

Method Summary

Modifier and Type	Method and Description
static BigQueryIO.Write.Method	valueOf(java.lang.String name) Returns the enum constant of this type with the specified name.
static BigQueryIO.Write.Method[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

DEFAULT

public static final BigQueryIO.Write.Method DEFAULT

The default behavior if no method is explicitly set. If the input is bounded, then file loads will be used. If the input is unbounded, then streaming inserts will be used.

FILE_LOADS

public static final BigQueryIO.Write.Method FILE_LOADS

Use BigQuery load jobs to insert data. Records will first be written to files, and these files will be loaded into BigQuery. This is the default method when the input is bounded. This method can be chosen for unbounded inputs as well, as long as a triggering frequency is also set using BigQueryIO.Write.withTriggeringFrequency(org.joda.time.Duration). BigQuery has daily quotas on the number of load jobs allowed per day, so be careful not to set the triggering frequency too frequent. For more information, see Loading Data from Cloud Storage.

STREAMING_INSERTS

public static final BigQueryIO.Write.Method STREAMING_INSERTS

Use the BigQuery streaming insert API to insert data. This provides the lowest-latency insert path into BigQuery, and therefore is the default method when the input is unbounded. BigQuery will make a strong effort to ensure no duplicates when using this path, however there are some scenarios in which BigQuery is unable to make this guarantee (see https://cloud.google.com/bigquery/streaming-data-into-bigquery). A query can be run over the output table to periodically clean these rare duplicates. Alternatively, using the FILE_LOADS insert method does guarantee no duplicates, though the latency for the insert into BigQuery will be much higher. For more information, see Streaming Data into BigQuery.

STORAGE_WRITE_API

public static final BigQueryIO.Write.Method STORAGE_WRITE_API

Use the new, exactly-once Storage Write API.

STORAGE_API_AT_LEAST_ONCE

public static final BigQueryIO.Write.Method STORAGE_API_AT_LEAST_ONCE

Use the new, Storage Write API without exactly once enabled. This will be cheaper and provide lower latency, however comes with the caveat that the output table may contain duplicates.

Method Detail

values

public static BigQueryIO.Write.Method[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (BigQueryIO.Write.Method c : BigQueryIO.Write.Method.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static BigQueryIO.Write.Method valueOf(java.lang.String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

java.lang.IllegalArgumentException - if this enum type has no constant with the specified name

java.lang.NullPointerException - if the argument is null