Package org.apache.beam.sdk.io.gcp.bigquery

Class UpgradeTableSchema

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.UpgradeTableSchema

public class UpgradeTableSchema
extends Object

Helper functions for SchemaUpdateOptions.

Constructor Summary

Constructors

Constructor	Description
UpgradeTableSchema()

Method Summary

Modifier and Type	Method	Description
static com.google.cloud.bigquery.storage.v1.TableSchema	getIncrementalSchema(TableRowToStorageApiProto.ErrorCollector errorCollector, com.google.cloud.bigquery.storage.v1.TableSchema oldSchema)	Given a list of schema errors, generate a new schema that represents the minimal changes needed to the schema in order to validate the new record.
static boolean	isPayloadSchemaOutOfDate(StorageApiWritePayload payload, org.apache.beam.sdk.util.ThrowingSupplier<byte[]> schemaHash, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor)
static com.google.cloud.bigquery.storage.v1.TableSchema	mergeSchemas(com.google.cloud.bigquery.storage.v1.TableSchema schema1, com.google.cloud.bigquery.storage.v1.TableSchema schema2)
static TableRowToStorageApiProto.ErrorCollector	newErrorCollector()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

UpgradeTableSchema

public UpgradeTableSchema()

Method Details

newErrorCollector

public static TableRowToStorageApiProto.ErrorCollector newErrorCollector()

getIncrementalSchema

public static com.google.cloud.bigquery.storage.v1.TableSchema getIncrementalSchema(TableRowToStorageApiProto.ErrorCollector errorCollector,
 com.google.cloud.bigquery.storage.v1.TableSchema oldSchema)
                                                                             throws TableRowToStorageApiProto.SchemaDoesntMatchException

Given a list of schema errors, generate a new schema that represents the minimal changes needed to the schema in order to validate the new record. We only generate the incremental schema here for performance reasons, as these schemas will be shuffled. The final transform will merge this back into the existing table schema before updating the table.

Throws:

TableRowToStorageApiProto.SchemaDoesntMatchException

mergeSchemas

public static com.google.cloud.bigquery.storage.v1.TableSchema mergeSchemas(com.google.cloud.bigquery.storage.v1.TableSchema schema1,
 com.google.cloud.bigquery.storage.v1.TableSchema schema2)

isPayloadSchemaOutOfDate

public static boolean isPayloadSchemaOutOfDate(StorageApiWritePayload payload,
 org.apache.beam.sdk.util.ThrowingSupplier<byte[]> schemaHash,
 org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor)
                                        throws Exception

Throws:

Exception