Class UpgradeTableSchema
java.lang.Object
org.apache.beam.sdk.io.gcp.bigquery.UpgradeTableSchema
Helper functions for SchemaUpdateOptions.
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic com.google.cloud.bigquery.storage.v1.TableSchemagetIncrementalSchema(TableRowToStorageApiProto.ErrorCollector errorCollector, com.google.cloud.bigquery.storage.v1.TableSchema oldSchema) Given a list of schema errors, generate a new schema that represents the minimal changes needed to the schema in order to validate the new record.static booleanisPayloadSchemaOutOfDate(byte @Nullable [] payloadSchemaHash, ByteString payload, org.apache.beam.sdk.util.ThrowingSupplier<byte[]> schemaHash, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) static booleanisPayloadSchemaOutOfDate(ByteString payload, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) static com.google.cloud.bigquery.storage.v1.TableSchemamergeSchemas(com.google.cloud.bigquery.storage.v1.TableSchema schema1, com.google.cloud.bigquery.storage.v1.TableSchema schema2) static booleanmissingUnknownField(AbstractMap<String, Object> unknownFields, Descriptors.Descriptor schemaDescriptor) static booleanmissingUnknownField(AbstractMap<String, Object> unknownFields, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) static booleanmissingUnknownField(List<?> unknownFields, Descriptors.Descriptor schemaDescriptor)
-
Constructor Details
-
UpgradeTableSchema
public UpgradeTableSchema()
-
-
Method Details
-
newErrorCollector
-
getIncrementalSchema
public static com.google.cloud.bigquery.storage.v1.TableSchema getIncrementalSchema(TableRowToStorageApiProto.ErrorCollector errorCollector, com.google.cloud.bigquery.storage.v1.TableSchema oldSchema) throws TableRowToStorageApiProto.SchemaDoesntMatchException Given a list of schema errors, generate a new schema that represents the minimal changes needed to the schema in order to validate the new record. We only generate the incremental schema here for performance reasons, as these schemas will be shuffled. The final transform will merge this back into the existing table schema before updating the table. -
mergeSchemas
public static com.google.cloud.bigquery.storage.v1.TableSchema mergeSchemas(com.google.cloud.bigquery.storage.v1.TableSchema schema1, com.google.cloud.bigquery.storage.v1.TableSchema schema2) -
isPayloadSchemaOutOfDate
public static boolean isPayloadSchemaOutOfDate(byte @Nullable [] payloadSchemaHash, ByteString payload, org.apache.beam.sdk.util.ThrowingSupplier<byte[]> schemaHash, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) throws Exception - Throws:
Exception
-
isPayloadSchemaOutOfDate
public static boolean isPayloadSchemaOutOfDate(ByteString payload, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) throws Exception - Throws:
Exception
-
missingUnknownField
public static boolean missingUnknownField(AbstractMap<String, Object> unknownFields, org.apache.beam.sdk.util.ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor) throws Exception- Throws:
Exception
-
missingUnknownField
public static boolean missingUnknownField(List<?> unknownFields, Descriptors.Descriptor schemaDescriptor) -
missingUnknownField
public static boolean missingUnknownField(AbstractMap<String, Object> unknownFields, Descriptors.Descriptor schemaDescriptor)
-