org.apache.beam.sdk.io.gcp.bigquery

Class TableRowToStorageApiProto

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto

public class TableRowToStorageApiProto
extends java.lang.Object

Utility methods for converting JSON TableRow objects to dynamic protocol message, for use with the Storage write API.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TableRowToStorageApiProto.SchemaDoesntMatchException
static class	TableRowToStorageApiProto.SchemaTooNarrowException
static class	TableRowToStorageApiProto.SingleValueConversionException

Constructor Summary

Constructors

Constructor and Description
TableRowToStorageApiProto()

Method Summary

Modifier and Type	Method and Description
static com.google.protobuf.Descriptors.Descriptor	getDescriptorFromTableSchema(TableSchema jsonSchema, boolean respectRequired, boolean includeCdcColumns)
static com.google.protobuf.Descriptors.Descriptor	getDescriptorFromTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema tableSchema, boolean respectRequired, boolean includeCdcColumns) Given a BigQuery TableSchema, returns a protocol-buffer Descriptor that can be used to write data using the BigQuery Storage API.
static java.lang.Object	jsonValueFromMessageValue(com.google.protobuf.Descriptors.FieldDescriptor fieldDescriptor, java.lang.Object fieldValue, boolean expandRepeated, java.util.function.Predicate<java.lang.String> includeField, java.lang.String prefix)
static com.google.protobuf.DynamicMessage	messageFromMap(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, com.google.protobuf.Descriptors.Descriptor descriptor, java.util.AbstractMap<java.lang.String,java.lang.Object> map, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable java.lang.String changeType, @Nullable java.lang.String changeSequenceNum)
static com.google.protobuf.DynamicMessage	messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, com.google.protobuf.Descriptors.Descriptor descriptor, TableRow tableRow, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable java.lang.String changeType, long changeSequenceNum) Forwards to #messageFromTableRow(SchemaInformation, Descriptor, TableRow, boolean, boolean, TableRow, String, String) via Long.toHexString(long).
static com.google.protobuf.DynamicMessage	messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, com.google.protobuf.Descriptors.Descriptor descriptor, TableRow tableRow, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable java.lang.String changeType, @Nullable java.lang.String changeSequenceNum) Given a BigQuery TableRow, returns a protocol-buffer message that can be used to write data using the BigQuery Storage API.
static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode	modeToProtoMode(@Nullable java.lang.String defaultValueExpression, java.lang.String mode)
static java.lang.String	protoModeToJsonMode(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode protoMode)
static TableSchema	protoSchemaToTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema protoTableSchema)
static TableFieldSchema	protoTableFieldToTableField(com.google.cloud.bigquery.storage.v1.TableFieldSchema protoTableField)
static java.lang.String	protoTypeToJsonType(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type protoType)
static com.google.cloud.bigquery.storage.v1.TableSchema	schemaToProtoTableSchema(TableSchema tableSchema)
static com.google.cloud.bigquery.storage.v1.TableFieldSchema	tableFieldToProtoTableField(TableFieldSchema field)
static TableRow	tableRowFromMessage(com.google.protobuf.Message message, boolean includeCdcColumns, java.util.function.Predicate<java.lang.String> includeField)
static TableRow	tableRowFromMessage(com.google.protobuf.Message message, boolean includeCdcColumns, java.util.function.Predicate<java.lang.String> includeField, java.lang.String namePrefix)
static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type	typeToProtoType(java.lang.String type)
static com.google.protobuf.Descriptors.Descriptor	wrapDescriptorProto(com.google.protobuf.DescriptorProtos.DescriptorProto descriptorProto)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TableRowToStorageApiProto

public TableRowToStorageApiProto()

Method Detail

modeToProtoMode

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode modeToProtoMode(@Nullable java.lang.String defaultValueExpression,
                                                                                         java.lang.String mode)

protoModeToJsonMode

public static java.lang.String protoModeToJsonMode(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode protoMode)

protoTypeToJsonType

public static java.lang.String protoTypeToJsonType(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type protoType)

typeToProtoType

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type typeToProtoType(java.lang.String type)

protoSchemaToTableSchema

public static TableSchema protoSchemaToTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema protoTableSchema)

protoTableFieldToTableField

public static TableFieldSchema protoTableFieldToTableField(com.google.cloud.bigquery.storage.v1.TableFieldSchema protoTableField)

schemaToProtoTableSchema

public static com.google.cloud.bigquery.storage.v1.TableSchema schemaToProtoTableSchema(TableSchema tableSchema)

tableFieldToProtoTableField

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema tableFieldToProtoTableField(TableFieldSchema field)

getDescriptorFromTableSchema

public static com.google.protobuf.Descriptors.Descriptor getDescriptorFromTableSchema(TableSchema jsonSchema,
                                                                                      boolean respectRequired,
                                                                                      boolean includeCdcColumns)
                                                                               throws com.google.protobuf.Descriptors.DescriptorValidationException

Throws:

com.google.protobuf.Descriptors.DescriptorValidationException

getDescriptorFromTableSchema

public static com.google.protobuf.Descriptors.Descriptor getDescriptorFromTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema tableSchema,
                                                                                      boolean respectRequired,
                                                                                      boolean includeCdcColumns)
                                                                               throws com.google.protobuf.Descriptors.DescriptorValidationException

Given a BigQuery TableSchema, returns a protocol-buffer Descriptor that can be used to write data using the BigQuery Storage API.

Throws:

com.google.protobuf.Descriptors.DescriptorValidationException

wrapDescriptorProto

public static com.google.protobuf.Descriptors.Descriptor wrapDescriptorProto(com.google.protobuf.DescriptorProtos.DescriptorProto descriptorProto)
                                                                      throws com.google.protobuf.Descriptors.DescriptorValidationException

Throws:

com.google.protobuf.Descriptors.DescriptorValidationException

messageFromMap

public static com.google.protobuf.DynamicMessage messageFromMap(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
                                                                com.google.protobuf.Descriptors.Descriptor descriptor,
                                                                java.util.AbstractMap<java.lang.String,java.lang.Object> map,
                                                                boolean ignoreUnknownValues,
                                                                boolean allowMissingRequiredFields,
                                                                @Nullable TableRow unknownFields,
                                                                @Nullable java.lang.String changeType,
                                                                @Nullable java.lang.String changeSequenceNum)
                                                         throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

messageFromTableRow

public static com.google.protobuf.DynamicMessage messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
                                                                     com.google.protobuf.Descriptors.Descriptor descriptor,
                                                                     TableRow tableRow,
                                                                     boolean ignoreUnknownValues,
                                                                     boolean allowMissingRequiredFields,
                                                                     @Nullable TableRow unknownFields,
                                                                     @Nullable java.lang.String changeType,
                                                                     long changeSequenceNum)
                                                              throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Forwards to #messageFromTableRow(SchemaInformation, Descriptor, TableRow, boolean, boolean, TableRow, String, String) via Long.toHexString(long).

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

messageFromTableRow

public static com.google.protobuf.DynamicMessage messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
                                                                     com.google.protobuf.Descriptors.Descriptor descriptor,
                                                                     TableRow tableRow,
                                                                     boolean ignoreUnknownValues,
                                                                     boolean allowMissingRequiredFields,
                                                                     @Nullable TableRow unknownFields,
                                                                     @Nullable java.lang.String changeType,
                                                                     @Nullable java.lang.String changeSequenceNum)
                                                              throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Given a BigQuery TableRow, returns a protocol-buffer message that can be used to write data using the BigQuery Storage API.

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

tableRowFromMessage

public static TableRow tableRowFromMessage(com.google.protobuf.Message message,
                                           boolean includeCdcColumns,
                                           java.util.function.Predicate<java.lang.String> includeField)

tableRowFromMessage

public static TableRow tableRowFromMessage(com.google.protobuf.Message message,
                                           boolean includeCdcColumns,
                                           java.util.function.Predicate<java.lang.String> includeField,
                                           java.lang.String namePrefix)

jsonValueFromMessageValue

public static java.lang.Object jsonValueFromMessageValue(com.google.protobuf.Descriptors.FieldDescriptor fieldDescriptor,
                                                         java.lang.Object fieldValue,
                                                         boolean expandRepeated,
                                                         java.util.function.Predicate<java.lang.String> includeField,
                                                         java.lang.String prefix)