Package org.apache.beam.sdk.io.gcp.bigquery

Class TableRowToStorageApiProto

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto

public class TableRowToStorageApiProto
extends Object

Utility methods for converting JSON TableRow objects to dynamic protocol message, for use with the Storage write API.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TableRowToStorageApiProto.SchemaDoesntMatchException
static class	TableRowToStorageApiProto.SchemaTooNarrowException
static class	TableRowToStorageApiProto.SingleValueConversionException

Constructor Summary

Constructors

Constructor	Description
TableRowToStorageApiProto()

Method Summary

Modifier and Type	Method	Description
static Descriptors.Descriptor	getDescriptorFromTableSchema(TableSchema jsonSchema, boolean respectRequired, boolean includeCdcColumns)
static Descriptors.Descriptor	getDescriptorFromTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema tableSchema, boolean respectRequired, boolean includeCdcColumns)	Given a BigQuery TableSchema, returns a protocol-buffer Descriptor that can be used to write data using the BigQuery Storage API.
static Object	jsonValueFromMessageValue(Descriptors.FieldDescriptor fieldDescriptor, Object fieldValue, boolean expandRepeated, Predicate<String> includeField, String prefix)
static DynamicMessage	messageFromMap(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, Descriptors.Descriptor descriptor, AbstractMap<String,Object> map, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable String changeType, @Nullable String changeSequenceNum)
static DynamicMessage	messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, Descriptors.Descriptor descriptor, TableRow tableRow, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable String changeType, long changeSequenceNum)	Forwards to messageFromTableRow(SchemaInformation, Descriptor, TableRow, boolean, boolean, TableRow, String, String) via Long.toHexString(long).
static DynamicMessage	messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation, Descriptors.Descriptor descriptor, TableRow tableRow, boolean ignoreUnknownValues, boolean allowMissingRequiredFields, @Nullable TableRow unknownFields, @Nullable String changeType, @Nullable String changeSequenceNum)	Given a BigQuery TableRow, returns a protocol-buffer message that can be used to write data using the BigQuery Storage API.
static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode	modeToProtoMode(@Nullable String defaultValueExpression, String mode)
static String	protoModeToJsonMode(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode protoMode)
static TableSchema	protoSchemaToTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema protoTableSchema)
static TableFieldSchema	protoTableFieldToTableField(com.google.cloud.bigquery.storage.v1.TableFieldSchema protoTableField)
static String	protoTypeToJsonType(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type protoType)
static com.google.cloud.bigquery.storage.v1.TableSchema	schemaToProtoTableSchema(TableSchema tableSchema)
static com.google.cloud.bigquery.storage.v1.TableFieldSchema	tableFieldToProtoTableField(TableFieldSchema field)
static TableRow	tableRowFromMessage(Message message, boolean includeCdcColumns, Predicate<String> includeField)
static TableRow	tableRowFromMessage(Message message, boolean includeCdcColumns, Predicate<String> includeField, String namePrefix)
static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type	typeToProtoType(String type)
static Descriptors.Descriptor	wrapDescriptorProto(DescriptorProtos.DescriptorProto descriptorProto)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TableRowToStorageApiProto

public TableRowToStorageApiProto()

Method Details

modeToProtoMode

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode modeToProtoMode(@Nullable String defaultValueExpression,
 String mode)

protoModeToJsonMode

public static String protoModeToJsonMode(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Mode protoMode)

protoTypeToJsonType

public static String protoTypeToJsonType(com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type protoType)

typeToProtoType

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema.Type typeToProtoType(String type)

protoSchemaToTableSchema

public static TableSchema protoSchemaToTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema protoTableSchema)

protoTableFieldToTableField

public static TableFieldSchema protoTableFieldToTableField(com.google.cloud.bigquery.storage.v1.TableFieldSchema protoTableField)

schemaToProtoTableSchema

public static com.google.cloud.bigquery.storage.v1.TableSchema schemaToProtoTableSchema(TableSchema tableSchema)

tableFieldToProtoTableField

public static com.google.cloud.bigquery.storage.v1.TableFieldSchema tableFieldToProtoTableField(TableFieldSchema field)

getDescriptorFromTableSchema

public static Descriptors.Descriptor getDescriptorFromTableSchema(TableSchema jsonSchema,
 boolean respectRequired,
 boolean includeCdcColumns)
                                                           throws Descriptors.DescriptorValidationException

Throws:

Descriptors.DescriptorValidationException

getDescriptorFromTableSchema

public static Descriptors.Descriptor getDescriptorFromTableSchema(com.google.cloud.bigquery.storage.v1.TableSchema tableSchema,
 boolean respectRequired,
 boolean includeCdcColumns)
                                                           throws Descriptors.DescriptorValidationException

Given a BigQuery TableSchema, returns a protocol-buffer Descriptor that can be used to write data using the BigQuery Storage API.

Throws:

Descriptors.DescriptorValidationException

wrapDescriptorProto

public static Descriptors.Descriptor wrapDescriptorProto(DescriptorProtos.DescriptorProto descriptorProto)
                                                  throws Descriptors.DescriptorValidationException

Throws:

Descriptors.DescriptorValidationException

messageFromMap

public static DynamicMessage messageFromMap(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
 Descriptors.Descriptor descriptor,
 AbstractMap<String,Object> map,
 boolean ignoreUnknownValues,
 boolean allowMissingRequiredFields,
 @Nullable TableRow unknownFields,
 @Nullable String changeType,
 @Nullable String changeSequenceNum)
                                     throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

messageFromTableRow

public static DynamicMessage messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
 Descriptors.Descriptor descriptor,
 TableRow tableRow,
 boolean ignoreUnknownValues,
 boolean allowMissingRequiredFields,
 @Nullable TableRow unknownFields,
 @Nullable String changeType,
 long changeSequenceNum)
                                          throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Forwards to messageFromTableRow(SchemaInformation, Descriptor, TableRow, boolean, boolean, TableRow, String, String) via Long.toHexString(long).

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

messageFromTableRow

public static DynamicMessage messageFromTableRow(org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaInformation schemaInformation,
 Descriptors.Descriptor descriptor,
 TableRow tableRow,
 boolean ignoreUnknownValues,
 boolean allowMissingRequiredFields,
 @Nullable TableRow unknownFields,
 @Nullable String changeType,
 @Nullable String changeSequenceNum)
                                          throws org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

Given a BigQuery TableRow, returns a protocol-buffer message that can be used to write data using the BigQuery Storage API.

Throws:

org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto.SchemaConversionException

tableRowFromMessage

public static TableRow tableRowFromMessage(Message message,
 boolean includeCdcColumns,
 Predicate<String> includeField)

tableRowFromMessage

public static TableRow tableRowFromMessage(Message message,
 boolean includeCdcColumns,
 Predicate<String> includeField,
 String namePrefix)

jsonValueFromMessageValue

public static Object jsonValueFromMessageValue(Descriptors.FieldDescriptor fieldDescriptor,
 Object fieldValue,
 boolean expandRepeated,
 Predicate<String> includeField,
 String prefix)