org.apache.beam.sdk.extensions.protobuf

Class ProtoByteUtils

java.lang.Object

org.apache.beam.sdk.extensions.protobuf.ProtoByteUtils

public class ProtoByteUtils
extends java.lang.Object

Utility class for working with Protocol Buffer (Proto) data. This class provides methods to retrieve Beam Schemas from Proto messages, convert Proto bytes to Beam Rows, and vice versa. It also includes utilities for handling Protocol Buffer schemas and related file operations.

Users can utilize the methods in this class to facilitate the integration of Proto data processing within Apache Beam pipelines, allowing for the seamless transformation of Proto messages to Beam Rows and vice versa.

Constructor Summary

Constructors

Constructor and Description
ProtoByteUtils()

Method Summary

Modifier and Type	Method and Description
static Schema	getBeamSchemaFromProto(java.lang.String fileDescriptorPath, java.lang.String messageName) Retrieves a Beam Schema from a Protocol Buffer message.
static Schema	getBeamSchemaFromProtoSchema(java.lang.String schemaString, java.lang.String messageName) Parses the given Protocol Buffers schema string, retrieves the Descriptor for the specified message name, and constructs a Beam Schema from it.
static SerializableFunction<byte[],Row>	getProtoBytesToRowFromSchemaFunction(java.lang.String schemaString, java.lang.String messageName)
static SerializableFunction<byte[],Row>	getProtoBytesToRowFunction(java.lang.String fileDescriptorPath, java.lang.String messageName)
static SerializableFunction<Row,byte[]>	getRowToProtoBytes(java.lang.String fileDescriptorPath, java.lang.String messageName)
static SerializableFunction<Row,byte[]>	getRowToProtoBytesFromSchema(java.lang.String schemaString, java.lang.String messageName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ProtoByteUtils

public ProtoByteUtils()

Method Detail

getBeamSchemaFromProto

public static Schema getBeamSchemaFromProto(java.lang.String fileDescriptorPath,
                                            java.lang.String messageName)

Retrieves a Beam Schema from a Protocol Buffer message.

Parameters:

fileDescriptorPath - The path to the File Descriptor Set file.

messageName - The name of the Protocol Buffer message.

Returns:

The Beam Schema representing the Protocol Buffer message.

getBeamSchemaFromProtoSchema

public static Schema getBeamSchemaFromProtoSchema(java.lang.String schemaString,
                                                  java.lang.String messageName)

Parses the given Protocol Buffers schema string, retrieves the Descriptor for the specified message name, and constructs a Beam Schema from it.

Parameters:

schemaString - The Protocol Buffers schema string.

messageName - The name of the message type for which the Beam Schema is desired.

Returns:

The Beam Schema constructed from the specified Protocol Buffers schema.

Throws:

java.lang.RuntimeException - If there is an error during parsing, descriptor retrieval, or schema construction.

getProtoBytesToRowFromSchemaFunction

public static SerializableFunction<byte[],Row> getProtoBytesToRowFromSchemaFunction(java.lang.String schemaString,
                                                                                    java.lang.String messageName)

getProtoBytesToRowFunction

public static SerializableFunction<byte[],Row> getProtoBytesToRowFunction(java.lang.String fileDescriptorPath,
                                                                          java.lang.String messageName)

getRowToProtoBytesFromSchema

public static SerializableFunction<Row,byte[]> getRowToProtoBytesFromSchema(java.lang.String schemaString,
                                                                            java.lang.String messageName)

getRowToProtoBytes

public static SerializableFunction<Row,byte[]> getRowToProtoBytes(java.lang.String fileDescriptorPath,
                                                                  java.lang.String messageName)