Package org.apache.beam.sdk.extensions.protobuf

Class ProtoByteUtils

java.lang.Object
org.apache.beam.sdk.extensions.protobuf.ProtoByteUtils
public class ProtoByteUtils extends Object
Utility class for working with Protocol Buffer (Proto) data. This class provides methods to retrieve Beam Schemas from Proto messages, convert Proto bytes to Beam Rows, and vice versa. It also includes utilities for handling Protocol Buffer schemas and related file operations.

Users can utilize the methods in this class to facilitate the integration of Proto data processing within Apache Beam pipelines, allowing for the seamless transformation of Proto messages to Beam Rows and vice versa.

  • Constructor Details

    • ProtoByteUtils

      public ProtoByteUtils()

  • Method Details

    • getBeamSchemaFromProto

      public static Schema getBeamSchemaFromProto(String fileDescriptorPath, String messageName)
      Retrieves a Beam Schema from a Protocol Buffer message.
      Parameters:
      fileDescriptorPath - The path to the File Descriptor Set file.
      messageName - The name of the Protocol Buffer message.
      Returns:
      The Beam Schema representing the Protocol Buffer message.

    • getBeamSchemaFromProtoSchema

      public static Schema getBeamSchemaFromProtoSchema(String schemaString, String messageName)
      Parses the given Protocol Buffers schema string, retrieves the Descriptor for the specified message name, and constructs a Beam Schema from it.
      Parameters:
      schemaString - The Protocol Buffers schema string.
      messageName - The name of the message type for which the Beam Schema is desired.
      Returns:
      The Beam Schema constructed from the specified Protocol Buffers schema.
      Throws:
      RuntimeException - If there is an error during parsing, descriptor retrieval, or schema construction.

    • getProtoBytesToRowFromSchemaFunction

      public static SerializableFunction<byte[],Row> getProtoBytesToRowFromSchemaFunction(String schemaString, String messageName)

    • getProtoBytesToRowFunction

      public static SerializableFunction<byte[],Row> getProtoBytesToRowFunction(String fileDescriptorPath, String messageName)

    • getRowToProtoBytesFromSchema

      public static SerializableFunction<Row,byte[]> getRowToProtoBytesFromSchema(String schemaString, String messageName)

    • getRowToProtoBytes

      public static SerializableFunction<Row,byte[]> getRowToProtoBytes(String fileDescriptorPath, String messageName)