org.apache.beam.sdk.schemas.utils

Class JsonUtils

java.lang.Object

org.apache.beam.sdk.schemas.utils.JsonUtils

public class JsonUtils
extends java.lang.Object

Utils to convert JSON records to Beam Row.

JSON-Schema (https://json-schema.org) support

This class provides utility methods to parse, validate and translate between JSON Schema-formatted schemas and Beam Schemas. The support is based on the everit-json-schema package, which is not provided by default.

Therefore, functionality in JsonUtils::beamSchemaFromJsonSchema requires that you include everit-json-schema in your project like so:

  <dependency>
 	  <groupId>com.github.erosb < /groupId>
 	  <artifactId>everit-json-schema < /artifactId>
 	  <version>${everit-json.version} < /version>
  < /dependency>
 

Note: This functionality has been tested with everit-json-schema version 1.14.2.

JSON-Schema supported features

The current Beam implementation does not support all possible features of JSON-schema. The current implementation supports:

• String, boolean and numeric values (integer and floating-point).
• Arrays with any one of the supported types as elements. This includes nested arrays.
• Objects with any of the supported types as fields. This includes nested objects.
• Fields marked as required are non-null. Other fields are nullable.

The following JSON-schema features are not supported:

• Tuple-like arrays (or arrays with multiple item types).
• Validation of row regular expressions, enum values, array lengths, etc.
• Special annotations for types (e.g. contentMediaType) are ignored.
• Composite schemas (schemas made out of a combination of other schemas)

Constructor Summary

Constructors

Constructor and Description
JsonUtils()

Method Summary

Modifier and Type	Method and Description
static Schema	beamSchemaFromJsonSchema(java.lang.String jsonSchemaStr)
static SimpleFunction<byte[],Row>	getJsonBytesToRowFunction(Schema beamSchema) Returns a SimpleFunction mapping JSON byte[] arrays to Beam Rows.
static SimpleFunction<java.lang.String,Row>	getJsonStringToRowFunction(Schema beamSchema) Returns a SimpleFunction mapping JSON Strings to Beam Rows.
static SimpleFunction<Row,byte[]>	getRowToJsonBytesFunction(Schema beamSchema) Returns a SimpleFunction mapping Beam Rows to JSON byte[] arrays.
static SimpleFunction<Row,java.lang.String>	getRowToJsonStringsFunction(Schema beamSchema) Returns a SimpleFunction mapping Beam Rows to JSON Strings.
static org.everit.json.schema.ObjectSchema	jsonSchemaFromBeamSchema(Schema beamSchema)
static java.lang.String	jsonSchemaStringFromBeamSchema(Schema beamSchema)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonUtils

public JsonUtils()

Method Detail

getJsonBytesToRowFunction

public static SimpleFunction<byte[],Row> getJsonBytesToRowFunction(Schema beamSchema)

Returns a SimpleFunction mapping JSON byte[] arrays to Beam Rows.

getJsonStringToRowFunction

public static SimpleFunction<java.lang.String,Row> getJsonStringToRowFunction(Schema beamSchema)

Returns a SimpleFunction mapping JSON Strings to Beam Rows.

getRowToJsonBytesFunction

public static SimpleFunction<Row,byte[]> getRowToJsonBytesFunction(Schema beamSchema)

Returns a SimpleFunction mapping Beam Rows to JSON byte[] arrays.

getRowToJsonStringsFunction

public static SimpleFunction<Row,java.lang.String> getRowToJsonStringsFunction(Schema beamSchema)

Returns a SimpleFunction mapping Beam Rows to JSON Strings.

jsonSchemaStringFromBeamSchema

public static java.lang.String jsonSchemaStringFromBeamSchema(Schema beamSchema)

jsonSchemaFromBeamSchema

public static org.everit.json.schema.ObjectSchema jsonSchemaFromBeamSchema(Schema beamSchema)

beamSchemaFromJsonSchema

public static Schema beamSchemaFromJsonSchema(java.lang.String jsonSchemaStr)