org.apache.beam.sdk.transforms

Class JsonToRow

java.lang.Object

org.apache.beam.sdk.transforms.JsonToRow

@Experimental(value=SCHEMAS)
public class JsonToRow
extends java.lang.Object

Experimental

Creates a PTransform to convert input JSON objects to Rows with given Schema.

Currently supported Schema field types are:

• Schema.TypeName#BYTE
• Schema.TypeName#INT16
• Schema.TypeName#INT32
• Schema.TypeName#INT64
• Schema.TypeName#FLOAT
• Schema.TypeName#DOUBLE
• Schema.TypeName#BOOLEAN
• Schema.TypeName#STRING

For specifics of JSON deserialization see RowJson.RowJsonDeserializer.

Conversion is strict, with minimal type coercion:

Booleans are only parsed from true or false literals, not from "true" or "false" strings or any other values (exception is thrown in these cases).

If a JSON number doesn't fit into the corresponding schema field type, an exception is be thrown. Strings are not auto-converted to numbers. Floating point numbers are not auto-converted to integral numbers. Precision loss also causes exceptions.

Only JSON string values can be parsed into Schema.TypeName.STRING. Numbers, booleans are not automatically converted, exceptions are thrown in these cases.

If a schema field is missing from the JSON value, an exception will be thrown.

Explicit null literals are allowed in JSON objects. No other values are parsed into null.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JsonToRow.JsonToRowWithErrFn
static class	JsonToRow.ParseResult The result of a withExceptionReporting(Schema) transform.

Constructor Summary

Constructors

Constructor and Description
JsonToRow()

Method Summary

Modifier and Type	Method and Description
static JsonToRow.JsonToRowWithErrFn	withExceptionReporting(Schema rowSchema) Enable Exception Reporting support.
static PTransform<PCollection<java.lang.String>,PCollection<Row>>	withSchema(Schema rowSchema)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonToRow

public JsonToRow()

Method Detail

withSchema

public static PTransform<PCollection<java.lang.String>,PCollection<Row>> withSchema(Schema rowSchema)

withExceptionReporting

@Experimental(value=SCHEMAS)
public static JsonToRow.JsonToRowWithErrFn withExceptionReporting(Schema rowSchema)

Enable Exception Reporting support. If this value is set errors in the parsing layer are returned as Row objects within a JsonToRow.ParseResult

You can access the results by using withExceptionReporting(Schema):

ParseResult results = jsonPersons.apply(JsonToRow.withExceptionReporting(PERSON_SCHEMA));

Then access the parsed results via, JsonToRow.ParseResult.getResults()

And access the failed to parse results via, JsonToRow.ParseResult.getFailedToParseLines()

This will produce a Row with Schema JsonToRow.JsonToRowWithErrFn.ERROR_ROW_SCHEMA

To access the reason for the failure you will need to first enable extended error reporting. JsonToRow.JsonToRowWithErrFn.withExtendedErrorInfo()

This will provide access to the reason for the Parse failure. The call to JsonToRow.ParseResult.getFailedToParseLines() will produce a Row with Schema JsonToRow.JsonToRowWithErrFn.ERROR_ROW_WITH_ERR_MSG_SCHEMA

Returns:

JsonToRow.JsonToRowWithErrFn