apache_beam.yaml.json_utils module

Utilities for converting between JSON and Beam Schema’d data.

For internal use, no backward compatibility guarantees.

apache_beam.yaml.json_utils.json_schema_to_beam_schema(json_schema: Dict[str, Any]) → Schema

Returns a Beam schema equivalent for the given Json schema.

apache_beam.yaml.json_utils.json_type_to_beam_type(json_type: Dict[str, Any]) → FieldType

Returns a Beam schema type for the given Json (schema) type.

apache_beam.yaml.json_utils.beam_schema_to_json_schema(beam_schema: Schema) → Dict[str, Any]

apache_beam.yaml.json_utils.beam_type_to_json_type(beam_type: FieldType) → Dict[str, Any]

apache_beam.yaml.json_utils.json_to_row(beam_type: FieldType) → Callable[[Any], Any]

Returns a callable converting Json objects to Beam rows of the given type.

The input to the returned callable is expected to conform to the Json schema corresponding to this Beam type.

apache_beam.yaml.json_utils.json_parser(beam_schema: Schema, json_schema: Dict[str, Any] | None = None) → Callable[[bytes], Row]

Returns a callable converting Json strings to Beam rows of the given type.

The input to the returned callable is expected to conform to the Json schema corresponding to this Beam type.

apache_beam.yaml.json_utils.row_to_json(beam_type: FieldType) → Callable[[Any], Any]

Returns a callable converting rows of the given type to Json objects.

apache_beam.yaml.json_utils.json_formater(beam_schema: Schema) → Callable[[Row], bytes]

Returns a callable converting rows of the given schema to Json strings.

apache_beam.yaml.json_utils.row_validator(beam_schema: Schema, json_schema: Dict[str, Any]) → Callable[[Any], Any]

Returns a callable that will fail on elements not respecting json_schema.