apache_beam.testing.datatype_inference module

apache_beam.testing.datatype_inference.infer_element_type(elements)[source]

For internal use only; no backwards-compatibility guarantees.

Infer a Beam type for a list of elements.

Parameters:elements (List[Any]) – A list of elements for which the type should be inferred.
Returns:A Beam type encompassing all elements.
apache_beam.testing.datatype_inference.infer_typehints_schema(data)[source]

For internal use only; no backwards-compatibility guarantees.

Infer Beam types for tabular data.

Parameters:data (List[dict]) – A list of dictionaries representing rows in a table.
Returns:An OrderedDict mapping column names to Beam types.
apache_beam.testing.datatype_inference.infer_avro_schema(data, use_fastavro=False)[source]

For internal use only; no backwards-compatibility guarantees.

Infer avro schema for tabular data.

Parameters:
  • data (List[dict]) – A list of dictionaries representing rows in a table.
  • use_fastavro (bool) – A flag indicating whether the schema should be constructed using fastavro.
Returns:

An avro schema object.

apache_beam.testing.datatype_inference.infer_pyarrow_schema(data)[source]

For internal use only; no backwards-compatibility guarantees.

Infer PyArrow schema for tabular data.

Parameters:data (List[dict]) – A list of dictionaries representing rows in a table.
Returns:A PyArrow schema object.