org.apache.beam.sdk.extensions.python.transforms

Class RunInference<OutputT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<?>,PCollection<OutputT>>

org.apache.beam.sdk.extensions.python.transforms.RunInference<OutputT>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

public class RunInference<OutputT>
extends PTransform<PCollection<?>,PCollection<OutputT>>

Wrapper for invoking external Python RunInference. @Experimental

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Method Summary

Modifier and Type	Method and Description
PCollection<OutputT>	expand(PCollection<?> input) Override this method to specify how this PTransform should be expanded on the given InputT.
static RunInference<Row>	of(java.lang.String modelLoader, Schema.FieldType exampleType, Schema.FieldType inferenceType) Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
static RunInference<Row>	of(java.lang.String modelLoader, Schema schema) Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
static <KeyT> RunInference<KV<KeyT,Row>>	ofKVs(java.lang.String modelLoader, Schema.FieldType exampleType, Schema.FieldType inferenceType, Coder<KeyT> keyCoder) Similar to RunInference#of(String, FieldType, FieldType) but the input is a PCollection of KVs.
static <KeyT> RunInference<KV<KeyT,Row>>	ofKVs(java.lang.String modelLoader, Schema schema, Coder<KeyT> keyCoder) Similar to of(String, Schema) but the input is a PCollection of KVs.
RunInference<OutputT>	withExpansionService(java.lang.String expansionService) Sets an expansion service endpoint for RunInference.
void	withExtraPackages(java.util.List<java.lang.String> extraPackages) Specifies any extra Pypi packages required by the RunInference model handler.
RunInference<OutputT>	withKwarg(java.lang.String key, java.lang.Object arg) Sets keyword arguments for the model loader.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

of

public static RunInference<Row> of(java.lang.String modelLoader,
                                   Schema.FieldType exampleType,
                                   Schema.FieldType inferenceType)

Instantiates a multi-language wrapper for a Python RunInference with a given model loader.

Parameters:

modelLoader - A Python callable for a model loader class object.

exampleType - A schema field type for the example column in output rows.

inferenceType - A schema field type for the inference column in output rows.

Returns:

A RunInference for the given model loader.

ofKVs

public static <KeyT> RunInference<KV<KeyT,Row>> ofKVs(java.lang.String modelLoader,
                                                      Schema.FieldType exampleType,
                                                      Schema.FieldType inferenceType,
                                                      Coder<KeyT> keyCoder)

Similar to RunInference#of(String, FieldType, FieldType) but the input is a PCollection of KVs.

Also outputs a PCollection of KVs of the same key type.

For example, use this if you are using Python KeyedModelHandler as the model handler.

Type Parameters:

KeyT - input and output Key type. Inferred by the provided coder.

Parameters:

modelLoader - A Python callable for a model loader class object.

exampleType - A schema field type for the example column in output rows.

inferenceType - A schema field type for the inference column in output rows.

keyCoder - a Coder for the input and output Key type.

Returns:

A RunInference for the given model loader.

of

public static RunInference<Row> of(java.lang.String modelLoader,
                                   Schema schema)

Instantiates a multi-language wrapper for a Python RunInference with a given model loader.

Parameters:

modelLoader - A Python callable for a model loader class object.

schema - A schema for output rows.

Returns:

A RunInference for the given model loader.

ofKVs

public static <KeyT> RunInference<KV<KeyT,Row>> ofKVs(java.lang.String modelLoader,
                                                      Schema schema,
                                                      Coder<KeyT> keyCoder)

Similar to of(String, Schema) but the input is a PCollection of KVs.

Type Parameters:

KeyT - input and output Key type. Inferred by the provided coder.

Parameters:

modelLoader - A Python callable for a model loader class object.

schema - A schema for output rows.

keyCoder - a Coder for the input and output Key type.

Returns:

A RunInference for the given model loader.

withKwarg

public RunInference<OutputT> withKwarg(java.lang.String key,
                                       java.lang.Object arg)

Sets keyword arguments for the model loader.

Returns:

A RunInference with keyword arguments.

withExtraPackages

public void withExtraPackages(java.util.List<java.lang.String> extraPackages)

Specifies any extra Pypi packages required by the RunInference model handler.

This should only be specified when using the default expansion service, i.e. when not using withExpansionService(String) to provide an expansion service.

For model handlers provided by Beam Python SDK, the implementation will automatically try to infer correct packages needed, so this may be omitted.

Parameters:

extraPackages - a list of PyPi packages. May include the version.

withExpansionService

public RunInference<OutputT> withExpansionService(java.lang.String expansionService)

Sets an expansion service endpoint for RunInference.

Parameters:

expansionService - A URL for a Python expansion service.

Returns:

A RunInference for the given expansion service endpoint.

expand

public PCollection<OutputT> expand(PCollection<?> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<?>,PCollection<OutputT>>