Package org.apache.beam.sdk.extensions.python.transforms

Class RunInference<OutputT>

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<PCollection<?>,PCollection<OutputT>>
org.apache.beam.sdk.extensions.python.transforms.RunInference<OutputT>
All Implemented Interfaces:
Serializable, HasDisplayData
public class RunInference<OutputT> extends PTransform<PCollection<?>,PCollection<OutputT>>
Wrapper for invoking external Python RunInference.
See Also:

  • Method Details

    • of

      public static RunInference<Row> of(String modelLoader, Schema.FieldType exampleType, Schema.FieldType inferenceType)
      Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
      Parameters:
      modelLoader - A Python callable for a model loader class object.
      exampleType - A schema field type for the example column in output rows.
      inferenceType - A schema field type for the inference column in output rows.
      Returns:
      A RunInference for the given model loader.

    • ofKVs

      public static <KeyT> RunInference<KV<KeyT,Row>> ofKVs(String modelLoader, Schema.FieldType exampleType, Schema.FieldType inferenceType, Coder<KeyT> keyCoder)
      Similar to of(String, FieldType, FieldType) but the input is a PCollection of KVs.

      Also outputs a PCollection of KVs of the same key type.

      For example, use this if you are using Python KeyedModelHandler as the model handler.

      Type Parameters:
      KeyT - input and output Key type. Inferred by the provided coder.
      Parameters:
      modelLoader - A Python callable for a model loader class object.
      exampleType - A schema field type for the example column in output rows.
      inferenceType - A schema field type for the inference column in output rows.
      keyCoder - a Coder for the input and output Key type.
      Returns:
      A RunInference for the given model loader.

    • of

      public static RunInference<Row> of(String modelLoader, Schema schema)
      Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
      Parameters:
      modelLoader - A Python callable for a model loader class object.
      schema - A schema for output rows.
      Returns:
      A RunInference for the given model loader.

    • ofKVs

      public static <KeyT> RunInference<KV<KeyT,Row>> ofKVs(String modelLoader, Schema schema, Coder<KeyT> keyCoder)
      Similar to of(String, Schema) but the input is a PCollection of KVs.
      Type Parameters:
      KeyT - input and output Key type. Inferred by the provided coder.
      Parameters:
      modelLoader - A Python callable for a model loader class object.
      schema - A schema for output rows.
      keyCoder - a Coder for the input and output Key type.
      Returns:
      A RunInference for the given model loader.

    • withKwarg

      public RunInference<OutputT> withKwarg(String key, Object arg)
      Sets keyword arguments for the model loader.
      Returns:
      A RunInference with keyword arguments.

    • withExtraPackages

      public RunInference<OutputT> withExtraPackages(List<String> extraPackages)
      Specifies any extra packages required by the RunInference model handler.

      This should only be specified when using the default expansion service, i.e. when not using withExpansionService(String) to provide an expansion service.

      The package can either be a PyPi package or the path to a locally available Python package.

      For model handlers provided by Beam Python SDK, the implementation will automatically try to infer correct packages needed, so this may be omitted.

      Parameters:
      extraPackages - a list of PyPi packages. May include the version.
      Returns:
      A RunInference with extra packages.

    • withExpansionService

      public RunInference<OutputT> withExpansionService(String expansionService)
      Sets an expansion service endpoint for RunInference.
      Parameters:
      expansionService - A URL for a Python expansion service.
      Returns:
      A RunInference for the given expansion service endpoint.

    • expand

      public PCollection<OutputT> expand(PCollection<?> input)
      Description copied from class: PTransform
      Override this method to specify how this PTransform should be expanded on the given InputT.

      NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      Specified by:
      expand in class PTransform<PCollection<?>,PCollection<OutputT>>