public class PythonExternalTransform<InputT extends PInput,OutputT extends POutput> extends PTransform<InputT,OutputT>
name, resourceHints| Modifier and Type | Method and Description |
|---|---|
OutputT |
expand(InputT input)
Override this method to specify how this
PTransform should be expanded on the given
InputT. |
static <InputT extends PInput,OutputT extends POutput> |
from(java.lang.String tranformName)
Instantiates a cross-language wrapper for a Python transform with a given transform name.
|
static <InputT extends PInput,OutputT extends POutput> |
from(java.lang.String tranformName,
java.lang.String expansionService)
Instantiates a cross-language wrapper for a Python transform with a given transform name.
|
PythonExternalTransform<InputT,OutputT> |
withArgs(java.lang.Object... args)
Positional arguments for the Python cross-language transform.
|
PythonExternalTransform<InputT,OutputT> |
withKwarg(java.lang.String name,
java.lang.Object value)
Specifies a single keyword argument for the Python cross-language transform.
|
PythonExternalTransform<InputT,OutputT> |
withKwargs(java.util.Map<java.lang.String,java.lang.Object> kwargs)
Specifies keyword arguments for the Python cross-language transform.
|
PythonExternalTransform<InputT,OutputT> |
withKwargs(Row kwargs)
Specifies keyword arguments as a Row objects.
|
PythonExternalTransform<InputT,OutputT> |
withOutputCoder(Coder<?> outputCoder)
Specifies the
Coder of the output PCollections produced by this transform. |
PythonExternalTransform<InputT,OutputT> |
withOutputCoders(java.util.Map<java.lang.String,Coder<?>> outputCoders)
Specifies the keys and
Coders of the output PCollections produced by this
transform. |
PythonExternalTransform<InputT,OutputT> |
withTypeHint(java.lang.Class<?> argType,
Schema.FieldType fieldType)
Specifies the field type of arguments.
|
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validatepublic static <InputT extends PInput,OutputT extends POutput> PythonExternalTransform<InputT,OutputT> from(java.lang.String tranformName)
The given fully qualified name will be imported and called to instantiate the transform.
Often this is the fully qualified name of a Python PTransform class, in which case the
arguments will be passed to its constructor, but any callable will do.
Two special names, __callable__ and __constructor__ can be used to define a
suitable transform inline if none exists.
When __callable__ is provided, the first argument (or source keyword
argument) should be a PythonCallableSource which represents the expand method of the
PTransform accepting and returning a PValue (and may also take additional
arguments and keyword arguments). For example, one might write
PythonExternalTransform
.from("__callable__")
.withArgs(
PythonCallable.of("def expand(pcoll, x, y): return pcoll | ..."),
valueForX,
valueForY);
When __constructor__ is provided, the first argument (or source keyword
argument) should be a PythonCallableSource which will return the desired PTransform
when called with the remaining arguments and keyword arguments. Often this will be a PythonCallableSource representing a PTransform class, for example
PythonExternalTransform
.from("__constructor__")
.withArgs(
PythonCallable.of("class MyPTransform(beam.PTransform): ..."),
...valuesForMyPTransformConstructorIfAny);
InputT - Input PCollection typeOutputT - Output PCollection typetranformName - fully qualified transform name.PythonExternalTransform for the given transform name.public static <InputT extends PInput,OutputT extends POutput> PythonExternalTransform<InputT,OutputT> from(java.lang.String tranformName, java.lang.String expansionService)
See from(String) for the meaning of transformName.
InputT - Input PCollection typeOutputT - Output PCollection typetranformName - fully qualified transform name.expansionService - address and port number for externally launched expansion servicePythonExternalTransform for the given transform name.public PythonExternalTransform<InputT,OutputT> withArgs(java.lang.Object... args)
args - list of arguments.public PythonExternalTransform<InputT,OutputT> withKwarg(java.lang.String name, java.lang.Object value)
name - argument name.value - argument valuepublic PythonExternalTransform<InputT,OutputT> withKwargs(java.util.Map<java.lang.String,java.lang.Object> kwargs)
public PythonExternalTransform<InputT,OutputT> withKwargs(Row kwargs)
kwargs - keyword arguments as a Row objects. An empty Row represents zero keyword
arguments.public PythonExternalTransform<InputT,OutputT> withTypeHint(java.lang.Class<?> argType, Schema.FieldType fieldType)
Type hints are especially useful for logical types since type inference does not work well for logical types.
argType - A class object for the argument type.fieldType - A schema field type for the argument.public PythonExternalTransform<InputT,OutputT> withOutputCoders(java.util.Map<java.lang.String,Coder<?>> outputCoders)
Coders of the output PCollections produced by this
transform.outputCoders - a mapping from output keys to Coders.public PythonExternalTransform<InputT,OutputT> withOutputCoder(Coder<?> outputCoder)
Coder of the output PCollections produced by this transform.
Should only be used if this transform produces a single output.outputCoder - output Coder of the transform.public OutputT expand(InputT input)
PTransformPTransform should be expanded on the given
InputT.
NOTE: This method should not be called directly. Instead apply the PTransform should
be applied to the InputT using the apply method.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).