apache_beam.ml.transforms.embeddings.tensorflow_hub module¶

class apache_beam.ml.transforms.embeddings.tensorflow_hub.TensorflowHubTextEmbeddings(columns: List[str], hub_url: str, preprocessing_url: Optional[str] = None, **kwargs)[source]¶

Bases: apache_beam.ml.transforms.base.EmbeddingsManager

Embedding config for tensorflow hub models. This config can be used with MLTransform to embed text data. Models are loaded using the RunInference PTransform with the help of a ModelHandler.

Parameters:

columns – The columns containing the text to be embedded.
hub_url – The url of the tensorflow hub model.
preprocessing_url – The url of the preprocessing model. This is optional. If provided, the preprocessing model will be used to preprocess the text before feeding it to the main model.
min_batch_size – The minimum batch size to be used for inference.
max_batch_size – The maximum batch size to be used for inference.
large_model – Whether to share the model across processes.

get_model_handler() → apache_beam.ml.inference.base.ModelHandler[source]¶

get_ptransform_for_processing(**kwargs) → apache_beam.transforms.ptransform.PTransform[source]¶: Returns a RunInference object that is used to run inference on the text input using _TextEmbeddingHandler.

class apache_beam.ml.transforms.embeddings.tensorflow_hub.TensorflowHubImageEmbeddings(columns: List[str], hub_url: str, **kwargs)[source]¶

Bases: apache_beam.ml.transforms.base.EmbeddingsManager

Embedding config for tensorflow hub models. This config can be used with MLTransform to embed image data. Models are loaded using the RunInference PTransform with the help of a ModelHandler.

Parameters:	columns – The columns containing the images to be embedded. hub_url – The url of the tensorflow hub model. min_batch_size – The minimum batch size to be used for inference. max_batch_size – The maximum batch size to be used for inference. large_model – Whether to share the model across processes.

get_model_handler() → apache_beam.ml.inference.base.ModelHandler[source]¶

get_ptransform_for_processing(**kwargs) → apache_beam.transforms.ptransform.PTransform[source]¶: Returns a RunInference object that is used to run inference on the text input using _ImageEmbeddingHandler.