MLTransform Image Embedding GPU Performance

Model: Sentence Transformers — clip-ViT-B-32 (image) Accelerator: NVIDIA Tesla T4 GPU Host: Dataflow Prime with throughput-based autoscaling

This batch pipeline reads image URIs from GCS, decodes images with Pillow, generates image embeddings through MLTransform with SentenceTransformerEmbeddings(image_model=True), and writes results to BigQuery using batch file loads.

See the glossary for definitions.

Full pipeline implementation is available here.

What is the estimated cost to run the pipeline?

RunTime and EstimatedCost

How has various metrics changed when running the pipeline for different Beam SDK versions?

AvgThroughputBytesPerSec by Version

AvgThroughputElementsPerSec by Version

How has various metrics changed over time when running the pipeline?

AvgThroughputBytesPerSec by Date

AvgThroughputElementsPerSec by Date

See also MLTransform Image Embedding CPU for the CPU right-fitting variant of this pipeline.