blog & release
We are happy to present the new 2.51.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.
For more information on changes in 2.51.0, check out the detailed release notes.
New Features / Improvements
- In Python, RunInference now supports loading many models in the same transform using a KeyedModelHandler (#27628).
- In Python, the VertexAIModelHandlerJSON now supports passing in inference_args. These will be passed through to the Vertex endpoint as parameters.
- Added support to run
mypyon user pipelines (#27906)
- Removed fastjson library dependency for Beam SQL. Table property is changed to be based on jackson ObjectNode (Java) (#24154).
- Removed TensorFlow from Beam Python container images PR. If you have been negatively affected by this change, please comment on #20605.
- Removed the parameter
parquetio.Write. The element type is derived from the input PCollection (Go) (#28490)
- Refactor BeamSqlSeekableTable.setUp adding a parameter joinSubsetType. #28283
- Fixed exception chaining issue in GCS connector (Python) (#26769).
- Fixed streaming inserts exception handling, GoogleAPICallErrors are now retried according to retry strategy and routed to failed rows where appropriate rather than causing a pipeline error (Python) (#21080).
- Fixed a bug in Python SDK’s cross-language Bigtable sink that mishandled records that don’t have an explicit timestamp set: #28632.
- Python containers updated, fixing CVE-2021-30474, CVE-2021-30475, CVE-2021-30473, CVE-2020-36133, CVE-2020-36131, CVE-2020-36130, and CVE-2020-36135
- Used go 1.21.1 to build, fixing CVE-2023-39320
- Python pipelines using BigQuery Storage Read API must pin
fastavrodependency to 1.8.3 or earlier: #28811
List of Contributors
According to git shortlog, the following people contributed to the 2.50.0 release. Thank you to all contributors!
Hai Joey Tran