blog & release
2025/05/12
Apache Beam 2.65.0Yi Hu
We are happy to present the new 2.65.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.
For more information on changes in 2.65.0, check out the detailed release notes.
Highlights
I/Os
- Upgraded GoogleAdsAPI to v19 for GoogleAdsIO (Java) (#34497). Changed PTransform method from version-specified (
v17()
) tocurrent()
for better backward compatibility in the future. - Added support for writing to Pubsub with ordering keys (Java) (#21162)
New Features / Improvements
- Added support for streaming side-inputs in the Spark Classic runner (#18136).
Breaking Changes
- [Python] Cloudpickle is set as the default
pickle_library
, where previously dill was the default in #34695. For known issues, reporting new issues, and understanding cloudpickle behavior refer to #34903. - [Python] Reshuffle now preserves PaneInfo, where previously PaneInfo was lost after reshuffle. To opt out of this change, set the update_compatibility_version to a previous Beam version e.g. “2.64.0”. (#34348).
- [Python] PaneInfo is encoded by PaneInfoCoder, where previously PaneInfo was encoded with FastPrimitivesCoder falling back to PickleCoder. This only affects cases where PaneInfo is directly stored as an element. (#34824).
- [Python] BigQueryFileLoads now adds a Reshuffle before triggering load jobs. This fixes a bug where there can be data loss in a streaming pipeline if there is a pending load job during autoscaling. To opt out of this change, set the update_compatibility_version to a previous Beam version e.g. “2.64.0”. (#34657)
- [YAML] Kafka source and sink will be automatically replaced with compatible managed transforms.
For older Beam versions, streaming update compatiblity can be maintained by specifying the pipeline
option
update_compatibility_version
(#34767).
Deprecations
- Beam ZetaSQL is deprecated and will be removed no earlier than Beam 2.68.0 (#34423). Users are recommended to switch to Calcite SQL dialect.
Bugfixes
- Fixed read Beam rows from cross-lang transform (for example, ReadFromJdbc) involving negative 32-bit integers incorrectly decoded to large integers (#34089)
- (Java) Fixed SDF-based KafkaIO (ReadFromKafkaViaSDF) to properly handle custom deserializers that extend Deserializer
interface(#34505)
- [Python]
TypedDict
typehints are now compatible withMapping
andDict
type annotations.
Security Fixes
- Fixed CVE-2025-30065 (Java) (#34573)
Known Issues
N/A
List of Contributors
According to git shortlog, the following people contributed to the 2.65.0 release. Thank you to all contributors!
Aaron Trelstad, Adrian Stoll, Ahmed Abualsaud, akashorabek, Arun Pandian, Bentsi Leviav, Bryan Dang, Celeste Zeng, Chamikara Jayalath, claudevdm, Danny McCormick, Derrick Williams, Ozzie Fernandez, Gabija Balvociute, Gayatri Kate, illoise, Jack McCluskey, Jan Lukavský, Jinho Lee, Justin Bandoro, Kenneth Knowles, XQ Hu, Luke Tsekouras, Martin Trieu, Matthew Suozzo, Naireen Hussain, Niel Markwick, Radosław Stankiewicz, Razvan Culea, Robert Bradshaw, Robert Burke, RuiLong J., Sam Whittle, Sarthak, Shubham Jaiswal, Shunping Huang, Steven van Rossum, Suvrat Acharya, sveyrie@luminatedata.com, Talat Uyarer, TanuSharma2511, Tobias Kaymak, Tom Stepp, Valentyn Tymofieiev, twosom, Vitaly Terentyev, wollowizard, Yi Hu, Yifan Ye, Zilin Du