Apache Beam 2.32.0

We are happy to present the new 2.32.0 release of Apache Beam. This release includes both improvements and new functionality.

See the download page for this release.

For more information on changes in 2.32.0, check out the

detailed release notes.

Highlights

I/Os

  • Added ability to use JdbcIO.Write.withResults without statement and preparedStatementSetter. (BEAM-12511)
  • Added ability to register URI schemes to use the S3 protocol via FileIO. (BEAM-12435).
  • Respect number of shards set in SnowflakeWrite batch mode. (BEAM-12715)
  • Java SDK: Update Google Cloud Healthcare IO connectors from using v1beta1 to using the GA version.

New Features / Improvements

  • Add support to convert Beam Schema to Avro Schema for JDBC LogicalTypes: VARCHAR, NVARCHAR, LONGVARCHAR, LONGNVARCHAR, DATE, TIME (Java)(BEAM-12385).
  • Reading from JDBC source by partitions (Java) (BEAM-12456).
  • PubsubIO can now write to a dead-letter topic after a parsing error (Java)(BEAM-12474).
  • New append-only option for Elasticsearch sink (Java) BEAM-12601

Breaking Changes

Deprecations

  • Python GBK will stop supporting unbounded PCollections that have global windowing and a default trigger in Beam 2.33. This can be overriden with --allow_unsafe_triggers. (BEAM-9487).
  • Python GBK will start requiring safe triggers or the --allow_unsafe_triggers flag starting with Beam 2.33. (BEAM-9487).

Bugfixes

  • Fixed race condition in RabbitMqIO causing duplicate acks (Java) (BEAM-6516))

List of Contributors

According to git shortlog, the following people contributed to the 2.32.0 release. Thank you to all contributors!

Ahmet Altay, Ajo Thomas, Alex Amato, Alexey Romanenko, Alex Koay, allenpradeep, Anant Damle, Andrew Pilloud, Ankur Goenka, Ashwin Ramaswami, Benjamin Gonzalez, BenWhitehead, Blake Williams, Boyuan Zhang, Brian Hulette, Chamikara Jayalath, Daniel Oliveira, Daniel Thevessen, daria-malkova, David Cavazos, David Huntsperger, dennisylyung, Dennis Yung, dmkozh, egalpin, emily, Esun Kim, Gabriel Melo de Paula, Harch Vardhan, Heejong Lee, heidimhurst, hoshimura, Iñigo San Jose Visiers, Ismaël Mejía, Jack McCluskey, Jan Lukavský, Justin King, Kenneth Knowles, KevinGG, Ke Wu, kileys, Kyle Weaver, Luke Cwik, Maksym Skorupskyi, masahitojp, Matthew Ouyang, Matthias Baetens, Matt Rudary, MiguelAnzoWizeline, Miguel Hernandez, Nikita Petunin, Ning Ding, Ning Kang, odidev, Pablo Estrada, Pascal Gillet, rafal.ochyra, raphael.sanamyan, Reuven Lax, Robert Bradshaw, Robert Burke, roger-mike, Ryan McDowell, Sam Rohde, Sam Whittle, Siyuan Chen, Teng Qiu, Tianzi Cai, Tobias Hermann, Tomo Suzuki, tvalentyn, Tyson Hamilton, Udi Meiri, Valentyn Tymofieiev, Vitaly Terentyev, Yichi Zhang, Yifan Mai, yoshiki.obata, Yu Feng, YuqiHuai, yzhang559, Zachary Houfek, zhoufek