Today, the Apache Software Foundation announced that Apache Beam has successfully graduated from incubation, becoming a new Top-Level Project at the foundation and signifying that its “community and products have been well-governed under the foundation’s meritocratic process and principles”.

Graduation is an exciting milestone for Apache Beam. Becoming a top-level project is a recognition of the amazing growth of the Apache Beam community, both in terms of size and diversity. It is fantastic to see ever-increasing participation on our development and user mailing lists, in pull requests, and at conferences and meetups. Contributions of additional runners and IO connectors further show the significant interest from other projects and organizations in being part of our growing community.

Although graduation is a statement about community, our technical progress has also been impressive. We started with code donations from several companies that were tailored with a specific vendor and scenario in mind and had fragile component boundaries. The whole community worked extremely hard over the past year to refactor the codebase and documentation into a truly vendor-neutral and extensible framework. We have established an impressive engineering system and processes that promote test-driven development. This concerted focus on quality and execution excellence enabled us to publish four releases, each significantly better than the previous.

Going forward, we will continue to extend the core abstractions to distill additional complex data processing patterns into intuitive APIs, and, at the same time, enhance the ability to interconnect additional storage/messaging systems and execution engines. Together, we are excited to push forward the state of the art in distributed data processing.

This is also an opportunity to thank those who helped and supported us reaching this milestone. None of this would be possible without the Apache Software Foundation and its amazing volunteers who have wholeheartedly welcomed us into the family. In particular, we’d like to thank the Apache Incubator community for their guidance producing releases and following processes. We are especially thankful to the Apache Infra team for their assistance establishing Beam’s engineering system, particularly in the areas of testing infrastructure and development productivity.

On behalf of the whole Apache Beam community, thank you to our incubation mentors Ted Dunning, Venkatesh Seetharam, and, previously, Bertrand Delacretaz and Jim Jagielski, for teaching us the Apache Way and guiding us through incubation. In particular, special thanks goes to our incubation champion and mentor Jean-Baptiste Onofré, whose extraordinary help, day after day, was instrumental in getting us here, and who continues as an active code contributor and a member of the project management committee.

Please consider joining us, whether as a user or a contributor, as we work towards our first release with API stability. If you’d like to try out Apache Beam today, check out the latest 0.4.0 release. We welcome contribution and participation from anyone through our mailing lists, issue tracker, pull requests, and events.