Roadmap for connectors and cross-cutting efforts that benefit multiple SDKs.

Last updated on Aug 2026.

Extended Metadata and SDK level CDC (Change Data Capture) support

Beam is introducing Extended Metadata to make element values extensible across SDKs. This provides native support for Change Data Capture (CDC) operations (INSERT, UPDATE, DELETE) consumed by lakehouse and database connectors, SDK-level pipeline drain, and fluent OutputBuilder APIs.

Learn more from the Beam Element Extended Metadata Design Doc.

OpenTelemetry Support

Beam is adding native OpenTelemetry support across SDKs, runners, and connectors (#33176) for distributed tracing. This leverages Extended Metadata to propagate W3C Trace Context (traceparent and tracestate) across transform stages and worker boundaries, integrates trace header propagation into streaming I/Os (such as Kafka, Pub/Sub, and Spanner change streams), and enables runner-level trace and log correlation.

IO Connector Ecosystem

Beam is actively expanding the Beam I/O connector portfolio with a focus on modern data lakehouse formats, native Change Data Capture (CDC), and unified access patterns:

• First-Class Lakehouse Connectors:
  • Apache Iceberg: Full support for batch and streaming reads, streaming appends, dynamic destinations, and changelog CDC reading. Available natively in Java, via cross-language transforms in Python, in Beam YAML, and via the Managed I/O API.
  • Delta Lake: Native read capabilities (#38551) and batch changelog (CDC) reading (#39492), exposed through core transforms and the Managed I/O API.
• Managed I/O Expansion: Exposing more storage systems through the Managed I/O API, which standardizes connector configurations using Beam Schemas, provides frictionless cross-language access (Java, Python, YAML), and enables runner-level optimizations.
• Native CDC Integration: Integrating database change streams (such as Spanner change streams and Debezium-based sources) directly into lakehouse sinks using Beam’s native CDC ValueKind metadata.

See the full list of available connectors in the Beam Connectors Overview.