Beam SDK for Java dependencies
The Beam SDKs depend on common third-party components which then import additional dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution.
Compile and runtime dependencies for your Beam SDK version are listed in BeamModulePlugin.groovy
in the Beam repository. To view them, perform the following steps:
Open
BeamModulePlugin.groovy
.https://raw.githubusercontent.com/apache/beam/v<VERSION_NUMBER>/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
Replace `<VERSION_NUMBER>` with the major.minor.patch version of the SDK. For example, https://raw.githubusercontent.com/apache/beam/v2.50.0/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy will provide the dependencies for the 2.50.0 release.
Review the list under
project.ext.library
.Note: Some dependencies in the list use version variables, such as
google_cloud_bigdataoss_version
. These variables are defined prior to theproject.ext.library
map definition.
You can also retrieve this list by creating a new project through Maven and resolving the dependencies.
Define the Beam SDK and Java versions for the new project.
export BEAM_VERSION=2.50.0 export JAVA_VERSION=11
Create the project.
mvn archetype:generate \ -DinteractiveMode=false \ -DarchetypeGroupId=org.apache.beam \ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-starter \ -DarchetypeVersion=$BEAM_VERSION \ -DtargetPlatform=$JAVA_VERSION \ -DartifactId=check-pipeline-dependencies \ -DgroupId=org.apache.beam.samples
Change to the new project directory.
cd check-pipeline-dependencies
Resolve and list the dependencies.
mvn dependency:resolve && mvn -o dependency:list