Beam SDK for Python dependencies
The Beam SDKs depend on common third-party components which then import additional dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution.
Dependencies for your Beam SDK version are listed in
setup.py in the Beam repository. To view them, perform the following steps:
<VERSION_NUMBER>with the major.minor.patch version of the SDK. For example, https://raw.githubusercontent.com/apache/beam/v2.24.0/sdks/python/setup.py will provide the dependencies for the 2.24.0 release.
Review the core dependency list under
Note: If you require extra features such as
test, you should review the lists under
INTERACTIVE_BEAMfor additional dependencies.
You can also retrieve the dependency list from the command line using the following process:
Create a clean virtual environment on your local machine.
$ python3 -m venv env && source env/bin/activate
$ pip install virtualenv && virtualenv env && source env/bin/activate
Retrieve the list of dependencies.
$ pip install pipdeptree && pipdeptree -p apache-beam