Beam SDK for Python dependencies

The Beam SDKs depend on common third-party components which then import additional dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution.

To see the compile and runtime dependencies for your Beam SDK version, expand the relevant section below.

2.15.0

Beam SDK for Python 2.15.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
cachetools>=3.1.0,<4
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.2.0,<4.0.0; python_version < "3.0"
google-apitools>=0.5.28,<0.5.29
google-cloud-bigquery>=1.6.0,<1.18.0
google-cloud-bigtable>=0.31.1,<0.33.0
google-cloud-core>=0.28.1,<2
google-cloud-datastore>=1.7.1,<1.8.0
google-cloud-pubsub>=0.39.0,<0.40.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.12.0
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4; python_version < "3.0"
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.15.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<2
pymongo>=3.8.0,<4.0.0
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0; python_version < "3.0"
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.14.0

Beam SDK for Python 2.14.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
cachetools>=3.1.0,<4
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.2.0,<4.0.0; python_version < "3.0"
google-apitools>=0.5.28,<0.5.29
google-cloud-bigquery>=1.6.0,<1.7.0
google-cloud-bigtable>=0.31.1,<0.33.0
google-cloud-core>=0.28.1,<0.30.0
google-cloud-datastore>=1.7.1,<1.8.0
google-cloud-pubsub>=0.39.0,<0.40.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.12.0
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4; python_version < "3.0"
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.15.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<1.3
pymongo>=3.8.0,<4.0.0
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0; python_version < "3.0"
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.13.0

Beam SDK for Python 2.13.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
cachetools>=3.1.0,<4
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.2.0,<4.0.0; python_version < "3.0"
google-apitools>=0.5.28,<0.5.29
google-cloud-bigquery>=1.6.0,<1.7.0
google-cloud-bigtable>=0.31.1,<0.33.0
google-cloud-core>=0.28.1,<0.30.0
google-cloud-datastore>=1.7.1,<1.8.0
google-cloud-pubsub>=0.39.0,<0.40.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.12.0
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4; python_version < "3.0"
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.14.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<1.3
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0; python_version < "3.0"
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.12.0

Beam SDK for Python 2.12.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.2.0,<4.0.0; python_version < "3.0"
google-apitools>=0.5.26,<0.5.27
google-cloud-bigquery>=1.6.0,<1.7.0
google-cloud-bigtable==0.31.1
google-cloud-core==0.28.1
google-cloud-pubsub==0.39.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.11.3
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.12.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<1.3
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0; python_version < "3.0"
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.11.0

Beam SDK for Python 2.11.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.2.0,<4.0.0; python_version < "3.0"
google-apitools>=0.5.26,<0.5.27
google-cloud-bigquery>=1.6.0,<1.7.0
google-cloud-bigtable==0.31.1
google-cloud-core==0.28.1
google-cloud-pubsub==0.39.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.11.3
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.12.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<1.3
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0; python_version < "3.0"
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.10.0

Beam SDK for Python 2.10.0 has the following compile and runtime dependencies.

PackageVersion
avro-python3>=1.8.1,<2.0.0; python_version >= "3.0"
avro>=1.8.1,<2.0.0; python_version < "3.0"
crcmod>=1.7,<2.0
dill>=0.2.9,<0.2.10
fastavro>=0.21.4,<0.22
future>=0.16.0,<1.0.0
futures>=3.1.1,<4.0.0
google-apitools>=0.5.23,<=0.5.24
google-cloud-bigquery>=1.6.0,<1.7.0
google-cloud-pubsub==0.39.0
googledatastore>=7.0.1,<7.1; python_version < "3.0"
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.11.3
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<4
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4
protobuf>=3.5.0.post1,<4
pyarrow>=0.11.1,<0.12.0; python_version >= "3.0" or platform_system != "Windows"
pydot>=1.2.0,<1.3
pytz>=2018.3
pyvcf>=0.6.8,<0.7.0
pyyaml>=3.12,<4.0.0
typing>=3.6.0,<3.7.0; python_version < "3.5.0"
2.9.0

Beam SDK for Python 2.9.0 has the following compile and runtime dependencies.

PackageVersion
avro>=1.8.1, <2.0.0
crcmod>=1.7, <2.0
dill>=0.2.6, <=0.2.8.2
fastavro>=0.21.4, <0.22
future>=0.16.0, <1.0.0
futures>=3.1.1, <4.0.0
google-apitools>=0.5.23, <0.5.24
google-cloud-pubsub==0.35.4
google-cloud-bigquery>=1.6.0, <1.7.0
googledatastore>=7.0.1, <7.1
grpcio>=1.8, <2
hdfs>=2.1.0, <3.0.0
httplib2>=0.8, <=0.11.3
mock>=1.0.1, <3.0.0
nose>=1.3.7
numpy>=1.14.3, <2
oauth2client>=2.0.1, <4
parameterized>=0.6.0, <0.7.0
proto-google-cloud-datastore-v1>=0.90.0, <0.90.4
protobuf>=3.5.0.post1, <4
pydot>=1.2.0, <1.3
pyhamcrest>=1.9, <2.0
pytz>=2018.3, <=2018.4
pyyaml>=3.12, <4.0.0
pyvcf>=0.6.8, <0.7.0
typing>=3.6.0, <3.7.0
2.8.0

Beam SDK for Python 2.8.0 has the following compile and runtime dependencies.

PackageVersion
avro>=1.8.1, <2.0.0
crcmod>=1.7, <2.0
dill>=0.2.6, <=0.2.8.2
fastavro>=0.21.4, <0.22
future>=0.16.0, <1.0.0
futures>=3.1.1, <4.0.0
google-apitools>=0.5.18, <0.5.20
google-cloud-pubsub==0.26.0
google-cloud-bigquery==0.25.0
googledatastore==7.0.1
grpcio>=1.8, <2
hdfs>=2.1.0, <3.0.0
httplib2>=0.8, <=0.11.3
mock>=1.0.1, <3.0.0
nose>=1.3.7
numpy>=1.14.3, <2
oauth2client>=2.0.1, <5
parameterized>=0.6.0, <0.7.0
proto-google-cloud-datastore-v1>=0.90.0, <0.90.4
proto-google-cloud-pubsub-v1==0.15.4
protobuf>=3.5.0.post1, <4
pydot>=1.2.0, <1.3
pyhamcrest>=1.9, <2.0
pytz>=2018.3, <=2018.4
pyyaml>=3.12, <4.0.0
pyvcf>=0.6.8, <0.7.0
typing>=3.6.0, <3.7.0
2.7.0

Beam SDK for Python 2.7.0 has the following compile and runtime dependencies.

PackageVersion
avro>=1.8.1, <2.0.0
crcmod>=1.7, <2.0
dill>=0.2.6, <=0.2.8.2
fastavro==0.19.7
future>=0.16.0, <1.0.0
futures>=3.1.1, <4.0.0
google-apitools>=0.5.18, <0.5.20
google-cloud-pubsub==0.26.0
google-cloud-bigquery==0.25.0
googledatastore==7.0.1
grpcio>=1.8, <2
hdfs>=2.1.0, <3.0.0
httplib2>=0.8, <=0.11.3
mock>=1.0.1, <3.0.0
nose>=1.3.7
numpy>=1.14.3, <2
oauth2client>=2.0.1, <5
proto-google-cloud-datastore-v1>=0.90.0, <0.90.4
proto-google-cloud-pubsub-v1==0.15.4
protobuf>=3.5.0.post1, <4
pydot>=1.2.0, <1.3
pyhamcrest>=1.9, <2.0
pytz>=2018.3, <=2018.4
pyyaml>=3.12, <4.0.0
pyvcf>=0.6.8, <0.7.0
six>=1.9, <1.12
typing>=3.6.0, <3.7.0
2.6.0

Beam SDK for Python 2.6.0 has the following compile and runtime dependencies.

PackageVersion
avro>=1.8.1,<2.0.0
crcmod>=1.7,<2.0
dill>=0.2.6,<=0.2.8.2
future>=0.16.0,<1.0.0
futures>=3.1.1,<4.0.0
google-apitools>=0.5.18,<=0.5.20
google-cloud-bigquery==0.25.0
google-cloud-pubsub==0.26.0
googledatastore==7.0.1
grpcio>=1.8,<2
hdfs>=2.1.0,<3.0.0
httplib2>=0.8,<=0.11.3
mock>=1.0.1,<3.0.0
oauth2client>=2.0.1,<5
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4
proto-google-cloud-pubsub-v1==0.15.4
protobuf>=3.5.0.post1,<4
pydot>=1.2.0,<1.3
pytz>=2018.3,<=2018.4
pyvcf>=0.6.8,<0.7.0
pyyaml>=3.12,<4.0.0
six>=1.9,<1.12
typing>=3.6.0,<3.7.0