WordCount quickstart for Go
If you’re interested in contributing to the Apache Beam Go codebase, see the Contribution Guide.
Set up your environment
The Beam SDK for Go requires
go version 1.20 or newer. It can be downloaded here. Check what go version you have by running:
If you are unfamiliar with Go, see the Get Started With Go Tutorial.
The Apache Beam examples directory has many examples. All examples can be run by passing the required arguments described in the examples.
For example, to run
go run github.com/apache/beam/sdks/v2/go/examples/wordcount@latest --input gs://dataflow-samples/shakespeare/kinglear.txt \ --output gs://<your-gcs-bucket>/counts \ --runner dataflow \ --project your-gcp-project \ --region your-gcp-region \ --temp_location gs://<your-gcs-bucket>/tmp/ \ --staging_location gs://<your-gcs-bucket>/binaries/
# Build and run the Spark job server from Beam source. # -PsparkMasterUrl is optional. If it is unset the job will be run inside an embedded Spark cluster. ./gradlew :runners:spark:3:job-server:runShadow -PsparkMasterUrl=spark://localhost:7077 # In a separate terminal, run: go run github.com/apache/beam/sdks/v2/go/examples/wordcount@latest --input <PATH_TO_INPUT_FILE> \ --output counts \ --runner spark \ --endpoint localhost:8099
- Learn more about the Beam SDK for Go and look through the godoc.
- Walk through these WordCount examples in the WordCount Example Walkthrough.
- Clone the Beam Go starter project.
- Take a self-paced tour through our Learning Resources.
- Dive in to some of our favorite Videos and Podcasts.
- Join the Beam users@ mailing list.
Please don’t hesitate to reach out if you encounter any issues!
Last updated on 2023/11/30
Have you found everything you were looking for?
Was it all useful and clear? Is there anything that you would like to change? Let us know!