Post-commit test task guides

These guides provide steps for common post-commit test failure tasks.

Finding someone to triage a post-commit test failure

To find the proper person to triage a test failure, you can use these suggestions:

  1. If you can triage it yourself, go for it.
  2. Look at the GitHub blame for the files with problematic code.
  3. Ask in the Beam Slack chat.
  4. Write to the dev list: dev@beam.apache.org

Rolling back a commit

Rolling back is usually the fastest way to fix a failing test. However it is is often inconvenient for the original author. To help the author fix the issue, follow these steps when you rollback someone’s change.

  1. Rollback the PR (or individual commit of the PR). The rollback PR should be green except in rare cases.
  2. Create a GitHub issue that contains the following information:
    • the reason for the rollback
    • a link to the test failure’s GitHub issue
    • triage information
    • any other relevant details
  3. Assign the new GitHub issue to the original PR author.
  4. Consider re-opening the GitHub issue associated with the original PR (if there is one).
  5. Send a notification email with information about the rollback, links to the original PR and the rollback PR, and the reasons for the rollback to:
  6. Close the test failure GitHub issue. Your work is done here!

Disabling a failing test

If a test fails, our first priority is to rollback the problematic code and fix the issue. However, if both: rollback and fix will take awhile to implement, it is safer to temporarily disable the test until the fix is ready.

Use caution when deciding to disable a test. When tests are disabled, contributors are no longer developing on top of fully tested code. If you decide to disable a test, use the following guidelines:

While the test is disabled, contributors should not push code to the failing test’s coverage area. The code area is not properly tested until you fix the test.