What is a flaky test?
Why flaky tests are a problem
Flaky tests are a problem for test suites because not only because they reduce the success rate of a test suite, but because they reduce confidence in the tests. The cost of this adds up quickly across an engineering team working on a project. A symptom of this is spending time re-running failed tests or incorrectly diagnosing a failed test as a flaky test.
What makes flaky tests hard to fix
Flaky tests produce a small signal over time, without a tool to track that signal it is incredibly hard to find a root problem or demonstrate progress. Sometimes this data gets captured as war-stories from veterans on the project or
@retry-3 annotations around tests.
How TestRecall make this easier
Finding flaky tests should not require a superhero, and demonstrating increased stability of a test suite should be easy. TestRecall helps by tracking tests over time, making it easy to spot flaky tests - and measure the improvements you are making