# 🔄 Rerun failed tests and identify flaky tests automatically **URL:** \ **Category:** Website\ **Tags:** testing, website, feature-update\ **Created:** [October 14, 2025, 4:04pm UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617 "2025-10-14T16:04:20Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![kaushal.bitrise](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/kaushal.bitrise/32/5305_2.png) [@kaushal.bitrise](https://discuss.bitrise.io/u/kaushal.bitrise)\ **Post date:** [October 14, 2025, 4:04pm UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/1 "2025-10-14T16:04:20Z")
You can now configure specific build steps to automatically rerun the failed tests . This helps prevent flaky tests from blocking your pull requests and gives you clear insight into which tests are unstable. ### đŸ˜„ The problem with flaky tests - flaky tests block PRs even when the code is fine. A single transient failure forces you to dig through logs, hit rerun, and wait, slowing merges and masking real regressions. - With targeted reruns, transient failures are retried automatically. If they pass on retry, the build can go green, and the test is marked flaky, so you can fix or quarantine it later. ### 🆕 What’s new The following steps allow automatic reruns for failed tests: 1. Xcode Test for iOS 2. Xcode Test without building 3. iOS Device Testing 4. Android unit test (via [Gradle plugin](https://github.com/gradle/test-retry-gradle-plugin)) 5. Virtual Device Testing for Android This feature introduces new inputs to our testing steps, allowing you to specify the number of retry attempts for failed tests. When a test fails during a run, the step will automatically retry that specific test up to the maximum configured. If the test passes on a subsequent attempt, it is marked as “flaky” in the new test report; however, the step itself will still succeed, allowing your build and PR check to turn green. This allows you to merge your changes without being blocked, while still providing the data you need to identify and fix unstable tests later. For example, a UI test might occasionally fail because a network request times out or an animation hasn’t completed. Instead of failing the entire build, you can now configure it to retry failed tests up to 3 times. If it passes on the second try, your PR is unblocked, and you can see in the test report that this specific test was flaky. ### 🩼 How to set it up Configuration is done via inputs in the specific workflow step. * * * **For** **`xcode-test-for-ios`** **and** **`xcode-test-without-building`** **steps:** Set the following inputs in your step configuration: - `test_repetition_mode`: Defines the repetition mode. For example, `retry_on_failure` will only rerun the tests that failed. - `maximum_test_repetitions`: The maximum number of times to repeat a test. - `relaunch_tests_for_each_repetition`: Controls whether the app is relaunched for each repetition. ![Test repetition mode](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/2X/e/e4d851fae9580f80750e188dac50e0a2e4b357e1.png) * * * **For** **`ios-device-testing`** **and** **`virtual-device-testing-for-android`** **steps:** Set the following input in your step configuration: - `num_flaky_test_attempts`: Specifies the number of times to retry a failed test execution. An execution that fails initially but succeeds on a reattempt is reported as flaky. The maximum value is 10. The default is 0 (no reruns). You can very well do this via the UI for the step input below. ![Reattempt test execution](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/2X/c/c913898ba2bf1dcecf11718f7eaf44ebec8a299d.png) * * * **For the** `android-unit-test` **step:** This functionality is configured directly through the [Gradle plugin](https://github.com/gradle/test-retry-gradle-plugin). You can enable retries for failed tests within your project’s `build.gradle` file. ### Link to docs [here](https://docs.bitrise.io/en/bitrise-ci/testing/detecting-and-quarantining-flaky-tests.html#configuring-flaky-test-detection) ---
**Author:** ![jeremyauclert](https://avatars.discourse-cdn.com/v4/letter/j/8491ac/32.png) [@jeremyauclert](https://discuss.bitrise.io/u/jeremyauclert)\ **Post date:** [October 24, 2025, 10:27am UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/2 "2025-10-24T10:27:19Z")
Hi there! On Android it seems like it re-runs the entire test suite, for example if one test fails from 100, it will re-run all 100 and if the original failure passes then the test is marked as flaky (I’m guessing this is what is explained here: “Specifies the number of times to retry a failed test execution.”) We were hoping that the process would work the same as iOS: “`test_repetition_mode`: Defines the repetition mode. For example, `retry_on_failure` will only rerun the tests that failed.” Thanks. Jeremy ---
**Author:** ![kaushal.bitrise](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/kaushal.bitrise/32/5305_2.png) [@kaushal.bitrise](https://discuss.bitrise.io/u/kaushal.bitrise)\ **Post date:** [October 24, 2025, 10:47am UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/3 "2025-10-24T10:47:52Z")
Hi @jeremyauclert - can you confirm which step you’re referring to on Android’s side, please? ---
**Author:** ![jeremyauclert](https://avatars.discourse-cdn.com/v4/letter/j/8491ac/32.png) [@jeremyauclert](https://discuss.bitrise.io/u/jeremyauclert)\ **Post date:** [October 24, 2025, 12:57pm UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/4 "2025-10-24T12:57:55Z")
Hi @kaushal.bitrise and thank you for getting back to me, this is the area: ![Screenshot 2025-10-24 at 13.54.44](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/2X/0/0909931987e9bbfc5fed2ec5f2419284b7b538db.png) This: **Number of times a test execution is reattempted\*** in the Virtual device testing for Android config section. Thanks. Jeremy ---
**Author:** ![kaushal.bitrise](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/kaushal.bitrise/32/5305_2.png) [@kaushal.bitrise](https://discuss.bitrise.io/u/kaushal.bitrise)\ **Post date:** [October 24, 2025, 1:17pm UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/5 "2025-10-24T13:17:20Z")
Thanks for clarifying @jeremyauclert. You are right. Our device testing steps are currently based on Firebase Test Lab. And they don’t allow us granular control on this. We’ve included this in the docs (screenshot below), but I appreciate it is not super clear. ![Screenshot 2025-10-24 at 14.14.12](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/2X/5/5bfb411ced222602a2ea88cf0d88962a8354b16e.png) I wish we could help with this. I take your feedback, though. And appreciate you sharing it. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/2X/d/d901cf2aad020f2dd11918afc89ec2089bb87b35.png) [@system](https://discuss.bitrise.io/u/system)\ **Post date:** [December 23, 2025, 1:17pm UTC](https://discuss.bitrise.io/t/rerun-failed-tests-and-identify-flaky-tests-automatically/25617/6 "2025-12-23T13:17:38Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.