Support test retries in the Xcode Test Step

Description of the feature request

Would like to request an option to retry failed tests in the Xcode Test Step if the test is flaky or the Simulator crashes. This allows for more reliable test runs and avoids having to re-run PR builds, allowing for quicker turnaround.

Use case / for what or how I would use it

We could minimize the number of failed builds caused by testing related failures. This would prevent having to manually rebuild those failed builds. If there are flaky tests in the project, they could be automatically identified without contacting a developer or Bitrise Support.

Also if the Xcode Simulator crashes during the Xcode Test Step, only the whole test suite can be rerun. Ideally, you could rerun only the failed tests or the ones that have not been run yet. Rerunning test that has not been run yet would reduce build time.

Related info - WWDC reference

Affected components

3 Likes

+1 on this

Hi there, thanks for your feature request! :slight_smile:

1 Like

should_retry_test_on_fail utilizes the new test repetition if Xcode version >= 13 and Test Plan support introduced via the new test_plan Step Input. The changes are available in version 2.7.0.