What’s new?
We have given the test results experience a comprehensive facelift and added a bunch of new features to help you recover from test failure super-fast. The following features are included for all customers:
-
Tests intuitively arranged by test suite and if appropriate device type
-
All attachments (e.g. screenshots, UI test videos, logs) are shown against each test, without you needing to visit the Testing Add-on
-
If you perform device testing with Firebase TestLab all your test results and attachments will be present in the main test report
-
Flaky test reporting: if you configured your build to automatically re-try failed tests (e.g. using Bitrise’s Xcode testing steps) and your test delivers a mixture of pass and fail on retry, we will flag the test as flaky
What problem does this solve?
When a test fails, developers want to discover the root cause as soon as possible so they can make a fix and merge their code. The new test report format brings everything needed to analyse a failure into one place - info about the failure reason, test suite (and where appropriate) screenshots, videos failing devices are all presented in one page.
The introduction of flaky test identification and reporting means developers can take action to fix or bypass troublesome tests, which can be such a time sink in the dev process. We will soon introduce a flaky test quarantine capability to help with this process - watch this space!
How to get started
This is the best bit… you don’t need to change anything in your current setup to see the new test report experience. Even if your builds are configured to display rich test reports in the Testing Add-on, all the info will be provided in the new test report.
Happy testing! 