Hey! I’ve started uploading my JUnit reports to use Tests Insights. How long does it take the dashboards to update? I’ve seen somewhere that it has a 30 min refresh rate but I’ve tests that haven’t appeared after a couple of hours.
Tíarnán
Hi @tiarnansqsp 
Should be less than 30 mins, might be a bit longer, but definitely shouldn’t be multiple hours.
Where do you upload the JUnit report? Do you see it in the Test Reports addon?
On the build’s page you can find the uploaded test report here:
If you can see the test report there then we’d need a build URL so we can check what happened.
If you can’t see the test report there that means that you’re not uploading the JUnit test report as a test report, but maybe you’re uploading it as an Artifact/as a common file (in which case you’ll find the file under the Artifacts tab on the build details page, but “common files” are not processed as test reports).
Hey @viktorbenei
Where do you upload the JUnit report?
I upload the JUnit report inside my workflow. I’ve a step which copies it over to the bitrise test dir
Do you see it in the Test Reports addon?
Yeah, I see all the tests appear in the Test Reports add-on. They are definitely not being uploaded to Artifacts. I checked the logs of the Deploy step and it has my JUnit report listed as a test result and not an artifact.
But when I go to the Insights page then to Tests. It displays incorrect information. Here are a list of the issues I found:
- The dashboards took 1 week to update. The total test runs are a fraction of what they should be. I have around 1300 test cases running for each build. It says there are a total of 4177 test runs for the past week. I think it should be closer to 80k.
- The test cases are listed at the bottom of the page. Under the “Test Runs” column, nearly all of them say 2 but when I tap a test case and go to the detailed view. There are like 67 related test case runs which is correct
I’ve created a ticket on Zendesk #35661 to try and help resolve the issue.
Thanks,
Tíarnán
Just saw the Zendesk ticket with the additional details, and it indeed looks like there’s a bug in the algorithm used for the charts. As you mentioned here, the Test Runs section properly lists the runs and it’s up to date as expected (no more than 1h delay), but the charts for some reason don’t receive the proper aggregated data.
Team is looking into it, we’ll respond as soon as we find out what’s causing this.
Thanks for reporting and sorry for the issue.
No worries, thanks for checking it out!
Tíarnán
@tiarnansqsp bug fix has been implemented and it’s now deployed into production. Please let us know if you’d still notice anything off, or if it’s working as expected now. We checked your linked app and the data seemed correct now as far as we can tell. Thanks again for reporting the bug and sorry for the issue.
@viktorbenei Awesome. I’ve checked the dashboard now and everything seems like its working. Thanks for looking into this!
1 Like