Test Report Addon Shows invalid overview numbers for JUNIT xml test reports

addon
#1

We are running appium tests and generate Junit reports using Mocha and Node.js
Tests overview shown in test reports add-on is not accurate!

You can clearly see from the image there are more tests overall and more passed than reported.

I can provide the generated junit xml if it helps.

0 Likes

#2

This is the number of Test Suites that failed.

When you click to view the Test Cases in the Test Suite you will see the count of failed Test Cases

0 Likes

#3

The issue is not that, Damien, see the overall numbers and count just a few passed test cases from tet results column, thez do not match, seems like this addon just shows the first level of tests, as seen on the image. Which, at least in our case, are test suites (describe in mocha) .

0 Likes

#4

Can you share the JUnit XML please?

0 Likes

#5

Here is a link to a single test suite report
https://drive.google.com/file/d/1283oTnVfwsqrVVAtJy0EBVqsDML2fBgb/view?usp=sharing

Here is a link to all of our test cases merge together https://drive.google.com/file/d/1Z971QeVJpjtYQDrJ61jI3OgEUBeJb7NS/view?usp=sharing

0 Likes

#6

It looks like a lot of your Test Suites contain zero Test Cases.

We only include TestSuites that have test cases in our reported numbers.

Without a single passing or failing test case the Test Suite is ignored.

0 Likes

#7

Hm dropping out things is arguable, as a good idea, user might wanna see this for whatever reason, however that does not change the fact that the numbers are wrong in the overview and it seems they just show a number of test suites.

0 Likes

closed #8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

0 Likes