Surface Xcode Test Results as Bitrise Output Variable

[juliancmg]

EDITED:
Feature request to have number of tests passing vs. failing as an output variable for Xcode tests. Or at least the names of the tests that failed.

[viktorbenei]

Thanks for the Feature Requests @juliancmg!

@juliancmg

Would be nice to be able to send a list of tests that passed, failed, or just a list of all tests that were run and their result (pass/fail) as a BitRise output variable.

How exactly would that look like? Can you provide an example?

Wouldn’t that be a “too long” list? E.g. what if you have 100+ tests?

@juliancmg

This way , we can surface this on Slack to have better alerting for tests that are failing.

But to see why exactly that test failed you’d still have to open the related bitrise.io page, right?
I mean, from only “Test X failed” you won’t have enough information why it failed.

P.S.: you can include the build URL in your slack message, with $BITRISE_BUILD_URL

[juliancmg]

Hi Viktor, noted - I didn’t think through the implications of if ALL the tests were reported as in my original feature request. I’ve updated the summary to match maybe more of a regular use case for Slack alerts (high level).

[viktorbenei]

Thanks!

Still, do you have any example idea?
All I can think of won’t solve that you still have to go to the build’s page to check the logs/details, which don’t really solve the problem (I guess the problem is that you don’t even want to check the build page, only Slack).

[simonswift]

Would be nice at least to get numbers of tests ran, how many passed / failed… Other CI platforms I’ve used like Jenkins make that available. Currently BITRISE_XCODE_TEST_RESULT just outputs “failed” which doesn’t tell me much. Yes I can go into the build on Bitrise then click into the test results but it’s nice to have the numbers there in Slack for my team to see.

[gabor_tachtler]

Have you found a solution to the problem?