# Surface Xcode Test Results as Bitrise Output Variable

**URL:** https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630
**Category:** Steps
**Created:** [May 2, 2017, 9:24pm UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630 "2017-05-02T21:24:46Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![juliancmg](https://avatars.discourse-cdn.com/v4/letter/j/f04885/32.png) [@juliancmg](https://discuss.bitrise.io/u/juliancmg)
#### Post date: [May 2, 2017, 9:24pm UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/1 "2017-05-02T21:24:46Z")

</div>

EDITED:  
Feature request to have number of tests passing vs. failing as an output variable for Xcode tests. Or at least the names of the tests that failed.

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [May 3, 2017, 9:12am UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/2 "2017-05-03T09:12:34Z")

</div>

Thanks for the #Feature Requests @juliancmg!

> [@juliancmg](#):
>
> Would be nice to be able to send a list of tests that passed, failed, or just a list of all tests that were run and their result (pass/fail) as a BitRise output variable.

How exactly would that look like? Can you provide an example?

Wouldn’t that be a “too long” list? E.g. what if you have 100+ tests?

> [@juliancmg](#):
>
> This way , we can surface this on Slack to have better alerting for tests that are failing.

But to see why exactly that test failed you’d still have to open the related [bitrise.io](http://bitrise.io) page, right?  
I mean, from only _“Test X failed”_ you won’t have enough information why it failed.

**_P.S.: you can include the build URL in your slack message, with `$BITRISE_BUILD_URL`_**

---

<div class="post-metadata">

### Author: ![juliancmg](https://avatars.discourse-cdn.com/v4/letter/j/f04885/32.png) [@juliancmg](https://discuss.bitrise.io/u/juliancmg)
#### Post date: [May 3, 2017, 6:57pm UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/3 "2017-05-03T18:57:10Z")

</div>

Hi Viktor, noted - I didn’t think through the implications of if ALL the tests were reported as in my original feature request. I’ve updated the summary to match maybe more of a regular use case for Slack alerts (high level).

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [May 4, 2017, 11:08am UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/4 "2017-05-04T11:08:07Z")

</div>

Thanks! 🙂

Still, do you have any example idea?  
All I can think of won’t solve that you still have to go to the build’s page to check the logs/details, which don’t really solve the problem (I guess the problem is that you don’t even want to check the build page, only Slack).

---

<div class="post-metadata">

### Author: ![simonswift](https://avatars.discourse-cdn.com/v4/letter/s/858c86/32.png) [@simonswift](https://discuss.bitrise.io/u/simonswift)
#### Post date: [June 17, 2020, 5:04pm UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/5 "2020-06-17T17:04:02Z")

</div>

Would be nice at least to get numbers of tests ran, how many passed / failed… Other CI platforms I’ve used like Jenkins make that available. Currently BITRISE\_XCODE\_TEST\_RESULT just outputs “failed” which doesn’t tell me much. Yes I can go into the build on Bitrise then click into the test results but it’s nice to have the numbers there in Slack for my team to see.

---

<div class="post-metadata">

### Author: ![gabor\_tachtler](https://avatars.discourse-cdn.com/v4/letter/g/a4c791/32.png) [@gabor\_tachtler](https://discuss.bitrise.io/u/gabor_tachtler)
#### Post date: [April 29, 2021, 5:48pm UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/6 "2021-04-29T17:48:42Z")

</div>

Have you found a solution to the problem?

---

<div class="post-metadata">

### Author: ![markturner](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@markturner](https://discuss.bitrise.io/u/markturner)
#### Post date: [May 15, 2024, 3:15am UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/7 "2024-05-15T03:15:03Z")

</div>

Happy to contribute and add support for this.

To add some technical details it seems relatively straightforward by using `xcresulttool` to parse the results to output json:

```shell
xcrun xcresulttool get --path ./Test.xcresult --format json > result.json

```

We can parse the output and set the environmental vars.  
Here’s some code as example:

```go
    // Get test results using the test summary ID
    cmd = exec.Command("xcrun", "xcresulttool", "get", "--path", xcresultPath, "--id", testSummaryId, "--format", "json")
    output, err = cmd.Output()
    if err != nil {
        fmt.Printf("Error running xcresulttool for test results: %v\n", err)
        os.Exit(1)
    }

    // Parse JSON output to get the test results
    var results TestResults
    err = json.Unmarshal(output, &results)
    if err != nil {
        fmt.Printf("Error parsing JSON: %v\n", err)
        os.Exit(1)
    }

    // Set environment variables for the number of tests passed and failed
    passedCount := results.Summary.TestsPassedCount.Value
    failedCount := results.Summary.TestsFailedCount.Value

    os.Setenv("BITRISE_XCODE_TESTS_PASSED_COUNT", fmt.Sprintf("%d", passedCount))
    os.Setenv("BITRISE_XCODE_TESTS_FAILED_COUNT", fmt.Sprintf("%d", failedCount))

```

---

<div class="post-metadata">

### Author: ![markturner](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@markturner](https://discuss.bitrise.io/u/markturner)
#### Post date: [May 15, 2024, 3:35am UTC](https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630/8 "2024-05-15T03:35:16Z")

</div>

> <https://github.com/bitrise-steplib/steps-xcode-test/pull/245>
>
> \<!--
> You are amazing! Thanks for contributing to our Step library!
> Please …fill this template with the details of your change.
> \--\>
> 
> \### Checklist
> \<!--
> Put an \`x\` in the boxes that apply. You can also fill these out after
> creating the PR. This is simply a reminder of what we are going to look
> for before merging your code.
> \--\>
> \- \[x\] I've read and followed the \[Contribution Guidelines\](https://github.com/bitrise-steplib/.github/blob/main/CONTRIBUTING.md)
> \- \[\] \`step.yml\` and \`README.md\` are updated with the changes (if needed)
> \- \[\] Updated \`bitrise.yml\` with necessary environment variables
> 
> \### Version
> 
> Requires a \*MINOR\* \[version update\](https://semver.org/)
> 
> \### Context
> 
> \<!--- 
> One sentence summary on why the change is needed.
> Describe the big picture of your changes here to communicate to the
> maintainers why we should accept this pull request.
> \--\>
> This PR adds functionality to parse \`.xcresult\` files and set the number of tests passed and failed as environment variables (\`BITRISE\_XCODE\_TESTS\_PASSED\_COUNT\` and \`BITRISE\_XCODE\_TESTS\_FAILED\_COUNT\`).
> 
> \<!-- Please link the issue that the PR fixes.
> Resolves: #GITHUB\_ISSUE\_ID or https://link\_to\_the\_issue\_on\_discuss.bitrise.io.
> \--\>
> Resolves: https://discuss.bitrise.io/t/surface-xcode-test-results-as-bitrise-output-variable/1630
> 
> \### Changes
> 
> \<!-- 
> Details are important, and help maintainers processing your PR.
> Please list additional details, for example:
> - Update dependencies
> - Make \`foo\` optional in \`main.go\`
> - \`foo\` now returns an \`error\` for better error handling
> \--\>
> \- Added structs \`TestSummary\` and \`TestResults\` to \`main.go\` for parsing \`.xcresult\` JSON output.
> \- Added \`ExportTestResults\` method in \`main.go\` to parse the \`.xcresult\` file and set the environment variables for the number of tests passed and failed.
> \- Updated \`ExportXCResultBundle\` method to call \`ExportTestResults\`.
> 
> \### Investigation details
> 
> 
> Alternative solutions involved parsing the \`.xcresult\` files using different tools, but \`xcresulttool\` was chosen for its reliability and ease of use with Xcode-generated files.
> 
> \### Decisions
> 
> 
> \- Decided to use \`xcresulttool\` for parsing \`.xcresult\` files.
> \- Chose to add environment variables \`BITRISE\_XCODE\_TESTS\_PASSED\_COUNT\` and \`BITRISE\_XCODE\_TESTS\_FAILED\_COUNT\` to provide clear metrics on test results.
