Test Reports wasn't configured correctly

#1

Hi! I’m evaluating Bitrise and set up a macOS project using the assistant. The project builds successfully except for the Test Reports add-on that gives the message “Test Reports wasn’t configured correctly. Please make sure you configure it in the Workflow Editor.”

I verified the workflow and the steps against the documentation as well as the video showing how to set it up. Everything seems as it should be.

The build log gives a clue as to something going wrong. It says it ran 12 tests:

Executed 12 tests, with 0 failures (0 unexpected) in 0.164 (0.184) seconds
▸ Test Succeeded

But then in the deploy step, it says it uploaded 0 of them:

Upload test results
- uploading (0) test results
Success

The steps used in my workflow are:

  • Xcode Test for Mac, version 1.4.1
  • Deploy to Bitrise.io, version 1.10.0

Is it possible that the test step is not generating artifacts as the deploy step expects them? Or maybe the deploy step doesn’t support macOS project tests?

0 Likes

#2

Hi there @steveroy :wave:

Could you please send me a build URL of the related build and enable Support Access on the Settings tab of the app (https://devcenter.bitrise.io/troubleshooting/enabling-bitrise-support-user/), so that I may take a closer look? :slightly_smiling_face:

0 Likes

#3

Hi @Roland-Bak and thanks for following up!

I turned on Support Access and here’s a link to the latest build. You should be able to see in the log that the Deploy step uploads 0 test results even though 12 tests ran successfully above.

https://app.bitrise.io/build/a0f945c828b5326e#?tab=log

0 Likes

#4

Thanks for your patience, @steveroy! :slightly_smiling_face:

By Default, the Test Reports plugin works with the following steps:

  • Xcode Test for iOS .
  • Android Unit Test .
  • iOS Device Testing .
  • Virtual Device Testing for Android .
  • Flutter Test.

Since you are using Xcode Test for Mac, you need to do a bit of working around to make it available for Test Reports.
Have you tried exporting your test results via this guide: https://devcenter.bitrise.io/testing/exporting-to-test-reports-from-custom-script-steps/ ? :slightly_smiling_face:

Seems like including an Export test results to the Test reports add-on step should help :wink:

0 Likes

#5

Thanks @Roland-Bak! That makes perfect sense. I gave that a go and still am not getting anything in Test Reports. It’s probably a configuration. In particular I suspect the value of Path where custom test results reside is incorrect. I tried using BITRISE_XCODE_TEST_RESULT, which is what’s output by the Xcode Test for Mac step. Still no go. Is there a variable that points to the derived data dir holding my project build output and xctest results?

Here are the relevant bits from my current yml:

- xcode-test-mac@1:
    inputs:
    - project_path: "$BITRISE_PROJECT_PATH"
    - scheme: "$BITRISE_SCHEME"
- custom-test-results-export@0:
    is_always_run: false
    inputs:
    - search_pattern: "*"
    - base_path: "$BITRISE_XCODE_TEST_RESULT"
    - test_name: Unit Tests
- deploy-to-bitrise-io@1: {}
0 Likes