Help required: Proper way to push combined code coverage of unit, robolectric- and instrumented-tests to SonarQube
Description of the issue
I’ve created a jacocoTestReport and a sonarqube gradle task. I can execute them locally on my machine and push a combined coverage report for the complete test suite, i.e. unit-, robolectric- and instrumented tests. The instrumented tests run on a locally spawned Android emulator. The psuh to SonarQube is successful and includes the complete coverage report.
Before implementing the jacocoTestReport and sonarqube gradle tasks, I used the Virtual Device Testing step (beta) to execute my instrumented tests, which worked properly. Since I need to get a combined xml report for the test coverage I have to execute all test categories on a single machine so that the tests can update the existing reports with more data. With the Virtual Device Testing step this does not seem to be possible easily. I can think of two options:
a) Since I have to execute the instrumented tests first and then the robolectric- and unit-tests, it would suffice to copy the coverage data from Firebase Test Labs to the Bitrise VM before executing the other tests in context of the Bitrise VM. But I do not have a clue if that is even possible.
b) Not using the Virtual Device Testing step and execute the new Gradle tasks all inside the Bitrise VM. This is what I’ve done and what does not work.
Here’s a description how I implemented way b) and what happens on building:
Basically I based my workflow on the default primary workflow, but made some changes. You can see the full workflow in the screenshot:
The first step is the AVD Manager, then after the Install Missing Android SDK components, I call the Gradle task assemble for my test suite to prepare while the emulator might still be launching. Then I wait with the “Wait for Android emulator” step. In a following script step I perform the commands
adb shell settings put global window_animation_scale 0
adb shell settings put global transition_animation_scale 0
adb shell settings put global animator_duration_scale 0
in order to speed up test execution. Then I call the Gradle task jacocoTestReport. This is where my problems arise:
About 70% of my tests fail with the following message:
java.lang.RuntimeException: Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds. If you specified a non default root matcher, it may be picking a root that never takes focus. Root:
Root{application-window-token=android.view.ViewRootImpl$W@4429e2b, window-token=android.view.ViewRootImpl$W@4429e2b, has-window-focus=false, layout-params-type=1, layout-params-string={(0,0)(fillxfill) sim={forwardNavigation} ty=BASE_APPLICATION wanim=0x10302f8
Environment:
Android & Docker, on Ubuntu 16.04
Involved steps: AVD Manager / Wait for Android Emulator / Script / Gradle UI test
→ last one fails
Reproducibility
- a Rebuild (even without caches) does not help
- the issue happens all the time, but it seems that different tests fail
- all steps are used in the latest version
- the issue was always there
Local reproduction: Linux / Android (docker based) stack builds
Have not set up local building yet.
