How to run Android UI tests on virtual devices

Hi, I’ve been trying many different configurations with the x86 emulators, but unfortunately all my test are failing now.
Getting messages like:

- RuntimeException: Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds.

I applied all techniques I found on my research but I can’t solve it.
I don’t even know how or where to pass the test-result and artifact to see them on the build tab.

Any help?

Hi @kuassivi ,

Can you please create an issue report in #issues:build-issues (http://discuss.bitrise.io/c/issues/build-issues) ? It’d help a lot (especially the log / build URL), so that we can look into it ASAP! :slight_smile:

Sure, I’ll do it in a moment. Thanks.
Anyway, about the env variable to use to pass the test-results, Am I right with the assumption that I need to do it manually (with a script), or Is it automatically collected by Bitrise after the Gradle Step somehow?

Depends, if you’re referring to output files of a tool (other than the base / standard ones, like APK in case of Gradle Runner) you might have to specify where the file(s) are located or move them into the $BITRISE_DEPLOY_DIR. Related docs: Build artifacts online - Bitrise Docs

I refer to the test-result files/folder generated after the connectedAndroidTest task. Do I need to move them manually to the $BITRISE_DEPLOY_DIR?

Yes, you most likely have to.

In general I’d suggest you to try the CLI locally on your Mac/Linux (https://www.bitrise.io/cli / How to experiment with Bitrise configs locally, on your Mac/Linux), that way you can check what’s moved automatically into the $BITRISE_DEPLOY_DIR and what isn’t.

The build log in general should also include the files which are moved into the deploy dir; e.g. the Gradle Runner prints every file at the bottom of it’s “step log box” which are moved into the deploy dir.

If you’d have any questions just let us know! :slight_smile:

1 Like

Hi @viktorbenei, all the links you provided were very helpful, thanks.

It’s working properly now, I don’t see the window-focus=false issue anymore. And it’s indeed saving build time, and probably more with a proper cache!

Only to remark that the Deploy Step doesn’t work from the local CLI.

1 Like

Glad to hear, thanks for reporting @kuassivi! :slight_smile:

Indeed; there’s a technical explanation for this, simply that Artifacts can only be uploaded for a specific Build, and when you run the step locally you don’t have any Build URL / “build context” to work with.

If you have any idea for this (where would it upload the artifact from your own local Mac/PC) just let us know!

Right now we’re thinking about allowing the step to run locally (in a “non CI environment”), but in “non CI mode” it would simply print out what files it would upload, it would simply skip the actual upload.

Yup, that was what I would have suggested. There is no real need of moving the files locally.

I also realised you have to create a new step for every different path you want to upload. Why not adding a multiline inputbox like other steps?

Regarding the zip option you have to choose for recursive folders, I can suggest to add a capability like the inline zip extractor functionality of google drive which is based on a js lib that can, perhaps, store temporally the extracted files into the “deployment build directory” and show off the files and paths in a prettify way on the Artifacts tab.

I hope to get some free time to contribute with some steps :slightly_smiling_face:.

1 Like

Definitely great ideas, thank you @kuassivi! :slight_smile:

I’d suggest you to create a Feature Requests at least for the zip extractor, actually we already have a related one, might worth to just add it as a comment there for better visibility / tracking: Artifacts: Make it possible to view HTML files with relative links (CSS etc. refs) - although zip extractor alone might not solve that issue directly…

If you’d have any questions just let us know! :wink:

Hi @tamaspapik, it’s working very well, thank you very much!
However, screenshots are missing e.g. https://www.bitrise.io/build/618fca3ce921e377
Are you able to help?
I’m using version 0.9.9 of the step

Hey @eric-grab!

In the build you’ve sent me shall not be any screenshots. Robo test is a test type that makes screenshots automatically, but you’ve ran instrumentation test which will create screenshots only if you code your test to do it. Instrumentation tests also has a video of it, what you can check under the Virtual Device Tests tab

1 Like

Hi @tamaspapik, do you know Spoon and how it can work with this awesome Bitrise step? :smiley:

I believe the things that need to happen are:

  • Replace the Gradle task that runs the tests with ./gradlew spoon (I’m using a Gradle plugin; this runs the tests + generate reports)
  • Copy the reports in “./app/build/spoon-outputs/” to “Apps & Artifacts”

Thanks a bunch

Hey @eric-grab!

Unfortunately Spoon looks like a client side tool, and to use VDT step, you will need to generate the exact test and app apks.

Just checked Spoon’s documentation, and I can’t see a way of generating them.

Sorry @tamaspapik, let’s forget Spoon for a moment, I’ll rephrase my question as follows:

  • Replace the Gradle task that runs the tests with ./gradlew someTask
  • Copy the files in ./app/build/someFolder/ to “Apps & Artifacts” or “Virtual Device Testing” output directory

Is that possible?

You should be able to replace the gradle task. And also if you have Deploy to Bitrise.io step in your workflow, then files copied to $BITRISE_DEPLOY_DIR should be deployed to the “Apps & Artifacts” tab.

Please contact us on our on-site chat if you encounter any issue doing these, or please send a detailed issue report in the

http://discuss.bitrise.io/c/issues/build-issues

section. :blush:

1 Like

Sorry @tamaspapik, how exactly can I “replace the Gradle task”?
There is absolutely no option to configure Gradle for the Virtual Device Testing step

Understood on the use of Deploy to Bitrise.io step, thanks a lot :smiley:

1 Like

So in your VDT step have the two APK path and Test APK path configured. To change gradle task to generate another flavor APKs, you will need to set “tasks” input in Gradle Runner step.

1 Like

How is that possible @tamaspapik? The virtual device/ emulator only lives within the time the Virtual Device Testing step right?
I’ve already tried having Gradle Runner task right after Virtual Device Testing but I got java.lang.RuntimeException: No device(s) found. e.g. https://www.bitrise.io/build/e48b280ad5781812

I believe that @eric-grab asked about Gradle task used to perform the tests like connectedCheck or spoon NOT about one generating APKs (like assemble).

In case of Firebase Test Lab devices are not connected to local ADB server. APKs (testing and tested) need to be uploaded and then you (or step) can execute a command which starts the tests: gcloud firebase test android run  |  Google Cloud CLI Documentation