# How to run Android UI tests on virtual devices **URL:** \ **Category:** Android\ **Tags:** build-cache, android, test, ui\ **Created:** [August 16, 2017, 11:02am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496 "2017-08-16T11:02:39Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![tamaspapik](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/tamaspapik/32/251_2.png) [@tamaspapik](https://discuss.bitrise.io/u/tamaspapik)\ **Post date:** [August 16, 2017, 11:02am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/1 "2017-08-16T11:02:40Z")
The new feature, Virtual Device Testing for Android is here! **It is currently in private-beta test phase, please ping us if you want to join! One test can be ran in one build at the moment, and robo test type has some limitations of watching the screenshots and the activity-map.** Let’s see an example flow for an espresso instruments test. I’ve used this sample project: > **[GitHub - bitrise-io/android-espresso-ui-test-sample](https://github.com/bitrise-io/android-espresso-ui-test-sample)** > > Contribute to bitrise-io/android-espresso-ui-test-sample development by creating an account on GitHub. ### Add your app, let the scanner setup your workflow It will create the default workflows, what we will need is the primary workflow. Originally this workflow will build a debug apk. ### App settings Turn on **Virtual Device Testing** on the app’s settings tab. ### How to change your primary workflow - Add an extra task **assembleDebugAndroidTest** in your gradle-runner’s gradle task input, so it will be something like: `assembleDebug assembleDebugAndroidTest`. - Add Virtual Device testing for Android step after your gradle-runner step. - As a minimum, in Virtual Device testing for Android step set **Test APK path** inputs to your built debug apk, in my case Test APK path will be: `./app/build/outputs/apk/app-debug-androidTest-unaligned.apk` Take care of configuring only one section of the step, which matches your desired test type! (APK path is exported by gradle-runner step, and it is set as the default for the step’s input, if you want to use different APK then don’t forget to change the step’s input accordingly!) ### If you’ve done everything well, this is what you’ll see in your log: ![27 PM](https://sea2.discourse-cdn.com/flex016/uploads/bitrise/original/1X/e36522ff74006dfaca4c0be810ba5154e73c1d87.png) ### The available devices and its versions: ```auto ┌─────────────┬──────────┬────────────────────┬─────────────┬────────────────┬ │ MODEL_ID │ MAKE │ MODEL_NAME │ RESOLUTION │ OS_VERSION_IDS │ ├─────────────┼──────────┼────────────────────┼─────────────┼────────────────┼ │ Nexus4 │ LG │ Nexus 4 │ 1280 x 768 │ 19,21,22 │ │ Nexus10 │ Samsung │ Nexus 10 │ 2560 x 1600 │ 19,21,22 │ │ Nexus5 │ LG │ Nexus 5 │ 1920 x 1080 │ 19,21,22,23 │ │ Nexus5X │ LG │ Nexus 5X │ 1920 x 1080 │ 23,24,25,26 │ │ Nexus6 │ Motorola │ Nexus 6 │ 2560 x 1440 │ 21,22,23,24,25 │ │ Nexus6P │ Google │ Nexus 6P │ 2560 x 1440 │ 23,24,25,26 │ │ Nexus7 │ ASUS │ Nexus 7 (2012) │ 1280 x 800 │ 19,21,22 │ │ Nexus9 │ HTC │ Nexus 9 │ 2048 x 1536 │ 21,22,23,24,25 │ │ NexusLowRes │ Generic │ Low-res MDPI phone │ 640 x 360 │ 23,24,25,26 │ └─────────────┴──────────┴────────────────────┴─────────────┴────────────────┴ ``` ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [August 18, 2017, 4:33pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/2 "2017-08-18T16:33:41Z")
I tried it, it’s awesome @tamaspapik!! 🎉 😉 ---
**Author:** ![burak.malkoc](https://avatars.discourse-cdn.com/v4/letter/b/e0b2c6/32.png) [@burak.malkoc](https://discuss.bitrise.io/u/burak.malkoc)\ **Post date:** [October 5, 2017, 10:18pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/3 "2017-10-05T22:18:40Z")
> [@tamaspapik](#): > > APK path is exported by gradle-runner step, Can you also share your full .yml script? I am wondering how you setup the output variables for consecutive gradle runners? The second(last) gradle runner output(debug-androidTest.apk) variable overridden on first gradle runner(debug.apk) output so UI test runner is not being able to see the first gradle runner output(debug.apk) and fails. How can I solve this issue? @viktorbenei @tamaspapik ---
**Author:** ![tamaspapik](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/tamaspapik/32/251_2.png) [@tamaspapik](https://discuss.bitrise.io/u/tamaspapik)\ **Post date:** [October 9, 2017, 8:30am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/4 "2017-10-09T08:30:37Z")
Hey @burak.malkoc! Thank you for asking! Please see my test workflow below for this case. 🙂 ```auto --- format_version: '3' default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git project_type: android trigger_map: - push_branch: "*" workflow: primary - pull_request_source_branch: "*" workflow: primary app: envs: - opts: is_expand: false GRADLE_BUILD_FILE_PATH: build.gradle - opts: is_expand: false GRADLEW_PATH: "./gradlew" workflows: primary: steps: - activate-ssh-key@3.1.1: run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}' - git-clone@3.5.2: {} - cache-pull@2.0.0: {} - install-missing-android-tools@2.0.2: {} - gradle-runner@1.7.7: inputs: - gradle_file: "$GRADLE_BUILD_FILE_PATH" - gradle_task: assembleDebugAndroidTest - gradlew_path: "$GRADLEW_PATH" - apk_file_exclude_filter: - script@1.1.4: title: Store Test APK path with envman inputs: - content: |- #!/bin/bash envman add --key "BITRISE_TEST_APK" --value "$BITRISE_APK_PATH" - gradle-runner@1.7.7: inputs: - gradle_file: "$GRADLE_BUILD_FILE_PATH" - gradle_task: assembleDebug - gradlew_path: "$GRADLEW_PATH" - cache-push@2.0.1: {} - virtual-device-testing-for-android@0.9.7: inputs: - test_type: instrumentation - test_apk_path: "$BITRISE_TEST_APK" ``` ---
**Author:** ![han.tran](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/han.tran/32/927_2.png) [@han.tran](https://discuss.bitrise.io/u/han.tran)\ **Post date:** [October 25, 2017, 6:24am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/5 "2017-10-25T06:24:43Z")
@tamaspapik I’m thinking how to config for appium test for Android (maven). How exactly to config in my case? ---
**Author:** ![tamaspapik](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/tamaspapik/32/251_2.png) [@tamaspapik](https://discuss.bitrise.io/u/tamaspapik)\ **Post date:** [October 25, 2017, 10:22am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/6 "2017-10-25T10:22:05Z")
Seems like there is no option to run appium test with an adb command like `adb shell am instrument -w /` so I think it is not possible at the moment. ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [October 25, 2017, 12:45pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/7 "2017-10-25T12:45:19Z")
@bootstraponline do you have a tutorial for running Appium on Firebase? Or is that indeed not possible? ---
**Author:** ![bootstraponline](https://avatars.discourse-cdn.com/v4/letter/b/c68b51/32.png) [@bootstraponline](https://discuss.bitrise.io/u/bootstraponline)\ **Post date:** [October 25, 2017, 1:43pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/8 "2017-10-25T13:43:26Z")
Firebase test lab does not currently support appium. It’s not possible. ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [October 25, 2017, 2:11pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/9 "2017-10-25T14:11:55Z")
Thanks for confirming @bootstraponline! 🙂 ---
**Author:** ![han.tran](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/han.tran/32/927_2.png) [@han.tran](https://discuss.bitrise.io/u/han.tran)\ **Post date:** [October 26, 2017, 2:53am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/10 "2017-10-26T02:53:46Z")
Thanks for the answer, so I need back to work with Emulator which always hang at the install apk step 😥 ---
**Author:** ![bootstraponline](https://avatars.discourse-cdn.com/v4/letter/b/c68b51/32.png) [@bootstraponline](https://discuss.bitrise.io/u/bootstraponline)\ **Post date:** [October 30, 2017, 3:29am UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/11 "2017-10-30T03:29:10Z")
You could try Sauce Labs for emulators which support appium. ---
**Author:** ![kuassivi](https://avatars.discourse-cdn.com/v4/letter/k/a3d4f5/32.png) [@kuassivi](https://discuss.bitrise.io/u/kuassivi)\ **Post date:** [December 9, 2017, 1:14pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/12 "2017-12-09T13:14:17Z")
Hi, first off, awesome job! Thank you. I’m worrying about the time it says it took the whole testing step process, the log says the Virtual step took about 20 minutes, but the UI tests video shows just 12 minutes (including some initial delays doing nothing), so I was wondering what’s the quota time the system is going to take into account, regarding the build time consumption? ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [December 11, 2017, 4:31pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/13 "2017-12-11T16:31:11Z")
Hi @kuassivi, > [@kuassivi](#): > > , so I was wondering what’s the quota time the system is going to take into account, regarding the build time consumption? There’s no special quota, it can run as long as your subscription’s build timeout limit. > [@kuassivi](#): > > about 20 minutes, but the UI tests video shows just 12 minutes (including some initial delays doing nothing) Indeed, especially for popular devices, Firebase Test Lab (the service we use right now under the hood for android device testing feature, as FBTL was the most reliable solution in our tests) might take a really long time to boot the emulator. Depends on their available resources. Initially my regular tests (scheduled to run every 30 mins) which are set to use `NexusLowRes,24,en,portrait` took about 4 mins to boot and run my minimal test suite, but lately it usually takes more than 8 mins to do the same (basically just boot the emulator, as my tests really are minimal). Unfortunately we don’t have control over FBTL, but we’re still searching to see if we can find a service which is at least as reliable as FTBL but can run the tests faster (boot faster). One thing you might want to try is our own Emulator steps **with x86** emulators (available on our Linux/Android stack right now). Not the same as FTBL but significantly faster than the old ARM emulators, and quite stable too. Related blog post: [https://blog.bitrise.io/introducing-avd-manager](https://blog.bitrise.io/introducing-avd-manager) If you have any questions just let us know! 🙂 ---
**Author:** ![kuassivi](https://avatars.discourse-cdn.com/v4/letter/k/a3d4f5/32.png) [@kuassivi](https://discuss.bitrise.io/u/kuassivi)\ **Post date:** [December 12, 2017, 3:54pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/14 "2017-12-12T15:54:16Z")
Hi, I’ve been trying many different configurations with the x86 emulators, but unfortunately all my test are failing now. Getting messages like: `- RuntimeException: Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds.` I applied all techniques I found on my research but I can’t solve it. I don’t even know how or where to pass the test-result and artifact to see them on the build tab. Any help? ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [December 12, 2017, 3:56pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/15 "2017-12-12T15:56:01Z")
Hi @kuassivi , Can you please create an issue report in #issues:build-issues ([http://discuss.bitrise.io/c/issues/build-issues](http://discuss.bitrise.io/c/issues/build-issues)) ? It’d help a lot (especially the log / build URL), so that we can look into it ASAP! 🙂 ---
**Author:** ![kuassivi](https://avatars.discourse-cdn.com/v4/letter/k/a3d4f5/32.png) [@kuassivi](https://discuss.bitrise.io/u/kuassivi)\ **Post date:** [December 12, 2017, 4:02pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/16 "2017-12-12T16:02:40Z")
Sure, I’ll do it in a moment. Thanks. Anyway, about the env variable to use to pass the test-results, Am I right with the assumption that I need to do it manually (with a script), or Is it automatically collected by Bitrise after the Gradle Step somehow? ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [December 12, 2017, 4:05pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/17 "2017-12-12T16:05:22Z")
> [@kuassivi](#): > > about the env variable to use to pass the test-results, Am I right with the assumption that I need to do it manually (with a script), or Is it automatically collected by Bitrise after the Gradle Step somehow? Depends, if you’re referring to output files of a tool (other than the base / standard ones, like APK in case of Gradle Runner) you might have to specify where the file(s) are located or move them into the `$BITRISE_DEPLOY_DIR`. Related docs: [Build artifacts online - Bitrise Docs](http://devcenter.bitrise.io/tips-and-tricks/attach-any-file-to-build/) ---
**Author:** ![kuassivi](https://avatars.discourse-cdn.com/v4/letter/k/a3d4f5/32.png) [@kuassivi](https://discuss.bitrise.io/u/kuassivi)\ **Post date:** [December 12, 2017, 4:13pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/18 "2017-12-12T16:13:03Z")
I refer to the **test-result** files/folder generated after the `connectedAndroidTest` task. Do I need to move them manually to the `$BITRISE_DEPLOY_DIR`? ---
**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\ **Post date:** [December 12, 2017, 4:16pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/19 "2017-12-12T16:16:06Z")
Yes, you most likely have to. In general I’d suggest you to try the CLI locally on your Mac/Linux ([https://www.bitrise.io/cli](https://www.bitrise.io/cli) / [How to experiment with Bitrise configs locally, on your Mac/Linux](http://discuss.bitrise.io/t/how-to-experiment-with-bitrise-configs-locally-on-your-mac-linux/1751)), that way you can check what’s moved automatically into the `$BITRISE_DEPLOY_DIR` and what isn’t. The build log in general should also include the files which are moved into the deploy dir; e.g. the Gradle Runner prints every file at the bottom of it’s “step log box” which are moved into the deploy dir. If you’d have any questions just let us know! 🙂 ---
**Author:** ![kuassivi](https://avatars.discourse-cdn.com/v4/letter/k/a3d4f5/32.png) [@kuassivi](https://discuss.bitrise.io/u/kuassivi)\ **Post date:** [December 14, 2017, 2:48pm UTC](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496/20 "2017-12-14T14:48:04Z")
Hi @viktorbenei, all the links you provided were very helpful, thanks. It’s working properly now, I don’t see the `window-focus=false` issue anymore. And it’s indeed saving build time, and probably more with a proper cache! Only to remark that the Deploy Step doesn’t work from the local CLI. [Next page](https://discuss.bitrise.io/t/how-to-run-android-ui-tests-on-virtual-devices/2496.md?page=2)