How to run Android UI tests on virtual devices

How To Android
, , ,
42

Thanks for replying.

Hereā€™s my current settings:

Environment Variables:

coverage=true
coverageFile=/sdcard/coverage.instrumentaltests

Directories to pull:

/sdcard

And I can see the coverage file downloaded, found and uploaded by the codecov script. But in codecov.io the coverage on the relevant classes are still zero. Could you suggest where should I start investigating?

1 Like
43

If youā€™re using android test orchestrator, thereā€™s a known bug that breaks coverage generation. Otherwise it should work.

Make sure your build.gradle is configured correctly for jacoco coverage and using the latest versions.

1 Like
44

After some further investigation I found that codecov canā€™t process the coverage.ec generated by instrument tests. When I turn it into an xml report, codecov can update the coverage correctly.

But instrument tests only seems to generate the .ec file, not the .em, so I canā€™t convert with ā€˜emma reportā€™. Well, anyways, itā€™s beyond bitriseā€™s power. I appreciate your great work.

45

Use jacoco, not emma. Jacoco can output different types of reports.

2 Likes
46

OK, I got a report with java -jar jacococli.jar report, finally! Thank you so much for your help!

3 Likes
47

Thanks for the tutorial.

I have about 5 different modules all containing instrumentation tests. How can I configure this with bitrise? Wouldnā€™t I need to specifiy 5 different apks for each module?

48

@PaulWoitaschek right now, with our built in Virtual Device Testing feature you can only run instrumentation tests for a single APK per build.

Feel free to create a #feature-request, or register the apps as separate apps on bitrise.io (you can register the same repo multiple times), or use Firebase TestLab directly via scripts (How-to run Android tests on Firebase TestLab).

If youā€™d have any questions just let us know! :wink:

49

Hi, I have a project which has multiple modules containing the UI tests. I want to test them all. However, when I install the apk generated by assembleDebugAndroidTest and run virtual device test, I run tests from the app module only. Can you help me with this?

50

Hi @kevalpatel2106,

Does the build generate multiple APKs? If it does, then please see my reply above :point_up_2: How to run Android UI tests on virtual devices

If not, then please create an issue report with the buildā€™s URL, so that we can look into it (http://discuss.bitrise.io/c/issues/build-issues)

51

My tests are failing on virtual device as the location is not enabled. All my tests uses espresso apiā€™s and to over come this I am running UI automator calls to enable the location whenever the location dialog is reported. But no luck. Is there anyway to get rid of this. ? Thanks in advance

52

Hello @ns.mesta!
Could you please write to us on the on-site chat, send a build url and please enable support user on the app?

53

Hi, I am trying to run this step but it is failing and it is not possible to determine the cause, the following is the only output I get:
Uploading app and test files
App path (/bitrise/deploy/GithubBrowserSample-app-debug.apk), is bundle: false
Assets requested: {isBundle:false testApp: Apk:{UploadURL: GcsPath: Filename:GithubBrowserSample-app-debug.apk} Aab:{UploadURL: GcsPath: Filename:} TestApk:{UploadURL: GcsPath: Filename:GithubBrowserSample-app-debug-androidTest.apk} RoboScript:{UploadURL: GcsPath: Filename:} ObbFiles:[]}
Uploading file(/bitrise/deploy/GithubBrowserSample-app-debug.apk) to (gs://bitrise-addon-ftl-prod-bucket/android-tests/1377cbd8db50543a/app.apk)
=> Files uploaded
Starting test
AndroidInstrumentationTest: &{AppApk:0xc0000ccbc0 AppBundle: AppPackageId: OrchestratorOption:DO_NOT_USE_ORCHESTRATOR TestApk:0xc0000ccc00 TestPackageId: TestRunnerClass: TestTargets:[] ForceSendFields:[] NullFields:[]}
=> Test started
Waiting for test results

  • Validating
  • (1/1) running
  • (0/1) running
    => Test finished
    Test results:
    Model API Level Locale Orientation Outcome
    NexusLowRes 24 en portrait failure
54

Hi,
I am running Android Instrumentation Unit tests on FTL and I could not find any option to disable video recording & performance monitoring.
I do not want to initialise the gCloud sdk using a shell script, rather I would like to go ahead with the available plugin. Is there any option which I can use to pass following options for my case, since it is of no use for me while running Instrumented Unit tests.
--no-record-video --no-performance-metrics
As of now, i found that the plugin records both of these by default (as these are enabled by default). An option to pass such flags would help a lot in reducing time for execution of Instrumented Unit tests.

Thanks!
Gaurav Dalal