Android: Virtual Device Testing on multiple modules

Feature Requests Steps
1

I have instrumentation tests in about 5 different gradle modules. Each should be run with different build flavors.

The current setup just allows instrumentation tests for (simple?) projects which rely on a single module.

It would be great if this structure would be supported.

2

Thanks for the #feature-request @PaulWoitaschek!

Possible workarounds in the meantime (if you’d have to run instrumentation tests on Firebase TestLab for multiple separate APKs/modules):

If you’d have any questions just let us know! :wink:

1 Like
3

Another work around is to use parallel flows: https://blog.bitrise.io/start-multiple-builds-with-the-same-trigger

1 Like
4

Hi @viktorbenei, do you have a solution in the workings for this?

I’d like to run UI tests (Virtual Device Tests) for two different build flavors in the same module. However, this also produces two different APK files.

Unless your account allows for build parallelization, I guess the only option still is to create a separate App for each build flavor that needs Virtual Device Testing?

Cheers,
Jeppe

5

Hy!

there is no need for 2 separate apps, use 2 separate workflows with workflow specific env vars to specify which one you would like to run.

6

Hi @fehersanyi-bitrise!

This works if we run the workflow after the other, or only if we have parallel workflows?

Cheers

7

Hyđź‘‹

only works if they are run after another, because on the same app you can’t run 2 firebase tests parallel unfortunately.

8

Hi again @fehersanyi-bitrise.

I tried to do that but unfortunately I’m still getting the same error (Failed to start test: 403, error: {“error”:“Build already exists”}).

Maybe I misunderstood something or I’m doing something wrong.

This is my bitrise.yml:

format_version: '7'
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
project_type: android
trigger_map:
- push_branch: "*"
  workflow: test-app-module
- pull_request_source_branch: "*"
  workflow: test-app-module
workflows:
  test-app-module:
    steps:
    - activate-ssh-key@4.0.3:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone@4.0.14: {}
    - script@1.1.5:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            gem uninstall bundler --force
            gem install bundler --force
        title: Update Bundler
    - cache-pull@2.0.1: {}
    - install-missing-android-tools@2.3.5:
        inputs:
        - gradlew_path: "$PROJECT_LOCATION/gradlew"
    - fastlane@2.4.0:
        inputs:
        - lane: unit_tests
        title: Run Unit Tests
    - fastlane@2.4.0:
        title: Assemble
        inputs:
        - lane: assemble
    - virtual-device-testing-for-android@1.0.5:
        inputs:
        - test_type: instrumentation
        - apk_path: "./app/build/outputs/apk/debug/app-debug.apk"
        - test_apk_path: "./app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk"
        - download_test_results: 'true'
        - test_devices: Pixel2,28,en,portrait
        title: Run App Instrumentation Tests
    - cache-push@2.2.0: {}
    after_run:
    - test-data-module
  test-data-module:
    steps:
    - activate-ssh-key@4.0.3:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone@4.0.14: {}
    - script@1.1.5:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            gem uninstall bundler --force
            gem install bundler --force
        title: Update Bundler
    - cache-pull@2.0.1: {}
    - install-missing-android-tools@2.3.5:
        inputs:
        - gradlew_path: "$PROJECT_LOCATION/gradlew"
    - fastlane@2.4.0:
        title: Assemble
        inputs:
        - lane: assemble
    - virtual-device-testing-for-android@1.0.5:
        inputs:
        - test_type: instrumentation
        - apk_path: "./app/build/outputs/apk/debug/app-debug.apk"
        - test_apk_path: "./data/build/outputs/apk/androidTest/debug/data-debug-androidTest.apk"
        - download_test_results: 'true'
        - test_devices: Pixel2,28,en,portrait
        title: Run Data Instrumentation Tests
    - cache-push@2.2.0: {}
    after_run: []
app:
  envs:
  - opts:
      is_expand: false
    PROJECT_LOCATION: "."
  - opts:
      is_expand: false
    MODULE: app
  - opts:
      is_expand: false
    VARIANT: ''
9

I encountered the same situation, and it seems that bitrise doesn’t support multiple virtual device testing in one workflow (see: https://devcenter.bitrise.io/testing/device-testing-for-android/ ).

I fixed this in our project by adding separate workflows for each module that i want to test.

2 Likes
11

@tdekoning How do you trigger all of the workflows with just one trigger?
Bitrise does not allow to add multiple workflows to one trigger-event.

The only solution I see so far is to add the project multiple times to Bitrise, which is really uncomfortable if you need to do this 6 times (3 modules, 2 flavors = 6 test APKs).

Update
Found the solution.

  • Chaining workflows does not work (same error as if you have multiple Virtual Device Testing steps in one workflow: “Build already exists”).
  • But you can start the next workflow with the build-router-start step