Allow multiple android VDT step invocations

Feature Requests Steps
1

Continuing the discussion from Error 403 on subsequent Android VDT steps:

It should be possible to run both instrumentation and robo tests in the single build. However, only first Android VDT step invocation may be successful. Subsequent ones fail with error 403.

2

Thanks for the feature request @koral

We started tracking it internally, our Tooling team will get back to you here when they can :slight_smile:

3

Hey @koral :wave:

Thanks for the #feature-request! :wink:

We’ll provide support for this use case once the integration is out of beta.

In the meantime as a workaround you can trigger parallel builds https://devcenter.bitrise.io/builds/triggering-builds/trigger-multiple-workflows/ to run multiple VDT tests :slight_smile:

4

Hi,

is there another workaround for this issue? I have several library modules I’d like to test, so it’s not something I can do in parallel, because there would be many concurrencies.

Sorry for the bump!

5

Unfortunately there is no way around this at the moment, so make sure to vote for the request!

1 Like
6

You can trigger “children” builds one after another from “parent” workflow with waiting for those builds. In such case, no matter how many modules you have it will occupy up to 2 concurrencies.

7

You can trigger “children” builds one after another from “parent” workflow with waiting for those builds. In such case, no matter how many modules you have it will occupy up to 2 concurrencies.

But to do that you need 2 concurrencies and thus an Org Standard plan ($90/mo) or higher when maybe you do not want to pay that plan just because of this limitation of VDT with multi modules.

Many Android apps are now following the multimodule approach and with VDT is not posible to properly tests different modules.

8

With only one concurrency available you can just start children builds but without waiting for them. In such case you may need some additional config eg. when using GitHub: additional steps for setting statuses.

9

With only one concurrency available you can just start children builds but without waiting for them. In such case you may need some additional config eg. when using GitHub: additional steps for setting statuses.

Thanks for the answer @koral , but could you give me more details about that? Because I can not:

  1. Launch multiple workflows with the same trigger, already reported in here. The Bitrise UI doesn’t allow it.
  2. Make this solution work with just one concurrency plan, without having the first workflow waiting forever for the second one to be executed.

I guess you mean starting a workflow just after other one but what happens if the second one fails? The first one, triggered by a GitHub push for instance, could be successful and it would send a green check to GH when that’s not true. Github webhooks work with Bitrise app tokens, not with Bitrise workflow tokens so I can not distinguish them. About statuses, basically the same, Github detects events from the Bitrise app, not the workflows.

So, what’s the solution? Creating as many apps as modules you want to test with Virtual Device Testing? It would be a mess.

Thanks

1 Like
10

To mitigate issue 1. you can start new workflow at the end of current one, of course you need to disable waiting for child workflows.
Ad 2. not sure what exactly you are doing but you must not add Bitrise Wait for Build step.

Regarding GH checks, I meant that you should use: https://www.bitrise.io/integrations/steps/github-status and make statuses sent by it required in your GitHub repo configuration.

11

Any updates or new workarounds for this? I’m trying to run UI tests for multiple variants of our app, and splitting them across multiple running workflows is not ideal.

12

You should be able to leverage Pipelines and Artifact Sharing for this.

See: Configuring a Bitrise Pipeline - Bitrise Docs

13

According to the docs:

Build Pipelines is only available on credit-based accounts.

which we are not, so we unfortunately can’t leverage Pipelines for a solution.

14

+1
@ Heetch we also have this need, to benefit from running multiple variants tests at once, without having to pull again the cache.

15

Hi @vlad.markovic - welcome to the Bitrise forum :wave:

Have you looked into Bitrise’s pipeline feature? It could be a potential solution.

16

Thanks @jeremyp ,
With pipeline, will the build number for the ones ran in parallel be the same?
That is our use-case. We want to produce several app-builds (for [staging,prod]*flavours variant combinations) with the same build number, as we moved to tying the Bitrise build number with the app build version number (version code on Android).

17

The build numbers should indeed be the same :+1:. It’s probably worth creating a small proof of concept pipeline to check that it behaves the way you need it to

1 Like
18

Thank you @jeremyp,
I tried the pipeline and can confirm that all the workflows in all stages in the pipeline share the same pipeline build number :+1:

So for anyone else looking for a solution for ops question:

  • use a pipeline
  • in first stage build your app files (including test app file) and
  • upload them with deploy-to-bitrise-io step
  • specifying app file paths as pipeline_intermediate_files
  • then in the next stage workflow use pull_intermediate_files step
  • now you have your app files for testing in any workflow you pull them, and can ran separate workflows, for each UI test
  • either in the same stage (parallel)
  • or in a separate stage (sequential)
1 Like