Hello. I’m running integration tests with VDT. I’m using Bitrise to trigger the tests. I build the two APKs and then I use the VDT step to send them to Firebase. When I install the APKs on my device I see all the tests running. On Firebase the Gherkin tests (Cucumber) do not run. I checked and the Runner in the APK is set to MultiDexCucumberJUnitRunner: after manually installing the two APKs that I’m sending to Firebase using Bitrise, running adb shell pm list instrumentation I see the line: instrumentation:my.package.name.test/.MultiDexCucumberJUnitRunner (target=my.package.name).
Please notice that I’m also setting the Runner in the VDT step.
Is there a way for me to investigate why the tests are not running?
I don’t know whether you are still monitoring this message. Anyway I’ll add another comment.
Can you please check these two builds?
You can notice that I haven’t made any change but the first time I have got an error and the second time a success.
The first time bitrise shows me that it did not manage to get the test results but when I open the tab for the device test I can see the result and it is a success.
This is going to be a problem if we will get this issue too often. Is there a way to avoid it?
really sorry for the late reply, we are a little bit overwhelmed by support tickets.
the first build of the last two you sent, shows a server side error. I don’t think you need to worry much about.
I see the second one was successful that is great. So now I am a little confused, this second build is now running the cucumber tests?
if yes that is great I will need to document the fix,
also we did not have any issues related to this kind of testing unfortunately so my experience with it is limited
also I will ask our tooling team about the orchestrator, I will definitely get back to you when I have a proper answer!
Thank you. It would be really good to know, at least, whether someone has managed to make the cucumber-runner working with the orchestrator enabled. I tried unsuccessfully.
We are starting now running integration tests in the on_push workflow, I’ll let you know if the issue I saw in the first build happens too often.