Cucumber tests not running with VDT

testing
android
#1

Hello. I’m running integration tests with VDT. I’m using Bitrise to trigger the tests. I build the two APKs and then I use the VDT step to send them to Firebase. When I install the APKs on my device I see all the tests running. On Firebase the Gherkin tests (Cucumber) do not run. I checked and the Runner in the APK is set to MultiDexCucumberJUnitRunner: after manually installing the two APKs that I’m sending to Firebase using Bitrise, running adb shell pm list instrumentation I see the line:
instrumentation:my.package.name.test/.MultiDexCucumberJUnitRunner (target=my.package.name).

Please notice that I’m also setting the Runner in the VDT step.

Is there a way for me to investigate why the tests are not running?

This is the build https://app.bitrise.io/build/36363d2bfc51ba48

Here the definition of the Runner that I’m using:


Thank you

0 Likes

#2

I managed to partially fix the problem by disabling the Orchestrator in the VDT step.

Are you aware of other people successfully running Cucumber tests with the Orchestrator enabled?

0 Likes

#3

I don’t know whether you are still monitoring this message. Anyway I’ll add another comment.
Can you please check these two builds?


You can notice that I haven’t made any change but the first time I have got an error and the second time a success.
The first time bitrise shows me that it did not manage to get the test results but when I open the tab for the device test I can see the result and it is a success.

This is going to be a problem if we will get this issue too often. Is there a way to avoid it?

Thank you

0 Likes

#4

Hello there!

really sorry for the late reply, we are a little bit overwhelmed by support tickets.

the first build of the last two you sent, shows a server side error. I don’t think you need to worry much about.
I see the second one was successful that is great. So now I am a little confused, this second build is now running the cucumber tests?
if yes that is great I will need to document the fix,
also we did not have any issues related to this kind of testing unfortunately so my experience with it is limited :cry:

also I will ask our tooling team about the orchestrator, I will definitely get back to you when I have a proper answer!

0 Likes

#5

Thank you. It would be really good to know, at least, whether someone has managed to make the cucumber-runner working with the orchestrator enabled. I tried unsuccessfully.
We are starting now running integration tests in the on_push workflow, I’ll let you know if the issue I saw in the first build happens too often.

0 Likes

#6

Hello. I managed to make it working. I needed to switch to androidx and use the new version of cucumber: 4.2.5
You can see the successful build here:

Thank you

1 Like

closed #7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

0 Likes