JUnit5 / JUnit Jupiter Tests won't fail on Bitrise

I have found a peculiar issue:
JUnit5 == JUnit Jupiter
JUnit4 == JUnit Vintage

Local Testing -> with JUnit Vintage & JUnit Jupiter tests are failing if broken
Bitrise -> only JUnit Vintage test are failing if broken

As it appears JUnit Jupiter test if broken will not fail with either the Android Unit Test or Gradle Unit Test workflow steps on Bitrise. Has anyone else come across this issue before?

Hi @ludwig,

Thanks for reporting this issue! Can you elaborate a bit?

When testing locally, are you running the tests with the Bitrise CLI locally or directly?

Which step versions are you using? Have you tried reverting to previous versions of the steps?

Hi @laszlo.szucs,

We are running tests directly via Bitrise so not locally via the Bitrise CLI.

Our testing step looks uses the latest available ‘android-unit-test’ step from the Bitrise Workflow editor. In this step I have defined my app module ‘mobile’ and the exact variant such as ‘productionBeta’.

I have tried to reverse back to running the gradle task for testing directly but also have not had success with catching faulty tests either.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Hi @ludwig,

Sorry for the late reply.

Can you confirm if this issue still persists? If yes, please supply a build URL where I can check the logs. You will have to enable the Bitrise Support User (https://devcenter.bitrise.io/troubleshooting/enabling-bitrise-support-user/).