I refer to the test-result files/folder generated after the connectedAndroidTest
task. Do I need to move them manually to the $BITRISE_DEPLOY_DIR
?
Yes, you most likely have to.
In general Iâd suggest you to try the CLI locally on your Mac/Linux (https://www.bitrise.io/cli / How to experiment with Bitrise configs locally, on your Mac/Linux), that way you can check whatâs moved automatically into the $BITRISE_DEPLOY_DIR
and what isnât.
The build log in general should also include the files which are moved into the deploy dir; e.g. the Gradle Runner prints every file at the bottom of itâs âstep log boxâ which are moved into the deploy dir.
If youâd have any questions just let us know!
Hi @viktorbenei, all the links you provided were very helpful, thanks.
Itâs working properly now, I donât see the window-focus=false
issue anymore. And itâs indeed saving build time, and probably more with a proper cache!
Only to remark that the Deploy Step doesnât work from the local CLI.
Glad to hear, thanks for reporting @kuassivi!
Indeed; thereâs a technical explanation for this, simply that Artifacts can only be uploaded for a specific Build, and when you run the step locally you donât have any Build URL / âbuild contextâ to work with.
If you have any idea for this (where would it upload the artifact from your own local Mac/PC) just let us know!
Right now weâre thinking about allowing the step to run locally (in a ânon CI environmentâ), but in ânon CI modeâ it would simply print out what files it would upload, it would simply skip the actual upload.
Yup, that was what I would have suggested. There is no real need of moving the files locally.
I also realised you have to create a new step for every different path you want to upload. Why not adding a multiline inputbox like other steps?
Regarding the zip option you have to choose for recursive folders, I can suggest to add a capability like the inline zip extractor functionality of google drive which is based on a js lib that can, perhaps, store temporally the extracted files into the âdeployment build directoryâ and show off the files and paths in a prettify way on the Artifacts tab.
I hope to get some free time to contribute with some steps .
Definitely great ideas, thank you @kuassivi!
Iâd suggest you to create a Feature Requests at least for the zip extractor, actually we already have a related one, might worth to just add it as a comment there for better visibility / tracking: Artifacts: Make it possible to view HTML files with relative links (CSS etc. refs) - although zip extractor alone might not solve that issue directlyâŚ
If youâd have any questions just let us know!
Hi @tamaspapik, itâs working very well, thank you very much!
However, screenshots are missing e.g. https://www.bitrise.io/build/618fca3ce921e377
Are you able to help?
Iâm using version 0.9.9 of the step
Hey @eric-grab!
In the build youâve sent me shall not be any screenshots. Robo test is a test type that makes screenshots automatically, but youâve ran instrumentation test which will create screenshots only if you code your test to do it. Instrumentation tests also has a video of it, what you can check under the Virtual Device Tests tab
Hi @tamaspapik, do you know Spoon and how it can work with this awesome Bitrise step?
I believe the things that need to happen are:
- Replace the Gradle task that runs the tests with
./gradlew spoon
(Iâm using a Gradle plugin; this runs the tests + generate reports) - Copy the reports in â./app/build/spoon-outputs/â to âApps & Artifactsâ
Thanks a bunch
Hey @eric-grab!
Unfortunately Spoon looks like a client side tool, and to use VDT step, you will need to generate the exact test and app apks.
Just checked Spoonâs documentation, and I canât see a way of generating them.
Sorry @tamaspapik, letâs forget Spoon for a moment, Iâll rephrase my question as follows:
- Replace the Gradle task that runs the tests with
./gradlew someTask
- Copy the files in
./app/build/someFolder/
to âApps & Artifactsâ or âVirtual Device Testingâ output directory
Is that possible?
You should be able to replace the gradle task. And also if you have Deploy to Bitrise.io step in your workflow, then files copied to $BITRISE_DEPLOY_DIR
should be deployed to the âApps & Artifactsâ tab.
Please contact us on our on-site chat if you encounter any issue doing these, or please send a detailed issue report in the
http://discuss.bitrise.io/c/issues/build-issues
section.
Sorry @tamaspapik, how exactly can I âreplace the Gradle taskâ?
There is absolutely no option to configure Gradle for the Virtual Device Testing step
Understood on the use of Deploy to Bitrise.io step, thanks a lot
So in your VDT step have the two APK path and Test APK path configured. To change gradle task to generate another flavor APKs, you will need to set âtasksâ input in Gradle Runner
step.
How is that possible @tamaspapik? The virtual device/ emulator only lives within the time the Virtual Device Testing step right?
Iâve already tried having Gradle Runner task right after Virtual Device Testing but I got java.lang.RuntimeException: No device(s) found.
e.g. https://www.bitrise.io/build/e48b280ad5781812
I believe that @eric-grab asked about Gradle task used to perform the tests like connectedCheck
or spoon
NOT about one generating APKs (like assemble
).
In case of Firebase Test Lab devices are not connected to local ADB server. APKs (testing and tested) need to be uploaded and then you (or step) can execute a command which starts the tests: gcloud firebase test android run | Google Cloud CLI Documentation
Hi @koral, are you saying that behind the scene, Virtual Device Testing step actually calls Firebase Test Lab?
Regardless, I believe Iâm very close to achieving my goals, please help me a just a bit more koral and @tamaspapik
Within the life span of Virtual Device Testing step, how can I trigger ./gradlew someTaskIwant
?
Exactly:
The tests run on Firebase Test Lab and we display the results we get from them.
Source:
Current version of Virtual Device Testing step waits until tests are finished by polling every 5 seconds:
It should be possible to split this step into separate âUpload APKs and start tests on Virtual devicesâ and âRetrieve Virtual devices test resultsâ. Between them you can insert whatever step you want.
Thanks @koral
Reading main.go and Firebase Test Lab documentation, itâs unclear to me whether you guys are calling a REST API (not found in their documentation) or Gcloud CLI (doesnât seem to support custom Gradle task).
Canât see myself being able to have both Virtual Device Testing + Spoon working together any time soon.
Will put this off for a while.
Thanks a lot for your help thus far @koral and @tamaspapik
Firebase REST API doesnât support stuff like that, we have our own server to handle requests, so this is why you wonât see our calls in the official documentation Thanks @koral!
So basically VDT uses an âofflineâ solution to test, which means that it prepares/uploads all stuff required for the test, and then runs the given configuration. not like you have a device connected that can be used from adb. I think this is why you wonât be able to use VDT for this case