Hello,
On a React Native application, I have some Detox tests I want to run in Bitrise CI.
It works perfectly on iOS with simctl like following logs show it:
detox[11958] INFO: [test.js] configuration="ios.sim.staging" cleanup=true reportSpecs=true useCustomLogger=true forceAdbInstall=false DETOX_START_TIMESTAMP=1595425817466 node_modules/.bin/jest --config e2e/config.json '--testNamePattern=^((?!:android:).)*$' --maxWorkers 1 e2e
detox[11959] INFO: [DetoxServer.js] server listening on localhost:51116...
detox[11959] INFO: [AppleSimUtils.js] fr.test launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 35B04E90-04AC-44D8-A15D-05DD460820D7 log stream --level debug --style compact --predicate 'process == "App"'
detox[11959] INFO: Diagnostic is assigned to 35B04E90-04AC-44D8-A15D-05DD460820D7 {"type":"iPhone 11"}
detox[11959] INFO: Diagnostic: displays no warning when camera and microphone access are authorized
detox[11959] INFO: [AppleSimUtils.js] fr.test launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 35B04E90-04AC-44D8-A15D-05DD460820D7 log stream --level debug --style compact --predicate 'process == "App"'
detox[11959] INFO: Diagnostic: displays no warning when camera and microphone access are authorized [OK]
detox[11959] INFO: Diagnostic: displays a warning message + button to app settings when camera access is denied
detox[11959] INFO: [AppleSimUtils.js] fr.test launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 35B04E90-04AC-44D8-A15D-05DD460820D7 log stream --level debug --style compact --predicate 'process == "App"'
detox[11959] INFO: Telehealth Diagnostic: displays a warning message + button to app settings when camera access is denied [OK]
detox[11959] INFO: Telehealth Diagnostic: displays a warning message + button to app settings when microphone access is denied
detox[11959] INFO: [AppleSimUtils.js] fr.test launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 35B04E90-04AC-44D8-A15D-05DD460820D7 log stream --level debug --style compact --predicate 'process == "App"'
detox[11959] INFO: Telehealth Diagnostic: displays a warning message + button to app settings when microphone access is denied [OK]
PASS e2e/telehealthDiagnosticTest.e2e.js (118.71 s)
Telehealth Diagnostic
✓ displays no warning when camera and microphone access are authorized (13040 ms)
✓ displays a warning message + button to app settings when camera access is denied (15035 ms)
✓ displays a warning message + button to app settings when microphone access is denied (16955 ms)
detox[11959] INFO: [DetoxServer.js] server listening on localhost:51352...
detox[11959] INFO: [AppleSimUtils.js] fr.test launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 35B04E90-04AC-44D8-A15D-05DD460820D7 log stream --level debug --style compact --predicate 'process == "App"'
detox[11959] INFO: Launch Application is assigned to 35B04E90-04AC-44D8-A15D-05DD460820D7 {"type":"iPhone 11"}
detox[11959] INFO: Launch Application: displays the splash screen during the loading of the webview
detox[11959] INFO: Launch Application: displays the splash screen during the loading of the webview [OK]
detox[11959] INFO: Launch Application: renders the webview
detox[11959] INFO: Launch Application: renders the webview [OK]
PASS e2e/launchAppTest.e2e.js (29.245 s)
Launch Application
✓ displays the splash screen during the loading of the webview (742 ms)
✓ renders the webview (835 ms)
But when I want to do the same on Android with emulator, only one test file succeed and the second failed because the emulator has disappeared as you can see in logs:
detox[6875] INFO: [test.js] configuration="android.emu.staging" cleanup=true reportSpecs=true readOnlyEmu=false useCustomLogger=true forceAdbInstall=false DETOX_START_TIMESTAMP=1595934596760 node_modules/.bin/jest --config e2e/config.json '--testNamePattern=^((?!:ios:).)*$' --maxWorkers 1 e2e
detox[6876] INFO: [DetoxServer.js] server listening on localhost:49623...
detox[6876] INFO: Diagnostic is assigned to emulator-5554 (Pixel_3_API_27)
detox[6876] INFO: Diagnostic: displays no warning when camera and microphone access are authorized
detox[6876] INFO: Diagnostic: displays no warning when camera and microphone access are authorized [FAIL]
detox[6876] INFO: Diagnostic: displays a warning message + button to app settings when camera access is denied
detox[6876] INFO: Diagnostic: displays a warning message + button to app settings when camera access is denied [SKIPPED]
detox[6876] INFO: Diagnostic: displays a warning message + button to app settings when microphone access is denied
detox[6876] INFO: Diagnostic: displays a warning message + button to app settings when microphone access is denied [SKIPPED]
FAIL e2e/telehealthDiagnosticTest.e2e.js (202.667 s)
Telehealth Diagnostic
✕ displays no warning when camera and microphone access are authorized (13958 ms)
○ skipped displays a warning message + button to app settings when camera access is denied
○ skipped displays a warning message + button to app settings when microphone access is denied
● Telehealth Diagnostic › displays no warning when camera and microphone access are authorized
Test Failed: No views in hierarchy found matching: (with text: is "TESTS" and view has effective visibility=VISIBLE)
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
detox[6876] INFO: [DetoxServer.js] server listening on localhost:49871...
detox[6876] ERROR: [exec.js/EXEC_FAIL, #37] ""/Users/vagrant/Library/Developer/Xamarin/android-sdk-macosx/platform-tools/adb" -s emulator-5554 reverse --remove tcp:49623" failed with code = 1, stdout and stderr:
detox[6876] ERROR: [exec.js/EXEC_FAIL, #37]
detox[6876] ERROR: [exec.js/EXEC_FAIL, #37] adb: error: device 'emulator-5554' not found
detox[6876] INFO: Launch Application is assigned to emulator-12040 (Pixel_3_API_27)
detox[6876] INFO: Launch Application: displays the splash screen during the loading of the webview
detox[6876] INFO: Launch Application: displays the splash screen during the loading of the webview [SKIPPED]
detox[6876] INFO: Launch Application: renders the webview
detox[6876] INFO: Launch Application: renders the webview [SKIPPED]
detox[6875] ERROR: [cli.js] Error: Command failed: node_modules/.bin/jest --config e2e/config.json '--testNamePattern=^((?!:ios:).)*$' --maxWorkers 1 e2e
Sometimes it works but it is rare and it is not possible to execute these tests if the CI is almost always flaky. I tried to run this workflow on default or android linux LTS but it seems the same. The only difference is I never succeed to have the both test file executed on default VM.
My workflow is quite simple.
- Bash script step with command “detox build --configuration android.emu.staging”
- AVD Manager step with these inputs “emulator_id: Pixel_3_API_27”, “api_level: ‘27’”
- Bash script step with command “detox test --configuration android.emu.staging”
On my local environment on macOS, I can launch an AVD emulator and then launching these shell commands on my terminal and it works all the time.
Do you have an idea why you have this strange and flaky behavior on Bitrise?
Best,
Mickael