Real device testing on iOS failing with INVALID(TEST_NOT_APP_HOSTED)

eggheadgames · October 24, 2018, 6:28am

Hi, I’m wondering what I might be doing wrong to get the following output in https://app.bitrise.io/build/f5efb7484a884d76 :

+------------------------------------------------------------------------------+
| (7) virtual-device-testing-for-ios@0.9.2                                     |
+------------------------------------------------------------------------------+
| id: virtual-device-testing-for-ios                                           |
| version: 0.9.2                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: go                                                                  |
| time: 2018-10-23T23:19:51-07:00                                              |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[23:19:51]  * [OK] Step dependency (go) installed, available. 
Configs:
- ZipPath: /Users/vagrant/deploy/testbundle.zip
- TestTimeout: 900
- TestDevices:
---
Model     OS version   Locale     Orientation   
iphone7   12.0         portrait   en            
---
Upload IPAs
=> .xctestrun uploaded
Start test
=> Test started
Waiting for test results
- Validating
Failed to get test status, error: Failed to get test status: INVALID(TEST_NOT_APP_HOSTED)
|                                                                              |
+---+---------------------------------------------------------------+----------+
| x | virtual-device-testing-for-ios@0.9.2 (exit code: 1)           | 51 sec   |
+---+---------------------------------------------------------------+----------+
| Issue tracker: ...itrise-steplib/steps-virtual-device-testing-for-ios/issues |
| Source: ...//github.com/bitrise-steplib/steps-virtual-device-testing-for-ios |
+---+---------------------------------------------------------------+----------+

To get to this point, I:

created a fresh Bitrise App (for my existing app)
kept everything as plain as possible, though I switched to Xcode 10.0
enabled Device Testing under the app’s Settings tab
uploaded certificates / provisioning
added the two steps as documented in https://blog.bitrise.io/introducing-solid-and-snappy-real-device-testing-for-ios-with-firebase-beta (and subsequent pages)
removed the (default) iOS testing step

Any insights are very welcome! We’ve been having a lot of reliability problems with the simulator testing and it would be super helpful if we can get “real devices” working!

Thanks!

BirmacherAkos · October 24, 2018, 1:09pm

Hi @eggheadgames !

I have found this:

github.com

googleapis/google-api-go-client/blob/master/testing/v1/testing-gen.go#L2521

    
      
          type TestEnvironmentCatalog struct {
          	// AndroidDeviceCatalog: Supported Android devices.
          	AndroidDeviceCatalog *AndroidDeviceCatalog `json:"androidDeviceCatalog,omitempty"`
          
          
	// DeviceIpBlockCatalog: The IP blocks used by devices in the test
          	// environment.
          	DeviceIpBlockCatalog *DeviceIpBlockCatalog `json:"deviceIpBlockCatalog,omitempty"`
          
          
	// IosDeviceCatalog: Supported iOS devices.
          	IosDeviceCatalog *IosDeviceCatalog `json:"iosDeviceCatalog,omitempty"`
          
          
	// NetworkConfigurationCatalog: Supported network configurations.
          	NetworkConfigurationCatalog *NetworkConfigurationCatalog `json:"networkConfigurationCatalog,omitempty"`
          
          
	// SoftwareCatalog: The software test environment provided by
          	// TestExecutionService.
          	SoftwareCatalog *ProvidedSoftwareCatalog `json:"softwareCatalog,omitempty"`
          
          
	// ServerResponse contains the HTTP response code and headers from the
          	// server.
          	googleapi.ServerResponse `json:"-"`

Could you please check in your app that your test target’s Host application has been set?

If not (or it’s not an app)

Set it to your app
Push the changes
Try a new build.

.xctestrun with host app:

.xctestrun without host app:

eggheadgames · October 24, 2018, 2:00pm

Thanks!

Yes, it does have that set in Xcode, but I’ll poke around and see if I can confirm that the xctestrun is being set correctly as well. (I expect it will take me a day or two, fyi.)

eggheadgames · October 24, 2018, 2:12pm

In case it makes a difference…

I don’t have a “workspace” – there are no CocoaPods or similar. It’s just a straight “project” with one .xcodeproj file.

eggheadgames · October 25, 2018, 8:53am

I don’t have the checkbox option for “Allow Host Testing APIs” as shown in the first screenshot below

I did some digging and ran this command:

xcodebuild build-for-testing -project foo.xcodeproj -scheme foo -destination "platform=iOS Simulator,name=iPhone 5s" -derivedDataPath "build"

Then I looked at the build/Build/Products/LogicPuzzles_iphonesimulator12.0-x86_64.xctestrun, and, as suggested, it was missing the IsAppHostedTestBundle value.

So, now I need to work out how to get that added. Or, maybe I need to convert my app to use a Workspace instead of a project file. (Though it’s quite happy being a project instead of a Workspace, and I’m not sure what side effects that might trigger.)

Thanks for the help so far. I welcome any thoughts going forward!

system · November 24, 2018, 8:53am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Virtual-device-testing-for-ios intermittent failing Tooling Issues	3	634	December 30, 2021
[BETA] iOS Device Testing Tooling Issues	1	594	September 2, 2020
Android Virtual Device Testing fails Tooling Issues testing	17	1737	August 7, 2019
Virtual Device Testing for Android Timeout Tooling Issues step , android , testing	8	2024	April 19, 2018
[BETA] iOS Device Testing v0.9.12 Changelog	1	508	August 31, 2020

Real device testing on iOS failing with INVALID(TEST_NOT_APP_HOSTED)

Related topics