# \[Xamarin\]\[iOS\]\[XamarinUITests on iPad Simulator\]

**URL:** <https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559>\
**Category:** Question & Answer\
**Tags:** xamarin\
**Created:** [February 2, 2017, 8:00am UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559 "2017-02-02T08:00:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![GrzegorzTrawinski](https://avatars.discourse-cdn.com/v4/letter/g/50afbb/32.png) [@GrzegorzTrawinski](https://discuss.bitrise.io/u/GrzegorzTrawinski)\
**Post date:** [February 2, 2017, 8:00am UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/1 "2017-02-02T08:00:43Z")

</div>

Hi Gents,

I am struggling with configuring a workflow that will fire Xamarin UI tests for both Android and iOS platform.  
In this topic I would like to describe issue related to iOS.

Assumptions:  
I am using bitrise for about half a year and I have plently of working CI’s.  
I have working Xamarin UI tests that I can fire locally.  
I have configured everything (I hope so) correctly accordingly to: [App Center Test - Visual Studio App Center | Microsoft Learn](https://developer.xamarin.com/guides/testcloud/uitest/quickstarts/)

What I know so far:  
This is how looks the machine: [https://github.com/bitrise-io/bitrise.io/blob/master/system\_reports/osx-xamarin-stable.log](https://github.com/bitrise-io/bitrise.io/blob/master/system_reports/osx-xamarin-stable.log) so I know which simulators can I use and which android emulators can I create and launch.  
Here are some tutorials: [http://blog.bitrise.io/2016/02/29/xamarin-ci-with-bitrise-tutorial-part-1-getting-started.html](http://blog.bitrise.io/2016/02/29/xamarin-ci-with-bitrise-tutorial-part-1-getting-started.html)  
[http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html](http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html) however they are very not accurate, not so informative as I wish it could be, and there is no step by step tutorial actually. But I know I have to use environmental variables in my code.  
Here is source of Xamarin iOS test step: [GitHub - bitrise-steplib/steps-xamarin-ios-test](https://github.com/bitrise-steplib/steps-xamarin-ios-test)  
Here is source of Xamarin Android step: [GitHub - bitrise-steplib/steps-xamarin-android-test](https://github.com/bitrise-steplib/steps-xamarin-android-test)  
…but they have different inputs, so I assume different approaches, and NO DESCRIPTION at all!  
(should I add that I am not familiar with GoLang, and can’t read this sources? 🙂)

First thing which is not clear and intuitive is:

1. Should I have Xamarin Archive step or Xamarin Builder step before running Xamarin iOS Test step?  
I testes with both configuration… and not going into details; still I am not sure.
2. In Xamarin iOS Test I do not provide IPA file, I provide only Xamarin project configuration and Xamarin platform. But is it the configuration of the UI test project itself, or Mobile.iOS project?  
Finally what is strange, I tried various configurations and end up with this:

> /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets: error : No installed provisioning profiles match the installed iOS signing identities.

What the heck? I copied workflow with exactly same configuration and provisioning profile, somehow I manage to build my solution withing Xamarin iOS Test step, but I get such strange error.

If anybody is willing to help I would be grateful. If you need more details I can provide of course.  
If somebody can provide example working workflow - that would be marvelous!

Cheers,  
G

---

<div class="post-metadata">

**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\
**Post date:** [February 2, 2017, 10:06am UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/2 "2017-02-02T10:06:38Z")

</div>

Hi,

Can you please check this #How To guide and let us know if you have any questions?

> [@How to do Calabash UITesting on Bitrise](http://discuss.bitrise.io/t/how-to-do-calabash-uitesting-on-bitrise/361):
>
> This guide introduces automated Calabash UITesting for Xamarin projects on [Bitrise](https://www.bitrise.io). Project setup The sample project, i will use in this guide is our [xamarin-cross-platform sample](https://github.com/bitrise-samples/sample-apps-xamarin-cross-platform). It contains iOS and Android projects, both with Calabash test. The test are placed into the project directories: for ios: REPO\_ROOT/iOS/features for android: REPO\_ROOT/Droid/fetaures and they are default generated/sample tests, read: [Calabash Quickstart for Xamarin.iOS](https://developer.xamarin.com/guides/testcloud/calabash/quickstarts/ios/) and [Creating A Calabash Test for an APK](https://developer.xamarin.com/guides/testcloud/calabash/quickstarts/android-studio/creating-the-test/) gu…

---

<div class="post-metadata">

**Author:** ![GrzegorzTrawinski](https://avatars.discourse-cdn.com/v4/letter/g/50afbb/32.png) [@GrzegorzTrawinski](https://discuss.bitrise.io/u/GrzegorzTrawinski)\
**Post date:** [February 2, 2017, 2:10pm UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/3 "2017-02-02T14:10:23Z")

</div>

This does not work for me 'cause I do not use Calabash, but Xamarin UI Test project instead.

---

<div class="post-metadata">

**Author:** ![godreikrisztian](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/godreikrisztian/32/22_2.png) [@godreikrisztian](https://discuss.bitrise.io/u/godreikrisztian)\
**Post date:** [February 2, 2017, 4:23pm UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/4 "2017-02-02T16:23:01Z")

</div>

Hi @GrzegorzTrawinski,

Regarding to your questions:

> [@GrzegorzTrawinski](#):
>
> Should I have Xamarin Archive step or Xamarin Builder step before running Xamarin iOS Test step?

You do not need any build/archive step, the `Xamarin iOS Test` step manages to build your solution.

> [@GrzegorzTrawinski](#):
>
> In Xamarin iOS Test I do not provide IPA file, I provide only Xamarin project configuration and Xamarin platform. But is it the configuration of the UI test project itself, or Mobile.iOS project?

Since the step manages to build your solution, you do not need to provide .app file path, it is detected by the step, based on the provided configuration and platform.

The configuration and platform are your **solution’s configuration and platform** and select the ones, **which maps to the desired project configuration and platform**.

**The desired project configuration and platform** is the one, which will generate .app file (most of the times this selects platform: **iPhoneSimulator** ) and launches the `Xamarin Test Cloud Agent` (most of the times this selects platform: **Debug** )).

_So if your desired project config is: Debug + iPhoneSimulator, by default your solution will have this configs as well, so you can go with this ones. (Most of the times solution config Debug + Any CPU maps to iOS project’s Debug + iPhoneSimulator config, so this is also a good choice)._

> [@GrzegorzTrawinski](#):
>
> But I know I have to use environmental variables in my code.

That is right, **do not forget to modify your BeforeEachTest method** :

```auto
...
			string appBundlePath = Environment.GetEnvironmentVariable("APP_BUNDLE_PATH");

			if (appBundlePath != null && appBundlePath != "")
			{
				// In case of Bitrise step: steps-xamarin-ios-uitest.
				app = ConfigureApp
					.iOS
					.AppBundle(appBundlePath)
					.StartApp();
			}
			else {
				// In case of Bitrise step: steps-xamarin-testcloud-for-ios or running in the IDE.
				app = ConfigureApp
					.iOS
					.StartApp();
			}
...

```

> [@GrzegorzTrawinski](#):
>
> No installed provisioning profiles match the installed iOS signing identities.

Please take a look on our [code-singing guide](http://devcenter.bitrise.io/ios/code-signing/).

* * *

Please let me know if you need any more informations!

---

<div class="post-metadata">

**Author:** ![GrzegorzTrawinski](https://avatars.discourse-cdn.com/v4/letter/g/50afbb/32.png) [@GrzegorzTrawinski](https://discuss.bitrise.io/u/GrzegorzTrawinski)\
**Post date:** [February 3, 2017, 3:02pm UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/5 "2017-02-03T15:02:30Z")

</div>

Hi again,

I know that I follow two different topics with almost same question, but issues must treated individually,definitely.  
So as I mentioned previously… I created brand new solution for both Android and iOS with Xamarin.Forms + UITests project. Created dummy button and two tests, these are working perfectly locally without issues.

I git push them and created bitrise workflow for testing purpose

> [@godreikrisztian](#):
>
> Please let me know if you need any more informations!

Let me kindly ignore previous message as I experience different issues right now.  
I assume that issues related to provisioning profile is connected to wrong configuration in my prime project that I can fix obviously. Later.

With the new project I am able to build both iOS app and Test dll, but I cannot get working tests. This is the result of the Xamarin iOS Tests:

> Testing (UISample.UITests) against (UISample.iOS)  
> test dll: /Users/vagrant/git/MobileTest/UISample/UITests/bin/Debug/UISample.UITests.dll  
> app: /Users/vagrant/git/MobileTest/UISample/iOS/bin/iPhoneSimulator/Debug/UISample.iOS.app

> Running Xamarin UITest  
> $ “/Library/Frameworks/Mono.framework/Versions/Current/Commands/mono” “/Users/vagrant/bitrise/xamarin/nunit/3.4.1/bin/nunit3-console.exe” “/Users/vagrant/git/MobileTest/UISample/UITests/bin/Debug/UISample.UITests.dll” “–result” “/Users/vagrant/deploy/TestResult.xml”

> NUnit Console Runner 3.4.1  
> Copyright (C) 2016 Charlie Poole

> Runtime Environment  
> OS Version: MacOSX 16.3.0.0  
> CLR Version: 4.0.30319.42000

> Test Files  
> /Users/vagrant/git/MobileTest/UISample/UITests/bin/Debug/UISample.UITests.dll

> =\> UISample.UITests.Tests(iOS).SecondTestToCheckIfMultipleTestsAreWorking  
> Full log file: /var/folders/90/5stft2v13fb\_m\_gv3c8x9nwc0000gn/T/uitest/log-2017-02-03\_06-05-19-589.txt  
> iOS test running Xamarin.UITest version: 2.0.5  
> Skipping IDE integration as important properties are configured. To force IDE integration, add .PreferIdeSettings() to ConfigureApp.  
> Skipping local screenshots. Can be enabled with EnableScreenshots() when configuring app.  
> Sim check: App not installed.  
> 1 - LaunchTestAsync:  
> deviceId: D0602EF4-CCAA-4169-A034-97AEEEB66952  
> 1 - Launching simulator if not already running  
> =\> UISample.UITests.Tests(iOS).WelcomeTextIsDisplayed  
> Full log file: /var/folders/90/5stft2v13fb\_m\_gv3c8x9nwc0000gn/T/uitest/log-2017-02-03\_06-06-35-056.txt  
> iOS test running Xamarin.UITest version: 2.0.5  
> Skipping IDE integration as important properties are configured. To force IDE integration, add .PreferIdeSettings() to ConfigureApp.  
> Skipping local screenshots. Can be enabled with EnableScreenshots() when configuring app.  
> Sim check: App not installed.  
> 7 - LaunchTestAsync:  
> deviceId: D0602EF4-CCAA-4169-A034-97AEEEB66952  
> 7 - Launching simulator if not already running

> Errors and Failures

> 1. SetUp Failed : UISample.UITests.Tests(Android)  
> One or more child tests had errors

> 1. SetUp Failed : UISample.UITests.Tests(iOS)  
> One or more child tests had errors

> Test Run Summary  
> Overall result: Failed  
> Test Count: 4, Passed: 0, Failed: 4, Inconclusive: 0, Skipped: 0  
> Failed Tests - Failures: 0, Errors: 4, Invalid: 0  
> Start time: 2017-02-03 14:05:19Z  
> End time: 2017-02-03 14:06:56Z  
> Duration: 97.572 seconds

> Results (nunit3) saved as /Users/vagrant/deploy/TestResult.xml  
> Test failed, error: exit status 4

Well, I did enabled Calabash agent obviously. I even commented EABLE\_TEST\_CLOUD directive to double check it is always included. It didn’t helped.

In step I configure that I wish to use iPhone 6 Plus + latest iOS (which is 10.2). And I see such info in log:

```
Collecting simulator info...
Simulator (iPhone 6s Plus), id: (EBD5719F-BB37-478E-AEC9-448EC2C03E26), status: Shutdown

```

I can even check that this id is correct accordingly to: https:// [[github.com](http://github.com)] /bitrise-io/bitrise.io/blob/master/system\_reports/osx-xamarin-stable.log

but during the tests it starts to run

> deviceId: D0602EF4-CCAA-4169-A034-97AEEEB66952

which is iPhone SE iOS 10.2. Btw. this one starts on my mac either as default if I do not provide any DeviceIdentifier and set UITests as startup project.  
Although, I didn’t ask for this device in step configuration 🙂.

Additionaly:

> 1. SetUp Failed : UISample.UITests.Tests(Android)  
> One or more child tests had errors

Why he tries to launch Android tests?

Finally, one thing bothers me. Here: [http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html](http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html) you mention that you have to embed IOS\_SIMULATOR\_UDID environment variable usage.

> string deviceUDID = Environment.GetEnvironmentVariable(“IOS\_SIMULATOR\_UDID”);

But I didn’t see the usage of IOS\_SIMULATOR\_UDID neither in Xamarin iOS Test step source code nor somewhere else, actually. So maybe should I set it manually? It didn’t help either…

Gents, any suggestions?

---

<div class="post-metadata">

**Author:** ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)\
**Post date:** [February 4, 2017, 11:12am UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/6 "2017-02-04T11:12:39Z")

</div>

> [@GrzegorzTrawinski](#):
>
> Let me kindly ignore previous message as I experience different issues right now.

We’ll try to answer here, just a friendly reminder: could you please create a new topic/question in the future when you face a new / separate issue? That would make it much easier to follow through the discussion and help you.

Thank you! 🙂

---

<div class="post-metadata">

**Author:** ![GrzegorzTrawinski](https://avatars.discourse-cdn.com/v4/letter/g/50afbb/32.png) [@GrzegorzTrawinski](https://discuss.bitrise.io/u/GrzegorzTrawinski)\
**Post date:** [February 6, 2017, 10:34am UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/7 "2017-02-06T10:34:37Z")

</div>

Well ViktorBenei, I cannot agree with you. It it still all related to the same issue: Xamarin iOS tests, I am just in different point on this long, painfull path to make it working.

I still get the issue with not linked Calabash. And my only guess is - it starts different iOS simulator or it does not do it at all.

> Collecting simulator info…  
> Simulator (iPhone SE), id: (D0602EF4-CCAA-4169-A034-97AEEEB66952), status: Shutdown

This status Shutdown make my worried. Should I start simulator manually somehow? I do not see such step anywhere.

and I am still wondering about this one:

> string deviceUDID = Environment.GetEnvironmentVariable(“IOS\_SIMULATOR\_UDID”);

Where do you set this parameter?

---

<div class="post-metadata">

**Author:** ![godreikrisztian](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/godreikrisztian/32/22_2.png) [@godreikrisztian](https://discuss.bitrise.io/u/godreikrisztian)\
**Post date:** [February 6, 2017, 1:59pm UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/8 "2017-02-06T13:59:37Z")

</div>

Hi @GrzegorzTrawinski, regarding to:

> [@GrzegorzTrawinski](#):
>
> Finally, one thing bothers me. Here: [http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html](http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html) you mention that you have to embed IOS\_SIMULATOR\_UDID environment variable usage.

The previous version does not handle correctly the specified simulator. I [fixed it](https://github.com/bitrise-steplib/steps-xamarin-ios-test/pull/28), the new version ([2.1.1](https://github.com/bitrise-steplib/steps-xamarin-ios-test/releases/tag/2.1.1)) is availabe in the steplib.

I recommend the following test setup to support both testing on bitrise and locally:

```auto
			iOSAppConfigurator iosConfigurator = ConfigureApp.iOS;

			string appBundlePath = Environment.GetEnvironmentVariable("APP_BUNDLE_PATH");
			if (!string.IsNullOrEmpty(appBundlePath))
			{
				iosConfigurator.AppBundle(appBundlePath);
			}

			string simulatorUDID = Environment.GetEnvironmentVariable("IOS_SIMULATOR_UDID");
			if (!string.IsNullOrEmpty(simulatorUDID))
			{
				iosConfigurator.DeviceIdentifier(simulatorUDID);
			}

			return iosConfigurator.StartApp();

```

The step’s new version will export the selected simulator’s UDID into `IOS_SIMULATOR_UDID `.

> [@GrzegorzTrawinski](#):
>
> Why he tries to launch Android tests?

If you are using cross platform Xamarin.UITest both android and ios tests are specified in the same UITest project, you need to tell to the step which test to run, related input: `Test name to run`.

You can figure out the test name based on your IDE’s test pad. In our [sample’s](https://github.com/bitrise-samples/sample-apps-xamarin-cross-platform) case, it looks like:

![](https://us1.discourse-cdn.com/flex016/uploads/bitrise/original/1X/25c840fb75c226cea0b59658b97235dfa1e3238d.png)

and the iOS test name to run is: `Multiplatform.UItest.Tests(iOS)`

---

<div class="post-metadata">

**Author:** ![GrzegorzTrawinski](https://avatars.discourse-cdn.com/v4/letter/g/50afbb/32.png) [@GrzegorzTrawinski](https://discuss.bitrise.io/u/GrzegorzTrawinski)\
**Post date:** [February 7, 2017, 3:52pm UTC](https://discuss.bitrise.io/t/xamarin-ios-xamarinuitests-on-ipad-simulator/559/9 "2017-02-07T15:52:38Z")

</div>

Hi @godreikrisztian,

Finally I did it.

Firstly, after your update proper iPhone Simulator launched + it was correctly used:

> Collecting simulator info…  
> Simulator (iPhone 6s Plus), id: (EBD5719F-BB37-478E-AEC9-448EC2C03E26), status: Shutdown  
> …  
> Sim check: App not installed.  
> 1 - LaunchTestAsync:  
> deviceId: EBD5719F-BB37-478E-AEC9-448EC2C03E26  
> 1 - Launching simulator if not already running  
> 6 - Launching simulator if not already running  
> 6 - InstallApp:  
> pathToBundle: /Users/vagrant/git/MobileTest/UISample/iOS/bin/iPhoneSimulator/Debug/UISample.iOS.app

Secondly, Indeed when I passed iOS test name properly to Test name to run field, e.g. “UISample.UITests.Tests(iOS)” It used only iOS tests.

Finally, here [http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html](http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html) you mentioned that you have to provide bundleId

> ```
> string deviceUDID = Environment.GetEnvironmentVariable("IOS_SIMULATOR_UDID");
> **string bundleID = "your.applications.bundle.id";**
> 
> ```

> ```
> ConfigureApp
> .iOS
> **.InstalledApp(bundleID)**
> .DeviceIdentifier(deviceUDID)
> .StartApp();
> 
> ```

Proposal from previous post was better 🙂 because it worked:

> ```
> string appBundlePath = Environment.GetEnvironmentVariable("APP_BUNDLE_PATH");
> if (!string.IsNullOrEmpty(appBundlePath))
> {
> iosConfigurator.AppBundle(appBundlePath);
> }
> 
> ```

So maybe you should update your article ;), currently is misleading? ( [http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html](http://blog.bitrise.io/2016/03/05/xamarin-ci-with-bitrise-tutorial-part-2-testing.html) )

By the way I keep finger crossed about issues raised in Xamarin Android Test step https:// [[github.com](http://github.com)] /bitrise-steplib/steps-xamarin-android-test/issues/15 → it would be nice to have mirrored functionality here with selecting tests to run.

Don’t you own me a beer, do you?  
Cheers,  
G
