Hi,
After upgrading to stack to osx-xcode-15.0.x
Our projects unit test time increases 5x (~60 mins) compared to Xcode 14 stack (~15 min)
We are unable to reproduce this on our mac (M1 Pro), unit tests runs smoothly on our side with same Xcode version
Haven’t been able to create a project to reproduce this issue yet, but my friend who is also using Bitrise Xcode 15 stack is also having this issue.
Any thoughts? Thank you!
@JohnnyTseng that looks odd. Can you provide two build urls to compare with support access enabled please?
Same here, my tests never finish now. Im running unit + ui tests. Builds are timing out.
Some comparison:
Xcode 15:
testAddingItemsToTheBagFromWishlistCarouselWhenSizeIsNotSelected (71.753 seconds)
testAddingItemsToTheBagFromWishlistCarouselWhenSizeIsSelected (223.265 seconds)
testAddingItemToTheBagFromPDP (0.268 seconds)
testAddPromoCodeIsShownAtTheTopWhenFeatureFlagIsOn (326.061 seconds)
testAddPromoCodeIsShownInTheListWhenFeatureFlagIsOff (13.230 seconds)
testApplyingWrongPromoCode (152.068 seconds)
testBagShowsCorrectPricingInformation (0.286 seconds)
testDeliveryInformationCanBeDisplayed (71.072 seconds)
testDonateToCharityAddOnDoesNotAppearWhenFeatureFlagDisabled (87.873 seconds)
Xcode 14.3:
testAddingItemsToTheBagFromWishlistCarouselWhenSizeIsNotSelected (5.816 seconds)
testAddingItemsToTheBagFromWishlistCarouselWhenSizeIsSelected (5.779 seconds)
testAddingItemToTheBagFromPDP (0.061 seconds)
testAddPromoCodeIsShownAtTheTopWhenFeatureFlagIsOn (4.411 seconds)
testAddPromoCodeIsShownInTheListWhenFeatureFlagIsOff (4.682 seconds)
testApplyingWrongPromoCode (7.832 seconds)
testBagShowsCorrectPricingInformation (0.039 seconds)
testDeliveryInformationCanBeDisplayed (4.598 seconds)
testDonateToCharityAddOnDoesNotAppearWhenFeatureFlagDisabled (4.063 seconds)
Using:
xcode15
Computer Name: vm-osx-ventura-15-g2-m1.4core-d050cfd2-80c7-41c5-a25a-4c394782ef94
Locally its fine
Only a bitrise employee would be able to view those.
I’m from Bitrise actually. For some reasons when I click or hover on copy the link url I can only see Bitrise - Mobile Continuous Integration and Delivery - iOS & Android Build Automation
Maybe it’s because discuss forum obscures Bitrise links. Please create a support request https://support.bitrise.io/hc/en-us/requests/new and pass us the ticket number.
We are also experiencing increased build times when updating to Xcode 15
Here is a build log with regular build time ~10 mins (link)
And here is a build log with Xcode 15
Best,
Killian
@silvercast.nguyen I’ve created ticket #42995
I also updated the link formatting above so the forum doesnt obscure it.
Chiming in as another data point. We’re having the same issue with a 2x increase of test runtimes from Xcode 14.3 to Xcode 15.0.
Support ticket: https://support.bitrise.io/hc/en-us/requests/43085
Xcode 14.3 build: Bitrise - Mobile Continuous Integration and Delivery - iOS & Android Build Automation
Xcode 15.0 build: Bitrise - Mobile Continuous Integration and Delivery - iOS & Android Build Automation
I have found that changing the Preferred Capture Format on your test plan from video to screenshot make it faster. Still not as quick but I managed to get a full run of ci without timing out.
Our build+test times have increased from 4-5 minutes to 7-12 minutes. One thing I noticed was that the default “-destination” device in Bitrise was “iPhone 8 Plus”, which doesn’t work with iOS17 builds, so I had to change it. I arbitrarily decided on “iPhone 14”, and I’m wondering if that is part of the reason for increased build times?
I expect this problem will be fixed in Xcode 15.0.1
Fixed: Executing Unit/UI tests from Xcode on the iOS Simulator takes an extended time to launch on first run of the suite. (115187363) (110330776) (FB12237092)