Test reports are incorrect

#1

Where the build-logs show that one of our Xcode UI tests failed, this is not reflected in the test reports add-on.

Log:

UITests.Registration
  testCarouselVisibleAfterValidationWithInFlowLogin, XCTAssertTrue failed - Expected to be on view object 'ValidationSuccessView', but identifying element 'ValidationSuccess' does not exist. (Did wait for 10.0 seconds)
  EnterValidationCodeView.swift:25
  testHouseholdNotNow, XCTAssertTrue failed - Expected to be on view object 'HouseholdConsentView', but identifying element 'GrantConsentController' does not exist. (Did wait for 1.0 seconds)
  RegisterCheckAddressView.swift:21
	 Executed 69 tests, with 2 failures (0 unexpected) in 1664.323 (1664.361) seconds

The test reports add-on state that these tests succeeded and did so in less than a second (along with all the other tests). This seems unlikely, because when running them locally it usually takes around half a minute each.

0 Likes