UI testing in Xcode 7

UI Testing in Xcode 7Dominique Stranz

UI TESTING IN XCODE 7

Example Test: Successful user login

1) tap my account button

2) tap log in button

3) tap & type e-mail

4) tap & type password

5) tap log in button


Example Test: Successful user login

@interfaceAccount:XCTestCase

@property(strong)TestUser*user;

@end

@implementationAccount

-(void)setUp{

[supersetUp];

[selfcreateUser];//wearecreatingtestuseraccounthere,itwillbeusedinalltestcases

self.continueAfterFailure=NO;}

-(void)tearDown{[supertearDown];}

-(void)testLogin_CredentialsAreValid_ShouldLogin{

}

RECORDING…


Recording…Problem with non-ascii characters

Not so useful with localised appsComplicated and not universal queries

Test

abili

ty

Quality of Accessibility Data

source: WWDC, Session 406


UIAccessibility identifiers

Unique identifiers allow recorder to use simpler queries

‣ Locale independent

‣ Ignored by Voice Over


How to set UIAccessibility identifiers?

@protocolUIAccessibilityIdentification<NSObject>

@required

/*

Astringthatidentifiestheuserinterfaceelement.

default==nil

*/

@property(nullable,nonatomic,copy)NSString*accessibilityIdentifierNS_AVAILABLE_IOS(5_0);

@end


We can choose from several options to find element

We should use more precise query, if our accessibility identifier or label isn’t unique:


Why not share our identifiers between app & test targets?

…and finally use user credentials created in setUp method.

EXPLANATION


XCUIElement - proxy for all application UI elements

XCUIElement UIButton

UITextField

UITableViewCell

and others…

‣ currentTitle ‣ currentAttributedTitle

‣ text ‣ placeholder

‣ textLabel.text ‣ detailTextLabel.text

‣ title ‣ label ‣ value ‣ placeholderValue ‣ identifier ‣ selected ‣ frame ‣ …


Interactions

XCUIElement


-(void)typeText:(NSString*)text;

-(void)tap;

-(void)doubleTap;

-(void)twoFingerTap;

-(void)tapWithNumberOfTaps:(NSUInteger)numberOfTapsnumberOfTouches:(NSUInteger)numberOfTouches;

-(void)pressForDuration:(NSTimeInterval)duration;

-(void)pressForDuration:(NSTimeInterval)durationthenDragToElement:(XCUIElement*)otherElement;

-(void)swipeUp;

-(void)swipeDown;

-(void)swipeLeft;

-(void)swipeRight;

-(void)pinchWithScale:(CGFloat)scalevelocity:(CGFloat)velocity;

-(void)rotate:(CGFloat)rotationwithVelocity:(CGFloat)velocity;


Element queries

XCUIElement


/*!Returnsaqueryforalldescendantsoftheelementmatchingthespecifiedtype.*/

-(XCUIElementQuery*)descendantsMatchingType:(XCUIElementType)type;

/*!Returnsaqueryfordirectchildrenoftheelementmatchingthespecifiedtype.*/

-(XCUIElementQuery*)childrenMatchingType:(XCUIElementType)type;


Element queries

XCUIElement


/*!Returnsaqueryforalldescendantsoftheelementmatchingthespecifiedtype.*/

-(XCUIElementQuery*)descendantsMatchingType:(XCUIElementType)type;

@property(readonly,copy)XCUIElementQuery*tables;

@property(readonly,copy)XCUIElementQuery*buttons;

@property(readonly,copy)XCUIElementQuery*staticTexts;

@property(readonly,copy)XCUIElementQuery*textFields;

@property(readonly,copy)XCUIElementQuery*secureTextFields;

@property(readonly,copy)XCUIElementQuery*textViews;


Element queries

XCUIElement


[elementdescendantsMatchingType:XCUIElementTypeButton];

element.buttons


Root element - XCUIApplication

@interfaceXCUIApplication:XCUIElement

-(void)launch;

-(void)terminate;

@property(nonatomic,copy)NSArray<NSString*>*launchArguments;

@property(nonatomic,copy)NSDictionary<NSString*,NSString*>*launchEnvironment;

@end


XCUIElementQuery

@interfaceXCUIElementQuery:NSObject<XCUIElementTypeQueryProvider>

@property(readonly)NSUIntegercount;

-(XCUIElement*)elementBoundByIndex:(NSUInteger)index;

-(XCUIElement*)elementMatchingPredicate:(NSPredicate*)predicate;

-(XCUIElement*)elementMatchingType:(XCUIElementType)elementTypeidentifier:(nullableNSString*)identifier;

-(XCUIElement*)objectForKeyedSubscript:(NSString*)key;

@end


XCUIElementQuery







@end

▸ Evaluates the query

▸ Return the number of matches found


XCUIElementQuery







@end

▸ Return nth element in query results

▸ Example:XCUIElement*firstCell=[app.tables.cellselementBoundByIndex:0];


XCUIElementQuery







@end

▸ Example:NSPredicate*labelPrefixPredicate=[NSPredicatepredicateWithFormat:

@"labelBEGINSWITH%@",prefix];

XCUIElement*label=[app.staticTextselementMatchingPredicate:labelPrefixPredicate];


XCUIElementQuery







@end

▸ Example:XCUIElement*button=[appelementMatchingType:XCUIElementTypeButtonidentifier:@"login"];


XCUIElementQuery







@end▸ Example:XCUIElement*button=app.buttons[@"login"];

DEMO


How to asset test results?

▸ We can use all variants of XCTAssert macro

▸ But more accurate for asynchronous UI are expectations

XCUIElement*loggedInLabel=app.tables.staticTexts[Identifier_Account_LoggedInLabel];NSPredicate*exists=[NSPredicatepredicateWithFormat:@"exists==1"];

[selfexpectationForPredicate:existsevaluatedWithObject:loggedInLabelhandler:nil];

[selfwaitForExpectationsWithTimeout:15handler:nil];


Performance test

-(void)testLogin_CredentialsAreValid_ShouldLogin{[selfmeasureBlock:^{

//UITestcode}];

}

▸ Xcode executes each test 10 times and compare average execution time with baseline (selected from previous results)

▸ Test will fail if new average has increased from baseline by 10% or more, but it will ignore regressions of less than a tenth of a second

TIPS & TRICKS


Handle UI interruptions

▸ Setup interruptions monitor[selfaddUIInterruptionMonitorWithDescription:@"LocationPermission"

handler:^BOOL(XCUIElement*element){XCUIElement*button=alert.buttons[@"Allow"];if(button.exists){[buttontap];returntrue;}returnfalse;}];

▸ Handlers are invoked in reverse order until one of them return true

1 2



▸ By default, XCode 7.2 will try to find & tap elements matching predicate:

userTestingAttributesCONTAINS"cancel-button"

userTestingAttributesCONTAINS"default-button"

1 2



▸ By default, XCode 7.2 will try to find & tap elements matching predicate:

userTestingAttributesCONTAINS"cancel-button"

userTestingAttributesCONTAINS"default-button"

2▸ Do you prefer confirm button as first choice? Add your own monitor:

NSPredicate*predicate=[NSPredicatepredicateWithFormat:@"userTestingAttributesCONTAINS\"default-button\""];

XCUIElement*button=[alert.buttonselementMatchingPredicate:predicate];if(button.exists){[buttontap];returntrue;}


Typing in secure text field doesn’t work

▸ Workaround: Disconnect Hardware Keyboard in Simulator

▸ „Neither element or any descendant has keyboard focus” error occurs when you are trying to type in secure UITextField

▸ Instead, we should check if element is hittable:


Why your test is not waiting for hidden elements?

▸ Hidden elements fulfil exists predicate:[NSPredicatepredicateWithFormat:@"exists==1"]

[NSPredicatepredicateWithFormat:@"hittable==1"]


How to detect that app is in test mode?

XCUIApplication*app=[[XCUIApplicationalloc]init];

[appsetLaunchArguments:@[@"UITESTS"]];

[applaunch];

NSArray*arguments=[[NSProcessInfoprocessInfo]arguments];

if([argumentscontainsObject:@"UITESTS"]){

//Customizeappfortests

}

▸ In test file:

▸ In application:


How to speed up tests?

▸ Disable animations in AppDelegate:NSArray*arguments=[[NSProcessInfoprocessInfo]arguments];

if([argumentscontainsObject:@"UITESTS"]){

[UIViewsetAnimationsEnabled:NO];

}

▸ Waiting for element is one of the most cases in UI tests, why not use convenient helper?


Waiting for elements shortcuts

NSPredicate*exists=[NSPredicatepredicateWithFormat:@"exists==1"];[selfexpectationForPredicate:existsevaluatedWithObject:elementhandler:nil];[selfwaitForExpectationsWithTimeout:15handler:nil];

[selfwaitForElement:element];

[selfwaitForElement:buttonwithTimeout:60];




NSPredicate*exists=[NSPredicatepredicateWithFormat:@"hittable==1"];[selfexpectationForPredicate:existsevaluatedWithObject:elementhandler:nil];[selfwaitForExpectationsWithTimeout:15handler:nil];

[selfwaitForElementHittable:element];

[selfwaitForElementHittable:buttonwithTimeout:60];




NSPredicate*exists=[NSPredicatepredicateWithFormat:@"hittable==1"];[selfexpectationForPredicate:existsevaluatedWithObject:elementhandler:nil];[selfwaitForExpectationsWithTimeout:15handler:nil];

[selfwaitForElementHittable:element];

[selfwaitForElementHittable:buttonwithTimeout:60];

https://github.com/dstranz/XCUITestsAdditionspod"XCUITestsAdditions"


Change device orientation

[[XCUIDevicesharedDevice]setOrientation:UIDeviceOrientationLandscapeRight];

[[XCUIDevicesharedDevice]setOrientation:UIDeviceOrientationLandscapeLeft];

▸ Landscape

[[XCUIDevicesharedDevice]setOrientation:UIDeviceOrientationPortraitFaceUp];

[[XCUIDevicesharedDevice]setOrientation:UIDeviceOrientationPortraitFaceDown];

▸ Portait


Simulates the user pressing a physical button

[[XCUIDevicesharedDevice]pressButton:XCUIDeviceButtonHome];

[[XCUIDevicesharedDevice]pressButton:XCUIDeviceButtonVolumeUp];

[[XCUIDevicesharedDevice]pressButton:XCUIDeviceButtonVolumeDown];

▸ Home button

▸ Volume up

▸ Volume down

▸ Volume up & down doesn’t work on simulator


Test coverage


How to reset simulator state before each test?

It’s not possible to force launching app on clean simulator (rdar://22455111).

What are workarounds?

rdar://22455111


How to reset simulator state before each test?

▸ If you are using CI, you can run xcrun simctl erase all between each test cases

▸ You can clean NSUserDefaults and Keychain in AppDelegate, when app is in UITests mode

▸ …or rollback changes in tearDown method (for example logout user after login test)

THANK YOU

Mobile

UI testing in Xcode 7