Strategies to Approach Mobile Web Testing

by Raj Subramanian

Agenda

• Introduction

• Testing Approaches

• Testing Responsive Websites

• Lessons Learned

• Future of Mobile Web

Introduction

My Bio

• Tester at Progressive Insurance (STE)

• 7 years experience in testing

• Actively involved in testing community

• NOSQAA

• NOTICE meetup

• Newsletter

• Testing articles

• Blogging

Mobile Ecosystem

Mobile Ecosystem

Mobile Ecosystem

The Context

• Complete redesign of mobile website

• 3 releases

• Made the website responsive

• Testing team size - 2 testers

Device Selection

• Apple

• Android

• Windows

• Blackberry

• Firefox

• Ubuntu

Device Selection - Strategy

• Step 1 - Usage Statistics

• OS versions

• Rendering engines

• Browsers

• Step 2 - Discuss findings and get feedback

• Step 3 - Select devices to test

Testing Approaches

Paired Scripted Testing

• Two sets of eyes. Domain/technical knowledge combination

• Both execute same high level test case and discuss results

Defects

Defects

Paired Exploratory Testing

• 45 - 90 min session

• Charter/Goal

• Noted ideas and scenarios

• Paraphrased/Debriefed

• Discussed observations

• Posted defects

Defects

Defects

Defects

Defects

Defects

Defects

Defects

Mobile Testing - Device

• Wi-Fi / 4G

• Interrupt testing

• Sending text messages

• Calling phone

• Disabling javascript

Mobile Testing - Web

• Back buttoning

• Hard buttons

• Soft buttons

Mobile Testing - Web

• Back buttoning

• Hard buttons

• Soft buttons

Mobile Testing - Web

• Back buttoning

• Hard buttons

• Soft buttons

Mobile Testing - Web

• URL editing

• New pages

• Old pages

• System error pages

Mobile Testing - Web

• URL editing

• New pages

• Old pages

• System error pages

Mobile Testing - Web

• Private browsing (Incognito)

Mobile Testing - Web

• Checking function calls

RST Techniques

• Galumphing

• Doing number of clicks around a button, page text etc.

RST Techniques

• Galumphing

• Doing number of clicks around a button, page text etc.

RST Techniques

• Blink test

• Used to compare two very similar pages and look for differences

• Testing to see visual patterns

Tools

• Browser tools

• Used Chrome/IE (F12) browser tools

• Inspect Javascript, HTML, CSS errors

Tools

• Browser tools

• Used Chrome/IE (F12) browser tools

• Inspect Javascript, HTML, CSS errors

Tools

• MITE

• Tool from Keynote

• Emulator - over 2200 mobile devices

• Useful for smoke testing

Tools

• MITE

• Tool from Keynote

• Emulator - over 2200 mobile devices

• Useful for smoke testing

Tools

• Fiddler

• Captures HTTP traffic

• “Login button” scenario

Tools

• Fiddler

• Captures HTTP traffic

• “Login button” scenario

Tools

• Fiddler

• Captures HTTP traffic

• “Login button” scenario

Testing Responsive Websites

What is “Responsive”?

What is “Responsive”

Free Tools

• screenfly - http://quirktools.com/screenfly/

• responsivepx - http://responsivepx.com/

• resizeMyBrowser - http://resizemybrowser.com/

• Responsinator - http://www.responsinator.com/

• Matt Kersley’s tool - http://mattkersley.com/responsive/

• Chrome plugin - Window Resizer

My Tools

• Break point test utility

• Media query test utility

Lessons Learned

From Testing

• Proof of concepts

• Pairing

• Prior investigation

• Continuous research

From Tools

• Sikuli

• Adobe Edge Inspect

• Test Studio

• SeeTest

• Meux

• DeviceAnywhere

• perfecto mobile

For more info - www.rajsubra.com

Future of Mobile Web

Future of Mobile Web

• Testing tools still evolving

• New OS, Plaform - Firefox, Fire 3.0, Android L

• Programming languages - Swift, Go

Future of Mobile Web

• Majority of the websites would be “Responsive”

• Mobile is moving in relation to customers

• Integration of mobile web with watch, cars and other devices

Summary

• Introduction

• Testing Approaches

• Testing Responsive Websites

• Lessons Learned

• Future of Mobile Web

Contact Details e-mail: raj@rajsubra.com website: www.rajsubra.com twitter: @epsilon11