MOBILE TEST PLAN OUTLINE · 2018-01-18 · A s you may have already guessed, mobile is here to stay. Not only is it here to stay, it will become more and more prevalent in our daily

MOBILE TESTPLAN OUTLINE

A s you may have already guessed, mobile is here to stay.

Not only is it here to stay, it will become more and more

prevalent in our daily lives as the Internet of Things (IoT) and

Wearables interface with mobile to give us access to and control

more than we could ever imagine. Most IoT and wearable

devices cannot stand alone, but must connect to a mobile

smartphone, and therefore, the quality of mobile applications

becomes even more important.

Because of this and due to popular demand, we've updated our

mobile test plan, now in Version 4 as an outline because the

intention is to provide a �exible guideline for you to pick and

choose those sections or components that apply most to your

mobile application and context.

In Section 1...

INSTALLATION

TEST CATEGORY TEST AREA GENERIC USE/TEST CASES

Install app via link (i.e. iTunes) CRITICAL1. App can be conveyed from hyperlink.

2. App can be successfully installed and no major software components are missing.

MEDIUM

3. Same app or old version can be detected if app exists.

LOW

4. Splash screen design is desired.

We've depicted test categories in a mobile application:

• COLUMN 1 depicting the general test category.

• COLUMN 2 represents a testing area within the test category.

• COLUMN 3 shows general use cases that you can use as a foundation for more detailed use cases,

We've listed out checkpoints for all apps preloaded on mobile:

• COLUMN 1 shows the functional breakdown for preloaded apps or utilities and functions on a mobile phone.

• COLUMN 2 has an explanation or description as necessary.

In Section 2...

11XBOSOFT.COM CONTACT US AT [email protected]

SECTION ONE


INSTALLATION

EXPLORATORYTESTING

Install app via download internet link (i.e. Web app store)

CRITICAL1. App can be conveyed from hyperlink.

MEDIUM2. App can be successfully installed and no major software components are missing.

3. Same app or old version can be detected if app exists.

LOW4. Splash screen design is desired.

Launch application in target language

Ensure that application starts up properly on launch

Check that orientation is correcton launch

CRITICAL1. Text are translated to target language instead of incorrectly translated to another language.

2. The target audience can interact with all primary controls with the anticipated input anatomy.

HIGH3. Optional functionality can be interacted with anticipatedinput anatomy.

MEDIUM4. The primary functionality can be interacted with related input anatomy as might be encountered with special needs persons.

LOW5. The primary functionality can be interacted with using alternate input methods such as voice, visual interpretation tools.

CRITICAL1. The app launches in the sequence and manner as expectedby end user.

HIGH2. Performance on legacy hardware is acceptable.

MEDIUM3. Any 3rd party applications that launch this app (i.e. from browser or social networking), can access and launch as expected.

LOW4. Friendly and desired use of splash screen.

CRITICAL1. Launch �ow can go to correct locations.

2. The app functionality works according to design wheninteracting with external applications such as contacts,email, Settings data.


EXPLORATORYTESTING


Check for application stability CRITICAL1. App can run under low battery power.

2. The app can be recovered from a force quit scenario.

HIGH3. App data can be restored to a usable state after power removal, reboot and relaunch.

MEDIUM4. App can run under recharged power.

5. App data can be recovered from a force quit scenario.

6. App can be recovered properly after sleep.

7. Performance is maintained when in low battery status.

HIGH1. User can set if receive the push from the app; if receive, user can receive noti�cation of app.

Download an update LOW1. Update can be downloaded and installed successfully.

2. Original data (i.e. pwd) can be inherited by new app version.

Check graphics appearing in:

a) Splash/Title/Logo/ Loading Screen

b) Main Menu and all its subsidiary menus

c) Help/Instructions Screen(s) d) About screen

e) Application Pause Menu and all its subsidiary menus (if present)

Phone should indicate via noti�cations of updating or other information from the app

LOW1. Using high resolution graphics for retina display.

Install app in di�erent location other than default

CRITICAL1. App can be installed in the speci�ed locations.

Launch and close 5-10 times or more CRITICAL1. App can be launched and closed without error.

2. App works as expected.


EXPLORATORYTESTING


App can accurately usegeolocation informationfrom the phone

CRITICAL1. Accuracy of 3G, 4G, LTE or WiFi to �nd correct location.

MEDIUM2. Location information can be read by app.

Uninstallation CRITICAL1. App can be Uninstalled smoothly.

LOW2. Uninstallation app > search, scan for remnant data -No sensitive data left.

Install when disk is full CRITICAL1. This scenario should not cause data loss.

MEDIUM2. Friendly noti�cation indicates to user 'installation cannotbe executed'.

SPELLING ERRORS The application must be free of spelling errors

CRITICAL1. All major headings, section and other primary text isspelled correctly.

HIGH2. All secondary areas of text such as annotations, etc.spelled correctly.

MEDIUM3. Standard conventions used for abbreviations.

TECHNICALTEXT ERRORS

The text in the application must be clear and readable

CRITICAL1. Translation will not cause end user to misunderstandmeaning of the text.

2. All text can be read by someone with typical 20-20 visionat a distance of 18 inches.

LOW3. Current translation is correct and understandable by the end user. Target text can have improvement suggestions.

A spelling error is de�ned as a strict misspelling of a word (no grammar or punctuation rules will be applied). Missing diacriticals and accents (e.g. acutes, cedillas, umlauts etc.) will not be reported as spelling errors

CRITICAL1. All major verbiage is grammatically correct.

HIGH2. All secondary verbiage is grammatically correct.


SPEED &PERFORMANCE

The application works in the deviceit was targeted for. It is usable on the device. The speed of the applicationis acceptable to the purpose of the application and must not alter the user experience by beinguncontrollable

CRITICAL1. App speed is steady and optimal for user usage.

HIGH2. App can work at low connection speeds or show friendly warning to user for 'low connection speed'.

TECHNICALTEXT ERRORS


The application must be free of technical text display issues such as: Text cut o� / Text overlapping

CRITICAL1. Text displays gracefully without truncation/overlapping.

UNEXPECTEDBEHAVIOR

The application must be able to handle unexpected user behavior, for example erroneous actions and multiple key presses

CRITICAL1. App will pop noti�cation message or voice to indicate user's improper behavior instead of crash.

HIGH2. Error reports sent properly.

MEDIUM3. Prompt for customer rating.

APPLICATIONRESPONSE

The application should never leave the user in a position where the state of the application is unknown or appears to be unresponsive (i.e. may have locked up)

CRITICAL1. Application works gracefully without crash.

CONFLICT/INTERRUPTS

An incoming call/video callwhile using app

CRITICAL1. App is functioning correctly as other apps are trying to interact with it's functionality.

2. Incoming app will not cause data loss or get stuck on current app.

An incoming SMS while using app CRITICAL1. App functionality is working according to design when interacting with external applications such as calling, email, SMS, Printing, etc.

Low battery when using app LOW1. Should maintain performance during low battery state.2. If battery level cannot support app functionality, app will notify user.

• Pause and launch music/movie player

• Pause and launch camera

• Pause (or not pause) and launch another app

1. App is functioning correctly as other apps are trying to interact with it's functionality.2. Incoming app will not cause data loss or get stuck on current app.


CONFLICT/INTERRUPTS


Plug or unplug USB whenusing app

LOW1. USB plugging/unplugging will not disturb application process.2. USB can be plugged or unplugged smoothly when using app.

Minimize the app to status bar and launch it

LOW1. App can be minimized to status bar or launched smoothly.

Switch to another app, then launch this app from status bar again

1. User can switch to another app smoothly.2. App can be resumed (or just relaunched) when it is switched back.

Change USB connectionwhen using app

CRITICAL1. USB connection can be changed among 'Mass storage mode', 'Charge-only mode' or 'camera mode' (depends on mobiletype, NA for iOS).

HIGH2. Picture or portrait etc. info could be recovered frominaccessible status in 'Mass storage mode'.

Force close the app and re-launch it. CRITICAL1. The App can be recovered from a force quit.

HIGH2. App data can be recovered.

MEDIUM3. Sensitive info (i.e. pwd) will be erased when relaunchAPP force quit.

LOW4. Saved data will not be lost due to force close.

Lose network connectionduring usage

CRITICAL1. App will not crash when there is sudden loss of connection.

Close/Shutdown mobilebefore quit app

CRITICAL1. The app can be recovered from a force quit.

HIGH2. App data can be recovered from a force quit.

LOW3. Sensitive info like password will be erased after force quit.

Close/Lock screen and open when using app

CRITICAL1. Automatically close/Lock screen will not cause data loss of current app.

HIGH2. User can Close/Lock and open screen smoothly when using app.


CONFLICT/INTERRUPTS


Lose network connectionwhen using, and returnwithout time out

CRITICAL1. App process will slow down or pause when lose network.

2. App process can be resumed automatically when connection returns without time out.

LOW3. Saved data will not be lost from connection loss.

STORAGE Disk usage CRITICAL1. Disk usage dynamically re�ects the disk storage status.

2. Disk usage should have chart and number to indicatecurrent usage status - 'pictures, videos', 'downloads','miscellaneous �les', 'available space'.

3. Phone can alter/delete related �les (pictures, videos,downloads) by clicking storage data.

Memory usage CRITICAL1. Memory usage dynamically re�ects the mobile memory storage status.

2. Disk usage should have graphical chart and number to indicate current usage status - 'available space', 'total space'.

Run app for long timecontinuously and checkmemory usage/leakage

CRITICAL1. Memory usage will change dynamically alongwith app running.

Use when disk is full CRITICAL1. Disk is full will not cause saved/existing data loss.

LOW2. Friendly warning message will pop up to indicate disk is full.

Save/Load process when in use CRITICAL1. Page can be saved/load smoothly when app is in use.

Lose network connection when using, and return with time out

CRITICAL1. App process will slow down or pause when lose network connection.

2. Friendly warning should pop up to indicate 'networkconnection time out' .

3. App can be resumed by refresh after connection returnwith time out.

LOW4. Saved data will not be lost from connection loss.


STORAGE


Synchronize with computer(if applicable)

CRITICAL1. User can set multi accounts (icloud, Skype, Facebook, etc.)on mobile to synch with computer.

MEDIUM2. Data can be synched between app and online account.

3. Data will be updated/deleted in mobile/computerwhen to do so in computer/mobile.

LOW4. Saved data will not be lost if connection is notavailable suddenly.

The application must indicate whether data will be permanently deleted or o�er easy reversal of the deletion

CRITICAL1. The remnant data will be read by app when reinstallapp again after deleting.

HIGH2. User can choose if clean remnant data after deleting.

MEDIUM3. All sensitive data should be deleted when delete app.

Using with WIFI signal MEDIUM1. Mobile can connect with network and then upload/download �le through WIFI signal.

2. App speed will be a�ected by WIFI signal.

Ensure updates do notcorrupt current saved data

1. Not for iOS.Transfer �les between internal storage and expanded storage

Using in signal handoverenvironment (e.g. handoverin 2G & 3G & 4G)

1. Check the app’s behavior as expected duringnetwork handover.

LOW1. Saved data can be read by app again after upgrading.

MEDIUM2. Login persists when upgrading app to a newer version.

SIGNAL ANDNETWORK

PERFORMANCE

Using with 3G signal CRITICAL1. '3G' label is shown on the GPRS signal when using with 3G.

Using with 4G signal CRITICAL1. '4G' label is shown on the GPRS signal when using with 4G.

Using with LTE signal CRITICAL1. 'LTE' label is shown on the GPRS signal when using with LTE.


SIGNAL ANDNETWORK

PERFORMANCE


Verify Bluetooth transfer and connections (if applicable)

CRITICAL1. Mobile can edit 'device name' and set name 'visible/invisible' in Bluetooth settings.

2. Mobile can pair with other Bluetooth device by searching 'device name' or matching 'match code'.

3. Mobile can transfer �les with di�erent device in valid distance.

LOW4. Friendly noti�cation 'can not �nd device', 'wrong match code'.

Using under weak/no signal (such as in high speed car, basement)

CRITICAL1. App performs the primary function end to end when in motion.

HIGH2. All app screens can be navigated while in motion.

MEDIUM3. App performs well instead of crash in elevator.

LOW4. App performs well instead of crash through tunnel.

Use with intermittent signal MEDIUM1. Intermittent signal will not cause data loss of app.

Check GPRS data usage

Using when switching betweenWLAN and mobile network

1. Network related app.

HIGH1. User can enable/disable GPRS data in Settings.

LOW2. User can view how data is used for all applications.

3. User can set limit or warning value for GPRS data usage.

Application that uses network capabilities must be able to handle situations where the network connection isnot allowed

HIGH1. Friendly warning message should pop up when connection is not allowed.

2. App still performs well without connection.

Application that uses network capabilities must be able to handle delays and any lossof connection

HIGH1. App process will slow down or pause when lose network connection.

2. Friendly warning should pop up to indicate 'network connectiontime out' .

3. App can be resumed by refresh after connection return with time out.


SIGNAL ANDNETWORK

PERFORMANCE


Application that uses network capabilities must be able to use the connection correctly and correctly close it after using it

LOW1. App will not use network bandwidth after closing app.

USERINTERFACE

Check all Function keys on mobile for both touch screen and keyboard

CRITICAL1. App can recognize typical user interaction with �ngers.

HIGH2. Tap and key should be sensitive to user's touch.

MEDIUM3. Path of least resistance for common actions. 2-3 touches away.

Check UI consistency CRITICAL1. All usages of app name and other branding are usedconsistently throughout the app.

HIGH2. Minimal number of colors are used in an appealing array.

LOW3. Consistent use of fonts.

Check each input methods: both touch screen and keyboard

CRITICAL1. User can interact with all primary controls with the anticipated input behavior.

HIGH2. Optional functionality is intiated with anticipated input behavior.

MEDIUM3. Primary functionality can be initiated with related input behavior as might be encountered with special needs persons.

LOW4. Primary functionality can be initiated with using alternate input methods such as voice, visual interpretation tools.

5. Phone numbers can be dialed using touch conventions.

From very small to tablet size screens, check UI for incorrect handling of components

LOW1. UI should adapt smoothly without freezing and crash.

When zooming in or out, checkUI for incorrect handling of components

LOW1. UI should adapt smoothly without freezing and crash.


CRITICAL1. User can interact with all primary controls with the anticipated input anatomy.

HIGH2. Optional functionality is interacted with anticipated input anatomy.

MEDIUM3. Primary functionality can be interacted with related input anatomy as might be encountered with special needs persons.

LOW4. Primary functionality can be interacted with using alternate input methods such as voice, visual interpretation tools.

5. Phone numbers can be dialed using touch conventions.

USERINTERFACE


Check each input methods: both touch screen and keyboard

CRITICAL1. App can read link/info from Game Center app.2. App can share/send info to Game Center app.

LOW3. Sensitive data won't be lost by sharing/interaction with Game Center.

Check that Game Center feature functions if available

CRITICAL1. App can read link/info from Facebook app.2. App can share/send info to Facebook app.

LOW3. Sensitive data won't be lost by sharing/interaction with Facebook.

Check that Facebook feature functions if available

CRITICAL1. App can read link/info from Twitter app.2. App can share/send info to Twitter app.

LOW3. Sensitive data won't be lost by sharing/interaction with Twitter.

Check that Twitter feature functions if available

CRITICAL1. App can alter to e-mail app to send e-mails.

Check that the e-mail functionsif available

CRITICAL1. Mobile can recognize di�erent kinds of scrolling gesture.

MEDIUM2. Primary functionality at edge of user's thumb.

Check 4/5 �nger gesturescrolling if available

CRITICAL1. Operate gesture for app is common with common gesture. Like minimize, maximize, etc.

Check swiping consistencyand awareness


HIGH1. App can provide minimum OS Support on di�erent OS.

2. App can support optimal OS performance.

COMPATIBILITY


Run on di�erent OS

CRITICAL1. All text can be read by someone with 20/20 vision at a distance of 18 inches.

2. The text is distinguished from the background image.

3. Items works well.

OTHER Find-ability/Usability for all items

Response time for each critical tasks (performance)

File format support

HIGH1. App functionality is consistent on di�erent devices.

Run on di�erent devices

HIGH1. User can set mobile's time/time zone/date manually.

2. User can synch mobile's time/time zone/date with the network.

Change mobile's time/time zone/date

MEDIUM1. Saved user data will not be lost after OS/�rmware update.

2. Original data can be read by updates.

Check after update OS/�rmware

MEDIUM1. Saved user data will not be lost after app update.

LOW2. Login information other than PW persists when upgrading app to a newer version.

MEDIUM1. App can handle with Large Files or provide friendly warning to user.

LOW2. App can handle many �les in sequence when required.

Update the app version

1. Check if the app still works and there is no con�ict for resources.Run a similar app simultaneously

CRITICAL1. All text can be switched to target language without mistakes.

LOW2. Display style should follow target language style guide.

Switch to di�erent languages


MEDIUM1. App can notify user related 'Personal InformationManagement' (messages, contacts, pictures) is not readable.

LOW2. App works well after failure to read PIM applications(Messages, Contacts, Pictures).

The application must be ableto handle the cases where the connection to the PIMapplications is not allowed

MEDIUM1. App can provide friendly noti�cation message or sound for user's mistaken operation.

OTHER


Error prevention and detection

CRITICAL1. When using password with network, password willbe encrypted.

CRITICAL

CRITICAL

CRITICAL

When connections are used, encryption is used for sending / receiving sensitive data

Cross App Interaction - Outgoing

Exiting

App is testable

LOGIN/LOGOUT Login with correct/wrong password

ITEMS TO CHECK THAT ARE THE SAMEFOR DESKTOP AND MOBILE

MEDIUM1. User is required to input other authenti�cation after correct password if required.

2. User is warned 'invalid password' after input wrong password.

Login on multiple devices MEDIUM1. User login through enter password.

2. User may login through scan QR code.

Login with other ways MEDIUM1. If user links phone number with phone, user can loginvia SMS.

LOW1. For a speci�c device, such as an iPhone X, user can log on through optical scanning.

2. User may login through Voiceprint.



LOGIN/LOGOUT Logout

ITEMS TO CHECK THAT ARE THE SAMEFOR DESKTOP AND MOBILE

LOW1. Password will be erased after logout if user didn’t choose ‘remember password’ option.

2. Password will stay in password �eld after logout if user chooses ‘remember password’ option.

Change password MEDIUM1. User can change password by clicking ‘forget password’ link before login. User will set new password through 3rd party or email application.

2. User can change password in application settings after login. User can set new password by inputting original password.

MEDIUM1. Password will be encrypted by choosing 'invisible' for password display.

2. Password will not be encrypted by choosing 'visible' for password display.

LOGIN/LOGOUT


Password display (Encryption)

CRITICAL1. App can be launched by di�erent browsers.

LOW2. UI and function will be the same for apps from di�erent browsers.

WEBSITE Launch by di�erent browsersif needed

MEDIUM1. App will send security validation if detects high risk behavior.

2. User can get security information through registered email or other means.

Security validation

CRITICAL1. User can create more than one account on mobile device.

HIGH2. Data will not be mixed up with multiple accounts.

LOW3. User can switch accounts smoothly.

Logout and Login withmultiple accounts

TEXT ENTRY

MEDIUM1. Time to load �rst characters or images (look time) and load time (time to load entire page) should be equivalent between mobile and web page, and congruent with user expecations based on amount of content displayed.

Check the look time and load time

MEDIUM1. Context should be consistent between mobile siteand regular website.

Check consistency between mobile site and regular website

CRITICAL1. App can be launched by di�erent browser versions.

LOW2. UI and function will be the same for apps from di�erent browser versions.

Launch di�erent versionof browsers

CRITICAL1. App works gracefully when switching di�erent input methods.

Switch di�erent input methods

HIGH1. Input method can suggest proper words/phrase according to user's input.

Check association for eachinput method

WEBSITE



SECTION TWO

ACCESSORY INTEROPERATION

ALBUM/PHOTO APP

FUNCTIONAL AREA COMMENTS

BATTERY & CHARGING

BROWSER

BLUETOOTH

CALCULATOR

i.e. Bluetooth headset:1. Manage accessories info in Settings (Name, Description, on-o�).2. Accessory performance in poor connection situation (low signal, far distance).3. Interaction with other app (Calling, voice Messaging, FM Radio).4. Compare common controls with other similiar accessaries.

CALL HISTORY

CAMCORDER

CLOCK/ALARM

CONTACTS

DRM

EMAIL

FM RADIO

GPS

INPUT METHOD

MASTER RESET

MESSAGING (SMS/MMS)

MULTIMEDIA

SEARCH

SENSORS

1. <Digital Rights Management> Audio and Video �les.


STANDBY

1. <SIM Tool Kit>.

USB

TELEPHONY

STREAMING/REALTIME PROTOCOLS

STK

SOUND RECORDER

VOICE INPUT & OUTPUT

WALLPAPER

WIDGETS

SETTINGS



1. <Media Transfer Protocol>.

1. App can be installed/uninstalled.

WI-FI

SECURITY

MTP

FILE EXPLORER

VOICE DIALER

LOCALIZATION

INTERACTION

USER SCENARIO

3RD PARTY APP



ABOUT XBOSOFT

As an experienced software testing and QA company, our team of nearly 100 has hundreds of years of cumulative work experience

serving clients, from Fortune 500 to small ISV’s, in North America, Europe, and Asia. Through long term multi-year relationships with

our clients, we’ve developed deep technical expertise in software quality assurance, covering web, mobile, embedded and desktop

apps, and spanning compliance-heavy industries, such as �nance and healthcare.

XBOSoft’s software quality processes are designed to provide con�dence every step of the way. We ensure that your software not

only performs optimally, but keeps your users coming back.

PHONE: +1 703 995 9896 EMAIL: [email protected] XBOSOFT

Documents

MOBILE TEST PLAN OUTLINE · 2018-01-18 · A s you may have already guessed, mobile is here to stay. Not only is it here to stay, it will become more and more prevalent in our daily