Upload
sundaresan-krishnaswami
View
204
Download
0
Embed Size (px)
DESCRIPTION
This is a draft presentation we had created for a speaker submission on mobile application performance testing. We couldn't make it to the presentation, however we thought it would be useful for Android Mobile Application Testers or Developers alike.
Citation preview
Cohesive approach to Android Application Performance testing
AuthorsPankaj Saini
Sundaresan Krishnaswami
The power of Android Apps
Typical Performance Issues
Abuse of power in a shared world
Battery usage- Services running in the background
- Poll for emails instead of Push- Too many notifications (Like too many Ads)
- Request and lock a service, only never to release- Location- Wakelocks
- Your app running in the foreground- Automatically download in the background without
user intervention- Contd…
Memory Usage–App too slow to use–Crashing with Out of memory exceptions
CPU Usage-Full wake-locks instead of partial-Apps do not release wake-locks-Idle CPU usage
Why bother about performance?
• A Smartphone is Smart only till its battery lasts.
• Numerous apps compete within the limited resources like memory and CPU.
• Optimal use of resources along with smooth end user experience.
Performance Testing Today
• The proliferation of apps, combination of devices, networks, endpoint servers, inter-app communication and user behavior has made performance testing extremely complex.
• What is the best approach?
The Cohesive Approach
• Understand App Usecase• Glean through Design Document• Come up with Techniques• Identify the tools• Put them together• Fit them into your QA cycle
Techniques
• Battery Usage• Network Usage • Server Latch Time• Log analysis• GPS Usage• Wake-locks• Memory leaks• CPU Usage
Usecase vs Techniques Matrix
CricinfoCricinfo Ola CabsOla Cabs FlipkartFlipkart Flipkart E-readerFlipkart E-reader TouchdownTouchdown
Battery UsageBattery Usage
Network UsageNetwork Usage
Server Latch TimeServer Latch Time
Log AnalysisLog Analysis
Wake-locks Wake-locks
Memory LeaksMemory Leaks
GPS UsageGPS Usage
CPU usageCPU usage
Tools – Battery Usage
Recommended tools –
1.Little Eye
2.Rational Test workbench
Poor man’s tool –
1.Power usage under Android Settings
2.eStar battery monitor
Battery Usage - What to test?
• Perform a scenario• Allow the application to idle• Keep a time limit based on application
usecase– Ex: On an email app, allow the device to idle for
an hour
• Observe the battery usage
Tools – Network Usage
Recommended tool – Fiddler
What to test and how?-Setup device to route network calls via fiddler
-Contd…
• Observe the network calls made• Ask questions like– Is the call relevant?– Does the app make too many calls within a
timeframe?– Does the app make networks calls in idle
scenarios?– Should a scenario run on WiFi instead of Data
network for optimum battery usage?
Tools – Server Latch Time• Observe the ‘Server latch time’ using Fiddler
• Ask questions like– Is the device connected to the server for a longer
period– Can the server performance be improved
Tools – Log Analysis
Recommended Tools – ADB Monitor
Why?
-Repeated logs consume memory
-Does the application wake up on every broadcast and log it too?
-Example scenario above
-Collect logs and analyze with Notepad ++
Tools – GPS Usage
• Look for GPS battery usage under Settings -> Location
Tools – Wake Locks
What is a Wake Lock?-Background services may require CPU or Screen to be on to complete tasks
Who requires it?
- Mostly PIM applications
How does it impact performance?
-Too many wake locks-Wake locks are not released post task completion
Recommended Tool to monitor Wake locks-Wakelock detector-Little Eye (discontinued)
What to test?-Observe the number of Wake locks in a time period-Question if the behavior is valid
Tools – Memory Leaks
Comprehensive Slideshow on Memory leaks by Zachary D. Blair http://www.slideshare.net/zblair/identifying-memory-leaks-in-android-applications
Recommended Tools – ADB Monitor, Eclipse Memory Analyzer tool
Also look for rogue processes under ‘Process Stats’ available under Developer Options
Easy Memory Leaks Testing*1) Install the app on the phone.2) Note lower memory usage of the app in DDMS.3) Run the application. Go through all possible screens.4) Note the higher memory usage.6) Using the Back key (not the Home key), close the app.7) Press "Cause GC" as many times as necessary in the DDMS tool to bring the memory usage down. 8) The memory usage should stabilize at a lower value than at step (4).
*Credit: Marius Marinescu, Senior Software Developer, IBM
Tools – CPU Usage
Recommended Tools –
1.Trepn Profiler - Snapdragon
2.Android ADB Monitor
What to test?
-Does the app show up in the stack trace once you tap the back button
-Is the usage too high compared to an app of similar nature
Summary
• Understand App Usecase• Glean through Design Document• Come up with Techniques• Identify the tools• Put them together• Fit them into your QA cycle