Webinar: Performance Testing for Mobile Applications by RTTS

© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050

Performance Testingfor Mobile Applications

Moderator

Laura PoggiMarketing Manager

Matthew AdcockDivision Manager, Performance & Scalability

Presenter


The Software Quality Experts

1996 Founded

Locations New York | Philadelphia | Atlanta | Phoenix

400 + Projects


What we offer

Extensive knowledge of the leading test tools

Unmatched experts for both strategy and implementation

Innovative services for all project sizes and lengths

1

2

3


Objectives• Review the current mobile load testing ecosystem & challenges.• Provide paths for interim load testing solutions for mobile

applications.• Limit additional software costs related to mobile load testing

requirements.• Enable collaboration amongst peers.

"Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime."


Load Testing in Our ContextHTTP

HTTP

Resource Monitoring

Test Agent

Virtual Users

Test CoordinatorTest Agent

Test Results

Virtual Users

Web Application


What’s Happening?• Mobile usage is taking off. Consumers have high expectations on

usability and performance.• Consumers are using tablets and smart phones; i.e. there are many

platforms to support.• The technology is evolving rapidly; i.e. HTML5 vs. Native Apps.


There are Billions of Installs


What’s the Future of Mobile?The future is complex:

• HTML5• Single codebase running across many platforms using a web browser.• Inherent maintenance benefits & upgrade pathways.• Possibly shorter time to market.

• Native Apps• Multiple codebases each targeting a specific platform.• Monetary incentives for software developers.• Increased brand management.• Sophisticated user interface.


Testing is Always Catching Up


Mobile Load Testing Future?

Business Challenges+ Technical Challenges Bumpy Ride

• Mobile load testing is nascent; i.e. we’re in an early phase.

• There’s a myriad of choices and/or vendors cropping up weekly.

• Technology is changing rapidly.


Business Challenges• Load test tool licensing constraints and/or budget limitations.• Load test tool procurement lengthy and/or cumbersome.• Accelerating time-to-market; i.e. “hurry-up and test”.• A plethora of target devices & operating systems to support.


Technical Challenges• Mobile architecture not supported out the box.• Limited features of load testing tool; i.e. not keeping pace.• Creating scripts may be unorthodox and/or unclear.• Emulating large numbers of concurrent users may be required.• Incorporating network conditions becomes important.


Some Existing Tools

Commercial w/ Traditional Install Hosted/SAAS Open Source


Short-Term Solutions• Scrutinize the mobile architecture; know the pain points and

challenges.• Be creative and use what you have to limit costs related to tool

transition, vendor selection, and/or constraints within your organization.

• Leverage open source technology.• Take advantage of free utilities and web-based services.


Our FocusPrimary Focus

− Creating automated test scripts for mobile applications.Honorable Mentions

− Emulating network conditions of mobile devices.− Scaling up load generation to support increasing consumption of on-demand

services.


Test Automation Challenges• Many load test tools run on desktops NOT mobile devices.• Mobile network traffic occurs on a Wi-Fi or Cellular network.

Desktop Web Application

LAN LAN


Test Automation Solutions1. Spoof mobile device from a desktop web

browser.2. Proxy recording using actual mobile

device.3. Packet captures (i.e. PCAP) using actual

mobile device.

Script Capture


Spoofing Mobile DevicesUse Case: Applicable if the target application runs within a mobile web browser. This can’t be used for native apps.

Strategy• Most web browsers can be modified to appear to be running on a mobile

device.• With the web browser configured to imitate a mobile device, script recording

can occur from the desktop where the load testing tool is installed.


Spoofing Relies on Metadata• HTTP clients must use standardized metadata in the header of a HTTP

request to indicate the client’s OS, browser type and version, along with extensions.

• The “User-Agent” field is the key to this solution.• Many web applications use this metadata to redirect customers to

mobile-specific URLs and/or mobile-specific content.

Desktop Web Application

HTML, CSS, JPEG JSON, XML


User-Agent Metadata

GET http://www.gasbuddy.com/ HTTP/1.1Host: www.gasbuddy.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Language: en-US,en;q=0.5Accept-Encoding: gzip, deflateConnection: keep-alive

Mozilla/[version] ([system and browser information of client]) [browser platform] ([browser platform details]) [extensions]

Example:

The User-Agent is composed of the following tokens:

Desktop


User-Agent for Windows XPGET http://www.gasbuddy.com/ HTTP/1.1Host: www.gasbuddy.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Language: en-US,en;q=0.5Accept-Encoding: gzip, deflateConnection: keep-alive

Desktop


User-Agent Glossary


User-Agent for Google NexusGET http://gasbuddy.com/ HTTP/1.1Host: gasbuddy.comConnection: keep-aliveUser-Agent: Mozilla/5.0 (Linux; Android 4.2.2; Nexus 7 Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Encoding: gzip,deflate,sdchAccept-Language: en-US,en;q=0.8Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3


User-Agent Makes a Difference

Desktop


Modifying the Web BrowserCapabilities differ by web browser and version:

• Firefox has configuration web page.• Earlier versions of Internet Explorer required Add-ons; latter versions have

built-in features using its Developer Tools menu item.• Google Chrome has Developer Tools; earlier version required command-line.• Safari as a Develop menu bar that can toggle the User-Agent.

Detailed Instructions:http://www.howtogeek.com/113439/how-to-change-your-browsers-user-agent-without-installing-any-extensions/


Make Firefox an iPad


Firefox Meets iPad• Customer goes to the GasBuddy home page at

http://www.gasbuddy.com/.• The GasBuddy application detects that the

browser session emanates from a mobile device and redirects the web browser to the mobile version of the web site.

• The mobile functionality is now accessible within a desktop browser, such as location aware browsing.

Begin test script capture from the load testing tool…

http://www.gasbuddy.com/


User-Agent Spoofing CaveatSome desktop web browsers may not behave the same as a mobile web browser…

− Application may require device-specific information, such as GPS.

− UI may want to use location aware browsing

There are other approaches if this is the case…


Proxy RecordingUse Case: Applicable for browser-based applications on mobile devices and native apps. The mobile device and load testing tool desktop must use Wi-Fi.Strategy

• “Point” the client’s user interface to an intermediate proxy running on your desktop.

• Some load testing tools can capture HTTP traffic directly by its recording mechanism.

• Ancillary utilities can capture the HTTP traffic, if the load testing tool is not capable to so.


Proxy Recording Steps

Desktop

Web Application

LAN

1. Set the Wi-Fi settings on the mobile device to use a HTTP Proxy.

2. Launch the load testing tool recording mechanism or launch the web proxy utility on your desktop.

3. Navigate through the mobile application; either via a web browser or native app.

4. After recording work with the automated test script or transform the captured conversation into the format for the load testing tool.


Enabling a Proxy for a Device• Proxy settings are found within the Wi-Fi options of the SSID.• Access to the settings is dependent upon the device.• Enter the Wi-Fi IP address and port of the web proxy or test tool.


JMeter Proxy Recording• JMeter uses a web proxy for script capture.• The proxy runs on the desktop on user-defined port number.• The mobile device needs to “point” to this desktop’s IP address and port

number to be captured by JMeter.


JMeter Proxy Recording

DesktopWeb

Application


Example: JMeter Test Plan


Fiddler Proxy RecordingWhat happens if the load testing tools don’t support proxy recording?

• Use an ancillary tool (that’s free) to capture the HTTP conversation.• Manually transform the HTTP conversation into an automated test script.• Programmatically convert the HTTP conversation into an automated test

script.

http://www.fiddler2.com/fiddler2/


Fiddler Setup


Fiddler Proxy Recording

DesktopWeb

Application


Fiddler Session• Captures all HTTP requests.• The raw details of each HTTP

request are stored.


Fiddler Session to Script• Manually edit load test tool script with Fiddler Session data. Or• Use Fiddler Plug-ins to automate the transformation process.

Support for Visual Studio Web Tests already exists. Custom formats can be created using any .NET language.


Packet Capture (PCAP)Refers to the act of capturing network traffic, as well as a well-defined structure for saving and manipulating the data.

• Libraries exist for both UNIX-like systems (libcap) and Windows systems (WinPcap); i.e. network capture is not tied to your Windows desktop and can even be performed on the mobile device.

• Network captures can be enabled via tools, such as tcpdump, WireShark, and Microsoft Network Monitor (all are free!).

• There is an API built in C. There are other programming languages that have a wrapper API built; i.e. network captures can be programmatically manipulated for our scripting purposes.


Packet Capture (PCAP)Use Case: Applicable for browser-based applications on mobile devices and native apps. You only need access to the PCAP traces; i.e. the mobile device and your desktop do not need to be coupled.

Strategies• Setup a Wi-Fi hotspot on a desktop computer with a packet capture

mechanism. Point the mobile device to the hotspot. You can also use the Wi-Fi proxy strategy that was discussed previously.

• Enable PCAP capture on mobile device. This will require the mobile device to be rooted.

• Plug packet capture device into network switch that will eventually “see” the mobile traffic.


PCAP Using Wireshark

DesktopWeb

Application


Using Wireshark• Ability to open existing captures,

filter conversations, and view packet details.

• Ability to launch new packet captures.


Starting a Wireshark Capture• You need to pick the network

interface to capture packets off of.

• We need to make sure that the wireless network interface is used, if we are using the Wi-Fi hotspot or web proxy setup.

• Start and Stop the capture when appropriate.


A Wireshark Capture


Filtering a Wireshark Capture


What’s in Wireshark Capture?


There is PCAP in the “Cloud”


What Can We Do?• Wirehark provides useful views

into the HTTP packets.• This information can be manually

transformed into an automated test script using any load testing tool.

• The transformation can be automated, if you are comfortable with programming.


PCAP ProgrammingPCAP traces can be programmatically parsed. We can use this functionality to automate the test script transformation process.

• Pcap.NET: http://pcapdotnet.codeplex.com/• jNetPcap: http://jnetpcap.com/• dpkt: http://code.google.com/p/dpkt/• node-pcap-parser:

https://github.com/nearinfinity/node-pcap-parser/blob/master/test/pcap-parser-test.js

http://pcapdotnet.codeplex.com/

http://jnetpcap.com/

http://code.google.com/p/dpkt/




PCAP Programming Examples

http://jnetpcap.com/tutorial/usage http://jon.oberheide.org/blog/2008/10/15/dpkt-tutorial-2-parsing-a-pcap-file/


Test Automation Re-Cap1. Spoof mobile device from a desktop web

browser. Relies on modifying your browser’s User-Agent metadata.

2. Proxy recording using actual mobile device. Record directly into test tool or use web proxy debugger.

3. Packet captures (i.e. PCAP) using actual mobile device. Use packet capture to get HTTP contents and/or program a transformation solution.

Script Capture


Emulating Network Conditions• Networks can influence the end-

user experience, as well as the scalability of the mobile application.

• Typical testing lab environments do not introduce network variability into the performance equation.


Emulating Network ConditionsCharacteristics to consider:

− Latency− Bandwidth− Packet Loss− Jitter

Challenges:• Not a lot of “accessible” options; i.e.

open source and/or turnkey.• Mostly commercial vendors.


Emulating Network Conditions

Network Emulator for Windows Toolkit

(NEWT)

Commercial Solutions Open Source Solutions


Scaling Up Load GenerationTypical Requirements:

− Large numbers of users requires hardware to emulate.− Hardware may only be needed on-demand.− Requirement to geographically distribute virtual users.

Solutions:• Use cloud services to build out infrastructure.• Leverage self-service solutions w/ pre-configured infrastructure.• Nothing is “free” here.


Scaling Up Load Generation


Moving Forward• We’re accessible for further discussions.• Potential for future webinars to continue our discussion.• Foster collaboration. We’ve created a LinkedIn User Group in order to

expand upon this discussion.


Thank you! Questions?

Linkedin GroupPerformance Testing- Mobile Applications

Matthew [email protected]

Websitewww.rttsweb.com

Call us(646) 329-9309