Testing Ajax, Mobile Apps the Agile Way

March 4, 2014 Frank Cohen, fcohen@appvance.com, (408) 364-5508

PepsiCo Needs Deep Reach To Consumers

•High User Experience Web App Campaigns

• Do Us A Flavor, Every2Minutes using Ajax, Web, Flash/Flex,

•Broad Reach Mobile App Campaigns

• iOS/iPhone, iPad, Android, Tablets Native Apps

•Time-On-Page Using Publishing Platforms

• Tumblr, WordPress, Google+, Facebook, Twitter

•Partners Via ERP, Workflow, Integration Platforms

• Oracle, TIBCO, IBM, Microsoft, Software AG, Redhat BPM, EMS, MQ

Testers Are Under Big New Pressure

•Agile Software Methods

•Ajax Environments

•Mixed Mobile Environments

“The Business Managers Tell Us They Need 1000 VUs in 10 Mins”

“The Devs used JMeter”

“The App Runs on WebKit Browsers and Android Native”

How Does It All Fit Together

•Agile Environment featuring Agile Stories

• Story < UseCase < Test < Schedule < Deploy < Build

• Continuous Integration, Source Repository, Test Scripting, Deploy Manager, and Agile Test Management

• CloudBees, Travis CI+Deploy, JMeter, Rally, Appvance

•Select Test Tools, Test Management Tools, And Create Operational Test Data

Test Runtime Architecture

Appvance Performance Cloud

Virtualized Environment

Shared ServicesEnvironment

AppvanceController

Map Story to TestsDeploy to Cloud

Log Results

Test Repository

Clear

Continuous Integration

JenkinsDevelopers

Business,DevOps

Test Management

Clear

Instance 2

Instance 1

BrowserChrome, IE

App Under Test

DPL RepositoryOracle

Scalability Index

Dashboard

Results RepositoryOracle

DriverSOAP, REST

Test InjectorScript Runner, Monitor, DPL

Test InjectorScript Runner, Monitor, DPL

Service Under Test

App Under Test

Service Under Test

BrowserChrome, IE

DriverSOAP, REST

Aric New User SLA WarningAverage perf: 32 Seconds

Modern Deployment Requires Modern Testing

•Desktop browsers, iOS and Android native apps, WebKit and Mobile Web browsers

•End-To-End means HTTP Protocol level testing

•Beginning-To-End means Driving The Entire App

•Combination Testing Surfaces Root Case To Bottlenecks

The Hybrid Model

•End-To-End Component Testing at API/Protocol Level

•Beginning-to-End App Integration Testing

•Scalability Testing of End-User Experience

•Data Driven Tests from live operational data sources

•Functional Tests Repurposed To Performance Tests

•Validate Digital Agency Tests (normally not LoadRunner)

Typical Solution Combines Process and Teams

•LoadRunner Best Practices for Ajax, Mobile

• Tool and license for LoadRunner team to do its own testing

• Train testers and business managers on Test Methodology

•Consulting Practice, Integration Expertise

• Build upon the tools

•Identify Requirements for Digital Agencies, Product Teams

• 10-15 best practices required by digital agencies for app development

• Server Configuration Requirements Document

• SLA Requirements Definition for Publishing PAAS Providers

Missing Integration Expertise

•App Integration Bridges Digital Agencies

Agency Architecture

Diagram

Host Implements

Frank: “Where’s the Node.js load balancer?”

Host: “Oh, We missed that.”

Digital Agencies Making Same Mistakes

•Frank asks: “Why is the deployed app not accepting 13 digit cap codes?” Agency: “We missed that.”

•Frank asks: “Why is network limiting throughput to PHP requests to Node.js?” Host: “The F5 load balancer does that by design. We’ll remove it.”

•Frank asks: “Does the delivered app have testing APIs?” Agency: “We will write some.”

Mistakes Happen At App Integration Time

•Frank asks: “How could more than 1 person log-in per second using Gigya?” Gigya: “Oh, we didn’t think of that.”

•Frank asks: “Why isn’t the Log In service scalable above 100 log-ins per second?” Agency: “You’re running it on Windows, all our experience is on Linux.” Host: “We don’t know.”

•Frank asks: “Why is Apache set to only use 25 worker threads, when the app needs hundreds?”

Example Architecture - Integrated App

Distributed Edge Cache

Browser

App Web 1Apache, PHP, Zend

MySQL MasterUtility

Apache, PHPEmail Services

NodeJSBid History

NodeJSView Counts

Load Balancer Load Balancer

MySQL Read

MySQL Read

Load Balancer

App Integration

App Integration Test Readiness Requirements

App code from Agency

Installation at Host

Voting Service (SAAS)

Configured

Preference Center

Configured

Database Created and Populated

Apache, PHP, Zend, .NET,

Java optimized

Last Minute Changes from the Business

Number of Server

Machines Identified and

Provided

Best Practices for LoadRunner on Ajax, Mobile

•Beginning-to-End App Integration Testing

• Run Ajax App In Real Browsers: Ie, Firefox, Chrome, Safari, Webkit, Mobile Web. Tests 100% Of The End-User Experience.

• LoadRunner TruClient does not work reliably with Ajax apps, does not test on Chrome, Safari, WebKit, and older IE, Firefox

• LoadRunner works great for end-to-end and protocol level tests. For example, making HTTP/JSON requests to a Node.js server through a load balancer

•Combine Beginning-To-End and LoadRunner for test of app’s integration at app and protocol tiers

Models That May Work

•Product and License - Augment What You Already Have, Teach and Train your team

•Services, Experts in Load Runner and other products, need outsource, short staff

•We know your product, we know your tools, we know our tools, we know what reduces app performance

•Maestro Service between agencies

•App Integration and Performance Test Services

Appvance

•When You Build Modern Apps And Wonder “How Risky Is That App Launch Going To Be?”

•Call Us At (855) 254-1164 Or Email Us Sales@Appvance.ComAnd We Will Show You