Testing in the Cloud

Introduction

- What is “The Cloud”?- What is different / the same for cloud testers?- Testing Philosophy / Test phases- What was easier / harder than expected?- Key success factors- Questions?

What is “The Cloud”?

Internet (“The Cloud”) accessible solution Off-premise multi-tenant architecture Accessed via a browser (PC or Wireless) Delivered as Software as a Service (SaaS)

Common cloud companies are: Google, Amazon, WorkDay, and Salesforce

What things are the same about testing cloud solutions?

Requirements coverage based testing Risk-based structured test approach Test Plan / Test Phases Test Cases / Test Data / Test Automation Defect Management / Functional Test Use of off-shore resources / Virtualization Politics {Quality / Schedule / Resources}

What is different about testing cloud solutions?

Shared multi-tenant test environmentsSecurity (Test User IDs / SSO)Integration of on/off premise systemsPerformance / volume testDefect isolationDocumentation of “Out Of the Box” reqsRelease Management

Project Philosophy

Risk-based structured testing approach {std} “Exploratory Testing” to discover the out of the

box cloud solution features {non-std} Data Migration – Moving legacy data off premise

{Non-Std} Performance / volume test only our new

infrastructure – not the cloud app {Non-Std}

Testing Phases

Overlapping Test Phases

Unit / CodeCoverage

Inspections& Reviews

Exploratory& System

UAT &Pilot

End-To-EndIntegration

A mix of waterfall (with heavy phase overlapping), and iterative delivery

Load &Performance

DataMigration

What was harder than expected? Isolating the architectural components for a

performance test Understanding that a cloud solution vendor

delivers new releases which you have to take! • Accepting that you have a “black box” in your

environment Determining who owns defects Data Migration Mock Runs

What was easier than expected?

Getting the IT leaders to embrace change Overlapping Test phases End User Adoption Use of off-shored test resources

Functional Performance Data ETL

Key Success Factors

Learn the “Out of the Box” cloud application features, security model, and configuration capabilities

Understand the holistic system architecture Understand the migrated test data Establish tight communications with the vendor