Upload
anne-marie-charrett
View
2.373
Download
0
Tags:
Embed Size (px)
DESCRIPTION
A webinar by Joel Montevilsky on testing Practitest, a SaaS service.
Citation preview
About me - Joel Montvelisky
PractiTest – Co Founder & Prod. Architect
QA Instructor & Consultant
Mercury Interactive - QA Manager (retired )
TD, QC, WR, QTP, etc
ITCB (IL) Advisory Board
QABlog.practitest.com
© 2010 -
Today’s Agenda
SaaS - Software as a Service
Things we Can Stop Testing
Non-trivial Stuff to Think About & TEST
Leveraging Live Feedback into the Testing Lifecycle
Open Season for Questions…
© 2010 -
© 2010 -
What is SaaS?
Software as a Service
Hosted Platforms – Total Control – No Maintenance
No Client Installations
Customers buys each month – keep them happy
© 2010 -
Users don’t care it is SaaS,
it needs to be better than the alternative
SaaS QA Management
End-to-End QA Management (Reqs – Tests – Bugs)
Enterprise Level SaaS Solution - Methodology - Customizations & Flexibility
Global platform - Communication - Languages - Available 24x7 Worldwide
Intuitive & Simple
Agile Dev, RoR, on Amazon EC2 © 2010 -
© 2010 -
What does SaaS mean to the Testing Process?
Things we can STOP Testing
✘ Client or Server installations
✘ Multiplatform backend support
✘ Multiple version upgrades
✘ Backwards compatibility
© 2010 -
“Savings” of approx. 1/3 of the Traditional Enterprise
Application Testing Cycle
The regular testing scenarios
Functionality - Manual scripts - Exploratory sessions - Checklists - End-2-End Scenarios - Selenium - Cucumber & RSpec
Load & Stress
Multi-platform & Multi-browser support
I18N English, Swedish, German, Chinese Simplified, Hebrew
© 2010 -
Agile Sprints & Updates of 3 – 4 weeks
Each sprint’s cycle is based on its content
Non-trivial stuff to think about & TEST
1. Remote accessibility & usage
Staging Servers in USA - Amazon EC2 Development & Testing teams in Israel (& WW collaborators)
Globally distributed monitoring services
(www.pingdom.com)
© 2010 -
Non-trivial stuff to think about & TEST
2. Security
Application-related:
Cross-site scripting
SQL injections
HTTP header injections
etc…
© 2010 -
Infrastructure-based:
Secured communication
Backups & storage policies
Controlled Access to the site & servers
Non-trivial stuff to think about & TEST
3. Live updates & deployments (3 to 4 weeks)
No concrete separation between Dev & IT
Minimal service disruption
Dressed rehearsals (including rollbacks!)
Automated sanity & manual verification
Patches & hot-fixes (extraordinary occasions )
Prove of Concepts & Beta Releases
© 2010 -
QA & Testing as a facilitator for Balance: Stable & Professional, yet Flexible Environment
Non-trivial stuff to think about & TEST
4. Disaster recovery procedures
Two Main scenarios:
(a) System down to be brought up quickly
Configure machines
Install & deploy software
Restore data
(b) Rollback to last known stable data (with or without data restore)
© 2010 -
Define a policy & schedule regular drills to ensure you can achieve it!
© 2010 -
Leveraging Real Data into the Testing Process
Levering live feedback into the Testing Process
(1) Internally developed monitoring tools
- Check usage patterns
- Validate need & usability assumptions
- Analyze patterns and create realistic & prioritized testing scenarios
© 2010 -
No need to continue guessing!!!
Levering live feedback into the Testing Process
(2) Site Monitoring systems
LiveAdmin (www.liveadmin.net)
Real-time usage to schedule updates
Allow users to contact the company with questions or issues
Pingdom (www.pingdom.com)
Up to date response times for real projects and data.
Proactive alerts when things start slowing down
© 2010 -
Levering live feedback into the Testing Process
(3) Proactive mail notifications on issues
(www.hoptoad.com)
Real-time notifications on issues on the system.
Information about the issue including system parameters that allow you to do an initial trouble-shoot of the issue
© 2010 -
Levering live feedback into the Testing Process
(4) Ask users for Real Projects
No better way to test than using real data
Most users will trust you not to abuse their information in exchange for assurance of working software
Since the data sits in your system it is easier to copy projects to your testing environment
Need to be careful with who has access to the data; making sure your testing environment is as secured as your production environment
© 2010 -
© 2010 -
Wrapping up…
Wrapping up
SaaS is only the delivery model, customers will not compromise on Functionality, Stability or Response Time
Since the system is hosted you can save all tests related to backend installation and support
You need to pay more attention to stuff like accessibility, security, deployments & DRPs
The lines between the R&D and IT organizations in SaaS are blurry
SaaS application allows better access to real data to be leveraged to improve the effectiveness of our testing operations.
© 2010 -
© 2010 -
Open Season for questions!