View
221
Download
0
Category
Preview:
Citation preview
QWise software engineering – refactored! www.qwise.se
Testing, testingA first-look at the new testing
capabilities in Visual Studio 2010
Mathias OlaussonCallista Knowledgebase | QWise
QWise software engineering – refactored! www.qwise.se
About Mathias...
QWise software engineering – refactored! www.qwise.se
About the Presentation...
• What’s on the menu?– Test planning and execution– Issue tracking with rich bugs – Eliminating the "works on my machine / cannot
reproduce" scenario– Automated UI Testing– Virtual Lab Management
QWise software engineering – refactored! www.qwise.se
TEST PLANNING AND EXECUTION
QWise software engineering – refactored! www.qwise.se
Agile Testing
QWise software engineering – refactored! www.qwise.se
Core Practices
QWise software engineering – refactored! www.qwise.se
Agile Testing Quadrants
Functional Tests Exploratory Tests
Unit TestsComponent Tests
Performance TestsLoad Tests
Security Tests”-ilitets” Tests
Business-facing
Technology-facing
Supp
ortin
g th
e te
amCritisizing the product
Automated &Manual
Manual
AutomatedAutomated
QWise software engineering – refactored! www.qwise.se
Test TypesManual Tests
UI Tests
Acceptance Tests
Unit Tests / Component Tests
QWise software engineering – refactored! www.qwise.se
Testing in an Iteration
sprint plan
sprint plan
write tests for us1
write tests for us1
write tests for us2
write tests for us2
sprint plan
sprint plan
build 1build 1 build 2build 2 build 3build 3
implement user story ‘1’
(us1)
implement user story ‘1’
(us1)
test us1 & file bugs
test us1 & file bugs
Implement US2
Implement US2
build 4build 4
fix bugsfix bugs
build 6build 6 build 7build 7
regress impacted
tests
regress impacted
tests
build 5build 5
implement us2
fix bugsfix bugs
test us2 & file bugs
test us2 & file bugs
verify fixesverify fixes
verify fixesverify fixes
QWise software engineering – refactored! www.qwise.se
Microsoft Test Manager and Visual Studio
Team Foundation Server Reporting
Development (Teams)Testing (Teams)
Web Test
Load Test
Coded UI Test Unit Testing
Test Case Management
Lab Management
Microsoft Test Runner
Fast Forward For Manual Testing
GeneralistSpecialistEngineering Support
Data Collectors (IntelliTrace, Test Impact, Metrics)
QWise software engineering – refactored! www.qwise.se
The Fundamentals
QWise software engineering – refactored! www.qwise.se
Defining and Running Tests
QWise software engineering – refactored! www.qwise.se
ISSUE TRACKING WITH RICH BUGS
QWise software engineering – refactored! www.qwise.se
Should We Do Bug Tracking?
Pros– Tracability– Priority of work– Visibility
Cons– Overhead
Should we log bugs within an iteration?Source: Pixar ”A bugs’ life”
QWise software engineering – refactored! www.qwise.se
Defect Tracking in MTM and MTR
QWise software engineering – refactored! www.qwise.se
No-more-no-repro
• Automatic data collection
• Test settings– Configuration of
diagnostic data collectors
– Assigned to test plan• Or per test run
QWise software engineering – refactored! www.qwise.se
Diagnostic Data Adapters
QWise software engineering – refactored! www.qwise.se
No-more-no-repro
QWise software engineering – refactored! www.qwise.se
AUTOMATED UI TESTING
QWise software engineering – refactored! www.qwise.se
Why Automated Testing?
Källa: Tools for Agility, Kent Beck
QWise software engineering – refactored! www.qwise.se
iteration 1iteration 1 iteration 2iteration 2 release iterationrelease iteration
feature afeature a feature bfeature b feature cfeature c feature dfeature d
unit testing
manual testing
regression testing
performance testing
feature afeature a
Automation Strategy
QWise software engineering – refactored! www.qwise.se
Coded UI Tests
• Unit test for UI• High-level framework• Create test from– Recording– Test result (action
log)– Using API
QWise software engineering – refactored! www.qwise.se
Automation Platform Support
QWise software engineering – refactored! www.qwise.se
Coded UI Tests
QWise software engineering – refactored! www.qwise.se
LAB MANAGEMENT
QWise software engineering – refactored! www.qwise.se
Motivation
QWise software engineering – refactored! www.qwise.se
Lab Management Aims To...
QWise software engineering – refactored! www.qwise.se
Lab Management Infrastructure
Team Foundation Server (TFS)
Lab Management
Test Case management
Build management
Work Item Tracking
Source Control
Hyper-V HostsVirtual
Machine1
Virtual Machine2
Virtual Machinen
Library SharesLibrary Share1
Library Share2
Library Sharen
System Center Virtual Machine Manager (SCVMM)
Team Foundation Server (TFS)
Lab Management
Test Case management
Build management
Work Item Tracking
Source Control
Hyper-V HostsVirtual
Machine1
Virtual Machine2
Virtual Machinen
Library SharesLibrary Share1
Library Share2
Library Sharen
System Center Virtual Machine Manager (SCVMM)
QWise software engineering – refactored! www.qwise.se
Environments
• An environment is a set of machines with different roles
• An environment can– Be deployed, saved, removed– Started, stopped– Snapsshot
VM1
Lab Agent
VM2
Lab Agent
QWise software engineering – refactored! www.qwise.se
Manual Tests in Lab
• Testing from the outside– Easy for web applications– Limitations in data collection
• Testing from the inside– Run application in lab via environment viewer
QWise software engineering – refactored! www.qwise.se
Automatic Tests in Lab
• Testing in VM– Test case connected to test environment– Test component comes from selected build
• Testing in build– Test case connected to build definition– Runs test list or test component
QWise software engineering – refactored! www.qwise.se
Lab Management Build Workflow
QWise software engineering – refactored! www.qwise.se
Build and Deploy
QWise software engineering – refactored! www.qwise.se
Summary
• Structured test planning with MTM• Integrated test execution• Effective defect tracking with rich bugs• Seamless integration with automated tests• Powerful lab management
QWise software engineering – refactored! www.qwise.se
Questions?
mathias.olausson@callista.sehttp://msmvps.com/blogs/molausson
Recommended