Upload
briana-tyler
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Model-Based Testing made easy with Spec Explorer 2010
Name: Wolfgang GrieskampTitle: Software ArchitectCompany: MicrosoftSession Code: DEV316
3
Modeling in Science and Engineering
A model:Is an abstraction of the system from a particular perspectiveSupports investigation, construction and prediction Is not necessarily comprehensiveCan be expressed as a table, graphical diagram, formal notation,etc.
Ubiquitous and Universal
Software Modeling Styles and Notations
4
Software Models
Structure
Class Diagram
Component Diagram
Behavior
Interaction Based
Use Cases
Message Sequence Charts
Traces/Patterns
State Based
Guarded Update Machines (Code)
State Charts
Focus of this session
Model-Based Testing in a Nutshell
Model
Implementation
Expected Outputs (Test Oracle)
Inputs(Test Sequences)
Control
Requirements
Generate
Observe
Feedback
Feedback
Author
Issue
Feedback
Test Cases
Verdict
Feedback
Model-Based Testing with Spec Explorer
Explore
Analyze
Generate
Execute
C# Model (or other .Net Language)
Model Graph
Test Suite
VSTT Result
Walkthrough: Modeling and Testing a Web Service
Example: A Simple Web Shop
Web Shop Service
Web Shop Client
Billing
Shipping
AddToCartCheckout
Bill
Ship
System Under Test
Web Shop Service
Web Shop Implementation in WCF [ServiceContract(CallbackContract = typeof(IWebShopCallback))] public interface IWebShop { [OperationContract] // Add item to stock of web shop void AddItem(string itemName, int price); [OperationContract] // Add item to shopping cart of user bool AddToCart(string userName, string itemName);
[OperationContract] // Check out void Checkout(string userName); }
public interface IWebShopCallback { [OperationContract] // Initiate billing, returns true on success bool Bill(string userName, int amount);
[OperationContract(IsOneWay = true)] // Initiate shipping void Ship(string userName, string itemName); }
Understanding the Web ShopWhich of the following traces is valid?
(assuming an initialization of the shop with AddItem(“wine”,10) and AddItem(“cheese”, 5))
1 AddToCart(“Peter”, “wine”) Checkout(“Peter”) Bill(“Peter”,10)/true Ship(“Peter”,”wine”)
2 AddToCart(“Peter”, “wine”) Checkout(“Peter”) Bill(“Peter”,10)/false Ship(“Peter”,”wine”)
3 AddToCart(“Peter”, “wine”) Checkout(“Peter”) Bill(“Peter”,12)/true Ship(“Peter”,”wine”)
4 AddToCart(“Peter”, “wine”) AddToCart(“Peter”,”cheese”) Checkout(“Peter”) Bill(“Peter”,15)/true Ship(“Peter”,”wine”) Ship(“Peter”,”cheese”)
5 AddToCart(“Peter”, “wine”) AddToCart(“Peter”,”cheese”) Checkout(“Peter”) Bill(“Peter”,15)/true Ship(“Peter”,”cheese”) Ship(“Peter”,”wine”)
Modeling Web Shop: Basic Approach
1. Import or declare action vocabulary in configuration script
2. Define state variables in C# model program3. Define state update rules in C# model program4. Define scenarios and generate state transition
system
config Actions{ action static void
TestAdapter.AddItem(string itemName, int price); action static void
TestAdapter.AddToCart(string userName, string itemName); action static void
TestAdapter.Checkout(string userName); action event static void
TestAdapter.Billed(string userName, int amount, bool success); action event static void
TestAdapter.Shipped(string userName, string itemName);}
Modeling Web Shop: Action VocabularyAdapter for the
System-Under-TestReference to test adapter method
Indicates that this action is an event raised by adapter
Modeling WebShop: Define Model State in C#
enum ShopperStatus{ Shopping, CheckedOut, Billed }
class ShopperInfo{ // Current status of this user internal ShopperStatus Status;
// Items on cart so far internal SequenceContainer<string> Items;}
// Foreach user shopping, contains info static MapContainer<string, ShopperInfo> shoppers;
// Foreach item in the shop, contains pricestatic MapContainer<string, int> items;
Modeling WebShop: Defining State Update Rules
[Action(“AddToCart(userName,itemName)”)]static void AddToCart(string userName, string itemName){ Contracts.Requires(items.ContainsKey(itemName));
if (!shoppers.ContainsKey(userName)) shoppers[userName] = new ShopperInfo();
Contracts.Requires(InStatus(userName, ShopperStatus.Shopping)); shoppers[userName].Items.Add(itemName);}
static bool InStatus(string userName, ShopperStatus status){ return shoppers.ContainsKey(userName)
&& shoppers[userName].Status == status;}
Binding rule to action in action vocabulary
Condition under which rule is enabled.
Fires a state update
State update
Modeling WebShop: Define Scenarios
machine MultipleItemScenario() : Config{ AddItem("Bread", 2); AddItem("Cheese", 5); AddItem("Wine", 10) AddToCart("Paul","Wine"); AddToCart("Paul", "Cheese"); (Checkout | Billed | Shipped)*}
machine MultipleItemSlice() : Config { MultipleItemScenario || construct model program from Config}
Combining scenario with C# model program
Or’ing actions in repetition (Note parameters omitted)
Sequence of actions
Modeling WebShop: Execution of generated test code
Model expects bill of $2, implementation bills $4
“BlueLine” Technical Document Testing Project
222 open protocol specifications tested22,847 pages
66,962 person days (250+ years)Vendor test teams
Hyderabad: 250Beijing: 100
Tool dev center in Beijing (Spec Explorer and other)~10,000 document bugs filed
Comparison Model-Based/Traditional Testing
Vendor teams had (guided) choice which approach to useOut of 222 protocols approx. ½ modeled, ½ traditional Modeling resulted in 42% productivity gain in average (some teams reached more than 300%)
Overall, modeling saved 50 person-years (12,547 days)
No special education background was requiredTeams of junior engineersMostly vendors, fresh out of college
Spec Explorer on the Web
DevLabs home page and download http://msdn.microsoft.com/devlabsSpec Explorer team blog http://blogs.msdn.com/specexplorerSpec Explorer forum http://social.msdn.com/forums/specexplorer
Wolfgang Grieskamp’s blog http://blogs.msdn.com/wrwg
A Model-Based Testing tool from Microsoft
www.microsoft.com/teched
Sessions On-Demand & Community
http://microsoft.com/technet
Resources for IT Professionals
http://microsoft.com/msdn
Resources for Developers
www.microsoft.com/learning
Microsoft Certification & Training Resources
Resources
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.