Upload
robert-van-lieshout
View
280
Download
1
Embed Size (px)
DESCRIPTION
Presentation for the European SEPG conference in 2005.
Citation preview
1
Using Formal Methods inIndustrial Software Development
European SEPG ConferenceLondon, June 2005
Robert van Lieshout, Quality [email protected]
2
“Quality through Commitment and Creativity”
3
Engineering and MathematicsEvery branch of Engineering uses Mathematics for Specification, Design and Verification
Mechanical Engineering => Differential EquationsStructural Engineering => Finite Element AnalysisCircuit Engineering => Boolean Algebra, etc
Except Software EngineeringOriginally a specialization within Mathematics!
Most Software is specified and designed without using mathematicsSoftware specifications and designs cannot be verified before
implementationSoftware testing must find specification, design and
implementation errors
4
Presentation Outline
Formal Methods – then and nowI-Mathic – an overviewApplying I-Mathic – some resultsBenefits and DrawbacksCurrent status
5
Formal Methods
Then and now
6
Formal Methods – Then ...
Formal Methods have promised much and delivered little:The solution is often more complicated than the problem
Formal specifications use difficult notations and require extensive mathematical backgroundCritical Stakeholders - Business Analysts, Domain Experts and Customers - cannot understand the formal specifications
Critical Stakeholders excluded from the process
7
Formal Methods – Now
Growing interest in Formal MethodsSee: http://www.fmeurope.org/Several methods with different objectivesProof of selected properties, rather than full correctness proof
Tool support and computing power have:Reduced laboriousnessImproved user-friendliness (sometimes)Made it less time consuming
8
The I-Mathic Formal Method
An overview
9
I-Mathic Principles
Design PrincipleDevelopers can and should strive to produce software that is nearly error free when entering testing
Testing PrincipleThe purpose of testing is quality measurement and not an attempt to “test in” quality
10
I-Mathic – Origins
Used a small part of Cleanroom Sw. Eng.Sequence Enumeration
Developed tool supportExcel template with VB automationScripts for visualisationScripts for code & test generation
Linked it with CSPCSP: Communicating Sequential ProcessesTools to check the CSP model
11
I-Mathic – Overview
12
Applying I-Mathic
Some results
13
Run outRun in
PP PP PP … PP PP
Transport system containing PCB’s
1..20 Pick & Place Robots
Example Project – Assembléon AX
14
AX Kernel
TCPPC SVS UTLMonitor
GUI intf(29 states)
Process Ctrl
‘Glue’
GPETERM GPEGPEGPE
Module intf(10 states)
Glue intf(11 states)
GPE intf(17 states)
Example Project – AX Kernel
15
Example Sequence Enumeration (fragment)0:<> IDLE
1 Cl.rqStartProduction Mod.rqStartProduction 1 GPE.Req12 Cl.rqPauseProduction Illegal - GPE.Req23 Cl.rqRecover Illegal - GPE.Req34 Srv.ntErrorOccured Illegal - GPE.Req15 Srv.ntProductionStarted Illegal - GPE.Req26 Srv.ntProductionPaused Illegal - GPE.Req37 Mod.ntErrorOccured Illegal - MOD.Req18 Mod.ntProductionStarted Illegal - MOD.Req29 Mod.ntProductionPaused Illegal - MOD.Req3
1:<Cl.rqStartProduction> STARTING_MOD10 Cl.rqStartProduction Illegal - D011 Cl.rqPauseProduction Illegal - D012 Cl.rqRecover Illegal - D013 Srv.ntErrorOccured Illegal - GPE.Req114 Srv.ntProductionStarted Illegal - GPE.Req215 Srv.ntProductionPaused Illegal - GPE.Req316 Mod.ntErrorOccured Cl.ntErrorOccured 16 Module in ERROR MOD.Req117 Mod.ntProductionStarted Srv.rqStartProduction 17 Module in RUN state MOD.Req218 Mod.ntProductionPaused Illegal - MOD.Req3
The Component Function is completeMaps every possible input sequence to response
The Component is the right systemEvery transition rule justified, full requirements tracingDerived requirements fill the gaps
16
Results – Model CheckingModel checker explores all state combinations of the CSP
model ensuring that:
Model is deterministicModel implements interface according to specificationThere are no deadlocksFinite queue size is sufficientQueues are never full (processes behave freely)
17
Results – Project Performance
122131143Productivity (eLocs/man day)
38,581,5112Effort (man days)
46901071516050Code Size (eLocs)
39012403000Transitions
2038194States
CBA
0.00.30.2Post Release Defects per 1000 eLoc
033Post Release
000During Acceptance
575During Internal verification
CBADefects
Size and Effort
18
Example – Quick ScanUsed to model part of an existing system
System Characteristics:Data driven multi-media application Multi-threaded components“Mature software”
Approach:Reverse-engineered models for a few scenario’s with help from domain expertRan model checks
19
Example – Quick ScanScope:
2 interfaces & 1 implementation modelledInterface 1: 5 states, 12 transitionsInterface 2: 10 states, 39 transitionsImplementation: 6 states, 44 transitions
Result:Potential deadlock detectedSeveral undocumented requirements revealedTotal effort spent: <12 man days
Including customer effort & writing report
20
Applying I-Mathic
Benefits and Drawbacks
21
Benefits
Encourages interaction with stakeholdersSufficiently understandableConsistent and complete requirements
Few defectsDeadlock free; No race conditionsSimple defects, easy to solve
High productivityPartly due to code generationPartly due to reduced test effort
22
Drawbacks
Limited integration with other methods and tools
I-Mathic is a very different approach
Requires abstract thinking and disciplinewhich not all software engineers have!
23
How does it scale?
Current data is on relatively small projectsExpected to scale well, considering the integration effort and defects
Maybe next year we’ll know?
24
I-Mathic
Current Status
25
Current status
Mandated for use in in-house projectsQuick ScanSupport for multi-threaded componentsRegular additions to toolset
E.g. context sensitive menu’sOngoing development:
Research together with Dutch universities
26
References
Cleanroom Software EngineeringMills, H.; Dyer, M.; & Linger, R.IEEE Software, September 1987
Communicating Sequential ProcessesHoare, C.A.R.Prentice Hall, April 1985ISBN: 0131532715