Software Architecture, Process and Management · SAPM: Waterfall Problems with the Waterfall Model I In my opinion, the waterfall model is simply a fundamentally awed metaphor for

Software Architecture, Process and Management

Allan Clark

School of InformaticsUniversity of Edinburgh

http://www.inf.ed.ac.uk/teaching/courses/sapmSemester Two 2012-13

SAPM

Quick Message for Level 10s

I Don’t forget to check back on your blog post for comments!I Don’t forget that the blog is live to the world, readers may

not be familiar with the fact that your blog post is related toa particular piece of reading

I So your associated article should probably be mentioned in thefirst paragraph

I Good idea to have a title separate to the title of the readingthat you are reviewing

SAPM

Quick Message for Level 11s

I Many of you are sending me topic and thesis ideas for approval

I It has been fun seeing theseI A small tip for everyone to bear in mind:

I The more precisely you define your thesis, the easier it will beto defend it

I Both in terms of the fact that your thesis is less open toattack, and in terms of the fact that the process of refiningyour thesis should help guide the structure of your report

I You don’t have to send me a refined thesis, and you don’t haveto specify it in a single sentence, but your introduction shouldmake clear your exact thesis

SAPM

Development Methodologies

I A methodology is a system of methods and principles used ina particular “school” of software design

I There is a wide variety of published methodologies, and aneven larger set of informal and/or company-specificmethodologies

I The most mature methodologies are often codified usingspecialist tools and techniques

I All methodologies are controversial, because some peopleargue that any fixed methodology is an affront to aprofessional, creative, independent designer, while the restargue about which methodology is best

SAPM

Example MethodologiesI In this course we will focus on three main methodologies:

1. The Waterfall Model (discussed in many courses)2. The Unified Process (UP) (partly covered in CS2)3. Extreme Programming (XP)

I But there are of course lots of others:I Cleanroom,I DSDMI V-modelI ScrumI Crystal, etc.

I The relationships between them are complex as well

I We will also discuss open-source design, which is more of aphilosophical approach than a methodology like the others,but which has implications for methodology

SAPM

Types of Methodologies

I “Cowboy hacking” and micromanaging are at the extremes ofa continuum (see related reading “Get ready for agilemethods, with care”)

I Basic distinction: agile vs. heavyweight

I Agile methods are more fashionable to discuss, but it’s hard totell what people are actually using

SAPM

Searching for a Solution

I We can think of “building” as more of a search problem inwhich we search for the correct solution

I We can imagine that we search by heading off in some chosendirection

I Backtracking occurs when we realise we have gone of in thewrong direction

I Heavyweight methodologies focus on making as correct achoice of direction as possible to avoid backtracking

I Agile methodologies focus on assuring that changing directionis as painless as possible thus avoiding actual backtracking

I There are other alternatives:I Going off in several directions at onceI Sending a scout who has to come back and report, also known

as prototyping

SAPM: Waterfall

Plan Driven Model: Waterfall

I Inspired by older engineering disciplines, such as civil andmechanical, for example, how cathedrals are built

I Development of a release is broken into phases, each of whichis completed and “signed-off” before moving on

I When problems are found, must backtrack to a previous phaseand start again with the sign-off procedures

I Much time and effort is spent on getting early phases right,because all later phases depend on them

SAPM: Waterfall

Waterfall Model of Release One

SAPM: Waterfall

Problems with the Waterfall Model

I In practice it is rarely possible to go straight through fromrequirements to design to implementation, withoutbacktracking

I There is no feedback on how well the system works, and howwell it solves users’ needs, until nearly the very end

I Large danger of catastrophic failure:I Any error in key user requirements dooms entire processI Big chance that the design is not actually feasibleI Big potential for unacceptable performance

SAPM: Waterfall

Problems with the Waterfall Model

I In my opinion, the waterfall model is simply a fundamentallyflawed metaphor for software development

I Design and debugging together account for nearly all ofsoftware development, with almost no construction step, justcompilation

I This is a huge difference from electronic hardware designwhere manufacturing and procurement typically dominate theprocess,

I or civil engineering, where construction dominates the process

SAPM: Unified Process

Unified Modelling Language

I UML is obviously a modelling language and not amethodology in and of itself

I But designed to be compatible with developmentmethodologies of the time and in particular object-orientedspecific development methodologies

I UML has 14 kinds of diagramsI 7 for structural informationI 7 for behavioural information

I 3 system behaviour diagram kindsI 4 interaction diagrams

I You won’t need to know any of this for the exam

I For what follows, when someone says “a UML diagram” theyjust mean a diagram in a standard format

SAPM: Unified Process

UML Problems

I The size, it contains many diagrams and constructs which areredundant or infrequently used, but at least such standardsare all in one place

I Complaints of ambiguity and inconsistencyI Intersection of UML and implementation language may

influence the implementation, this is true of any standardnotation language

I Model interchange format, since UML is a graphical notationyou still require some standard to promote exchange ofmodels between two software products

SAPM: The Unified Process

Relatives of the Unified ProcessI The IBM Rational Unified Process is a commercial product

and toolset, superseding:I The Objectory ProcessI The Booch MethodI The Object Modeling Technique

I The relationship is that these methods inspired the design ofUML which in turn drove the creation of the Unified Process

I The Unified Software Development Process is a published,non-proprietary method based on the Rational UnifiedProcess, but without specific commercial tools or proprietarymethods


The Unified Process

I Typical heavyweight approachI Iterative modification of waterfall model using modeling to

forestall backtracking:I Component basedI Uses UML for all blueprintsI Use-case drivenI Architecture centricI Iterative and incremental

I We just give an overview; for more details see:I Jacobson, I., Booch, G., & Rumbaugh, J. (1998). The Unified

Software Development Process. Reading, MA: Addison-Wesley.


Phases of Unified Process DesignI Each software release cycle proceeds through a series of

phases, each of which can have multiple modeling iterations:I Inception : Produces commitment to go ahead, business case

feasibility and scope knownI Elaboration : Produces basic architecture; plan of

construction; significant risks identified; major risks addressedI Construction : Produces beta-release systemI Transition : Introduces system to users


Waterfall Iterations within Unified Process Phases

I Each phase can havemultiple iterations — projectproceeds top-to-bottom andthen left-to-right

I Each iteration may includeall workflows but some aremore heavily weighted indifferent phases

I Still relatively hard tochange requirements onceimplementation is underway


Unified Process vs Waterfall Cycle


Use Cases and the Unified Process

I “A use case specifies a sequence of actions, including variants,that the system can perform and that yields an observableresult of value to a particular actor.”

I These drive:I Requirements captureI Analysis and design of how system realises use casesI Planning of development tasksI Acceptance/system testingI Traceability of design decisions back to use cases


The Product is a series of models


Unified Process Example 1












Models and Diagrams

I The specifics of the models and diagrams are unimportant

I What matters is that you start off with a high-level use-casediagram

I Refine that producing more detailed models of eachcomponent of the use-case diagram

I Until the models are sufficiently detailed such that refinementtakes the form of actual implementation


Assumption of Unified Process

I Unified Process, and other heavyweight methodologies,concentrate on carefully controlled, up-front, well-documentedthinking

I Based on assumption that cost of making changes risesexponentially through the development stages

I To minimise backtracking, Unified Process establishes rigorouscontrol over each stage

I At each stage of Unified Process, a model acts as a proxy forthe whole system, helping to bring out problems as early aspossible before they are set in code


Problems with Unified Process

I Heavy training, documentation, and tools requirements —learning and using UML, modeling, process, tools, techniques

I UML is not a native language for customers, and so it can behard for them to provide good feedback until system isimplemented

I Requirements are very difficult to change at late stages, ifneeded to match changes in business world, address newcompetition, or fix mistakes in requirements capture

SAPM: Extreme Programming

Extreme Programming (XP)

I What if it were possible to make the cost of change constantacross all stages, so that design and requirements can bechanged even at late stages?

I XP tries to prevent backtracking by keeping the systemcontinuously flexible, eliminating the need for determining thefinal correct requirements and design before implementation

I XP started the trend toward “agile” processes, such as Scrumand Crystal, focusing on closely knit, fast movingdesign/coding teams and practices

I see Beck, K. (1999). Extreme Programming Explained.Addison-Wesley.


Unified Process vs Extreme Programming


Extreme Programming is ControversialI An IBM Java poll on XP from: www.xprogramming.com said

roughly this:I “I’ve tried it and loved it” (51%)I “I’ve tried it and hated it” (8%)I “It’s a good idea but it could never work” (25%)I “It’s a bad idea — it could never work” (16%)

I Of course, the Unified Process is widely resented as well...

www.xprogramming.com


How Extreme Programming Imposes ControlI Through a set of “practices” to which designers adhere

I using whatever other compatible methods and tools they preferI See: www.extremeprogramming.org/rules.html

I Not strongly influenced by a particular design paradigm —unlike Unified Process

I Does require a strongly held (“extreme”) view of how toapproach design

I We consider some key practices in the following slides

www.extremeprogramming.org/rules.html


1. The Planning Process

I An XP project starts with a “Planning Game”

I The “customer” defines the business value of desired “userstories”

I The programmers provide cost estimates for implementing theuser stories in appropriate combinations

I No one is allowed to speculate about producing a total systemwhich costs less than the sum of its parts


User Stories vs Use Cases

I A user story meets a similar need as a use case, but is textual,not graphical, and is something that any customer can dowithout training in UML

I A user story deliberately does not include all the possibleexceptions, variant pathways, etc. that go into use cases

I Short example: “A bank customer goes up to an ATM andwithdraws money from her account.”


2. On-site Customer

I Someone who is knowledgeable about the business value ofthe system sits with the design team

I This means there is always someone on hand to clarify thebusiness purpose, help write realistic tests, and make smallscale priority decisions

I The customer acts as a continuously available source ofcorrections and additions to the requirements

I Best case scenario: developers are their own customersI Development tool creators: Fogbugz, github, Visual StudioI Tools for general population, gmail, facebook etcI Known as “eating your own dog food”


3. Small Releases

I Put a simple system into production early, implementing a fewimportant user stories

I Re-release it as frequently as possible while adding significantbusiness value in each release

I Significant business value will usually equate to a set ofimportant user stories

I For example: aim for monthly rather than annual releasecycles

I The aim is to get feedback as soon as possible

I The difficulty in fixing a particular bug is proportional to thetime between the developer developing the buggy code andthe time when it is reported and worked upon


4. Continuous Testing

I Write the tests before writing the software

I Customers provide acceptance tests

I Continuously validate all code against the tests

I Tests act as system specification


5. Simple Design

I Do the simplest thing that could possibly workI Don’t design for tomorrow — you might not need it

I Recall from the last lecture YAGNI

I Extra complexity added “just in case” will:I fossilise your designI For example your class hierarchies get in the way of the

changes you will need to make tomorrow


6. Refactoring

I When tomorrow arrives, there will be a few cases where youactually have to change the early simple design to a morecomplicated one

I Change cannot occur only through small, scattered changes,because over time such changes will gradually turn the designinto spaghetti

I To keep the design modifiable at all stages, XP relies oncontinuous refactoring

I improving the design without adding functionalityI Refactoring may fix bugs, but the intention is that the

behaviour is exactly the same, and bugs fixed are only done soas an unintended consequence


7. Collective Ownership

I Anyone is allowed to change anyone else’s code modules,without permission, if he or she believes that this wouldimprove the overall system

I To avoid chaos, collective ownership requires a good revisioncontrol (configuration management) tool, but those are nowwidely available

I especially see git blame

I This also helps to keep each developer’s “bus factor” low


8. Coding Standard

I Since XP requires collective ownership (anyone can adaptanyone else’s code) the conventions for writing code must beuniform across the project

I This requires a single coding standard to which everyoneadheres


9. Continuous Integration

I Integration and full-test-suite validation happens no morethan a day after code is written

I This means that individual teams don’t accumulate a libraryof possibly relevant but obscure code

I Moreover, it enables everyone to freely modify code at anytime, because they know that they have access to the latestdesign

I Even more extreme is the idea of Continuous Deployment ,this has advocates but even those generally agree it isn’tsuitable for all projects


10. Pair ProgrammingI All code is written by a pair of people at one machine.

I One partner is doing the codingI The other is considering strategy

I Is the approach going to work?I What other test cases might we need?I Could we simplify the problem so we don’t have to do this?

I This is unpalatable to some but appears vital to the XPmethod, because it helps make collective code ownership work

I Many also say it is a very enjoyable way to code


11. 40 Hour Work Week

I XP is intense so it is necessary to prevent “burnout”I Designers are discouraged from working more than 40 hours

per weekI which is low compared to the rest of the software world!

I If it is essential to work harder in one week then the followingweek should drop back to normal (or less)


Problems with XP

I Published interfaces (e.g. APIs): some code is not practical torefactor, because not all uses can be known, so that codemust anticipate all reasonable tomorrows

I Many programmers resist pair programming or other XPguidelines; teams are often spread geographically, and even atone site, sharing a computer is often awkward

I You may have/covet a corner desk but they are awful for evenshowing someone your work much less pair programming

I The customer isn’t always available or willing, and may not beable to agree to an open-ended process

I Over time XP has become more heavyweight, trying toincorporate new realisations, just as Unified Process did


Problems with XP

I XP is deliberately ‘low ceremony’ and will therefore notnormally produce documents describing its architecture, forexample.

I Needs special care to ‘wrap up’ properly when the team isdisbanded — all crucial information must be visible in thecode, tests or other durable deliverables

I If this is not arranged, long-term support and maintenance ofan XP-developed product can be problematic

I Additionally you may fail to produce data for future projects


The rules

I These are rules or guidelines standard for a project using whatit calls an XP methodology

I Of course project managers are free to pick and choose whichof those rules to use and which to leave out

I It is important that the project manager makes clear what therules and guidelines are

SAPM

Extreme Programming Offshoots

I Extreme programming has fostered many developmentpracticies some of which have matured into consideration as amethodology in their own right

I For example:I ScrumI Test-driven DevelopmentI Lean

I I will briefly talk about TDD

SAPM

Test Driven Development

SAPM

Test-Driven DevelopmentI Advantages

I Aids in refactoringI Version control can be used to indicate the changes to code

which caused a test to go from green to redI Tests become an executable specification of the applicationI Can be useful for debugging legacy code

I DisdvantagesI Tests are yet more code to maintainI Changing may requirements require changes to testsI Tests written by the same developer as the code may share the

same blind spots

SAPM

Agile Methodologies and the Web

I With the growth in popularity of the web agile methodologieshave become more popular/relevant

I Some factors I believe have contributed to this:I Web application deployment is easier, no CDs etcI You can ensure no-one is using and old version

I A new version is downloaded for each visit and/orI The running code is on the server

I Feedback from users is easier to gatherI Sometimes even automagically, for example “Does anyone

ever click here/use this feature?”

I Easier deployment has meant greater competition and morefrequent changes to requirements

I Security fixes must often be released with extreme haste

SAPM

Methodologies and MeI I have probably betrayed something of a bias towards agile

methodologies rather than heavyweight methodologiesI I believe the waterfall method is always inappropriateI I could believe The Unified Process, or some other heavyweight

iterative methodology, has some appropriate applications,though I’m (near-)convinced these are few

I Objections to agile methodologies for a specific project oftencome in the form of: “Half a product is of no business valueto us and cannot be tested”

I For example the control software for a nuclear power-plant orMars rover

I I argue that such efforts can be simulated and half productsthereby tested

I My major concern is that requirements do change, so if yourmethodology should not rely on that not being the case

SAPM

Summary

I Methodologies: principled ways to manage large projects

I Waterfall model works in other disciplines, where most of thework is on the physical implementation, but in softwareengineering nearly all work is conceptual

I Unified Process constructs gradually more elaborate models touncover risks and solidify requirements and design as early aspossible

I Extreme Programming relies on continuous customerinvolvement, testing, and refactoring to deliver code early andcontinuously, minimising risk of complete failure

I We have done overviews only — you will need to read more toactually implement any process

I Though I may be convinced to do a more in-depth ExtremeProgramming/Agile lecture

SAPM — Related Reading

I Required ReadingI B. W. Boehm. A Spiral Model of Software Development and

Enhancement. IEEE Computer, May 1988.I B. Boehm. Get ready for agile methods, with care. IEEE

Computer 35(1):64-69, 2002.

I Suggested ReadingI M. C. Paulk. Extreme Programming from a CMM

Perspective. IEEE Software, November/December 2001I P. Schuh. Recovery, Redemption, and Extreme Programming.

IEEE Software, November/December 2001.I Brooks 1982, Mythical Man-month , ISBN 0201835959 (for

updated 1995 edition).I Brooks 1987, No Silver BulletI ...

SAPM

Don’t Forget

I Next week is Innovative Learning Week

I http://www.inf.ed.ac.uk/student-services/teaching-organisation/informatics-innovative-learning-week-2013

http://www.inf.ed.ac.uk/student-services/teaching-organisation/informatics-innovative-learning-week-2013



Any Questions

Any Questions?

Documents

Software Architecture, Process and Management · SAPM: Waterfall Problems with the Waterfall Model I In my opinion, the waterfall model is simply a fundamentally awed metaphor for