View
30
Download
0
Category
Preview:
DESCRIPTION
Is Agility Useful In Military Software Projects?. Terry Shepard Electrical and Computer Engineering Royal Military College of Canada. University of Toronto, 6 July 2007 shepard@rmc.ca 613-541-6000 ext. 6031. - PowerPoint PPT Presentation
Citation preview
Is Agility Useful In Military Software Projects?
University of Toronto, 6 July 2007 shepard@rmc.ca 613-541-6000 ext. 6031
Terry Shepard
Electrical and Computer Engineering
Royal Military College of Canada
Terry Shepard RMC2
An Aside:Software Engineering at RMC
• 4 civilian profs, 3 military profs– 7 in ECE, 1 in Math & CS
• one emeritus prof (as of today!)• undergrad option as part of computer engineering
– small classes!
• thesis and project pattern master’s degrees• PhD: 3 grads so far (examining 4th one July 13)• may have graduated the first Software Eng’g PhD
in Canada (2004)
Terry Shepard RMC3
from the Abstract
• Three obvious characteristics of [agile] projects are that – a single representative customer can speak with
authority on behalf of all stakeholders, – development is a collaborative effort and not
adversarial, – the size of the group is small enough that the
customer can always be available to any developer with a question
Terry Shepard RMC4
‘a single representative customer can speak with authority on behalf of all stakeholders’
• works when there is a clearly defined user community, with relatively uniform needs, and few unresolved issues among the stakeholders
• consider two examples– automobile parts warehouse in Pakistan (case
study from a grad student for a requirements course at Queen’s)
– Canadian Land Forces command and control system acquisition
Terry Shepard RMC5
One Warehouse: simple ?
At the start: very limited computer
use…
Example 1
Terry Shepard RMC6
Warehouse Stakeholders – not so simple!
There were six (6) directly and six (6) indirectly related stakeholders in this warehouse situation:
1. Inventory Management2. Order Placement3. Audit4. Sales5. Management Tier6. Invoicing Department7. Customers8. General Supplier9. Parts Manufacturer10. Garage11. After Market Manufacturers/Refurbishers12. Transportation and Delivery
Terry Shepard RMC7
Warehouse example illustrates the role of a champion
• champion can act as a surrogate customer– gathers information from all the stakeholders– synthesizes the real needs, how to keep
everyone happy– champion can then act as a single customer
point of contact in an agile development team
• role of champion can also be used to capture requirements in more conventional ways
Terry Shepard RMC8
Example 2: Canadian Land Forces command and control system acquisition
• more stakeholders than the warehouse example
• operational commanders in the field are key– needed to field early versions of system in
training exercises – another kind of agility– didn’t happen
• delivery of whole unusable system in 2000– portions are starting to be used in Afghanistan
• Master’s thesis in progress on the 50 year history of these requirements
Terry Shepard RMC9
‘development is a collaborative effort and not adversarial’
• a major challenge in most military software projects
• consider two examples:1) military officers embedded in supplier
organization while contract is in progress
2) writing requirements while the contractor is already writing code to their internal requirements
Terry Shepard RMC10
Example 1) : military officers embedded in supplier organization while contract is in progress
• consider Brook’s Law: “adding more people to a late project makes it later”– even more true if development is agile?– agility might demand that an officer try to stop
the people from being added• corporate and military chains of command have to
be agile• in the specific case, they were not
• hierarchical structure on both sides of the contract can be adapted to agility?– what does agility mean?
Terry Shepard RMC11
many senses of agile/agility
• narrow sense refers to the ‘Agile Manifesto’ for software development (next slide)
• broader sense refers to flexibility in an organization, ability to respond to change, management openness to decisions being questioned– can be a recipe for inaction
• military in action tends to be highly agile, less so in peacetime– e.g. land forces doctrine changes normally take months or
years, but only weeks or even days in Afghanistan
Terry Shepard RMC12
Agile Manifesto
“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and toolsWorking software over comprehensive documentation
Customer collaboration over contract negotiationResponding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.”
Terry Shepard RMC13
Example 2) : writing requirements while the contractor is already writing code to their
internal requirements
• it is often the case in military software contracts that there are two or more parallel requirements documents– e.g. CCS 330 command and control system had
about 3000 requirements written by DND• parallel 14 volume Combat System Users Manual
(CSUM) written by Lockheed Martin
• a number of defects reference incompatibility between the two – CSUM tends to reflect reality
• CSUM needed for training and operations
Terry Shepard RMC14
Halifax Class Frigate – CCS 330(entered service in 1989)
Terry Shepard RMC15
recent RMC Master’s thesis on CCS 330
• Exploration Of A Heterogeneous Tracing Strategy To Support Naval Command And Control Software – by Paul Mondoux– retrospective study tracing between known
defects and requirements
• are tracing and agility compatible?• introducing heterogeneity is a form of
agility
Terry Shepard RMC16
Mondoux Thesis AbstractRequirements traceability aims to improve project communication, to
support dealing with the effects brought upon by change, to preserve system and decision knowledge and to help assure quality.
Even though mandated for Naval command and control projects, there remains poor software requirement traceability take-up due to a perceived lack of benefit in its use. Static link based traceability schemes, if not properly conceived and maintained, can create an
unmanageable tangle of links, often providing little to no return on the time and money invested. Based on analysis of a subset of the
roughly 3000 individual requirements for the Navy’s CCS330 command and control system, this thesis will investigate whether a heterogeneous tracing strategy based on requirement-defect risk analysis may be an alternative to traditional link-based traceability
strategies. Attributes considered include probability of defect, degree of impact, standards/mandates, evolution/maintainability of requirements, requirement type (Functional/Non-Functional) and granularity. A successful application of a heterogeneous tracing strategy will simplify tracing by better matching link types to
requirement types while minimizing cost and maximizing value as compared to a link based strategy and a no trace strategy.
Terry Shepard RMC17
Heterogeneous Tracing
[from Mondoux thesis]
Terry Shepard RMC18
‘the size of the group is small enough that the customer can always be available to any
developer with a question’
• many project leaders have had very good success scaling this one up
• solution seems to be to do some serious planning up front to allocate the work to smaller teams
• for example, article by Philippe Kruchten now of UBC, on ‘Scaling down large projects to meet the agile sweet spot’
Terry Shepard RMC19
article by Philippe Kruchten (1)• “This initial architecture team proceeds
iteratively, and the team gradually grows in size. When this team reaches fifteen members or more, they split into:– The architecture team, mostly customer
involved, and making key architectural choices– The prototyping team, gradually building up
and testing the architectural prototype, refactoring it, and pushing back up any issues detected, thus closing the loop”
Terry Shepard RMC20
article by Philippe Kruchten (2)“ The architecture team (seven to twelve people) is
the customer for two or three prototyping teams (consisting of thirty-five or more people).
“ Staffing the project: Construction and Transition– Towards the end of the elaboration phase, when the
structure of the code starts to take shape and stabilize, additional teams are created, each one "owning" a chunk of the architecture. The implementation view (according to the "4+1 view model" serves as the basis for the partition of the code and the allocation of ownership to a set of teams. Each chunk of code now belongs to one and only one team.”
Terry Shepard RMC21
article by Philippe Kruchten (3)
“ Ideally, each of the newly created teams is seeded with one or two members of the original architecture and prototyping teams to bring some of the project knowledge and emerging culture. Typically, each of these seed members is the new team's "technical lead." This person will play the role of the local architect and serve as the communication conduit to the core architecture team that remains, and, in some cases, with the customer.”
Terry Shepard RMC22
Can military contracts call up this style of agility?
• agility is mostly internal to the contractor– telling the contractor how to manage the
contract internally causes finger pointing
• finding contractors who work this way of their own accord may get easier …– but it’s a very large change for most large
aerospace contractors
Terry Shepard RMC23
Diversity of Military Software Projects
• many shapes and sizes – we will consider a few more …
• some officers develop local DB applications privately – they work and are used – good outcome! – probably agile …– e.g. home grown project tracking system
• agility in projects requiring safety certification– recent RMC Master’s thesis on agility in the context of
DO178B – by Ron Chisholm
• very large projects
Terry Shepard RMC24
Agility in the Context of DO178B
• DO178B is the avionics safety standard enforced by the FAA – 5 levels– four Stages of Involvement by the Certification
Authority
• Agility is compatible with DO178B– but changes are needed eg. to increase the
amount of planning
• Agility may bring benefits– e.g. test cases developed with code– e.g. write requirements near end of each
iteration, when they are stable and complete
Terry Shepard RMC25
A very large project: The Fox is more Agile than the Chickens
• US Coast Guard Deepwater project started in 2002– $24B over 20 years to refurbish whole fleet of
ships and planes– handed requirements responsibility to prime
contractor, who was also to build the ‘system of systems’
• the prime (fox) can eviscerate systems (chickens), thereby improving its financial nourishment
– Puts contractor in position of both specifying what to build, and building it
• Coast Guard has now changed the terms of the prime contract
Terry Shepard RMC26
Deepwater Planned Assets
Terry Shepard RMC27
Deepwater is Software?
• Integrated Deepwater System Program, often simply called "Deepwater" or "IDS," is the largest and most innovative acquisition in the Coast Guard's history. Deepwater is not just "new ships and aircraft," but an integrated, performance-based approach to upgrading existing assets while transitioning to newer, more capable platforms, with improved systems for Command, Control, Communications and Computers, Intelligence, Surveillance, Reconnaissance (C4ISR) and innovative logistics support
Terry Shepard RMC28
C4ISR
Terry Shepard RMC29
“Performance Based Acquisition”“… Deepwater focuses on system-wide capabilities and not
assets. The Coast Guard began the design process with the goal to acquire the performance capabilities required to execute the full range of Coast Guard deepwater missions. Traditionally, acquisition programs have replaced a single type of asset - a type of cutter, for example, or class of helicopter - on a one for one basis. The Deepwater Program takes a different approach - a system-of-systems perspective. The Coast Guard is focusing on the overall required capabilities - the entire system’s capabilities and the life cycle costs, including people, equipment, infrastructure, and operations - rather than the individual assets. Performance-based acquisitions are consistent with the Budget and Performance Integration initiative of the President’s Management Agenda. In simple terms, this initiative states that the Government should be results-oriented and should be guided by performance not by process.”
Terry Shepard RMC30
Department of Homeland Security (DHS) Office of Inspector General (OIG)
Audit released in Aug. 2006
• DHS OIG conducted an audit in Aug. 2006 of “the development and implementation of information technology (IT) systems to support the U.S. Coast Guard’s Integrated Deepwater System program.”
• Purpose was “to evaluate the effectiveness of Coast Guard efforts to identify C4ISR systems requirements, and to determine how well these systems are being implemented to support the Deepwater program.”
Terry Shepard RMC31
Sample Audit Finding 1
“ agency officials are involved in high-level Deepwater requirements definition processes, they have limited influence over the decisions that the contractor makes toward meeting those IT requirements. Due to a lack of discipline in adhering to systems requirements change management processes, there is little assurance that the requirements are complete or effective in meeting program goals.”
Terry Shepard RMC32
Sample Audit Finding 2
“ many IT requirements documents do not require prior Coast Guard approval. For example, according to one contract official, the contractor generally gives the Coast Guard 30 days to review documents. However, because of a shortage of personnel to conduct the document reviews, the agency has difficulty responding within that time frame.”
Terry Shepard RMC33
Main Lesson to be learned from Deepwater
• Attempted to give contractor maximum flexibility (agility) while reducing oversight
• Balance between oversight and contract freedom must be carefully nuanced– Deepwater had too little oversight– Too much oversight can slow progress to a
crawl
• There are no simple solutions in the management of large projects– ideas of Kruchten and others on large project
agility should be applicable
Terry Shepard RMC34
Can DND be the fox and the contractors be the chickens?
• One tactic:– you have a high level vision of what is needed– put out an RFP to develop requirements– contractors develop their proposals at their own
expense– you get a range of approaches to the next level
of detail in requirements– contract is let to develop the requirements– not clear how well this scales to large projects– feels agile at the RFP stage, less so once
contract is let
Terry Shepard RMC35
What’s wrong with the fox and chickens model?
• One side wins – the other side loses– fox may kill more chickens than it needs right away– farmer may increase security – fox may starve
• Best arrangement is win-win– if any party feels threatened, they will act to protect
themselves– the software agility movement essentially promotes
win-win• win-win is the ideal – practice is more complex• win-win contracting can be especially difficult
Terry Shepard RMC36
Agility, Open Source and Military Software
• It may be possible to combine agility with the approach used in the open source community – code base in continuous evolution– all parties share in that evolution
• These ideas are being considered at US DoD deputy secretary level
Terry Shepard RMC37
Open Technology Development
Roadmap Plan
April 2006
Prepared for:
Ms. Sue Payton
Deputy Under Secretary of Defense
Advanced Systems & Concepts
Terry Shepard RMC38
Open Technology Development – DoD Roadmap Plan
“ OSS and open source development methodologies are important to the National Security and National Interest of the U.S. for the following reasons:– Enhances agility of IT industries to more rapidly adapt
and change to user needed capabilities.– Strengthens the industrial base by not protecting
industry from competition. Makes industry more likely to compete on ideas and execution versus product lock-in.
– Adoption recognizes a change in our position with regard to balance of trade of IT.
– Enables DoD to secure the infrastructure and increase security by understanding what is actually in the source code of software installed in DoD networks.
– Rapidly respond to adversary actions as well as rapid changes in the technology industrial base.”
Terry Shepard RMC39
Open Technology Development – DoD Roadmap Plan
“ Open Technology Development combines salient advances in the following areas:1. Open Standards and Interfaces
2. Open Source Software and Designs
3. Collaborative/Distributive culture and the and online support tools
4. Technological Agility”
Terry Shepard RMC40
Open Technology Development – DoD Roadmap Plan
“ the hinge issues are:– How can DoD leverage military-funded software
development more effectively?– How can OTD’s business-process advantages increase
both the rate of innovation and the sustainability of software developed using DoD funds?
– What changes in acquisition practice and policy may be required to capture the benefit of OTD within and across the Defense Department?
– How can DoD leverage existing external OSS resources?”
Terry Shepard RMC41
Conclusions
• Opportunities exist for agility in military software
• Diversity dominates: agility needs to be applied appropriately
• Evolution will be slow: there is ponderous machinery to change
Terry Shepard RMC42
Requirements Engineering Guide
• Goal: describe what needs to be known to be successful in requirements engineering tasks for DND software intensive projects– ambitious: many levels, layers of complexity
• To have value, the guide needs to be put into initial use and then revised
Terry Shepard RMC43
what people responsible for requirements for military software
projects need to know about requirements engineering
• what can go wrong
• how it can go wrong
• things to do to make it go right
Terry Shepard RMC44
examples of what can go wrong
– too many changes• high costs in adversarial environments
– missing requirements– goldplating– wrong granularity
• untestable requirements
– unsatisfied stakeholders
Terry Shepard RMC45
examples of how it can go wrong
– wrong people– not enough resources– poor judgment– misunderstanding– fox in charge of the chicken coop
Terry Shepard RMC46
things to do to make it go right
– tell truth to power
– know the whole history
– use contracting creatively
– find things that will work
– know who all the stakeholders are, and find out what they know
– iterate: approach the solution gradually
– build trust with the contractor
– work really hard, but pray!
Recommended