Agile Methods and the CeBASE Method Dan Port, USC Vic Basili, UMD

Agile Methods and the CeBASE Method

Dan Port, USC

Vic Basili, UMD

Outline

• Overview of the CeBASE Method

• The Challenge of Application to CS577

• Our track record

• Critical Agility Strategies– Risk driven specifications and modeling

– Rapidly achieving shared vision, tacit knowledge

– Experience Factory and knowledge reuse

CeBASE Method Overview• We needed to create a unified framework for empirical SE

– Reconciled our processes as well as our data definitions

• We found that EF/GQM and MBASE/Spiral were compatible and complementary– GQM Goals map to Spiral Objectives

– GQM Questions & Metrics map to Spiral evaluation of Alternatives

– MBASE focuses on projects; EF covers organizations as well

• Integrated CeBASE Method presented at STC 2000 by Boehm and Vaughn– Led to adoption by several DoD organizations, including

Army/DARPA’s biggest software project

• Org. Improvement Goals– Goal-related questions, metrics

• Org. Improvement Strategies– Goal achievement models

Org. Shared Vision & Improvement Strategy

Experience Factory Framework - I

Initiatives

Planning context

Progress/Plan/ Goal Mismatches

Experience Base

Analyzed experience, Updated models

Achievables, Opportunities



Org. Improvement Initiative Planning & Control

• Initiative Plans– Initiative-related questions,

metrics

• Initiative Monitoring and Control

– Experience-Base Analysis


Experience Factory Framework - II

InitiativesPlanning context


Experience Base

Analyzed experience, Updated models

Achievables, Opportunities



Org. Improvement Initiative Planning & Control

• Initiative Plans– Initiative-related questions,

metrics

• Initiative Monitoring and Control

– Experience-Base Analysis


Experience Factory Framework - III

Project Shared Vision and Strategy

Planning Context

Models and data

Project experience

Org. Goals

Project Planning and Control

Models and data


The CeBASE Method

1. Org. Value Propositions (VP’s)a-Stakeholder values

2. Current situation w.r.t. VP’s3. Improvement Goals, Priorities4. Global Scope, Results Chain5. Value/business case models

Org-Portfolio Shared Vision

1. Strategy elements2. Evaluation criteria/questions3. Improvement plans

a. Progress metricsb. Experience base

Org. Strategic Plans

Organization/Portfolio:

ExperienceFactory,GMQM

1. Monitor environment-Update models

2. Implement plans3. Evaluate progress

-w.r.t. goals, models4. Determine, apply

corrective actions5. Update experience base

Org. Monitoring & Control

Monitoring& ControlContext

1. Project Value Propositions a-Stakeholder values2. Current situation w.r.t. VP’s3. Improvement Goals, Priorities4. Project Scope, Results Chain5. Value/business case models

Project Shared Vision

Project:MBASE

Planningcontext

Plan/Goal mismatches

Project Plans

PlanningContext

Initiatives

OFB: Progress/Plan/Goal mismatches-shortfalls, opportunities,

risks

Projectvision,goalsO-PSV

Shortfalls,opportunities,risks; P-OSV

Scopingcontext

Shortfalls,opportunities,

risks: P-OP

PlanningContext(O-PP)

1. Monitor environmenta-Update models

2. Implement plans3. Evaluate progress

-w.r.t. goals, models, plans

4. Determine, apply corrective actions5. Update experience base

Proj. Monitoring & Control

Monitoring& Controlcontext

PFB: Progress/Plan/goal mismatches

-Shortfalls, opportunities, risks

Plan/goal mismatches

Monitoring& Controlcontext

Projectexperience,

progress w.r.t.plans, goals

LCO: Life Cycle ObjectivesLCA: Life Cycle ArchitectureIOC: Initial Operational CapabilityGMQM: Goal-Model-Question-Metric ParadigmMBASE: Model-Based (System) Architecting and Software Engineering

-Applies to organization’s and projects’ people, processes, and products

Project Plans (PP)1. LCO/LCA Package

-Ops concept, prototypes, rqts, architecture, LCplan, rationale

2. IOC/Transition/Support Package-Design, increment plans, quality plans, T/S plans

3. Evaluation criteria/questions4. Progress metrics

Outline







The CS577 Challenge• 20 USC eServices Applications

– 2 sentence problem statements

– USC Information Services clients

• 100 Graduate Students– 30% with industry experience

– Largely unfamiliar with each other, Library ops.

* Develop LCA packages in 12 weeks• Re-form teams from 30-40 continuing students* Develop IOC packages in 12 more weeks

– Including 2-week beta test and transition

Application of CeBASE• Within the constraints of the previous slide, student

teams must:– Rapidly assimilate the key stakeholders organizational

shared vision

– Rapidly converge on an organization shared vision with key stakeholders

– Understand and align project with the organizations strategic plans

– Formulate and execute feasible project plans

– Adapt to frequent client changes

– Transition a system in alignment with organization monitoring and control process

Impossible?

Focused Represen- O & M Collabo- Domain Clienttative Resources rative Knowledge Envir. Success

EDGAR Business Data + + + + + + +Medieval Manuscripts + + + + + +Technical Reports + + + + + +Latin American Pamphlets + + + + + + +Cinema-TV + + + + + + + (+) + (+)Image Archives + + +

S-Charts + + + + + + (+) + (+) + (+)Global Express + + + + + + + +Hancock Virtual Museum + + (+) + + + + + +Serial Control Records + + + + + + + + (+) + (+)B-School Working Papers + + + + + + + + + + +

Data Mining + + + + + + + + (+) + (+)Dissertations + + + + + + + (+) + + (+)Hispanic Archive + + + + + + + + +WWI Archive + + + + + + + + + (+)

1996-97

1997-98

1998-99

ApplicationOutcome

AdoptedSoftware Site PeopleStable

Client Characteristics Transition Preparation

Critical Success Factors for Adoption - I

Critical Success Factors for Adoption - II

Focused Represen- O & M Collabo- Domain Clienttative Resources rative Knowledge Envir. Success

Oversized Object Viewing + + + + + + + + + + +East Asian Ingest + + + + + + + + + + +New Books List + + + + + + + + (+) + (+)Chicano/Latino Serials + + + + + + (+) + +Vacation/Sick Leave Tracking + + + + + + + + +Business/Reference Q&A's + + + + +

Web Mail + + + + + + + + (+) + +Full text DB Search + + + + + + + + +Dental Journal ToC + + + + + + + + + + +Pathology Slides (+) + + + + (+) (+) (+) (+) (+)Arch/F. Arts Slides (+) + + + + + + + +Velero Archive + + + + + + + + + + +

PeopleStable

Client Characteristics

2000-2001

Transition Preparation

1999-2000

ApplicationOutcome

AdoptedSoftware Site

Summary of Results 1996-2000Metric USC

1996-97

USC 1997-98

USC 1998-99

USC 1999- 00

Columbia U-grad. S99

Columbia Grad. S99

Columbia U-grad. F99

Columbia Grad. F99

Fall Semester: LCA Package

Teams 15 16 19 22 20 13 10 7 Students 86 80 102 100 107 59 44 26 Applications 12 15 17 22 10 10 10 7 Teams failing LCO review

4 4 1 1 10 6 5 1

Teams failing LCA review

0 0 0 0 0 1 1 0

Pages, LCO package 160 103 114 - 124 116 107 95

Pages, LCA package Client

230 154 167 - 142 142 140 109

Evaluation (1-5, 5 best)

4.46 4.67 4.74 4.48 - - - -

Spring Semester: IOC Package

Teams 6 5 6 8 Students 28 23 28 35 Applications 8 5 6 8

Remained the same since projects were only one semester long

Teams failing IOC acceptance review

0 0 0 1 0 0 1 0

Applications satisfying clients (*teams)

5 5 6 7 20* 12* 10* 7*

Applications not overtaken by events

6 4 4 4 10 9 10 6

Applications continued 3 3 4 4 2 3 1 2 Applications used 1 3 3 5 10 5 7 3 Client evaluation - 4.15 4.3 4.75 4.44 4.21 3.9 4.38

Outline







Risk-Driven Specifications and Activities

• Basic driving principle for CeBASE activities and specifications (i.e. modeling, model content, degree of detail, etc.)

If it’s risky to do something, Don’te.g. specify firm GUI requirements early

If it’s risky not to do something, Doe.g. document shared protocols

• Seems obvious, but often not explicitly done or managed!– People must be educated to perform effective risk

identification, assessment, mitigation, prioritization, tolerance– Varies considerably over the project and people

Tactical and Strategic Risk Management

• CeBASE makes use of both tactical and strategic risk management

• Tactical Risk Management– Rmall-scale actions serving to contain or respond to risks made or

carried out with only a limited or immediate end in view • Risk identification and assessment

• Top-10 risk monitoring (EF)

• Risk contingency plans

• Strategic– Risk management as an integrated whole or to a planned effect

(e.g. Expected Return on Investment)• Risk driven “how much is enough?” approach

• Risk/value based feasibility assessments (GQM)

• Risk based development processes (e.g. SAIV)

Rapidly achieving shared vision and tacit knowledge

• Critical challenge is to converge on a shared vision for the project within 12 weeks

– Involves many factors such as teambuilding, stakeholder identification, requirements solicitation and negotiation, domain modeling, etc.

• CeBASE uses many agile approaches to building shared vision and experience factory techniques to utilize tacit knowledge

– Win-win requirements negotiation, early prototyping, group planning exercises, stakeholder lists, results chains

Results Chain: Hispanic Digital Archive (HDA)

Major donor

funding

Viable HDA

Archive

Digitize HDAArchive

ViableIBM DL

package

HDA PR, training for USC, community

Develop HDA Software

Assumption

Outcome

Initiative

Contribution

Staff, trainHDA

Ops/Maintpersonnel

Viable HDA

System

DigitalHDA assets

Sustainable HDA Archive

World-class

Hispanicresearch, education,outreachsupport

CS577 Experience Factory• Teams are supplied with and educated in

the use of an experience base with:– Domain model descriptions

– Pre-architectures

– Specialized WinWin taxonomies

– Specialized top-n risks

– Previous project examples

• Each year the experience base is updated– E.g. in 1998 we added “simplifier and complecators” to reduce LCO

failure rate

Example S&C’s

1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 31, 32, 35, 36, 37, 39

Type ofApplication

Simple Block Diagram Examples Simplifiers Complicators

MultimediaArchive

Use standardquery languages

Use standard orCOTS searchengine

Uniform mediaformats

Natural languageprocessing

Automatedcataloging orindexing

Digitizing largearchives

Digitizingcomplex or fragileartifacts

Automatedannotation/description/ or meaningsto digital assets

Integration oflegacy systems

MM assetinfo

Catalog

MMArchive

query

MM assetupdate

query updatenotification

Rapid access tolarge Archives

Access toheterogeneousmedia collections

The Results• Projects That Failed LCO Criteria

- 1996: 4 out of 16 (25%)- 1997: 4 out of 15 (27%)

- 1998: 1 out of 20 (5%)

- 1999: 1 out of 22 (4%)

• 40% of Student critiques cited S&C’s as helpful (and more since)- In focusing on achievable requirements set within

tight schedule- In understanding project risks and tradeoffs

Summary• Overall success rate 92%

– compared with 26% Standish

• Primarily agile approaches used– Scenarios, prototypes, patterns, metaphors

• Primarily plan driven approaches used– Risk driven requirements, life cycle architecture,

stakeholder roles and responsibilities, feasibility rationale, risk management plan

• CeBASE not one size fit all– Risk tailoring produces appropriate balance of

discipline and flexibility

Backup Slides

Further CS577 CeBASE Experience Factory examples

Examplewin-win

taxonomy

1. Operational Modes 1.1 Classes of Service (research, education, general public) 1.2 Training 1.3 Graceful Degradation and Recovery

2. Capabilities 2.1 Media Handled

2.1.1 Static (text, images, graphics, etc.) 2.1.2 Dynamic (audio, video, animation, etc.)

2.2 Media Operations 2.2.1 Query, Browse 2.2.2 Access 2.2.3 Text Operations (find, reformat, etc.) 2.2.4 Image Operations (zoom in/out, translate/rotate, etc.) 2.2.5 Audio Operations (volume, balance, forward/reverse, etc.) 2.2.6 Video/Animation Operations (speedup/slowdown, etc) 2.2.7 Adaptation (cut, copy, paste, superimpose, etc.) 2.2.8 File Operations (save, recall, print, record, etc.) 2.2.9 User Controls

2.3 Help 2.4 Administration

2.4.1 User Account Management 2.4.2 Usage Monitoring and Analysis

3. Interfaces 3.1 Infrastructure (SIRSI, UCS, etc.) 3.2 Media Providers 3.3 Operators

4. Quality Attributes 4.1 Assurance

4.1.1 Reliability/Availability 4.1.2 Privacy/Access Control

4.2 Interoperability 4.3 Usability 4.4 Performance 4.5 Evolvability/Portability 4.6 Cost/Schedule 4.7 Reusability

5. Environment and Data 5.1 Workload Characterization

6. Evolution 6.1 Capability Evolution 6.2 Interface and Technology Evolution 6.3 Environment and Workload Evolution

Example top-n Risks

Source of Risk Risk Management Techniques

1) Performance risks for image/video distribution systems

Simulation; benchmarking; modeling; prototyping; instrumentation; tuning

2) Finding a proper search engine

Software evaluation of search engines, prototyping, experience factory investigation

3) Legacy software integration

Reengineering; code analysis; interviewing; wrappers; incremental deconstruction

4) Information Systems Division limitations

Interviewing, alternative analysis, benchmarking

5) Effective indexing and access of assets

Technical analysis, prototyping, modeling, tuning

6) Digitizing complex/fragile assets

Effort/schedule estimation, equipment analysis, benchmarking, instrumentation; tuning

Multimedia Archive Risks

Exampleprevious project

data

Multimedia Archive Example Projects: 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 31, 32, 35, 36, 37,39, 47, 48, 50, 55, 57 COTS:

Name Vendor address or URL Project Uses Mr. Sid http://www.roktech.net

/ROK/docs/ Products/MrSid.cfm

Serving and viewing of large images in a web browser. Provides image manipulation functions such as zoom, rotate, select region. Used in projects 47,..

MS Access Microsoft A database management system. Provides facility to organize data, find and retrieve information effectively. Used in projects 10,..

BANS http://www.bans.com Operates by connecting to your mail server and filtering each waiting message, searching for what you specify. Used in projects 41,...

… … … DSDP: Model View Controller, Façade, Composite, Reactor Common Pitfalls: Dependency on the use of cgi scripts, ISD limits the use of these scripts due to security reasons; Plan your schedule taking into delays in the negotiation activity; Restricted usage of legacy software due to unexpected occasions, Integrated Library System’s test server could not be used for prototyping because it was needed in transitioning from old to new system; Lack of required expertise in library information systems; Doing software evaluation for Search Engine without investigating the experience factory, ….

Domain ModelsSub-domain Description 1) Multimedia Archive Provides a user interface for a collection of

multimedia content) 2) Selective Dissemination of Information Distributes content according to user interests and

selection rules 3) Data Analysis Process data from multiple sources and reporting 4) Activity Monitoring and Control Implements agents that invoke policy in response to

events and provides status reporting and activities management

5) Automated Reference Services Provides a uniform source for frequently requested, relatively static information

6) Data Migration Aggregates and converts data from one format to another

7) Virtual Access to Special Collection Provides a virtual environment that implements access policies for multimedia content

8) COTS Package Extension Externally adds capabilities to COTS 9) Distributed Borrowing Provides an interface to manage non-digital assets

and implements a borrowing policy 10) Interactive Communication Provides interactive user interface for rich media

access

ExampleDomain Model

S y s t e m B l o c k D i a g r a m : T h i s d i a g r a m s h o w s t h e u s u a l b l o c k d i a g r a m f o r e x t e n s i o n s p r o v i d i n g a c c e s s t o n e w i n f o r m a t i o n a r c h i v e a s s e t s f r o m a n e x i s t i n g i n f o r m a t i o n a r c h i v e ( I A ) S y s t e m : I A S y s t e m O & M S u p p o r t

N e w A s s e t A c c e s s

E x i s t i n g I A S y s t e m

U s e r s

I A S y s t e m I n f r a s t r u c t u r e

N e w A s s e t N e w A s s e t M a n a g e r s

E x i s t i n g A s s e t

E x i s t i n g A s s e t M a n a g e r s

I A S y s t e m I n f r a s t r u c t u r e O p e r a t i o n s a n d M a i n t e n a n c e ( O & M )

T h e s y s t e m b o u n d a r y ( m a r k e d b y t h i c k e r b o u n d ) f o c u s e s o n t h e a u t o m a t e d a p p l i c a t i o n s p o r t i o n o f t h e o p e r a t i o n , a n d i n c l u d e s s u c h e n t i t i e s a s u s e r s , o p e r a t o r s , m a i n t a i n e r s , a s s e t s , a n d i n f r a s t r u c t u r e ( c a m p u s n e t w o r k s , e t c . ) a s p a r t o f t h e s y s t e m e n v i r o n m e n t . T h e d i a g r a m a b s t r a c t s o u t s u c h c a p a b i l i t i e s a s a s s e t c a t a l o g u e s a n d d i r e c t u s e r a c c e s s t o O & M s u p p o r t a n d a s s e t m a n a g e r s . S o m e S t a k e h o l d e r R o l e s a n d R e s p o n s i b i l i t i e s

A s s e t M a n a g e r s . F u r n i s h a n d u p d a t e a s s e t c o n t e n t a n d c a t a l o g u e d e s c r i p t o r s . E n s u r e a c c e s s t o a s s e t s . P r o v i d e a c c e s s i b i l i t y s t a t u s i n f o r m a t i o n . E n s u r e a s s e t - b a s e r e c o v e r a b i l i t y . S u p p o r t p r o b l e m a n a l y s i s , e x p l a n a t i o n , t r a i n i n g , i n s t r u m e n t a t i o n , a n d o p e r a t i o n s a n a l y s i s .

O p e r a t o r s . M a i n t a i n h i g h l e v e l o f s y s t e m p e r f o r m a n c e a n d a v a i l a b i l i t y . A c c o m m o d a t e a s s e t a n d s e r v i c e s g r o w t h a n d c h a n g e . P r o t e c t s t a k e h o l d e r p r i v a c y a n d i n t e l l e c t u a l p r o p e r t y r i g h t s . S u p p o r t p r o b l e m a n a l y s i s , e x p l a n a t i o n , t r a i n i n g , i n s t r u m e n t a t i o n , a n d o p e r a t i o n s a n a l y s i s .

U s e r s . O b t a i n t r a i n i n g . A c c e s s s y s t e m . Q u e r y a n d b r o w s e a s s e t s . I m p o r t a n d o p e r a t e o n a s s e t s . E s t a b l i s h , p o p u l a t e , u p d a t e , a n d a c c e s s a s s e t - r e l a t e d u s e r f i l e s . C o m p l y w i t h s y s t e m p o l i c i e s . P r o v i d e f e e d b a c k o n u s a g e .

A p p l i c a t i o n S o f t w a r e M a i n t a i n e r . P e r f o r m c o r r e c t i v e , a d a p t i v e a n d p e r f e c t i v e ( t u n i n g , r e s t r u c t u r i n g ) m a i n t e n a n c e o n s o f t w a r e . A n a l y z e a n d s u p p o r t p r i o r i t i z a t i o n o f p r o p o s e d c h a n g e s . P l a n d e s i g n , d e v e l o p , a n d v e r i f y s e l e c t e d c h a n g e s . S u p p o r t p r o b l e m a n a l y s i s , e x p l a n a t i o n , t r a i n i n g , i n s t r u m e n t a t i o n , a n d o p e r a t i o n s a n a l y s i s .

S e r v i c e p r o v i d e r s ( e . g . n e t w o r k , d a t a b a s e , o r f a c i l i t i e s m a n a g e m e n t s e r v i c e s ) . S i m i l a r r o l e s a n d r e s p o n s i b i l i t i e s t o A s s e t M a n a g e r s .

ExamplePre-

architectures

1) Multimedia Archive

2) Selective Dissemination of Information

3) Data Analysis

4) Activity Monitoring and Control

5) Automated Reference Services

6) Data Migration

7) Virtual Access to Special Collection

8) COTS Package Extension

9) Distributed Borrowing

MM asset info

Catalog

MM Archive

Query

MM asset Update

Query Update Notification

Filter

Info Base

Interest

New assets

New asset notification

New assets of interest

Analysis Request

Analysis display

Data Sources

Updates

Parser / Searcher

Pattern Detection/Processing

Reporting

Parse Format

Relevant data Queries

Status Reporting

Event

Activity Agent

Status Assessment

Activity

Controls

Monitoring & Control

Policy

Revised controls

Action

Status

non-automated query

Filter

Reference Administration

user input

Reference Repository

policy update references

query

results

filter policy

data out

Data Assimilation

data in

Parser data in

Converter

parse format

reporting

Policy

Generator

policy

Collection Access

user

Collection Media

Display Policy

Data, Controls Preparation

user input

COTS Package

output

Converter

input

to user

asset request

asset

Locator Asset Source

policy request

asset

Specialized S&C’s

Simplifiers Risks and Trade-offs

Generic Uniform Media FormatsSpecific All video clips are stored using an open file format for video/audio (e.g., MPEG). All film stills are stored using an open image file format (e.g., JPEG). The inverse complicator is to store film clips using streaming video technologies

This means that we may have to convert existing digital assets or digitize the original media, which may be costly. A unique file format limits the user base to those who have viewers for that particular file format The chosen file format may not be the most efficient for the various types of media (in terms of compression rates, quality, etc...)

Generic Use Standard Query LanguagesSpecific Organize catalog and archive relationally so that queries will be limited to standard search formats,: match exactly by value on any of the fields with or without using boolean combinations (AND, OR, NOT, etc...), or using pattern matching (SQL LIKE keyword)

May not be as effective for "discovering" assets in the archive: users must know what they're looking for, in order to search for it

Generic Use Standard COTSSpecific Use a standard Relational Database Management System (RDBMS) that supports storing multi-media assets

A Relational Database Management System may not be most suited for archival of multi-media assets. A Relational Database Management System may have a high initial cost, high implementation, and high administration cost (requires specialized knowledge skills)

Complicators Risks and Trade-offs

GenericNatural Language ProcessingSpecificStore the information only in one language (e.g., English) and provide dynamic translation into Chinese, Japanese and Korean The inverse simplifier is to store the same information in 4 different languages (English, Chinese, Japanese and Korean).

The first approach is a complex, error-prone, expensive natural language processing issue The second approach will require more storage space, in addition to acquiring the translations

GenericDigitizing Large ArchivesSpecific Digitizing film clips from the entire collection of films (which grows at a very fast rate of 800 films per year for Indian films alone)

If each film clip requires around 10 MB, then the rate of growth of the database will be of 8GB a year (exclusive of catalog information)

GenericIntegration of "Legacy" SystemsSpecific Do not require Real-Video plug-in for Web browsers to allow users to view streamed film clips

We cannot use more effective multi-media formats, which are becoming standard technologies

Documents

Agile Methods and the CeBASE Method Dan Port, USC Vic Basili, UMD