Lecture #2 CS 473 EX

February 6, 2016

Agenda

• Course Structure and Expectations • Logistics

– Term Project team, submissions – Assignments – Quizzes

• Module 2 – CSM – Estimation

• Justify, implement and manage a global product development • Solicit, define and scope requirements as part of the product backlog grooming. • Play an effective role of a Software Engineering Manager in a context of IEEE CSDP • Select an estimation method that is appropriate for a specific phase of a product life cycle. Oversee adoption of a consistent methodology to narrow the Cone of Uncertainty. • Support the Scrum delivery framework; become aware of several agile certification paths. • Play a role in a peer review, request and provide constructive and concise comments. • Evaluate software development tools (approved, allowed, restricted), while following the Magic Quadrant technique. Maintain a logical relationship between tools and processes to optimize their variety throughout an organization. • Articulate the strategy for system and unit test leading to continuous integration and delivery. • Structure a project asset library aiming at a single-click navigation to a requested artifact. • Provide leadership to a process program using SEI CMMI as an improvement model.

Learning Objectives Upon successful completion of this course, you will be prepared to:

Introduction

Focus on learning . You should assume that you will be successful with the course and hence, grades will be reflective of that. Focus on those best practices that you learn here and be able to actually apply. This is an overview course. It delves into multiple topics. Buckle up. You cannot afford spending all your time on a single topic. Time box your assignments. This is not about a PhD thesis. This is about becoming aware of an important concept and then moving to the next concept. This course is not about a specific language or a framework. Some assignments and discussions - are optional, aimed to stimulate your thinking Term project brings it all together. All predefined deliverables of a term project are introduced in lectures. How does it all fits together? It is all about how to do software better, about software engineering and program management. You cannot function as a professional software engineer or a project manager if you are unaware of these key concepts covered in the class.

Introduction

Software Configuration Management

Estimation

Peer Reviews

Agile Software Design

SW Tools Evaluation

Unit Test

Test Essentials

Continuous Delivery

Globalization Offshoring Taxonomy

Requirements Backlog

Grooming

Engineering Management Process

Architecture

Process Improvement

1

2

6

5

3 4

Identification , Control, Auditing, Reporting

Do not offshore if unable to monetize

If it’s not on a backlog it does not exist

Myth of organizational hierarchy

Precision equal to accuracy

Wisdom of vertical slices

It is cheaper to find defects early

Low coupling and high cohesion

Scale optimization

Data-driven design

Mocks are not Stubs

Deployment pipeline

Process has no goal in itself

A Glue. A Heart bit. A Librarian.

MET CS473 Principles

Software Configuration Management

Estimation

Peer Reviews

Agile Software Design

SW Tools Evaluation

Unit Test

Test Essentials

Continuous Delivery

Globalization Offshoring Taxonomy

Requirements Backlog

Grooming

Engineering Management Process

Architecture

Process Improvement

1

2

6

5

3 4

Control changes

Document Project Taxonomy

Maintain requirements in canonical form Manage flow of features

Focus your team

Retain historical estimates

Split

Ask for and provide comments

Fit components into architecture

Facilitate convergence of tools

Select best tests

Fail, Pass, Refactor

Merge each commit

Find the shortest path

Separate released from drafts

MET CS473 Responsibilities

Estim

atio

n Peer Reviews

Engineering Management

Software Configuration Management

Requ

irem

ents

Git

Pivo

tal

Vers

ion

One

Ra

lly

Agile Manifesto

SW To

ols

Eval

uatio

n

Process Improvement

Test Essentials

Unit Test

Cont

inuo

us

Deliv

ery

CMM

ISO Process Architecture

Asse

t Lib

rary

UM

L

Low Coupling and High Cohesion Scrum

Backlog Grooming Stor

y Po

ints

Black Swan Cost Of Delay

Cone

Of U

ncer

tain

ty

Mot

ivat

ion Mocks & Stubs

Quality Center Re

gres

sion

MVC

5/20

Rul

e

Defe

ct D

ensi

ty &

Ex

amin

atio

n R

ate

Faults & Failures

SCM

Software Configuration Management

What is SCM ?

There is a certain tradition to the way SCM (Software Configuration Management) is described and taught. SEI CMMI has a process area at level 2 called SCM. Eric Braude book on Software Engineering has a Chapter Six dedicated to this topic. Such long-standing tradition also stems from the unmovable CM audits conducted by government agencies to all their large and small vendors. Whether you use Agile or Waterfall methodology, whether you are a small startup or a Fortune 500 company, the format and template of a CM Plan that you have to produce and demonstrate to an auditor - remain identical.

• Software Configuration management (SCM) is a discipline of maintaining integrity of product parts as they propagate along a project life cycle.

• There are many natural organizational forces that could impede the consistency, completeness and traceability among various product pieces. If no systematic improvement effort is applied, the amount of disorder is bound to increase. This is a fundamental entropy law we learn at high school.

• SCM discipline decreases the risk of losing integrity by establishing the four overriding processes, Identification, Control, Auditing and Reporting.

Four Basic Essential Interdependent Activities It is useful to trace the chain of events of a software change, so to understand the details and mechanics of, first, how does it happen that following these four steps will prevent the un-intended software changes, and second, if you will not follow these steps, then sooner or later the chaos will prevail.

reporting

identification

control

auditing

Configuration Identification – Discipline for identifying the item, its configuration, and documenting its functional and physical characteristics; "What version of the file is this?"

Configuration Control – Creation and exercising of established procedures to classify, approve or disapprove, release, implement and confirm changes to agreed specifications and baselines; "How many changes and which changes went into the latest version of this product?"

Configuration Accounting – Recording and reporting of data concerning configuration identification, approval status of proposed changes and implementation status of approved changes during all phases of the project; "How many files were affected by fixing this one bug?"

Here is the quote from James Bach article "The highs and lows of change control", where he describes the balancing act. Both, too much or too little of a change control could cause issues. Change Control is vital. But the forces that make it necessary also make it annoying. We worry about change because a tiny perturbation in code can create a big failure in the product. But it can also fix a big failure or enable a wonderful new capabilities. We worry about change because a single rouge developer could sink the project; yet brilliant ideas originated in the minds of those rogues, and the burdensome change control process could effectively discourage them from doing creative work.

Configuration Items List Here is an illustration of a CI list that is similar to the one you do at your term project. In a commercial environment this is an extensive list that brings together different Owners and various Repositories. One has to accustom to dealing with multiple tools of a large organization. It is best if a CI List is a report produced auto-magically from various repositories. Although in a real-life-project a manual intervention is common. The list matches contents

of the project repository

The list itself has a version

Ideally, CI List is a reportdump from a repository

File names in repositoryfollow the naming convention from CI list

Some items have no version,since they are not “controlled”

Version, date, other attributesare not a part of the name, sodistributed link remains the same

Baselines confirm traceability between items

One has to decide upfront which items are “controlled” and which ones are “stored”

Version structureshows whether an item is a draft

Example. After a Baseline Audit, all versions are bumped to 5.0

ALL items are listedwithout exceptions

Configuration Item Name Version Date Owner Repository

1 Definition of users (personas) 1.4 9-Sep John S Word file (Drop Box)2 Definition of Scope and Limitations 1.0 9-Sep Mary W Word file (Drop Box)3 Engineering Requirements List 1.3 9-Sep Joe B EXCEL file (Drop Box)4 Estimation Record 2.0 16-Sep Joe B EXCEL file (Drop Box)5 State Transition Diagram 2.1 23-Sep Alex B Power Point (Drop Box)6 Definition of UseCases 1.7 23-Sep John S Visio file (Drop Box)7 Definition of Fields 4.3 23-Sep Harry V EXCEL file (Drop Box)8 Definition of Reports 5.8 23-Sep Harry V Word file (Drop Box)9 Test Cases 0.8 6-Oct Bill K EXCEL file (Drop Box)10 Defect Tracking System 0.4 23-Sep Sam C Web App (Drop Box)11 Defects from testing recorded na na na Defect Tracking System12 Issues from Peer Reviews retained na na na Discussions Blackboard13 Configuration Items List 1.0 9-Sep Peter Z EXCEL file (Drop Box)

QA Engineer

Developer

Submits &Assigns CR

VerifyResolution

start close

Is resolutionacceptable?

SUB1

DEV3

ResolvesCR

QA1

No

New Assigned Resolved

ReOpened

Verified

Closed

Accepts

Pass to someone

else

Already Resolved

Submit new bug

Resolved

Resolution incorrect

Submitter Verifies

Test Case Exists &

Closed for Good

Zombie

Change Requests A simple bug fix Change Management Team (Bug Review)

State Transition Diagram for a Change Request System

QA Engineer

Developer

Submits &Assigns CR

Prioritize

Resolve

accept

What is theoutcome?

Managerejects

reject

start

SUB1

CMT3

CMT2CMT1 CMT4

DEV3

ChangeManagement

Team(BugReview)

defer

Managedeferred list

VerifyResolution

close

Is resolutionacceptable?

QA1

no

SCM Scope - Adoption Chart

Identification Control Auditing Reporting

Code

UserDocs

Internal Docs

Tools

This chart shows four parts of SCM applied to four different types of work products. It could serve as an indicator of adopting best practices. You can see that Code is well-controlled, but the User Docs are not under control.

Baseline Audit (example) A project baseline consists of a set of baselined work products that are required at the end of each phase or at some designated interim quality checkpoint. The project baseline evolves through the life of a project. A baseline audit checks the integrity of each baseline. Artifacts are traceable to each other within each baseline, although might NOT be traceable between different baselines.

PRD ProjPlanERL

FuncSpec

ERLTest

Cases

Requirements Baseline

Development Baseline Baseline Audit• ERL matches Test Plan• test cases match FS• FS is updated

Baseline audit

• release contentis clear and stable

Release Baseline

TestPlan

ProjSched

DocPlan

FuncSpec

ProjSched

TestReport

UserDoc

Baseline Audit

• Test Rep covers ERL• User Doc covers ERL• FCS is successful

Three Levels of Control

Standard Configuration Management offers three levels of control. The purpose of differentiating between these levels is to focus on critical tasks at hand. Not to over-control those items that need no control. And not to lose control over items that are key to our operation and business success. Configuration Items List reflects the decision about which items belong to which level of control.

• No Control

• Identification (or Stored Control) Items are stored in the CM repository Naming convention applied (with limitations)

• Version control Revisions to items are documented Each version is maintained (any previous version can be retrieved)

• Change control Includes all attributes of Version Control Change requires formal peer review and approval Related work products are also updated

Examples of levels of control

• An example of an asset from "No Control" category is a weekly e-mail distributed by BU News Central. Such an e-mail is not maintained in our repository and is usually deleted after being read. Since the today's news will surely become obsolete tomorrow.

• The so-called "stored" items could be easily found in our repository. Their name complies with the naming convention and they are searchable through the regular queries. Recorded video to our Live Session is an example of a stored item. Its naming could be "MET CS473 Live Classroom 1/24/15". You could immediately notice a violation of a naming convention, since the file name includes a date. So the only reason this is possible, since this item has no version number, it will remain unchanged over time.

• Items under "version control" have a version number that is being increased each time an item is changed. The configuration item list, which you submit as part of the term project - covers all such items. Your first configuration item called "Definition of Personas" is an example of a version control. It is expected to have a version that is specific to this definition.

• Change Control implies that an approval is required to implement a change to an item. In most companies, the list of requirements is under a Change Control. It would be strange if a random person or even an author is able to single-handedly add or delete requirements. What about the other thousands people who need to implement these requirements? They might have something to say about changes.

Here is the so-called "Not-To-Do-List", as the following actions are greatly discouraged,

• Promote within organization or release major changes without an appropriate peer review and approval

• Distribute a documents as an attachments instead of distributing it as a link

• Include a date and a version - as part of the document’s name

• Assign names to documents that do not reflect the document’s content and library’s convention

• Maintain duplicate documents in a library

• Keep an important documents on individual desktops

Not To Do List

Versioning Standard A mature organization adopts a consistent definition of versioning. It could take you a full page to describe various examples of versions, as well as scenarios of a transition between versions, for both code and docs. A common question is why do I need to manually insert a version (at the bottom right corner of a document's footer), if a repository itself assigns a version automatically? The point is that each time you hit the commit button, the version to your artifact is bumped. You can insert ten blank spaces resulting in exactly ten additional versions. Obviously these new versions have no corresponding software change that worth mentioning. You still have to tie the meaningful software changes to the sequence of versions. A versioning standard could be quite simple, X.Y •<X> for major •<Y> for minor or it could be more complex, One of the Quiz questions reads as, "If a consistent versioning standard is adopted, it is enough to examine the structure of a version number to determine whether a configuration item is a draft or a released." The expected answer is "YES", regardless of the specifics of a versioning standard. It would be quite unusual for a versioning not to distinguish a released from a draft. Often folks make "C" and "D" equal to zero - after a customer release or after a baseline audit. Alternatively, customers are never exposed to an internal versioning, so the only numbers customers see are "A" and "B".

A.B.C.D •<A> designates a Major Release •<B> ... Minor Release •<C> ... Major Draft •<D> ... Minor Cosmetic Change

Examples Here are several common scenarios,

• version 1.0 means that a document just been released to the field for the first time. • version 0.0.0.1 means that a document has been introduced as an initial draft into the

repository • version could be advanced from 0.0.0.1 to 0.0.0.2 with a minor alteration • version could be advanced from 1.0 to 2.0 as a major customer release • from 1.1 to 1.2 - as a minor customer update or a bug fix • Note that a versioning is usually influenced by marketing, industry standards, customers, • You may have a product line with a monthly public release, 15.1, 15.2, 15.3....15.12. and then an internal scheme as, 15.1.8.3456 ... 15.1.23.3500 , that includes various

builds.

More Examples Here are scenarios that are not supposed to happen,

• from 1.3 to 1.6 ... missing versions • from 2.6 to 0.8 ... going backward

Transition from one version to another corresponds to a certain software change. A missing version is indicative of a software change that has not been accounted for - which could raise a red flag.

1. Branch specifications A and B are created for two project codelines, A and B, respectively. 2. Both project codelines are integrated from deployed 3. Change list #12345 is made to project A 4. Projects A and B are rebased and then change #12345 is shared (integrated) from project A to project B 5. Change list #12346 is made to project A 6. Project A codeline is moved to deployed. 7. A release is deployed to the network 8. Project B is rebased, during which a conflict occurs 9. Branch specification C is created and integrated 10. Project codeline C is rebased 11. A move to deployed is scheduled for Project C

Basic Source Control Source control is extremely important to the software development process. Its importance is amplified in the distributed environment where people have no chance to talk, but rely on a process and a tool to carry the day. Pictorial below illustrates the eleven basic scenarios of code control.

Taxonomy Here is some commonly accepted taxonomy. Codeline An area in the repository file system, consisting of all the files contained under a certain directory. No two files in a codeline can share a common ancestor, except in the case of a `rename' operation. Examples are the deployed codeline, and project codelines. Branch Specification An object that stores a list of relationships between codelines. It is used to simplify integration commands. Branch specifications are used to indicate how a project's code changes are to be integrated into the deployed codeline Rebase To integrate changes from a `parent' codeline into a `child' codeline. This is used to integrate changes made on the parent codeline into the child codeline, to keep the child codeline up to date. Most rebases will be from the deployed codeline (the parent) to a project codeline (the child). Projects are not released unless they are rebased from the head of the deployed codeline; this ensures that changes to a component are not unintentionally lost. Promote To integrate changes from a `child' codeline into a `parent' codeline. Promotions occur from project codelines to the deployed codeline, when projects are deployed

Key Concepts

•The only way to ship software to the field is through Deployed Codeline. In some systems it is called Trunk, or Master. On the pictorial such event is designated with a yellow square and called "Release is Deployed". Understanding that the sole purpose of all our activities is to succeed with the Release to customers. Here is the link to an interesting discussion about the "Power of One" covering concept of a single delivery codeline. http://blogs.workday.com/why_weve_moved_to_single_codeline_development_at_workday.html

• Synchronization among all branches of the repository should be as frequent as possible. The longer branches are kept out of synch, the more difficult it is to merge them. "Building Often" - is the key principle of a modern organization. Reading the Google's front site "Fast is Better Than slow", one could appreciate the fact that the effectiveness of a huge enterprise is driven by a simple parameter "how fast can you build".

• Resolution of code conflicts is inherently manual process. Each source control tool offers a variety of features to simplify the merge; however the final decisions are always done by a developer who is familiar with the context.

• More to the above paragraph, in most cases, resolution of a conflict could be only done by the developer who introduced this conflict on a first place. This brings about the principle aired by Jez Humble "Never Go Home On A Broken Build". Imagine yourself shutting down the Trunk and forcing folks in different time zones to sit on their thumbs for the next twenty-four hours while waiting for your return.

• Branching and Merging Strategy should be well-documented. Who has his own branch and who does not - should be determined and not improvised. Multiple branches kept for a long time without a rebase - are bound to hinder the operation.

Tutorials Module 4 assignment covers several source control tools, AccuRev, CVS, Subversion, GIT, ClearCase, and Perforce. Here is the link to an explicit GIT tutorial. http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide

Note that terminology from each tool differs slightly, while basic concepts remain common.

References to Software Configuration Management

• Wayne A. Babich, "Software Configuration Management: Coordination for Team Productivity", Addison-Wesley ; Original text establishing a taxonomy and scope of the SCM subject

• IEEE Std 1042 "Guide for Software Configuration Management" Consists of several examples (templates) of SCM Plans covering various types of software projects • MIL Std 483 "Configuration Management Practices" If you work for a government contractor, as a minimum, you need to know the specific taxonomy, e.g. allocated baseline, contract change proposal, etc. • James Bach, "The Highs and Lows of Change Control", Computer, vol.31, #8, August 1998.

• Mario Moreira, "SCM Implementation Guide", Willey A detailed treatment of SCM adoption steps for a large organization Mario Moreira, "Adapting Configuration Management for Agile Teams", Willey Balancing Agile speed with a prudent discipline leading to a long term sustainability • The importance of the topic is supported by the fact that every text book on software engineering, (e.g. Pressman and Braude) has a chapter dedicated to SCM. CMMI (Capability Maturity Model) has a prominent PA (Process Area) that is specific to this topic.

Create your account

Search class

repository

Fork class

repository

Merge changes

1

2

3 4

https://github.com/ Exercise with GIT

Suppose each student has few lines of a poem. The task is to integrate all lines into a complete poem.

It takes exactly 60 seconds for all students in a class to obtain a new account with GIT. Then it takes exactly one hour to integrate a complete poem. Most of the time is spent on coordination, waiting for each other and then correcting conflicts, e.g. lines out of order and doubles.

Consistent Estimation Practice

Rationale for Adopting Estimation as a Defined Process

At this section we shall explore the key software process - Estimation. Its importance is emphasized in CMM (Capability Maturity Model) framework where it is rightly called "the heart of Level 2". Meaning... an organization cannot advance from initial level to the next, unless there is a consistent estimation practice in place. Key concepts and rationale for adopting estimation are as follows. Estimation is not optional

a mandatory part of product delivery, as Design or Testing features could not be included into a release without sound scoping

Does not involve black art based on calculation and previous experience

Various groups have their own assumptions enterprise collaboration involves multiple dependencies if you do not provide estimates, then someone else will give it to you

How to improve Estimation accuracy? acquire the skill, learn the craft avoid surprises, prevent risks

What is Estimation? Let us zero in on a definition of estimation process. • Estimation is an iterative process consisting of the following steps, (a) Size, (b) Effort, (c)

Duration (Schedule). • It involves Program / Project Managers, Feature/Test Leads, Owners of Components -

maintaining estimation records in a predefined program repository. • The process includes reporting accuracy trends to drive improvement recommendations.

A typical Project Management Policy is illustrated below. Hence, Estimation is a part of a broad Program Management process. It highlights an important part of the process, namely the historical estimation repository, that we shall explore a bit later.

reconstruct & correlate historical

data

Product Delivery Life Cycle A successful software business uses a product delivery life cycle to guide its many projects. We should examine estimation process with a back drop being a defined product delivery methodology.

Organizational Life Cycle creates a context for key processes of, Software Hardware Service Manufacturing

Also covers: Documentation Architecture Program Management Training

While one size does not fit all, there are common features of a life cycle leading to success: Project plans are tailored to fit into a Product Delivery Life Cycle Folks charge their time to activities inside each phase The delivery model covers thousands of people contributing to a single delivery Phases have a significant overlap System Test is automated

To clarify the notion of a Product Life Cycle, it is recommended to ponder over the following discussion points.

How does a simple release of documentation fit into the Life Cycle Which phase covers daily stand-ups of software development What comes first, a Functional Spec or a Test Plan What comes first, Code or Test Cases How does MRD is traceable to unit test cases How does bug fixing of customer-reported defects fit into the Life Cycle

These points shed light into grey areas and have to be responded to assure the life cycle is adopted at your specific environment.

A lifecycle is the set of steps a program follows from its conception, through its design, development, test, manufacture, service, and disposal.

Multiple programs follow the different steps of a predefined lifecycle.

Life Cycle

A product life cycle is an organizational concept, not a project-level concept. Refinements and improvements can be identified in a program and in a lifecycle. A

lifecycle is a general guidance, not a strict instruction set. Program deliverables differ greatly depending on the program type.

The lifecycle is an important part of the organizational process set. As all controlled processes, a lifecycle is under a version control that is periodically reviewed and updated.

Phase 1Initiate

Legend:

Checkpoints• Control Docs

Review Meetings

CP-4: Feature Done

CP-6 Release Ready

CP-7 General Availability

CP-0: Begin• Project Charter

CP-3: Project Commit• Commit to Release date

CP-5: Ready• Quality assessment

RTSRTM

EOLEOS

MRD

CP-1: Defined Requirements• PRD Complete• Rough Schedule

CP-8 Steady State

Phase 2Plan

Phase 3Define

Phase 4Build

Phase 5Optimize

Phase 6 Release

Phase 7Introduce

Phase 8Support

42 31

CP-2: Engineering Commit• Functional Specs• Test Plans

System Test Gates

How to Beat the Cone Here we introduce a concept of Cone of Uncertainty as thick red lines overlaying the product life cycle. I encourage you to examine the text book to this course by Steve McConnell "Software Estimation" that has a chapter 4.2 dedicated to this topic. The red lines on the pictorial below show the Estimation Accuracy. There are several immediate observations we could derive from its shape.

• First, you see that Estimation is iterative and the same features are being re-estimated • Second, the accuracy of estimation is improving dramatically. At the end, it is one hundred percent accurate since, while standing on the shipping dock we know exactly when the product is shipped. In the beginning, the gap is huge. When the initial concept is being analyzed, an estimate could be several hundred percent off. The fundamental question that is being asked - How to Beat the Cone? How to reduce variability on a project? The cone does not narrow itself. We need to maintain well-defined requirements, consistent code reviews and effective unit tests.

Three Gurus of Estimation

It’s worth mentioning the names of three gurus, that are industry practitioners who have been advancing the subject of estimation over the last thirty years. Barry Boehm, Capers Jones and Steve McConnell have each a written well-known books on the subject. It is prudent to use the wealth of knowledge these software gurus offer. Here we map the Cone first published by Barry Boehm – into our standard life cycle. Note that “variability of Scope” in text book’s Cone (4X) is much greater than “variability of Schedule” (1.6X).

Example from a Real Project The fundamental reason we introduce this training is to improve the Estimation Accuracy on your project. Usually, Program Management is in a position to track the delivery dates of all release types. Here is below we illustrate the concept with a Cone based on dates.

Several conclusions and lessons one could immediately derive from the chart that is a slightly modified example of a real data.

• The chart is lopsided, as none ships early. Schedules are always over-optimistic, as programs frequently slip and hardly ever completed before a commitment.

• Bigger programs have a greater margin of error.

• The earlier we acknowledge an upcoming slippage – the easier it is to implement a

correction. Late corrections are costly, as it is difficult to prepare for a dramatic change of a course.

• 15 percent correction is expected at a program start • 50 percent correction is the maximum one could anticipate at program start.

• Let's compare these numbers (15% middle and 50% max) against the numbers quoted

by Steve McConnell and Burry Boehm. The text book chart above has 1.6X (60%) variability. We should be able to justify this difference.

• In our example, at "program start", a significant part of analysis has been completed which mitigated some large sources of variability.

• The data shown relates to Schedule, not to Effort and not to Size. As we all know, some releases are compelled to drop features, so to keep the train on schedule.

Estimation Framework - Three Buckets Let us review the framework for estimation in some detail. There are three buckets that all measurements should fall into. Size. Effort, and Duration.

• Example of Size would be Lines of Code, Screens, APIs, and, of course, Story Points. • Effort is measured in Person-Hours or (ideal hours). Only 7 hours per day are counted

considering this is the time that spent productively. Staff meetings, vacations, and other non-productive work is excluded.

• Duration is measured in Months. The structured

estimation process proceeds in this same order. From Size to Effort to Duration. Skipping a step has its ramifications. Managers are encouraged not to ask for a date, and not to focus on when you are going to finish. But instead to ask first the details of ….what are you going to do.

Estimating Size - is the Game Changer Estimating Size is the key to a prudent process. It is a Game Changer so to speak. Instead of first giving dates and then dealing with consequences. Our strategy is to, • retain size measures mapped into Effort and Duration, • maintain a historical estimation repository and • eventually re-use this historical data on next projects. In fact these three points constitute the most basic method called Estimating By Analogy.

Key Principle of Estimation An important estimation principal is that its Accuracy shall be equal to precision There’s no point in being precise about something if you don’t even know what you’re talking about. —John von Neumann [Steve McConnell, p.33, p.51]

• This principle is more fundamental than it might be understood initially

• The distinction between accuracy and precision is critical. Project stakeholders make assumptions about project accuracy based on the precision with which an estimate is presented.

• When you present an estimate of 395.7 days, stakeholders assume the estimate is accurate to 4 significant digits! The accuracy of the estimate might be better reflected by estimating 1 year, 4 quarters, or 13 months rather than 395.7 days. Using an estimate of 395.7 days instead of 1 year is like representing pi as 3.37882—the number is more precise, but it’s really less accurate.

Common Questions and Answers

• (Q) A common question managers ask. If I give you another week to work on your estimate, can you refine it so that it contains less uncertainty.

(A)The answer is No. The reason the estimate contains variability is that the software project itself contains variability. The only way to reduce the variability in the estimate is to reduce the variability in the project.

• (Q) How to combine sizing from different features; how to aggregate estimates? (A) Estimates are aggregated at an Effort-level not at the Size-level For example, How would you aggregate five test cases with ten bug fixes? Although, if each

of these activities take 2 days then combined, it will take 4 days to complete them. • (Q) What do I do if it is unfeasible to estimate Size? (A) Try using Analogy for Size, then derive Effort from Size. When people say that they

cannot estimate Size it usually means that they have no idea how to proceed with the project. So the issue in front of us is little to do with estimation and everything to do with domain expertise.

• (Q) How to select the most effective Size Measures? (A) Those Size measures that have the greatest correlation with Effort - are the best. Think

of the next step, when you have to estimate Effort. Focus on your strongest Effort drivers.

Estimation Record - Leading to Historical Repository Here is an example of an estimation record for a medium size project. The Decomposition method is used. As the Project has been decomposed into Phases and then Tasks. • An appropriate Size measures were selected to drive the corresponding Effort. • The column "Typical Effort Per Size Measure" is taken from previous experience of similar projects. • The Size, for example 10,000 lines is also estimated from previous historical data. • Effort is computed and

derived from Typical Effort multiplied by Size.

Phase Task Size MeasureTypical Effort per

Size Measure (person / hours)

SizeEffort

(person / weeks)

Feasibility Study write a white paper # white papers 40 1

Definition document Requirements # requirements 1 20 0.5document new fields # fields 0.5 50 0.625document new reports # reports 2 5 0.25develop Functional Spec # pages 2 20 1peer review Functional Spec # issues reported 1 20 0.5develop Design Spec # use cases 8 5 1

design complexity (S/M/L) M

Coding LOC (changed / added / deleted) 10,000 12GUI Screens (ch / add / del) 10 4APIs exposed 9 5New objects 5 6

Test document test cases 4execute test cases 4

40 • There are many assumptions and decisions that are made while completing this estimation record. Making decisions about small project components (for example about GUI Screens) are much easier than to make such decisions about the whole project. That is why the Decomposition is an effective method producing estimates within acceptable ranges. • Analyzing an estimation record could lead to some useful conclusions. One might observe that too much effort is spent on coding. If we re-distribute the effort to put additional emphasis on requirements and design then coding and testing could be significantly reduced, so the total effort will be smaller. Needless to say that without a consolidated estimation record, such analysis is impossible. • A collection of estimation records constitute a historical estimation repository. Some organizations maintain databases with thousands of estimation records along with their corresponding actuals. Should note the significant value of actuals, since all future estimates are based on past actuals.

Estimation Record - Example / Template Here is an estimation record example/template that fits well into the context of the term project. It should account for each member of the project team. Following is the basic order of its completion, • identify deliverables

that need to be done • Size Measures • Size (how many of each

of the size measure) • Effort • Round-up to the next

Fibonacci number

Expert A Expert B

Size MeasureTypical Effort

per Size Measure (person / hours)

SizeFinal

(Fibonacci)

Requirements Definition of users (personas) # roles 0.20 5 1 1 1Definition of Scope and Limitations # attributes 0.10 10 1 1 1Engineering Requirements List # requirements 0.40 5 2 1 2

Config Mngmt Configuration Items List # CI items 0.09 11 1 1 1

Estimation Estimation Record # activities 0.08 13 1 1 1

Design State Transition Diagram # states 0.25 4 1 0.5 1Definition of UseCases # use cases 0.50 4 2 2 2Definition of Fields # fields 0.10 20 2 2 2Definition of Reports # reports 1 2 2 2 2

Peer Reviews Issues from Peer Reviews retained # issues 0.16 25 4 4 5

Implementation Defect Tracking System # User Screens 6 1 6 8 8

Test Design Test Cases # test cases 0.33 6 2 2 2

Test Execution Defects from testing recorded # defects 0.30 10 3 3 3Total: 31

Effort (person / hours)

Expert A Expert B

Size Measure

Typical Effort per Size Measure (person / hours)

SizeFinal

(Fibonacci)

Requirements Definition of users (personas) # roles 0.20 5 1 1 1Definition of Scope and Limitations # attributes 0.10 10 1 1 1Engineering Requirements List # requirements 0.40 5 2 1 2

Config Mngmt Configuration Items List # CI items 0.09 11 1 1 1

Estimaton Estimation Record # activities 0.08 13 1 1 1 Design State Transition Diagram # states 0.25 4 1 0.5 1

Definition of UseCases # use cases 0.50 4 2 2 2Definition of Fields # fields 0.10 20 2 2 2Definition of Reports # reports 1 2 2 2 2

Peer Reviews Issues from Peer Reviews retained # issues 0.16 25 4 4 5

Implementation Defect Tracking System # User Screens 6 1 6 8 8 Test Design Test Cases # test cases 0.33 6 2 2 2

Test Execution Defects from testing recorded # defects 0.30 10 3 3 3

Total: 31

Effort (person / hours)

Decomposition method is used

Analogy method draws from previous projects

Expert Judgmentmethod relies ontwo SMEs

Final estimate isrounded up to the nextFibonacci numberto create a buffer

Those Size measures are selected that correlate well with Effort

Estimates are done several times along the life cycle

Schedule is notestimated, sinceduration is time boxed

Planning Poker Planning Poker is the key technique used to estimate stories. The whole SCRUM team is involved in a Planning Poker session. The same or a similar technique could be used successfully for non-agile projects to estimate features and individual requirements. Here are several considerations. • Prepare to estimate. The process will not work if you are surprised when the Product

Owner describes a story. In fact the biggest myth about this method is that one can waltz into a planning poker session and be able to respond with a reasonable prediction.

• The reason process is so effective since experiences of several SMEs are combined and the total is greater than the simple sum of its parts.

• Ask questions and be ready to reconcile assumptions. • During initial discussion, numbers must not be mentioned at all in relation to feature size

to avoid “anchoring”, when team members are influenced prematurely. • Reveal cards all at once. This prevents folks from influencing each other’s estimates. • A value of 3 needs to be 3 times bigger than the value of one. Need to ground this proxy

measures to an actual Size. For example, to a number of test cases or bug fixes or GUI changes.

• The larger the estimate, the larger the error. Precision should be equal to accuracy. Remember Leonardo Fibonacci who brought us the buffer.

The order of process steps is as follows. 1) Scrum Master takes a minute to review the process and roles 2) Then presents the top item of a product backlog to the team 3) The team discusses the essence and then details of the story 4) The Product Owner clarifies questions, assumptions as well as story’s acceptance criteria. 5) Each team member privately decides how big this story is, writes a number (of points) on

an index card. Estimates are rounded to the next Fibonacci number. 1, 2, 3, 5, 8, 13, 21, 34. Stories exceeding X are split or are marked with “infinity” sign. “X” being a team-specific parameter.

6) At the count of three, everyone shows his or her index card simultaneously. 7) If there are significant differences, then reconcile assumptions. 8) Repeat the last steps no more than two times. Spend no more than 10 minutes on each

Story. Make sure the team consensus is reached and derived estimates are unaffected by emotions or peer pressure.

9) Scrum Master documents Points and then documents Assumptions

Pictorial below shows standard cards. For example, Fibonacci number "13" on the card represents the size of a story equal to 13 points.

Common Deployment Issues Adopting a consistent estimation practice across a large enterprise usually meets with a considerable resistance that is natural for any organizational change. Here is the relevant LinkedIn discussion on the topic that might provide a taste for a common deployment issues. We could continue this discussion within the framework of our class.

• As Product Management is searching for a direct measure of Customer Value and stating that Story Points are inappropriate measure of Value. Please suggest a way of measuring Value, so to help Product Management.

• What are the ramifications of missing the first step in our estimation process? Namely, if we drop the idea of Size and jump directly to Effort. Here is an interesting site called "Ballpark" http://www.stridenyc.com/ballpark/ It has a certain methodology for a rough-level initial estimate. Apparently the site is trying to differentiate itself by juxtaposing Budgeting and Estimation. Obviously these two concepts are not in any competition. First goes estimation, second goes budgeting. They are not a substitute for each other. I feel that there is an apparent confusion with terminology, as methodology like T-Shirt Sizing is what these folks call Ballpark. Although glancing at their site it looks like Monte Carlo Simulation is used as a first estimation method.

Estimation Tools Most commercial tools are based on a proprietary database from thousands of completed projects. Initially, you derive an estimate from an industry averages. After your company-specific data is collected, you start customizing and substituting the industry data with your specific data. Checkpoint originally introduced by Capers Jones the founder of SPR (Software Productivity Research) of Burlington, with projects contributed from AT&T. It uses Function Points and employs more than a hundred parameters in five categories. • Process • Technology • Personnel • Environment • Special factors

SLIM (Software Lifecycle Management) has a proprietary algorithm, supported by QSM (Quantitative Software Management). Commonly used by government contractors, uses a variety of parameters, including software size and Productivity Index (PI). The PI is calibrated to an organization's historical software development environment.