Leveraging Agile Concepts & Techniques To Enable Enterprise …mercatorg.com › ... › Final-TMG-Agile-Workshop-Material-v1.pdf · 2013-07-17 · Agile Is Supported by Multiple

Agile Workshop

Leveraging Agile Concepts & Techniques To Enable Enterprise Modernization

7/9/13 © The Mercator Group 1

Agenda

•  Workshop I: Guide to Leveraging Agile (3 to 4 hours) •  Workshop II: Hands-on Exercise (1.5 days)

–  Develop a Product Backlog –  Develop a Sprint Backlog –  Conduct a Sprint –  Lessons Learned


Objective of Workshop I

A Practical Guide to leveraging Agile concepts & techniques

•  Brief primer on Agile •  Introduction to Agile Methodologies

–  Scrum –  Agile BPM and Model Driven Requirements

•  Primer on how to manage Agile efforts •  Guidelines on how to stand up an Agile capability and

adopt Agile to meet your organizational needs


What Is Agile?

Agile is set of principles that proponents believe to be a better way of developing software

•  Agile grew out of a “Manifesto” issued in 2001 by a group of 17 influential programmers, that included the following:

–  Individuals and interactions over processes and tools –  Customer collaboration over contract negotiation –  Working software over comprehensive documentation –  Responding to change over following a plan.

•  Agile gains its substance through the methodologies that implement its principles, such as Scrum, which is not only a methodology, but a framework most other methods are based on.

•  All Agile methodologies have in common a set of core principles: –  Focus on collaboration –  Shared-ownership and responsibility –  Adaptation and continuous learning –  Iteration –  Open-ended requirements and time-boxed implementation


How Agile Fundamentally Changes the Development Process

•  Under Agile, cross-functional and self-organizing teams work together to collaborate in all phases of the development process

•  Unlike other approaches to software development, changing requirements are not only accepted, but thought of as fundamental to the process

•  Agile completely transforms the development process, fundamentally altering roles, responsibilities, required skills, process and culture:

–  End users are no longer “customers,” but part of a team with joint responsibility for the outcomes

–  Documentation is more of an output of the process, than a control mechanism to the process

–  The quality of delivered functionality takes precedence over completing and end-to-end solution, at least as initially perceived

•  Strong governance, software architecture, and software change management processes are even more important to manage the higher level of change that occurs as part of the process


How Do You Implement Agile?

Figuring out what works for your organization – not one single orthodoxy

Start With Agile Development As A Philosophy

Understand Agile Methodologies

Adopt One To Fit Your Organizational Needs

Agile fundamentally changes roles, relationships, culture, and power structures, and needs to be viewed as an organization change effort


Agile Is Supported by Multiple Methodologies

•  All Agile methodologies stress collaboration, joint ownership, flexibility, and iteration.

•  While Scrum is the most used framework for Agile methodologies, there are many other influences (eXtreme Programming, Model Driven Architecture, Lean Manufacturing).

•  Many vendors have their own proprietary Agile approach, e.g. IBM has created Disciplined Agile Delivery (DAD) methodology.

•  TMG has developed Agile BPM for process oriented application development.

•  It is up to the individual organization to decide what best fits its need.

•  For the purpose of this workshop we focus on Scrum, with some level of discussion on Agile BPM, which we have successfully implemented at GSA.


Agile Enables the Leveraging of Modern Software Capabilities

•  Software has typically been treated as an expensive and rigid asset requiring a lot of upfront planning and design to ensure a return on investment, and control risk

•  While still expensive, software environments have become very flexible and adaptive to change:

–  Modern environments allow for easy configuration of Lego-block like capability, versus laborious and error prone coding

–  Service Oriented Architecture, combined with powerful object oriented abstraction, have further improved the flexibility of software, and the availability of reuse

–  Most packages today have easy to understand modeling environments that can be mastered by non-programmers

–  APIs allow for easy integration of diverse environments, including email, portals, office automation tools, and other applications

•  Adopting Agile is required for an organization to leverage these capabilities, and provide quicker and iterative turnaround of capabilities, cheaper and more flexible environments, and higher quality and better aligned software.


Agile Using The Scrum Method

•  Scrum is an industry standard framework for implementing Agile: –  Integrated project teams (IPTs) share joint responsibility for achieving the

goals of the effort –  Business stakeholders, end users, analyst, trainers, testers and developers

work as a unit to develop a solution; usually with interim releases of functionality

–  Iterations, referred to as Sprints (Scrum methodology); typically organized into two to six week timeframes for defining and developing a functional subset of the required capabilities

–  The team is guided by a Product Owner who is responsible for prioritizing work activities, and manage the addition of new or changing requirements into the effort.

–  A Scrum Master helps lead the team by providing expertise in the methodology. The Scrum Master is not the Project Manager.

–  The outcome of the effort is working software, while minimum documentation tends to be a byproduct of Development, not an input to Development


Key Scrum Roles and Responsibilities

Scrum adds some additional role descriptions to the software development process


Scrum Role Responsibilities Product Owner Sets objectives and priorities, ensure outcome reflect the

needs of the business, and their priorities Team Self-managed teams sharing the responsibilities for

getting the work done Scrum Master Facilitates Scrum activities, coaches and focuses on

removing impediments Product Assurance Team

Works with Product Owner to ensure Scrum Objectives and prioritization reflect a wide organizational view

Business System Architect (TMG Opinion)

Provides expertise in facilitating tradeoffs among business objectives, technology, and project resource or timeline limitations

Notional Scrum Project Organization

Executive Sponsors Steering Committee

Product Owner

Project Lead

Project Management

Team

End Users Analysts Software Professionals

Scrum Master

Bus. Systems Architect

Product Assurance

Team


Roles in blue boxes represent roles particular to the Scrum methodology

Agile Teams

•  Should be ten or less people •  Geographically co-located, with shared team space, or

mitigation strategy for dispersed teams •  Fulltime •  Receive Agile training •  Open-minded


The Product Backlog

•  The Product Backlog is an ordered list of requirements/features controlled by the Product Manager

•  The list is ordered according to the order in which the requirements should be implemented

•  Many Agile methodologies define the list as the ordered set of User Stories to be implemented

•  TMG advises the Product List be made up of a more definitive and detailed set of requirements that represent capabilities to be implemented (business requirements), and that it be supported by some level of process design

•  The Product Backlog will be broken into a set of time-boxed Sprints for implementation


The Sprint Backlog

•  The Sprint Backlog further decomposes the Product Backlog into a set of activities required to implement the the Product Backlog included in that Sprint

•  The activities represented in the Sprint are what is managed


The Scrum Lifecycle

Prioritized Requirements – Divided into a time-boxed set of activities - To deliver an increment of functionality – Which will become part of an iterative release of capabilities supporting a business process


User Stories - Process Modeling - Product Backlog

As a Requestor, I need to enter CLINs in my procurement request.

USER STORY <Role> <Activity> <Context>

Subset of sub-process within Procurement Request process

PROCESS MODEL

PRODUCT BACKLOG All of the components of the process models (role, activities, inputs, outputs, deliverables, rules, etc.) are then documented as part of the product backlog to be developed iteratively.

<Role> <Activity>

Many User Stories

Few Process Models

One Backlog


The Agile Business Process Management (BPM) methodology

•  TMG’s Agile BPM methodology brings together Agile with Business Process Management Systems (BPMS)

•  BPMS allows for the quick configuration and building of process-based software by a non-programmer

•  Business Process Management System capabilities available in standalone tools, and as part of many software packages (SAP, Force.com, etc.) provide an excellent way to quickly model and develop system capabilities in an interactive and collaborative environment.

•  It allows for easy collaboration between end-user subject matter experts and business analyst to build functionally operational capabilities


The Agile Business Process Management (BPM) methodology

•  Using BPMS, substantial applications can go from conception to implementation in anywhere from 3 to 16 weeks.

•  Taking a business process approach to defining software services has proven to be the most optimal approach to developing effective system capabilities.

•  This approach also has the important benefit of eliminating the stove-pipes and data issues so common to approached using a more traditional functional decomposition method of defining software needs.


TMG Agile BPM Methodology

* See TMG “practical Guide to Agile BPM”


TMG Agile BPM Methodology

Stage Phase Tools/Methods

Inception Scope • User stories • Value chain • High-level process modeling

Iteration Model • Business workflow modeling • BPMS workflow modeling • Process Improvement • Scrum • Facilitated stakeholder sessions • Process simulation

Assemble • BPMS coding • System integration • Scrum • Stakeholder/user testing • Data management

Execute • System deployment • User acceptance testing • Change management

Maturation/Evolution Measure • Process performance metrics • Stakeholder feedback

Analyze • Performance expectation analysis • Continuous process improvement • ROI analysis

Refine • BPMS model refinement (major and minor) • Scrum • Facilitated stakeholder sessions


Why Agile Makes Sense?

•  Agile recognizes that software development is an empirical discipline: –  The effort is time and resourced boxed, what varies is the functionality that is delivered. –  Time and resources are fixed. –  Requirements are subject to change as further insight is achieved as part of the process.

•  Agile takes advantage of “lessons learned” from traditional approaches. Allows for:

–  Changes in requirements –  Flexibility in both time and resources constraints –  Stakeholder acceptance and buy-in –  Better focus and responsiveness to “must have” requirements

•  The majority of time is spent building what the end user needs, but may not get through the complete list of what is desired

•  Agile focuses on optimizing the quality of what is delivered over completeness, cost and schedule:

•  Achieving 80% of initial scope is considered a success, as long as the 80% of delivered capabilities is of excellent quality


Agile Doesn’t Lessen the Importance of Program Management

•  Agile requires strong program management just like any software development effort

•  It differs in: –  Plans are less static –  There is a realization that upfront estimates are subject to change as

discovery takes place as a byproduct of the development effort –  Work is always time-boxed and prioritized based on business need –  Focus on quality of what is delivered, versus completing the upfront plan

•  Agile does require: –  Strong governance processes to ensure the added flexibility provided by Agile

does not get abused –  Independent architectural oversight to ensure it doesn’t become an excuse for

non-delivery


Project Planning

•  The necessity of planning and the amount of planning necessary. –  Underplanning results in the inability to answer questions like, “When

will you be done?” –  Overplanning results in creating a plan that may not be accurate or

useful in execution.

•  Planning in Agile –  Knowledge and insight are being constantly updated, as such, Agile

planning is an ongoing process and Agile plans are subject to change. –  Focus is on features rather than activities. –  Take uncertainty into account in the planning phase.


Estimation

•  Estimation of a project refers to the question, “how long does it take to complete the project?”

•  The entire project’s amount of work are broken down in a Work Breakdown Structure (WBS) into smaller units of work.

•  The units of work can be based relatively to the amount of work required such as story points and ideal days, or time based such as days or hours.

•  Relative estimates based off of work required rather than time required are generally more accurate, because it is difficult to estimate how long a project will take the people actually implementing it to do.


•  The rate of progress within software development pertaining to Agile methods is called “Velocity.”

•  The equation of the velocity of a software development team is similar to the velocity of an object (Velocity of an object = Distance / Time).

•  Velocity would be determined by taking the amount of work done (distance), divided by the length of time taken to do the work (time). –  Ex: if 20 story points are achieved in the initial week, the velocity

of the team is 20 story points per week. •  Velocity is then used to determine overall time the project

will take. –  If the said project is a 200 story point project, it is assumed the

project would take 10 weeks. 7/9/13 © The Mercator Group 25

Measuring Progress

Tracking Progress: Burn Rate and Velocity


A Program Management View of Agile

Managing Agile Projects Description

Carefully plan iterations Start with a business process design, and move from start to finish through the process. This will result in a logical sequencing of capabilities and resolution of data disconnects across the process

Get team comfortable with shared-ownership

The agile process blurs lines of responsibilities and puts the end user front and center in the process. Establishing team chemistry and implementing an integrated project team approach is essential to success.

Manage change and churn The Agile process, supported by the right tools, can make changing requirements too easy, resulting in excessive churn (“should we make this blue or red?”)

Manage operational risks Agile projects can over-concentrate on end-user functionality, at the cost of administrative and back end integration vital to the operational efficacy of the system.

Ensure operational support capabilities are in place

The level of software change activity goes way up in an agile environment, requiring added attention to the software configuration management function.

Leverage modeling tools The efficacy of the end system increases dramatically with upfront modeling of functionality (Model Driven Requirements – MDR). The team should leverage modeling software to elicit requirements.

Establish an organizational change management process

Agile is a major transformation of the software development process, and needs to be treated as an organizational transformation effort

Develop and maintain project structure

An agile approach, in a subtle, but substantial way, effects the roles of everyone involved in the software lifecycle process. If not managed correctly, it could become a bit chaotic. It is important to maintain project management discipline and leverage the tools that provide for participation and visibility into the effort.

Agile leverages all we have learned in 40 years of software development


Refactoring

•  Refactoring is improving the quality of the underlying software without affecting behavior.

•  In the rush to meet deadlines, it is almost certain that shortcuts will be taken in developing the software.

•  These short cuts may allow for 200 to 1600 users to more quickly take advantage of productivity improvements, and provide a huge ROI

•  However, over time, they can make the software more brittle and costly to maintain

•  To prevent this, all projects should: –  Architect the software so refactoring can take place (good object

oriented techniques) –  Factor in resources to ensure refactoring can occur


Misconceptions of Agile

•  Agile means no plan. –  Agile doesn’t support development without planning. Rather,

planning and estimation is a continuous process.

•  Agile doesn’t allow for documentation. –  Documentation should be written at the right time, and only just

the right amount.

•  Agile only works for trivial projects. –  Agile has been used for:

•  End to end hospital healthcare processes •  Complex financial applications •  Adopted by DoD •  Etc.


Benefits of an Agile Mindset

•  Typically results in software that aligns most closely to business needs.

•  Working valuable software is delivered on a frequent basis, allowing for immediate feedback, and use.

•  Agile provides more flexibility than Waterfall in requirement changes, even late in development.

•  The end product will have fewer defects than if Waterfall methodology was applied.

•  Early feedback (non-scientific) is there is a lower rate of failure, even given the level of change required.


Agile in the Government

•  Key components of Agile from the Agile Manifesto are collaboration takes precedence over contracting, and change should be welcomed

•  Of course, most development efforts in the government take place in a structured budgeting and contractual environment, with change being difficult

•  Despite these potential limitations, the Government can greatly benefit from adopting some form of Agile:

–  Collaboration is beneficial to all parties, and increases buy-in by all participant –  Software development environments are much more amenable to a more flexible

and agile development process, and it would be a mistake not to take advantage of this

–  Iteration provides for better oversight and program management and reduces risk across the board

–  Accepting some level of change is only acknowledging the realities of software development.


Adopting Agile Techniques to the Enterprise

Technique Method Difficulty Value Collaborative cross-functional teams

Implement Integrated Project Teams (IPTs)

Low to Medium High

Iterative joint-development Model Driven Requirements (MDR)

Medium High

Self organization and management

Culture change High High

Abstract requirements definition

More flexible contracting

High to Very High Very High

Time-box based management (Open-ended Requirements)

Shared risk contracting

Very High High

Ope

n-en

ded

Req

uire

men

ts

Pre

scrib

ed

Req

uire

men

ts

Government Comfort Zone

Functional Specifications Model Driven Requirements

Middle Ground

User Stories

Agile


Modified Agile for Government or a First Step

•  A pure Agile approach is extremely difficult to implement in a Government environment due to the general contractual requirements imposed by regulations, and organizational structures and culture that promote clear delineations of responsibilities

•  Something has to give: –  Agile needs to be broken down into three stages that allow for better definition of the work to be performed

–  SOWs need to provide more leeway in how requirements are defined (somewhere between User Stories and detailed functional requirements)

•  The TMG three-stage Agile approach is as follows: 1.  Conduct business modeling and define Product Backlog: Identify and define Business scope, Roles,

Processes, System touch points and capabilities, Data Requirements

2.  Conduct Working Model Sprints: Work with end users to build Working Models and identify functional system integration requirements

3.  Assemble and Deploy Working Models: Integrate working models into the operational environment, and provide any additional required customization.

•  The three-stage approach: –  Maintains much of the collaboration benefits of Agile –  Allows for improved control and standard contractual relationships –  Provides for overlapping of stages to leverage cross-team collaboration


A Stage-Based Approach to Agile for the Government

Conduct business modeling and define Product Backlog

User Stories – Business Processes – Capability Descriptions – Product Backlog

……

……

Conduct Working Model Sprints

Standalone functioning modules

……

Assemble and Deploy Working Models

Integrated operational Modules

Detailed Prioritized Business Capability Requirements

Col

labo

rativ

e C

ross

-Fun

ctio

nal T

eam

s


Changing the Process

•  Agile is a drastic transformation of the development process, and must be implemented in the context of an organizational change effort.

•  A center of excellence should be established, and the required support mechanisms need to be put in place.

•  Infrastructure support processes, such as software change management and software integration need to be iron clad.

•  The effort should start with a manageable, but significant application.

•  The Agile life cycle should be well documented, and all participants properly trained.

•  Performance measures should be put in place that not only measure ROI, but measure the quality of the software released.

•  A broader program promoting cross-function collaboration should be implemented.


Recap and Closing


Why Agile?

•  Agile Scrum provides a mechanism for significantly reducing risk associated with software development, and significantly improving the quality of the delivered product: –  The iterative nature of Agile, with interim deliverables of functionality,

provides natural check points for assessing progress, and reducing the risk so obvious in traditional waterfall methods

–  The intense collaborative nature of Agile, along with the focus on the quality of what is built, versus a focus on timelines and resources, results in much improved software that is better aligned to the business needs.

•  Large Waterfall efforts have a very high mortality rate –  Too often, large enterprises trust the delivery of major software asset

investments to system integrators, only to find out, after most of the money is spent, they will not get what was agreed to, and will be faced with a savvy contractor making a case for them to spend a lot more money.

–  This acquisition approach has inherent conflict of interest built into it. Adopting an acquisition strategy that enables Agile, and implementing Agile, significantly reduces the chances of the above scenario occurring.


Why Not Agile?

•  Moving to Agile is a significant cross-organizational effort: –  A true implementation of Agile requires a significant change in

budgeting, contracting, and, most importantly, organizational software development process and culture.

–  Agile changes governance, processes, roles, power structures, and skill requirements simultaneous across organizational and functional areas. It is hard to do, and requires a prescriptive organizational change effort.

•  An organization may have in place long-term existing contractual relationships that preclude implementing Agile.


Is there a hybrid approach?

In the long-term, Agile is clearly displacing traditional waterfall as the primary approach to software development. Modern software development environments provide incredible productivity increases, and improvements in the quality of what can be made possible with software. These environments can only be leveraged by implementing some form of Agile. In the short term, organizations have implemented four strategies for moving to Agile:


Ten steps to standing up Agile

1.  Allow and encourage vendors to implement Agile on their own, and incorporate the changes in how you manage and interact with their projects

2.  Educate senior management across the organization on Agile

3.  Have the CIO appoint a senior manager in the role of what is referred to as a Scrum Master (appoint a senior manager with responsibility to implement the program)

4.  Carefully select a set of projects to pilot a first implementation of the complete Agile Scrum process

5.  Just as carefully, select the participants for those projects

6.  Stand-up a center of excellence to support the effort

7.  Put in place the appropriate governance and software change processes required

8.  Provide just in time training for the team

9.  Implement the pilots and measure results

10. Put in place continuous process improvement


Documents

Leveraging Agile Concepts & Techniques To Enable Enterprise …mercatorg.com › ... › Final-TMG-Agile-Workshop-Material-v1.pdf · 2013-07-17 · Agile Is Supported by Multiple