11/29/2014

1

A Systems View of Software Development in Government

Moving From Software Development

to Software Engineering

Working Draft (ver 11)

@tonyjoyce

Looking at software with the Cynefin framework

11/29/2014 @tonyjoyce 2

11/29/2014

2

Pro

gram

min

g D

efin

ed

• "Programming is a process that leads from

an original formulation of a computing problem to executable programs.” http://en.wikipedia.org/wiki/Software_programming

• In Government, the processes for programming are always formulated in terms of Requirements

11/29/2014 @tonyjoyce 3

Programming Defined

• The formulation of a software program proceeds in phases of interlinked processes

– from the initial requirements and needs,

– to a decision about a significant level of work,

– to an initial construction of a capability,

– to acceptance as fit for the intended purpose.

11/29/2014 @tonyjoyce 4

Software engineering

11/29/2014

3

Systems Engineering Concerns

• Inertia

• Expertise

• Age of tech

• Backlogs

• Knowledge transfer

• Capabilities

• Portfolios

• Tradeoffs

• Features

• Functions

• Projects

IT Business

Status Quo

New Rqmts

11/29/2014 @tonyjoyce 5

• Arguments contrasting new agile ideals against traditional waterfall frameworks have been useful in enumerating some practices

• In one way it is a simplistic discussion, because in contrasting the two viewpoints we have missed other use-cases and other developer needs

• Much discussion has looked at development from the programmers’ point of view, where the work supports greenfield development, which prioritizes adding new capabilities

• That is not the way that most systems evolve. Many system improvements are organic or opportunistic, which is to say exaptative as opposed to deductive design or inductive invention – http://cognitive-edge.com/blog/entry/6103/design-thinking-complexity-pt-3/

• We must change the focus of agile discussions from the green fields of software developers towards the make-do & bricolage of software maintainers

• We need to consider other dimensions of system growth to understand how agile could work in what we should call brownfield software development – http://cognitive-edge.com/blog/entry/6227/brown-field-architecture/

Original title: Brownfields Agile

11/29/2014 @tonyjoyce 6

11/29/2014

4

• Next slide is a representation of the issues that must be addressed at an initial decision -- Milestone A of the DOD 5000 acquisition process

• At this stage we have not set a cost target – the cost threshold is critical to a variety of contract and

budget actions

– costs are estimates before the decision is made

– after the decision, cost is a constraint

• Nor have we received the recommendations of the senior official – typically as “conditions” for actions that follow

– these are further constraints

DOD prescribes Program Approval as formal Acquisition start

11/29/2014 @tonyjoyce 7

System Engineering Considerations - Pre-Decisional

Business Architecture

Enterprise Architecture

Status Quo (inertia)

New Requiremen

ts

Portfolios, Capabilities & Tradeoffs

Features, Functions & Projects

Aging & Expertise

Knowledge (loss/gain/

flows)

$$$ & Approvals

11/29/2014 @tonyjoyce 8

11/29/2014

5

• Many named programming methods attempt to define a detailed process to structure programming tasks into small units that can be accomplished by one or two individuals

• This approach may be counterproductive if it centers the focus of the programming work exclusively in the complicated and complex domains – Dave Snowden warns us of the limitations of this specialization approach in his

comments on recipes and formulas

• From a CIO and IT management perspective, the projects we direct have many aspects which are not visible to the programming activities and products produced

• As the project progresses the work becomes more complex, and we must find or build increasingly complicated frameworks to measure our progress

• Illusion occurs when we try to force fit a project into the wrong domain – Illusion can quickly turn into pervasive disorder, hiding the “ground truth”

– This helps explain why it is so hard to obtain consistent status reports and metrics for programming projects

Different organizations will have different lexicons

11/29/2014 @tonyjoyce 9

• We define the original needs in terms of contract requirements which must be met (mandatory) or evaluated (as options in a best value selection)

• Upon award of a contract we direct the contractor to begin construction of a set of capabilities

• This is a new start – a greenfield

What is a requirement? It depends on it’s context

11/29/2014 @tonyjoyce 10

11/29/2014

6

Greenfield Development

Business Needs

Requirements

Design

Partitioning (Modules, Classes)

Code

Unit tests Interdependencies

(Interfaces)

Tradeoffs

Test

User Acceptance

Internal Focus External Focus

Customer Focus 11/29/2014 @tonyjoyce 11

• (blank slide)

11/29/2014 @tonyjoyce 12

11/29/2014

7

• Detailed or traditional waterfall is a “requirements-up-front” or “requirements first” approach, where the adjective “all” is implicitly understood

• A focused enhancement is also a careful and detailed activity (which may be waterfall-like and slow)

• Workarounds and quick fixes are characteristic of opportunistic reuse and exaptation

• High capability solutions introduce greater complexity and increase technical debt, as up-front planning and architecture is typically quite limited

• Redundancy and overlapping functions are very natural – a lot of customer centric innovation occurs through this sort of discovery

• These considerations complicate our definition of completeness and “done”

• This type of programming is a form of rework – a brownfield

11/29/2014 @tonyjoyce 13

Requirements and their contexts, continued

Brownfield Development

Full Featured

Traditional Requirements

Focused Enhancement

Multiple Solutions

Status quo

Measured Timeboxed

High Capability

Opportunistic Reuse

Frugal Minimum

Detailed Waterfall

Agile

Parallel Paths

Prototypes

Requirements Up Front

Use-Case & Stories

11/29/2014 @tonyjoyce 14

11/29/2014

8

• Spreadsheets are ad-hoc codes which may not be programming

– They are disposable formulas & bits of code

– Combining or upsizing a bunch of spreadsheets may not meet the threshold for serious programming

• A hackathon is a novel simple case

– a customer mockup or a simple non functional prototype also is a simple case

• These are Back-of-the-Envelope use-cases

11/29/2014 @tonyjoyce 15

And on the Back-of-the-Envelope

• The JCIDS process, upon which DOD 5000 series instructions are based, prescribes a sequence of decisions, i.e. Milestones

• At each point, A/B/C, a go/no-go gate is required

• Milestone A is the decision to start a purchase (e.g. Pre-Decisional step)

• Milestone B authorizes Initial Operational Capability (IOC)

• Milestone C is the Full Operational Capability (FOC)

11/29/2014 @tonyjoyce 16

The DOD Joint Capabilities Integration Development System defines acquisition requirements

11/29/2014

9

System Engineering Viewpoints

New Rqmts

Ent Arch

Bus Arch

Status Quo

Customer

External

Internal

High Labor

Leadership

High Maint

Complexity

Pre-Decisional Greenfield Brownfield

Milestone A

Milestone B

Milestone C

11/29/2014 @tonyjoyce 17

• It takes a long time to develop a significant system or program, and few people (i.e. Program or Project Managers) have stayed in control (or a devoted job) long enough to see all of these phases

11/29/2014 @tonyjoyce 18

A Knowledge Management View

11/29/2014

10

• A good way to look at this is through the Cynefin framework

• There are different patterns of knowledge & knowing in each of the different domains

– Dave Snowden has illustrated many of these patterns on his Cognitive Edge blog and related works

– See http://cognitive-edge.com/blog/entry/6227/brown-field-architecture

A systems engineering complexity assessment

11/29/2014 @tonyjoyce 19

The Cynefin Framework Captures the Increasing Complexity of Development

Greenfields Pre-

decisional

Brownfields Back of

Envelope

Illusion

Simple

Complicated

Chaotic

Complex

11/29/2014 @tonyjoyce 20

11/29/2014

11

• With software engineering knowledge we can define Done with some measure of confidence (e.g. LOC) – In the complex domain – Glen Alleman’s Herding

Cats blog and System Engineering briefings present a well balanced approach for greenfields

• In brownfields we can not be precise, as fitness and purpose are codependent

• As a result we have fractal patterns with their intricate complexities – In the chaotic domain – see Dave Snowden’s

fragmented knowledge principles, ideas of resilience, and safe-to-fail probes

11/29/2014 @tonyjoyce 21

Engineering is a set of disciplines

• “[you need a ] measurable level of inefficiency in the system to be effective”

• Dave Snowden 2011 Kumvana Panel https://www.youtube.com/watch?v=ejnlwc7W8VE

• Also see Dave Snowden “Managing under conditions of uncertainty | State of the Net 2014” https://www.youtube.com/watch?v=APB_mhpsQp8 (~ 38 min)

• The magic sauce -- praxis -- is incremental change by smart people who can access the data with tools and techniques (disintermediation)

11/29/2014 @tonyjoyce 22

Discipline requires practice

11/29/2014

12

Both Focus and Constraints Change

• Needs

• Ambiguous requirements

• Clean sheet

Back of Envelope (Simple)

• Requirements

• Requirements as mandatory, optional , qualities

• Sky vs. ground

Pre-Decisional (Complicated)

• Capabilities

• Requirements are built-up or decomposed

• Iron Triangle

Greenfields (Complex)

• Features

• Requirements are interrelated or mutable

• Co-evolutionary

Brownfields (Chaotic)

11/29/2014 @tonyjoyce 23

Evidence of Phase Changes

Brownfields

Features Velocity, Satisfaction,

Quality metrics Time-boxing, queuing

Greenfields

Capabilities Master Schedule, Done

defined Loose Coupling, Gartner

(A. Bradley’s) ISP

Pre-Decisional

Requirements Budget/cost/resource

commitments Multiple sources, Open

Architecture

Back of Envelope

Needs Ad-Hoc Good enough

11/29/2014 @tonyjoyce 24

11/29/2014

13

• This is a perfect summary of the necessary and complete outcomes of "software development as a disciplined practice"

• “These procedures are intended to assure that the engineer’s product: will be fit for the use for which it is intended;

will conform to precise stable standards;

is robust enough to survive all foreseeable circumstances (including incorrect input); and

is conservatively designed with appropriate allowance for a margin of error.”

• These principles map to the Cynefin domains

“Inside Risks: Risks of Undisciplined Development” David L. Parnas

11/29/2014 @tonyjoyce 25

Software Development as Disciplined Practice (from Parnas “Inside Risks”)

Robust enough to

survive

Conservative with error

margin Fit for use

Conforms to standards

Simple

Complicated

Chaotic

Complex

11/29/2014 @tonyjoyce 26

11/29/2014

14

• Consider the 4 distinctive procedures as exit criteria applicable to the indicated domains

• Parnas’ procedures are fundamental to all types of engineering – Explained as distinct practices supporting disciplined design

• Practices that are less than conservative -- decisions that do not include appropriate affordances -- will regress into (or be stuck within) the disordered domain – the resulting ritualized practices can be termed Software

Alchemy

• This means that, absent luck and some serendipitous alignment of activities, there can be no single all-encompassing software development routine – This alignment is the tacit knowledge of deep expertise

Practice is only eventually perfect

11/29/2014 @tonyjoyce 27

• Requirements can be articulated by logical structures whereby, in the: – Complicated domain we have linear or

hierarchical (tree) structures

– Complex domain has V-shaped or pyramid like structures, with 3 or 4 points leading to lattice and meshes (including hexagonal tiles) • Traceability, such as tests to requirements, may be

an immutable constraint here

– Chaotic domain has fractal structures resulting from codependency and coevolution, e.g. a feature is “fit for use” (form and function)

The Requirements complexity that Cynefin exposes

11/29/2014 @tonyjoyce 28

11/29/2014

15

Quality Increases With The Increasing Complexity of Development

Fail safe; fault tolerant; DONE is

simple or complicated

Success is binary – all or nothing

Safe fail; resilient recovery; fitness is

unordered (can not be structured)

Success is binary – do something for arbitrary non-

Programming reasons

(Cynefin)

11/29/2014 @tonyjoyce 29

• In the simple domain the structures are inconsistent or short lasting

• We have with software development and software engineering a meta-pattern of a closed loop system with a clear mathematical or logical foundation underneath – This meets the tests for complexity described by

Snowden in his Star Canada 2012 video (url…) – As explained in video (~40 min mark), the logic

must be discoverable by many, not too difficult to share, and independently validated by experts, or those willing to do the legwork to learn. These are all characteristics of disintermediation

11/29/2014 @tonyjoyce 30

Software development is an ecosystem

11/29/2014

16

Enterprise Architecture Views

As-Be, Work on Capability will result in viable

standards

As-Is, the current theory of what we

have

To-Be, Fitness for use is emergent

Classification schemas,

taxonomy, artifacts (definitions)

(Cynefin)

11/29/2014 @tonyjoyce 31

• The Cynefin framework shows us the modulators which drive the system (a different meta-pattern than the ideal CAS with its attractors) – The fundamental role of expertise in the simple domain is what

holds this thesis together – And a surprising conclusion is acknowledging the complexity of

the discipline of computer engineering -- of which programming is a part

• From a KM point of view, without good anchors in the domains we drift into disorder and illusion – Taken as a whole we can see the broad skill set and deep

resources which are necessary for the CIO position – To be proficient in all domains is exceedingly difficult (similar to

systems engineering)

• Because requirements change in a mathematical progression, attempts to force use of a singular (requirements) database or standard (structured) documentation will not be sufficient to meet knowledge needs across the domains

11/29/2014 @tonyjoyce 32

There clearly is a mathematical progression in the complexity underlying this model

11/29/2014

17

Programming Practice Changes In Each Domain

Success in one domain does not grant success in another domain

11/29/2014 @tonyjoyce 33

• It illustrates various sources of information that I find I rely on as a CIO. It is organized according to Dave Snowden’s famous diagram of sensemaking practice – These four domains are the beginning of a knowledge map – Each of the four is a different type of conversation, both internally and externally.

CIO’s need to be conversant in all four! – Indeed, in larger audiences, as is the norm for Government work (with customer,

supplier, Gov’t PM and CIO/IT managers all in the room or phone call), a discussion may bounce through all 4 viewpoints in a single meeting

• This is the shape of the Software Development Elephant: – Status quo is a single, generally fixed framework that can’t easily be transferred

because the changing contexts are not easily captured – This map has evolved from a 2x2 matrix (Tacit vs. Explicit, Theory vs. Practice) – Note one domain is noisy with weak signals, the other is noisy with sparse strong

signals (fragmented knowledge, spaghetti-style KM) – In the complicated domain, remember that history is biased – the Gov’t is too

cautious about errors and doesn’t have forums equivalent to Twitter – Gov’t is intensely risk adverse; rightly so given the visibility and criticism of even the

smallest failure – For checklists, see Atul Gawande’s “Checklist Manifesto.” Checklists are contextual,

like a collection of evolving recipes. It is their shared use and adaptation that is an indicator of being in brownfield vice the back-of-envelope domain

What follows is a personal knowledge map

11/29/2014 @tonyjoyce 34

11/29/2014

18

CIO Practice Knowledge Map

• Tacit theories

• Experience

• Expertise

• Narratives (uncodified best practice)

• Implicit practices

• Checklists

• Safe-to-fail trials

• Disputation arenas (e.g. Twitter)

• Explicit theories

• Successful Programs

• History

• Contract clauses as good practice

• Explicit practices

• Emergent practice

• Community of Practice

• Curated websites

Greenfield Pre-

Development

Back of Envelope

Brownfield

Explicit-- Im

plicit

Practice - Theory 11/29/2014 @tonyjoyce 35

• Are these practices held in place as part of the system, as the result of the system, or as history (path dependencies)? – In Gov’t work, explicit knowledge is placed into estimates, schedules, scopes, contracts and

so on. These inform the pre-decision phase, and lock us into place in the complicated domain, because it takes a considerable amount of knowledge work to put all the paperwork together

– We can infer that tacit knowledge of the same kind informs and locks-in the back-of-the envelope decisions; this is the common sense definition of expertise

– Frameworks might be more apt to the explicit knowledge of Greenfields. Although I suspect many of them are closer to theories and are unproven or unreliable in practice. (See Dave Snowden’s SAFE polemics)

• The lower domains are filled with implicit practices and theories, and the Gov’t may not do well here because of the paperwork fetish of bureaucracy – Extensive documentation may fix us into a cycle – oscillating between the two explicit states

– The Gov’t demand for paper, despite frequent attempts to tame it (by tailoring and redefining various events), is a constant force – a skilled incompetence learned in the risk aversion of frequent criticism, blame and grief?

– We also suffer from risk aversion in communications, a “bad news blame game,” where the Gov’t is always wrong for not imposing an adequate framework, contract, oversight or control over the actions of all subsidiaries

The rigors of practice are not nearly as simple as what our common understanding of the terms “best, emergent & novel” are

11/29/2014 @tonyjoyce 36

11/29/2014

19

• My framework is an ecosystem, not a system, because engineering is a social (group) practice. Engineering is not a solitary pursuit, as Parnas notes in “Inside Risks”

• Best and other practices in my framework are a far cry from what our expectations of proficiency are. But that is the point of using the Cynefin framework; it is a knowledge tool that lets us explore our intuitions and beliefs – Also see Gary Klein's “Streetlights and Shadows: Searching for the Keys to Adaptive Decision

Making”

• Our understanding of this ecosystem is different when we are inside an organization vs. outside – Inside an organization, the knowledgeable or the responsible

people have names and we can easily say who did or didn’t do some phase of the practice

– But on the outside it is mostly invisible, as it is masked by the organizations image and identity

– Therefore from the outside we can mostly see only self-organized teams, failed teams, or failed opportunities to team

The patterns of learning here are of an autopoetic system

11/29/2014 @tonyjoyce 37

Patterns of Learning

Ad-hoc

Pre-D

Green

Brown

improvement

repeatability Status Quo

Knowledge leakage, loss & illusions

11/29/2014 @tonyjoyce 38

11/29/2014

20

• What works in one domain does not work nearly as well in another. There are contextual and cultural factors that come into play. This is one reason why transitions are confusing and fuzzy

• The outcome is a set of continually evolved computing frameworks that encapsulate knowledge

• We can see the slow advancement of science and engineering in this illustration

• We can also see the destructive power of illusions that keep us trapped in the status quo

• Innovation can break the cycle, though innovation is an out-of-band and unpredictable occurrence in this framework

11/29/2014 @tonyjoyce 39

What we can now see behind the curtain

Conclusion

• My view of software engineering through the lens of the Cynefin framework diverges from a traditional systems engineering approach

• And the view of IT shown here is considerably different from other uses of Cynefin for IT; particularly in looking for the types of practice manifest in each domain

• We need to pay attention to these frameworks of complexity as we seek to improve our practices for DOD Software Engineering

11/29/2014 @tonyjoyce 40