Upload
codemotion
View
59
Download
3
Embed Size (px)
Citation preview
1
Microservices and Conway’s Law James Lewis @thoughtworks
ROME 24-25 MARCH 2017
2
CONSULTANTS!HELL YEAH!
4
5
BDD
6
8
“…organizations which design systems … are constrained to produce designs which are copies of the communication structure of those organizations”Melvyn Conway, 1968
The mirroring phenomenon is consistent with two rival causal mechanisms. First, designs may evolve to reflect their development environments. In tightly-coupled organizations, dedicated teams employed by a single firm and located at a single site develop the design. Problems are solved by face-to-face interaction, and performance “tweaked” by taking advantage of the access that module developers have to information and solutions developed in other modules. Even if not an explicit managerial choice, the design naturally becomes more tightly-coupled.
By contrast, in loosely-coupled organizations, a large, distributed team of volunteers develops the design. Face-to-face communications are rare given most developers never meet. Hence fewer connections between modules are established. The architecture that evolves is more modular as a result of the limitations on communication between developers.
10
http://www.hbs.edu/faculty/Publication%20Files/08-039_1861e507-1dc1-4602-85b8-90d71559d85b.pdf
"Exploring the Duality between Product and Organizational Architectures : A Test of the “Mirroring” Hypothesis"
loosely-coupled organizations ⇒ the architecture is more modular
11
tightly-coupled organizations ⇒ the design becomes more tightly-coupled.
organised around business capabilities
componentisation via services
products not projects
smart endpoints and dumb pipes
decentralised governance
decentralised data management
infrastructure automation
evolutionary design
designed for failure
organised around business capabilities
componentisation via services
products not projects
smart endpoints and dumb pipes
decentralised governance
decentralised data management
infrastructure automation
evolutionary design
designed for failure
How big are they?
18
microservices should be:
cheap to replace
and should allow you to go as “fast as possible”?
quick to scale
resilient
A TYPICAL COMPANYI got approximately 5 problems out of a maximum of about 20- I reckon
19
develop-ment
20
testing
architects
ops
PMO
HR
sales
marketingfinance
21
develop-ment
22
testing
architects
ops
PMO
HR
sales
marketing
finance
23
Project(process(
1(
Business
IT Solutions
dev
QA
Support
Idea
High level Estimation
and decision to proceed
Deployment
Testing
Business Requirement Document
Business Case
Project Brief
Functional spec
Initiation Doc
reqs. matrix
Detailed Estimate
Project Approve
d
TDD
PM docs
Test Script
s
Change Request
Requirements
clarification and
estimation
1010100010100101011010 Dev
Kick off Coding (Dev)
Additional <thing> raised
>10%
CAPEX
<10%
CAPEX
Support handover
User Acceptance
Testing
Normal Flow
Data Flow
Alternate Process
Flow
KEY
Dropped requirements become
Enhancements
24
develop-ment
25
testing
architects
ops
PMO
HR
sales
marketing
finance
26
each of these chords represents at least one queue
27
28
“There is nothing so useless as doing efficiently that which should not be done at all” Peter Drucker
29
SYSTEMS WOT I HAVE SEENa synthesis of many different experiences…
30
31
A capability is a combination of people, processes, systems that provides value to customers (internal or external)
The what of the business, not the how
32
delivering stuff selling stuff
Often capabilities are built by large teams
33
delivering stuff selling stuff
Using project-thinking, not product thinking*
* http://lmgtfy.com/?q=GDS+service+thinking
34
And the nicely decoupled capabilities end up looking like this
Feature starts dev
iteration 2
iteration 1
iteration 3
Feature deployed
regression testingperformance testing
deployment tests
cycle time
Anyone seen 6 week periods of “hardening”?
Unit TestsBuild Service TestsService A
BuildService B
End-to-end Tests
BuildService C
BuildService D
Service TestsUnit Tests
Unit Tests
Unit Tests
Service Tests
Service Tests
Anyone seen “fan-in” to end to end tests?
Anyone had to deploy everything all at once?
write spec
write code
test release
Without deploying into production, inventory is built up - inventory costs money and the more we have the more risky our deployments
?
38
beware the distributed monolith!
39
this is a recipe for a nervous breakdown
this is not “as fast as possible”
WHAT MIGHT GOOD LOOK LIKE?
41
42
Pauline Cafferkey
44
Team at the Royal Free
infectious
45
lab services
facilities
domestic
physio
micro biology
estates
pharmacy
finance
Royal Free Hospital
Capabilities
<other><other>
infectious
46
lab services
facilities
domestic
physio
micro biologyestates
pharmacy
finance
47“The approach favors agility over raw power <snip>”http://en.wikipedia.org/wiki/OODA_loop
48
1 Dr Rebecca Lester, Infectious diseases, Specialist registrar
2 Dr Damien Mack, Microbiology, Consultant
3 Dr Ruaridh Buchanan, Infectious diseases, Specialist registrar
4 Dr Simon Warren, Infectious diseases, Consultant
5 Oliver Carpenter, Infectious diseases, Nurse clinical practice educator
6 Breda Athanasopoulos, Infectious diseases, Senior matron lead high level isolation unit
7 Dr Tabitha Mahungu, Infectious diseases, Specialist registrar
8 Susan Coath, Infectious diseases, Ward sister
9 Zenaida Pandio, Domestic services, Housekeeper
10 Claire Harawa, Domestic services, Domestic team leader
11 Renato Pandio, Domestic services, Deep cleaner
12 Sheila Connolly, Patient environment, Head of patient environment
13 Peter Williams, Facilities, Security manager
14 Marchito Tumibay, Domestic services, Support worker
15 Michael Campo, Facilities, Porter
16 Lauren Geddes, Physiotherapy, Clinical lead cardiorespiratory care
17 Victoria Maddox, Laboratory service, Senior health care scientist
18 Nikki Williams, Infectious diseases, Ward sister
19 Dr Sanjay Bhagani, Infectious diseases, Consultant
20 Sarah Stanley, Patient at risk team, Nurse consultant
21 Dr Susan Hopkins, Infectious diseases and microbiology, Consultant
22 Torrain Exall, Physiotherapy, Clinical lead critical care
23 Dr Stephen Mepham, Infectious diseases and microbiology, Consultant
24 Dr Michael Jacobs, Infectious diseases, Consultant
25 Simon Rattenbury, Laboratory service, Head of laboratory service
26 Bryan O’Farrell, Pharmacy, Senior Pharmacist
27 Adrien Cooper, Estates, Director of estates
28 Robert Elliot-Cooke, Pharmacy, Senior pharmacist
29 Allan Harris, Estates, Engineering supervisor
30 Sumal Kuranaratne, Facilities, Support services manager
31 Dave Reavy, Facilities, Head of security and support services
32 Garba Ibrahim, Domestic services, Domestic team leader
33 Stephen King, Royal Free Charity, Head of finance
49
1 Dr Rebecca Lester, Infectious diseases, Specialist registrar
2 Dr Damien Mack, Microbiology, Consultant
3 Dr Ruaridh Buchanan, Infectious diseases, Specialist registrar
4 Dr Simon Warren, Infectious diseases, Consultant
5 Oliver Carpenter, Infectious diseases, Nurse clinical practice educator
6 Breda Athanasopoulos, Infectious diseases, Senior matron lead high level isolation unit
7 Dr Tabitha Mahungu, Infectious diseases, Specialist registrar
8 Susan Coath, Infectious diseases, Ward sister
9 Zenaida Pandio, Domestic services, Housekeeper
10 Claire Harawa, Domestic services, Domestic team leader
11 Renato Pandio, Domestic services, Deep cleaner
12 Sheila Connolly, Patient environment, Head of patient environment
13 Peter Williams, Facilities, Security manager
14 Marchito Tumibay, Domestic services, Support worker
15 Michael Campo, Facilities, Porter
16 Lauren Geddes, Physiotherapy, Clinical lead cardiorespiratory care
17 Victoria Maddox, Laboratory service, Senior health care scientist
18 Nikki Williams, Infectious diseases, Ward sister
19 Dr Sanjay Bhagani, Infectious diseases, Consultant
20 Sarah Stanley, Patient at risk team, Nurse consultant
21 Dr Susan Hopkins, Infectious diseases and microbiology, Consultant
22 Torrain Exall, Physiotherapy, Clinical lead critical care
23 Dr Stephen Mepham, Infectious diseases and microbiology, Consultant
24 Dr Michael Jacobs, Infectious diseases, Consultant
25 Simon Rattenbury, Laboratory service, Head of laboratory service
26 Bryan O’Farrell, Pharmacy, Senior Pharmacist
27 Adrien Cooper, Estates, Director of estates
28 Robert Elliot-Cooke, Pharmacy, Senior pharmacist
29 Allan Harris, Estates, Engineering supervisor
30 Sumal Kuranaratne, Facilities, Support services manager
31 Dave Reavy, Facilities, Head of security and support services
32 Garba Ibrahim, Domestic services, Domestic team leader
33 Stephen King, Royal Free Charity, Head of finance
50
1 Dr Rebecca Lester, Infectious diseases, Specialist registrar
2 Dr Damien Mack, Microbiology, Consultant
3 Dr Ruaridh Buchanan, Infectious diseases, Specialist registrar
4 Dr Simon Warren, Infectious diseases, Consultant
5 Oliver Carpenter, Infectious diseases, Nurse clinical practice educator
6 Breda Athanasopoulos, Infectious diseases, Senior matron lead high level isolation unit
7 Dr Tabitha Mahungu, Infectious diseases, Specialist registrar
8 Susan Coath, Infectious diseases, Ward sister
9 Zenaida Pandio, Domestic services, Housekeeper
10 Claire Harawa, Domestic services, Domestic team leader
11 Renato Pandio, Domestic services, Deep cleaner
12 Sheila Connolly, Patient environment, Head of patient environment
13 Peter Williams, Facilities, Security manager
14 Marchito Tumibay, Domestic services, Support worker
15 Michael Campo, Facilities, Porter
16 Lauren Geddes, Physiotherapy, Clinical lead cardiorespiratory care
17 Victoria Maddox, Laboratory service, Senior health care scientist
18 Nikki Williams, Infectious diseases, Ward sister
19 Dr Sanjay Bhagani, Infectious diseases, Consultant
20 Sarah Stanley, Patient at risk team, Nurse consultant
21 Dr Susan Hopkins, Infectious diseases and microbiology, Consultant
22 Torrain Exall, Physiotherapy, Clinical lead critical care
23 Dr Stephen Mepham, Infectious diseases and microbiology, Consultant
24 Dr Michael Jacobs, Infectious diseases, Consultant
25 Simon Rattenbury, Laboratory service, Head of laboratory service
26 Bryan O’Farrell, Pharmacy, Senior Pharmacist
27 Adrien Cooper, Estates, Director of estates
28 Robert Elliot-Cooke, Pharmacy, Senior pharmacist
29 Allan Harris, Estates, Engineering supervisor
30 Sumal Kuranaratne, Facilities, Support services manager
31 Dave Reavy, Facilities, Head of security and support services
32 Garba Ibrahim, Domestic services, Domestic team leader
33 Stephen King, Royal Free Charity, Head of finance
“If you imagine all the functions a hospital has to have, not only the doctors and nurses and physiotherapists and pharmacists but estates management and food and facilities and laundry — they all have to be involved, and they all have to be specially trained to be safe to work in that environment. “ Dr Michael Jacobs - Consultant, infectious disease
51
infectious
52
lab services
facilities
domestic
physio
micro biology
estates
pharmacy
finance
Royal Free Hospital
Capabilities
<other><other>
infectious diseases
53
Ebola treatment
54
single purposecross-functional teams
focussed entirely on ‘customer’ outcome
55
In our study at a large Australian Telco, we found that stories leaving a team cycled 12 times slower than those that stayed in the team.
56
This is true for Trauma teams in hospitals
And Formula 1 teams
And “reinforced battle groups”
A SOFTWARE EXISTENCE PROOF…
57
develop-ment
58
testing
architects
ops
PMO
HR
sales
marketing
finance
x-func teams organised around lines of business
59
ops
HR
sales
marketing
finance
insurance company
insurance company
separate lines of business
home motor life
separate lines of business
and cross-cutting capabilities
home motor life
my account
cross-functional teams delivering lines of business
x-func teams organised around lines of business
66
ops
HR
sales
marketing
finance
x-func teams organised around lines of business
67
ops
HR
sales
marketing
finance
x-func teams organised around lines of business
68
ops
HR
sales
marketing
finance
x-func teams organised around lines of business
69
ops
HR
sales
marketing
finance
71
72
DEVOPS!!!Well, build it, run it…
x-func teams organised around lines of business
73
ops
HR
sales
marketing
finance
74
ops
HR
sales
marketing
finance
build it / run it
build it / run it
75
HR
sales
marketing
finance
each of these capabilities can be tested and deployed independently
home motor life
my account
build it / run it
77
HR
sales
marketing
finance
78
79But can we go further?
getting there…
“If you imagine all the functions a hospital has to have, not only the doctors and nurses and physiotherapists and pharmacists but estates management and food and facilities and laundry — they all have to be involved, and they all have to be specially trained to be safe to work in that environment. “ Dr Michael Jacobs - Consultant, infectious disease
80
build it / run it
81
HR
sales
marketing
finance
HR
sales
marketing
finance
build it / run it
83
http://nomad8.com/squadschaptersguilds/https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/
84
market segments
85
Value streams
86
teams
each teamowns one or more services
~160-200~10-20
multiples thereof
89Thomas J. Allen, 1977
0 10 20 30 40 50 60 70 80 90 1000
0.05
0.10
0.15
0.20
0.25
0.30
m
Probability of weekly interaction
x
x x
xxxxxxxxxxx
x xx
x xx x
The effect of distance on communication
communication frequency varies with the inverse square of distance
co-locate as much as possible
take advantage of serendipitous conversations
as you chunk up from team to value stream to market segment to organisation, use Conway’s Law, Allen’s research and Dunbar’s number to preserve low-coupling and high cohesion
and remember - whenever work leaves your team, you incur cost
92
The “chunking up from microservices to teams to value streams to market
segments to organisations" practice onion*
*I might need a better name for this
My hypothesis is that you can use organisational boundaries to reason about which testing types to apply and which integration patterns to use
94
Low change rate
inter-market segment
High stability
Semantic Versioning
Contract Testing
Tolerant Reader“Conversational change”
95
higher change rate
inter-value stream
lower stability
Semantic Versioning
Contract Testing
Tolerant Reader“Conversational change”
96
higher change rate
inter-team
lower stability
Semantic Versioning
Contract Testing
Tolerant Reader“Conversational change”
97
highest change rate
intra-team
lowest stability
Semantic Versioning
Contract Testing
Tolerant Reader“Conversational change”
https://www.flickr.com/photos/snippets101/2686592689/https://creativecommons.org/licenses/by-nd/2.0/
Thoughts to finish
successfully adopting the microservice-style of architecture requires more than just componentisation via services
organised around business capabilities
componentisation via services
products not projects
smart endpoints and dumb pipes
decentralised governance
decentralised data management
infrastructure automation
evolutionary design
designed for failure
The Inverse Conway ManoeuvreDesign the organisation you want, the architecture will follow (kicking and screaming).Evan Bottcher, ThoughtWorks
CONWAY’S LAW SENDS US SIGNALS
•constantly ordering work across capabilities
•splitting stories between teams
•moving people around
•deploying in lock-step
• fan-in for end-to-end testing
THERE IS A HIGH COST ASSOCIATED WITH WORK LEAVING YOUR TEAM
• raising tickets with the “platform team”
•handing off work to other departments
•constantly starting and stopping projects
the last word
[email protected] @boicy
thanks!https://creativecommons.org/licenses/by-nc-sa/3.0/