Architecting a Large Software

Project – Lessons LearnedJoão Pedro Martins

IASA WORLD SUMMIT

Ground Rules and Expectations

∃ Break

Interaction expected – share your experiences

Software Architecture level (yes, technology is included)

João Pedro “jota” Martins

CTO @ |create|it|

Software Architect

TechEd 2006 – “Iron Architect” Winner

BizTalk Server MVP 2006-2011

Co-founder - GASP + APPU

Windows Azure Insider

What’s this session about?

Architecting a

Large

Software Project –

Lessons Learned

Overview

Internal Banking application for corporate credit

allocation. 3000+ users.

Dev started Jun-2011, R1 installed Jul-2014. Currently on

progressive rollout with updates.

Web-based, running inside a Windows Forms shell,

developed on Microsoft technologies.

Scrum - 56 sprints, ~15 dev-years

Core team: 14 (5 devs) + ~10 in other teams

Integrates with 11 systems

THE PROJECT

Current statistics

Application code - LOCs

Presentation: 63,664

Backend: 116,145

Business Rules (xml): 36,473

Database: ~4,000

Automated tests: 157,586

And also…

GeneratedCode LOCs

81 Visual Studio Projects

24 Services with213 Operations

86 Presentation views

1,481 classes, 96 database tables

1,755 Automated Tests, ~80% coverage of

backend code.

100% StyleCop compliance

> 15 libraries and packages~800,000

THE PROJECT

Stories and Issues

~550user stories

1,064 issues: 98% closed, 78% bugs. 0.04%

show stoppers.

In last 5 sprints…

56% time new developments, 44%

fixes/enhancements

Production updates every 4 days

Conceptual Architectureand Technologies

THE PROJECT

browser

Controller

View

ViewModel

dataservicesweb

Controller

View

Model

business logic

rule engine

integration

tables + views

data access

stored procedures

browser web services data

NET 4.0

CODESMITH +

NETTIERS TEMPLATES

T-SQL

SQL SERVER + VS

DATA TOOLS

SSMS

NET 4.0

WCF

NXBRE

LOG4NET

UNITY (ENTLIB)

VS UNIT TESTING

RHINO.MOCKS

SPREADSHEET.NET

QUICKGRAPH.NET

TOOLKIT CREATE

.NET 4.0

ASP.NET MVC3

LOG4NET

JSON.NET

UNITY (ENTLIB)

TOOLKIT CREATE

BALSAMIQ MOCKUPS

HTML + CSS

JQUERY

KNOCKOUTJS

UNDERSCOREJS

KENDOUI

MOMENTJS

TOASTR

Architecture design: from Theoretical

to Pragmatic to Implementable

Theoretical Implementable

ESB/Integration Bus BizTalk Server SOAP Web Services with

bank’s custom ServiceHost

Enterprise Rules

Engine

BizTalk Server BRE / Excel

Services

NxBRE (OSS) – XML Rules

Engine

Distributed Cache VelocityCTP3 / Windows

AppFabric

Cookies + Web Server

Afinity

Load Tests VS2010 Test with

distributed agents

VS2010 single server load

Application Lifecycle

Management (ALM)

TFS SVN + Issue Tracker + Excel

Product/Sprint Backlogs

Increasing levels of contraints limit the choices:

- Cost, time, available IT, risks, governance, security, skillsets, …

How does it look?THE PROJECT

Enough context

Now, what we learned…

Secrets for the success

Scrum

[Dev] Agility

Retrospectives

Frequent releases/user demos

Notion of Progress

User Interface

Motivated Team Focus

Usable and simple

Aesthetic

Innovative(not what you would expect in a bank)

Team principles - quality

Continuous improvement

Individual initiative&strengths

Redundancy in skills

Very low turnover

Work-life balance

Challenges

Latest&Greatest technology

Physical workspace

Great relationships

OVERVIEW

Classes with too many responsibilities

LEARNING #1

Service Implementation

Business Manager

Data Access Layer

Operation Contracts

Data Contracts

Business Manager

These

go

tto

o larg

e

WCF Physical Hosting (.svc’s)

How to work around this?

SOLID principles - Single Responsibility

Domain Driven Design – vertical slices of behavior

Separation of Concerns – partial classes + smaller

classes

When in doubt:

• Create another class

• Use interfaces

LEARNING#1

Logging

Do you have detailed logging enabled in your

production environments?

LEARNING#2

Is this useful?

Logging

Instead of this (the “Laconic Logging” Anti-Pattern)…

… do something like this…

LEARNING #2

Logging -hints

Log operation details vs Security/Privacy

Use end-to-end correlation.

Define your logging policy.

Beware of impact on performance.

Beware of storage space required. DO cleanup/archive.

LEARNING #2

Interfaces vs Inheritance

“Why extends is evil - Improve your code by replacing

concrete base classes with interfaces” (Allen Holub)http://www.javaworld.com/article/2073649/core-java/why-extends-is-evil.html

LEARNING #3

Dependency injection interfaces.

Very limited use of OO inheritance- Base Data Contracts with minimum property set (e.g., id + name)

- About ~10 uses in total

Classes and Interfaces

Class: state + behavior, represent entity

Abstract class: non-instantiable class, includes definition

of mandatory capabilities of subclasses.

Interface≅Abstract Class. First class citizen in C#.

Inheritance: is-a relationship, specialization.

Interface: implements/”can-do” relationship.In C#, multiple inheritance not supported. But classes can implement

multiple interfaces. And interfaces can inherit from other interfaces.

LEARNING #4

Dependency Injection

Indirection when instantiating objects:

- Container builds and reuses objects

- Flexibility and reduced coupling

- Supports mocking for automated tests

Interception:

- Cache

- Logging

- Profiling

- Exception Shielding

Fully configurable.

LEARNING #4

Caller

Target

Interceptor – Call handler

Dependency Injection

"All problems in computer science can be solved by

another level of indirection“ (David Wheeler)

LEARNING #4

ConfigIf asked for IServiceA create ServiceA instance

creates&usesA

Dependency Injection pitfalls

Initial setup can be demanding (skills+time)

Programming configurations (complex debugging)

Impact on runtime performance

Productivity (F12 goes to interface and not implementation)

LEARNING #4

Cache

Cache transparently via interception +

configuration.

Cache before accessing the network.

Initially designed for 3 freshness configs.

Idempotence as a side-benefit.

LEARNING #5

Presentation Layer

Whenever this assembly is used

… and a method with this name is called

… apply this interceptor with this configuration

Cache: ooops!

Business information presented must be accurate –

and data is not stable in external systems.

Very little external reference data.

User authorization.

Transparent, configuration-based cache, is convenient -

however, you can’t selectively expire contents

LEARNING #5

Transparent caching pitfals

Per operation call, not per business entityE.g., “method_1_2_3” as cache_key, insteadof “client_1”

You don’t control cache keys on the runtime (general purpose

cache-key: hash generator of input parameters)

Hard Impossible to track dependencies in complex business

models.

=> Cache invalidation is all-or-nothing

LEARNING #5

Automated tests

Requirement: 80% coverage by automated tests in

service layer.

Team principle: the AGILE team is not afraid to change

any piece of code for fear of breaking something.

Approach: service-level, end-to-end tests

- Visual Studio Tests framework

- Not unit tests integration, end-to-end tests

- Depend on external data

- Sprint Backlog: One service operation one test set

LEARNING #6

Automated tests – mmmm…

Test suite takes too long to run (~2-3h) :

- SQL scripts SQL Database Snapshots

- Service layer tests Business layer tests

External data not stable – mocks

But: how to test complex business cases dependent on

external data of which we can’t be sure?

LEARNING #6

Automated tests – more mmmm…

Have a Test King in the team to nurture and run tests

VS2012 test runner worse/slower than VS2010 (!)

Smart asserts can help improve code-coverage:

Tool recommendation: SSMS to generate T-SQL from data

LEARNING #6

Code Conventions

Agree on coding conventions and stylecop compliance

at start of project.

Architect/Dev Lead name all the main artifacts: service

contracts, database artifacts, etc. Strive for consistency.

LEARNING #7

Code Conventions notes

Focus on code legibility:

- Comment your code (take special care with algorithms)

- Don’t use var for non-anonymous types

- Don’t overdo Linq statements

Mistakes will happen, and rename refactors will be

needed (mixing PT with EN is frequent).

Standardize verbs in services/methods, db naming (ex:

List vs GetAll)

Do NOT argue tab size. When in doubt, use defaults.

LEARNING #7

Negotiation… with your team, and with the customer

Always voice your opinion, focusing on what you think

is the best for the project architecture-wise. Create a

trust relationship.

When your recommendation is not followed, and you

are sure you are right, present objective arguments –

don’t be emotional. Argue for as long as you must, but

no longer. And don’t say “I told you so”.

Accept defeat, make compromises.PS: you’ll losea lot.

LEARNING #8

Negotiation… some more notes

Be attentive of the other’s possible hidden motivations,

but be careful in exposing them.

Consensus is not always possible. Your options will be

questioned, and sometimes you will be wrong.

Remember the 3 views of architecture: theoretical,

pragmatic, implementable.

Ask open questions.

LEARNING #8

Functional team/domain experts

They are your peers, and part of the team. You

depend on well written and clear user stories.

Domain experts that understand Scrum,

priorities and constraints make the difference.

Rely heavily on them and their tools. When you

don’t understand, ask questions until you do.

LEARNING #9

Functional Team do’s and don’ts

Sometimes the way a story is written crystalizes a way

of implementation.

Some stories will be hard to understand and

decompose into tasks. Ask for clarifications and don’t

implement blindly.

Business context is sometimes missing.

Tendency to “follow the old ways”.

Tendency to abide to single-user/hierarchical requests.

Use Your Brain: design elegantly

You are not paid to write code, you are paid to think

and communicate.

Think things through before committing to a solution.

Try to isolate and design autonomous and change-

tolerant components.

Step back, look at the larger picture. As an Architect,

you DO NOT have to be a technical expert in

everything: focus on capabilities and structure.

LEARNING #10

Impediments to using your brain

Interruptions, background noise, phones, no

whiteboard, lack of natural light, music on

headphones, time or budget pressure, too

much coffee, personal problems, lack of sleep, …

What’s your style: collaborate then design, or design and then

collaborate? Isolate yourself to design. Make drawings, and then

document your proposed solution.

* Thinking is hard work.

Wireframes

Create and discuss

mockups pre-

implementation.Ps: if you have an UI

expert in your team,

don’t let business

experts create them.

Tool recommendation:

Balsamiq Mockups

LEARNING #11

Usability Tests

Usability tests are simple! Just looking at users during

training uncovered both problems and ideas for

improvement.

LEARNING #12

Revisiting 4 technical choices

KnockoutJS or MVC3?

Took time to decide and spike,

there was an initial setback with

KO and adoption was reversed.

2nd attempt and investment

proved correct.

NxBRE Rules Engine

QuickGraph.net 3.6 Distributed Cache

XML-based rules engine DLL.

XML file can be replaced without

recompilation. Works fine and is

fast, but hard to code and read.

Jury is still out.

Formal Architectural feedback was

tacitally dismissed as non-

pragmatic, and package was used.

VelocityCTP3 was refused as non-

supported. AppFabric not available

in Windows 2003. Oracle

Coherence never provided.

Lightning round

Your team is an

extension of your

body.

Use diagrams to communicate

and structure your ideas.

NOT MY SKETCH

Use an issue tracker, designate someone(s) to

do the triage, and configure mail alerts, your

pages/modules, team, and sprints.

Teach the client

how to use it for bugs

& enhancements.

Know thy user’s pc:

@start, 1GB RAM, IE8, WinXp, 1024px

Javascript + IE8 + 1GB RAM recipe

for disaster.

4 layers & no distributed cache

mean no real-time features

Be lazy. Don’t waste time

coding your own, special, data-

access layer/library/....

Scavenge codeplex, github,

nugget, etc. for assets&tools to

reuse or buy.

Use extension methods – don’t

pollute your classes with

auxiliary methods (ex: finders

in collections)

* and kill those «helper» classes, too

Just 3 more…

Humans make mistakes. Scripts don’t. Use scripts and

obsessively automate repetitive tasks or installations.

Know your branches, merges, shelves, labels,

versioning (just use best practices, don’t invent).

Innovating and surprising your customer, and the

cherry on the cake, makes a world of difference.

Closing message

It’s an architect job to address the

clients’ needs and deliver quality

products.Three stone cutters were asked about their jobs.

The first one replied, “I’m paid to cut stones.”

The second replied, “I use special techniques to shape stones in an exceptional

way, here let me show you.” He proceeded to demonstrate.

The third just smiled and said, “I build cathedrals.”

- Ricardo Semler Hope I helped!

João Pedro “jota” Martins

jota@create.pt

(+351) 96 782 5537

blogit.create.pt/joaomartins/

twitter.com/lokijota

pt.linkedin.com/in/joaopedromartins/

IASA WORLD SUMMIT

Thanks!Questions? Thoughts?

|create|it|

Started 2001 @ Lisboa, Portugal

Systems Integrator

Team of 26

Microsoft Gold Certified Partner

Azure

BizTalk

Office 365

SharePoint

Umbraco

NopCommerce