Embed Size (px)
Citation preview
How important it is to invest in a good software factory and automate the build process as much
as possible
...if possible
Nicolas Mas - [email protected]
https://www.linkedin.com/in/nicolasmas
Acknowledgment
“Alone we can do so little; together we can do so much”
Eric Domingo Shuang Xia
Lokesh Boghte
Cedric Arnoult
Andrew Tiu
What this talk won’t do
⬗ Elaborate on Yet Another Software Factory Definition
⬘ Or only if, as a consultant once told me: “a common language needs to be
defined to understand each other” © Ajit Alwe ;-)
⬗ Enumerate a list of tools
⬗ Talk about any PM methodology
⬗ Show any code in action
⬗ Entertain you with funny pictures (sorry)
So what is it going to be?
⬗ A discussion about a real experience
⬗ back to basic: breaking down a complex problem into smaller chunks and
finding solution
⬗ An example of software factory designed to handle a specific problem
⬗ Bonus: Discuss if it’s worth automating the build of the software factory
as well as the automation of the build process (inception style)
⬗ B Tracks: if we really have time
Actually we have two topics here...
...and they are complementary
1. Investing in a good software factory
2. Automate the build process as much as possible
Investing in a good software factory: YES YOU CAN!
Part One:
Quick Survey
⬗ What’s a software factory for you?
⬗ Are you using one?
For us, it all started with a unique context...
⬗ A Java Spring project
⬘ Team of 5 developers
◆ A “A Team”, with distinct and partially overlapping skills and experience
⬘ A delivery manager
◆ NOT from a technical background
⬘ A product owner on the client side
◆ NOT from a technical background
◆ NOT collocated
⬘ Tight deadline
...which induced
⬗ Small team with high pressure
⬗ Delivery manager does not speak the same language as the developers
⬗ Product owner in another country
⬗ Customer dev. processes are very different
⬗ Deliverables:
⬘ Code quality
⬘ Documentation
⬘ processes on top of the usuals
⬗ Start from scratch
⬗ A bit of innovation
So we needed to:
Rethink the way we would build the product
&
Customize the process to build the product
WHY?
To manage the risks generated by these factors
&
Make sure we deliver an amazing great product
which will provide our customer the greatest
satisfaction (hint: bullshit)
Moving on from there
We can segregate our problem by vertical domains/aspects:
⬗ The product scope
⬗ Delivery management
⬗ Development process
⬘ Dependency Management
⬘ Code repository
⬘ Build process
⬘ Software Quality
⬘ Deployment process
⬗ Delivery process
⬗ Feedback
⬗ Communication
Mapping
⬗ The product scope
⬗ Delivery management
⬗ Development process
⬘ Dependency Management
⬘ Code repository
⬘ Build process
⬘ Software Quality
⬘ Deployment process
⬗ Delivery process
⬗ Feedback
⬗ Communication
=> Jira
=> Human (yes!)
=> Maven
=> GitHub
=> Jenkins
=> Sonar
=> Jenkins
=> Shell
=> Jira
=> Slack
A few remarks
⬗ The tools chosen are NOT the only one available:
⬘ Actually many !!
⬗ Rather than being a fixed structure, the Software Factory is a living
system and must be design accordingly
⬗ Setup, test before the development starts
⬗ Complete transparency within the team and with the client
What did we gain doing this?
⬗ We are confident we can deliver anything, anytime
⬗ Newcomers to the project are not a potential problem
⬗ We minimized the impact of human factor
⬗ The delivery pipe is unlikely to be a blocking factor for the project
⬘ We won’t be able to put it on the tools if we fail
Is there anything else we can improve?
⬗ We designed our software factory in such a way that our main point of
failure is the human factor
How could we minimize that?
Automate as much as possible the build process: YES YOU
CAN! (BUT...)
Part Two:
What we can’t automate
⬗ code commit by the developers
⬗ stakeholders communication
⬘ at least partially (Slack technically allows us to do it, but can’t adapt to the
stakeholder feedback).
What we decided not automate
⬗ Code review
⬗ Delivery
⬘ (YES this can be discussed … and YES I am aware of continuous delivery).
So finally, what we can automate
The build process + UAT deployment
“Jenkins is an open source continuous integration tool written in Java.Jenkins provides continuous integration services for software development. Itis a server-based system running in a servlet container such as ApacheTomcat.”
⬗ A maven build is triggered (code pulled from github)
⬗ Shell scripts deploy on UAT targets
⬗ Postprocess action: email + Slack message
So finally, what we can automate
The software quality - SonarQube
“SonarQube is an open platform for continuous inspection of code quality”
Code coverage
Code Quality
So how did it go?
⬗ Anytime delivery was real
⬘ BUT complexity increased with time and integration with other systems
(what a surprise!)
⬗ Code coverage tools sides effects
⬘ Require dev to code in a sometimes not so efficient manner
⬗ Client communication is better
⬘ Scope management
⬘ improved transparency
Delivery happened no matter what
Was it worth it?
YES
Bonus: Automating the creation and setup of the
software factoryCan or Cannot?
Part Three:
Automating the whole thing
⬗ We want to be able to recreate from a master the whole software
factory stack
⬘ In case of a crash (but can it be stateful?)
⬘ A new project is started in //, with similar constraints
⬗ We want to maintain a backup infra just in case
⬘ How do we keep them aligned?
⬘ AWS + local VBOx for example
Our current setup (manual)
What we manually setup so far
DB Stack CI Stack Staging Stack
Github
virtualbox
vagrant
Physical server
Ubuntu server Ubuntu serverUbuntu server
Dev machine: MB Pro
⬗ Eclipse IDE + JDK 1.7
⬗ Spring 4.0.4 + Apache CXF
⬗ Maven 3.0.3
⬗ Tomcat 7
⬗ Findbugs
Staging Server: VM: Ubuntu 14.04
LTS
⬗ JDK 1.7.0_11
⬗ Tomcat 7.0.37
DB Server: VM: Ubuntu 14.04 LTS
⬗ Oracle Entreprise Edition
11.2.0.3
⬗ Gemfire
CI Server: VM: Ubuntu 14.04 LTS
⬗ JDK 1.7.0_11
⬗ Jenkins
⬗ Archiva
⬗ GitHub
⬗ Jira Agile
⬗ Jira Crucible
⬗ Sonar Cube 4.1.2
Infra: Ubuntu 14.04 LTS
⬗ Vagrant
⬗ Saltstack
⬗ packer
⬗ Virtualbox
Our target through automation
DB Stack CI Stack Staging Stack
Github
virtualbox
vagrant
AWS
Physical server
WebApp
-
Software
Factory
Builder
Ubuntu serverUbuntu serverUbuntu server
The Software Factory Builder
⬗ Node.js (locally hosted) front end
⬗ Underlying technologies (main ones)
⬘ Packer (OS baking)
⬘ SaltStack (provisioning)
⬘ Vagrant (VM management)
What it looks like
⬗ A node.js interface
⬘ You can imagine anything for the VM type (VMWare, digital ocean etc.)
What it looks like
What it looks like
What it looks like
Pros & Cons
What’s cool..
⬗ Traditionally, software factories are built once and used for projects for
years. Stacks are updated, patched, reconfigured => lower and lower
reliability. In our case, we can rebuild it every time
⬗ We can duplicate on different infrastructures local or in the cloud
⬗ Once defined, the build pattern is fast to execute
What’s not…
⬗ Maintaining the setup and testing it before a “gold” version takes time
and is complexe => it’s an upfront investment you might not benefit from
⬗ You can’t automate 100%, rather 90%. So you still need a bit of work
Bonus Track - Did you know ?
“The mother of all demos”
http://en.wikipedia.org/wiki/The_Mother_of_All_Demos
⬗ Douglas Engelbart's December 9, 1968
⬗ 90-minute live public demonstration of the online system, NLS, they had
been working on since 1962
⬗ 1000 attendees and a demo of almost all features we use today
⬘ Mouse
⬘ hypertext, object addressing and dynamic file linking
⬘ shared-screen collaboration involving two persons at different sites
communicating over a network with audio and video interface.
http://web.stanford.edu/dept/SUL/library/extra4/sloan/mousesite/1968Dem
o.html
https://www.youtube.com/playlist?list=PL76DBC8D6718B8FD3
Watch it !!
