Agile 2013 Talk: How DevOps Changes Everything

Preview:

DESCRIPTION

The most important DevOps things I’ve learned over the last 4 years. I presented this at Agile 2013 in Nashville, TN. #agile #devops #automation #culture #distributedTeams #measurement #sharing #bestPractices

Citation preview

How DevOps changed everything

Karthik Gaekwad

@iteration1 #AgileDevOpsWednesday, August 7, 13

Howdy!• I’m Karthik Gaekwad

• Senior Web Engineer

• Mentor Graphics Embedded

DevOps and the CIA

• From Austin, TX

• We have great BBQ, chill people and brutal summers!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

My Background

• National Instruments (NI)

• Ecommerce Team (Agile)

• R&D Cloud Team (DevOps)

• Mentor Graphics Embedded

• Cloud Team (DevOps)

• Organizer: CloudAustin, DevOpsDays Austin, TheAgileAdmin

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Audience Survey

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

The most important DevOps things I’ve learned over the last 4 years

Let’s not waste time!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

First, we have to talk about agile...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

There was a waterfallRequirements

Design

Implement, Test

Maintain

Deploy

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

RolesRequirements

Design

Implement

Maintain

Deploy

Product Manager, Business Analyst

Business Analyst, Web Designer, Architect

Programmer Analyst, Web Developer

Programmer Analyst, System Administrator

System Administrator

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

S#AgileDevOps@iteration1 Agile 2013

Waterfall..

Wednesday, August 7, 13

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013

“But I told you this already”

Communication

Wednesday, August 7, 13

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013

“That is not my job!”

Accountability

Wednesday, August 7, 13

DevOps and the CIA#AgileDevOps@iteration1 Agile 2013

“That is not a priority”

Time to deliver

Wednesday, August 7, 13

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

**About the time I started...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

New Product: Partslist

Collaborate and tag parts to build large systems

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Transitioned to SCRUM2 Week Iterations

Targeted Feature setsDemo at end of iteration

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

The Product Team > Traditional Organization

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

ResultsWorking together, building things that were

wanted

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Results

Between Business and IT

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Results

Better products being delivered, at a faster pace.

Methodology expanded to all other web teams...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Results

I love my job! #ForReal

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Agile: Retrospective

• Scrum brought team together.

• Building products on time, which were actually needed.

• Superior collaboration between business and IT.

• Organization shift -> Scrum

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Ops conversations

“Agile is cool! but it’s hard for ops to be

agile because it takes months to buy and provision servers.”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Ops conversations

Deployer Dude

“All day everyday”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

But in the community...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

But in the community...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Go build us cloud products

now”Sincerely,

Management

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Cloud is AWESOME!”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Cloud is OSSM”

-Dave Nielson

-NIST

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Cloud is AWESOME OSSM”

On-demand

Self-service

Scalable

Measurable

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“Give me API or give me death”

@littleidea

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Agile DevOps

Cloud Team”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps

“A cultural and professional movement”

Adam Jacobs

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps

“System administrators participating in an agile development process alongside developers and using many

of the same agile techniques for their systems work.”

The Agile Admin Blog

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Alison ChaikenEmbedded Linux, Thought Leader

“Get rid of the devs; get rid of the ops; let’s

just find people who can solve problems”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULTURE

MEASUREMENT

AUTOMATION

SHARING

Pillars of DevOps

John Willis

Damon Edwards

#AgileDevOps@iteration1 Agile 2013

By:

Wednesday, August 7, 13

DevOps 101Ideas and Stories

Free!! No charge!!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

“We’re making cloud products now”

“Agile DevOps

Cloud Team”

#AgileDevOps@iteration1 Agile 2013

Buzzword friendly!

Wednesday, August 7, 13

We had a goal:

“Build a cloud authentication prototype”

Wednesday, August 7, 13

CULT

URE

Our 1st sprint(s) sucked!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

But we suffered from “Wall of confusionitis”

Devs: “REST Services!”

Ops: “System Stability!”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Not seeing eye to eye#Frustrating #Angry

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Really, this is “Platform Architecture”, but I’m too lazy to redo the image

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Devs and ops understand good architecture. #GoodPlaceToStart

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Think platform first, then app later. #PaaS #REST

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

• Everyone understood architecture

• Helped everyone speak same language

• Reduced friction

• Called it “The System Model”

AUTO

MATIO

N

#AgileDevOps@iteration1 Agile 2013

1st win: Architecture

Wednesday, August 7, 13

1st win: Architecture

• “Model Driven Automation”

• Championed by ops

• Everyone needs to understand the model

• Figure out how to deploy the model

AUTO

MATIO

N

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

1st win: Architecture• Programmable Infrastructure Environment

(PIE)

• Based on version controlled architecture diagram

• Provisioned cloud boxes

• Ran deploys

• Set standard naming for boxes

• On demand deploy for apps

AUTO

MATIO

N

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Outcomes

• Ops learns Java!

• Ops learns version control!

• Developers get comfortable with production environments!

• ...Now we’re speaking many common languages!

AUTO

MATIO

N

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

No Chef? No Puppet?

AUTO

MATIO

NAre y’all crazy???

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

AUTO

MATIO

NEverybody be cool

• We believe the cleanest abstraction layer is starting with the entire system you’re trying to model

• In 2009, chef and puppet were younger

• We were scared of Ruby

• We needed windows cloud support

• Plus dawg, cloud was a wild wild world then!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

AUTO

MATIO

N

Please automate!!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

AUTO

MATIO

N

Do I pick Chef or Puppet??

“whatever makes you happy”

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

SHARIN

G

For chef/puppet, you’ll need to code.

#NotJust4Devs #NotScary

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

SHARIN

G

Stuck? Just ask your DevOps dev guy!

#NotJust4Devs

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

SHARIN

G

Don’t hide.Share the knowledge

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

SHARIN

G App configs

Wednesday, August 7, 13

SHARIN

G App configs

Easy Change Management

Let your automation control these kind of files

Wednesday, August 7, 13

Tools PhilosophyDEVO

PS

• Needed to work for cloud

• Facilitate collaboration

• Charts and graphs

• Everyone wants to use it

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

• Monitoring = Cloudkick

• Monitor servers

• Monitor code (custom synthetic checks)

We use #datadog now

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

Teach your devs how to monitor their code, and they will love the tool

Example: #NewRelic

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

If it measures business metrics, management will love you too!

#statsd #dogstatsd

HINT: Management pays you. Make them happy

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

WASTE OF TIME:

SSH’ing into a box to read log files

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

Get a logging solution.

Plethora of choices!

#Logstash #Graylog #Splunk #Sumologic

Which one? CloudAustin July Meetup

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

SHARIN

G

You’ll need more toolsFind them or write them!

You’re devops now!

Tools Philosophy

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Tools PhilosophyM

EASU

RE

DevOps victory:

“People” use tools that solve their core problems

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps 201MOAR Ideas and StoriesFor ops who can code

and dev not scared of linux

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Is different for every team.

What works for your team, works for your team

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

This worked for us...#AlwaysBeInclusive

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

Stuff Breaks

Like...all the time

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE

#DontDoThis#AgileDevOps@iteration1 Agile 2013

Wednesday, August 7, 13

CULT

URE

Be Cool. Fix the issue at handIf you panic, everyone else does too.

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE Post Mortem

Do a Post mortem

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE Post Mortem

It’s okay to admit mistakes

You’re not Batman...

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE Post Mortem

Why x5Add more tests

Make sure it doesn’t happen again

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

CULT

URE RELEASE OFTEN

Release with 4 changes is less risky than

Release with 40,000 changes

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS Testing

Wednesday, August 7, 13

DEVO

PS Test Everything

Test the codeTest the infrastructureIt’s all the same now

#infrastructureAsCode

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS Free Time?

Write more tests!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS

CONTINUOUSLY

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS CONTINUOUSLY

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS

Manual ChangesTo

Production

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DEVO

PS Manual Changes

Don’t do this.

Do this:1. Make manual changes in dev

2. Add to your chef/puppet script3. Release to test/prod

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps 301MOAR MOAR Tips

For when your team grows

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Security in the loop

Add security audits as a part of your sprint.

We do an audit periodically

#AgileDevOps@iteration1 Agile 2013

@Gauntlt @RuggedDevOps

Wednesday, August 7, 13

Security in the loop

Hire an infosec pro to your devops team

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Security in the loop

No budget?

Try a cross functional working group

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Expanding Teams?

• Rotate times for meetings (status/planning)

• Scrum standups were recorded and dropbox(ed)

• “OK to call” philosophy

• Campfire chat (saves history)

• Ownership of new features

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Expanding Teams?

• No secret: Face to face is always better

• Prioritize travels for visits to teammates

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Everyone asking for DevOps?

The embedded DevOps model works*

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Everyone asking for DevOps?

Make sure your team culture is strong enough so you don’t lose person

permanently :)

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Start Small...

Don’t say “WE’RE 100% DEVOPS TODAY”

Wednesday, August 7, 13

DevOps 401MOAR MOAR MOAR Ideas

For next time. I don’t have any. Tweet #agiledevops for your best ideas!

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

STILL STUCK?Just ask!

We’re devops, and this is real(even though we can’t define it)

Twitter: #devopsamahttp://devopsweekly.com/

Go to a devopsdayshttps://groups.google.com/forum/#!forum/devops

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps: Retrospective• DevOps and CAMS

bring team together.

• Building stable products on time, which were actually needed.

• Superior collaboration between devs and ops

• Organization shift -> DevOps

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

DevOps: Retrospective• DevOps and CAMS

bring team together.

• Building stable products on time, which were actually needed.

• Superior collaboration between devs and ops

• Organization shift -> DevOps

#AgileDevOps@iteration1 Agile 2013

• Scrum brought team together.

• Building products on time, which were actually needed.

• Superior collaboration between business and IT.

• Organization shift -> Scrum

Wednesday, August 7, 13

Thank you!

Let’s work together, and solve the problems that our business wants us

to!

#DevOps

#AgileDevOps@iteration1 Agile 2013Wednesday, August 7, 13

Recommended