Upload
puppet-labs
View
1.491
Download
0
Embed Size (px)
DESCRIPTION
Julien Rottenberg, CityGrid. Presentation of the tools and workflow for our puppet setup. How puppet helps us managing 500+ servers in a hybrid environment O&O Datacenter and EC2, hands free. Watch the video at http://youtu.be/FPwga7HwomM PuppetCamp LA, May '12.
Citation preview
Puppet @ Citygrid
PuppetCamp LA 2012
Who am I ?
My name is Julien RottenbergRecovering System AdministratorPuppet user since 2009 (at GN)I like to write code and answer questions
Puppet promises
No need to ssh to a server to understand how it works
Look at the puppet code
"Those who cannot remember the past are condemned to repeat it."
George Santayana
literally... go to a server, run `history` as root.
Puppet promises
Guarantee that a server is setup exactly the same way on all environments
Puppet promises
Puppet promises
All the building steps are auditable - SDLCDrift management Open knowledge, no blackbox
Puppet promises
and finally...
Toolset
Open source tools :
● No need to reinvent the wheel● You are not alone with 'that problem',
get help
Toolset
Toolset
puppet doc ● validate code (puppet parser validate)● produce html doc
Running on commit through Jenkins ~ 10 sec
Toolset
Toolset
puppet-lint == coding standard
Takes ~ 2 minPart of the 'Ci-Game'
Toolsettrunk = work in progress - could break but fix ASAPproduction = tested code - do not break
Toolset Trunk --> Production (CherryPicking)We use svnmerge.py
Code in progress = technical debtIt needs to be merged and deployed or reverted Fail the build if pending Cherry Picks > 30 Can't CherryPick yourself ~ code review
ToolsetSvn even with svnmerge has limitations
Git ?
ToolsetVagrant
Sits on top of VirtualboxAllows to 'go crazy'Test, experiment, share configured box$ vagrant provision == puppet apply
ToolsetnVentory
As an External Node ClassifierClasses:
- default_install- gcd
ToolsetnVentory
Helper to build configuration files, etc.ie replacement for Stored Config
ToolsetForeman Api for puppet status, visibility of puppetrun
(Failed attempt on ENC - probably pebkac)
Toolset
Man ! that's a lot of tools !1!!
And it's fine, design by contract. Each application does one thing and was designed to do that thing only (so does it well)Pipe them
Dennis Ritchie
Toolset Boto for
● ec2● route53● elb● sqs● sns
River dolphin native to the amazon WP
Culture cloudEC2 glitches, expect them It used to `just work` :
● gmond (split view dns side effect, using gmetric)
● jmx (need to specify public name at runtime)● ELB pseudo load balancer● Bad neighbors
Culture cloud
One AMI to rule them all (map set per region)
# Build 10 nodes $ create_ec2_instance.py -e dev -s m1.medium -c thd -n 10 # assign them to the hadoop::datanode class (specific install on top of our vanilla)$ nv --name aws1devthd --addtonodegroup hadoop::datanode # send traffic & enable monitoring (configs to be updated on the next puppet run)$ nv --name aws1devthd --set status=inservice
Culture cloud
Culture Collaborate
Culture Collaborate
"If the company's doing well and people don't hate each other, you're probably doing ok."
John Allspaw, speaking at Devopsdays 2011 MV.
You guys ! Make it better : - Integrated function for 'secrets' (no get_var or hiera + gpg)Something like an encrypted data luggage
- puppet agent stability, puppetd vs cron ?
Guys !
Yesterday...
Two last things...
On call rotation side effect
Don't be a human keyboard
Shameless plug
We aggregate advertising & content from local businesses and distribute across our network of web & mobile sites
● Ingest : Hadoop & Cassandra● Serve : Solr & MongoDb + Tomcat● Code : Java mostly
Q/A
puppet doc : http://docs.puppetlabs.com/guides/style_guide.html#puppet-docpuppet-lint : http://puppet-lint.com/Jenkins : http://jenkins-ci.org/svn strategy : http://weblogs.java.net/blog/johnsmart/archive/2009/05/ci_branching_an.htmlsvnmerge.py : http://www.orcaware.com/svn/wiki/Svnmerge.pyvagrant : http://vagrantup.com/nventory : http://sourceforge.net/apps/trac/nventory/wikiforeman : http://theforeman.org/boto : https://github.com/boto/boto Don't be an Ahole : http://www.hollenback.net/DevOpsMeansDontBeAnAholeDevelopers on call - Mitchell Hashimoto : http://vimeo.com/31367609Don't be a human keyboard : http://sysadvent.blogspot.com/2010/12/day-13-dont-be-human-keyboard.html Citygrid is hiring : http://careers.citygrid.com/ Puppet Master LA User Group : http://www.meetup.com/LAPMUG/
This page : goo.gl/soxEs