Embed Size (px)
DESCRIPTION
The shift to cloud computing means that organizations are undergoing a major shift as they develop scale-out infrastructure that can respond to apace of business change faster than ever before. Opscode Chef® is an open-source systems integration framework build specifically for automating the cloud by making it easy to deploy and scale servers and applications throughout your infrastructure. Join us for this session containing an introduction to Chef including: An Overview of Chef The Chef Architecture Cookbook Components System Integration Live demo launching a Java Stack on Amazon EC2, Rackspace, Ubuntu, and CentOS [Presented as part of the Open Source Build a Cloud program on 2/29/2012 - http://cloudstack.org/about-cloudstack/cloudstack-events.html?categoryid=6]
Citation preview
•U has a cloud
•Now what?
http://www.flickr.com/photos/ian_munroe/4758240536/
Congratulations!!!Congratulations!!!
But then what?But then what?
http://www.flickr.com/photos/doctorow/2698336843
You need to configure them
APIs are awesome!APIs are awesome!
•You can provision compute resources in seconds
•You can provision storage resources in seconds
•That’s cool.
http://www.flickr.com/photos/jdhancock/3634246981/
Chef can help with thatChef can help with that
•knife ec2 server create
•knife rackspace server create
•knife terremark server create
•knife voxel server create
•knife cloudstack server create
•knife kvm server create
•knife vsphere server create
•knife eucalyptus server create
•knife openstack server create
http://www.flickr.com/photos/kyz/3122499444/
See NodeSee Node
Application Server
See NodesSee Nodes
Application Server
Application Database
See Nodes GrowSee Nodes Grow
Application Server
Application Databases
Application Servers
Application Databases
See Nodes GrowSee Nodes Grow
Application Servers
Application Databases
Load Balancer
See Nodes GrowSee Nodes Grow
See Nodes GrowSee Nodes Grow
Application Servers
Application Databases
Load Balancers
See Nodes GrowSee Nodes Grow
Application Servers
Application Database Cache
Load Balancers
Application Databases
Tied together with Config
Tied together with Config
Application Servers
Application Database Cache
Load Balancers
Application Databases
Infrastructure is a Snowflake
Infrastructure is a Snowflake
Application Servers
Application Database Cache
Load Balancers
Floating IP?
Application Databases
Evolving ComplexityEvolving Complexity
Load Balancers
Application Servers
NoSQL
Database Slaves
Application Cache
Database Cache
Database
Complexity Grows Quickly
Complexity Grows Quickly
DC1
DC3
DC2
http://www.flickr.com/photos/16339684@N00/2681435235/
And it Continues to EvolveAnd it Continues to Evolve
Golden Images are not the answerGolden Images are not the answer
•Gold is heavy
•Hard to transport
•Hard to mold
•Easy to lose configuration detail
http://www.flickr.com/photos/garysoup/2977173063/
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
Typical Boring InfrastructureTypical Boring Infrastructure
•Move SSH off port 22
•Lets put it on 2022
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
New Compliance MandateNew Compliance Mandate
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite 1 2
3
4
5
6
•edit /etc/ssh/sshd_config
6 Golden Image Updates6 Golden Image Updates
•Delete, launch
•Repeat
•Typically manually
8
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite 1 2
3 4 5 6 7
9
10 11
12
12 Instance Replacements12 Instance Replacements
5
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite 1 2
4 5 6 7
8 9
10 11
12
3
•Don't break anything!
In a Maintenance WindowIn a Maintenance Window
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
• Invalid Configs
•Bob just got fired :(
With Different IP Addresses?
With Different IP Addresses?
http://www.flickr.com/photos/francoforeshock/5716969942/
Configuration DesperationConfiguration Desperation
Configuration Management and Automated Systems Integration
is the Answer
http://www.flickr.com/photos/philliecasablanca/3354734116/
Chef Solves This ProblemChef Solves This Problem
•But you already guessed that, didn’t you?
Collections of ResourcesCollections of Resources
•Networking
•Files
•Directories
•Symlinks
•Mounts
•Routes
•Users
•Groups
•Tasks
•Packages
•Software
•Services
•Configurations
•Other Stuff
http://www.flickr.com/photos/stevekeys/3123167585/
http://www.flickr.com/photos/glowjangles/4081048126/
Acting in ConcertActing in Concert
http://www.flickr.com/photos/28309157@N08/3743455858/
To Provide a ServiceTo Provide a Service
Chef is Infrastructure as Code
Chef is Infrastructure as Code
http://www.flickr.com/photos/louisb/4555295187/
•Programmatically provision and configure
•Treat like any other code base
•Reconstruct business from code repository, data backup, and bare metal resources.
Declarative Interface to Resources
Declarative Interface to Resources
•Define policy
•Say what, not how
•Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/
That looks like thisThat looks like this
extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
Or thisOr this
search(:users, '*:*') do |u| user u['id'] do uid u['uid'] shell u['shell'] home "/home/#{u['id']}" end directory "#{home_dir}/.ssh" do owner u['id'] group u['gid'] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode "0600" variables :ssh_keys => u['ssh_keys'] endend
Recipes and CookbooksRecipes and Cookbooks
•Recipes are collections of Resources
•Cookbooks contain recipes, templates, files, custom resources, etc
•Code re-use and modularity
•Hundreds already on Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/
•Chef-Client generates configurations directly on nodes from their run list
•Reduce management complexity through abstraction
•Store the configuration of your programs in version control
http://www.flickr.com/photos/ssoosay/5126146763/
NodesNodes
Upload your infrastructureUpload your infrastructureknife cookbook upload chef-client
knife cookbook upload java
knife cookbook upload jpackage
knife cookbook upload ntp
knife cookbook upload sudo
knife cookbook upload tomcat
knife cookbook upload users
knife cookbook upload sample
knife role from file base.rb
knife role from file tc.rb
knife role from file sample.rb
knife data bag create users
knife data bag from file users mray.json
Build it somewhereBuild it somewhere
#EC2
knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce -f m1.small -d omnibus -r 'role[base],role[tc],role[sample]’
#Rackspace
knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample]’
#CloudStack
knife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample]’
#Ubuntu Linux VM
knife bootstrap test.lab -i ~/.ssh/mray.pem -x ubuntu --sudo -d omnibus -r 'role[base],role[tc],role[sample]'
Tomcat stack deployedTomcat stack deployedec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Chef Run complete in 125.548799554 seconds
ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Running report handlers
ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Report handlers complete
Instance ID: i-ee18148b
Flavor: m1.small
Image: ami-0c6ebd65
Region: us-east-1
Availability Zone: us-east-1b
Security Groups: default
SSH Key: mray
Root Device Type: instance-store
Public DNS Name: ec2-107-21-179-169.compute-1.amazonaws.com
Public IP Address: 107.21.179.169
Private DNS Name: ip-10-120-255-91.ec2.internal
Private IP Address: 10.120.255.91
Environment: _default
Run List: role[base], role[tc], role[sample]
Tomcat stack deployedTomcat stack deployed
Update the sample cookbookUpdate the sample cookbook
$ knife cookbook upload sample
Uploading sample [0.0.2]
upload complete
Update the nodesUpdate the nodes$ knife ssh "role:base" "sudo chef-client" -i ~/.ssh/mray.pem -x ubuntu -a cloud.public_ipv4107.21.179.169 [Thu, 23 Feb 2012 03:30:22 +0000] INFO: *** Chef 0.10.8 ***107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List is [role[base], role[tc], role[sample]]107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List expands to [chef-client, ntp, sudo, users::sysadmins, java, tomcat, sample]107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Starting Chef Run for i-ee18148b107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Running start handlers107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Start handlers complete.107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Loading cookbooks [chef-client, java, jpackage, ntp, sample, sudo, tomcat, users]107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Storing updated cookbooks/sample/recipes/default.rb in the cache.<SNIP>107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] updated content107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] sending restart action to service[tomcat] (immediate)107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: Processing service[tomcat] action restart (tomcat::default line 42)107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: service[tomcat] restarted107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Chef Run complete in 15.170097638 seconds107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Running report handlers107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Report handlers complete
Tomcat stack updatedTomcat stack updated
http://www.flickr.com/photos/kathycsus/2686772625
• IP addresses
• Hostnames
• FQDNs
• Search for nodes with Roles
• Find configuration data
SearchSearch
pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb” owner "root" group "root” mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]”end
Pass results into Templates
# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |memb| -%> server <%= memb[:hostname] %> <%= memb[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>
Pass results into Templates
Jboss App
Memcache
Postgres Slaves
Postgres Master
So when thisSo when this
NagiosGraphite
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
Becomes this
Becomes this
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
Updates can be automatic
Updates can be automatic
NagiosGraphite
Count the resourcesCount the resources
Jboss App
Memcache
Postgres Slaves
• Load balancer config
• Nagios host ping
• Nagios host ssh
• Nagios host HTTP
• Nagios host app health
• Graphite CPU
• Graphite Memory
• Graphite Disk
• Graphite SNMP
• Memcache firewall
• Postgres firewall
• Postgres authZ config
12+ resource changes for 1 node addition
http://www.flickr.com/photos/evelynishere/2798236471/
CLONING CANNOT COPE WITH THISCLONING CANNOT COPE WITH THIS
• Chef can.
Build anythingBuild anything
•Simple internal applications
•Complex external applications
•Workstations
•Hadoop clusters
• IaaS infrastructure
•PaaS infrastructure
•SaaS applications
•Storage systems
•You name it
http://www.flickr.com/photos/hyku/245010680/
And manage it simplyAnd manage it simply
http://www.flickr.com/photos/helico/404640681/
•Automatically reconfigure everything
•Linux, Windows, Unixes, BSDs
•Load balancers
•Metrics collection systems
•Monitoring systems
•Cloud migrations become trivial
The Chef Community
The Chef Community
•Apache License, Version 2.0
•550+ Individual contributors
•100+ Corporate contributors
•Dell, Rackspace,VMware, RightScale, Heroku, and many more
•400+ Community cookbooks
•http://community.opscode.com
Questions?Questions?
http://www.flickr.com/photos/mrchippy/443960682/
Questions?Questions?
