Upload
puppet-labs
View
5.706
Download
3
Tags:
Embed Size (px)
DESCRIPTION
"Oscar: Rapid Iteration with Vagrant and Puppet Enterprise" by Adrien Thebo, Software Engineer, Puppet Labs. Presentation Overview: When trying to debug software problems it's critical to be able to reproduce the original situation, and Puppet Enterprise is no exception to this. The Puppet Labs support team needs a way to rapidly reproduce customer issues across a wide range of operating systems and various versions of Puppet Enterprise. Oscar is a set of Vagrant plugins that handles machine provisioning and configuration to install Puppet Enterprise. It's designed to make building a Puppet Enterprise as simple as running `vagrant up`. While Oscar was originally built for supporting Puppet Enterprise, it provides a general platform for developing and testing against Puppet Enterprise. This talk will go over the history of Oscar, the current state, how it's used, and where to get it. Speaker Bio: Adrien Thebo has been in the Operations/Software development field since 2005, starting at small IT shop in Boise, Idaho. He started at Puppet Labs in 2011 on the Operations team, and used Puppet to run the Puppet Labs infrastructure. In 2013 he transferred to the Community platform team, working with contributors to merge their contributions in Puppet Core. He develops and maintains a number of Puppet modules and tools around Puppet, and when he's not writing code for Puppet then he's probably blogging about it.
Citation preview
OscarRapid Iteration with Vagrant
and Puppet EnterpriseAdrien Thebo | Puppet Labs
@nullfinch | Freenode: finch
Hi.
Introductions n’ stuffOn/Off Operations/Development, ~8 years
Devops before it was cool
Introductions n’ stuffPuppet Labs, 2.5 years
Operations Engineer, 2 yearsCommunity Software Engineer, 6 months
Things I do
r10kBetter Puppet deployment, powered by robots
Puppet modulespuppet-networkpuppet-portagepuppet-filemapper
So how about that Vagrant?
Mandatory audience fun time!Who has…
heard of Vagrant?tried Vagrant?used Vagrant regularly?developed on Vagrant or plugins?created Vagrant?
A brief introduction to Vagrant
Good idea, bad ideaGood idea: hand crafted, one of a kind artisan furnitureBad idea: hand crafted, one of a kind artisan VMs
Before VagrantHand rolled VMs
No automationUnreproducibleFull of forgotten hacks
Vagrant is…Local VM infrastructure without the pain
Easy to configureReproduciblePortable
Vagrant Workflowgenerateprovisionuse/breakdiscardIt’s your very own cloud!
Laying the GroundworkOperations @ Puppet Labs… Support @ Puppet Labs?Infrastructure & client support
The problem space
Supporting PE 2Need to reproduce customer problems
Various supported platformsVarious node configurations
Supporting PE 2Speed/reproducibility paramount
Have to meet SLAAlso handle other operations work
Support ♥ VagrantReceive ticketDuplicate client environmentBuild environmentReproduce & debugHelp customer^W^Wprofit!!!
LimitationsIf this presentation was all roses and butterflies
Then it would be very boring.
Choose your own adventureLike any tool, Vagrant makes tradeoffsSome use cases are easier than others
Vagrant excels at…Static/Stable environmentsConfigure VM definitionSet up provisioninggit commit -m 'Add Vagrantfile'
Vagrant struggles with…Highly mutable environmentsComplex provisioning stepsConfiguration reuse
Partial configurationDistributable configuration
The Vagrantfile DSLPro - all the power of RubyCon - all the power of Ruby
A descent into madness
(I’m sorry.)
DONT. DO. THIS.
# vi: set ft=ruby :require 'yaml'
begin # Load config yaml_config = File.join(File.dirname(__FILE__), "config.yaml") config = nil
File.open(yaml_config) do |fd| config = YAML.load(fd.read) end
nodes = config["nodes"] profiles = config["profiles"]
nodes.each do |node|
# Set default PE configuration, and allow node overriding of these values defaults = {"pe" => config['pe']}
node.merge!(defaults) do |key, oldval, newval|
if oldval.is_a? Hash newval.merge oldval else warn "Tried to merge hash values with #{key} => [#{oldval}, #{newval}], but was not a hash. Using #{oldval}." oldval end end
profile = node["profile"] node.merge! profiles[profile] endrescue => e puts "Malformed or missing config.yaml: #{e}" puts e.backtrace exit!(1)end
# This is an extension of the common node definition, as it makes provisions# for customizing the master for more seamless interaction with the base# systemdef provision_master(config, node, attributes)
# Manifests and modules need to be mounted on the master via shared folders, # but the default /vagrant mount has permissions and ownership that conflicts # with the master and pe-puppet. We mount these folders separately with # loosened permissions. config.vm.share_folder 'manifests', '/manifests', './manifests', :extra => 'fmode=644,dmode=755,fmask=022,dmask=022' config.vm.share_folder 'modules', '/modules', './modules', :extra => 'fmode=644,dmode=755,fmask=022,dmask=022'
# Update puppet.conf to add the manifestdir directive to point to the # /manifests mount, if the directive isn't already present. node.vm.provision :shell do |shell| shell.inline = <<-EOTsed -i '2 {/manifest/ !i\ manifestdir = /manifests}' /etc/puppetlabs/puppet/puppet.confEOT end
# Update puppet.conf to add the modulepath directive to point to the # /module mount, if it hasn't already been set. node.vm.provision :shell do |shell| shell.inline = <<-EOTsed -i '/modulepath/ {/vagrant/ !s,$,:/modules,}' /etc/puppetlabs/puppet/puppet.confEOT end
# Rewrite the olde site.pp config since it's not used, and warn people # about this. node.vm.provision :shell do |shell| shell.inline = %{echo "# /etc/puppetlabs/puppet/manifests is not used; see /manifests." > /etc/puppetlabs/puppet/manifests/site.pp} end
# Boost RAM for the master so that activemq doesn't asplode node.vm.customize([ "modifyvm", :id, "--memory", "1024" ])
# Enable autosigning on the master node.vm.provision :shell do |shell| shell.inline = %{echo '*' > /etc/puppetlabs/puppet/autosign.conf} endend
# Adds the vagrant configuration tweaksdef configure_node(config, node, attributes)
node.vm.box = attributes["boxname"]
# Apply all specified port forwards attributes["forwards"].each do |(src, dest)| node.vm.forward_port src, dest end if attributes["forwards"] # <-- I am a monster
# Add in optional per-node configuration node.vm.box_url = attributes["boxurl"] if attributes["boxurl"] node.vm.network :hostonly, attributes["address"] if attributes["address"] node.vm.boot_mode = attributes[:gui] if attributes[:gui]end
# Provide provisioning details for this nodedef provision_node(config, node, attributes)
# Hack in faux DNS # Puppet enterprise requires something resembling functioning DNS to be # installed correctly attributes["hosts_entries"].each do |entry| node.vm.provision :shell do |shell| shell.inline = %{grep "#{entry}" /etc/hosts || echo "#{entry}" >> /etc/hosts} end end
# Set the machine hostname node.vm.provision :shell do |shell| shell.inline = %{hostname #{attributes["name"]}} end
node.vm.provision :shell do |shell| shell.inline = %{domainname puppetlabs.test} endend
def install_pe(config, node, attributes)
# Customize the answers file for each node node.vm.provision :shell do |shell| shell.inline = %{sed -e 's/%%CERTNAME%%/#{attributes["name"]}/' < /vagrant/answers/#{attributes["role"]}.txt > /tmp/answers.txt} end
# If the PE version is 0.0.0, bypass puppet installation. # This could easily make later provisioning fail.
if attributes['pe']['version'].match %r[̂0\.0] warn "Requested PE version was set to #{attributes['pe']['version']}; will not automatically install PE" return end
# Assemble the installer command fragments = [] fragments << "2>&1" fragments << attributes['pe']['installer']['executable'] fragments << '-a /tmp/answers.txt' fragments << attributes['pe']['installer']['args'].join(' ')
installer_cmd = fragments.join(' ').gsub(':version', attributes['pe']['version'])
# Install Puppet Enterprise if it hasn't been installed yet. If the machine # is rebooted then this provisioning step will be skipped. node.vm.provision :shell do |shell| shell.inline = <<-EOTif ! [ -f /opt/pe_version ]; then #{installer_cmd}fi EOT endend
Vagrant::Config.run do |config|
# Generate a list of nodes with static IP addresses hosts_entries = nodes.select {|h| h["address"]}.map {|h| %{#{h["address"]} #{h["name"]}}}
# Tweak each host for Puppet Enterprise, and then install PE itself. nodes.each do |attributes| config.vm.define attributes["name"] do |node|
attributes["hosts_entries"] = hosts_entries
configure_node(config, node, attributes) provision_node(config, node, attributes) install_pe(config, node, attributes)
if attributes["role"].match /master/ provision_master(config, node, attributes) end end endend
Vagrant and networkingNo default internal networkingFlaky DHCPDNS resolution
Vagrant and networkingSolutions!
Run your own infrastructure!DHCPDBIND
The realityCan’t support every configurationShouldn’t support every configurationBatteries not included
Introducing Oscar
Oscar in a nutshellSet of Vagrant pluginsBuilt for mutable environments
An overviewvagrant-hostsvagrant-auto_networkvagrant-pe_buildvagrant-config_builderoscar
Vagrant hostsDNS record types:
starting with A: A, AAAA, AFSDB, APLstarting with C: CAA, CERT, CNAMEstarting with D: DHCID, DLV, DNAME, DNSKEY, DS…
We need: name -> ip address
Vagrant hostsInside of a transitory environmentQuery private network addresses-> /etc/hostsGo do something you care aboutOr manage BIND on $platform
Vagrant Auto-networkHave to add extra interfacesf$&k it. STATIC IP ADDRESSES FOR ALL
config.vm.network :private_network, :auto_network => true
Vagrant PE BuildPE configuration optimized for VagrantDownload installers on demand
Vagrant config builderConfiguration as data
Before config builderVagrantfile
config.vm.define :puppetmaster do |box| flavor = :centos_6 set_box box, S3_BOXES[:centos_64_nocm]
# NOTE: Hostonly _may_ mean no internets if this adapter gets found first!!! # Check /etc/resolv.conf ! box.vm.network :hostonly, "192.168.23.20"
# NOTE: Share folders, such as Git checkouts of the Puppet source code share_puppet_source box
box.vm.provision :shell, :inline => BOOTSTRAPS[flavor] provision_box box, 'server.pp'end
After config builderroles.yaml
---roles: puppetmaster: private_networks: - {auto_network: true} synced_folders: - {host_path: ~/Source/puppetlabs, guest_path: /puppetlabs} provisioners: - {type: hosts} - {type: pe_bootstrap, role: master}
puppetagent: private_networks: - {auto_network: true} provisioners: - {type: hosts} - {type: pe_bootstrap}
After config buildervms.yaml
---vms: - name: master box: centos_64_nocom roles: puppetmaster - name: agent box: debian_64_nocm roles: puppetagent
Pick your data sourceYAMLJSONXMLinterpretive dance
What does Oscar do?Dependencies!Everything is a standalone pluginMix and match
What does Oscar do?Templates and defaultsSane defaults to get you started
PE stack in a boxConfigure your development environment like productionDevelop your modules in complete isolationSimulate app deployments before going livePre-production in a box!Stable Puppet environment
What Oscar gets youAll the perks of VagrantMinimal user setupComplex config made easy
What Oscar gets youGoal - from zero to PE in vagrant up
Who uses it?Commercial SupportOpen Source SupportSales EngineeringPeople… and stuff…
Demo time!Basic spin upSales eng env
Questions?
Try it yourselfCode on GithubUnder active developmentcontributions accepted!
ThanksTom Linkin!Chris Barker!Charlie Sharpsteen!Hunter Haugen!Adam Crews!The fabulous Puppet and Vagrant community!Exclamation points!Many others!
CreditsLayout by NanocPresentation by Reveal.jsConsciousness by caffeine