Upload
puppet-labs
View
13.687
Download
0
Tags:
Embed Size (px)
DESCRIPTION
"Hacking The Data out of Puppet" by Dan Bode, Business Development at Puppet Labs Watch videos from PuppetConf 2012 here: www.puppetlabs.com/community/videos/puppetconf Speaker Bio: Dan has worked in the technology industry as a consultant and software developer for almost a decade. He has spent most of the last 4 years building infrastructure automation solutions and teaching people how to use Puppet. He currently works in the Business Development at PuppetLabs where he spends his time researching technologies and figuring out how they can be integrated with Puppet to provide value. Learn more about puppet: http://bit.ly/QQoAP1
Citation preview
Dan Bode| Puppet Labs [email protected]
Hacking The Data out of Puppet
# puppetconf # puppetize @ puppetlabs
Who is this talk for?
• People who already understand Puppet
• Developers or people who are dev-curious
# puppetconf # puppetize @ puppetlabs
Shameless plug
# puppetconf # puppetize @ puppetlabs
What is it about?
• Deconstructing Puppet to data
Dissecting a Puppet Run
Puppet as Data
# puppetconf # puppetize @ puppetlabs
Facter, who am I?
Agent
Hi! your facts are: kernel=linux ipaddress=10.0.0.3 macaddress=…
# puppetconf # puppetize @ puppetlabs
Hi Mr. Master, I need a catalog. Here
are my facts
http://www.dgcomputers.org/testimonials.php
Agent
facts
# puppetconf # puppetize @ puppetlabs
facts
Agent Thanks for you facts. I’ll just store them in
PuppetDB
PuppetDB
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
ENC
Agent
Mr. ENC, is this host defined as an external
node? Yep, he should be an
apache server. Here is the definition
nodes
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent Just compiled your
catalog. One sec while I store it in PuppetDB.
catalog
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent
catalog
Here is your catalog. Send me a report and let me know how it
went!
catalog
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent
catalog
I hate to be a bother, but can you compute
the md5sums of a few files?
catalog
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent Just finished applying. Here are the results.
report
catalog
Interacting with Puppet’s Data
Termini and the indirector
# puppetconf # puppetize @ puppetlabs
facts find from terminus facter
Agent
# puppetconf # puppetize @ puppetlabs
catalog find from terminus rest
http://www.dgcomputers.org/testimonials.php
Agent
facts
# puppetconf # puppetize @ puppetlabs
facts
Agent facts save to terminus
puppetdb
PuppetDB
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
ENC
Agent
node find from terminus exec (or
ldap)
nodes
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent catalog find from
terminus compiler
catalog
# puppetconf # puppetize @ puppetlabs
PuppetDB
facts
Agent catalog save to
terminus puppetdb
catalog
# puppetconf # puppetize @ puppetlabs
Facter ENC
Disecting a Puppet Run
Com
piler
Config Catalogs
Nodes/Manifest
Reports
Facts
# puppetconf # puppetize @ puppetlabs
CLI commands
puppet facts find
puppet node find
puppet catalog find
# puppetconf # puppetize @ puppetlabs
CLI Puppet Facts
# mkdir –p /tmp/yaml/facts
# puppet facts find node_name --render-as yaml \
> /tmp/yaml/facts/node_name.yaml
# puppetconf # puppetize @ puppetlabs
Creating a node (optional):
# puppet node find node_name \
--node_terminus=exec \
--external_nodes=/etc/puppet/nodes.sh \
--facts_terminus=yaml \
--clientyamldir=/tmp/yaml/ --render-as=yaml \
> /tmp/yaml/nodes/node_name.yaml
# puppetconf # puppetize @ puppetlabs
Creating a catalog:
# puppet catalog find node_name \
--facts_terminus=yaml \
# puppet catalog find node_name \
--node_terminus=yaml \
--manifest=/etc/puppet/manifest/site.pp \
--modulepath=/etc/puppet/modules/
--clientyamldir=/tmp/yaml/ > /tmp/catalog.yaml
# puppetconf # puppetize @ puppetlabs
Fun with IRB
Puppet::Node::Facts.indirection.find facts
Puppet::Node.new nodes
Puppet::Catalog.indirection.find catalog
# puppetconf # puppetize @ puppetlabs
IRB Facts
irb:> require ‘puppet/face’
> facts=Puppet::Face[:facts, :current].find('node’)
# puppetconf # puppetize @ puppetlabs
Access a Fact value (irb):
…
> facts.values['ipaddress']
=> "10.0.2.15"
# puppetconf # puppetize @ puppetlabs
Creating a node (from irb):
…
> node=Puppet::Node.new('node_name',
{:classes => {:foo => {:bar => :baz}}})
>node.merge(facts.values)
# puppetconf # puppetize @ puppetlabs
Creating a catalog:
…
irb> catalog=Puppet::Resource::Catalog.indirection.\
find('node_name’, :node => node)
Interacting with Puppet’s Data
Use Cases
# puppetconf # puppetize @ puppetlabs
Inspecting the catalog:
• What types are in the catalog?
irb> catalog.resources.collect do |r| r.type end.uniq
• Gimme a resource:
irb>catalog.resource(‘Package[httpd]’)
# puppetconf # puppetize @ puppetlabs
Rspec Puppet:
let :facts do
{:operatingsystem => ‘Redhat’}
end
let :params do
{:bind_address => ‘0.0.0.0’
end
it { should contain_file(‘/tmp/foo.conf’) }
# puppetconf # puppetize @ puppetlabs
Thundering Herd
Pre-compile catalogs for faster auto-scaling
# puppetconf # puppetize @ puppetlabs
Applying pre-compiled catalogs:
puppet apply --catalog /tmp/catalog.json –server
puppet-fileserver
# puppetconf # puppetize @ puppetlabs
DMZ
tcp over USB
Interacting with Puppet’s Data
Use Cases
# puppetconf # puppetize @ puppetlabs
Hacking reports
Everything in Puppet is a state transition
User[‘dan’] : absent -> present
User[‘dan’][‘shell’] -> ‘/sbin/nologin’ -> /bin/bash
# puppetconf # puppetize @ puppetlabs
Setting up the agent:
[agent]
report=true
# puppetconf # puppetize @ puppetlabs
Archive reports in your yamldir
[master]
reports = store
# puppetconf # puppetize @ puppetlabs
Puppet reports
$ cd `puppet config print reportdir`
$ ls
node1 node2 node3
$ ls node1
# puppetconf # puppetize @ puppetlabs
Every report from every run ever
$ ls node1
201206060256.yaml 201206060303.yaml 201206060519.yaml 201206122349.yaml 201206122354.yaml 201206130002.yaml
# puppetconf # puppetize @ puppetlabs
Lets crack one open!
Irb > require ‘yaml’
>reports=YAML.load_file('201206130002.yaml')
# puppetconf # puppetize @ puppetlabs
Have a look
>(reports.methods - Object.methods).sort
Notice the following methods:
# puppetconf # puppetize @ puppetlabs
High level data
> reports.exit_status
⇒ 0
> reports.status
=> "unchanged"
> reports.host
⇒ ”node1”
# puppetconf # puppetize @ puppetlabs
metrics
> reports.metrics.keys
⇒ ["resources", "events", "changes", "time"]
> reports.metrics['resources']
⇒ [‘failed’, 0],[ ‘changed’, ‘7’]
# puppetconf # puppetize @ puppetlabs
And the awesome sauce
> reports.resource_statuses.keys
=> ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe-puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]
# puppetconf # puppetize @ puppetlabs
And the awesome sauce
> status = reports.resource_statuses
> status.keys
=> ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe-puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]
# puppetconf # puppetize @ puppetlabs
And the awesome sauce >events = status["File[/etc/swift/swift.conf]"].events
> events.first.status
⇒ "success”
> events.first.desired_value
⇒ :present
> events.first.previous_value
=> :absent
Thank You Dan Bode| Puppet Labs [email protected]