Chef_Intermediate_v1.0.0.pdf

8/9/2019 Chef_Intermediate_v1.0.0.pdf

1/321

• THIS IS A WIP ONLY VERSION OF THE SLIDES

• Before releasing these slides (say v1.2.1)

• Make a copy of this file and call it "Chef-Intermediate_v1.2.1.key"

• Set the version number on Master Slides• Seek and destroy all "


2/321

2

Friday, 4 July 14

mailto:[email protected]:[email protected]


3/321

3

Friday, 4 July 14


4/321

• Name

• Current job role

• Previous job roles/background

• Experience with Chef and/or config managem

• Location

4

Friday, 4 July 14


5/321

5

Friday, 4 July 14


6/321

• Upon completion of this course you will be able to:

• Extend Chef with custom resources and providers• Understand the internals of a Chef Client run

• Create, debug, and distribute custom Ohai plugins

• Use the chef-shell debugger to interactively debug c

• Configure report and exception handlers• Avoid common cookbook errors using Foodcritic an

Rubocop

• Gain an introduction to unit testing using ChefSpec

6

Friday, 4 July 14


7/321

• Completed Chef Fundamentals or equivalent

experience• Chef Client 11.12.2 or better

• Needed for the Ohai 7 lessons

7

Friday, 4 July 14


8/321

• You bring the domain expertise about your bu

and problems• Chef provides a framework for solving those

problems

• Our job is to work together to teach you how texpress solutions to your problems with Chef

8

Friday, 4 July 14


9/321

• We will be doing things the hard way

• We’re going to do a lot of typing

• You can’t be:

• Absent

• Late• Left Behind

• We will troubleshoot and fix bugs on the spot

• The result is you reaching fluency fast9

Friday, 4 July 14


10/321

• I’ll post objectives at the beginning of a sectio

• Ask questions when they come to you

• Ask for help when you need it

10

Friday, 4 July 14


11/321

11

Friday, 4 July 14


12/321

• Chef Fundamentals Refresher

• Building Custom Resources• chef-shell - The Chef Debugger

• Writing Ohai Plugins

• Chef Client Run Internals

• Implementing Chef Handlers

• Linting Your Cookbooks

• An Introduction to ChefSpec

• Further Resources12

Friday, 4 July 14


13/321

• We’ll take a break about every 2 hours

• We’ll obviously break for lunch :)

• Around 11:30pm (ish?). At the Atrium level

13

Friday, 4 July 14


14/321

14

Friday, 4 July 14


15/321

$ whoamii-am-a-workstation

This is an example of a command to run on your workstat

user@hostname:~$ whoamii-am-a-chef-node

This is an example of a command to run on your target nod

15

Friday, 4 July 14


16/321

$ ifconfig

lo0: flags=8049 mtu 16384

options=3

inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1

inet 127.0.0.1 netmask 0xff000000

inet6 ::1 prefixlen 128gif0: flags=8010 mtu 1280stf0: flags=0 mtu 1280

en0: flags=8863 mtu 1500

ether 28:cf:e9:1f:79:a3

inet6 fe80::2acf:e9ff:fe1f:79a3%en0 prefixlen 64 scopeid 0x4 inet 10.100.0.84 netmask 0xffffff00 broadcast 10.100.0.255 media: autoselect status: activep2p0: flags=8843 mtu 2304 ether 0a:cf:e9:1f:79:a3 media: autoselect status: inactive

16

Friday, 4 July 14


17/321

OPEN IN EDITOR:

SAVE FILE!

~/hello_world

Hi!

I am a friendly file.

17

Friday, 4 July 14


18/321

18

Friday, 4 July 14


19/321

• Register and login to CloudShare (see invite)

• Start Using This Environment

19

Friday, 4 July 14


20/321

20

Friday, 4 July 14


21/321

$ ssh chef@

The authenticity of host 'uvo1qrwls0jdgs3blvt.vm.cld.sr(69.195.232.110)' can't be established.RSA key fingerprint is d9:95:a3:b9:02:27:e9:cd:74:e4:a2:34:23:f5:a6:8b.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'uvo1qrwls0jdgs3blvt.vm.cld.sr,69.195.232.110' (RSA) to the list of known [email protected]'s password:Last login: Mon Jan 6 16:26:24 2014 fromhost86-145-117-53.range86-145.btcentralplus.com [chef@CentOS63 ~]$

21

Friday, 4 July 14

mailto:[email protected]:[email protected]:[email protected]


22/321

• At this point you should have

• One virtual machine (VM) or server that youfor the lab exercises

• The IP address or public hostname

• An application for establishing an ssh conne

• 'sudo' or 'root' permissions on the VM

22

Friday, 4 July 14


23/321

23

Friday, 4 July 14


24/321

• After this lesson the, you will be able to

• Create a new Organization• Manually configure your pem files & knife.rb

your workstation so it can communicate withServer

• Bulk upload all cookbook, role, environmentdata bag files to the Chef Server

24

Friday, 4 July 14


25/321

• Problem: We want to pick up where we left off after ChFundamentals class

• Proposed Solution: We need to

• Set up a new Organization in Enterprise Chef

• Configure your workstation to communicate with new

• Download the Chef Fundamentals content from a Gitrepo - cookbooks, data bags, environments and roles

• Upload all artifacts to your new Org

• Configure server run list

• Bootstrap server

25

Friday, 4 July 14


26/321

• Sign into Hosted Enterprise Chef

• Or create a new account if you wish• Create a new Org - name must be globally un

26

1 2

Friday, 4 July 14


27/321

• In Chef Fundamentals you set our workstation

the easy way using 'Starter Kit'

• In this class you will download your .pem filesknife.rb and configure it manually!

27

Friday, 4 July 14


28/321

• Download the knife.rb and validator pem for y

Org

28

Friday, 4 July 14


29/321

• Reset and download your private client pem

• ONLY DO THIS if you don't already have thefile available on your laptop

29

Friday, 4 July 14


30/321

• You should now have these 3 files from Enter

Chef •.pem

•-validator.pem

•knife.rb

• Later you will move these to your .chef directo

30

Friday, 4 July 14


31/321

• Download the Chef Fundamentals working rehttps://github.com/learnchef/chef-fundamenta

31

Friday, 4 July 14


32/321

• For the purposes of this class make a working direc

under your home directory called '~/intermediat• Windows:-

• C:\Users\you\intermediate

• Mac/*nix:-

• /Users/you/intermediate

• Navigate to this working Directory

32

Friday, 4 July 14


33/321

$ cp ~/Downloads/chef-fundamentals-repo-master.zip .$ unzip chef-fundamentals-repo-master.zip

Archive: chef-fundamentals-repo-master.zipbb06ea2c0cabaa855e4cb1d1c43bbe4d75caf70d creating: chef-fundamentals-repo-master/ inflating: chef-fundamentals-repo-master/Berksfileinflating: chef-fundamentals-repo-master/README.mdinflating: chef-fundamentals-repo-master/Vagrantfilecreating: chef-fundamentals-repo-master/cookbooks/

creating: chef-fundamentals-repo-master/cookbooks/apache/ inflating: chef-fundamentals-repo-master/cookbooks/apache/CHANGEinflating: chef-fundamentals-repo-master/cookbooks/apache/READMEcreating: chef-fundamentals-repo-master/cookbooks/apache/attrib

creating: chef-fundamentals-repo-master...

33

Friday, 4 July 14


34/321

$ cd chef-fundamentals-repo-master$ ls -a

. Berksfile Vagrantfile data_bags roles

.. README.md cookbooks environments

34

• Notice there is no '.chef' directory here

• You need to create one and place your 'knife.rb' file, 'validator.pyour 'client.pem' in it

Friday, 4 July 14


35/321

35

• These are the artifacts created in Chef F

• Everything should be uploaded to the Ch

cookbooks!"" apache

!"" chef-client!"" cron

!"" logrotate

!"" motd

!"" ntp

!"" pci

!"" starter

#"" users

9 directories

roles!"" base.rb

!"" starter.rb#"" webserver.rb

3 files

data_bags!"" groups

$ #"" clowns.json

#"" users !"" bobo.json

#"" frank.json

2 directories, 3 filesenvironments!"" dev.rb

#"" production.rb

2 files

Friday, 4 July 14


36/321

$ mkdir .chef

$ cp ~/Downloads/.pem .chef$ cp ~/Downloads/-validator.pem .chef$ cp ~/Downloads/knife.rb .chef

36

Friday, 4 July 14


37/321

$ knife client list

37

-validator.pem

Friday, 4 July 14


38/321

$ knife cookbook upload -a

Uploading apache [0.2.0]Uploading chef-client [3.4.0]Uploading cron [1.2.8]Uploading logrotate [1.4.0]

Uploading motd [0.1.0]Uploading ntp [1.5.4]Uploading pci [0.1.0]Uploading starter [1.0.0]Uploading users [0.1.0]Uploaded all cookbooks.

38

Friday, 4 July 14


39/321

$ knife upload data_bags

Created data_bags/groupsCreated data_bags/users

Created data_bags/groups/clowns.jsonCreated data_bags/users/bobo.jsonCreated data_bags/users/frank.json

39

Friday, 4 July 14


40/321

$ knife role from file base.rb starter.rb webserver

Updated Role base!

Updated Role starter!Updated Role webserver!

40

Friday, 4 July 14


41/321

$ knife environment from file dev.rb production.rb

Updated Environment devUpdated Environment production

41

Friday, 4 July 14


42/321

$ knife bootstrap --sudo -x chef -P chef -N

uvo164727i3mvh1jup2.vm.cld.sr --2014-05-13 04:31:10-- https://www.opscode.com/chef/installuvo164727i3mvh1jup2.vm.cld.sr Resolving www.opscode.com... 184.106.28.90uvo164727i3mvh1jup2.vm.cld.sr Connecting to www.opscode.com|184.106.28.90|:443... connecteduvo164727i3mvh1jup2.vm.cld.sr HTTP request sent, awaiting response... 200 OKuvo164727i3mvh1jup2.vm.cld.sr Length: 15934 (16K) [application/x-sh]uvo164727i3mvh1jup2.vm.cld.sr Saving to: `STDOUT'uvo164727i3mvh1jup2.vm.cld.sr100%[======================================>] 15,934 --.-K/s in 0suvo164727i3mvh1jup2.vm.cld.sr

uvo164727i3mvh1jup2.vm.cld.sr 2014-05-13 04:31:10 (538 MB/s) - written to stdout [15934/159uvo164727i3mvh1jup2.vm.cld.sruvo164727i3mvh1jup2.vm.cld.sr Downloading Chef 11.8.2 for el...uvo164727i3mvh1jup2.vm.cld.sr downloading https://www.opscode.com/chef/metadata?v=11.8.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64uvo164727i3mvh1jup2.vm.cld.sr to file /tmp/install.sh.41533/metadata.txtuvo164727i3mvh1jup2.vm.cld.sr trying wget...uvo164727i3mvh1jup2.vm.cld.sr url https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86chef-11.8.2-1.el6.x86_64.rpm ...

42

Friday, 4 July 14

$ knife bootstrap IPADDRESS --sudo -x USERNAME -P PASSWORD -N


43/321

local workstation manag

(V

chef

Opscode Hosted Chef

chef_server_url

validation_client_name

validation_key

SSH!

bash

install

configur

run c

43

Friday, 4 July 14


44/321

• Chef and all of its dependencies installed via an

operating system-specific package ("omnibus in• Installation includes

• The Ruby language - used by Chef

• knife - Command line tool for administrators

• chef-client - Client application• ohai - System profiler

• ...and more

44

Friday, 4 July 14


45/321

$ ssh chef@

chef@node1:~$ ls /etc/chefclient.pem client.rb first-boot.json validati

chef@node1:~$ which chef-client

/usr/bin/chef-client

chef@node1:~$ chef-client -vChef: 11.12.4

45

Friday, 4 July 14


46/321

$ knife node run_list add node1 'role[webserver]'

node1:

run_list: role[webserver]

46

Friday, 4 July 14


47/321

chef@node1$ sudo chef-client

...[2014-01-07T06:57:48-05:00] INFO: template[/etc/httpd/conf.d/bears.conf] updated file cohttpd/conf.d/bears.conf

- update content in file /etc/httpd/conf.d/bears.conf from 30610b to 92e1e4 --- /etc/httpd/conf.d/bears.conf

2014-01-07 05:13:42.450113970 -0500 +++ /tmp/chef-rendered-template20140107-18263-5563t5 2014-01-07 06:57:48.09139

@@ -1,6 +1,6 @@ - Listen 81 + Listen 8081

- + ServerAdmin webmaster@localhost

DocumentRoot /srv/apache/bears...

47

Friday, 4 July 14


48/321

• You may have seen an SSH error

48

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *SSL validation of HTTPS requests is disabled. HTTPS connections are stillencrypted, but chef is not able to detect forged replies or man in the middleattacks.

To fix this issue add an entry like this to your configuration file:

``` # Verify all HTTPS connections (recommended) ssl_verify_mode :verify_peer

# OR, Verify only connections to chef-server verify_api_cert true

```

To check your SSL configuration, or troubleshoot errors, you can use the`knife ssl check` command like so:```

• This is harmless for the purpose of this course

we will fix it in the next sectionFriday, 4 July 14


49/321

chef@node1$ ps awux | grep chef-clien

root 8933 0.3 2.2 130400 37816 ?

03:19 0:01 /opt/chef/embedded/bin/ruby /usr

chef-client -d -c /etc/chef/client.rb -L /varchef/client.log -P /var/run/chef/client.pid -

-s 300

49

Friday, 4 July 14


50/321

50

Friday, 4 July 14


51/321

• What files did we download from Chef Server toconfigure our workstation?

• Where did we place these files?

• What other way could we have copied the repofrom GitHub

• Bonus point - why did we not do this?• How did we upload all cookbooks to the Chef S

51

Friday, 4 July 14


52/321

52

Friday, 4 July 14


53/321

• After completing the lesson, you will be able t

• Use the chef-client cookbook to manage theClient service on your nodes

• Configure nodes to use SSL Certificate Valid

53

Friday, 4 July 14


54/321

54

• The chef-client cookbook manages chef-clien

configuration on nodes• client.rb configuration

• chef-client service management

• validation key deletion

Friday, 4 July 14


55/321

• We're already using two recipes on the node

the chef-client cookbook• chef-client::service

(via chef-client::default)

•chef-client::delete_validation

55

Friday, 4 July 14


56/321

$ knife node show node1

Node Name: node1Environment: _defaultFQDN: centos63.example.com IP: 10.160.201.90Run List: role[webserver]

Roles: webserver, baseRecipes: chef-client::delete_validation, chef-client, ntp, motd,apache, chef-client::default, chef-client::service, chef-client::inintp::default, motd::default, users::default, users::groups, apache::Platform: centos 6.4Tags:

56

Friday, 4 July 14


57/321

OPEN IN EDITOR: cookbooks/chef-client/recipes/config.rb

template "#{node["chef_client"]["conf_dir"]}/client.rb" do

source 'client.rb.erb' owner d_owner group d_group mode 00644 variables( :chef_config => node['chef_client']['config'], :chef_requires => chef_requires, :ohai_disabled_plugins => node['ohai']['disabled_plugins'],

:start_handlers => node['chef_client']['config']['start_handler :report_handlers => node['chef_client']['config']['report_handl :exception_handlers => node['chef_client']['config']['exception_handlers'] ) notifies :create, 'ruby_block[reload_client_config]', :immediatelend

57

Friday, 4 July 14


58/321

• The config recipe is used to dynamically generate the /etc/client.rb config file

• The template walks all attributes in node['chef_client'['config'] and writes them out as key:value pairs

• The key should be the configuration directive

• The chef_server_url, node_name and

validation_client_name are set by default in the attribufrom Chef::Config

58

Friday, 4 July 14


59/321

• For example, the following attributes (in a roledefault_attributes(

"chef_client" => { "config" => { "ssl_verify_mode" => ":verify_peer", "client_fork" => true } })

59

• Will render the following configuration (/etc/chef/cchef_server_url "https://api.opscode.com/organizations/MYORG"validation_client_name "MYORG-validator"ssl_verify_mode :verify_peernode_name "config-ubuntu-1204"client_fork true

Friday, 4 July 14

https://api.opscode.com/organizations/MYORGhttps://api.opscode.com/organizations/MYORG


60/321

• Chef defaults to not verifying certificates when it makes HTTconnections - hence the error when running chef-client

• Enable SSL Verification by settingssl_verify_mode :verify_peer in your config file

• Two useful knife commands:-

• knife ssl check - makes an SSL connection to your Cor any other HTTPS server and tells you if the server presevalid certificate

• knife ssl fetch - allows you to automatically fetch a scertificates to your trusted certs directory

60

Friday, 4 July 14


61/321

OPEN IN EDITOR:

SAVE FILE!

roles/base.rb

name "base"

description "Base Server Role"run_list "recipe[chef-client::delete_validation]", "recipe[chef-client::config]", "recipe[chef-client]", "recipe[ntp]", "recipe[motd]", "recipe[users]"default_attributes( "chef_client" => {

"config" => { "ssl_verify_mode" => ":verify_peer" } })

61

Friday, 4 July 14


62/321

$ knife role from file base.rb

Updated Role base!

62

Friday, 4 July 14


63/321


Starting Chef Client, version 11.12.8resolving cookbooks for run list: ["chef-client::delete_validation", "chef-client::"chef-client", "ntp", "motd", "users", "apache"]Synchronizing Cookbooks: - chef-client - cron - logrotate

- ntp - motd - pci - users - apache - ohaiCompiling Cookbooks......

63

Friday, 4 July 14


64/321

• How do you remove the validation .pem on thnode?

• What recipe is used to configure chef-client?

• How do you enable SSL Verification on chef-c

• How do you fetch a server's certificates to you

trusted certs directory?

64

Friday, 4 July 14


65/321

65

Friday, 4 July 14


66/321


• Describe the components of the LightweightResource/Provider (LWRP) framework

• Explain the Resource DSL

• Explain the Provider DSL

• Build a resource and provider from scratch

• Look at examples of other LWRPs

66

Friday, 4 July 14


67/321

• Resources are declarative interfaces that describe whwant to happen, rather than how

• Resources take action through Providers that perform

• Resources

• have a type

• have a name

• can have one or more parameters• take action to put the resource into a desired state

• can send notifications to other resources

67

Friday, 4 July 14


68/321

• Definitions

• “recipe macro”• stored in definitions/

• cannot receive notifications

• Heavyweight Resources

• pure Ruby code

• stored in libraries/

• cannot use core resources (by default)68

Friday, 4 July 14


69/321

• LWRPs have two components: the resource aprovider

• Resources are used in Recipes to declare tstate to configure on our system

• Providers configure that state on the system

during convergence• They are defined in the resources/ and

providers/ directories of cookbooks

69

Friday, 4 July 14


70/321

• The Problem: We want to abstract the Apachvirtual host configuration pattern.

• Success Criteria: We will create an

apache_vhost resource that will let us man

Apache virtual hosts.

70

Friday, 4 July 14


71/321

maintainer "apache" maintainer_email "YOUR_EMAIL" license "All rights reserved" description "Installs/Configures apache" long_description IO.read(File.join(File.dirname( __FILE__ ), ‘README version "0.3.0"

OPEN IN EDITOR: cookbooks/apache/metadata.rb

SAVE FILE!

• Major, Minor, Patch• Semantic Versioning Policy: http://semver.org

71

Friday, 4 July 14

http://semver.org/http://semver.org/


72/321

+ cookbooks | + apache | + resources | + vhost.rb

• Name of new resource is implied by its name

cookbook

• In the above scenario, the new resource will b

called apache_vhost

72

Friday, 4 July 14


73/321

• Three methods: actions, attribute,

default_action• actions specifies allowed actions that can

used in a recipe

• attribute specifies a new parameter attri

the resource• default_action specifies one of the allow

actions to be used when the user leaves it o

73

Friday, 4 July 14


74/321

• Note the file cookbooks/apache/attributes/defaucontains the following

default["apache"]["sites"]["clowns"] = { "port" =>default["apache"]["sites"]["bears"] = { "port" =>

OPEN IN EDITOR: cookbooks/apache/recipes/default.r

# Iterate over the apache sites

node["apache"]["sites"].each do |site_name, site_da# Set the document root

document_root = "/srv/apache/#{site_name}"

74

Friday, 4 July 14


75/321


# Add a template for Apache virtual host configurat

template "/etc/httpd/conf.d/#{site_name}.conf" do source "custom.erb" mode "0644" variables( :document_root => document_root, :port => site_data["port"]

) notifies :restart, "service[httpd]" end

75

Friday, 4 July 14


76/321


# Add a directory resource to create the document_r

directory document_root do mode "0755" recursive true end

76

Friday, 4 July 14


77/321


# Add a template resource for the virtual host's

index.html template "#{document_root}/index.html" do source "index.html.erb" mode "0644" variables( :site_name => site_name,

:port => site_data["port"] ) end

77

Friday, 4 July 14


78/321

actions :create, :remove

OPEN IN EDITOR: cookbooks/apache/resources/vhost.r

SAVE FILE!

• We have allowed two resource actions

78

Friday, 4 July 14


79/321

• Only one method used: action

• Specify the action name with a Ruby sym• Name maps to allowed actions defined in th

resource actions

• You can also re-use any Chef Resources insi

providers

79

Friday, 4 July 14

/ / i /


80/321

action :create do

puts "My name is #{new_resource.name}"end

OPEN IN EDITOR: cookbooks/apache/providers/vhost.r

SAVE FILE!

• When our apache_vhost resource calls

action :create, execute this block.

80

Friday, 4 July 14


81/321

# Enable an Apache Virtualho

apache_vhost "lions" do action :create

end

• We call resource

apache_vhost

• With name “lions”

• With action :create in the parameter block

81

Friday, 4 July 14


82/321

$ knife cookbook upload apache

Uploading apache [0.3.0]

Uploaded 1 cookbook.

82

Friday, 4 July 14


83/321


Starting Chef Client, version 11.10.4resolving cookbooks for run list: ["chef-client::delete_validation", "chef-client", "ntp", "motd", "useSynchronizing Cookbooks: - chef-client - cron - logrotate - ntp - motd - pci - users

- apache... * apache_vhost[lions] action createMy name is lions (up to date)...Running handlers:Running handlers complete

Chef Client finished, 12/35 resources updated in 12.045321863 seconds

83

Friday, 4 July 14

OPEN IN EDITOR: cookbooks/apache/providers/vhost r


84/321

use_inline_resources

action :create do log "My name is #{new_resource.name}"end


SAVE FILE!

• The log resource uses Chef’s logger object to messages at Chef::Config[:log_level]

84

Friday, 4 July 14


85/321

• use_inline_resources instructs the emb

resources (“log”) to notify the parent

(“apache_vhost”) if their states change

• More info: docs.opscode.com/lwrp_custom_provider.html#use-inline-res

85

Friday, 4 July 14


86/321

• A regular resource collection is just a big arraresources.

86

Friday, 4 July 14


87/321

• use_inline_resources creates a mini resoucollection (execution context)

• Notifications are rolled up to the master resourccollection.

87

Friday, 4 July 14


88/321




88

Friday, 4 July 14


89/321



- apache... * apache_vhost[lions] action createRecipe: * log[My name is lions] action write...Running handlers:Running handlers complete


89

Friday, 4 July 14



90/321


attribute :site_name, :name_attribute => true, :kind_of => Stringattribute :site_port, :default => 80, :kind_of => Fixnum


SAVE FILE!

• attribute takes the name of the attribute and an(optional) hash of validation parameters

• These validation parameters are specific to the LWRResource DSL

90

Friday, 4 July 14



91/321


SAVE FILE!

• :name_attribute set the file_name to the resour

• :default sets the default value for this parameter

• :kind_of ensures the value is a particular class

• More validation parameters listed in the docs.91


attribute :site_name, :name_attribute => true, :kind_of => Stringattribute :site_port, :default => 80, :kind_of => Fixnum

Friday, 4 July 14

Validation Meaning


92/321

Validation Meaning

:callbacks Hash of Procs, should ret

:default Default value for the para:equal_to Match the value with

:kind_ofEnsure the value is of a p

class

:name_attribute Set to the resource na

:regex Match the value against th

:required The parameter must be s

:respond_toEnsure the value has a

method92

Friday, 4 July 14



93/321

SAVE FILE!

p p

use_inline_resources

action :create do # Set the document root document_root = "/srv/apache/#{new_resource.site_name}"

# Add a template for Apache virtual host configuration template "/etc/httpd/conf.d/#{new_resource.site_name}.conf" do source "custom.erb" mode "0644" variables(

:document_root => document_root, :port => new_resource.site_port ) end

93

Friday, 4 July 14



94/321

SAVE FILE!

p p

# Add a directory resource to create the document_root directory document_root do

mode "0755" recursive true end

# Add a template resource for the virtual host's index.html template "#{document_root}/index.html" do source "index.html.erb" mode "0644" variables(

:site_name => new_resource.site_name, :port => new_resource.site_port ) endend

94

Friday, 4 July 14



95/321

SAVE FILE!

# Enable an Apache Virtualhost

apache_vhost "lions" do

site_port 8080 action :create notifies :restart, "service[httpd]"end

95

Friday, 4 July 14


96/321




96

Friday, 4 July 14


97/321



- apache... * apache_vhost[lions] action createRecipe: * template[/etc/httpd/conf.d/lions.conf] action create...Running handlers:Running handlers complete


97

Friday, 4 July 14


98/321

98

Friday, 4 July 14

Resource Collection


99/321

99

resource_collection = [ ...,package[httpd],

service[httpd], apache_vhost[lions], resource_collection = [

template["/etc/httpd/conf.d/lions.conf"], directory["/srv/apache/lions"], template["/srv/apache/lions/index.html"], ], template["/etc/httpd/conf.d/clowns.conf"], directory["/srv/apache/clowns"], template["/srv/apache/clowns/index.html"], template["/etc/httpd/conf.d/bears.conf"], directory["/srv/apache/bears"], template["/srv/apache/clowns/bears.html"]]

All notificatioresources are

'master' resou

Friday, 4 July 14

• Now that we have an easy way to add virtual


100/321

Now that we have an easy way to add virtual we will quickly run into a data management p

• We could keep adding more attributes to the cookbook, but...

• that doesn’t scale well across teams

• you can end up with a long, messy list of att

100

Friday, 4 July 14


101/321

default["apache"]["sites"]["clowns"] = { "port" => default["apache"]["sites"]["bears"] = { "port" => 8

• How should we structure our data bags?

• Data bag for each site?

• One data bag for apache with items for each

101

Friday, 4 July 14

$ k if d t b t h it


102/321

$ knife data_bag create apache_sit

Created data_bag[apache_sites]

102

Friday, 4 July 14

$ mkdir data ba s/a ache sites


103/321

$ mkdir -p data_bags/apache_sites

No output...

103

Friday, 4 July 14

OPEN IN EDITOR: data_bags/apache_sites/clowns.json


104/321

SAVE FILE!

{

"id": "clowns", "port": 80}

104

Friday 4 July 14

$ knife data bag from file apache sites clown


105/321

$ knife data_bag from file apache_sites clown

105

Updated data_bag_item[apache_sites::clowns]

Friday 4 July 14

OPEN IN EDITOR: data_bags/apache_sites/bears.json


106/321

SAVE FILE!

{

"id": "bears", "port": 81}

106

Friday 4 July 14

$ knife data bag from file apache sites bears


107/321

$ knife data_bag from file apache_sites bears

107

Updated data_bag_item[apache_sites::bears]

Friday 4 July 14

OPEN IN EDITOR: data_bags/apache_sites/lions.json


108/321

SAVE FILE!

{

"id": "lions", "port": 8080}

108

Friday 4 July 14

$ knife data bag from file apache sites lions


109/321

$ knife data_bag from file apache_sites lions

109

Updated data_bag_item[apache_sites::lions]

Friday 4 July 14



110/321

# Iterate over the apache sites

search("apache_sites", "*:*").each do |site|

# Enable an Apache Virtualhost apache_vhost site['id'] do site_port site['port'] action :create notifies :restart, "service[httpd]" endend

110

• Delete existing node[“apache”][“sites”]

• Move apache_vhost LWRP inside a search l

• Change variable namesFriday, 4 July 14



111/321




111

Friday, 4 July 14



112/321

chef@node1$ sudo chef client

Starting Chef Client, version 11.10.4resolving cookbooks for run list: ["chef-client::delete_validation", "chef-client", "ntp", "motd", "usersSynchronizing Cookbooks: - chef-client - cron - logrotate - ntp - motd - pci - users - apache... * apache_vhost[clowns] action createRecipe: ... * apache_vhost[bears] action createRecipe: ... * apache_vhost[lions] action createRecipe: ...


112

Friday, 4 July 14



113/321

SAVE FILE!

action :remove do file "/etc/httpd/conf.d/#{new_resource.site_name}.conf" do action :delete

endend

113

Friday, 4 July 14



114/321

# Disable the default virtual host

apache_vhost "welcome" do

action :remove notifies :restart, "service[httpd]"end

114

• Delete existing execute resource that disables the wsite

• Add a new apache_vhost resource with action

Friday, 4 July 14



115/321

p p



115

Friday, 4 July 14



116/321

Starting Chef Client, version 11.10.4resolving cookbooks for run list: ["chef-client::delete_validation", "chef-client", "ntp", "motd"apache"]Synchronizing Cookbooks: - chef-client - cron - logrotate - ntp - motd - pci - users

- apache... * apache_vhost[welcome] action removeRecipe: * file[/etc/httpd/conf.d/welcome.conf] action delete (up to date) (up to date)...


116

Friday, 4 July 14

• We wrote an LWRP using out-of-the-box Che


117/321

resources

• You can write pure Ruby in LWRPs as well• http://docs.opscode.com/lwrp_custom_provider_

• You can also write pure Chef resources/provid

inheriting from Chef::Resource::Base an

Chef::Provider::Base and putting them

libraries/

• AKA HWRPs or heavyweight resources & p117

Friday, 4 July 14

• What are the two components of an LWRP?

http://docs.opscode.com/lwrp_custom_provider_ruby.htmlhttp://docs.opscode.com/lwrp_custom_provider_ruby.htmlhttp://docs.opscode.com/lwrp_custom_provider_ruby.htmlhttp://docs.opscode.com/lwrp_custom_provider_ruby.htmlhttp://docs.opscode.com/lwrp_custom_provider_ruby.html


118/321

• What is the difference between these two

components?• What language can LWRPs written in?

• What classes can you inherit from Chef to wriHWRPs

118

Friday, 4 July 14


119/321

119

Friday, 4 July 14



120/321

• Use chef-shell, the interactive Chef console

(REPL)• List chef-shell’s execution contexts

• Use chef-shell to inspect and debug recipes

• Interact with a Chef Server API• Discuss the magic that is knife exec

Friday, 4 July 14

• Interactive Ruby console (REPL) for Chef


121/321

• Three startup modes (standalone, solo, client

• Three run-time contexts (main, attribute, recip

• Configured with a specific or implied config fil

Friday, 4 July 14

• Three modes for startup

S d l kb k d li


122/321

• Standalone: no cookbooks and empty run lis

(default)• Solo: attempts to act as chef-solo, using solo

and JSON attributes. (-s or --solo)

• Client: contacts chef server for node object,

retrieving cookbooks and using specified run or --client).

Friday, 4 July 14

chef@node1$ sudo chef-shell -z


123/321

loading configuration: /etc/chef/client.rb

Session type: clientLoading.....resolving cookbooks for run list: ["apache::ohai_plugin", "chef-client::delete_validation", "chef-client", "n"users", "apache"]..Synchronizing Cookbooks: - apache - chef-client - cron - logrotate - ntp - motd - pci - users./opt/chef/embedded/lib/ruby/gems/1.9.1/gems/ohai-7.0.4/lib/ohai/plugins/linux/network.rb:49: warning: already initializeIPROUTE_INT_REGEX

........[2014-05-14T06:27:24-04:00] WARN: Cloning resource attributes for apache_vhost[lions] from prior resource (CHEF-3[2014-05-14T06:27:24-04:00] WARN: Previous apache_vhost[lions]: /var/chef/cache/cookbooks/apache/recipes/default.rb:27:in[2014-05-14T06:27:24-04:00] WARN: Current apache_vhost[lions]: /var/chef/cache/cookbooks/apache/recipes/default.rb:42:infrom_file'done.

This is the chef-shell. Chef Version: 11.12.4 http://www.opscode.com/chef http://docs.opscode.com/

Friday, 4 July 14

chef > help


125/321

Command: nodes================================================================================## SUMMARY ## +nodes+ Allows you to query your chef server for information about your nodes.

## LIST ALL NODES ## You can list all nodes using +all+ or +list+

nodes.all #=> [, , ...]

To limit the information returned for each node, pass a code block to the +all+ subcommand:

nodes.all { |node| node.name } #=> [NODE1_NAME, NODE2_NAME, ...]

Friday, 4 July 14

chef > node['fqdn']


126/321

=> "centos63.example.com"

Friday, 4 July 14

chef > attributes_mode


127/321

chef:attributes >

Friday, 4 July 14

chef:attributes > node.normal["interactive"] = "i


128/321

=> "is cool"

Friday, 4 July 14

chef:attributes > node["interactiv


129/321

=> "is cool"

Friday, 4 July 14

$ knife node show node1 -a interactive


130/321

node1: interactive:

Friday, 4 July 14

chef:attributes > node.save


131/321

=>

Friday, 4 July 14

$ knife node show node1 -a interactive


132/321

node1: interactive: is cool

Friday, 4 July 14

$ knife node show node1 -Fj -l |more


133/321

{ "name": "node1", "chef_environment": "_default", "run_list": [ "role[webserver]" ], "normal": { "tags": [

], "interactive": "is cool" }, "default": {

"logrotate": { "global": { "weekly": true, "rotate": 4, "create": "", "/var/log/wtmp": { "missingok": true, "monthly": true, "create": "0664 root utmp",

Friday, 4 July 14

chef:attributes > recipe_mode


134/321

chef:recipe >

Friday, 4 July 14

chef:recipe > package "tree"


135/321

=> run_chef


136/321

[2014-05-13T12:20:27-04:00] INFO: Processing package[tree] action install ((irb#1) line 2)

[2014-05-13T12:20:27-04:00] DEBUG: package[tree] checking yum info for tree[2014-05-13T12:20:27-04:00] DEBUG: package[tree] installed version: (none) candidate version: [2014-05-13T12:20:27-04:00] INFO: package[tree] installing tree-1.5.3-2.el6 from base reposit[2014-05-13T12:20:27-04:00] DEBUG: Executing yum -d0 -e0 -y install tree-1.5.3-2.el6[2014-05-13T12:20:32-04:00] DEBUG: ---- Begin output of yum -d0 -e0 -y install tree-1.5.3-2.e[2014-05-13T12:20:32-04:00] DEBUG: STDOUT:[2014-05-13T12:20:32-04:00] DEBUG: STDERR:[2014-05-13T12:20:32-04:00] DEBUG: ---- End output of yum -d0 -e0 -y install tree-1.5.3-2.el6 [2014-05-13T12:20:32-04:00] DEBUG: Ran yum -d0 -e0 -y install tree-1.5.3-2.el6 returned 0[2014-05-13T12:20:32-04:00] INFO: apache_vhost[lions] sending restart action to service[httpd[2014-05-13T12:20:32-04:00] INFO: Processing service[httpd] action restart (apache::default l[2014-05-13T12:20:32-04:00] DEBUG: service[httpd] supports status, running

httpd (pid 49825) is running...[2014-05-13T12:20:32-04:00] DEBUG: service[httpd] is runninghttpd 0:off

1:off

2:on

3:on

4:on

5:on

6:offStopping httpd: [ OK ]Starting httpd: [ OK ][2014-05-13T12:20:35-04:00] INFO: service[httpd] restarted => true

Friday, 4 July 14

chef:recipe > resources


137/321

... "apache_vhost[welcome]","apache_vhost[lions]", "apache_vhost[clowns]"apache_vhost[bears]", "package[tree]"]

Friday, 4 July 14

chef:recipe > resources("user[fran


138/321

false, :non_unique=>false}@ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/var/chef/cache/coorecipes/default.rb:11:in ̀ block in from_file'" @elapsed_time: 0.001173478 @resource_n@username: "frank" @comment: "Frank Belson" @uid: 2001 @gid: 0 @home: "/home/frank" @bash" @password: nil @system: false @manage_home: false @non_unique: false @cookbook_@recipe_name: "default"> => false, :non_unique=>false} @ignore_failure: false @retries: 0 @retry_d@source_line: "/var/chef/cache/cookbooks/users/recipes/default.rb:11:in `block in fro@elapsed_time: 0.001173478 @resource_name: :user @username: "frank" @comment: "Frank 2001 @gid: 0 @home: "/home/frank" @shell: "/bin/bash" @password: nil @system: false @false @non_unique: false @cookbook_name: "users" @recipe_name: "default">

Friday, 4 July 14


139/321

chef:recipe > exit


140/321

chef >

Friday, 4 July 14

chef > reset


141/321

Loading.....INFO: Run List is [role[webserv

INFO: Run List expands to [motd, users, ntp

apache]

...INFO: Loading cookbooks [apache, motd, n

pci, users]

.done.

=> nil

Friday, 4 July 14

chef > nodes.list


142/321

=> [node[node1]]

Friday, 4 July 14

chef > nodes.all {|n| "#{n.name}:#{n.chef_environ


143/321

=> [“node1:_default”]

Friday, 4 July 14

chef > nodes.transform(:all) {|n| n.chef_environment("


144/321

=> [node[node1]]

Friday, 4 July 14



145/321

=> [“node1:dev”]

Friday, 4 July 14

$ knife exec -E 'nodes.transform(:all) {|n| n.chef_environment("pro


146/321

Has...no output >:)

Friday, 4 July 14

$ knife search node "*:*" -a chef_enviro


147/321

1 items found

node1: chef_environment: production

Friday, 4 July 14



148/321

=> ["node1:production"]

Friday, 4 July 14

• Changes are permanent

• You can do the same tricks on roles, data bag


149/321

items), clients... all kinds of stuff • You can (and will) blow off your foot if you are

careful

•

Friday, 4 July 14

$ knife exec -E 'nodes.transform(:all) {|n| n.chef_environment("_de


150/321

Has...no output >:)

Friday, 4 July 14

• What are chef-shell's three startup modes?

• What are chef-shell's three run-time contexts?


151/321

• How do you save attribute changes back to theobject?

• How do you invoke a chef run from within chef-

• How would you list nodes from within chef-shel

• What knife command can be used to invoke opthat you perform on the node object in the chef-

Friday, 4 July 14


152/321

152

Friday, 4 July 14


• Explain the function Ohai and Ohai Plugins


153/321

• Explain what’s new in Ohai 7?• Write your own Ohai Plugins

• Force Ohai Plugins to run

• Disable Ohai Plugins

153

Friday, 4 July 14

• Ohai writes automatic attributes - they can’t bchanged by other components of Chef


154/321

• Automatic attributes form the base for every nobject at the beginning of every Chef run

• Many plugins are enabled by default, but can disabled if desired

154

Friday, 4 July 14

chef@node1:~$ ohai


155/321

{ "languages": { "ruby": { "platform": "x86_64-darwin13.0.0", "version": "1.9.3",

"release_date": "2013-11-22",...

155

Friday, 4 July 14

• Platform details: redhat, windows, etc.

• Processor information


156/321

• Kernel data• FQDNs

• Cloud provider data (EC2, Azure, Rackspace

• Windows device drivers• and more...

156

Friday, 4 July 14

• New DSL

• collect_data() - block of Ruby code cal


157/321

Ohai when it runs• depends - comma-separated list of attribute

collected by another plugin

• provides - comma-separated list of attribudefined by a plugin

• In Chef 11.12.0 or newer

157

Friday, 4 July 14

chef@node1:~$ ohai -v


158/321

Ohai: 7.0.2

158

Friday, 4 July 14

• Problem: We want to capture all installed Apmodules as part of our node data


159/321

• Success: The Apache module data is visible node attributes

159

Friday, 4 July 14

chef@node1:~$ apachectl -t -D DUMP_MODU


160/321

Loaded Modules: core_module (static) mpm_prefork_module (static) http_module (static) so_module (static) auth_basic_module (shared)

auth_digest_module (shared) authn_file_module (shared) authn_alias_module (shared)...

160

Friday, 4 July 14

• Custom Ohai plugins are installed with the Ohcookbook


161/321

• Configures plugin path for Ohai• Delivers plugin to Ohai plugins directory with

• http://community.opscode.com/cookbooks/o

• http://docs.opscode.com/ohai.html

161

Friday, 4 July 14

$ knife cookbook site download ohai 2.0

http://docs.opscode.com/ohai.htmlhttp://community.opscode.com/cookbooks/ohaihttp://docs.opscode.com/ohai.htmlhttp://docs.opscode.com/ohai.htmlhttp://community.opscode.com/cookbooks/ohaihttp://community.opscode.com/cookbooks/ohai


162/321

Downloading ohai from the cookbooks siversion 2.0.0 to /Users/YOU/chef-repo/cookbooks/ohai-2.0.0.tar.gz

Cookbook saved: /Users/YOU/chef-repo/cookbooks/ohai-2.0.0.tar.gz

162

Friday, 4 July 14

$ tar -zxvf ohai-2.0.0.tar.gz -C cookbo


163/321

x ohai/x ohai/CHANGELOG.mdx ohai/README.mdx ohai/attributesx ohai/attributes/default.rbx ohai/filesx ohai/files/defaultx ohai/files/default/pluginsx ohai/files/default/plugins/READMEx ohai/metadata.jsonx ohai/metadata.rb

x ohai/providersx ohai/providers/hint.rbx ohai/recipesx ohai/recipes/default.rbx ohai/resourcesx ohai/resources/hint.rb

163

Friday, 4 July 14

$ knife cookbook upload ohai


164/321

Uploading ohai [2.0.0]Uploaded 1 cookbook.

164

Friday, 4 July 14

OPEN IN EDITOR: ohai/attributes/default.rb

# FHS location would be /var/lib/chef/ohai_plugins or

case node["platform_family"] when "windows"


165/321

default["ohai"]["plugin_path"] = "C:/chef/ohai_plugielse default["ohai"]["plugin_path"] = "/etc/chef/ohai_pluend

# The list of plugins and their respective file locati

default["ohai"]["plugins"]["ohai"] = "plugins"

165

Friday, 4 July 14

• We could create our modules.rb file and add ohai cookbook, but...


166/321

166

• We now have to maintain a forked version of community cookbook

• Let’s use our apache cookbook instead!

Friday, 4 July 14

OPEN IN EDITOR: cookbooks/apache/metadata.rb

name 'apache'maintainer 'Your Name'maintainer_email '[email protected]'

mailto:[email protected]:[email protected]


167/321

SAVE FILE!

license 'All rights reserved'description 'Installs/Configures apache'long_description IO.read(File.join(File.dirname( __FILE__ ),'README.md'))version '0.4.0'

depends 'ohai'

167

Friday, 4 July 14

Ohai.plugin(:Name) do

• Ohai.plugin(:Name) - used toidentify the plugin


168/321

end

168

Friday, 4 July 14


• provides - comma-separated listof attributes defined by the plugin

Ohai.plugin(:Name) do provides "attribute_a", "attr


169/321

169

end

Friday, 4 July 14



Ohai.plugin(:Name) do provides "attribute_a", "attr depends "attribute_x", "attri


170/321

170

• depends - comma-separated list ofattributes collected by anotherplugin

end

Friday, 4 July 14





171/321

171


• collect_data(:default) - thedefault code run if the platform isnot defined

collect_data(:default) do # some Ruby code attribute_a Mash.new end

end

Friday, 4 July 14





172/321

172


• collect_data(:default) - thedefault code run if the platform isnot defined

• collect_data(:platform) - thecode run for a specific platform

collect_data(:default) do # some Ruby code attribute_a Mash.new end

collect_data(:platform ...) do # platform-specific Ruby co attribute_b Mash.new endend

Friday, 4 July 14

OPEN IN EDITOR: apache/files/default/plugins/modul

Ohai.plugin(:Apache) do require 'mixlib/shellout' provides "apache/modules"


173/321

SAVE FILE!

collect_data(:default) do modules = Mixlib::ShellOut.new("apachectl -t -D DUMP modules.run_command apache Mash.new apache[:modules] = modules.stdout

endend

173

Friday, 4 July 14

OPEN IN EDITOR: cookbooks/apache/recipes/ohai_plug

ohai 'reload_apache' do plugin 'apache' action :nothing


174/321

SAVE FILE!

end

cookbook_file "#{node['ohai']['plugin_path']}/modules.rb" source 'plugins/modules.rb' owner 'root' group 'root' mode '0755' notifies :reload, 'ohai[reload_apache]', :immediatelyend include_recipe 'ohai::default'

174

Friday, 4 July 14



175/321

175



Friday, 4 July 14

$ knife node run_list set node1'recipe[apache::ohai_plugin]','role[base]','recipe[apa


176/321

node1: run_list: recipe[apache::ohai_plugin] role[base] recipe[apache]

176

Friday, 4 July 14


177/321

* cookbook_file[/etc/chef/ohai_plugins/README] action create - create new file /etc/chef/ohai_plugins/README - update content in file /etc/chef/ohai_plugins/README from none --- /etc/chef/ohai_plugins/README

2014-02-27 09:31:04.558032 +++ /tmp/.README20140227-33098-tpmpwc

2014-02-27 09:31:04.62

0500


178/321

-0500 @@ -1 +1,2 @@ +This directory contains custom plugins for Ohai. - change mode from '' to '0644' - restore selinux security context

Recipe: ohai::default * ohai[custom_plugins] action reload/opt/chef/embedded/lib/ruby/gem

gems/ohai-7.0.0.rc.0/lib/ohai/plugins/linux/network.rb:49: warning: ainitialized constant IPROUTE_INT_REGEX

- re-run ohai and merge results into node attributes

178

Friday, 4 July 14

Converging 3 resourcesRecipe: apache::ohai_plugin * cookbook_file[/etc/chef/ohai_plugins/modules.rb] action create - create new file /etc/chef/ohai_plugins/modules.rb - update content in file /etc/chef/ohai_plugins/modules.rb from none to 2d1a51 --- /etc/chef/ohai_plugins/modules.rb 2014-02-27 09:31:09.667647850 -0500

+++ /tm / mod les rb20140227 33098 sa id 2014 02 27 09 31 09 718673962


179/321

+++ /tmp/.modules.rb20140227-33098-savqid

2014-02-27 09:31:09.718673962 - @@ -1 +1,9 @@ +Ohai.plugin(:Apache) do + provides "apache/modules" + + collect_data(:default) do + modules = shell_out("apachectl -t -D DUMP_MODULES") + apache Mash.new + apache[:modules] = modules + end

+end - change mode from '' to '0755' - change owner from '' to 'root' - change group from '' to 'root' - restore selinux security context

179

Friday, 4 July 14

Recipe: ohai::default * ohai[custom_plugins] action reload/opt/chef/embedded/lib/ruby/1.9.1/gems/ohai-7.0.0.rc.0/lib/ohai/plugins/linux/network.rb:49: walready initialized constant IPROUTE_INT_REGEX


180/321

- re-run ohai and merge results into node attributes

* remote_directory[/etc/chef/ohai_plugins] action nothing (skippaction :nothing) * ohai[custom_plugins] action nothing (skipped due to action :no

Running handlers:

Running handlers complete

Chef Client finished, 5/5 resources updated in 16.185836857 second

180

Friday, 4 July 14

$ knife node show node1 -a apache

d 1


181/321

node1: apache: indexfile: index1.html modules: Loaded Modules: core_module (static) mpm_prefork_module (static) http_module (static) so_module (static) auth_basic_module (shared)...

181

Friday, 4 July 14

chef@node1:~$ cat /etc/chef/ohai_plugins/modu

Ohai plugin(:Apache) do


182/321

Ohai.plugin(:Apache) do require 'mixlib/shellout' provides "apache/modules"

collect_data(:default) do modules = Mixlib::ShellOut.new("apachectl -t -D DUMP_M modules.run_command

apache Mash.new apache[:modules] = modules.stdout end

182

Friday, 4 July 14

• We have Apache module data!

• It’s saved as one big string... that’s a bummer

Let’s sort this into a group of static and share


183/321

• Let’s sort this into a group of static and sharemodules

183

Friday, 4 July 14

Ohai.plugin(:Apache) do

require 'mixlib/shellout' provides "apache/modules"

ll d d f l d



184/321

collect_data(:default) do apache Mash.new apache[:modules] = {:static => [], :shared => []}

184

• Let’s start by setting up our node[‘apache’] M

Friday, 4 July 14


modules = shell_out("apachectl -t -D DUMP_MODULES") modules.stdout.each_line do |line| fullkey, value = line.split(/\(/, 2).map {|i| i.stri

apache mod = fullkey gsub(/ module/ "")


185/321

SAVE FILE!

apache_mod = fullkey.gsub(/_module/, ) dso_type = value.to_s.chomp("\)") if dso_type.match(/shared/) apache[:modules][:shared]


186/321

186



Friday, 4 July 14

chef@node1:~$ sudo chef-client

Starting Chef Client, version 11.10.4.ohai7.0

resolving cookbooks for run list: ["apache::ohai plugin"]


187/321

resolving cookbooks for run list: [ apache::ohai_plugin ]Synchronizing Cookbooks: - apache - ohaiCompiling Cookbooks...Recipe: apache::ohai_plugin * ohai[reload_apache] action nothing (skipped due to action :nothin * cookbook_file[/etc/chef/ohai_plugins/modules.rb] action create

- update content in file /etc/chef/ohai_plugins/modules.rb from e809f46 --- /etc/chef/ohai_plugins/modules.rb

2014-04-02 23:22:30.727-0400

187

Friday, 4 July 14

$ knife node show node1 -a apache.mod

node1:

apache.modules:


188/321

188

p shared: auth_basic auth_digest authn_file authn_alias authn_anon... static: core mpm_prefork http so

Friday, 4 July 14

chef@node1$ /opt/chef/embedded/bin


189/321

irb(main):001:0>

Friday, 4 July 14


190/321

irb(main):002:0> require 'ohai'

=> true

190

Friday, 4 July 14

irb(main):003:0> Ohai::Config['plugin path']


191/321

irb(main):003:0> Ohai::Config[ plugin_path ] o = Ohai::System.new

=> #


192/321

@data {}, @provides_map # Ohai::ProvidesMap:0x0000000186ac30 @map={}>, @v6_dependency_solver={@loader=#,@runner=#>

192

Friday, 4 July 14


193/321

irb(main):005:0> o.all_plugins=> [......]

193

Friday, 4 July 14

irb(main):006:0> o.refresh_plugins(attribute_filte

'apache/modules')


194/321

p / )

=> [#{"modules"=>{"static"=>["core","mpm_prefork", "http", "so"],"shared"=>["auth_basic", ...

194

Friday, 4 July 14

• Ohai can be forced to run, even if detected systsays that they shouldn't!

• For example its impossible to reliably detect wh


195/321

For example, its impossible to reliably detect whserver is in EC2, so EC2 plugin may not run

• Solution: Force the EC2 plugin to run by placingJSON document ("{}") in/etc/chef/ohai/hints/ec2.json

195

Friday, 4 July 14

• Some plugins can execute slowly on differentplatforms

• You can disable plugins whose data you don’

save speed things up!


196/321

save speed things up!

• Only disables plugins for Ohai when run with client

196

Friday, 4 July 14

• The Problem: Our systems are LDAP/ActiveDirectory joined and thus the Ohai data contathousands of user accounts, slowing down th

run


197/321

run.• Success Criteria: We will disable the :Passw

plugin to avoid collecting this information

197

Friday, 4 July 14

$ knife node show node1 -a etc.passwd

node1:

etc.passwd: abrt:


198/321

dir: /etc/abrt gecos:

gid: 173 shell: /sbin/nologin uid: 173 adm: dir: /var/adm gecos: adm gid: 4 shell: /sbin/nologin uid: 3 ...

198

Friday, 4 July 14

OPEN IN EDITOR: roles/base.rb

name "base"description "Base Server Role"run_list "recipe[chef-client::delete_validation]", "recipe[chef-client::config]", "recipe[chef-client]", "recipe[ntp]", "recipe[mot

"recipe[users]"default attributes(


199/321

SAVE FILE!

default_attributes( "chef_client" => { "config" => { "ssl_verify_mode" => ":verify_peer" } }, "ohai" => {

"disabled_plugins" => [":Passwd"] })

199

Friday, 4 July 14

$ knife role from file base.rb


200/321

Updated Role base!

200

Friday, 4 July 14

chef@node1:~$ sudo chef-client

Starting Chef Client, version 11.12.4resolving cookbooks for run list: ["apache::ohai_plug


201/321

"chef-client::delete_validation", "chef-client", "ntp"motd", "users", "apache"]...Running handlers:Running handlers complete

Chef Client finished, 1/4 resources updated in 9.6900seconds

201

Friday, 4 July 14

$ knife node show node1 -a etc.passwd


202/321

node1: etc.passwd:

202

Friday, 4 July 14

chef@node1:~$ sudo cat /etc/chef/client.rb

chef_server_url "https://api.opscode.com/organizations/intermediate050614"

validation_client_name "intermediate050614-validator"verify_api_cert truel if d if


203/321

ssl_verify_mode :verify_peernode_name "node1"

Ohai::Config[:plugin_path]


204/321

Ohai::Config[:disabled_plugins]

["passwd"]

• Ohai 7

•Ohai::Config[:disabled_plugins] [:Passwd]

204

Friday, 4 July 14

• What command can you run at the commandinvoke Ohai?

• How are Ohai plugins installed?


205/321

o a e O a p ug s sta ed

• What is wrong with this first line of an Ohai de

"Ohai.plugin(:name) do"?

205

Friday, 4 July 14


206/321

206

Friday, 4 July 14

• After completing the lesson, you will be able to:

• Understand what happens under the hood whClient runs

• Describe the purpose of the run context an


207/321

_run_status objects

• Use node.run_state to pass information arthe recipe at execution time

• Describe the event subsystem and how it can used

207

Friday, 4 July 14


208/321

208

Friday, 4 July 14

• Ohai Pl

• StartHandle

• Event


209/321

Dispatc

209

Friday, 4 July 14

• LWRPs

• HWRPs

• Librarie


210/321

210

Friday, 4 July 14

• ExceptiHandle

• Report

Handle


211/321

211

Friday, 4 July 14

• Master object for the Chef run for this node

• Important parts of the hierarchy:

• run_context

• node• chef environment


212/321

• chef_environment

• run_status

• run_state

• cookbook_collection

• resource_collection

• events

212

Friday, 4 July 14

• Hangs off node

• Stores current status of run

• Resources that are converged, resources thachanged during this run


213/321

• Run status methods (success? / failed?)

• We’ll use this object later when we write even

handlers

213

Friday, 4 July 14

• A Hash that hangs off

node

• A place for you to

stash transient datad i h


214/321

during the run

• Transmits databetween resources

214

Friday, 4 July 14

• Problem: We can’t decidewhat scripting language wewant to use to write our web

application.


215/321

• Success Criteria: Chef hasrandomly chosen one for us

each time it runs.

215

or

Friday, 4 July 14

OPEN IN EDITOR: apache/recipes/default.rb

package "httpd" do action :installend

ruby_block "randomly_choose_language" do

block do if Random .rand > 0.5 node.run_state['scripting_language'] = 'php'


216/321

SAVE FILE!

else node.run_state['scripting_language'] = 'perl' end endend

package "scripting_language" do

package_name lazy { node.run_state['scripting_language'] } action :installend

216

Friday, 4 July 14

ruby_block "randomly_choose_language" do block do if Random .rand > 0.5 node.run_state['scripting_language'] = 'php' else node.run_state['scripting_language'] = 'perl'

end endend


217/321

end

217

• ruby_block declares a block of Ruby to run at execute tim

• It has direct access to the node structure and other aspects • Store something on node.run_state for later use

Friday, 4 July 14

node.run_state['scripting_language'] = 'php'

• node.run state is an empty Hash


218/321

ode. u _state s a e p y as

• Discarded at the end of the run

218

Friday, 4 July 14

package "scripting_language" do package_name lazy { node.run_state['scripting_language'] } action :installend

• Normally resource parameters must be specifie


219/321

y p pcompletely at compile time

• Sometimes their value is not known until execu

• Use the lazy{} block to have parameters evathen

219

Friday, 4 July 14



220/321



220

Friday, 4 July 14


* ruby block[randomly choose language] action run


221/321

ruby_block[randomly_choose_language] action run - execute the ruby block randomly_choose_language

* package[scripting_language] action install - install version 5.3.3-27.el6_5 of package php

221

Friday, 4 July 14

• Used by logging, Enterprise Chef reporting

• Simple queue-free pub/sub model

• All listeners get all events

• Log formatters are examples of Event Dispatc


222/321

subclasses

222

Friday, 4 July 14

• Write a subclass of

Chef::EventDispatch::Base

• Base methods are all set to take no action

• Implement methods in subclass to be called


223/321

• Examples: run_start, run_completed

• Create a start handler to load your event disp

• Register your handler with Chef (next chapter

223

Friday, 4 July 14

• What is the name of the root object of the Che

• Where would you stash transient data during t


224/321

• What object stores current status of run

• What can be used to populate resource attribuexecute time

224

Friday, 4 July 14


225/321

225

Friday, 4 July 14

• After completing this lesson, you will be able

• Describe Chef handlers

• Write custom report handlers

• Distribute handlers to nodes


226/321

• Configure Chef to use custom handlers

226

Friday, 4 July 14

• Handlers run at the start or end of a chef-clien

• They are Ruby programs that can read the stinformation of your chef-client run

• Three types:

R t


227/321

• Report

• Exception

• Start

227

Friday, 4 July 14

• Used when a chef-client run succeeds

• Runs when the run_status.success? =

• Community Examples:

• http://ampledata.org/splunk_storm_chef_ha

https://github.com/realityforge/chef-graphite_handlerhttp://onddo.github.io/chef-handler-zookeeper/http://ampledata.org/splunk_storm_chef_handler.htmlhttp://ampledata.org/splunk_storm_chef_handler.html


228/321

• http://onddo.github.io/chef-handler-zookeep

• https://github.com/realityforge/chef-graphite_

228

Friday, 4 July 14

• Used to identify situations that have caused aclient run to fail

• Runs when the run_status.failed? = t

• Community Examples:

htt // ith b / th/ i b k h dl

http://onddo.github.io/chef-handler-zookeeper/http://ampledata.org/splunk_storm_chef_handler.htmlhttps://github.com/realityforge/chef-graphite_handlerhttps://github.com/realityforge/chef-graphite_handlerhttp://onddo.github.io/chef-handler-zookeeper/http://onddo.github.io/chef-handler-zookeeper/http://onddo.github.io/chef-handler-sns/http://onddo.github.io/chef-handler-sns/https://github.com/jblaine/syslog_handlerhttps://github.com/morgoth/airbrake_handlerhttps://github.com/morgoth/airbrake_handler


229/321

• https://github.com/morgoth/airbrake_handle

• https://github.com/jblaine/syslog_handler

• http://onddo.github.io/chef-handler-sns/

229

Friday, 4 July 14

• Used to run events at the beginning of the chrun

• May NOT be loaded using the chef_handler

resource• Install with chef gem or add to the

https://github.com/morgoth/airbrake_handlerhttp://onddo.github.io/chef-handler-sns/http://onddo.github.io/chef-handler-sns/https://github.com/jblaine/syslog_handlerhttps://github.com/jblaine/syslog_handlerhttps://github.com/morgoth/airbrake_handlerhttps://github.com/morgoth/airbrake_handlerhttps://github.com/opscode/chef-reportinghttps://github.com/opscode/chef-reportinghttps://github.com/opscode/chef-reporting


230/321

Install with chef_gem or add to thestart_handlers setting in the client.rb

• https://github.com/opscode/chef-reporting

230

Friday, 4 July 14

• Use any ruby gem or library

• Distribute to nodes with the chef_handler chef-client cookbooks

• Optionally configure through client.rb sett

https://github.com/opscode/chef-reportinghttps://github.com/opscode/chef-reporting


231/321

231

Friday, 4 July 14

$ knife cookbook site download chef_handler 1

Downloading chef_handler from the cookbooks at version 1.1.6 to /Users/YOU/chef-repo/

h f h dl 1 1 6 t


232/321

chef_handler-1.1.6.tar.gz

Cookbook saved: /Users/YOU/chef-repo/

chef_handler-1.1.6.tar.gz

232

Friday, 4 July 14

$ tar -zxvf chef_handler-1.1.6.tar.gz -C cook

x chef_handler/

x chef_handler/CHANGELOG.mdx chef_handler/README.mdx chef_handler/attributes


233/321

x chef_handler/attributes/default.rbx chef_handler/filesx chef_handler/files/defaultx chef_handler/files/default/handlers

x chef_handler/files/default/handlers/READMEx chef_handler/librariesx chef_handler/libraries/matchers.rb

233

Friday, 4 July 14

$ knife cookbook upload chef_handler

Uploading chef handler [1 1 6]


234/321

Uploading chef_handler [1.1.6]


234

Friday, 4 July 14

• We’re going to write a new handler that will emwhen chef-client runs

• It will send us all of the resources that change

during the chef-client run• http://docs.opscode.com/handlers.html

http://docs.opscode.com/handlers.html


235/321

p p

235

Friday, 4 July 14

• We could modify the chef_handler cookboadd a recipe for the email handler

• However, we’d basically be “forking” an upst

cookbook• Instead, let’s create our own cookbook and us

http://docs.opscode.com/handlers.htmlhttp://docs.opscode.com/handlers.html


236/321

,

chef_handler as a “library”

• Code reusability!

236

Friday, 4 July 14

$ knife cookbook create email_hand

** Creating cookbook email_handler** Creating README for cookbook: email hand


237/321

** Creating README for cookbook: email_hand** Creating CHANGELOG for cookbook: email_h** Creating metadata for cookbook: email_ha

237

Friday, 4 July 14

OPEN IN EDITOR: cookbooks/email_handler/recipes/def

chef_gem "pony" do action :installend

include recipe "chef handler"


238/321

SAVE FILE!

include_recipe chef_handler

238

Friday, 4 July 14

• chef_handler is a resource packaged with the chef_handle

• It has two actions, :enable and :disable

• It has three arguments

• source - the file containing the handler code

• arguments - any pieces of information needed to initialize• supports - :report, :exception


239/321

• Defaults:

• :enable

• :report => true, :exception => true

239

Friday, 4 July 14

OPEN IN EDITOR: cookbooks/email_handler/recipes/def

cookbook_file "#{node['chef_handler']['handler_path']}/email_handle source "handlers/email_handler.rb" owner "root" group "root" mode "0755"

end

chef_handler "MyCompany::EmailMe" do"#{ d [' h f h dl ']['h dl th']}/ il h dl b"


240/321

SAVE FILE!

source "#{node['chef_handler']['handler_path']}/email_handler.rb" arguments [node['email_handler']['from_address'], node['email_handler']['to_address']] action :enableend

240

Friday, 4 July 14

OPEN IN EDITOR:

SAVE FILE!

cookbooks/email_handler/attributes/de

default['email_handler']['from_address'] = "chef@localdefault['email_handler']['to_address'] = "chef@localho


241/321

Documents

Chef_Intermediate_v1.0.0.pdf