Cluster management with
the Ruby shell and Unix integration library
Adam WigginsApril 18, 2008http://rush.heroku.com/
bash+ssh are the cornerstone of unix systems administration
but...
Ruby is its own universe
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
unix = awesomeruby = awesome
So put them together and you should get awesome x awesome...
...right?
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
What are the problems here?
Quoting
Try inserting a single quote into the regexp here. How many backslashes do you need?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Security
What if this value can be entered by the user, and they type in: ";rm -rf /
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Output is all text
Why do we need the inverse grep, or for that matter the complex regexp, just to get a simple list of PIDs?
pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"`
Error Trapping
How can we catch errors, like permission denied or file not found, and log them?
log_contents = `ssh #{host} "cat log/mongrel.log"`
Unit Testability
Um... yeah. Good luck with that.
`ssh #{host} "echo 'mongrel_rails start -d' | sudo -u #{user} -H bash"`
In short, Unix and Ruby do not play well together.
And that’s really a shame.
Introducing...
Bringing Ruby and Unix together.awesome x awesome!
Let’s try that again.
rush-style.
def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |
sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`
# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)
# do it `ssh #{host} "echo 'mongrel_rails start -d' |
sudo -u #{user} -H bash"`end
def start_mongrel # kill any existing mongrels box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
# don't try to restart if previous mongrel crashed log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
# do it box.bash 'mongrel_rails -d', :user => userend
Processes box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end
Files log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)
Bash Commands box.bash 'mongrel_rails -d', :user => user
Basic concepts of rush
Boxes box = Rush::Box.new('remote.example.com') box['/var/log/nginx/*.log'].search(/error/)
Files & Dirs dir = home['myapp/'] dir['config/environment.rb'].contents
dir['**/*.rb'].search(/^module /)
dir['README'].copy_to other_dir
Processes mongrels = box.processes.find_all_by_cmdline /mongrel_rails/ mem_hogs = mongrels.select { |p| p.mem > 50.mb } mem_hogs.each { |p| p.kill }
Permissions file.access = { :user_can => :read_and_write, :group_can => :read }
Bash Commands box.bash 'rake db:migrate'
Bash Commands box.bash 'rake db:migrate', :user => 'www', :env => { :RAILS_ENV => 'production' }
Bash Commands begin box.bash 'rake db:migrate' rescue Rush::BashFailed => e log_warning "Failed to migrate: #{e.message}" end
It’s Ruby! spec_lines = myproj['**/*_spec.rb'].line_count total_lines = myproj['**/*.rb'].line_count spec_percent = (spec_lines * 100 / total_lines).round puts "Specs are #{spec_percent}% of the total code." puts (spec_percent >= 30) ? "Nice!" : "Tsk, tsk."
It’s Ruby!
“I can not adequately convey the sheer joy it was to rewrite my Bash deployment functions in Ruby.”
- Jamie Hoover
What’s good about the traditional unix shells?
Lessons to carry forward.
Remote and local are seamless
box1['myproj/'].move_to box2
Small sharp tools, chained together
dir1['**/*.rb'].search(/^class/).copy_to dir2
Why now?
Cloud computing.Why now?
Cloud computing.
Which means automation of systems administration tasks.
Why now?
Pre-cloud computing, sysadmin tasks are occasional and manual
In cloud computing, sysadmin tasks are frequent and automatic
In cloud computing, sysadmin tasks are frequent and automatic
So we need a real programming language for them!
My need came from:
hostnames = %w(host1 host2 host3 host4)boxes = hostnames.map { |h| Rush::Box.new(h) }
arc_host = Rush::Box.new('archive')archive = arch_host['/arc/nginx_logs/']
boxes.each do |box| log = box['/var/log/nginx/access.log'] archive_name = "#{box.host}_#{log.name}" log.copy_to archive[archive_name] log.rename log.name + ".old"end
Clustering (finally)
gem install rush