Deploying Rails Applications on EC2

With help from Capistrano/Rubber

About Me

• Matthew Conway

• Director of Engineering at SnapMyLife

• email: matt@snapmylife.com

• twitter: mattconway

• github: wr0ngway

About SnapMyLife

• Mobile Photo Sharing Site

• http://snapmylife.com

• Deploys to EC2 with rubber/capistrano

• Approximately 10m pageviews/month

Rubyists turn me on!

Amazon Elastic Compute Cloud (EC2)• Hardware (virtual) as a service

• Scriptable cloud computing

• Have to assume all instances are transient (this has benefits!)

• Easy ami bundling for custom servers, support for static IPs, security groups

EC2 Problems

• Transient

• Redundancy

• Backups!

• Scripted instance construction for short downtimes while rebuilding

EC2 Problems (cont.)

• DNS Issue

• Use elastic (static) ips for web/mail roles

• DNS round robin with a short TTL

• Monitor web instances from each other, remove from round robin if a failure

EC2 Problems (cont.)

• Not good for sending email

• SPAM central

• IPs are blacklisted

• Use something like authsmtp.com or smtp.com, or host your mail server elsewhere

Our Deployment Needs

• Configure multiple instances as easily as one

• Easy bootstrapping and extension - didn't want to force need for a custom image at start

Rubber

• Free and open sourced

• http://github.com/wr0ngway/rubber/wikis

• Makes it easy to get up and running on ec2 with a single instance

• Allows one to add in instances to scale as needed on a role by role basis

Start Demo

• Create a simple rails app

• Setup a single instance to host it

• Lifted from quickstart on rubber wiki: http://github.com/wr0ngway/rubber/wikis

Demo - create rails app

rails rubbertest; cd rubbertest./script/generate scaffold post title:string body:text

Add to app/views/posts/index.html.erb :<%= `hostname` %> <br /><%= ObjectSpace.each_object(Mongrel::HttpServer) {

|i| @port = i.port};@port %> <br />

<%= `hostname -i` %> <br /><%= require 'open-uri' open("http://169.254.169.254/latest/meta-data/public-ipv4").read() %> <br />

Demo - Install rubber./script/plugin install git://github.com/wr0ngway/rubber.git./script/generate vulcanize complete_mysql

edit config/rubber/rubber.yml:

aws_access_key: ****aws_secret_access_key: ****aws_account: ****ec2_key_name: ec2_key_file:staging_roles: app,memcached,mysql_master,web,web_munin

Demo - create instancecap rubber:create# use alias: production# use roles: app,memcached,mysql_master,web,web_munincap rubber:bootstrapcap deploy:setupcap rubber:monit:stopcap deploy:cold

Or use convenience task to do all the above:cap rubber:create_staging

Demo - view results

• Uses /etc/hosts for demo

• site: http://production.foo.com

• munin: http://production.foo.com:8080

Rubber (cont.)

• My attempt at getting people to share deployment recipes

• Uses rails generators, so you can customize your config as you grow

• ec2 conveniences such as easy ami bundling, support for static IPs, security groups

Rubber Features

• Capistrano rocks! rubber is a capistrano plugin

• DRY configuration

• Role and/or host based configuration of instances

Rubber Configuration System

• All configuration for your instances lives in your project's source tree and gets applied on deploy

• Configuration system uses ERB templates for config files

• Configuration system handles dynamically adding instances

Config templatemunin host config

<% rubber_instances.each do |i| %>[<%= i.full_name %>] address <%= i.internal_ip %> use_node_name yes<% end %>

Configuration (cont.)

• Configuration templates for sharing deployment scenarios (vulcanize generator)

• Start with a single instance and add more up as needed - no configuration edits needed

• Use whatever AMIs you want (so long as they are debian based :)

Rubber Features (cont.)

• Use /etc/hosts or updates dynamic dns entries automatically

• Easily/quickly create standalone staging/dev/loadtest instances that mirror production setup

Anti-Features

• Initial experience is fairly turnkey, but you will need some expertise as you grow

• Tied to Rails, shouldn’t be hard to divorce

• Not (yet) a gem

• Many other TODOs, feel free to contribute

• No dynamic instances (like poolparty) - you create them all statically. Works for us.

Demo +1 app serverALIAS=app01 ROLES=app,memcached cap rubber:createcap rubber:bootstrapcap deploy:setupcap deploy... check load balancing at http://production.foo.com/postsALIAS=app01 cap rubber:destroy... check load balancing at http://production.foo.com/posts

Individual Staging Servers

• Incredibly valuable, you know it works before you deploy for real

• Quick and easy

• cp config/environments/production.rb config/environments/matt.rb

• RAILS_ENV=matt cap rubber:create_staging

• Mimics production

Complex setup

ALIAS=web01 ROLES=web,internal_web cap rubber:createALIAS=app01 ROLES=app cap rubber:createALIAS=app02 ROLES=app cap rubber:createALIAS=db01 ROLES=mysql_master cap rubber:createALIAS=db02 ROLES=mysql_slave cap rubber:createcap rubber:bootstrapcap deploy:setupcap deploy:cold

Mysql cluster setup./script generate mysql_clusterALIAS=data01 ROLES=mysql_data cap rubber:createALIAS=data02 ROLES=mysql_data cap rubber:createALIAS=data03 ROLES=mysql_data cap rubber:createALIAS=sql01 ROLES=mysql_sql cap rubber:createALIAS=sql02 ROLES=mysql_sql cap rubber:createALIAS=mgm01 ROLES=mysql_mgm cap rubber:createcap rubber:bootstrapcap deploy:setupcap deploy:cold

Bring it all Down

• cap rubber:destroy_all

• Elapsed cost, a few cents

• No idle hardware

• No obsolete hardware

• We can do it all again without any additional effort

Questions?

• Alternatives listed in FAQ in rubber github wiki