In Today’s Session You Will Learn how to: •  Gain visibility on site performance •  Improve scalability and uptime •  Find and fix key bottlenecks

•  Database •  Web Servers •  Caching •  Background Processing

New Relic + Engine Yard

Web Request Overview

Web Application Overview

DATA BASE

Lazy loading associated data can quickly lead to an N+1 query problem.

ORMs (ActiveRecord, DataMapper, etc.) make it easy to get our data but also make it easy to forget to optimize and refactor.

N+1 problems are hard to spot in development since you are working with limited data sets.

Database Performance

# app/models/customer.rb class Customer < ActiveRecord::Base has_many :addresses

end

# app/models/address.rb class Address < ActiveRecord::Base belongs_to :customer

end

# app/controllers/customers_controller.rb class CustomersController < ApplicationController def index @customers = Customer.all end

end

# app/views/customers/index.html.erb <% @customers.each do |customer| %> <%= content_tag :h1, customer.name %>

<% end %>

N+1 Query Creep

# app/views/customers/index.html.erb

<% @customers.each do |customer| %>

<%= content_tag :h1, customer.name %>

<%= content_tag :h2, customer.addresses.first.city %>

<% end %>

If @customers has 100 records, you'll have 101 queries:

SELECT "customers".* FROM "customers"

SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 1 AND "addresses"."primary" = 't' LIMIT 1

SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 2 AND "addresses"."primary" = 't' LIMIT 1

SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 3 AND "addresses"."primary" = 't' LIMIT 1

...

...

SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 100 AND "addresses"."primary" = 't' LIMIT 1

N+1 Query Creep

# app/controllers/customers_controller.rb

class CustomersController < ApplicationController

def index

@customers = Customer.includes(:addresses).all

end

end

If @customers has 100 records, now we only have 2 queries:

SELECT "customers".* FROM "customers"

SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100)

Eager Loading with .includes

New Relic > App Server > Web Transactions > Performance Breakdown

Finding N+1 in New Relic

Missing Indexes == Slow Queries

# db/migrate/20120201040247_add_index_for_shop_id_on_orders.rb

class AddIndexForShopIdOnOrders < ActiveRecord::Migration

def change

add_index :orders, :shop_id

end

end

Adding an Index is Simple

Index Protips:

•  Searching an index on a table with 1,000 rows is 100x faster than searching a table without an index.

•  Put an index on any columns you will likely query against, it's better to have too many than too few indexes.

•  Adding an index to a table will lock the table!

RUBY WEB SERVERS

•  Simple to operate •  Simple configuration •  Handles worker management •  Great for multi-application environments •  Great for low resource environments •  Attached to Nginx/Apache HTTPD

Passenger 3

Passenger Request Queue

solo i-c3f2d8a2 ~ # passenger-status ----------- General information ----------- max = 3 count = 3 active = 0 inactive = 3 Waiting on global queue: 0 ----------- Application groups ----------- /data/john_yerhot_org/current: App root: /data/john_yerhot_org/current * PID: 19802 Sessions: 0 Processed: 3 Uptime: 3h 10m 13s /data/scalingrails/current: App root: /data/scalingrails/current * PID: 28726 Sessions: 0 Processed: 3 Uptime: 59m 22s /data/sites/clmeisinger/current: App root: /data/sites/clmeisinger/current * PID: 22147 Sessions: 0 Processed: 70 Uptime: 10h 45m 57s

•  Independent of front end web server •  More configuration options •  Master process will reap children on timeout •  Great for single application environments •  Allows for zero downtime deploys

Unicorn

Unicorn Request Queue?

Raindrops solo i-5b74313d ~ # gem install raindrops Fetching: raindrops-0.10.0.gem (100%) Building native extensions. This could take a while... Successfully installed raindrops-0.10.0 1 gem installed solo i-5b74313d ~ # ruby -rubygems -e "require 'raindrops'; puts Raindrops::Linux.unix_listener_stats(['/var/run/engineyard/unicorn_appname.sock']).inspect" {"/var/run/engineyard/unicorn_appname.sock"=>#<struct Raindrops::ListenStats active=0, queued=0>}

Request Queuing in New Relic

Request Queuing in New Relic

NOT COOL

Request Queuing in New Relic

•  Time between first ActionContoller hit - X-Queue-Start = Time spent in queuing.

Internet => LB inserts X-Queue-Start => Nginx => Ruby Webserver => Rack => Application

Track Rack Middleware as well def call(env) env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}" @app.call(env) end

CACHING

Cache Everything

Rails makes it stupid easy to

cache everything. Do it.

Static Files & Nginx

The best cache is a static file served by Nginx.

# create it on #index, #show, etc.. caches_page :index # expire it on #creates, #updates, #destory, etc... expire_page :action => :index

A Note About Static Files:

Use the front end server. upstream upstream_enki { server unix:/var/run/engineyard/unicorn_enki.sock fail_timeout=0; } location ~ ^/(images|assets|javascripts|stylesheets)/ { try_files $uri $uri/index.html /last_assets/$uri /last_assets/$uri.html @app_enki; expires 10y; } location / { if (-f $document_root/system/maintenance.html) { return 503; } try_files $uri $uri/index.html $uri.html @app_enki; }

Memcached: The Standard

# config/initializers/memcached.rb config.cache_store =:mem_cache_store,

"server-1:11211", "server-2:11211", "server-3:11211",

"server-4:11211"

Next Best: ActionCaching

Will still go through Rack/Rails, but the action gets cached. before_filter :make_sure_youre_ok caches_action :all_the_things def all_the_things @all_things = Thing.all_in_a_complex_way end def expire expire_action :action => :all_the_things end

Fragment Caching

<% cache('my_cache_key') do %> <%= render_large_tag_cloud %> <% end %> ... def update_large_tag_cloud TagCloud.update expire_fragment('my_cache_key') end

Baremetal

Rails.cache.write("john", "yerhot") Rails.cache.read("john")# => "yerhot" # execute a block on miss and cache it. Rails.cache.fetch("miss") do "yerhot" end Rails.fetch("miss")# => "yerhot" Rails.cache.exists("john") # => true Rails.cache.delete("john") # => true Rails.cache.exists("john") # => false

Background Processing

•  send email •  process images •  grab feeds and cache them •  complex computations/reports •  create/expire caches/pages (like Reddit)

Why Background Processing?

Best Practice:

Use a utility server for background jobs and cron.

Resque to the Rescue

Resque in New Relic

Delayed Job Too

Background Processing baked in.

•  Allow an application to switch job systems with minimal code change due to common API

•  Very basic queuing system built in •  Roll your own wrapper class that responds to push & pop

Rails 4

# application.rb config.queue = QueueName Rails.queue.push(Job.new)

•  You need to be monitoring your application.

•  Performance has to be reviewed on a regular basis.

•  Database indexes are cheap, make lots of them.

•  Every application can take advantage of some level of caching: page, action or fragment.

•  Background any work that you can.

•  Don't neglect front-end performance.

Review

New Relic Standard is Free at Engine Yard

1.  If you’re an Engine Yard Customer, select your plan in your Engine Yard Account Settings

2.  Add newrelic_rpm to your Gemfile

3.  Enable monitoring in the Engine Yard Dashboard

Full Installation Details: http://ey.io/install-newrelic

How to Install New Relic

Questions?

Chris Kelly @amateurhuman www.newrelic.com

John Yerhot @yerhot www.engineyard.com

Thanks for Watching!