J Á N S U C H A L

@ J S U C H A L

Profiling and monitoring ruby/rails

Optimalization

“If you can’t measure it, you can’t improve it.” – Lord Kelvin Environment

development vs. production (hw, sw, load) data (synthetic vs. real)

Bottlenecks 10sec * 1 run vs. 0.5s * 100 runs

run time * runs vs. development time

Microbenchmarks waste of time vs. 20 * 1% = 20%

“There are three kinds of lies: lies, damned lies, and statistics.” – Benjamin Disraeli single run vs. multiple runs average vs. standard deviance, percentiles cold vs. hot cache

Profiler - Example

# profiler/dates.rb require 'date' def create_days_after(date_str, n) after = Date.strptime(date_str) + n after.strftime("%Y-%m-%d") end 1000.times do create_days_after("1982-10-27", 5) end

Ruby Profiler

$ gem install ruby-prof

$ ruby-prof dates.rb How long does each method take?

$ ruby-prof dates.rb –m 3 Just methods above 3% time

Thread ID: 7749480 Total: 0.653076 %self total self wait child calls name 12.97 0.19 0.08 0.00 0.18 2000 String#scan 11.57 0.08 0.08 0.00 0.00 205000 String#=== 6.87 0.18 0.04 0.00 0.14 1000 String#gsub 6.28 0.05 0.04 0.00 0.01 14000 Hash#values_at 4.08 0.03 0.03 0.00 0.00 80470 Hash#default 3.21 0.03 0.02 0.00 0.01 3000 Date#emit

Ruby Profiler

total – time in method and children calls

self – time in method call

wait – wait time

child – time in child calls

call – number of times method invoked

Thread ID: 7749480 Total: 0.653076 %self total self wait child calls name 12.97 0.19 0.08 0.00 0.18 2000 String#scan 11.57 0.08 0.08 0.00 0.00 205000 String#=== 6.87 0.18 0.04 0.00 0.14 1000 String#gsub 6.28 0.05 0.04 0.00 0.01 14000 Hash#values_at 4.08 0.03 0.03 0.00 0.00 80470 Hash#default 3.21 0.03 0.02 0.00 0.01 3000 Date#emit

Ruby Profiler

$ ruby-prof dates.rb -m 3 -p graph

Which method calls what and how many times?

Thread ID: 15523700 Total Time: 0.675587048 %total %self total self wait child calls Name 0.20 0.10 0.00 0.20 2000/2000 <Class::Date>#_strptime_i 30.28% 14.70% 0.20 0.10 0.00 0.20 2000 String#scan 0.04 0.04 0.00 0.00 110000/205000 String#=== 0.02 0.01 0.00 0.01 3000/5000 Date::Format::Bag#method_... 0.02 0.00 0.00 0.01 2000/3005 Class#new 0.01 0.01 0.00 0.00 5000/5000 String#sub! 0.01 0.00 0.00 0.01 2000/2000 Range#=== 0.00 0.00 0.00 0.00 3000/3000 String#to_i 0.00 0.00 0.00 0.00 2000/2000 <Class::Regexp>#quote 0.00 0.00 0.00 0.00 2000/2000 Regexp#=== 0.00 0.00 0.00 0.00 1000/1000 <Class::Date>#num_pattern? 0.00 0.00 0.00 0.00 1000/2000 <Class::Date>#_strptime_i

Ruby Profiler

Thread ID: 15523700 Total Time: 0.675587048 %total %self total self wait child calls Name 0.20 0.10 0.00 0.20 2000/2000 <Class::Date>#_strptime_i 30.28% 14.70% 0.20 0.10 0.00 0.20 2000 String#scan 0.04 0.04 0.00 0.00 110000/205000 String#=== 0.02 0.01 0.00 0.01 3000/5000 Date::Format::Bag#method_..

three parts

parent calls

method

children calls

calls – number of calls from method/total number of calls

Ruby Profiler

$ ruby-prof dates.rb -p graph_html -m 3 > graph.html

$ ruby-prof dates.rb -p graph_html -m 3 –s self > graph.html

Profiler - KCacheGrind

$ ruby-prof dates.rb -p call_tree -m 3 > dates1.grind

$ kcachegrind dates1.grind

benchmark-ips

require 'benchmark/ips'

require 'ostruct'

require 'hashr'

require 'hashugar'

SMALL_HASH = {:a => 1, :b => 2}

Benchmark.ips do |x|

x.report 'OpenStruct create small hash and access once', 'OpenStruct.new(SMALL_HASH).item5'

x.report 'Hashr create small hash and access once', 'Hashr.new(SMALL_HASH).item5'

x.report 'Hashugar create small hash and access once', 'Hashugar.new(SMALL_HASH).item5‘

end

OpenStruct create small hash and access once

43858.0 (±5.5%) i/s - 221820 in 5.074250s (cycle=3697)

Hashr create small hash and access once

67408.9 (±5.0%) i/s - 339780 in 5.053728s (cycle=5663)

Hashugar create small hash and access once

230217.9 (±4.2%) i/s - 1152670 in 5.015705s (cycle=15790)

Memory profiling

Patched ruby

$ rvm install 1.9.3 --patch railsexpress --name gc

$ ruby-prof --mode=allocations dates.rb –m 3

%self total self wait child calls name 59.23 6004.00 6004.00 0.00 0.00 1000 <Class::Date>#strptime 9.87 1000.00 1000.00 0.00 0.00 1000 Date#strftime 9.87 9004.00 1000.00 0.00 8004.00 1000 Object#create_days_after 9.87 1000.00 1000.00 0.00 0.00 1000 Date#+ 9.87 10004.00 1000.00 0.00 9004.00 1 Integer#times

Rails / NewRelic Developer mode

gem 'newrelic_rpm‘

http://localhost:3000/newrelic

gem 'newrelic_rpm', git: 'git://github.com/jsuchal/rpm.git', branch: 'feature-profile-sorting'

NewRelic Developer mode

Rails / NewRelic Developer mode

NewRelic Developer mode

NewRelic Developer mode Profiler

Custom Method Tracers

# initializers/elastic_search_traces.rb

require 'new_relic/agent/method_tracer‘

ElasticSearch.class_eval do

include NewRelic::Agent::MethodTracer

add_method_tracer :search, 'Custom/elasticsearch/search'

add_method_tracer :index, 'Custom/elasticsearch/index'

end

NewRelic Production Monitoring

Web Transactions – controller actions drilldown

Transaction Traces – detailed slow requests

Slow SQL – slow queries

Background job monitoring

Availabality monitoring

Deployment tracking

Scalability analysis

...

Server monitoring (load, disks, …)

Error tracking

ExceptionNotifier

www.airbrake.io

Most common performance problems

“1 + N query problem” joins FTW!

Lack of proper indexing

Unnecessary ActiveRecord loading e.g. count vs. size vs. length

Default GC parameters 37signals params

RUBY_HEAP_MIN_SLOTS=600000 # This is 60(!) times larger than default

RUBY_GC_MALLOC_LIMIT=59000000 # This is 7 times larger than default

RUBY_HEAP_FREE_MIN=100000 # This is 24 times larger than default

Unknown abstraction internals e.g. OpenStruct