Debugging Ruby

Debugging RubyDebugging Ruby code & Rails applications by example

Ryan BiggThis book is for sale at http://leanpub.com/debuggingruby

This version was published on 2014-10-16

This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishingprocess. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools andmany iterations to get reader feedback, pivot until you have the right book and build traction onceyou do.

2013 - 2014 Ryan Bigg

Tweet This Book!Please help Ryan Bigg by spreading the word about this book on Twitter!The suggested hashtag for this book is #debuggingruby.Find out what other people are saying about the book by clicking on this link to search for thishashtag on Twitter:https://twitter.com/search?q=#debuggingruby

Contents

General Debugging Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Debugging frame of mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Debugging Ruby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Basic Example #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Basic Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Basic Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Debugging Rails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Rails Example #1 - form_for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Rails Example #2 - Routing HTTP verbs/methods . . . . . . . . . . . . . . . . . . . . . . . 20Debugging slow code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27TODO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Handling Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Advanced Rails Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Advanced Rails Example #1 - Broken Devise Application . . . . . . . . . . . . . . . . . . . 32

CONTENTS i

Thank you for reading Debugging Ruby.If you find any misteaks while reading this book, please email them to [email protected]. If emailsnot your thing, I have a review tool that you can use too. Ill need your email address to add you tothat though. Tweet me (@ryanbigg) or email is cool.If its a problem to do with your code, then please put the code on GitHub and link me to it so I canclone it and attempt to reproduce the problem myself. If you dont understand something, then itsmore likely that Im the idiot and rushed it when I wrote it. Let me know!If you want to see some code examples and compare them to what you have, you can go view themover on GitHub. For now, theyre the absolute latest code from the book.This far into the book and theres already one error. You should expect that it is not alone. It hasfriends and their ways are devious. They are coming after your perception of reality. Beware.

Ive been helping out in the #rubyonrails channel on Freenode now for about 7 years and Ive seen awide variety of skill levels come through. I also help out frequently on Stack Overflow and my dayjob requires doing some of that as well. More often than not though, the visitors of the channel orpeople on Stack Overflow are just lacking the basic skills for debugging their code.The ability to debug code is arguably more important than the ability to write code. Writing code iseasy! Just throw some things together and bash them a couple of times until it works. What couldpossibly go wrong? Anything and everything.Code is not perfect the first time it is written, or even the 10th time it is rewritten. The abilityto debug code written by ourselves, strangers or versions of us from the past that can feel likestrangers sometimes is a valuable ability to have. When things go wrong and theywill go wrong having that ability allows you to dive in without fear into fixing whatever code is causing the faultquickly and efficiently so you can get back to doing more productive things.Debugging can be fun because it provides a technical challenge; a puzzle. Deciphering that puzzleis usually just a matter of walking through the flow of execution within the piece of code which isbreaking, but it is sometimes hard to know where the right place to start is. Once youve found theplace to start, knowing where to go from there can also be difficult. The solution is its own rewardand if youre anything like me you should get a nice little Dopamine hit every time you solve a bug.Or even one of those from just helping out other people by squashing bugs theyve reported.Within this book are common examples things that can go wrong in Ruby and Rails code. Ranginganywhere from a typo in the code, to performance issues and nasty exceptions. This book will covera wide gamut of how code can break, and exactly what we can do about it to fix those breakagesand ensure that we know about them as soon as possible.Lets go!

https://github.com/radar/debug_book_examples

General Debugging TipsThe ability to debug code is arguably more important than the ability to write code. Written code issometimes not completely perfect the first time its written, especially when humans are involved inthe writing of that code. Humans make mistakes, and those mistakes can cost time and money. Beingable to fix those mistakes in an efficient manner is a core competency to being a good programmer.These mistakes are sometimes extremely simple; a typo here, a missing bracket there, defining codeoutside of the correct scope. Other times they involve interconnected bits of code and tracking downthe misbehaving cog (or cogs) in that wonderful machine can take some time.In this book, were going to go through some common coding mistakes that people make and wewill look at how to solve them. The deeper and deeper you get into this book, the more complicatedthe examples will get. All the examples in this book will be written in Ruby.Before we get to see any code, lets talk about getting in the right frame of mind for debugging.

Debugging frame of mindHave you ever come across a bug thats been too hard to fix, only to have the answer come to youafter you distract yourself from the problem for an extended period of time? I most certainly have.I think that just by sitting there for a bit longer that I can magically know the answer. Most of thetime that this is not true. There are many other ways to solve the problem, but I focus on my currentline of thinking and get caught up in a frustrating loop. The loop is thinking that I can solve theproblem, attempting to solve it and then not being able to solve it.Ive solved these kinds of problems by sleeping, having lunch, exercising or by talking people in thecommunity and asking them about the problem. These are all common sense, but its worth coveringthem even if it is for a short while.Sleep has the best effect onmy ability to debug. A good nights sleep is easy to come by as Im a newlymarried man who lives in a first-world country and Im thankful for all of those things. Sleepingwell makes me less grumpy, more rational in my thinking and generally improves my mood andoutlook on things. Debugging on low sleep leads only to frustration.Similarly, eating (and eating healithly) also helps. Having to debug something when my stomach isrumbling is hard work. Staying the course and attempting to fix the problem is not the best courseof action, but I still do it anyway. Grabbing something to eat helps a lot with that. If its some lollies,then the initial sugar hit boosts me up, but then comes the inevitable crash. Eating healthily improvesmy long-term ability to debug well, and so I try to do that as often as I can.

Hence the books subtitle How to find the answer to your problems in a sandwich. Realistically for me though it would be an Indian curry,Thai dish, salad, burger or yiros. Sandwich just has a better ring to it than all those other things.

General Debugging Tips 2

Exercising has been scientifically proven to enhance brain power (and general happiness!), andalong with that comes the ability to reason better about code. Going for a walk, riding a bike or asession at the gym are all ways that I think have boosted my ability to debug code.Lastly, talking to other people is absolutely worthwhile. Ive found the answer to many a problemby just simply explaining whats going wrong to someone else. This technique is called the RubberDucky Technique. You talk to the person, they say nothing (or very little), and suddenly an epiphanyhappens. The answer is known. Its a funny thing.If the answer doesnt come by way of explanation, other people have their own brains to reasonabout things with and their own takes on things. The way that you debug something might bedifferent to the way that they debug it. Getting someone else to look over the code and work withme in reasoning about a problem is, by far, the most productive debugging technique that I knowof.So in summary: make sure youve caught up on your sleep, have eaten well, have exercised well,and dont be afraid to speak to other people in the Ruby community about any issues you have.Lets now dive into some practical examples of debugging in Ruby.

http://www.psychologytoday.com/blog/the-athletes-way/201401/what-is-the-best-way-improve-your-brain-power-lifehttp://www.apa.org/gradpsych/2013/09/exercise.aspxhttp://www.nytimes.com/2012/04/22/magazine/how-exercise-could-lead-to-a-better-brain.html?pagewanted=all

Debugging RubyIn this chapter well cover some basic Ruby code that has some small problems with it, and figureout how to fix it so that the code works once again.

Basic Example #1Our first problem we can cause ourselves in the irb console. Lets start this up now by using irband put this in it:

1 "test".join

When we type this and hit enter, well see a NoMethodError exception:

1 irb(main):001:0> "test".join2 NoMethodError: undefined method `join' for "test":String3 from (irb):14 from .../ruby-2.1.3/bin/irb:11:in `'

This exception shows us that something exceptional happenedwith our code namely a NoMethodError.The text after NoMethodError shows more detail about what happened, and then the lines under-neath it show us how the script got to that point, starting from the bottom up, and thats called astack trace.Why are we getting this error? Well, the "test" object is an object of type String, as is indicated onthe first line of our exception output with the words "test":String. If we look at the documentationfor Rubys String class, we can see that there is indeed no method called join.This error is happening because were calling a method that doesnt exist. In this example weveseen what a stack trace is and how to read it. The example is slightly contrived, but neverthelessdemonstrates how we can find whats happening.Lets take a look at a proper Ruby program which has another, similar bug and talk more about stacktraces.

Basic Example #2Lets take a look at our first very simple Ruby script with an error:

http://www.ruby-doc.org/core-2.1.3/String.html

Debugging Ruby 4

class Carattr_accesssor :onend

This is some fairly simple Ruby code. All were doing is defining a class called Car and then callingthe attr_accesssormethod inside that class which should define a setter method called on= wherewe can store a value, and a getter method called on where we can retrieve that value. We dontnecessarily care about the setting and getting just yet. All we care about is that our code works.Put this code into a new file called car.rb and try running it with ruby car.rb. Ideally, nothingshould be output to the terminal, because we have not told Ruby to output anything. Instead ofnothing, you should see this:

car.rb:2:in `':undefined method `attr_accesssor' for Car:Class (NoMethodError)from car.rb:1:in `'

The output here is modified slightly to fit into the book. The first two lines in the above outputshould be one line. This is another stack trace, just like the one we saw in the earlier example. Theseare output by any Ruby program that encounters an exceptional circumstance and it shows us theexecution flow of the program in reverse order. Lets start from the final line of the stacktrace:

from car.rb:1:in `'

This line tells us that the execution flow of the program started on line 1 of the car.rb file. The here indicates the main namespace that all code within Ruby operates in. This line of thestacktrace is not too important. All its indicating is that on line 1 of our program, something isexecuting and altering what our program is going to do.The first line of the stacktrace begins with this:

car.rb:2:in `'

This line tells us that on line 2 of car.rb, were operating within the scope of a class called Car(). Because this is the top line of the stacktrace, this is probably where our error hasoccurred. In most cases, the top line of the stacktrace will point to the exact line where an erroroccurs.The remainder of this line is this:

undefined method `attr_accesssor' for Car:Class (NoMethodError)

Debugging Ruby 5

This part of it tells us the error thats happened. In this case, were trying to call a method that Rubyhas no knowledge of, the attr_accesssormethod. Were trying to call that method on the Car class,but that method does not exist there. When we try to call a method that Ruby does not know about,it raises a NoMethodError exception, which is shown in parentheses at the end of the line.Our program is 3 lines long, but the third line is just an end, and those are never included instacktraces because theyre just there to demarcate the end of a scope within the code, rather thanperform a function.From this stacktrace, we can gather that the error is happening on line 2. All were doing on line 2is this:

attr_accesssor :on

Do you see the mistake weve made yet? It might be glaringly obvious or it might not be. Look at itfor a moment longer. Do you see it now?The mistake weve made is dead simple: weve typed three ses rather than two. Weve done attr_-accesssor, rather than attr_accessor. The code should be this:

attr_accessor :on

Change the code on line 2 to that now and re-run the program. It should run without showinganything. This is the standard for Ruby programs when they execute successfully. Unless we havespecifically told it to output anything i.e. with a call to the puts method nothing will happen.

Verify program successYou can verify that a program has run successfully by running this command (assumingyoure using Bash or a similar shell):

1 echo $?

If that outputs 0, then everything has worked correctly.TODO: Bash error codes mention goes here.

Now that weve fixed that error, lets cover another.

Basic Example #2Look at this code:

Debugging Ruby 6

class Carattr_accessor :ondef start

on = trueendendcar = Car.newcar.startputs car.on

Can you tell what its doing just by looking at it? It defines a new class called Car, and a virtualattribute inside that class by way of attr_accessor called on. This time, attr_accessor is spelledcorrectly. Remember that the attr_accessor call defines a getter method called on and a settermethod called on=.After the attr_accessor, the code defines a method called startwhich sets on to true. It then endsthe method and class definition.On the final two lines, the code creates a new instance of this class with Car.new and assigns it toa local variable, calls start which should do what its told, and finally outputs the value of car.onusing puts. Just by looking at this code, you may expect that when it is run that it will output true.Youd be wrong.Put this code into car.rb and try running it. This is what will happen:

$ ruby car.rb$

Rather than nothing being output this time, a blank line has been output. This is because the programhas done what weve told it to do. It does not output true, even though the start method is settingon to true. This time we dont have a stacktrace to tell us what line the error is probably on. Wehave to step through the programs execution ourselves.Since there are no exceptions raised within the code, we can assume that the code itself is valid.Lets step through the final three lines of the code:

car = Car.newcar.startputs car.on

The first line here creates a new instance of the Car class. Thats Rubys code, so thats probably notthe source of our frustration. The second line calls the startmethod which sets on to true. The finalline just outputs the value of on, which isnt too special a task. The problem probably lies withinour second line. Lets look at how that method is defined:

Debugging Ruby 7

def starton = trueend

This code inside the method is not dissimilar to the first line of the previous example:

car = Car.new

When we call car = Car.newwere setting a local variable to the value of whatever Car.new returns.In the start method, were setting an on local variable to true. This is not our intention! Ourintention is to set the virtual attribute for the instance to true. Beware this very fine difference!Theres more than one way to set this virtual attribute. We can go through the setter method call,like this:

def startself.on = trueend

The caveat of this is that the on= method may define other behaviour. This is a rare occurrence inRuby code, but it can happen. In this case, its obvious that the on= method isnt redefined to addextra behaviour, so its OK to do this. Another way of doing it would be to set the instance variableof the same name, like this:

def start@on = trueend

Doing it this way has the benefit of the code being shorter and there being no unintended side-effects with the on=method. Using either the setter method or setting the instance variable are bothlegitimate ways of solving the issue with our code. Go ahead and try both ways now. They shouldboth cause the program to output this when its run:

$ ruby car.rbtrue

Great, so weve solved this problem. The issue here was that we were trying to set a local variableinside the method, rather than referencing the virtual attribute setter method or instance variable.With that out of the way, lets look at the possibility of a setter method behaving badly as we talkedabout before. Lets change our code to this:

Debugging Ruby 8

class Carattr_accessor :ondef start

self.on = trueenddef on=(value)

@on = !valueendendcar = Car.newcar.startputs car.on

The startmethod is now using the setter method, on=, to set the value of the virtual attribute calledon. The setter method is overriden from the default here directly underneath that. It takes the valueand negates it, and then stores it on that @on instance variable. This means that when we call on,well get the opposite of what it should be: setting it to true will make it false and vice versa.Try running this code yourself now and youll see:

$ ruby car.rbfalse

The on= method is maliciously negating our value, giving us an unexpected outcome. If we set theinstance variable within the startmethod directly, this would bypass the on=method and we wouldsee the right value. The start method would look like this:

def start@on = trueend

And the output of the program would look like this:

$ ruby car.rbtrue

Its extremely rare that a setter method will perform strange functions like this, but it still isworthwhile knowing that it can happen, just in case we come across a situation like this in ourdebugging experiences.

Debugging RailsRails applications are generally more complex than one or two files. More often than not, youllhave many different pieces of the application all working together: the routes, the controllers, themodels, the views and the helpers. In this chapter, well cover some examples of errors that peoplemake within Rails applications and how to fix them.

WorkflowEach section below works through one particular problem in a Rails application. These Railsapplications are kept on GitHub at https://github.co m/radar/debugging_book_examples and arenumbered the same way as their examples; i.e. Example #1s code is within the directory called 1underneath the rails directory.To work through the examples of these Rails applications, youll need to clone that repository toyour computer:git clone git://github.com/radar/debugging_book_examples

Each Rails application comes fitted with RSpec and Capybara tests which will pass once the codehas been fixed. We can run these tests to execute some code which wil check if the applicationis working. Running automated tests is just an easier way compared with manually viewing theapplication ourselves and saves time in the long run as our applications get more and more complex.Lets begin with the first example now.

Rails Example #1 - form_forIn this example well cover:

The debug helper How to read log output How constant lookup in Ruby works

In this first application example, we have a very basic Rails application. The thing that we want thisapplication to do right now is for the users to be able to go to the New Post page without anyissues, but unfortunately theres a problem thats preventing them from doing that which well seeshortly.Before we do anything with this application, lets install all the dependencies:

https://github.com/radar/debugging_book_examples

Debugging Rails 10

bundle install

Now lets look at the automated test we have inside spec/features/posts_spec.rb inside theapplication, written using RSpec and Capybaras DSL:

require 'spec_helper'feature "Posts" do

it "can begin to create a new post" dovisit root_pathclick_link 'New Post'find_field("Title")find_field("Body")endend

This test will pass if it can navigate to the root page, click New Post and see two fields, one forTitle and one for Body. If we run the test now, well see this error:

1) Posts can begin to create a new postFailure/Error: click_link 'New Post'ActionView::Template::Error:

First argument in form cannot contain nil or be empty# ./app/views/posts/new.html.erb:1:in `...'# ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

This is a typical failure output from RSpec, shown when code raises an exception. The first lineof the message shows us which test is failing. We should have a pretty good idea which test thatis out of the one test that we have so far. The particular exception that were seeing with this testis ActionView::Template::Error, and the message for that exception is First argument in formcannot contain nil or be empty. Well get to that in a moment.The two remaining lines in this output show some of the stacktrace for the error. It shows us thatposts_spec.rb line 6 does something that triggers some code in app/views/posts/new.html.erbto run, and thats where the error is (probably) occurring. According to the stacktrace, thats line 1of app/views/posts/new.html.erb.All this application has inside it is a PostsController which does nothing other than inherit fromApplicationController. This controller is routed to by the two routes within config/routes.rb:

Debugging Rails 11

Bug::Application.routes.draw doroot "posts#index"resources :postsend

We have a view at app/views/posts/index.html.erb which just contains a link to the new action:

When we go to that new_post_path, it will render app/views/posts/new.html.erb which containsthis content:

Writing a new post

So thats all we have: some routes, a controller that does nothing special, a template at app/views/posts/index.html.erband another at app/views/posts/new.html.erb. Just four interconnected pieces in this application,nothing too special.Lets jump back to our test now and see what it said when we ran it with bundle exec rspecspec/features/posts_spec.rb.

1) Posts can begin to create a new postFailure/Error: click_link 'New Post'ActionView::Template::Error:

First argument in form cannot contain nil or be empty# ./app/views/posts/new.html.erb:1:in `...'# ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

The first line of the error is pointing directly to the first line in app/views/posts/new.html.erb:

Debugging Rails 12

All were doing on this line is calling the form_formethod and passing it the @post instance variableand then starting a block. Later on in that file, were calling some methods on the block argument,but that doesnt really matter. The error is fair-and-square happening on this line. The most usefulpart of the error message is the message for the exception:

First argument in form cannot contain nil or be empty

The argument that its talking about here is the @post variable that were passing. Its claiming thatthe variable is nil or empty, and it probably is.

The debug helperWe can see for ourselves if this variable is actually nil by removing all the code in app/views/posts/new.html.erband replacing it with this single line:

The debug method here is provided by Rails and will output a YAMLized version of whatever it ispassed. Were passing to it the output of the @post.inspect call. The inspect method is a methodprovided by Ruby which outputs a human-friendly version of the object. If youve ever written Rubyin irb, youve seen the inspected versions of objects perhaps without even realising it.Start a new irb session now and try entering these things:

1 1.inspect puts 1.inspect "1" "1".inspect puts "1".inspect [1,2,3] [1,2,3].inspect puts [1,2,3].inspect nil nil.inspect puts nil.inspect

Debugging Rails 13

Well see that the non-inspect versions are almost identical to the inspect versions. The inspectversions just have more quotes around them. This is because the inspect call always returns aString object, as those are easiest for humans to read. Whatever the final thing is on the line forsome code entered into IRB is what will be returned in the IRB prompt. In the case of the putscalls, IRB returns nil because puts returns nil. IRB automatically uses inspect to return a human-friendly representation of whatever is entered into the prompt.The debugmethod in our view will not automatically call inspect on whatever it is passed. Instead,it calls another method: to_s. In some cases this method gives back similar output to inspect. Trythe above examples with to_s rather than inspect and see what happens.For everything including nil we see a string representation of that object. For the number 1 we see"1". For the string "1", we see the object again because its already a string. Converting an Arrayto a String gives us "[1,2,3]", which clearly shows us that the object is an array consisting of theelements 1, 2 and 3.Calling nil on the other hand, produces nothing; just an empty string (). Thats because nil isnothing. This whole explanation was to demonstrate that calling this code is not enough:

Due to the debug method calling to_s rather than inspect on the objects it is passed. We must callinspect ourselves:

This way, the debug method receives a String object already and will just output that.Now lets see this in action by firing up a new server process with this application:

rails server

Navigating to http://localhost:3000 will show just the New Post button. We know that the rootaction works because our test is not failing on that line. Clicking New Post now will show us theoutput of the debug call, which will be this:

--- nil...

We can see here that our @post variable is indeed nil, just like the error message said: Firstargument in form cannot contain nil or be empty. Now why is this?We know that the route is working correctly because were currently on the page at /posts/newlooking at the debug message we put there. The route routes to the controller, and the controller is

Debugging Rails 14

empty. After the controller runs, the view template is rendered. Nowhere along the chain is the @postvariable defined for our form_for call in app/views/posts/new.html.erb and that is the cause ofthis bug.Where should we be defining this variable? Not in the view itself, because it is never theresponsibility of the view to collect data. That is the controllers job, and so the code to definethe @post variable should go in the controller. But where in the controller?A clue lies in the server output over in our console.

Reading log outputWhen we made a request to /posts/new, it shows this:

Started GET "/posts/new" for 127.0.0.1 at 2013-11-09 11:29:09 +1100Processing by PostsController#new as HTML

Rendered posts/new.html.erb within layouts/application (0.6ms)Completed 200 OK in 4ms (Views: 3.6ms | ActiveRecord: 0.0ms)

This text has a whole lot of information compressed into a little bit of space. Knowing how to readand interpret logs from Rails is an important skill, so lets go through the details of this now. On thefirst line we have the details about the request:

Started GET "/posts/new" for 127.0.0.1 at 2013-11-09 11:29:09 +1100

This shows us that we have made a GET request to the application, requesting the /posts/new path.The next two bits of information is the IP address of our local computer and the timestamp for therequest.On the second line we have this:

Processing by PostsController#new as HTML

This indicates to us that the route has beenmatched by Rails and has routed to the PostsControllersnew action. The request is a standard HTML format, meaning that HTML output will be returnedby this request.The third line is this:

Rendered posts/new.html.erb within layouts/application (0.6ms)

This tells us that the posts/new.html.erb template was rendered within layouts/application.This means that the controller has automatically chosen to display this template for this action, andhas used the default layout for the application to wrap around that template. All the rendering took0.6ms in this case.The fourth and final line shows this:

Debugging Rails 15

Completed 200 OK in 4ms (Views: 3.6ms | ActiveRecord: 0.0ms)

This line tells us that the response was completed successfully and returned a 200 OK response. Thisis the HTTP status part of the response which indicates to browsers the final status of their request.We can see that the request completed in 4ms total, with the views taking 3.6ms of that time andActiveRecord taking no time at all. The remaining 0.4ms were taken up by unknown things.

Logs are written to filesIf you want to go back and see what happened after youve shut down the server, well beable to do that by viewing logs/development.log, which stores the exact same data thatis displayed when the server is running. The logs directory is where Rails writes its logdata to, and the filename is simply the environment that the Rails application is runningwithin. By default, that environment is development, so the log file that will be writtento will be logs/development.log.

We know that we need to define the @post variable within the controller to make it available to theview, but where exactly? The logs tell us where exactly:

Processing by PostsController#new as HTML

This line from the logs is telling us that the new action within the PostsController is being runbefore the view is rendered. This would be a perfect place to set up the variable, so lets do that nowby defining this code:

app/controllers/posts_controller.rb

1 class PostsController < ApplicationController2 def new3 @post = Post.new4 end5 end

Is this enough to fix our test? Lets find out by running bundle exec rspec spec/features/posts_-spec.rb.

Debugging Rails 16

1) Posts can begin to create a new postFailure/Error: click_link 'New Post'NameError:

uninitialized constant PostsController::Post# ./app/controllers/posts_controller.rb:3:in `new'# ./spec/features/posts_spec.rb:6:in `block (2 levels) in '

Not quite! Were now seeing a new error which is a NameError exception. This time its happeningfrom line 3 of the PostsController, which is this line:

@post = Post.new

The error says uninitialized constant PostsController::Post, but on this line were not looking upthe PostsController::Post constant, we just want Post! So whats happening here? Why does itsay PostsController::Post and not just Post?

Constant lookups in RubyWhen Ruby attempts to look up a constant, it will first attempt to look it up within the currentconstant context. Because were within the PostsController, it will attempt to look it up there. Itwill then travel up the hierarchy looking for that constant until it reaches the top-level namespace.If it cant find it there, then it gives up and shows an error message saying that it couldnt find it inthe current context. We can demonstrate this constant lookup using a very basic Ruby program:

FOO = "foo"module Foo

class Putterdef self.put

puts FOOendendendFoo::Putter.put

With this code, weve defined a FOO constant at the very top level. After that, weve defined a modulecalled Foo and a class called Putter, and that class has a method called put which calls puts FOO.This code will search up the hierarchy, looking for the constant in the Putter class, then the Foomodule and then finally the main namespace.Go ahead and put this code in a new file and try to run it. Well see it outputs foo. The constantlookup is working correctly.Now comment out the FOO constant and try running it again. Well see this happen:

Debugging Rails 17

foo-putter.rb:5:in `put': uninitialized constant Foo::Putter::FOO (NameError)from foo-putter.rb:10:in `'

Ruby is telling us here that it cannot find the constant any more, which is true because wecommented it out! The most important part of this error message is that it cant find the constantwithin the Foo::Putter namespace.Try now uncommenting FOO and moving the constant to inside the module Foo, like this:

module FooFOO = "foo"class Putter

def self.putputs FOOendendendFoo::Putter.put

When we run this code again, well see that it works just the same as if the constant was defined inthe main namespace. The code will work also if the FOO constant is inside the class:

module Fooclass Putter

FOO = "foo"def self.put

puts FOOendendendFoo::Putter.put

This should demonstrate quite well how constant lookup works within Ruby. Lets go back to solvingour new problem, the uninitialized constant PostsController::Post message, armed now with ournew knowledge of constant lookup.

Creating the Post modelWe need to define a Post constant within the application for our test to be happy. The best way todo this would be to generate a new Post model, which we can do with this command:

Debugging Rails 18

rails g model post

Along with this model comes a migration to create the table. If we attempt to view our applicationwithout running this migration, well see this error when we make a request to http://localhost:3000.

Migrations are pending;run 'bin/rake db:migrate RAILS_ENV=development' to resolve this issue.

We can fix this by running the command that it tells us to:

bin/rake db:migrate RAILS_ENV=development

Well see a similar errorwhenwe try to run our automated test, bundle exec rspec spec/features/posts_-spec.rb:

... Migrations are pending;run 'bin/rake db:migrate RAILS_ENV=test' to resolve this issue.

We can run the recommended command here as well to fix the problem.

bin/rake db:migrate RAILS_ENV=test

When we run our test again, well see a different error:

Failure/Error: click_link 'New Post'ActionView::Template::Error:

undefined method `title' for ## ./app/views/posts/new.html.erb:4:...# ./app/views/posts/new.html.erb:1:...# ./spec/features/posts_spec.rb:6:...

Weve now gotten past the error in our controller and now were back to an error withinapp/views/posts/new.html.erb. While line 1 of this file is mentioned in the stacktrace, it is notthe final line and therefore the error is probably not occurring on that line. The very first line of thestacktrace points to line 4, which is this line:

Debugging Rails 19

The error were seeing is happening because somehow title is being called on an instance of thePost class. This is happening because the text_field helper, along with many other form helpersin Rails, will attempt to populate the form with the value from the attribute. It does this by trying tocall the attributes method. If the attributes method is not there, then we see this error happening.Whats happening here is that we didnt define any columns in the posts table, which means thatthere will be no title or body attribute defined for the form to use.We can fix this by altering the migration for the posts table, which is the only migration in thedb/migrate folder. Before we alter it, we need to undo the migration, which we can do by runningthis command:bin/rake db:rollback

Now the database is back to its pristine state, we can alter that migration:class CreatePosts < ActiveRecord::Migration

def changecreate_table :posts do |t|

t.string :titlet.text :bodyt.timestampsendendend

Now that the migration is correct, we can run this command to create the table with the correctcolumns:bin/rake db:migrate

We need to do the same thing with the test database, and that can be done with this command:bin/rake db:test:prepare

Lets run that test again and see if everythings running smoothly:1 example, 0 failures

Yay! Our test is now passing. The router is receiving the request and passing it to the PostsController.The new action in that controller is defining a @post instance variable set to a new instance of the

Post model. The app/views/posts/new.html.erb template is being run and rendering the form.The form is attempting to fetch the attributes from the new Postmodel instance, but since there arenone then the fields will be left blank. All the parts are working in unity and weve debugged thisbug.

Debugging Rails 20

Rails Example #2 - Routing HTTP verbs/methodsIn this example well cover:

Debugging routing issues The rake routes command

This example is very similar to the first example. The changes are that our tests in spec/features/posts_-spec.rb have now changed to this:

require 'spec_helper'feature "Posts" do

it "can create a new post" dovisit "/"click_link "New Post"fill_in "Title", :with => "Hello world"fill_in "Body", :with => "This is the first post."click_button "Create Post"page.should have_content("Created post.")endit "can update an existing post" do

Post.create({:title => "Hello world",:body => "This is the first post."})visit "/"click_link "Hello world"fill_in "Title", :with => "Hello World"click_button "Update Post"page.should have_content("Updated post.")endend

When we run the tests using bundle exec rspec spec/features/posts_spec.rb, well see thiserror:

Debugging Rails 21

1) Posts can update an existing postFailure/Error: click_button "Update Post"ActionController::RoutingError:

No route matches [POST] "/posts/1"# ./spec/features/posts_spec.rb:21 ...

Lets mull over this error while we look through the code in the application.Rather than having a test to ensure that we can see the new post form correctly, the first test now goesthrough all the motions of creating a new post. It does this by going to the root path of the applicationand then clicking New Post. This routes to the new action within the PostsController:

def new@post = Post.newend

This then renders the app/views/posts/new.html.erb view which has changed since the last timewe saw it:

Writing a new post

The form has been moved into a partial so that it can be used within the edit actions template aswell. The partial is located at app/views/posts/_form.html.erb and contains this content:

{ :method => :post } do |f| %>

"btn btn-primary" %>

For the new action, this form will send a request to the create action. For the edit action, we canexpect it to send the data to the update action instead because the record represented by @post ispersisted within the database.When this form is submitted from the new template, it will go to the create action within

PostsController:

Debugging Rails 22

def create@post = Post.create(post_params)flash[:notice] = "Created post."redirect_to root_pathend

This simply creates the post from the data posted in and sets a flash notice. We know that all of thisis working because the first test within spec/features/posts_spec.rb is working. Its the secondtest which is failing. Lets look at that failing test again:

it "can update an existing post" dopost = Post.create({

:title => "Hello world",:body => "This is the first post."})visit "/"click_link post.titlefill_in "Title", :with => "Hello World"click_button "Update Post"page.should have_content("Updated post.")end

This test is failing like this:

1) Posts can update an existing postFailure/Error: click_button "Update Post"ActionController::RoutingError:

No route matches [POST] "/posts/1"# ./spec/features/posts_spec.rb:21 ...

This test first creates a new Post instance. It then visits the root path, which is a listing of posts. Theindex action within PostsController is responsible for collecting all the posts:

def index@posts = Post.allend

The template at app/views/posts/index.html.erb renders all these posts within a table, showinga link with the posts title for each post:

Debugging Rails 23

Posts "btn btn-primary" %>

Title

The test then clicks on the newly created posts title which takes us to the edit action withPostsController:def edit

@post = Post.find(params[:id])end

This action renders the template at app/views/posts/edit.html.erb:Editing

This renders the same form partial that we saw earlier used by the template in app/views/posts/new.html.erb: { :method => :post } do |f| %>


After the edit template has been rendered, the test edits the title of the post to capitalize the wordworld within the title. It then clicks Update Post and thats where our test is failing. The mainpart of the error, one more time:

Debugging Rails 24

ActionController::RoutingError:No route matches [POST] "/posts/1"# ./spec/features/posts_spec.rb:21 ...

The stacktrace that follows this isnt really that helpful because it only shows us the line in the specwhich is triggering this problem and doesnt point to the error in our code so far. We could get abigger stacktrace by running this command:

bundle exec rspec spec/features/posts_spec.rb -b

But this shows us a lot of Rails and Capybara internal code, and its more likely that our code is atfault than Railss or Capybaras. This time, were not being pointed in any particular direction andso we need to walk through the steps by ourselves to figure this out.We know in the test that were about to navigate to the root page, and then from there navigate to theedit page and fill in the Title field. Its the clicking the Update Post button that is failing, claimingthat there is no route defined. We can check the routes defined for the application by running therake routes command which will show us this:

Prefix Verb URI Pattern Controller#Actionroot GET / posts#indexposts GET /posts(.:format) posts#index

POST /posts(.:format) posts#createnew_post GET /posts/new(.:format) posts#newedit_post GET /posts/:id/edit(.:format) posts#editpost GET /posts/:id(.:format) posts#show

PATCH /posts/:id(.:format) posts#updatePUT /posts/:id(.:format) posts#updateDELETE /posts/:id(.:format) posts#destroy

This output shows us all routes which are defined within our small application. We want a routethat goes to the update action. There are two of these within the rake routes output:

PATCH /posts/:id(.:format) posts#updatePUT /posts/:id(.:format) posts#update

When making a request within Rails, we need to be careful that we use the correct HTTPverb/method. The routes defined for the update action use either a PATCH or a PUT request. Fromthe feedback in our test, we can quite clearly see were not doing that:

Debugging Rails 25

ActionController::RoutingError:No route matches [POST] "/posts/1"# ./spec/features/posts_spec.rb:21 ...

Were using POST rather than PATCH or PUT. Sincewe know that we can get to the edit action templatejust fine, lets start our debugging from there. What were looking for is somewhere that is tellingthe code to make a POST request rather than a PUT or PATCH. The app/views/posts/edit.html.erbtemplate contains this code:

Editing

This is fairly standard and nothing is standing out here. Lets look at the partial within app/views/posts/_-form.html.erb:

{ :method => :post } do |f| %>


On the very first line here is where were explictly telling the code to use the POST verb rather thanPUT or PATCH. If we change this to use put or patch, like this:

{ :method => :put } do |f| %>

Or:

{ :method => :patch } do |f| %>

Then run our tests with bundle exec rspec spec/features/posts_spec.rb, well see our first testis now failing with a similar reason:

Debugging Rails 26

1) Posts can create a new postFailure/Error: click_button "Create Post"ActionController::RoutingError:

No route matches [PATCH] "/posts"# ./spec/features/posts_spec.rb:9 ...

Doomed if you do, doomed if you dont. What were failing to realise here at this point is thatform_for automatically takes care of using the right HTTP verb/method. We dont actually needto specify the method. Lets remove that option from the form_for now, leaving that first line likethis:

When we run our tests again, they will now pass:

2 examples, 0 failures

The issue all along was that we were explicitly specifying the HTTP verb/method for the form, butwe didnt need to do that because Rails deals with this automatically. Rails does this by checkingthe persisted? methods return value. This method returns true if the object is represented in thedatabase with a record in the table, or false if not. Lets go into the rails console now and seewhat this does.When we try that method with a new Post object, well get this:

irb(main):001:0> post = Post.new=> #irb(main):002:0> post.persisted?=> false

If we try it with a Post object thats fetched from the database, well get this:

irb(main):001:0> post = Post.create=> #irb(main):002:0> post.persisted?=> true

If the return value of this persisted? method is false, the form_for helper will route to the pluralname of the resource using posts_path in this case. It will also use the POST HTTP verb/method. Ifthe return value of the method is true, then it will route to the singular name of the resource usingpost_path(post), using the PATCH HTTP verb/method.Lets leave the HTTP verb/method deciding up to form_for, unless we absolutely know better.

Debugging Rails 27

Debugging slow codeWithout proper care, applications can get slow over time. Pages that once loaded quickly can degradeinto taking more than a second to load. This can be caused by a slow database lookup, too manydatabase lookups or just even too much information displayed on the page at any particular time.Slowness within an application can almost be called a bug itself. Its definitely not a feature!In this chapter, well cover some common pitfalls that people can inadvertently stumble upon duringRails application development. You might even be surprised at how easy some of the fixes are.

Rails Example #4 - Reducing Repeated QueriesIn this section well cover:

Includes queries Pagination

This example app is a slight modifications on the earlier blogging application. In this application, wehave a Post model which has_many :comments, and that Comment model has_many :users. Beforewe start this application, were going to need to run this command to set up some test data:

rake db:seed

All this task will do is run the code within the db/seeds.rb file within the context of our Railsapplications development environment. This code looks like this:

post = Post.create(:title => "Hello World", :body => "This is the first post.")users = 5.times.map do

User.create(:name => "User#{rand(9999)}")end1000.times do

Comment.create(:post => post,:user => users[rand(5)],:body => "This is just a comment.")end

This code creates a single post, then 5 randomly numbered users, then 1000 comments with the samepost and text, but a randomly chosen user for each comment. Lets look at what this data makes ourapplication do by firing up the Rails server:

Debugging Rails 28

rails s

Going to http://localhost:3000 will show us the familiar list of posts:

Posts

Clicking on this posts link will now show us the post itself, along with a whole bunch of comments:

Posts

Your numbers will probably be different, but thats alright. If we switch over to the terminal sessionwhere rails s is running, well see a quite large amount of queries that are being run to load ourdata:

Processing by PostsController#show as HTMLParameters: {"id"=>"1"}Post Load (0.1ms) SELECT "posts".* FROM "posts" ...Comment Load (2.8ms) SELECT "comments".* FROM "comments" ...User Load (0.2ms) SELECT "users".* FROM "users" ...CACHE (0.0ms) SELECT "users".* FROM "users" ...User Load (0.2ms) SELECT "users".* FROM "users" ...CACHE (0.0ms) SELECT "users".* FROM "users" ...

At the top of this mountain of queries is the query to load the post. This needs to happen, otherwisewe would not be able to show the posts data. The second query is one to load the comments, whichruns for a similar reason: we need to show the comments. The 1000 queries (dont count them) thatfollow are loading all the users for the comments. But why are there 1000 queries when theresonly 5 users to load? Because Rails doesnt know any better. Luckily for us though, Active Recordis caching the results of these queries and fetching the back from the cache when it goes to run thatquery again. These lines in the output begin with CACHE.At the end of the output, well see this:


The entire action is taking 681ms to render, 655ms of that is within the view and 18.4ms is happeningwithin Active Record. A lot of this slowness is due to the number of queries and query cache hitsthat Rails is undergoing during this request.

Interesting trivia fact: The probability of the numbers not being different is 1 in 10000 to the power of 5, or 1 in 100 quintrillion (1 in100,000,000,000,000,000,000).

Debugging Rails 29

When any action within our application runs, we should try and minimize the database queryingneeded for that action. Even a single query takes time, and so ideally we would like to reduce it downto 0 queries. Lets not go all the way to that extreme yet. We should focus our efforts on makingActive Record stop performing 1000 lookups for these users.The way we can fix this problem is with something in Active Record called eager loading. Eagerloading will load the required data in as few queries as possible. We can use this within the showaction of the PostsController like this:

def show@post = Post.includes(comments: :user).find(params[:id])end

The includes statement here triggers eager loading for this query. The Hash object passed as theargument to this tells Active Record to eager load the comments association from the Postmodel, aswell as the user association for each comment.When we reload the page in our browser and look at the queries again, theres no longer over athousand of them:

Started GET "/posts/1" for 127.0.0.1 at 2013-11-24 11:52:14 +1100Processing by PostsController#show as HTML

Parameters: {"id"=>"1"}... SELECT "posts".* FROM "posts"

WHERE "posts"."id" = ? LIMIT 1 [["id", "1"]]... SELECT "comments".* FROM "comments"

WHERE "comments"."post_id" IN (1)... SELECT "users".* FROM "users"

WHERE "users"."id" IN (1, 3, 2, 5, 4)

There are now only 3 queries for the data that needs to be displayed on this page: one for the post,one for the comments and one for all the users for all the comments. The eager loading here hasaltered the query for the comments to load it using an IN query, rather than the query it ran before:

... SELECT "comments".* FROM "comments"WHERE "comments"."post_id" = ? [["post_id", 1]]

This doesnt make too much of a different in the query speeds. The two queries are going to producethe same result. The major difference here is that the 1000 queries for the users for the commentshas now been reduced to one simple query:

Debugging Rails 30

... SELECT "users".* FROM "users"WHERE "users"."id" IN (1, 3, 2, 5, 4)

We can also see that the page which once took around 700ms to load (650ms in the view, 25ms inActive Record), is now loading much, much faster:


The page is now loading seven times faster with this one easy trick. Thats great!

TODO Pagination Fragment caching (?) Mention the Bullet gem

Handling ExceptionsNot all exceptions known to humankind have been documented in this book so far. There will bemore than these which pop up in the applications that you will write. By default, when users causean exception to happen within a Rails application, this is what theyll see:

500 Error Page

Exceptions will happen in your applications, because nobody writes perfect code. Sometimes theseexceptions might happen under a set of very, very specific circumstances and only for one user.Unless that user contacts you specifically, you may never know that this exception is happening.Thats where exception notification services come into play. These range from the basic exception_-notification gem (http://smartinez87.github.io/exception_notification/) which you can install andconfigure to email you whenever an exception happens, all the way up to the large scale New Relicwhich does much much more than exception notifications. There are others too, such as Airbrake(http://airbrake.io) and Honeybadger (honeybadger.io).

Advanced Rails DebuggingAdvanced Rails Example #1 - Broken DeviseApplicationThis example was from a debugging session I did with Steven Baker on November 25th, 2013.In this example well cover:

bundle show Debugging with the pry gem How to find where methods are defined

Not all applications that we debug are going to be on the latest and greatest versions of everything.Sometimes well be asked to debug code that is years old and more often than not written bysomebody else. Unfamiliarity with an older version of Rails or another gem may seem like a hugehurdle at first, but on closer inspection that hurdle is a lot smaller than it seems.Debugging sometimes requires a large amount of patience and concentration, as this (almost 5,000word) example shows. The pay off at the end of such a long debugging session is worth it, in myopinion. I get a little kick out of it. Lets start debugging.Here in this application, we are having a problem where (imaginary) users are unable to sign in. The(imaginary) user is reportedly seeing a 500 error page when they try to sign in, which is indicative ofa bigger problem. Thankfully another (imaginary) developer within this aplication has written a testto cover this breakage using RSpec + Capybara. This test lives at spec/features/sign_in_spec.rb:require 'spec_helper'describe "sign in" do

before doUser.create!(:email => "[email protected]", :password => "password")endit "can sign in successfully" do

visit "/"click_link "Sign in"fill_in "Email", :with => "[email protected]"fill_in "Password", :with => "password"click_button "Sign in"page.should have_content("Signed in successfully.")endend

Advanced Rails Debugging 33

When we run this test using bundle exec rspec spec/features/sign_in_spec.rb, well see thatit is indeed breaking:

1) sign in can sign in successfullyFailure/Error: click_button "Sign in"ActionController::RoutingError:

uninitialized constant SessionsController# ./spec/features/sign_in_spec.rb:13...

It says uninitialized constant SessionsController, but Devise should be handling the authentication,shouldnt it? Devise has a controller within its own code called Devise::SessionsControllerwhich should be handling this request, but for some reason it is not. Its trying to use theSessionsController without the Devise namespace.Running the test with bundle exec rspec spec/features/sign_in_spec.rb -b yields no betterinformation, showing us only stacktrace from within Rails or Rack.Lets investigate this problem ourselves by starting up this application now:

rails server

If we now go to http://localhost:3000 and click Sign in, well see the standardDevise form consistingof an email and password field.

Sign in form

If we attempt to sign in with [email protected] and password, it will fail and show the sameerror that we saw from our tests:


Routing error for SessionsController

Why is this happening? The path for this request looksweird. It shows as localhost:3000/session.user.Where is that coming from? The best first place to look would be back on the sign in page itself tosee if the form tag is sending us there. If we go back to that page, right click on an element of theform and click Inspect Element, well see that the forms HTML makeup begins with this:

This explains howwere being taken to /session.user, but not thewhy. Lets investigate howDevisegenerates that form now. The template that displays this form lives inside the Devise gem rather thanour application. To get to the template, were going to need to open the current version of Devisethat the application is using. To get there, were going to need to find where Devise is installed onour system. Lucky for us, theres a command to do that:

bundle show devise

This command will show the path to the gem that the applications Gemfile will use when it sourcesits dependencies. It will show something like this:

/Users/user/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/devise-1.1.9

We can open this path in our editor by running this command:

$EDITOR `bundle show devise`

If that doesnt work, then you will need to open the path manually in your editor.Now that we have Devise open, we need to figure out what template is being shown that rendersthis form. We can find that out by looking at the log output from our rails server session when arequest to the sign in page is made:


Started GET "/users/sign_in" for 127.0.0.1 ...Processing by Devise::SessionsController#new as HTMLRendered .../devise-1.1.9/app/views/devise/shared/_links.erb (0.9ms)Rendered .../devise-1.1.9/app/views/devise/sessions/new.html.erb

within layouts/application (5.6ms)

In this output, we can see that were making a request to /users/sign_in and that request is beingrouted to the new action within Devise::SessionsController. This action then goes on to rendera couple of templates, namely devise/shared/_links.erb and devise/sessions/new.html.erb.The code for the form is probably going to be in the sessions/new.html.erb template. Inside thattemplate well see this code to generate the form:

resource_name,:url => session_path(resource_name)) do |f| %>

For the URL of the form, this form_for call is calling the session_path helper, which is probablya routing helper. We can find out where this is defined within our application by running thiscommand:

rake routes

This command will show this output for this application:

root /(.:format) ...new_user_session GET /users/sign_in(.:format) ...

user_session POST /users/sign_in(.:format) ...destroy_user_session GET /users/sign_out(.:format) ...

user_password POST /users/password(.:format) ...new_user_password GET /users/password/new(.:format) ...edit_user_password GET /users/password/edit(.:format) ...

PUT /users/password(.:format) ...user_registration POST /users(.:format) ...new_user_registration GET /users/sign_up(.:format) ...edit_user_registration GET /users/edit(.:format) ...

PUT /users(.:format) ...DELETE /users(.:format) ...

session POST /session(.:format) ...new_session GET /session/new(.:format) ...edit_session GET /session/edit(.:format) ...GET /session(.:format) ...PUT /session(.:format) ...DELETE /session(.:format) ...


On the left-hand side of this output, we can see the path helpers defined for the routes if they arepresent. We can see that there is one line in particular that defines a session helper:session POST /session(.:format)

{:action=>"create", :controller=>"sessions"}

This is the helper that is being used by Devise to generate the route for the form incorrectly. Whereis this coming from though? Lets look in config/routes.rb and see if anything there sticks out:Bug::Application.routes.draw do

root :to => "welcome#index"devise_for :usersresource :sessionend

At the very bottom of this file there is a resource :session line, which is what is definingthe session_path helper. Since there is no SessionsController for this route to use within theapplication, it probably got left over in this application from a previous developer. Lets remove thisline from config/routes.rb now:resource :session

For this route change to take full-effect, well need to restart our rails server session so that thesession_path helper is completely forgotten about. Once weve restarted rails server, we can tryto sign in again with [email protected] and password. This time well see a completely differenterror message:

Invalid Email or Password


Ok, a different error message is progress because it means something new is happening! First thingto check: is there actually a user with the email address [email protected] within the database?The easiest way to check that would be to launch a new console session:

rails c

Within this console session, we want to find if a user with the email [email protected] exists,which we can do like this:

User.exists?(:email => "[email protected]")

When we run this code, well see that it doesnt exist:

irb(main):001:0> User.exists?(:email => "[email protected]")=> false

This is happening because the only place where we have been creating this user is within thetest environment, and not the development environment. These two environments use separatedatabases and after a test is run within the test environment, that database is emptied anyway.Therefore this problem may be caused by a simple case of a missing user. Lets create this new userin the database now:

User.create!(:email => "[email protected]", :password => "password")

With this user created, we can try to sign in again.

Invalid Email or Password


Unfortunately, that did not fix the problem yet. The error message is still telling us invalid email andpassword, although were pretty certain that its the right email address and the right password. Wecan double-check this by first finding the user in the console:

>> user = User.find_by_email("[email protected]")=> #

With this, we can see that the user does actually exist. Now lets check to see if the password is validusing Devises valid_password? method:

>> user.valid_password?("password")=> true

The user definitely exists and their password is definitely valid, even though the error message istelling us that this is not true. The error message is lying to us. Lets take a look at the logs whichmay give us more leads as to where to look to figure this one out.Heres what happening for this request:

Started POST "/users/sign_in" for 127.0.0.1 at 2013-11-27 18:03:40 +1100Processing by Devise::SessionsController#create as HTMLParameters: ...Completed in 4ms

Processing by Devise::SessionsController#new as HTMLParameters: ...Rendered .../app/views/devise/shared/_links.erb (0.8ms)Rendered .../app/views/devise/sessions/new.html.erb within layouts/application (\5.0ms)Completed 200 OK in 97ms ...

We can see here that Rails is receiving the request from the sign in form and that the request is beinghandled by the create action within Devise::SessionsController. Somehow, the action does notrespond in any particular fashion, indicated with the Completed in 4ms message. Typically wewould expect to see a 200 OK or 302 Redirected message there, but were not seeing that. Afterthis action complese, the new action within the same controller gets the same parameters and thenre-renders the app/views/devise/sessions/new.html.erb template.To begin to discoverwhats happening here, lets check out the codewithin Devise::SessionsController.If the gem is not open any more, re-open it using this command:

$EDITOR `bundle show devise`

Within Devise, this controller is located at app/controllers/devise/sessions_controller.rb andits create action looks like this:


devise/app/controllers/devise/sessions_controller.rb1 def create2 resource = warden.authenticate!(:scope => resource_name, :recall => "new")3 set_flash_message :notice, :signed_in4 sign_in_and_redirect(resource_name, resource)5 end

The create action first calls out to the wardenmethod which is provided by the Warden gem whichDevise depends on. The warden method is a proxy object which keeps track of the current sessionstate. We can find out where the authenticate! method for this proxy object is defined by firstlyadding the pry gem to our Gemfile:

gem 'pry'

We can install this gem using bundle install. This gem provides us with some useful debuggingtools which well see used as we go along. The first of these is the breakpoint helper calledbinding.pry. This stops code execution right in its tracks and will bring up an IRB-esque promptwhich we can then use to debug code. Were going to use this within the create action in

Devise::SessionsController like this:devise/app/controllers/devise/sessions_controller.rb1 def create2 binding.pry3 resource = warden.authenticate!(:scope => resource_name, :recall => "new")4 set_flash_message :notice, :signed_in5 sign_in_and_redirect(resource_name, resource)6 end

To ensure that our server has this latest and greatest code, well need to restart it. Once its beenrestarted, if we try signing in again the browser should hang and then in the server window wellsee the Pry prompt:

12: def create=> 13: binding.pry14: resource = warden.authenticate!(:scope => resource_name, :recall => "new")15: set_flash_message :notice, :signed_in16: sign_in_and_redirect(resource_name, resource)17: end

We can treat this prompt just like any other Ruby prompt. We can call whatever Ruby code we wishwithin it. Right now though, all we want to know is where the authenticate! method is defined.Lets find that out now by using this code:


warden.method(:authenticate!).source_location

The method method returns a Method object that represents the method. Calling source_locationon that object tells us this:

=> [".../gems/warden-1.0.6/lib/warden/proxy.rb", 112]

This indicates to us that the method is defined within the Warden gem, and so we should go lookthere to see whats happening. Now that we know where that method is defined, we no longer needthe binding.pry within Devise::SesionsController. We should remove this call now before wecontinue.Lets open the warden gem using this command in our terminal:

$EDITOR `bundle show warden`

Once weve got the gem open, well then open lib/warden/proxy.rb and navigate to line 112 withinthat file. The authenticate! method will be here:

warden/lib/warden/proxy.rb:112

1 def authenticate!(*args)2 user, opts = _perform_authentication(*args)3 throw(:warden, opts) unless user4 user5 end

This method takes any number of arguments and stashes them in the args variable. These argumentsare from Devise::SessionsControllers create action:

resource = warden.authenticate!(:scope => resource_name, :recall => "new")

What resource_name is isnt too important right now. What is important is tracking down why thismethod is sending us off to the new action in Devise::SessionsController rather than processingour request properly.It would seem here that the _perform_authenticationmethod returns two things: a user and a setof options. If our authentication is failing, then it would make sense that it would probably be failingwithin this _perform_authentication method. A quick search for this method within this file willshow us where it is: down on line 272.



1 def _perform_authentication(*args)2 scope, opts = _retrieve_scope_and_opts(args)3 user = nil4 # Look for an existing user in the session for this scope.5 # If there was no user in the session. See if we can get one from the request.6 return user, opts if user = user(scope)7 _run_strategies_for(scope, args)8 if winning_strategy && winning_strategy.user9 opts[:store] = opts.fetch(:store, winning_strategy.store?)10 set_user(winning_strategy.user, opts.merge!(:event => :authentication))11 end12 [@users[scope], opts]13 end

The first thing this method does is call out the _retrieve_scope_and_opts method, passing it thesame args that this method takes. This method is the next method defined in this file:


1 def _retrieve_scope_and_opts(args) #:nodoc:2 opts = args.last.is_a?(Hash) ? args.pop : {}3 scope = opts[:scope] || @config.default_scope4 opts = (@config[:scope_defaults][scope] || {}).merge(opts)5 [scope, opts]6 end

We can see what this method returns by altering _perform_authentication to have a binding.pryright after the _retrieve_scope_and_opts call:


1 def _perform_authentication(*args)2 scope, opts = _retrieve_scope_and_opts(args)3 binding.pry

To trigger this breakpoint, well need to restart the server so that the code for all the gems, includingWarden, is reloaded. After that, well try signing in again which will bring up a new Pry prompt:


1 272: def _perform_authentication(*args)2 273: scope, opts = _retrieve_scope_and_opts(args)3 => 274: binding.pry4 275: user = nil

We can inspect the values for scope and opts easily in the prompt:

[1] pry(#)> scope=> :user[2] pry(#)> opts=> {:scope=>:user, :recall=>"new"}

Now that we knowwhat the value of scope and opts is, we can go through the rest of the _perform_-authentication methods flow. Lets make sure to remove this binding.pry first, now that wevefound the information we wanted.The next part of the _perform_authentication method is this:


1 # Look for an existing user in the session for this scope.2 # If there was no user in the session. See if we can get one from the request.3 return user, opts if user = user(scope)

This part of the code is pretty well-documented. It attempts to fetch a user from the session, butthere probably isnt one because were attempting to sign in as a user now. If there was a user, thenthis method would return the user and the options at this point.There isnt a user, and therefore its going to fall to the next code within that method:


1 _run_strategies_for(scope, args)2 if winning_strategy && winning_strategy.user3 opts[:store] = opts.fetch(:store, winning_strategy.store?)4 set_user(winning_strategy.user, opts.merge!(:event => :authentication))5 end

The _run_strategies_for method is called, passing in the scope and args arguments. Lets seewhat this method does:



1 # Run the strategies for a given scope2 def _run_strategies_for(scope, args) #:nodoc:3 self.winning_strategy = @winning_strategies[scope]4 return if winning_strategy && winning_strategy.halted?5 if args.empty?6 defaults = @config[:default_strategies]7 strategies = defaults[scope] || defaults[:_all]8 end9 (strategies || args).each do |name|10 strategy = _fetch_strategy(name, scope)11 next unless strategy && !strategy.performed? && strategy.valid?12 self.winning_strategy = @winning_strategies[scope] = strategy13 strategy._run!14 break if strategy.halted?15 end16 end

Theres quite a lot going on inside this method. At the top, it sets the winning_strategy virtualattribute (defined on line 9 of this file), to whatever the output of @winning_strategies[scope] is.If @winning_strategies[scope] is not nil and is halted?, then the method stops executing and_perform_authentication continues.Is @winning_strategies[scope] returning nil? Lets use Pry to find out by sticking a binding.pryas the first line in that method:


1 def _run_strategies_for(scope, args) #:nodoc:2 binding.pry3 self.winning_strategy = @winning_strategies[scope]

We will need to restart our rails server process here for the changes to take effect. Lets try tosign in again. This will bring up the Pry prompt again:

298: def _run_strategies_for(scope, args) #:nodoc:=> 299: binding.pry300: self.winning_strategy = @winning_strategies[scope]

With the Pry prompt, we can see what @winning_strategies[scope] is:


[1] pry(#)> @winning_strategies[scope]=> nil

Since this call returns nil, the method will continue executing past the line that checks for theobjects presence. Lets keep reading this method.


1 if args.empty?2 defaults = @config[:default_strategies]3 strategies = defaults[scope] || defaults[:_all]4 end

This part of the method is just setting up some defaults for the strategies to use within this method,unless some arguments explicitly telling us what strategies to use are passed in.With our Pry promptstill open, we can see if args is empty by simply asking:

[2] pry(#)> args=> []

Since args is empty, this will set defaults to the return value of @config[:default_strategies].Whats that? Lets ask Pry again:

[3] pry(#)> @config[:default_strategies]=> {:user=>[:rememberable, :database_authenticatable]}

This method returns the default configured strategies from Devise, namely rememberable anddatabase_authenticatable. We know that scope is :user, and therefore we can work out fromthis that strategies is going to be set to [:rememberable, :database_authenticatable]. We cansee this in practice if we run that code in Pry:

[4] pry(#)> defaults = @config[:default_strategies]=> {:user=>[:rememberable, :database_authenticatable]}[5] pry(#)> strategies = defaults[scope] || defaults[:_all]=> [:rememberable, :database_authenticatable]

Now that we know what that part of the method does, lets remove that binding.pry and move onto the next couple of lines:



1 (strategies || args).each do |name|2 strategy = _fetch_strategy(name, scope)3 next unless strategy && !strategy.performed? && strategy.valid?

This method iterates through either strategies or args. We know that args is empty, and becauseof this the strategies variable is being set, therefore this will iterate through strategies. For eachstrategy, the _run_strategies_for method calls _fetch_strategy and passes it the name of thestrategy, as well as the scope variable.Lets see what this _fetch_strategy method does now; its the last method within this file.


1 # Fetchs strategies and keep them in a hash cache.2 def _fetch_strategy(name, scope)3 @strategies[scope][name] ||= if klass = Warden::Strategies[name]4 klass.new(@env, scope)5 elsif @config.silence_missing_strategies?6 nil7 else8 raise "Invalid strategy #{name}"9 end10 end

Lets stick a binding.pry at the top of this method now, so that we can investigate what its doing:


1 # Fetchs strategies and keep them in a hash cache.2 def _fetch_strategy(name, scope)3 binding.pry

Restarting the server and attempting to sign in again will bring up a new Pry prompt:

319: def _fetch_strategy(name, scope)=> 320: binding.pry321: @strategies[scope][name] ||= if klass = Warden::Strategies[name]

The first part of this code that will execute is the Warden::Strategies[name] part. Lets see whatthat returns now:


[1] pry(#)> Warden::Strategies[name]=> Devise::Strategies::Rememberable

This call is going to return Devise::Strategies::Rememberable, so that means the if part of themethod will be executed:


1 @strategies[scope][name] ||= if klass = Warden::Strategies[name]2 klass.new(@env, scope)

This code calls the initializemethod located within Devise::Strategies::Rememberable, passs-ing it the @env object (which is the current requests environment), as well as the scope. This callreturns a new object of the Devise::Strategies::Rememberable class, which is then sent back tothe _run_strategies_for method.Now that the _run_strategies_for method actually has a strategy to run, it does that with thisline:


1 next unless strategy && !strategy.performed? && strategy.valid?

We know that strategy here is going to be an instance of the Devise::Strategies::Rememberableclass, and thats definitely not nil, so the first check here completes. We dont know however whatperformed? or valid? return.To find that out, lets remove the binding.pry from _fetch_strategy and add a new one directlyabove line 308 in warden/proxy.rb:


1 binding.pry2 next unless strategy && !strategy.performed? && strategy.valid?

Restarting the server and attempting a sign in again will bring up the Pry prompt at the new location:

307: (strategies || args).each do |name|308: strategy = _fetch_strategy(name, scope)

=> 309: binding.pry

With this prompt, we can now check to see what performed? and valid? return.


[1] pry(#)> strategy.performed?=> false[2] pry(#)> strategy.valid?=> false

It makes sense here that performed? is returning false, since this is the first time (as far as we haveseen) that the strategy is being fetched and checked. The reason why valid? is returning false isunclear. So lets investigate where thats coming from so that we can rule it out as a cause of ourproblem.We can find out where the valid? method is defined by using source_location:

>> strategy.method(:valid?).source_location=> ["...gems/devise-1.1.9/lib/devise/strategies/rememberable.rb", 11]

Lets go back to the open Devise project and find that file. On line 11, it has this code:

devise/lib/strategies/rememberable.rb:11

1 def valid?2 remember_cookie.present?3 end

Where is remember_cookie? Its a little further down, near the bottom of the file:


1 def remember_cookie2 @remember_cookie ||= cookies.signed[remember_key]3 end

This method calls cookies.signed[remember_key]. What is remember_key? Its defined a littlefurther up:


1 def remember_key2 "remember_#{scope}_token"3 end

The scope variable here is going to be user, so therefore remember_key is going to be remember_-user_token. Does remember_cookie return anything? Lets check back in our Pry console:


[4] pry(#)> strategy.send(:remember_cookie)=> nil

So no, this doesnt return anything. Therefore this particular strategy is not valid. This means thatWarden will now move onto the next strategy. Lets type exit to allow Wardens code to continueto run. It will stop on the next strategy. We can find out that strategys class by running this:

[1] pry(#)> strategy.class=> Devise::Strategies::DatabaseAuthenticatable

This strategy is used by Devise to authenticate user sessions using the database. It finds a user withan email address and checks that their password is valid. This sounds like exactly the right place tobe looking to find out whats going wrong. Lets look at this very closely.We can do the same checks as we did with the previous strategy. The strategy does exist, becausewe were able to find out the class of it, and that class wasnt NilClass. Next, lets see if this strategyhasnt been performed yet:

[2] pry(#)> strategy.performed?=> false

Ok, so thats the same as the first strategy. Now lets see if this strategy is valid:

[3] pry(#)> strategy.valid?=> false

This strategy isnt valid either. Again, theres not much information to go on and so lets investigatewhat that valid? method is doing.

[4] pry(#)> strategy.method(:valid?).source_location=> [".../gems/devise-1.1.9/lib/devise/strategies/authenticatable.rb", 11]

The valid? method in this strategy is defined like this:

devise/lib/strategies/authenticatable.rb:11

1 def valid?2 valid_for_http_auth? || valid_for_params_auth?3 end

What do each of these methods return? Lets see in our Pry console:


[5] pry(#)> strategy.send(:valid_for_http_auth?)=> false[6] pry(#)> strategy.send(:valid_for_params_auth?)=> false

We should investigate the valid_for_http_auth? method first. Its defined like this:


1 def valid_for_http_auth?2 http_authenticatable? &&3 request.authorization &&4 with_authentication_hash(http_auth_hash)5 end

Lets see what the first method is doing here:

[7] pry(#)> strategy.send(:http_authenticatable?)=> false

The http_authenticatable?method returns false for this strategy, which means that valid_for_-http_auth? is going to return false as well. Therefore valid_for_http_auth? is a dead-end. Letslook at valid_for_params_auth?:


1 def valid_for_params_auth?2 params_authenticatable? && valid_request? &&3 valid_params? && with_authentication_hash(params_auth_hash)4 end

Lets see what these methods do.


[8] pry(#)> strategy.send(:params_authenticatable?)=> true[9] pry(#)> strategy.send(:valid_request?)=> true[10] pry(#)> strategy.send(:valid_params?)=> true[11] pry(#)>

strategy.send(:with_authentication_hash, strategy.send(:params_auth_hash))=> false

Its that final method, with_authentication_hash, that is failing. Lets look at what this methoddoes:devise/lib/strategies/authenticatable.rb:1041 # Sets the authentication hash and the password from params_auth_hash or http_au\2 th_hash.3 def with_authentication_hash(hash)4 self.authentication_hash = hash.slice(*authentication_keys)5 self.password = hash[:password]6 authentication_keys.all?{ |k| authentication_hash[k].present? }7 end

This method takes a hash, namely the one defined by params_auth_hash:devise/lib/strategies/authenticatable.rb:621 # Extract the appropriate subhash for authentication from params.2 def params_auth_hash3 params[scope]4 end

What do the params look like?

{"utf8"=>"","authenticity_token"=>"8Lreg3wifRd81HlP0Spo3oQSNrgOhq5odrEAekHqnYQ=","user"=>

{"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"},"commit"=>"Sign in","action"=>"create","controller"=>"devise/sessions"}

We know from before that scope is user, so the only parameters used in with_authentication_-hash are params[:user]:


{"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}

Now that we know what hash with_authentication_hash has, lets see what the method does next:


1 self.authentication_hash = hash.slice(*authentication_keys)

The hash gets sliced using authentication_keys. What is authentication_keys?

[12] pry(#)> strategy.send(:authentication_keys)=> [:login]

The authentication_keysmethod returns a 1-element array with the symbol :login inside it. Thismeans that when hash.slice is called within with_authentication_hash, it will pick out any keycalled :login from our params[:user] hash. Is there any key with that name?

{"email"=>"[email protected]", "password"=>"password", "remember_me"=>"0"}

No, there isnt! We only have the keys of "email", "password", and "remember_me". It would seemstrange that authentication_keys is returning :login when our hash doesnt contain that key atall. This is most likely where our bug is happening. If we look at the authentication_keysmethod,we can find out where this is coming from:


1 # Holds the authentication keys.2 def authentication_keys3 @authentication_keys ||= mapping.to.authentication_keys4 end

Lets see where that authentication_keys method is coming from:

[13] pry(#)>strategy.mapping.to.method(:authentication_keys).source_location=> ["/Users/ryanbigg/.gem/ruby/2.1.3/gems/devise-1.1.9/lib/devise/models.rb", 22]

The method is defined like this:


devise/lib/devise/models.rb:221 def self.config(mod, *accessors) #:nodoc:2 accessors.each do |accessor|3 mod.class_eval


[1] pry(User)> @authentication_keys=> nil

Nope, thats not defined at all. Lets move on to the next statement:

[2] pry(User)> superclass.respond_to?(:authentication_keys)=> false

Thats also not defined. Therefore it must be in Devise.authentication_keys:

[3] pry(User)> Devise.authentication_keys=> [:login]

Yes! Thats where this value is coming from. Now that we know we can remove the binding.prythat we just added.Since this method is defined on the Devise class itself, we can expect it to be defined inlib/devise.rbwithin Devise, because thats where the Devisemodule itself is defined. Lets checkin that file for authentication_keys. Well come across this code:

lib/devise.rb:64

1 # Keys used when authenticating an user.2 mattr_accessor :authentication_keys3 @@authentication_keys = [ :email ]

If we look through this whole file, it looks like the mattr_accessor methods are defining (quite alot of) configuration for Devise. It would seem that our authentication_keys value of [:login]doesnt match the default value, specified here in lib/devise.rb, as [:email]. Something in ourcode is changing this. Where could that place be?Well, we have one big block of Devise configuration in our application at config/initializers/devise.rb.On line 23 of that file, we have this:

config/initializers/devise.rb:23

1 config.authentication_keys = [:login]

Methods defined on classes and modules in Ruby are typically found in the same files as those classes and modules; its just really best practiceto do so.


The culprit has been here all along! Lets try commenting out that line and see what happens whenwe restart our application and sign in once again.

Signed in successfully

Great! So now weve been able to sign in successfully. Why was the authentication_keyssetting failing us in the first place though? It was Devises with_authentication_hash method inDevise::Strategies::Authenticatable which was returning false:

devise/lib/strategies/authenticatabl

Documents

Debugging Ruby