Effectively Testing Services

Neal Kemp

$ whoami Iowa native

Now: Californian

Software Developer

Ruby, Rails, Javascript, etc

what, why & how of testing services

NOT Building testable services

NOT Test-driven development

(necessarily)  

… and because I don’t want @dhh to rage

what

What is a service?

Internal “SOA”

Any time you make an HTTP request to an endpoint in another repository

why

Why are services important? Build faster

Makes scaling easier

Use them on virtually every application

Increasingly prevalent

Services are critical to modern Rails development

Why is testing services important? You (should) test everything else

Services compose crucial features

You may encounter problems…

Internal API Sometimes null responses

Inconsistencies

Catastrophe

Okay? But what about external APIs?

{"id": 24}{"code": "ANA"}

"goals":[ { "per":"1", "ta":"CGY", "et":"14:11", "st":"Wrist Shot" }, { "per":"2", "ta":"ANA", "et":"11:12", "st":"Backhand" } ]

"goals": { "per":"1", "ta":"CGY", "et":"14:11", "st":"Wrist Shot" }

No versioning!

Snapchat Client Haphazard documentation

What are the requests?

Bizarre obfuscation

github.com/nneal/snapcat

how

What is different about services? External network requests

You don’t own the code

On an airplane…

Failure is bad!

No network requests

Don’t interact with services from test environment* **

* Includes “dummy” APIs

** Using pre-recorded responses is okay

Assuming: Rails, rspec

Time to stub!

Built-in Stubbing Typhoeus

Faraday

Excon

Simplify.

gem 'webmock'

ENV['RAILS_ENV'] ||= 'test'require File.expand_path('../../config/environment', __FILE__)require 'rspec/autorun'require 'rspec/rails’Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)RSpec.configure do |config| config.infer_base_class_for_anonymous_controllers = false config.order = 'random’endWebMock.disable_net_connect!

spec/spec_helper.rb

module FacebookWrapper def self.user_id(username) user_data(username)['id'] end def self.user_data(username) JSON.parse( open("https://graph.facebook.com/#{username}").read ) endend

lib/facebook_wrapper.rb

require 'facebook_wrapper'

config/intializers/facebook_wrapper.rb

require 'spec_helper'describe FacebookWrapper, '.user_link' do it 'retrieves user link' do stub_request(:get, 'https://graph.facebook.com/arjun'). to_return( status: 200, headers: {}, body: '{ "id": "7901103","first_name": "Arjun", "locale": "en_US","username": "Arjun" }' ) user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' endend

spec/lib/facebook_wrapper_spec.rb

require 'spec_helper'describe FacebookWrapper, '.user_link' do it 'retrieves user link' do stub_request(:get, 'https://graph.facebook.com/arjun'). to_return( status: 200, headers: {}, body: '{ "id": "7901103","first_name": "Arjun", "locale": "en_US","username": "Arjun" }' ) user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' endend

spec/lib/facebook_wrapper_spec.rb

require 'spec_helper'describe FacebookWrapper, '.user_link' do it 'retrieves user link' do stub_request(:get, 'https://graph.facebook.com/arjun'). to_return( status: 200, headers: {}, body: '{ "id": "7901103","first_name": "Arjun", "locale": "en_US","username": "Arjun" }' ) user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' endend

spec/lib/facebook_wrapper_spec.rb

require 'spec_helper'describe FacebookWrapper, '.user_link' do it 'retrieves user link' do stub_request(:get, 'https://graph.facebook.com/arjun'). to_return( status: 200, headers: {}, body: '{ "id": "7901103","first_name": "Arjun", "locale": "en_US","username": "Arjun" }' ) user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' endend

spec/lib/facebook_wrapper_spec.rb

Even Better No network requests

Fast!

No intermittent failure

Mock-Services AWS

FB graph mock

OmniAuth

Etc…

gem 'fb_graph-mock'

ENV['RAILS_ENV'] ||= 'test'require File.expand_path('../../config/environment', __FILE__)require 'rspec/autorun'require 'rspec/rails’require 'fb_graph/mock' Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)RSpec.configure do |config| config.infer_base_class_for_anonymous_controllers = false config.order = 'random' config.include FbGraph::MockendWebMock.disable_net_connect!

spec/spec_helper.rb

describe FacebookWrapper, '.user_link' do it 'retrieves user link' do mock_graph :get, 'arjun', 'users/arjun_public' do user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' end endend

spec/lib/facebook_wrapper_spec.rb

describe FacebookWrapper, '.user_link' do it 'retrieves user link' do mock_graph :get, 'arjun', 'users/arjun_public' do user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' end endend

spec/lib/facebook_wrapper_spec.rb

Even Better Already stubbed for you

Pre-recorded responses (sometimes)

Don’t need to know API endpoints

gem 'sham_rack'

gem 'sinatra'

ENV['RAILS_ENV'] ||= 'test'require File.expand_path('../../config/environment', __FILE__)require 'rspec/autorun'require 'rspec/rails’Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)RSpec.configure do |config| config.infer_base_class_for_anonymous_controllers = false config.order = 'random’endWebMock.disable_net_connect!

spec/spec_helper.rb

ShamRack.at('graph.facebook.com', 443).sinatra do get '/:username' do %Q|{ "id": "7901103", "name": "Arjun Banker", "first_name": "Arjun", "last_name": "Banker", "link": "http://www.facebook.com/#{params[:username]}", "location": { "id": 114952118516947, "name": "San Francisco, California" }, "gender": "male" }| endend

spec/support/fake_facebook.rb

ShamRack.at('graph.facebook.com', 443).sinatra do get '/:username' do %Q|{ "id": "7901103", "name": "Arjun Banker", "first_name": "Arjun", "last_name": "Banker", "link": "http://www.facebook.com/#{params[:username]}", "location": { "id": 114952118516947, "name": "San Francisco, California" }, "gender": "male" }| endend

spec/support/fake_facebook.rb

ShamRack.at('graph.facebook.com', 443).sinatra do get '/:username' do %Q|{ "id": "7901103", "name": "Arjun Banker", "first_name": "Arjun", "last_name": "Banker", "link": "http://www.facebook.com/#{params[:username]}", "location": { "id": 114952118516947, "name": "San Francisco, California" }, "gender": "male" }| endend

spec/support/fake_facebook.rb

ShamRack.at('graph.facebook.com', 443).sinatra do get '/:username' do %Q|{ "id": "7901103", "name": "Arjun Banker", "first_name": "Arjun", "last_name": "Banker", "link": "http://www.facebook.com/#{params[:username]}", "location": { "id": 114952118516947, "name": "San Francisco, California" }, "gender": "male" }| endend

spec/support/fake_facebook.rb

describe FacebookWrapper, '.user_link' do it 'retrieves user link' do user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103’ endend

spec/lib/facebook_wrapper_spec.rb

Even Better Dynamic

Expressive

Readable

gem 'vcr'

ENV['RAILS_ENV'] ||= 'test'require File.expand_path('../../config/environment', __FILE__)require 'rspec/autorun'require 'rspec/rails’Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)RSpec.configure do |config| config.infer_base_class_for_anonymous_controllers = false config.order = 'random’endWebMock.disable_net_connect!VCR.configure do |c| c.cassette_library_dir = 'spec/fixtures/vcr_cassettes' c.hook_into :webmock end spec/spec_helper.rb

describe FacebookWrapper, '.user_link' do it 'retrieves user link' do VCR.use_cassette('fb_user_arjun') do user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' end endend

spec/lib/facebook_wrapper_spec.rb

describe FacebookWrapper, '.user_link' do it 'retrieves user link' do VCR.use_cassette('fb_user_arjun') do user_id = FacebookWrapper.user_id('arjun') expect(user_id).to eq '7901103' end endend

spec/lib/facebook_wrapper_spec.rb

Even Better Record API automatically

Replay responses without network

Verify responses

Additional Build Process Runs outside normal test mode

Rechecks cassettes for diffs

Avoids versioning issues

gem 'puffing-billy'

Puffing-Billy Built for in-browser requests

Allowed to record and reuse (like VCR)

Be brave, venture out of ruby

I also like…

Chrome Dev Tools

Postman

HTTPie

Charles

Additional Reading martinfowler.com/bliki/IntegrationContractTest.html

timrogers.uk/2014/07/12/discovering-private-apis-with-charles-app/ robots.thoughtbot.com/how-to-stub-external-services-in-tests

joblivious.wordpress.com/2009/02/20/handling-intermittence-how-to-survive-test-driven-development

railscasts.com/episodes/291-testing-with-vcr

Bringing it all together Testing services is crucial

If in doubt, stub it out

Determine the flexibility you want

Record responses to save time

Next Up How to Consume Lots of Data

with Doug Alcorn

Thank you! me@nealke.mp

neal@women.com (I like getting email)

@neal_kemp

(I tweet)