Super fast end-to-end testsLars ThorupZeaLake Software Consulting

March, 2016

Lars Thorup

● Software developer/architect● JavaScript, C#● Test Driven Development

● Coach● Agile engineering practices

and tools

● Founder● BestBrains● ZeaLake

● @larsthorup

Agenda● What's so bad about end-to-end tests?

● Why are unit tests not sufficient?

● Getting the best of both worlds

● Demo!

What's so bad about end-to-end tests?

* Thanks to Randall Munroe: xkcd.com/303/

What's so bad about end-to-end tests?● Not fast

● Not precise

● Not simple

● Not robust

End-to-end tests are slow● Selenium tests: 0.1 / second

● Unit tests: 61 / second

End-to-end tests give poor feedback● End-to-end tests: "something went wrong"

● Unit tests: the problem is "in this method in this class"

End-to-end tests are complicated● Load specific set of test data into the database

● Run the application on a web server

● Start a browser

● Writing the test ● Depend on test data loaded elsewhere● Ignore side-effects of other tests

End-to-end tests are fragile● End-to-end tests fails sporadically:

● Unit tests fails for a reason:

Unit tests have many nice properties● Fast

● Precise

● Simple

● Robust

Why are unit tests not sufficient?● They test code in isolation

● Dependencies are mocked

● So what happens when a dependency changes interface?

● The application may break, while all unit tests pass!

● Because...

● Mocks lie to you!

The problem with unit tests: mocks lie to you!● All is well for now:

The problem with unit tests: mocks lie to you!● Then watch what happens:

The problem with unit tests: mocks lie to you!● How can mocks lie to us?

● ...because we hand-write them

● ...we copy assumptions that can change

● Can we automatically generate correct mocks?

Getting the best of both worlds

Demonstration of a sample implementation● Node.JS

● Back-end: a REST-ful API

● Front-end: HTML + JavaScript

● Almost 100 end-to-end tests per second

● github.com/larsthorup/http-auto-mock-demo

End-to-end tests now also become● Fast

● Precise

● Simple

● Robust

Questions!