1. Lean Analytics requires a Lean project
Illustration from commons.wikimedia.org/wiki/File:Kepler-solar-system-1.png
Challenging for a consultancy?
Lean Analytics works well with Lean Startup, Lean UX and
Agile projects
Question designs, question authorities, ask difficult questions,
be prepared to kill features.
Requires freedom to experiment
Was this feature used more than Google+ login?
If feature isn’t successful, just kill it.
Measure only actionable metrics.
Is the feature for acquisition, retention or revenue?
Typically at least 10k events, such as visits or people using a
feature.
Doesn’t work for small startups.
4. Make big changes
Illustration from http://quod.lib.umich.edu/m/moa/agb8710.0001.001/26
If you cannot see the changes from 2 meters away, most likely
your AB test won’t either.
Google A/B tested 41 shades of blue. Don’t do that — leads to
local optimum.
Illustration from http://www.doxsey.net/blog/too-practically-minded-to-be-of-any-theological-good/
Never terminate earlya running test
Use a split test calculator, for example:
http://getdatadriven.com/ab-significance-test
Google Analytics can be used for tracking events but it is
painful.
Mixpanel more suited for the job. Consider Keen.io, Heap as
well.