Boomerang: How fast do users think your site is?

IntroductionHow does it work?Using boomerang

Data

Boomerang: How fast do users think your siteis?

Philip Tellis / [email protected]

ConFoo.ca/2011 – 2011.03.09-11

ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?

http://confoo.ca/

http://confoo.ca/


Data

$ finger philip

Philip [email protected]

@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/


http://twitter.com/bluesmoon

http://bluesmoon.info/

http://confoo.ca/


Data

TimeThe adversaryMeasure twiceboomerang


http://confoo.ca/


Data



http://confoo.ca/


Data



http://confoo.ca/


Data


Only 10–20% of page load time is spent on systems we control


http://confoo.ca/


Data


It’s what we can’t control that users notice

http://twitter.com/stoyanstefanov/status/43747252317077504


http://twitter.com/stoyanstefanov/status/43747252317077504

http://confoo.ca/


Data


We can’t control

browsers


http://confoo.ca/


Data


We can’t control

plugins


http://confoo.ca/


Data


We can’t control

OSes


http://confoo.ca/


Data


We can’t control

viruses


http://confoo.ca/


Data


We can’t control

antiviruses


http://confoo.ca/


Data


We can’t control

microwaves


http://confoo.ca/


Data


We can’t control

baby monitors


http://confoo.ca/


Data


We can’t control

naughty neighbours


http://confoo.ca/


Data


We can’t control

file shares


http://confoo.ca/


Data


We can’t control

governments


http://confoo.ca/


Data


We can’t control

rodents


http://confoo.ca/


Data


We can’t control

Try simulating all that in the lab!


http://confoo.ca/


Data


We need to measure real end-user performance


http://confoo.ca/


Data


We need to measure real end-user performance from the realend-user’s perspective


http://confoo.ca/


Data



http://confoo.ca/


Data


While this might work, it isn’t necessarily representative


http://confoo.ca/


Data


What about JavaScript?


http://confoo.ca/


Data



http://confoo.ca/


Data


boomerang is...

A piece of javascript that you add to your web page where itmeasures and beacons back to you, the end user’s perceivedperformance of your page


http://confoo.ca/


Data


How?

<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({

user_ip: "<network ident>",beacon_url: "http://mysite.com/beacon.php"

});</script>


http://confoo.ca/


Data


What does it do?

Measures user’s network throughput and latency to yourserverMeasures the current page’s load timeBeacons these results back to your server


http://confoo.ca/


Data

LatencyBandwidth/ThroughputLoad timeAccuracy

How does boomerang work?


http://confoo.ca/


Data


Let’s take that one at a time


http://confoo.ca/


Data


Measuring latency

Download a 32 byte gif 10 times in sequenceMeasure the time to download eachDiscard the first measurement because it’s overpriced9

Calculate the arithmetic mean, standard deviation andmargin of error of the rest


http://confoo.ca/


Data


Measuring throughput

After the latency test is done, we download progressivelylarger imagesStop at the first image that times outRedownload that image a few more timesCalculate the median, standard deviation and margin oferror of the largest image


http://confoo.ca/


Data


Measuring latency before throughput helps here

Those 10 latency images do a lot to widen the TCPwindow sizeThe bandwidth images make much better use of availablebandwidthThe image we end with makes the best use of bandwidth


http://confoo.ca/


Data


How do we measure page load time?

In the onbeforeunload event, measure the time andstore it in a cookieIn the onload event, check the cookie, and measure thedifference with the current timeWe also make sure that the page that set the cookie is thereferrer of the current page


http://confoo.ca/


Data


What? Two pages?

Yes, this needs two pages and cookies. If those aren’tsupported, we try to use the WebTiming API10.


http://confoo.ca/


Data


How accurate is it?

Latency measurements are very accurate (±1%)Bandwidth is to an order of magnitude. For badconnections can be ±30%Page load time sometimes has outliers, you needpost-filteringThe margin of error tells you how good your data is


http://confoo.ca/


Data

BasicIn-page timersAJAX

Include it on your page

<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({

user_ip: "<network ident>",beacon_url: "http://mysite.com/beacon.php"

});</script>


http://confoo.ca/


Data


For most sites, that’s about it


http://confoo.ca/


Data


You probably want to do more


http://confoo.ca/


Data


Measure more than just load time

<html><head><script>var t_pagestart=new Date().getTime();</script>...<script>var th=new Date().getTime();</script></head><body>...<script>var tj=new Date().getTime();</script>... ConFoo.ca/2011 – 2011.03.09-11 Boomerang: How fast do users think your site is?

http://confoo.ca/


Data


...<script src="boomerang.js"></script>...var te=new Date().getTime();BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).

setTimer("t_body", te-th).setTimer("t_js", te-tj);

</script></body></html>


http://confoo.ca/


Data


This adds the t_head, t_body and t_js fields to the beacon


http://confoo.ca/


Data


Loading dynamic content

BOOMR.init({user_ip: "<network ident>",beacon_url: "http://mysite.com/beacon.php",auto_run: false

});


http://confoo.ca/


Data



// Just before download startsBOOMR.plugins.RT.startTimer("t_done");


http://confoo.ca/


Data



// Just after download finishesBOOMR.plugins.RT.done();


http://confoo.ca/


Data


Much more

http://yahoo.github.com/boomerang/doc/howtos/


http://yahoo.github.com/boomerang/doc/howtos/

http://confoo.ca/


Data

shouldacouldawoulda

The beacon

GET request to the beacon URL (response ignored)All parameters passed in the query stringExtra timers are passed in as a comma separated list int_other

before_beacon JavaScript event fired just before thebeacon is sent


http://confoo.ca/


Data

shouldacouldawoulda

What should we do with the data?

Sanity checking to:Remove fake dataRemove abusive dataMaybe just rate limiting


http://confoo.ca/


Data

shouldacouldawoulda

What can we do with the data?

Statistical analysis to:Remove outliersAggregate based on bandwidth blocksMeasure trends over time and correlate them with codechanges


http://confoo.ca/


Data

shouldacouldawoulda

Bandwidth blocks


http://confoo.ca/


Data

shouldacouldawoulda

Bandwidth blocks


http://confoo.ca/


Data

shouldacouldawoulda

Bandwidth blocks


http://confoo.ca/


Data

shouldacouldawoulda

Bandwidth blocks

Data points from some countries may require narrower bands


http://confoo.ca/


Data

shouldacouldawoulda

Geographic data

Looking at latency from different geographic locations can tellyou where your next mirror should be


http://confoo.ca/


Data

shouldacouldawoulda

ISPs

Grouping data by ISP can tell you who’s behaving badly


http://confoo.ca/


Data

shouldacouldawoulda

More data

Write plugins to get more performance dataWe already have a DNS pluginI’ve just completed a plugin to measure IPv6 support andlatencyWhat about a full WebTiming plugin?I’m also looking at measuring connection setup time


http://confoo.ca/


Data

shouldacouldawoulda

You decide

Once you have the data, you can do anything with it


http://confoo.ca/


Data

Boomerang: It (almost) always comes back

http://github.com/yahoo/boomerang

http://yahoo.github.com/boomerang/doc/


http://www.github.com/yahoo/boomerang/

http://yahoo.github.com/boomerang/doc/

http://confoo.ca/


Data

Photo credits

flickr.com/photos/21233184@N02/4389412851http://xkcd.com/445/


http://www.flickr.com/photos/21233184@N02/4389412851/

http://xkcd.com/445/

http://confoo.ca/


Data

Contact me

Philip [email protected]

@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/slideshare.net/bluesmoonjoind.in/talk/view/2913


http://twitter.com/bluesmoon

http://bluesmoon.info/

http://www.slideshare.net/bluesmoon

http://joind.in/talk/view/2913

http://confoo.ca/


Data

References

github.com/yahoo/boomerangMore bandwidth doesn’t matter (much) – Mike BelsheAnalysing Bandwidth & Latency – YUI BlogIt’s the latency, stupid – Stuart CheshireThe statistics of web performance


http://github.com/yahoo/boomerang/

http://www.belshe.com/2010/05/24/more-bandwidth-doesnt-matter-much/

http://www.yuiblog.com/blog/2010/04/08/analyzing-bandwidth-and-latency/

http://www.stuartcheshire.org/rants/Latency.html

http://www.slideshare.net/bluesmoon/index-3441823

http://confoo.ca/


Data

Overpriced images

The first image might require a DNS lookup and TCPhandshakeSlow start is not an issue since 32 bytes fit in 1 packet


http://confoo.ca/


Data

WebTiming API

JavaScript interface to network timing information includingDNS, TCP connect, download, etc. Supported by:

Microsoft Internet Explorer 9Google Chrome (WebKit)W3C specification


http://confoo.ca/

Technology

Boomerang: How fast do users think your site is?