Embed Size (px)
Citation preview
Metrics, metrics everywhere(but where the heck do you start?)Tammy Everts & Cliff CrockerVelocity New York 2015
@tameverts @cliffcrocker
#velocityconf
Who cares about performance today?How do I measure performance?
How fast am I?How fast should I be?How do I get there?
The myth of a single metric
Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
Who cares about performance?
“47% of consumers expect a web page to load in 2 seconds or less.”
Akamai, 2009
1s = $27M DNS144ms
Start render1.59s
Hero image render2.01s
How do I measure performance?
Andr
oid
devi
ce fr
agm
enta
tion
Ope
nSig
nal,
Augu
st 2
015
How do we measure UX for mobile?
https://gist.github.com/larahogan/b681da601e3c94fdd3a6
RUM versus plus synthetic
Real User Monitoring 101
Technology for collecting performance metrics directly from the end user’s browser
Involves instrumenting your site via JavaScript
Measurements are fired across the network to a collection point through a small request object
(beacon)
What makes RUM great
Always on Every user, every browser, everywhere Able to capture human behavior/events Only getting better
Questions your RUM data can answer
What are my users’ environments?
How do visitors move through my site?
How are my third-party scripts
performing in real time?
What impact does performance have
on my business?
Synthetic Monitoring 101
Uses automated agents (bots) to measure your website from different physical locations
A set “path” or URL is defined
Tests are run either ad hoc or scheduled, and data is collected
What makes synthetic monitoring great Rich data collected (waterfall, video,
filmstrip, HTTP headers) Consistent “clean room” baseline Nothing to install Doesn’t require users / ability to
measure pre-production and competition
Questions your synthetic data can answer
How do I compare to the competition?
How does the design of my pages affect
performance?
How does the newest version of my site compare to previous versions?
How well am I sticking to my performance budget?
What if my third parties fail?
Original: 3.5s
SPOF: 22.7s
39© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Common things we hear about RUM & synthetic
Why are these numbers so different?I don’t trust your data. Your numbers are wrong.How are you calculating page load time?I can’t share two sets of numbers with the business.
“But it loads so much faster for me!?!”
2015 Macbook ProWarm browser cache
FIOS
X86 – Windows 7 VMCompletely cold cache/DNS
Throttled bandwidth
boomerang.js
Episodes
W3C Performance Working Group
How fast am I?
NavigationTiming API
Browser support for NavigationTiming
http://caniuse.com/#feat=nav-timing
49© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Role-specific use cases
Network operationsFront-end developerMarketing and site operationsDesignerC-level
Use case: Measure network performance
I need visibility into… issues with authoritative DNS servers impact of denial of service attacks
on end users efficiency of connection re-use tier 1 connectivity issues (load balancer,
web server)
Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
Measuring DNS and TCP
function getPerf() { var timing = window.performance.timing; return {
dns: timing.domainLookupEnd - timing.domainLookupStart};
connect: timing.connectEnd - timing.connectStart};}
What’s with all those zeros! Connection reuse DNS caching Prefetching
Focus on higher percentiles
85th percentile
Median (50th)
Use case: Measure front-end browser events
How do I… understand the impact of back-end
versus front-end on page speed? investigate how DOM complexity affects
performance? measure a “fully loaded” page?
Start render DNS TCP TTFB
DOM load event DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
Isolate front-end vs. back-end
Isolate front-end vs. back-end
function getPerf() { var timing = window.performance.timing; return {
ttfb: timing.responseStart - timing.connectEnd};basePage: timing.responseEnd - timing.responseStart};frontEnd: timing.loadEventStart -
timing.responseEnd};}
Front-end
Back-end
Measuring the critical rendering path
Investigate DOM eventsfunction getPerf() { var timing = window.performance.timing; return {
DLoading: timing.domLoading – timing.navigationStart};
DInt: timing.domInteractive – timing.navigationStart};
DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart;
DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart};
DComplete: timing.domComplete - timing.navigationStart};
PLoad: timing.loadEventStart - timing.navigationStart};}
Understanding critical rendering path DOM Loading – browser begins parsing initial
bytes of the document DOM Interactive – doc parsed, DOM has been
constructed DOM Content Loaded – No blocking style sheets DOM Complete – All processing complete, all
assets downloadedhttps://developers.google.com/web/fundamentals/performance/critical-rendering-path
2618 DOM nodes
86 DOM nodes
Visualizing DOM complexity
Use case: Measure object-level performance
I need to understand… how third-party content affects my
performance how long a group of assets takes to
download how assets served by a CDN are
performing
Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
ResourceTiming interface
Browser support for Resource Timing
http://caniuse.com/#feat=resource-timing
Cross-Origin Resource Sharing (CORS)
Start/End time only unless Timing-Allow-Origin HTTP Response Header defined
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"
ResourceTimingvar rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’;var me = performance.getEntriesByName(rUrl)[0];var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart}
Measuring a single resource
Other uses for ResourceTiming Slowest resources Time to first image (download) Response time by domain Time a group of assets Response time by initiator type (element type) Cache-hit ratio for resources
For examples see: https://github.com/lognormal/beyond-page-metrics
Using Resource Groups
PLT impact not due resource groups
PLT impact correlates with improvement from image domains
Use case: Measure the user experience
I just need to understand… when users perceive the page to
be ready how long until my page begins
to render when content above the fold is visible
Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
The fallacy of “First Paint” in the wild Support for First Paint is not standardized between browsers Metric can be misleading (i.e. painting a white screen)
First Paint is not equal to Start Render!Chrome – “First Paint” True Start Render
Start Render and filmstrips
UserTiming Interface Allows developers to measure performance of
their applications through high-precision timestamps
Consists of “marks” and “measures” PerformanceMark: Timestamp PerformanceMeasure: Duration between
two given marks
Browser support for UserTiming
http://caniuse.com/#feat=user-timing
Measure duration between two marksperformance.mark(“startTask”);
//Some stuff you want to measure happens here
performance.mark(“stopTask”);
//Measure the duration between the two marks
performance.measure(“taskDuration”,“startTask”,“stopTask”);
How long does it take to display
the main product image on my site?
Record when an img loads
<img src=“image1.gif” onload=“performance.mark(‘image1’)”>
For more interesting examples, see:
Measure hero image delayhttp://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/
Measure aggregate times to get an “above fold time”http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-experience.html
Adoption of UserTiming
for top 25 shopping sites
(Global)
Nope
User Timing
Sales
How do I measure performance when the onload event loses relevance?
Use case: Measure performance of SPAs
http://www.ryan-williams.net/hacker-news-hiring-trends/2015/october.html
What is a SPA?• Single Page Application• HTML loads once• Fluid user experience• Can be great for performance• Hard to measure
onload event
visible resources
Measuring SPAs• Accept the fact that onload no longer matters• Tie into routing events of SPA framework• Measure downloads during soft refreshes• (support in boomerang.js for Angular and other
SPA frameworks)
How fast should I be?
Use case: Measure business impact
I need to understand… how performance affects business KPIs how our site compares to our
competitors
Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation
http://www.soasta.com/blog/mobile-web-performance-monitoring-conversion-rate/
2% increase in conversions for every 1 second of improvement
Cut load time by 80%83% traffic increase32% increase in time on site108% increase in interaction rate with ads
So what?You must look at your own data.
Not all pages are created equalTop of funnel impact (“browse” pages)
For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds
Not all pages are created equalBottom of funnel impact (“checkout” pages)
However, there is much less impact to conversion when “checkout” pages degrade
What is the Conversion Impact Score?The Conversion Impact Score (CIS) is a relative score that ranks page groups by their propensity to negatively impact conversions due to high load times. For each page group, the Conversion Impact Score is calculated using the proportion of overall requests that are associated with that group, along with the Spearman Ranked Correlation between its load times and number of conversions. The Conversion Impact Score will always be a number between -1 and 1, though scores much greater than zero should be very rare. The more
negative the score, the more detrimental to conversions that high load times for that page group are, relative to the other page groups.
TL;DRThe Conversion Impact Score answers this question:
How much impact does the performance of this page have on conversions?
Conversion Impact Score
http://www.soasta.com/blog/website-monitoring-conversion-impact-score/
How do I get there?
Create a performance budget
Setting a Performance Budgethttp://timkadlec.com/2013/01/setting-a-performance-budget/
Performance Budget Metricshttp://timkadlec.com/2014/11/performance-budget-metrics/
Set meaningful, page-specific SLAs
Chapter 8: Changing Culture
at Your Organization
Free download (until November 4)
http://soasta.io/timeismoneybook
performancebacon.com
performancebeacon.com
Thanks!
Meet us at booth #51
