Upload
patrick-meenan
View
4.511
Download
2
Embed Size (px)
DESCRIPTION
Slides for my tutorial from Velocity 2014 on some of the more advanced features in WebPagetest. Video is available on Youtube: Part 1: http://youtu.be/6UeRMMI_IzI Part 2: http://youtu.be/euVYHee1f1M
Citation preview
WebPagetest Power UsersPatrick [email protected]@patmeenan
The Basics
● Presentation is available on slideshare○ http://www.slideshare.net/patrickmeenan○ Mostly contains links to everything I’ll be showing
● Video will be uploaded to YouTube○ https://www.youtube.com/user/patmeenan
● I’ll send links and announcements of both to Twitter and Google+○ https://twitter.com/patmeenan - @patmeenan○ https://plus.google.com/u/0/+PatrickMeenan/posts
What we’re covering1. Multiple Runs2. medianMetric3. Visual Comparisons4. Filmstrip URL Manipulation5. Video Comparison View6. Better Screen Shots7. Chrome timelines8. Mobile device vs emulation (data reduction)9. bodies/tcpdump/cloudshark
10. Chrome Tracing11. Images12. Keep UA String13. Resource blocking14. SPOF testing15. Filmstrip comparison debugging/overlays16. Scripting multi-step/authentication17. Scripting request manipulation (headers, DNS, command-line)18. User Timing19. Custom metrics/success criteria20. API21. CI - node wrapper, Travis, Jenkins, Grunt budget22. Bulk tests (url/host substitution)23. Results logging (splunk, logster, etc)24. Benchmarks
Using WebPagetest
The API
Private Instances
Plot Full Results
● Test with > 1 run● Click “Plot Full Results” link below data table
medianMetric
● Defaults to onload (“loadTime”)● Can be any metric by adding &medianMetric=<blah> to URL● Valid metric names are the identifiers returned in the API
Visual Comparisons
● Log in● Run Individual tests with the options you want
○ For best results, label each test run● Go to your Test History● Select the tests you want to compare● Click Compare
Filmstrip URL Manipulation
http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE
Filmstrip URL Manipulation
http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE
● Test ID’s for the tests that are being compared● In the order to be displayed● Comma-delimited● Selects the First View run with the median onload time
Each test can be configured
140609_KC_P6Z-l:First+View-r:3-c:0-e:7.3
● -l: (Label)● -r: (Run)● -c: (Cached? 0 = First View, 1 = Repeat View)● -e: (end time in seconds or category)
○ doc (onload)○ full (fully loaded)○ visual (visually complete - first time it gets to 100%)○ all (last visual change)
Video Comparison View
● Get the filmstrips configured the way you want● Click “Create Video”● Click “View as data comparison”
Better Screenshots
● PNG Screen Shots● Control JPEG compression level● Usually API-only, but also available through hidden parameters
pngss= 0/1, enables full-resolution PNG screen shots(not video frames)
iq= <jpeq quality level>, sets compression level for all JPEG images(screen shots and video frames)
http://www.webpagetest.org/?pngss=1&iq=75
Regular Resolution vs Improved
Chrome Timelines
● Advanced Settings -> Chrome -> Timeline○ Always enabled on Android with video capture○ Call stacks are optional (but helpful for debugging)
● Processing Breakdown link○ top of details page○ left of waterfall thumbnail
● Timeline CPU Time below waterfall including in filmstrip● Timeline links to the left of the waterfall (download or view)
CPU Impact - www.amazon.com
Nexus 5
Desktop VM
Moto G
Moto E
200ms ticks
Filmstrip Comparison
CPU Impact - www.google.com/search?q=flowers
Nexus 5
Desktop VM
Moto G
Moto E
200ms ticks
Nexus 7 (2013)
Filmstrip Comparison
Mobile Device vs Emulation - http://www.theguardian.com/uk?view=mobile
Nexus 5
Desktop VM
Moto G
Moto E
Nexus 7 (2013)
500ms ticks
Filmstrip Comparison
Data Reduction Proxy
● Android-specific● Only available with actual devices● Advanced Settings -> Chrome● Sample Test and Comparison
Moar Data!
● Response Bodies○ Desktop-only (or mobile emulation), all browsers○ Stores text resources served to test browser (HTML, CSS, JS)○ Available in “object” tab of waterfall dialog○ Also as download from request detail
● TCP Dump○ Desktop and Android Mobile○ pcap download left of waterfall○ Sharable through cloudshark (view link)
■ Filter “http”■ Find request■ Analysis tools->Follow TCP Stream■ Show only this stream
Chrome Tracing
● Filing a Chrome Performance bug (crbug.com)?● Debugging a performance issue and need more detail?
○ Miss the days of writing raw assembly? yeah, it’s like that● Provides tracing of internal browser activity● Downloadable or viewable directly in the browser
○ WASD to zoom/pan
Images
● View all images link below waterfall in detail view (main test)● Images displayed in the order they were requested● Look for:
○ visible images loading before slideshow/lower images○ things that should be in sprites○ Images with lots of savings○ PNGs that should be JPEGs○ Anything else “odd”○ Decide if you really need to deliver all of those images while loading
● JPEG Analysis view
Preserve Original UA String
● Advanced Settings -> “Preserve Original User Agent String”● Usually Appends PTST <version> to UA string
○ Identify/filter synthetic tests● Sometimes sites do weird things (sometimes on purpose)● AOL blocks ads for example
Resource Blocking - Desktop Only
● Good for what-if testing○ How does the site load without my javascript?○ How does it load without ads?
● Only works for external resources● Substring match● “Fails Fast” - returns an immediate error● Combine with visual comparison for great effect
SPOF Testing
● Good for testing impact of 3rd-party outages● Blackhole FQDN’s only (and entirely)
○ Coming soon, “SPOF all domains except…”● Goes directly to a Filmstrip comparison
○ Comparing Original vs SPOF● SPOF routes given domains to blackhole.webpagetest.org
○ blackhole.webpagetest.org is just a host with a firewall rule to drop all incoming packets
● Non-Responding behaves VERY differently from Fast-Fail● Would your monitoring detect it?
Test Comparison
● Waterfall overlay/slider● Comparison Graphs
Scripting - Multi-step
● Simplest case is multiple distinct pages● logData 0/1
○ logData 0○ navigate www.amazon.com○ logData 1○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites
● Test with shared site resources● vs combineSteps
○ combineSteps○ navigate www.amazon.com○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites
Scripting - Forms
● Common use is testing pages behind an auth curtain○ Check the “sensitive data” box to have cookies stripped
● Multiple ways to automate:○ Automate login form○ Re-use existing session cookie
● Script Structurenavigate <landing page>xxx (not measured)xxx (not measured)xxxAndWait (measured)
● WPT depends on network activity for end state
Scripting - Form Commands
● setValue● setInnerText● setInnerHtml● selectValue● click, sendClick● sendKeyDown, sendKeyUp, sendKeyPress● submitForm
● exec
exec(AndWait)
● Executes arbitrary javascript in the context of the page● Best to keep it to plain javascript (don’t depend on in-page code)● Debug scripts with local dev tools console before submitting
Script - Get the Weather in Santa Clara
logData 0navigate http://www.weather.com/exec document.getElementById("typeaheadBox").value = 95054logData 1execAndWaitdocument.getElementById("headerSearchForm").submit()
Result - Filmstrip
Scripting - Request Manipulation
● setDns, setDnsName, overrideHost● setCookie● setHeader, addHeader● block
Scripting - Misc
● setViewportSize● sleep
Full scripting docs:https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting
Custom Browser Settings
● Chrome○ Custom command-line options (advanced settings)
■ Specify PAC scripts (proxy)■ Custom host override■ Disable SPDY■ Toggle features
● Firefox○ firefoxPref script command
■ Sets any firefox pref value before launching browser
User Timing
● performance.mark(‘label’)● Exposed as metrics in WebPagetest results
○ In raw data○ As purple vertical lines in waterfall○ Last mark exposed as a unique metric
● Trivial polyfill for older browsers● Validate RUM markers using video capture/filmstrip
Start Render From JS
Light green line in waterfall
Chrome:
var startTime = window.chrome.loadTimes()['requestTime'] ?window.chrome.loadTimes()['requestTime'] :window.chrome.loadTimes()['startLoadTime'];
var render = window.chrome.loadTimes().firstPaintTime - startTime;
IE:var render = window.performance.timing.msFirstPaint;
Custom Metrics
● Runs arbitrary javascript snippet(s) at the end of the test● Defined on a per-test basis or instance-wide
○ New HTTP Archive metrics (DOM depth, iFrame count, etc)● Can be simple one-liners or more complex multi-function
operations● Can override existing metrics● Can return numeric or string values● Lots of uses:
○ Validate expected page content○ Collect RUM beacon results○ Separate results based on A/B or type of ad served○ Count widgets, etc
Custom Metrics - Simple
[iframe-count]return document.getElementsByTagName("iframe").length;
Custom Metrics - Validate Content[result]var result = 499;try {
var copyright = document.getElementsByClassName('copy-text')[0].innerText.slice(-8);result = copyright == "AOL Inc." ? 0 : 498;
} catch(e) {}return result;
Changes test result to:● 499 if the copyright block doesn’t exist● 498 if the copyright block exists but the values don’t match● Leaves it as a success (0) if it exists and matches
API
https://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis
1. Submit Test
2. Poll for test status
3. Get test result
API - Submitting a test
● API key required for public instance○ email me ([email protected]) for key○ keys are quite limited (~200 “page loads” per day)○ mostly for proof of concept, early dev before deploying private
instance
● Anything you can do with the UI you can do with the API○ If a field isn’t documented, just inspect the HTML on the UI to get the
field name :-)
● Make sure to encode parameters○ urlencode if GET○ form encode if POST
API Response
http://www.webpagetest.org/runtest.php?url=www.aol.com&f=json
{"statusCode":200,"statusText":"Ok","data":{
"testId":"140610_FY_N2F","ownerKey":"04121a1e5b17a59ba9ddd666ae020cf52cee4abe","jsonUrl":"http:\/\/www.webpagetest.org\/jsonResult.php?test=140610_FY_N2F","xmlUrl":"http:\/\/www.webpagetest.org\/xmlResult\/140610_FY_N2F\/","userUrl":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/","summaryCSV":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/page_data.csv","detailCSV":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/requests.csv"
}}
Polling status
● Periodically poll the JSON/XML URL● statusCode = 200 : Test Done
○ Test itself may still have failed● statusCode = 1xx : Test pending/waiting● statusCode = 4xx : Error locating the test
API test result
http://www.webpagetest.org/xmlResult.php?test=140610_FY_N2F&requests=1&breakdown=1
Bulk testing
● Submit a bunch of tests
● Periodically check status
● Do “something interesting” with the results
Bulk Testing
● PHP○ https://github.com/WPO-Foundation/webpagetest/tree/master/bulktest
● Python○ https://github.com/WPO-Foundation/webpagetest/tree/master/batchtool
● HTTP Archive○ http://httparchive.org/○ https://github.com/HTTPArchive/httparchive
Node API Wrapper
Created by Marcel Duran (@marcelduran)
https://github.com/marcelduran/webpagetest-api
$ npm install webpagetest -g
webpagetest test -k <key> http://twitter.com/marcelduran
webpagetest status 140610_F0_T5K
webpagetest results 140610_F0_T5K
webpagetest test http://twitter.com/marcelduran --poll 5 --timeout 60
CI Integration
https://github.com/marcelduran/webpagetest-api/wiki/Test-Specs
Match result to provided specs{ "median": { "firstView": { "requests": 20, "render": 400, "loadTime": 3000, "score_gzip": { "min": 90 } } }}
CI Integration cont.
Uses Mocha with reporters suitable for integration with:● Jenkins● Travis-CI● Drone.io● Just about anything else you can imagine
grunt-perfbudget
Created by Tim Kadlec (@tkadlec)
https://github.com/tkadlec/grunt-perfbudget
Bulk Testing in the UI (Private Instances)
● Submit list of URLs to test● UI options apply to each test● Scripts can be run against each URL
○ %URL% in script gets replaced with test URL○ %HOST% in script gets replaced with host from test URL○ %HOSTR% in script gets replaced with host from URL after redirects
● Aggregate results○ Download as CSV (subset of data)○ Compare tests against each other
Results logging (Private Instances)
● As each test comes in, full result data is written to log files○ Page Data (load time, Speed index, requests, custom metrics, etc)○ Request Data (details for every individual request)
● Each test run or request is on it’s own log file line● JSON-formatted● Config information is included
○ Test URL○ Run #○ First/Repeat View○ Test label○ Location○ Browser○ Connectivity○ Test ID○ URL to test result
Results logging - integration
● Splunk● Logster
○ Logster -> StatsD = Arbitrary metrics trending automatically● Track individual requests across all pages
○ ads JS performance○ CDN response times○ Effectiveness of image compression○ so much more
Benchmarks (Private Instances)
● Still pretty experimental● Recurring tests run automatically
○ Configuration is a bit ugly but flexible● Tests are scheduled, run and aggregated automatically
○ Top-level trended aggregate view○ All-metrics trended aggregate view○ Per-page trended view○ Scatter Plot for a given run○ Filmstrip comparisons
Google Booth
Wednesday 6/25● Lightning Talks
○ 10:45 am - Meet4Speed, Pat Meenan○ 2:40 pm - The PageSpeed Optimization Platform - Joshua Marantz
● Meet the Speaker○ 2:40 pm - WebPagetest Q&A, Pat Meenan
Thursday, 6/26● Lightning Talks
○ 10:45 am The Case for Tunneling Sockets Over HTTP, Wenbo Zhu○ 2:40 pm - Optimize For Bandwidth - the what and why, Matt Atterbury
● Meet the Speaker○ 2:40 pm - Making the Web POSH - Q&A, Jan Maessen
Feedback Please!
http://velocityconf.com/velocity2014/public/content/mobile-app
Thank You!