Internet Performance Measurements using Firefox Extensions Scot L. DeDeo Professor Craig Wills

Department of Computer Science

Internet Performance Measurements using Firefox Extensions

Scot L. DeDeo Professor Craig Wills


Motivation

There exists an API within the Firefox web browser that allows for interacting with its downloading and processing of websites

Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?


Outline Introduction to Firefox Extensions

Description Capabilities

Page Stats Extension Overview Approach and paths taken Demo

Summary


Introduction Firefox Extensions are

add-ons that integrate directly into the Firefox Web browser.

Function through open source API that allows for manipulating both the content being received by the web browser as well as the graphical outlook.


Capabilities Graphical add-ons (Use the XML User Interface Language – XUL)

Allows for changing the existing Firefox overlay, adding new buttons, menus, and windows.

E.g. Google toolbar allows for a toolbar to be inserted directly into Firefox.

Middleware modifications (Cross Platform Component Object Module – XPCOM Interfaces - Accessible through JavaScript or C++)

Allows for changing networking properties, modifying / blocking data requests / responses, etc.

E.g. Fasterfox allows you to tweak many network and rendering settings such as simultaneous connections, pipelining, cache, DNS cache, and initial paint delay.

E.g. AdBlock and AdBlock plus allow for use of a pre-formed list or to customize a list of sites to block downloads from.


Capabilities Combination of Graphical add-ons and Middleware modifications

can yield new applications that exist within Firefox. E.g. Gmail Space – Allows users to take advantage of Gmail’s 2 GB quota by

providing a graphical interface to easily transfer files.

Fixing bugs in other sites

Allows for changing sites html, style sheets, and inserting JavaScript. E.g. Style sheets can be modified to change the background color and text of a site

(change to black on white instead of yellow on white).


Website Request / Response Breakdown

1. Location Changes – Request for container (main) page goes out.

2. Response for container (main) page is received and processed.

3. Additional requests are spawned in response to the processed page: CSS, JavaScript, images, etc.

4. Responses come in and are processed and may / may not spawn additional requests.

Container Page

CSS

JavaScript

Jpeg

Spawns

JavaScript

Spawns

Browser location set to:

http://www.scotd.com/index.php

http://www.scotd.com/index.php?id=all

http://www.scotd.com/scot.css

http://www.scotd.com/background/valley2k.jpg

Spawns





Summary


Page Stats Extension Measures the HTTP requests associated with the load of a webpage.

Page Measurements Include:# of Request Request submit time (msec)Finish Time (msec) HTTP Response CodeSize (bytes) content typeURL

Page Summary Includes:URL total page size (bytes) total time (msec) total # of requests # dead requests

Capabilities: Control input and output with a graphical interface Output to a log file Run in a batch mode and visit a list of sites


Page Stats Extension Usage:

1. Graphical Interface found under the tools menu in Firefox - “Site Measurements”

• Ability to enable / disable measurement listeners, log file, and select a list of sites to visit.

• Also allows for saving and clearing the text on the measurement window.

• Provides a summary of all sites visited.


Page Stats Extension Usage:

2. Batch mode – Invoked by a configuration file specified on the command line (firefox pagestatsconfig=“/home/sdedeo/config.txt”).Parameters:

measurement: onlog: /home/sdedeo/log.txtsites: /home/sdedeo/sites.txtexitOnFinish: truerequestTimeout: 8000

pageTimeout: 45000

Allows for easily measuring pages in bulk

Pages are still fully rendered


Approach and paths taken Began in January 2006 based on work completed by

Paul Timmins Included initial work of setting up an extension with ideas on how

to make measurements

Initial version based on an XPCOM component called nsiWebProgress which reports to the browser: State, Progress, Location, Status, and Security changes


Approach and paths taken First Approach Issues

nsiWebProgress progress notification only occurs on text based files (E.g. html, CSS, JavaScript). Images and other static files are omitted.

No way to measure the time the request is submitted Occasionally the progress notification gives an unknown length of a file If a site was visited more then once, caching effects skew measurements

Second Approach – nsiHTTPChannel Intercepts requests browser sends – “http-on-modify-request” Intercepts responses before processing – “http-on-examine-response” Clears system cache between sites in order to maintain consistent

measurements


Approach and paths taken Second Approach (Continued)

nsiHTTPChannel provides no state notifications Solved using nsiWebProgress state change to detect state

stopped / finished Probe to insure all requests that have gone out have come back

(nsiWebProgress state stop returned before all images received)

Timeout Mechanism Two Timeouts:

Max time between receiving responses – defaults to 8 seconds Max time for a page – defaults to 45 seconds


Approach and paths taken

Sites with Issues: http://www.cnn.com routinely sends requests for images in which no

response comes. These are JavaScript invoked images on mouseovers This problem is solved by the request timeout and the requests are labeled

as ‘dead’

http://my.netscape.com uses a JavaScript based redirect: ‘document.location = http://my.netscape.com/index2.psp’ which when used in batch mode was causing the extension to move onto the next site and process two sites simultaneously

This was solved by giving a one milliseconds delay before switching sites after a site’s results were processed

http://www.dallasnews.com sends an image banner which receives an update once every two seconds

This was solved with the max site timeout


Demo

Launch Firefox





Summary


Summary

Firefox extensions are a powerful tool that allow for: Measuring and manipulating how a page is downloaded and

processed Adding additional features, both graphical and background

Building Extensions: Firefox 1.5 enables much easier testing of extensions XPCOM API is still in the beta stages and certain libraries must

be used with caution since they change Documentation available is still minimal, but the development

community is growing at a fast pace


Summary

Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?

The pagestats extension shows one way we can create quantitative measurements for a website

Pagestats can be used in conjunction with other extensions, such as adblock and fasterfox, to compute a measurement of effectiveness


Questions / Comments

Scot L. DeDeoProfessor Craig Wills

Page Stats Extension:http://www.wpi.edu/~sdedeo/pagestats

Related Sites:https://addons.mozilla.org/firefox/extensions/http://www.xulplanet.com (XUL and XPCOM API Reference)

Documents

Internet Performance Measurements using Firefox Extensions Scot L. DeDeo Professor Craig Wills