Upload
sampetruda
View
474
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
10-04-23 @ IPA Herfstdagen
Software Engineering Research Group
DelftUniversity ofTechnology
Dynamic Analysis and Testing of Ajax User Interfaces
Ali Mesbah, Arie van Deursen
Software Engineering Research GroupDelft University of Technology, The Netherlands
2Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Classical Web Applications
Shortcomings of the Multi-page style:• Low lever of user interactivity• Redundant data transfer• High user-perceived latency• Client is passive
ServerServerClientBrowserClient
Browser
HTTP
GET
HTML
3Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Ajax
Asynchronous JavaScript and XML• Asynchronous server communication
through XMLHttpRequest• Web standards-based (HTML, CSS, XML)• Dynamic display through DOM• JavaScript binds everything together
Features:• No full page refresh!• Single-page web interface• Google Suggest, Gmail, Yahoo! Mail
4Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Ajax-based Web Applications
• Client-side execution• State changes & navigation• Dynamic DOM• Delta-communication• Clickables: Elements having
event-listeners attached to them
<a href="javascript:OpenNewsPage();"><a href="#" onClick="OpenNewsPage();"><div onClick="OpenNewsPage();"><a href="news.html" class="news"><input type="submit" class="news"/><div class="news">
$(".news").click(function() { $("#content").load("news.php");});
5Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Research Question
How can we automatically analyze (test) Ajax web applications?
6Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Current Approaches
• General Approach: extract link, send a request to the server and analyze the response. i.e., HTML (based on the Multi-page style)
• Capture/Replay: demands a substantial amount of manual effort
• Static Analysis: misses the complex run-time behavior
• Dynamic Analysis: promising but detecting the various doorways to different dynamic states and providing proper interface mechanisms for input values is challenging.
7Dynamic Analysis and Testing of Ajax User Interfaces
| 26
A Method for Deriving Ajax States
• Conduct dynamic analysis of the user interface to find Clickable elements and state changes
• Reverse-engineer a State-flow Graph to abstract the navigation paths and state changes
8Dynamic Analysis and Testing of Ajax User Interfaces
| 26
User Interface State Changes
• In classic web applications, each UI state is represented by a URL and the corresponding web page
• In Ajax, it is the internal change of the DOM tree on the (single-page) user interface that represents a UI state change
• To adopt a generic solution, we define a UI state change as: a change on the DOM tree caused either by server-
side state changes propagated to the client, or client-side events handled by the Ajax engine
9Dynamic Analysis and Testing of Ajax User Interfaces
| 26
State-flow Graph
10Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Inferring the State Machine (Crawljax)
11Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Detecting Clickables
• Find a set of Candidate Elements (CC)• Expose each to en event (onclick, onmouseover, …)• Analyze the effect on the DOM
We distinguish between 3 ways of obtaining CCs:1. Full Auto Scan: HTML tagname (div, span, a, …)
with attribute constraints, e.g., div:{class=“menuitem”}
2. Annotation: element attribute crawljax=“true”, or “false”
3. A DSL: Crawling Ajax Specification Language (CASL)
12Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Detecting States
Input: DOM Trees before and after the event:
• Compare DOM Trees:• As is• Textual: extract textual content and compare• Structural: extract structural content and compare• Meta-model: reverse-engineer the Schema for each
DOM, then compare the schemas
• Comparison method: Edit Distance, Levenshtein method with a similarity threshold
13Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Processing Document Tree Deltas
Upon every new state entry:• Compute the differences between the previous document
tree and the current one • We use an enhanced Diff algorithm to find the “delta
updates” • Delta updates may be due, for example, to a server request
call that injects new elements into the DOM• Find new candidate elements in the delta updates, and
examine them in a recursive depth-first manner
14Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Navigating the States
• Problem: going ‘Back’ in the browser
• Solutions: 1. Register the state explicitly and use Back2. Click through from the initial state
• Use element IDs: not always present, not always persistent
• Use XPath Expressions: much more reliable, not 100% water-
proof
15Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Data Entry Points
Detecting DOM forms:• For each state change, extract all the forms• For each form, calculate a hashcode based on the form
properties• Check if form is already present in the database• If not, store the form completely in the database with the
hashcode as its ID• If present, use the corresponding custom values in the
database (if present) to fill the form in the browser and submit
• Upon submission, the resulting state change is recursively analyzed.
16Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Full Auto Scan
For each state:1. Retrieve all candidate clickables2. Fire event on each candidate element3. Compare DOM trees: Edit Distance. Is DOM changed?4. Make a new State and add to the State-flow graph5. Find the XPath Expression of the Clickable Element6. Add the clickable as an edge to the SFG between state
before and after the click7. Analyze forms8. Do all this recursively
17Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Testing Ajax States
• With access to different dynamic DOM states we can check the user interface against different constraints
• We use invariants as oracle• Invariants on the DOM tree• Invariants between DOM states, and• Application-specific invariants
18Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Generic DOM Invariants
Generic DOM Invariants:• Validated DOM• No error messages in DOM• Accessible State• Secure State• Discoverable links• No Dead Clickables• Consistent Back-Button
19Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Application-specific Invariants
Examples:• //case one: warn about collapsable divs within expandable
items String xpathCase1 = "//LI[contains(@class
,’expandable’)]/DIV[contains(@class ,’collapsable’)]";
• //case two: warn about collapsable items within expandable items
String xpathCase2 = "//LI[contains(@class ,’expandable’)]/UL/LI[contains(@class ,’collapsable’)]";
20Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Testing Ajax Paths
• While running the crawler to derive the state machine can be considered as a first full test pass, the state machine itself can be further used for testing purposes.
We generate test cases from the state machine:• K shortest paths from Index to sinks (nodes with no
outgoing edges)• Cycles included only once, All-transitions coverage• Transform each path found into a JUnit test case
21Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Testing Ajax Paths@Testpublic void testcase1 () {browser.goToUrl (url);/*Element-info: SPAN class=expandable-hitarea */browser .fireEvent(new Eventable (new Identification("xpath", "//DIV[1]/SPAN[4]"), "onclick"));
Comp.assertEquals(oracle.getState ("S_1"). getDom(), browser.getDom ());
/*Element-info: DIV class=hitarea id=menuitem2 */browser.fireEvent(new Eventable(new Identification("xpath","//P[2]/ DIV[2]"), "onmouseover"));…}
22Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Testing Ajax Paths
Generated test suite can be used in several ways:• It can be run as is on the current version of the Ajax
application, but for instance with a different browser to find browser incompatibilities
• It can be applied to altered versions of the Ajax application to support regression testing (against a Gold-standard)
• The typical use of the derived test suite will be to take apart specific generated test cases, and augment them with application-specific assertions, capturing specific fault-sensitive click trails.
23Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Tool Implementation
• Crawljax: crawling through dynamic states• ATUSA: plugin-based framework on top of Crawljax:
• preCrawling• inCrawling• postCrawling
• Java 1.6• Watij (IE), XULRunner (Mozilla)• Numerous libraries• Java Plugin Framework• Open source
24Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Tool Implementation
25Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Empirical Evaluation
• 4 case studies (3 open source), 1 industrial• TUDU, FreakTask, PetStore, CoachJeZelf
• Instrumented the server side code with Clover• Instrumented the client-side JavaScript code with
JSCoverage• Fault seeding and code coverage
• Code Coverage: 73% server-side, 70% client-side• More than 180 Violations per case• 80% fault detection
26Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Concluding Remarks
We have proposed a method for testing Ajax applications automatically through
1. A crawler that can detect clickables and forms and analyze state changes
2. A testing framework that can check the application behavior through generic invariants as well as application-specific invariants
3. A test suite generation from the inferred state machine.
• Tools are available for download: http://spci.st.ewi.tudelft.nl/content/software
27Dynamic Analysis and Testing of Ajax User Interfaces
| 26
Publications
• Ali Mesbah, Engin Bozdag, and Arie van Deursen (2008). Crawling Ajax by Inferring User Interface State Changes. In Proceedings of the 8th International Conference on Web Engineering (ICWE'08), IEEE Computer Society.
• Ali Mesbah and Arie van Deursen (2009). Invariant-Based Automatic Testing of Ajax User Interfaces. In Proceedings of the 31st International Conference on Software Engineering (ICSE'09), IEEE Computer Society.