Quality Assessment II Peter Dolog dolog [at] cs [dot] aau [dot] dk 2.2.05 Intelligent Web and...

Quality Assessment II

Peter Dologdolog [at] cs [dot] aau [dot] dk2.2.05Intelligent Web and Information SystemsNovember 2, 2010

Outline

Design models and testingUsage analysisStatistical testing

2Peter Dolog, Web Engineering, Quality Assesment

Design Models and Quality Assessment

Metrics for navigation

Navigation maps: size, depth, breadth, compactnessNavigation context: coupling of nodes, intrinsic

complexity

Can be checked in navigation design modelsFor example if represented in XML, analysis can be

done by XSLT

Correctness of navigation

Syntacticall the constructs are correct and consistent with respect to a given syntax

Semanticnon-determinismracing conditionsdeadlocks

Usability

Consistency:consistent use of patterns in content composition, navigation, and function invocation

Ease of Navigation:availability of links – from associations between data, in addition links to home, and other related upper pages to avoid dead ends.

Low page density:number of content units at the page

Modify Pattern

Termination Variant

Other design based techniques

See the last lecture:White box : Data flow basedBlack Box

From last lecture: performance testing

Testing Loop

11Peter Dolog, Web Engineering 2010, Quality Assesment I

Results

Additional response time testing and stressing

White box testing (2)

White Box Testing

Web Usage Analysis

Typical Web Log

Client IPTimestampMethod of transaction (Get, Post, …)URL requestedProtocol version (e.g HTTP1.1)HTTP return codeSize of response returned to clientCookie at the client browserURL of referrerClient used to access (e.g. Firefox, Mozzila, IE, …)

Usage Analysis

We can mine for related pages from the requestsA typical query would be:

Give me all URL’s accessed in one session by usersRank them according to number of occurance patterns in the log

Good for the web sites with less complex and static pages.

If the patterns correlate with the links then we are fine

However, for complex applications, more fine grained log structure is neccassary to assess the usage

You want to typically find out whetheryour designed links are followed or notwhether your pages are reachablewhich of the pages are not used and whywhich data items are usedare there any anomalies, i.e. invoking functions after strange navigation sequenceand so on

Goal’s achievements

You are able to do this if:you know the composition of pageswhich data entities and records are inhow they are processedwhich operations and functions are invokeddo data fulfil their role (central role, access role, interconnection)how the links are related to the items and so on

Design Model

Data Schema (In MVC Model)Hypertext Schema (In MVC Model and partly

Controller)Presentation Model (In MVC View and partly

Controller)…

You can extend the web log with these information

WebML QA approach

Design Schema Analysis

Performed at the design timeChecks for some quality attributes (see last

lectures)There are various metrics to check for those

attributes (see P. Fraternali et. al.: WQA: An XSL Framework for Analyzing the Quality of Web Applications. IWWOST 2002 workshop proceedings.)

Web Usage Analysis

Data Access AnalysisHypertext Access AnalysisNavigation Analysis

Web Usage Mining

Discovers patterns of navigating connections which are not designed

Results in rules: • XML Association Rule X=>Y• XML Sequential Pattern, where rule head and

body are bound to a position in the log statement (indicates temporal relation)

Information Extraction

You can extend your WA logs as well

http://logging.apache.org/log4j/1.2/index.html

To insert log about an event from application runtime

Such as user interacting with data unit, or part of a page, … when populating it and sending as a result of a request

Concepetual Log

WebML QA approach

Consistency Analysis

Most Accessed Entities

Missing Links

Statistical Testing

Web• Huge user population• Complex Architecture• Renders traditional coverage-based testing less

appropriateStatistical Testing

• Generation of test cases from usage data• Targeting high risk areas• Focuses on a probability distribution from

application domain

Outcome

Test cases generated randomlyOutcome of the test is used to predict reliability

given the usage profileThe amount of resources reduced significantly

Usual predictions focus on:• Reliability• Time to failure• Mean time between failures• All are of probabilistic nature

Process of Statistical Testing

Construct the statistical testing models or usage models basedon actual usage scenarios and frequencies.

Use these models for test case construction, selection andexecution.

Analyze the test results for reliability assessments andprediction, and for decision-making.

Usage Model

Characterizes the application operationGraph – nodes different states of usage model and

arcs transitions between themProbability distribution may be assigned to the arcs

Operation profiles are used to represent usage models

Help to guide allocation sequences

Unified Markov Model

Based on operation profilesStates and transitionsProbabilities on transitions – best estimate of actual

Web applications: hits and goesVisited pages or collections of related pages – statesUsage patterns/Navigation patterns and their

likelihood – transitions

Can be formed as hierarchical

Example

Hao, J. and Mendes, E. 2006. Usage-based statistical testing of web applications. In Proceedings of the 6th international Conference on Web Engineering (Palo Alto, California, USA, July 11 - 14, 2006). ICWE '06, vol. 263. ACM, New York, NY, 17-24. DOI= http://doi.acm.org/10.1145/1145581.1145585

Testing

Perform exhaustive complete coverage testing for the toplevelmodel.

Perform selective testing for some lower-level models, whichcovers frequently visited sub-parts.

Perform more selective testing for the remaining low levelUMMs.

Test case generation

Treshold set upThose which are above trashold contribute to the

critical navigation pathStart with higher numberLater decrease

Building a UMM

Web logsaccesserror

Calculating Reliability

One possibility is to look at (Nelson Model)number of errors discovered during testingamount of time needed to discover them

Relaibility = 1 – f/n

f is number of errorsn is number of hits for those errors

Mean time between failures = n/f

Quality Assessment II Peter Dolog dolog [at] cs [dot] aau [dot] dk 2.2.05 Intelligent Web and...

Documents

Jó dolog az Urat áldani, énekkel magasztalni nevedet, Felséges;

ERP Course: Human Resources Chapter 7 from Mary Sumner Peter Dolog dolog [at] cs [dot] aau [dot] dk E2-201 Information Systems September 29, 2006

Navigation and Presentation Design Peter Dolog dolog [at] cs [dot] aau [dot] dk 2.2.05 Intelligent Web and Information Systems October 7, 2010

101 dolog, amit jó lett volna tudni a nőkről.pdf

ERP Course: Enterprise Application Integration Readings: Chapter 3 from Gustavo Alonso et al Peter Dolog dolog [at] cs [dot] aau [dot] dk E2-201 Information

Fajta és mennyiség szerinti dolog határidős adásvételi ... · PDF fileFajta és mennyiség szerinti dolog határidős adásvételi szerződése amely létrejött egyrészről

Daniel Pink: Eladni emberi dolog - Honlapom.huhonlapom.hu/pages/korokanmasszazstanfolyam/contents/...Daniel Pink: Eladni emberi dolog Mire van szükség a sikeres eladáshoz? Mindannyian

10 dolog amitől megnyugodsz

Introduction Peter Dolog dolog [at] cs [dot] aau [dot] dk 2.2.05 Intelligent Web and Information Systems September 9, 2010

DM-D100 Inst Oper Manual DL850 Dolog 2X Replacement

TOWARDS THE ADAPTIVE SEMANTIC WEB (Peter Dolog et al)

ERDÉLYI NORBERT: 10 DOLOG, AMIT ROSSZUL TUDSZ · PDF fileerdÉlyi norbert: 10 dolog, amit rosszul tudsz a marketing szÖvegÍrÁsrÓl. futurizmo.hu 1 | 28 erdÉlyi norbert: 10 dolog,

Tárgy Személy Képzelt Dolog Család Mesebeli Történelem

Advanced Web Service Transactions Management Peter Dolog dolog@cs.aau.dk CS Department Intelligent Web Information Systems ://,

33 dolog az IBM-ről

ERP Course: Knowledge Management and Learning See articles at the course web site and in references Peter Dolog dolog [at] cs [dot] aau [dot] dk E2-201

Mi az az egy dolog, ami biztos a tőzsdén?

Dot Dot Dot

14806747 101 Dolog a Gyerekekrol

Requirements II - Business Processes and Workflows Peter Dolog dolog [at] cs [dot] aau [dot] dk 2.2.05 Intelligent Web and Information Systems September