8 th of February 2006WPG1 Stakeholders Usage Reporting - STUR aka NetTracker Upgrade

8th of February 2006 WPG 1

Stakeholders Usage Reporting - STUR

aka NetTracker Upgrade


STUR

Goal of the project Why the need? Problems/Solutions

How does it work?

Closer look at some issues

Project scope

Project planning


STUR – Need for this tool

NetTracker is a web analysis tool

>> NetTracker processes web server log data

• How many people visit my web site/web page?• How long does a person stay on my web site?• How successful is my campaign/newsletter?• Did the success of our site increase/decrease

compared to last month/year?• …


STUR – Problems/Solutions

Problems with the “old” version• Long time to process requests • Service sometimes unavailable• Data unreliable• Interface too complex


“New” version• Long time to process requests

> Data stored in robust Oracle database instead of proprietary NetTracker database supports larger data amounts provides faster querying more open – can be combined with other databases

> Need to understand product better• Service sometimes unavailable

> Server COR overloaded – migration to other servers• Data unreliable

> Close and automatic monitoring, distribute responsibility• Interface too complex

> Different roles

> Possibilities to customise further

STUR – Problems/Solutions


STUR – How does it work?

Web analysis tool – processed web server log files


Purpose of web log analysis• Make sense of log file data• Create a picture of what really happened

When did a user log on? Which pages did he visit? When did his visit end? …

Based on concepts of view and visit



57.250.245.249 - - [05/Feb/2006:00:32:28 +0100] "GET /corporate/gallery/content/public/homepage/imgs/civilmil.gif HTTP/1.1" 200 16977 "http://www.eurocontrol.int/corporate/public/subsite_homepage/index.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FreeprodTB; .NET CLR 2.0.50727)" GET /corporate/gallery/content/public/homepage/imgs/civilmil.gif - "HTTP/1.1" -

57.250.245.249 - - [05/Feb/2006:00:32:28 +0100] "GET /corporate/gallery/content/public/homepage/imgs/civilmil.gif HTTP/1.1" 200 16977 "http://www.eurocontrol.int/corporate/public/subsite_homepage/index.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FreeprodTB; .NET CLR 2.0.50727)" GET /corporate/gallery/content/public/homepage/imgs/civilmil.gif - "HTTP/1.1" -

57.250.245.249 - - [05/Feb/2006:00:32:28 +0100] "GET /corporate/gallery/content/public/homepage/imgs/arrow.gif HTTP/1.1" 200 81 "http://www

.eurocontrol.int/corporate/public/subsite_homepage/index.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FreeprodTB; .NET CLR 2.0

.50727)" GET /corporate/gallery/content/public/homepage/imgs/arrow.gif - "HTTP/1.1" -

84.193.8.202 - - [05/Feb/2006:00:32:30 +0100] "GET /muac/gallery/content/public/images/photobanner.jpg HTTP/1.1" 304 - "http://www.eurocont

rol.int/muac/public/standard_page/PDphotoGallery.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.0.370

5; Hotbar 4.6.1)" GET /muac/gallery/content/public/images/photobanner.jpg - "HTTP/1.1" -

57.250.245.249 - - [05/Feb/2006:00:32:28 +0100] "GET /corporate/gallery/content/public/homepage/imgs/dash_hor.gif HTTP/1.1" 200 56 "http://www.eurocontrol.int/corporate/public/subsite_homepage/index.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FreeprodTB; .NET CLR2.0.50727)" GET /corporate/gallery/content/public/homepage/imgs/dash_hor.gif - "HTTP/1.1" -

57.250.245.249 - - [05/Feb/2006:00:32:28 +0100] "GET /corporate/gallery/content/public/homepage/imgs/dash_hor.gif HTTP/1.1" 200 56 "http://www.eurocontrol.int/corporate/public/subsite_homepage/index.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FreeprodTB; .NET CLR2.0.50727)" GET /corporate/gallery/content/public/homepage/imgs/dash_hor.gif - "HTTP/1.1" -

84.193.8.202 - - [05/Feb/2006:00:32:29 +0100] "GET /muac/public/standard_page/PDphotoGallery.html HTTP/1.1" 200 24739 "http://www.eurocontr

ol.int/muac/public/standard_page/PublicationsAndDocumentation.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NE

T CLR 1.0.3705; Hotbar 4.6.1)" GET /muac/public/standard_page/PDphotoGallery.html - "HTTP/1.1" -



Hit= a request to a webserver= 1 line in the log file= many hits per click!View= load of a simple page= approximation of click by a userVisit= approximation of a visit (begin until end) to a website

by 1 user



How do we get the data representation which matches reality?

• Filter on gif, jpeg, css, js, …• Spiders and robots (Google, Eurocontrol search)• Combine clicks from identical hosts, with same

browser, clicking from 1 page to another in 1 visit

Not an exact science!Comparisons with other products/implementations are

difficult!



STUR - How does it work?

NetTracker:

“If it’s in the logfile, NetTracker can show it!”

NetTracker processes log files daily• Different web servers at Eurocontrol receive

requests: e.g. Public web site, ORBITe, OneSky Online, OneSkyTeams, CFMU


STUR – Data unreliable

Log file data is copied daily to NetTracker server (responsibility of SOM)

A script monitors:• If the log file is present• If it is not the same as the previous day• Notifies the web server responsible and the

NetTracker manager

File is uploaded into NetTracker database

Dashboard and custom reports are prepopulated.


STUR – interface too complex

• Average User Avoid that user is overwhelmed Data in compact view Limited possibilities CS AWG requirement list Customisation to subbranches

• Power User Ad hoc queries Drill down functionalities


STUR – Understand NetTracker better

• Ad hoc queries Filter: 1 or more pages Compare volumes Scenario’s

• Frequent need – save reports Updated automatically Data available without delay Can be made available to all


STUR – Other functionalities

• Look up wrong links• List of pages which are never visited• Combination with other databases


STUR – project

• Upgrade• Training• Support

SOM: transfer of files OCCS: first line support AST/COS: second line support

• Information/Documentation on intranet• Communication


STUR – planning

• 1/1/2006 – close monitoring• Migration to bigger server for database – 15/3/2006• Communication by 31/3/2006• On need basis - training

Documents

8 th of February 2006WPG1 Stakeholders Usage Reporting - STUR aka NetTracker Upgrade