16
User needs Vipavc Brvar Irena, ADP, Slovenia IASSIST/IFDO 2005, 24 - 27 May, 2005

U ser needs Vipavc Brvar Irena , ADP, Slovenia IASSIST/IFDO 2005 , 24 - 27 May , 200 5

Embed Size (px)

DESCRIPTION

U ser needs Vipavc Brvar Irena , ADP, Slovenia IASSIST/IFDO 2005 , 24 - 27 May , 200 5. Purpose – EVALUATION The most general definition of evaluation is that it is a process of assessment of the value of a certain activity or objects . - PowerPoint PPT Presentation

Citation preview

User needs

Vipavc Brvar Irena, ADP, Slovenia IASSIST/IFDO

2005, 24 - 27 May, 2005

Purpose – EVALUATION

The most general definition of evaluation is that it is a process of assessment of the value of a certain activity or objects.

In the case of libraries they evaluate how well the library satisfy the needs of its users and basic limitations and weaknesses in the library’s operation. The standard ISO 11620 (1998) defines evaluation as a process of assessing effectiveness, efficiency, usefulness and relevancy of a certain services, equipment or facilities.

Lancaster (1977) states that services (in our case also data archives services), can be evaluated from different viewpoints and that in the process we find:

- How well a service realises the goals set, which usually means how well it satisfies the needs of the users.

- How efficiently a service, system realises the goals set (operating costs).

- Does the data archive justify its existence, that is what is the usefulness of the data archive service in light of operating costs (measuring cost-performance-benefits)

Evaluation processes are important for decision making.

Evaluation is a part of strategic planning process and has practical significance.It is in some cases also necessary because of political reasons (to justify the expenditure of public funding and to show sense in its spending).

- > The future of data archives does not lie in supporting generic, broadly useful services such as information access to large collections, but in supporting “customization by community”, i.e. the development of services tailored to support the specific, and real, practices of different user constituencies.

In data archive area we do not have indicators that would measure our performance nor do we have standard questionnaire/questions that would measure satisfaction of our users with our services.There were some attempts to measure user satisfactions (UK DA, ICPSR, University of Tennessee, etc.).

-> Changes in IT brought to us new expectations from users. We experience change of practice in data distribution and collection, change of behaviour of users, and usability. Users want codebooks and data files to be easy available.

Establishment of data arhive and launching of Web page/Nesstar (Situation in 2005)

RO

GR

IR

FI

IT

CZSI

EECH

AT

HU

ES

SE

DK

NO

GBNL

DE

RO

FI

SI

EE

CH

AT

DKNO

GB

NLDE

1990

1992

1994

1996

1998

2000

2002

2004

2006

1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

Year data archive was established

Web page Nesstar

Decision was made to conduct a survey and evaluate users needs and their satisfaction with offered services.

We do want responses from different groups of users(researchers, users and potential users)

Registered Slovene researchers in the field of social science.

Known users- in our user database

Banner on FacultiesWeb pages

Cover letter by email

How to measure usability- Analytically: counter – log analyzer, tracing

paths,Web site usability evaluation, etc.

- Empirically: survey...

Information from Web statistics (log analysis):

* Number of visits, and number of unique visitors,* Visits duration and last visits,* Days of week and rush hours,* Domains/countries of hosts visitors (pages, hits, KB, 269 domains/countries detected, GeoIp detection),* Hosts list, last visits and unresolved IP addresses list,* Most viewed, entry and exit pages,* OS used,* Browsers used ,* Search engines, keyphrases and keywords used to find your site,* Number of times your site is "added to favourites bookmarks".* Screen size* ......

• Analysis of log file from Nesstar• Analysing user paths

ADP

EN

SLOPrice list

Mail-contactdepositorsOrder

form

List of studies

Particular study

news event

Colleting Study. Info.

Survey on user satisfaction

• Pretest (current stage)• Define Sampling Frame and Contact Design• Decide on data collecting method

Web based survey

Create separate URL addresses for each email address given. Demands for anonymity are satisfied.

Web survey with PHPSurveyorPHPSurveyor is a multi-question surveying tool that allows you to develop, publish and manage surveys. PHPSurveyor script allows you to display surveys as single questions, group by group or all in one page or use a dataentry system for administration of paper-based versions of the survey. PHP Surveyor can produce "branching" surveys (set conditions on whether individual questions will display), can vary the look and feel of your survey through a templating system, and can provide basic statistical analysis of your survey results. -Conversion to SPSS data file was written.

Most of current user satisfaction surveys ask users to tell them how much they are satisfied and how important particular mentioned topic is (5 point scale) .

To shorten the questionnaire we decided to do it a little bit different.We inquire about overall satisfaction (on general, and about web page) and to evaluate particular services.

Discriminant analysis!!-> create indexes that measure satisfaction and importance

Importance

Satisfaction

This need to be

changed!!!!

Relation between satisfaction with and importance of particular element

Questionnaire available athttp://www.adp.fdv.uni-lj.si/publikacije/userneeds.html

Literature- AMBROŽIČ, Melita (2003): A few countries measure impact and

outcomes – most want to measure at least something. In: Performance Measurement and metrics. Vol.4, No.2, p. 64-78.

- BARNES, Stuart J. And VIDGEN, Richard (2001): An Evaluation of Cyber-Bookshops: The WebQual Method. In: International Journal of Electronic Commerce. Fall 2001, Vol.6, No.1., pp.11-30.

- BOLLEN, Johan and LUCE, Rick (2002): Evaluation of Digital Library Impact and User Communities by Analysis of Usage Patterns. In: D-Lib Magazine. Vol. 8, No.6.

- COLEMAN, Anita and SUMNER, Tamara (2004): Digital Libraries and User Nees: Negotiating the Future. In: Journal of Digital Information. Vol 5, Issue 3.

- HERNON, Peter and ALTMAN, Ellen (1998): Assessing service quality: satisfying the expectations of library customers. Chicago, London: American Library Association.

- NORLIN, Elaina (2002): Usability testing for library Web Sites. A hands-on guide. Chicago, London: American Library Association.

- RUBIN, Jeffrey (1994): Handbook of usability testing. How to plan, design, and conduct effective tests. John Wiley & Sons, USA.