User research in the development of PoliMedia

Max Kemman

Erasmus University Rotterdam

User research

• Understanding the user

– Acceptance

– Performance

– Capabilities

– Weaknesses

• Goal

– Creating a system that is intuitive and helpful to the users

PoliMedia symposium: User research in the development of PoliMedia

2

User research in the development process

• Review related search systems– Staten Generaal Digitaal– Political Mashup– Newspaper database KB– Academia

• Learn search behaviour from users– Survey regarding search strategies– Interviews

• User wishes → user requirements• Wireframes• Prototype• Evaluation

PoliMedia symposium: User research in the development of PoliMedia

3

SurveyGeneral search strategies

• N=294

• Popular search engines

Very often

Often

Regularly

Sometimes

Never

Don’t know it G

oo

gle

Go

ogl

e Im

age

s

Go

ogl

e S

cho

lar

You

Tub

eJS

TOR

KB

Flic

kr

EBSC

ON

atio

naa

l Arc

hie

f

We

b o

f K

no

wle

dge

Uit

zen

din

g G

em

ist

Yah

oo

!B

ing

Aca

de

mia

.nl

Euro

pe

ana

Sco

pu

s

Mic

roso

ft A

cad

em

ic S

ear

chEU

scre

en

Ark

yve

s

4

SurveyGeneral search strategies

1. Keywords 4,752. Advanced search 3,363. Related terms 2,524. Boolean 2,425. Browsing subject

categories 2,296. Filters 2,197. Thesaurus 1,878. Visualization 1,22

PoliMedia symposium: User research in the development of PoliMedia

5

SurveyConclusions

• Google is the dominant search engine

• This has two consequences

1. People compare other search systems to their experience with Google

2. The search task is mainly performed by using keywords

PoliMedia symposium: User research in the development of PoliMedia

6

Interviews

• N=5

• Quantitative (n=2) as well as qualitative (n=4)

• Main themes– How do people search currently?

– What could be improved about current search systems?

– What should PoliMedia offer, given its goals?

• Results– 39 user wishes

– Prioritized internally • 19 user wishes deemed out of scope

• 20 user requirements

PoliMedia symposium: User research in the development of PoliMedia

7

Interviews Findings

• Key issue is to provide a good overview of data – Why are search results retrieved– How are search results ranked

• Assumptions of relevance– Higher frequency of keywords indicated higher relevancy to query?– Longer segments (speeches and articles) indicate higher importance?

• Many more or less out-of-scope wishes to make current research easier– Sentiment-metadata– Context metadata

• Roles of people• Links to external databases• Types of documents• Types of presentation (dramatic, humoristic, etc.)

– Ability to export to own software

PoliMedia symposium: User research in the development of PoliMedia

8

• Clear and immediate keyword-search

• Support for Booleans and (some) Google-search operators

• Separate advanced-search

PoliMedia symposium: User research in the development of PoliMedia

9

WireframesSearch interface

WireframesSearch results

• Keyword search remains prominent

• User chosen ranking of results

• Keyword highlighting

• Overview of related media

• Support for filtering

PoliMedia symposium: User research in the development of PoliMedia

10

WireframesDebate page

• Keyword search remains prominent

• Overview of people in debate

• Easy access to related material

11

Evaluation

• General usability evaluation

– N=5

– Check user requirements

– General usability issues

• Search behaviour evaluation

– What role do facets play in the search behaviour?

– What is the influence of a uniform search interface?

PoliMedia symposium: User research in the development of PoliMedia

12