How do UK students, researchers and academics use the Web?

Caroline WilliamsExecutive Director of Intute and Mimas Deputy Director

Lisa CharnockIntute Communications Officer, Mimas

Overview

1. Project Fusion market research2. Intute Web 2.0 and ViM project

market research3. What does this mean for online

information service development?

Project Fusion

4

Background

Project Fusion was conceived as means of developing a

system by which data from various online resources can

be aggregated for the benefit of postgraduate students.

Qualitative research commissioned to inform this project,

testing the thinking and concepts behind it.

In brief, the research concerned the way that

postgraduate humanities students conduct and manage

their online research.

5

Objectives

To understand the way in which postgraduate humanities

researchers conduct and manage online research;

To explore if (and how) an online tool could help them

work more efficiently;

To obtain feedback on how such a tool should be

designed (including, but not limited to, feedback on:

structure; user interface; range of resources; search

facilities and potential for personalisation).

6

Key question…

How do humanities researchers prefer

to receive Mimas services?

…by aggregated search facilities or by

links to other Mimas services?

7

Focus group details

3 focus groups conducted, all with PhD students: University of Bristol (2nd March 2009)

University of Manchester (3rd March 2009)

Manchester Metropolitan University (3rd March 2009)

Nine respondents in each group

Wide range of humanities-based PhDs: the majority of

respondents were at least one year into their PhD

All groups included a mix of ages, genders and nationalities

Each session lasted approximately 1½ hours

Discussions based on a broadly-structured discussion guide

Mock-ups of possible Mimas user interfaces presented to

groups

88

Discussion process:

Introductions and ‘warm-up’

Outline and discussion of online research practices

Discussion of Mimas: awareness,

familiarity, individual services etc. Views on possible

aggregation of services – prompted by interface mock-

ups

1

2

Discussion of common

frustrations, problems etc.

3

4

5

Respondents kept research ‘diaries’ ahead of groups: used to prompt

these discussions…

9

Typical research practices (1)

Books (including searches to check/complete references)

Journal articles, conference papers etc. (often with name of author

and/or title and/or date of publication)

Broad searches by subject and/or keywords (e.g. “city spaces”;

“geography and the avant-garde”; “school effectiveness”; “studying

youth gangs”)

Searches for works by a particular author

“I did one on ‘geography and the avant-garde’. Google Scholar was a bit useless – lots of stuff not related to it. Then I went to Web of Knowledge and

geography journals and that was better.”

“I did one on ‘geography and the avant-garde’. Google Scholar was a bit useless – lots of stuff not related to it. Then I went to Web of Knowledge and

geography journals and that was better.”

“At this stage I’m writing-up. I was trying to find a reference I’d included – missing dates. I looked up the library catalogue.”

“At this stage I’m writing-up. I was trying to find a reference I’d included – missing dates. I looked up the library catalogue.”

Searching processes and choice of resource depends on the nature

of the search. Most common searches:

10

Typical research practices (2) All see Google and Google Scholar as indispensable tools, used

extensively throughout their studies

However, other resources are used as appropriate to the nature of

the search being undertaken

All agreed that they vary the way they search and the resources

they used depending on what they are searching for

Certainly, the ‘centrifugal’ model of information gathering

(‘Scholarly Work and the Shaping of Digital Access’; Carole

Palmer; 2005) in which scholars move from lead to lead is

relevant in many search situations…

11

Commonly used resources / search engines (1)

Very widely used and/or familiar to almost all: Google Google Scholar Google Books Amazon (.co and .com) JSTOR

Widely used and/or familiar to many: ProQuest (dissertations) Web of Knowledge Web of Science

Discipline-specific: ERIC (education research) Literature Online 18th Century Collections Online Oxford Dictionary of National Biography Online ScienceDirect DBLP (computer science) ABI Inform (business) RIBM (business)

12

Typical research practices – common themes:

Mindmap

Resources: General and Specialised – used for different

purposes

Importance of habit

Habits deepen over time

Importance of ‘search inside’ Some supervisor

influence – especially early on

Expectation of direct access online to resources, docs,

journals etc.

Lack of contact / chances to refresh skills and range of

resources used

Postgraduate students: ‘private

worlds’

Unsophisticated use of ICT No strong desire to

change

Little personalisation /

saving / organisation of

searches

Constant anxiety about delivering new research

Little awareness or usage of alerts

(journals, articles etc.)

(However…)

13

Common themes:Satisfaction with ‘research skills’

Respondents admitted to variable research skills

Clear lack of confidence evident in some:

Knowledge of relevant resources

Using search-engines (especially optimising keyword search success)

Most admit that there are opportunities to improve skills…but many

have not taken advantage of these

14

Common themes:Many practices are habitual

Respondents admitted that they develop habits in relation to the

way they conduct online research…

…and in particular, they rely heavily on a fairly narrow range of

resources

Since many work in a largely private way, having little contact or

dialogue with others, habits tend to deepen over time

It was perhaps telling that during the focus groups, many

respondents showed interest in the resources mentioned by others

– even in some cases taking notes

“You fall into habits don’t you?...once you fall into a habit

it’s difficult to break it.”

“You fall into habits don’t you?...once you fall into a habit

it’s difficult to break it.”

15

Common themes:Importance of ‘search inside’

A very common theme, across all three groups was the usefulness

of ‘search inside’ features

This feature enables respondents to have a clearer idea of the

relevance of a particular book or article

Google Books and Amazon are both extensively used for this

purpose

“I really like Google Books – I find that very

useful. Previews are really helpful.”

“I really like Google Books – I find that very

useful. Previews are really helpful.”

“Another alternative is to use the search inside feature on Amazon –

especially Amazon.com.”

“Another alternative is to use the search inside feature on Amazon –

especially Amazon.com.”

“Search inside is so useful.”

“Search inside is so useful.”

“I also use Amazon – it lets you see if the book is

useful.”

“I also use Amazon – it lets you see if the book is

useful.”

16

Common themes:Relatively unsophisticated use of ICT

Although all respondents were heavily reliant on computers to

facilitate their research, it does not appear that the majority are

‘sophisticated’ in the way they use ICT

For the most part during discussions, interest was shown in new

resources, rather than in new ways of technology-assisted working

(with the possible exception of email alerts)

17

Common themes:Personalising and organising searches (1)

Consequently, very few respondents had gone to the trouble of

personalising their searches:

Searches done on a session-by-session basis – often without saving

With only a handful of exceptions, none had developed personalised

home search pages (for example, using iGoogle)

Some usage of favourites and limited usage of bookmarks

“I don’t like personalised homepages.”

“I don’t like personalised homepages.”

“…the hassle of setting it

up.”

“…the hassle of setting it

up.”

“I set it up a while ago but I stopped using it – I just ended up with garbage – hyperlinks

everywhere.”

“I set it up a while ago but I stopped using it – I just ended up with garbage – hyperlinks

everywhere.”

“I keep a record of the keywords I’ve used –

that’s about it.”

“I keep a record of the keywords I’ve used –

that’s about it.”

18

Common themes:Personalising and organising searches (2)

A handful mentioned MetaLib for searching library catalogues – with

slightly mixed opinions:

Overall however, there just did not appear to be much of an appetite

for organising searches (by bringing together the most commonly

used resources, by saving searches or by being generally more

organised):

“I can’t think of anywhere that would have everything I need. You’re only one click away with Google.”

“I can’t think of anywhere that would have everything I need. You’re only one click away with Google.”

“You create a list of your own databases that you use. I have six databases that I use most

– some that I have open all the time.”

“You create a list of your own databases that you use. I have six databases that I use most

– some that I have open all the time.”

Intute Web 2.0 and ViM

20

Background

Mindset commissioned by Intute to undertake two closely-related

research projects evaluating specific aspects of the way in which

users and potential users view Intute and to explore a number of

ways in which Intute might be developed for the benefit of key

audiences in the UK HE sector (undergraduates; postgraduates;

academics and library / information professionals)

The two projects were:

Value for Money in Automatic Metadata Generation (ViM)

Intute Web 2.0

21

Objectives:ViM – Specific objectives

What bibliographic and descriptive metadata fields are essential

and desirable?

How useful is the description?

Can user reviews and ratings serve the same purpose?

Are the ‘best’ descriptions summaries or reviews?

Are there any disciplinary differences?

Are there differences between undergraduates and postgraduates?

22

Objectives:Web 2.0 – overall purpose of

research…

To understand how the HE communities interact with

potential Web 2.0 approaches to the delivery of Intute

services: if they will be used; how they could be used

and what aspects are used for what purpose.

23

Focus group details Mindset conducted a series of mini-focus groups

Majority of session comprised 4-6 respondents

Research conducted across four areas of the UK: Bristol,

Birmingham, Manchester and Edinburgh

Sessions held in the summer recess, so in each location students

represented a very wide range of institutions

Focus groups based around discussion guides drawn-up by

Mindset and agreed in advance with Intute

Focus groups typically each lasted 1¼ to 1½ hours

Respondents were not recruited to be existing users of Intute…

Indeed, in the majority of cases, prior to be invited to participate in

the study, respondents had little or no awareness or experience of

Intute. Librarians and information professionals were the exception

to this rule: almost all of this group were very familiar with Intute.

24

Key findings: sections

General thoughts on and reactions to Intute

Intute Web 2.0

VIM

Using the internet

25

Using the internetInternet and research skills training

All audiences in agreement that:

Undergraduate – and sometimes postgraduate – skills in this area are

often very limited

Many fall back on bad habits (e.g. Googling everything)

Once habits and patterns are formed – including reliance on particular

resources – they are rarely changed

Few opportunities to squeeze proper training and instruction into

timetable – and relatively few courses have research skills as an

embedded module

26

Using the internetOrganising searches

Typical practices – level of organisation, tools used etc. - varies

widely…

…and appears to be based as much on the personality of the student

as on the course they are studying:

Minority catalogue and save meticulously

Most use favourites and bookmarks to some extent

Some rarely or ever save search results on their computer

Discussion also about the increasingly disposable nature of

information – some question relevance of saving searches in a

‘dynamic’ world

27

Using the internetLibrarians’, PGs’ and academics’ thoughts on

UGs Widespread view that undergraduates lack research skills and are

overly reliant on:

Google

Specified resources

Discussion among librarians about the reluctance of some academics

for librarians to be too involved in ‘teaching’

A welcome – and rare – experience is for academics to request

assistance in this area

“Typically they don’t get much guidance how to venture wider than

their reading list.” (Librarian)

“Typically they don’t get much guidance how to venture wider than

their reading list.” (Librarian)

“It’s nice to get academics who will come to the class or will ask you to do

a class just for them and their colleagues.” (Librarian)

“It’s nice to get academics who will come to the class or will ask you to do

a class just for them and their colleagues.” (Librarian)

28

Web 2.0 technologies:Understanding and familiarity

The term ‘Web 2.0’ was largely unfamiliar to students

There has however been widespread adoption of Web 2.0

technologies by this group, primarily through social networking

sites and to a more limited extent, through Web 2.0 approaches

used in an academic context (for example, discussion boards on a

VLE)

In contrast, amongst librarians, utilisation of Web 2.0 in education

was something of a ‘hot topic’

29

Web 2.0 technologies:Appropriateness of adoption in

education Clear and consistent finding across all groups was that although these

technologies and approaches have a place, they should not be adopted

simply because the technology exists

Majority of respondents dismissive of many aspects of Web 2.0

technologies or admitted to be novices in their use: (for example, few

respondents organise internet browsing beyond perhaps adding useful sites

to favourites; few used iGoogle – most had not even heard of it etc.)

Although Facebook was predictably popular, its usage limited in most cases

to being a way of staying in touch and finding out about social events

Some students spoke of Facebook groups being set up as part of their

studies but few if any were heavily reliant on applications of this type

30

Web 2.0 technologies:Appropriateness of adoption in

education Most suggested that they were not in the habit of ‘rating and

commenting’ (for example, on Amazon) and, to an extent were

suspicious of those who were

Additionally, it was explained that in many ways, there are

practical barriers to the adoption of interactive technologies in HE

institutions:

Some academics and lecturers are traditional in outlook and working

practices – and this is the audience that is most influential

To an extent, students wish to separate working and social dimensions of

their lives

Typically ICT is tightly controlled within universities which results in…

31

Web 2.0 technologies:Librarian community views

Very strong sense that the library community is currently trying to work out

how to proceed with Web 2.0 technologies – in fact, for some, this was a

key motivation for attending

Much interest in how other institutions are using emerging technologies

Apparent that developments are characterised by experimentation with

small projects – rather than a co-ordinated and strategic roll-out

A key benefit is that potentially students and lecturers can be in more

constant dialogue: useful at postgraduate level where a ‘wrong turn’ can be

especially damaging if not corrected early on

In short, support for Web 2.0 technologies where they offer choice, without

any evidence yet of ‘becoming mainstream’…

32

ViM

33

Overview of common search practices

Although many students are making use of their institutions’ own

resources, catalogues and data-base searching tools, more generic

search tools – and Google in particular – remain very popular

Course students at all levels use resources such as Google Scholar,

Web of Knowledge and JSTOR

However, groups across all audiences acknowledged that there is an

over reliance on generic search engines amongst the student

community

Most students will continue to use Google as their initial searching

tool and explain this in terms of habit, convenience and value in

scoping research requirements

34

Overview of common search practices

Several respondents mentioned using Amazon as a way of finding

related books – “If you like this, you might like this...”(PG)

Considered a useful tool – “The more you use it the more it gets to

understand you.” (PG)

With very few exceptions, undergraduate respondents admitted that

they are largely ignorant of how to undertake effective internet

searches

In most cases, respondents claimed that their typical practice would

be to ‘throw a few key words’ at a search engine and quickly scan

the results – without thinking too carefully about what is being

searched and how the results are presented

35

Evaluating search results (1)

For most, this is a fairly subjective process

It can vary significantly by discipline. For example:

Medical / law subjects – users have typically identified sources

which are provided for them; scientific / technical / computer based

subject areas – typically require a ‘black & white’ response to a

query and thus credibility and source are considered less important

Humanities and social sciences – often need to read widely around

a subject including reviews of sources outside the academic domain

– credibility of source is therefore more relevant here and perhaps

more difficult to evaluate

36

Evaluating search results (2) Many undergraduates simply rely on the ‘first 10’ results – believing

that, for example Google will deliver the most relevant results first. At

most, students might review three or four pages of results

Preferred two or three lines of description

Other initial ways of quickly evaluating the relevance and potential

usefulness of results include:

Checking the url (with, for example, an ‘.ac.uk’ domain being an

indicator of a credible resource)

Recognised subject authorities / governing bodies etc.

37

User ratings / reviews (1) A strong reaction – and one which was consistent across almost all

groups and audiences – was that user ratings and reviews have only

limited appeal

Most would have a quick look, but on the whole would disregard

them due to numerous identified concerns:

Who are the users and how can students be certain that relevance is a

universal quality? (i.e. what might be relevant to one student might not

be so to another)

Propensity to give ratings is not equal across all users. In crude terms

there is a certain type of person more inclined to rate and comment and

respondents felt that such a person is not necessarily a reliable

barometer of usefulness and relevance to all

38

Popularity ratings Popularity ratings based on site hit rates are also seen as potentially

misleading

Perception that they do not indicate who actually used the resource

but merely who opened the page

Additionally usefulness is not equal to popularity

“Hit rates don’t really add anything from an

academic point of view, what one person finds

useful has no bearing on anyone else.” (PG)

“Hit rates don’t really add anything from an

academic point of view, what one person finds

useful has no bearing on anyone else.” (PG)

“”Popularity is just another bit

of information, it’s up to you

whether you use it or not.”

(UG)

“”Popularity is just another bit

of information, it’s up to you

whether you use it or not.”

(UG)

“Hit rates can be misleading; most people

know you have to be a bit careful with this

type of information.” (UG)

“Hit rates can be misleading; most people

know you have to be a bit careful with this

type of information.” (UG)

What does this mean for online information services?

1. Don’t clutter things that won’t be used or would potentially undermine trust

2. Raise awareness of range of resources3. Do support development of Internet

research skills and digital literacy4. Be mindful of preferences for distinctions

between the social and work/study exchanges on the web