Digital Resource Study | UC Berkeley Why Study Users? Use and Users of Digital Resources in Humanities/Social Science (H/SS) Undergraduate Education Diane

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y Why Study Users? Use and Users of Digital Resources in Humanities/Social Science (H/SS)

Undergraduate Education

Diane Harley, Ph.D.Principal Investigator; Senior Researcher

Center for Studies in Higher Education (CSHE)University of California, Berkeley

Full Report/Project Website: http://cshe.berkeley.edu/research/digitalresourcestudy/

Copyright Diane Harley 2007. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational

purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Educause/ELI Conference: Atlanta, GA. January 23, 2007

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y

© 2006, Center for Studies in Higher Education | Diane Harley, Ph.D. | http://cshe.berkeley.edu/research/digitalresourcestudy 1.23.07

Research Project Overview

Three-year project (2003 – 2006)

Funding: A.W. Mellon and Hewlett Foundations

Goals: Environmental Scan

• Describe and map the vast universe of digital resources, uses, and users in the H/SS.

• Determine how digital resources are currently used or NOT used in H/SS undergraduate education.

• Explore how (or if) an understanding of use and users can help the integration of resources into teaching/ learning environments.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Why Bother?

Strategic Planning and Investments- More attention has been paid to supply than demand.

- How do you pay for something that you are giving away for free?

- Who supports “non-affiliated,” “non-matriculated,” “non-paying” users?

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Why Bother?

Focus on the Humanities and Social Sciences-Understand variation in user and non-user types by discipline and institution—focusing on a subset of the HE disciplinary landscape

-H/SS different than science and technical courses?

-H/SS not a monolith

-Primary source material and communication tools important

-Future role of technology in the delivery of general/liberal arts education?

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Why Bother?Consolidating Effective Strategies for Understanding Use

and Users Across Projects

• Is it possible, or even desirable, to have projects share methods and results?

• Challenge: Variation among projects along numerous dimensions

– Content, objectives, and targeted users

– Funding models and need for understanding users

– Knowledge of users (and methods for studying them)

• Lack of clear picture about users makes coordination (of user methods, findings, business models, strategic planning) across projects challenging.

• Importance of codifying content, users, and contexts

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Research Activities

• Ongoing discussion with faculty, librarians, educational technology professionals, and resource/site owners.

• Faculty survey:– Sampling opinions about digital resource use among

various disciplines and institutions.

• Consolidating knowledge about users of online educational resources (OER), and convening site owners, funders, and use researchers.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Associated Research Challenges

How can digital resources be defined?Challenge: Agreeing on a working definition/many different perspectives/stakeholders.

• Objects that employ rich media and span text, images, sound, maps, video, and many other formats.

• Sources include collections developed by large institutional entities (e.g., libraries and museums), those developed by individual scholars, canned curricula, and everything in between.

• Particularly interested in free, unrestricted content, but considered any resource faculty say they use, regardless of whether it is restricted or not—“authentic use”

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Ask Faculty: Survey

• Survey design and implementation:– Random stratified sample of:

• Calif. community colleges (CCs)• Liberal arts colleges (LACs)• University of California campuses (UCs)

– Range of H/SS disciplines – Challenges: IRB, finding/tracking respondents– Online survey: Pros and cons– Response rates (~18-20%: 830/4400)– Determining representativeness of sample—non-response survey– External check: HNet survey (~ 450 responses/~350 completes)

• Rigorous methodology is resource intensive, but reduces misinterpretation and guesswork

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Survey Instrument

Discussion Groups and Surveys

Four overarching questions:– What digital resources do you use in undergraduate

teaching?– How do you use them?– What obstacles do you encounter?– In a perfect world, what would you do with digital

resources?

We assiduously avoided judgments about “value” of specific resources.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


A Complex Universe

• Digital collections vary in type, purpose, and perceived value.– Dizzying range of objects used, including personal

collections. Mix of free and commercial resources.– Used for wide range of educational “purposes” and goals

• Variation in faculty enthusiasm and involvement.– A semblance of non, light, heavy, and fundamental users.

Some actively negative—“they can’t substitute for the teaching methods I use”

• Different disciplines/institutions, different needs

• Wide range of obstacles to using resources —cultural, economic, as well as technical.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Types of Resources Used• Images or visual materials

(75%)• News or other media sources

and archives (64%)*• "Portals" that provide links or

URL's relevant to particular disciplinary topics (63%)

• Online reference resources (62%)

• Digital film or video (62%)• Maps (53%)• Online or digitized documents

(50%)• Audio materials (46%)• Curricular materials and

websites that are created by other faculty and/or other institutions (35%)*

• Digital readers or coursepacks (30%)*

• Online class discussions (28%)*

• Government documents (27%)• Data archives (27%)• Digital facsimiles of ancient or

historical manuscripts (23%)• Simulations or animations

(19%)*• Personal online diaries/blogs

(9%)

(Write-ins < 2%: JSTOR, Google, OED, LOC, Lexis/Nexis, Project Muse, Perseus, MLA)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Faculty survey responses by discipline: What resources people use (principal components)

-0.12

-0.18

0.35

0.00

0.19

0.05

0.89

-0.01

-0.67

0.11

-0.05

-0.18

0.88

0.27

0.02

-0.20

-0.74

-0.16

0.99

0.02

0.44

0.00

-0.75

0.03

0.48

-0.09

0.53

-0.31

-0.25

-0.11

0.15

-0.13

-0.32

-0.28

-0.08

-0.35

0.20

0.51

0.97

-0.38

-0.34

-0.02

0.28

0.45

-0.26

0.16

0.51

-0.77

0.00

0.48

-1.50 -1.00 -0.50 0.00 0.50 1.00 1.50

Anthropology & archaeology Art & architecture HistoryPolitical science Writing Foreign languageLiterature & English language Geography Ethnic, gender, & cultural st.Media st. & communic.

Dat

a, n

ews/

med

ia, a

nd

gove

rnm

enta

l res

ourc

esD

iscu

ssio

n a

nd c

urr

icula

r m

ater

ials

Gen

eral

-purp

ose

and

refe

rence

mat

eria

lsH

isto

rica

l doc

um

ents

, map

s,

and p

rim

ary

sourc

esIm

ages

and a

udio

visu

al

mat

eria

ls

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Curricular Materials x Discipline

Curricular materials • Aggregate: 35%• CC’s higher @ 45%

Curricular Materials x Discipline• Foreign Language 49%• Writing 47 • Art/Architecture 46• Geography 44• Anthropology 35 • Literature/Language 31 • History 28• Political Science 21

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Where People Find Resources

• Search Engines (e.g., Google; 81%)

• Personal Collections (69%)

• Public Online Image Database (62%)

• Online Journals (62%)*

• Media Sites (58%)*

• Library Collections (58%)*

• Portals (55%)

• Online exhibits (37%)*

• Campus Image Databases (24%)

• Commercial Image databases (9%)*

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Where People Find x Institution

77%

69%

62%

5%

21%

53%

36%

57%

65%

52%

88%

64%

60%

10%

31%

59%

43%

70%

80%

60%

87%

71%

62%

15%

26%

58%

37%

52%

46%

69%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

6a. digital resources from... Search

engines/directories

6b. digital resources from... My own personal

collection of digital materials

6c. digital resources from... Public online image

databases

6d. digital resources from... Commercial image

databases

6e. digital resources from... Campus image

databases from my own institution

6f. digital resources from... "Portals" that provide

links or URL's relevant to particular disciplinary

topics

6g. digital resources from... Online exhibits

6h. digital resources from... Library collections

6i. digital resources from... Online journals

6j. digital resources from... Media sites

% of respondents (at least sometimes): Univ of Calif (N=522)

Liberal arts (N=90)Community Coll (N=206)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


How Are Resources Used in Teaching?

Aggregate Data

- Presented during my lectures/class (e.g., images, audio, etc.) (71%)

- Assigned to students for review and/or study (59%)

- Assigned for student research projects or problem-based learning assignments (56%)

- Posted directly on/linked from my course website (52%; 49%)

- Used in tests and quizzes (25%)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Motivations for Using

Aggregate

- To integrate primary source material into the course (78%)

- It improves my students’ learning (78%)

- To provide students a context for a topic (75%)

- To get students excited about a topic (73%)

- Allows me to do new things in the classroom (68%)

- Provides access to materials I don’t have at my college (61%)

- To teach critical thinking (56%)*

- To teach info lit skills (56%)*

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Motivation for Use x Institution (1 of 2)

72%

69%

76%

53%

25%

36%

47%

21%

77%

75%

82%

60%

35%

54%

61%

13%

81%

84%

82%

59%

49%

56%

75%

27%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

12a. ...to provide students a context for a topic.

12b. ...to get students excited about a topic.

12c. ...to integrate primary source material into the

course.

12d. ...to integrate my research interests into my course.

12e. ...to provide students with both good and bad

examples of different kinds of scholarship.

12f. ...to teach information literacy (i.e., evaluating the

online materials themselves).

12g. ...to teach critical thinking skills.

12h. ...to provide students a preview of the course before

they register.

% of respondents (at least sometimes): Univ of Calif (N=522)Liberal arts (N=90)Community Coll (N=206)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Faculty survey responses by discipline: How people use digital resources (principal components)

-0.23

0.40

-0.05

0.24

0.52

0.06

0.12

0.31

-0.18

0.05

-0.06

0.47

-0.31

0.29

-0.08

-0.17

0.32

-0.21

0.44

-0.51

0.13

0.12

-0.12

0.01

0.03

-0.28

-0.07

-0.31

-0.03

0.01

0.28

0.62

-0.37

0.10

-0.08

0.17

0.14

-0.22

0.55

0.18

-1.50 -1.00 -0.50 0.00 0.50 1.00 1.50

Anthropology & archaeology Art & architecture HistoryPolitical science Writing Foreign languageLiterature & English language Geography Ethnic, gender, & cultural st.Media st. & communic.

Studen

t ass

ignm

ents

Web

pos

ting

Online

cours

esIn

-cla

ss p

rese

nta

tion

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Barriers & Frustrations

Aggregate

- They cannot substitute for the teaching approaches I use. (75%)- I don’t have time (66%)- I don’t have reliable access to physical resources in my classroom.

(53%)- They distract from the core goals of my teaching (47%)- The digital resources are distributed in so many places that it is

difficult for me to organize them for use in my teaching. (45%)- There are too many resources out there for me to take advantage of

— I am overwhelmed.(44%)- I don’t have time to assess the credibility of the available resources.

(43%)- The content I need or want is just not available online. (41%)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


• Given a teaching style and materials that require one-on-one and/or group discussions regularly, … the computer is a poor substitute for being in a classroom where ideas bong off of each other and where we “talk” as people to people, where I can see body language, and where I can manage the flow.—Literature instructor, UCLA

• I find digital technology inherently alienating and a distraction from the sense of human community and interpersonal communication I try to create.—Writing and art instructor, UC Santa Barbara

• I think there is a real danger of students’ becoming too computer literate and “connected” in ways that undermine, or at least compete with, other crucial skills: argumentative writing, careful and critical reading of long texts, and oral argument.—Political science instructor, UCLA

• Frankly, I just don’t really want to use digital resources. What’s wrong with books anyway?—History instructor, UC Berkeley

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Faculty survey responses by discipline: Reasons for use and non-use (principal components)

0.05

-0.07

-0.25

0.01

-0.14

0.04

0.39

0.16 0.20

0.09

-0.32

-0.11

0.40

0.10

0.08

0.03

0.31

-0.22

0.08

-0.03

-0.25

0.07

-0.10

0.07

-0.27

-0.40

0.09

-0.21

0.10

-0.17

-0.02

-0.07

-0.10

0.54

0.31

0.11

0.22

0.61

-0.14

-0.44

0.39

0.06

0.19

0.03

0.12

-0.40

0.06

-0.10

-0.18

-0.01

0.26

0.07

-0.22

0.21

-0.11

-0.16

0.21

-0.30

-0.98

-0.02

0.26

0.26

0.12

0.11

0.27

-0.19

-0.03

0.36

-0.35

-0.14

0.28

-0.19

-0.18

0.16

-0.46

0.13

0.09

0.18

0.35

-0.35

-1.50-1.00

-0.500.00

0.501.00

1.50

Anthrop

ology & archaeology

Art &

architecture

History

Political science

Writing

Foreign language

Literatu

re & E

nglish language

Geograp

hyEthnic, gend

er, & cu

ltural st.

Med

ia st. & com

munic.

Pedagogical reasons

Inappropriate-ness

Concerns re: students'

interpretation and info. literacy

Time, convenience, and

access

Teaching info. literacy and

critical thinking

Making information

publicly available

Using free and publicly available

materials

Expectations and reputation

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


• I came across an adage that “email allows me to do in one hour what I never had to do before.” So it goes with course WWW sites and digital instructional media too. —Art history instructor, UC Berkeley

• I am excited about the possibilities here to truly enhance teaching. For me the primary stumbling block is in having the time to explore and evaluate sources, not a lack of sources or a lack of belief that quality resources are out there.—English and writing instructor, Mendocino Community College

• I have not devoted enough time to finding out what is out there. I feel like I need a sabbatical just to learn to make efficient use of digital matter.—History instructor, Pomona College

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Activities for Which Support or Assistance is Important

- Obtaining or setting up technical infrastructure (servers, computers, smart classrooms, etc.) (82%)

- Creating my own website; importing resources into my website (82%; 79%)

- Digitizing existing resources (80%)

- Learning how to use; integrating resources into a LMS (79%; 78%)

- Gathering, organizing, maintaining materials (78%)

- Interpreting/securing copyright (65%)

- Assessing credibility of; evaluating appropriateness of resources

(42%; 50%)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


• I find that the computer in class anchors me to a certain spot and at times to a certain order of presentation. I need freedom to improvise, change direction, and physically move around…Finally, I hate the tension that equipment introduces into the classroom, the fear of breakdown, the suspense, the frequent waste of time.…

—English instructor, UC Santa Barbara

• The physical teaching facility is a big issue. I am currently carrying my laptop and projector from classroom to classroom and having to reconnect two or three times per teaching day. The rooms in which I teach have no online hook-ups, which is also a limitation. The physical burden of this technology can sometimes discourage me from using it.

—History instructor, LA Harbor Community College

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Personal Collections

90%

42%

86%

46%

78%

27%

63%

41%

70%

35%

78%

43%

62%

31%

72%

50%

73%

32%

73%

22%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

9a. I gather or maintain my own collection of digital resources.

9b. I make my own digital resources available to others via the internet.

% of respondents (at least sometimes): Anthro & archaeology (N=65)

Art & architecture (N=102)

History (N=99)

Political science (N=90)

Writing (N-33)

Foreign language (N=93)

Lit & English (N=235)

Geography (N=18)

Ethnic, gender, & cultural st. (N=26)

Media studies (N=13)

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Good Enough?

• Faculty use a variety of strategies for negotiating the digital morass.

• Path of least resistance is the one usually taken. EASY trumps all.

• Google search, a walk down the hall, an e-mail to a colleague/family, visit to the website of a trusted archive, or ones own personal and eclectic collection of digital stuff.

• What’s good enough for users?

Depends on problem; single individual may have different standards and strategies that are determined by the immediate objective, time constraints, budgets, personal and institutional equipment, and support staff, among other variables.

• One-size-fits-all programs or resources unlikely to serve the wide range of potential users.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Challenges: Building Tools for Reaggregating Resources

• Difficulty of reaggregating objects that are bundled into fixed, often proprietary resources (incl w/in LMSs);

• Managing and interpreting digital rights (pulling data from one resource for integration into another);

• Unevenness of interface usability and aesthetics (and need for high-end);

• Growing demand from users for granularity (e.g., the ability to search/find particular image or piece of text within a large resource);

• Ubiquity of faculty “personal collections” and lack of support for digitizing, managing, preserving, integrating;

• Knowing about and finding digital objects—some have no idea about the availability of local and non-local resources (esp. licensed resources).

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Advice for Researchers

• Differentiate among types of OER content; the category of OER needs to be refined.

• Differentiate among OER users and the contexts in which OER can be potentially used.

• Differentiate users with different skill levels and learning objectives.

• Study non-users.

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y

Use of Digital Resources in Humanities and Social Science Undergraduate Education

Diane Harley, Ph.D.

Principal Investigator

Center for Studies in Higher Education (CSHE), UC Berkeley

Full Report/Project Website: http://cshe.berkeley.edu/research/digitalresourcestudy/

January 23, 2007

Dig

ital R

esou

rce S

tud

y |

UC

Berk

ele

y


Acknowledgements

Research Team:

Jonathan Henke, M.I.M.S.

Shannon Lawrence, M.A.

David Nasatir, Ph.D., Ian Miller, Cara Bautista

Charis Kaskiris, Alison Head, Ph.D., Sarah Ellinger

Sara Pashmfaroosh, Judy Lien

Documents

Digital Resource Study | UC Berkeley Why Study Users? Use and Users of Digital Resources in Humanities/Social Science (H/SS) Undergraduate Education Diane