Assessment of the Impact of Ubiquitous Computing on

Learning

Ross A. GriffithWake Forest University

Ubiquitous Computing Conference

Seton Hall UniversitySouth Orange, NJ

January 4-6, 2001

Wake Forest UniversityFall 2000

Programs Enrollment (HC) Undergraduate 3944 Graduate 584 Divinity 50 Law 476 MBA 628 Medicine 440 Allied Health 136 Total

6258

Overview of Presentation

• Major Elements of the Strategic Plan

• Assessment Structure

• The Computing Initiative

• Changes by Students and Faculty

• Academic Outcomes

• Summary

Major Elements of the Strategic Plan

• IBM laptop computers provided to all entering freshmen and faculty effective with the fall semester 1996

• A new first-year seminar, ensuring each freshman an in-depth intellectual encounter effective with the fall semester 1996

• 40 new tenure-track faculty members

representing a 15 percent increase

• Scholarships for 175 students to study

abroad

• Fellowships for 150 students to perform

joint research with faculty members

Major Elements of the Strategic Plan (continued)

Assessment Structure

Evaluation Committee consisting of faculty and administration formed to assess effectiveness of the strategic plan.

Evaluation Committee sanctioned:

• College Student Experiences Questionnaire (CSEQ)• Cooperative Institutional Research Program (CIRP)

Freshman Survey

• Freshman Essay

Evaluation Committee sanctioned: (continued)

• Higher Education Data Sharing (HEDS) Consortium Alumni/ae Survey

• HEDS Senior Survey

• Higher Education Research Institute (HERI) Faculty Survey

• In-house Faculty Computer Survey

• In-house Faculty Survey

• In-house Student Computer Survey

• Wake Forest Fact Book

• Wake Forest Key Measures of Quality

Evaluation Committee

• Reviews Fact Book and Key Measures

• Analyzes survey data

• Communicates results

• Conducts follow-up

Evaluation Committee sanctioned: (continued)

The Computing Initiative

• IBM laptop computers provided to all entering freshmen and faculty

• Standing Faculty Committee on Information Technology formed

• Entire campus wired

• Created new position of Academic Computing Specialist (ACS)

The Computing Initiative(continued)

• Information Systems Support Center (ISSC) revamped

• Student and faculty training provided by library

• Computer-Enhanced Learning Initiative (CELI) formed by faculty

• Student Technology AdvisoRS (STARS) created

Changes by Students and Faculty College Student Experiences Questionnaire

(CSEQ)

• The survey directed by George Kuh of the University of Indiana was administered to a random sample of one-half of the freshmen, sophomores and juniors in March of 1996, 1997 and 1998 and all freshmen, sophomores and juniors on the web in spring 2000.

• The main purpose of the survey is to determine the activities of students with respect to how they spend their time.

Items for 2000 CSEQ with Significantly Higher Mean Scores than 1998 CSEQ

Time spent:Used computer to prepare reports or papersUsed e-mail to communicate with instructor/other studentsUsed computer tutorial to learn material for a courseSearched the WWW or Internet for course materialUsed computer to retrieve material from outside libraryUsed computer to produce visual displaysUsed computer to analyze dataDeveloped a Web page/multimedia presentation

Items for 2000 CSEQ with Significantly Higher Mean Scores than 1998 CSEQ (continued)

Institutional Emphasis:Information literacy skills (using computers)

Made gains:Use Computers and other information technology

28 other “intellectually developing” items

Items for 2000 CSEQ with Significantly Lower Mean Scores than

1998 CSEQ

Time spent:Participate in class discussion via electronic mediumUse dictionary or thesaurusUse campus learning lab to improve study skillsUse campus recreational facilitiesHad discussions w/students of different valuesOpinion:Overall opinion of collegeAttend same institutional again

CSEQ - Quality of Effort: Computer and Information Technology Scale

Sum of Means for Nine Computer Items

23.2922.73

21.87

20.67

21.99

20.73

25.15

181920212223242526

2000 WF 1998 WF RU DU CCU SLA GLA

HEDS Senior Survey

• The HEDS Senior Survey was administered to the Classes of 1993 through 1998 and the Class of 2000 in the spring of their senior year.

• The HEDS Senior Survey is an excellent survey for indicating the degree of educational enhancement and satisfaction in a number of areas.

• The Class of 2000 is the first class to graduate under the Plan for the Class of 2000 while the Senior Survey results are compared to the Class of 1998 as well as a College Group and a University Group of peer institutions.

HEDS Senior SurveyUse of Quantitative Tools - Enhancement

Mean

2.94

2.722.66

2.76

2.5

2.6

2.7

2.8

2.9

3

WFU 2000 WFU 1998 College Peer Group 2000

University PeerGroup 2000

HEDS Senior SurveyUse of Technology – Enhancement

Mean

3.40

2.923.02

2.50

2.70

2.90

3.10

3.30

3.50

WFU 2000 College Peer Group 2000 University Peer Group 2000

HEDS Senior Survey - 2000 Wake Forest Items Significantly Above 1998 Wake Forest,

2000 College and 2000 University

Enhancement:Use Quantitative ToolsRead or Speak Foreign Language

BelowSatisfaction:Relive college experience at same institutionSocial life on campusEthnic/Racial diversityClimate for minority students on campus

In-house Faculty Survey

• The Faculty Survey was developed by the evaluation committee to assess the effectiveness of all portions of the Wake Forest strategic plan.

• The survey was administered to the undergraduate faculty in December 1995 and February 1998.

In-house Faculty Survey Results Mean Scores Significantly Higher:

1998 vs. 1995

• Computers in teaching

• Computers in communication

• Computers in individual instruction

• Computers for presentations

• Computers with information gathering

• Computers for modeling/simulation

• Computer skill

• Computer training & assistance

In-house Faculty Survey Results Mean Scores Significantly Higher

1998 vs. 1995 (continued)

• Students proficient with computers

• Technology changed effectiveness of teaching

• Effect of computers on communication

• Effect of computers on resource material

• Effect of computers on presentations

• Use of technology in teaching

• Intellectual climate among students

• Religious development of students

• Prepare students for graduate/advanced education

In-house Faculty Survey Results Mean Scores Significantly Higher:

1998 vs. 1995 (continued)

• Prepare students for employment after college

• End of course student evaluations

• Enroll more graduate students

In-house Faculty Survey Results Mean Scores Significantly Lower:

1998 vs. 1995

• Number of papers or compositions published since previous fall

• Number of professional meetings attended since previous fall

1998 HERI Faculty Survey

• Administered by UCLA in Fall 1998 to faculty nationally

• Several questions contained items regarding computer use and opinions

• Wake Forest full-time undergraduate faculty results compared with peer group of nine private institutions

Use of Computers(1998 HERI Faculty Survey)

62

93

22

36

48

97

71

40

31

12

39

98

91

75

59

41

0 20 40 60 80 100

Participate in on-line discussion groups

Conduct data analysis

Create presentations

Conduct research on internet

Work from home

Conduct scholarly research

Write memos/letters

Communication via e-mail

% F

aculty

Percent using computers at least twice a week

Wake ForestPeer Group

Sources of Stress(1998 HERI Faculty Survey)

7 12 17 22 27 32

Colleagues

Institutional procedures and "red tape"

Committee work

Teaching load

Keeping up with info technology

Personal finances

Review/Promotion process

Research/Publishing demands

% F

aculty

Percent indicating "extensive"Wake ForestPeer Group

Opinion in General(1998 HERI Faculty Survey)

55

36

22 20

37

1913

28 31

59

8

18

28

38

48

58

68

Diverse studentbody enhances

education

Tenure attractsbest to

academe

Computersenhance

student learning

Encouragestudents to do

communityservice

Western Civfoundation of

undergradcurriculum

Percent indicating "agree strongly"

% F

acu

lty

Wake Forest Peer Group

Academic Outcomes First-time Freshmen Retention

Following Year

Year Entered

94.3

91.0

93.4

94.5

92.0

89

90

91

92

93

94

95

1995 1996 1997 1998 1999

Average GPAEnd of Freshman Year

2.87

2.83

2.81

2.89

2.77

2.72.72

2.742.76

2.782.8

2.82

2.842.86

2.882.9

1995 1996 1997 1998 1999Year Entered

Summary

• Student & faculty computer usage has increased significantly since implementation of ubiquitous computing at Wake Forest

• Student & faculty computer usage is higher than peer group institutions

• Students are more engaged intellectually but are less satisfied with their overall college experience

• Freshmen GPA and retention rate have increased slightly overall

Assessment of the Impact of Ubiquitous Computing on

Learning

Ross A. GriffithWake Forest University

Ubiquitous Computing Conference

Seton Hall UniversitySouth Orange, NJ

January 4-6, 2001