Keyek-Franssen & Briggs Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support Deborah Keyek-Franssen University of Colorado

Keyek-Franssen & Briggs

Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support

Deborah Keyek-Franssen

University of Colorado at Boulder

Charlotte Briggs

Loyola University Chicago

Copyright Deborah Keyek-Franssen and Charlotte Briggs, 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational

purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written

permission from the author.


Take-aways

• Motivation to think strategically about research and decision making

• Understanding of the breadth of data collection methods

• Some data, just for fun


Strategic Research

• Timing is everything

• Need to think holistically

• Consider teaching and learning needs first, technology second


Backward Marketing

• ‘Backward’ Market Research by Alan Andreasen (1985)

• Know first what decisions you need to make before you begin your research, instead of beginning with a research problem.

• Focus first on why you need to know, and then on what you want to know.


Backward Marketing vs. Traditional Research

Standard research Backward marketing

Define research problem Determine key decisions

Check secondary sources Determine information needs

Determine primary resources

Determine questions to be asked

Research & survey Which questions already answered

Analysis & reporting Research & survey

Analysis & reporting


Defining research problem vs. determining key decisions

“Curiosity” questions

How do we invest our limited resources in instructional technology to have the greatest impact on the teaching &

learning of the greatest number of faculty and students?


Embedded Principles

How do we invest our limited resources in instructional technology to have the greatest impact on the teaching & learning of the greatest number of faculty and students?

1. Focus first on teaching and learning, and then on technology as a tool to support both


Embedded PrinciplesHow do we invest our limited resources in instructional technology to

have the greatest impact on the teaching & learning of the greatest number of faculty and students?

2. Aim for an impact on Roger’s Early Adopters and Early and only slightly Late Majority

Adapted From Sdmagazine.com


Embedded PrinciplesHow do we invest our limited resources in instructional technology to

have the greatest impact on the teaching & learning of the greatest number of faculty and students?

3. Emphasis on undergraduate education4. We don’t know what we don’t know


Less than PerfectBackward Marketing

How will we know what data will result in what decisions?

1. Number of responses?

2. Passion of responses?

3. Differences among schools and colleges?

4. Political position of responses?


Determining information needed

• Still worked within framework of existing technologies: learning management systems, classroom technologies

• For student data, also included communication technologies and administrative applications

• Did *not* ask about video, overhead projectors, slide projectors--because that information wouldn’t influence our decisions.


Sorta Curiosity Questions

• Who’s using what?

• How’s it working?

• What current use and support could be improved?


Strategic Questions

• What would you like to use but aren’t?

• What would enable that use?

• How can technology improve teaching and learning?


Research Partnerships

• Institutional Research

• IT Services

• IT Governance Groups


Why IR?• Expertise• Good citizens• Understanding of survey cycle• Ability to sample• Sweet survey application• Non-IT branding• Staff• Expertise!


Why ITS?• Empirical data• Potential impact to unit


Why IT Governance Groups?• Political vetting• Audience expertise• A nudge toward the strategic


Good Methods, Good Data

• Data mining

• Surveys

• Focus groups


Questions already answered: data mining

• Number of students served by WebCT? (over 23,000)

• How many courses use “clicker” technology? (about 60, with approximately 4000 students total)

• What instructional technology questions come through the help desk? (WebCT password problems and urgent classroom technology glitches)

• How many “smart” classrooms are managed centrally (just over 100)


Survey Design• Faculty survey

– Spring 2005

– About 40% of the faculty (statistical validity & good citizenship), only 33% response rate, lower on certain questions

– Stratified by school & college

• All-faculty survey– Fall 2005

– Somewhat shorter version

– Opportunity for all faculty to respond


Student Survey• Fall 2005• Stratified by year & declared major• 2000 undergraduates (just under 10%)

and 1000 graduates (about 20%)• Low response rate (25%)• Pivot questions to allow additional

information to be gathered about laptop owners, graduate students, and teaching assistants


Student Survey

• Important points– Ownership not as crucial as use (How often do you

bring your laptop to class? For more than taking notes?)

– Want to understand if there is a gap between faculty use (even intended) and student expectations or preferences (student expectations can be a driver for change)

– Teaching assistants asked both student and faculty questions


Effective Practices & Caveats

• Online surveying• Sampling• Short survey• Incentives• Good survey• Pilot, pilot, pilot• Combine questions

• Second bias• Not all voices heard• But, but, but…• Don’t always work• No caveat here!• It’ll cost ya• Increases complexity


Combining Questions


Focus Groups & Interviews

• Cutting edge

• “Normal Joe & Josephine”

• Distributed Academic Technology Coordinators

• Research computing

• Digital Asset Management


Effective Practices & Caveats: Focus Groups

• Conversation• Informal• Focused

• Tentative• Objectivity difficult• Hard to generalize


In-Class Use: Frequency and Importance

Tech use Frequency Important

Laptop as chalk replacement

60%all or regularly

73%

Laptop beyond chalk 33% all or regularly

72%

Feedback systems 15% all or regularly

45%


Online components: Frequency and Importance

Component Frequency Important

Syllabus 60% 65%

Readings 55% 64%

Interactive simulations

14% (24% more would like to use)

33%


Enablers to Using Technology

Enabling factor Would it help?

More time 40%

Training 56%

Money 31%

Staff 30%

Tenure & promotion 27%


Open-ended comments

• Hey! What about VHS, PAL, DVDs, slides, overheads, demonstrations, chalk, and rocks?

• “I still use chalk…nothing beats chalk’s flexibility, at least nothing I’ve seen yet.”


The clicker debate…

• “Clickers…are nothing but a heavy-handed way to force attendance. [They] measure outcome and not process.

• “It is time for the university to realize that this is an important technology and get ITS to support it.”


…morphs into musings about pedagogy.

• “I am not at all persuaded that clickers work better…as the kinds of interactive learning we already use, which do not require technological assistance.”

• “I am not comfortable with pre-prepared presentations that seem to thwart questions that lead to important diversions.”


But all roads lead to laments...

• “I would use the smart classroom stuff more if the equipment worked better.”

• “I’d like tech assistance and repair services available all the time.”

• “[We need] better maintenance of the equipment available in the classrooms.”

• “Updated facilities!”


…and comments to keep us honest.

• “I prefer to spend my time learning the subject matter better. I find instructional technology pretty boring actually.”

• “As with any technologies, these are just tools and it looks like that has been forgotten.”


Work yet to be done:

• In-depth analysis (including open-ended)

• Compare faculty use and desires with student expectations

• All-faculty survey

• Focus groups

• Communication

• Plug-in to the decision-making process


Effective Practices To Go• Know where you’re going (backward

marketing can help)• Make good use of strategic partnerships

on campus• Employ a mix of data collection methods• Aspire to high research standards• Don’t forget the grain of salt.• You’ll find flaws afterwards.• Expect unexpected consequences.


Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support

Deborah Keyek-Franssen

[email protected]

Charlotte Briggs

[email protected]

Documents

Keyek-Franssen & Briggs Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support Deborah Keyek-Franssen University of Colorado