Upload
leonard-mallinson
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
ASSESSMENT TOOL OR
EDUTAINMENT TOY
USING PERSONAL RESPONSE DEVICES FOR LIBRARY
INSTRUCTION ASSESSMENT
Patrick Griffis August 6, 2008
Outline
Overview of Personal Response Systems/Clickers
Scalable Options of Clicker Systems
Benefits/Drawbacks for Using Clickers
Using Clickers for Assessment
Experimenting with Clickers/Lessons Learned
Personal Response Systems ‘Clickers’
System allowing instructors to pose questions to students which students can answer anonymously
Anonymous response feature encourages participation from students which enhances student engagement in the classroom and provides instructors immediate feedback
Personal Response Systems were developed as a tool to increase student feedback and engagement in large class settings
Scalable Options: Common Model
Clicker Software + Receiver + Clicker Devices
Standard model
Classroom set of devices and receiver
Instructor responsible for receiver and software
Requires software download to instructor’s computer
Scalable Options: Clicker Only
Clicker Devices Only
LCD screen handheld receiver
Computer and projector not required
Answers from student keypads displayed and stored on handheld receiver and can be exported via USB to a computer
Easiest system to implement/Can be used anywhere
Scalable Options: Clicker Only
Scalable Options: Software Only
Clicker Software + Virtual Response Keypads
Receiver not required and Clicker Devices not required
Local Area Network and/or Wireless Network
Student computers connect via broadcasted IP Address
Requires software download to instructor computer and Virtual Clicker download to student computers
Scalable Options: Software Only
Alternative Web Polling Tools
DyKnow Vision (Licensed Software-One Time Cost)
SurveyMonkey (Subscription Web Survey Tool)
WordPress Polling (Ad On to WordPress Blog Software)
Tool: Clicker Benefits
Can increase student interactivity and engagement in bibliographic instruction with a ‘workshop’ feel
Can help instructor to assess what students already know coming into the session allowing instruction to be tailored to what students don’t know
Can help instructor assess what students have retained/learned in a session as well as areas that need reinforcement
Toy: Clicker Drawbacks
Learning curve for students can take time away from an instruction session which might already be hurting for time
Some instructors feel that Clickers are another of many slick gadgets/technologies that already serve to distract students in the classroom
Clickers good-cell phones bad-mixed message for students
Toy: Clicker Drawbacks
Clicker use more expensive for Libraries which have the burden of purchasing Clicker Devices
More feasible for classes that meet regularly to require students to purchase clicker devices
It costs more to use Clickers for Library Instruction Sessions than for Term Courses and Library Instruction Sessions have a much smaller window of opportunity to use these devices
Nascent need to experiment for feasibility
View from Library Instructors: Tool
“A Personal Response System (PRS), or clickers, is an effective method for gathering assessment data during library instruction sessions” (Page 258)
“A PRS is a useful tool for gathering data and customizing instruction to student needs” (Page 260)
Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.
View from Library Instructors: Toy
“New technology is entertaining but can quickly become the focus of a session if not used as part of an appropriate learning activity” (Page 260)
“We discovered it was important to carefully monitor the amount of time being devoted to clicker questions and evaluate if their use enhanced the instruction” (Page 260)
Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.
Critical Question
Can the use of clickers enhance library instruction enough to be worth the time needed to use them?
5-10 minutes?
10-15 minutes?
Critical Response
Effective Clicker use allows library instructors to identify a few key areas needing most emphasis for student needs.
Reduced need to cover everything
How much time would be saved in a 50 minute session if session student learning objectives were cut by a third or even by half?
Clickers could actually enhance effectiveness of sessions to save time.
An Unexpected Use of Clickers for Library Instruction
Assessment Julian & Benson claim that they
serendipitously discovered that Clickers can be an effective tool for instructional assessment.
Original intention was to add interactivity to instruction sessions to increase student engagement and retention.
Authors began considering using Clickers for instructional assessment.
This purpose will become a new area of consideration and experimentation.
Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.
Critical Question
Is a Clicker System an Effective, Sustainable, Practical Tool for Assessing Library Instruction?
Depends on the type/purpose of Instructional Assessment
Formative Assessment
Summative Assessment
Formative vs. Summative Assessment
Formative assessments are on-going assessments, reviews, and observations in a classroom. Teachers use formative assessment to improve instructional methods and student feedback throughout the teaching and learning process. The results of formative assessments are used to modify and validate instruction.
Summative assessments are typically used to evaluate the
effectiveness of instructional programs and services at the end of an academic year or at a pre-determined time. The goal of summative assessments is to make a judgment of student competency--after an instructional phase is complete. Summative evaluations are used to determine if students have mastered specific competencies and to identify instructional areas that need additional attention.
Critical Questions
Is Instruction Assessment more Formative or Summative?
Are Clicker Systems Designed more for Formative or Summative Assessment?
ACRL Information Literacy Standards [Bloom’s
Taxonomy]“In implementing these standards, institutions need to recognize that different levels of thinking skills are associated with various learning outcomes--and therefore different instruments or methods are essential to assess those outcomes.
For example, both "higher order" and "lower order" thinking skills, based on Bloom’s Taxonomy of Educational Objectives, are evident throughout the outcomes detailed in this document. It is strongly suggested that assessment methods appropriate to the thinking skills associated with each outcome be identified as an integral part of the institution’s implementation plan.”
Bloom’s Taxonomy
Original Revised
Knowledge RememberRecalling the information
Comprehension UnderstandExplain the ideas and/or concepts
Application ApplyUsing the newly acquired knowledge in another familiar situation
Analysis AnalyzeComparing and differentiating between constituent parts.
Evaluation EvaluateJustifying a decision or course of action
Synthesis CreateGenerating new ways of creating products, ideas or ways of viewing things
View from Clicker Instructors
Higher Order & Lower Order Thinking Skills “Different questions elicit different responses and require different
levels of cognitive engagement. Knowledge-level questions ask for simple recall of facts and data without assessing them critically, whereas analysis, synthesis and evaluation questions require critical thinking and judgment.”(Page 5)
“When faculty are simply assessing students’ basic understanding, a knowledge-level question may be appropriate. But when faculty wish to engage students in thinking critically about course content, a knowledge-level question may fall short of reaching the goal.”(Page 5)
Zhu, Erping . "Teaching With Clickers." Center for Research on Learning and Teaching Occasional Papers (2007): 1-7. http://www.crlt.umich.edu/publinks/CRLT_no22.pdf.
Critical Questions
Are Clickers an Effective Instrument for Measuring Information Literacy Outcomes Requiring Higher Order Thinking Skills?
What Type of Questioning Requires Higher Order Thinking Skills?
What Type of Questioning is Possible with Clickers?
Questioning Capabilities of Clickers Initial Models supported Fixed Response
Questions Multiple Choice True/False
Newer Models support Alpha-Numeric Answers 10-15 Character Text Entry LCD Display Screen
What About….. Matching Answers Short Answer/Essay Questions
Will Clickers Evolve to Fully Support Summative
Assessment Applications are being developed to enhance
the Assessment capabilities of Clickers.
TurningPoint’s ‘TestingPoint’ Application
Higher Education Assessment Tool
With TestingPoint, professors and lecturers can create and administer self-paced assignments, quizzes and tests with ease, allowing for real-time summative and formative assessment.
TestingPoint
TestingPoint supports questions in multiple formats: Multiple Choice Multiple Response Numeric Response Matching Fill in the Blank Bimodal True/False Short Answer Essay
http://www.turningtechnologies.com/k12studentresponsesystem/testingpoint.cfm
Summative Assessment Considerations
Clickers were initially designed for Formative Assessment
Instructors have serendipitously experimented with using Clickers for Formative and Summative Assessment
Clicker developers are taking the need for Summative Assessment capabilities to heart
Library Instructors will have opportunity to experiment fully with Clicker Summative Assessment soon
My Own Experience
Chaired a campus committee tasked with evaluating Clicker systems and selecting one to be the campus standard.
Developed knowledge about Clicker systems and a desire to try them in library instruction to increase interactivity and student engagement.
Experimentation Issues: Trial Kit Size
Vendor ‘Trial Kits’ limited in scale
Trial Kits typically no larger than 10 Clicker Devices
Could not experiment with using Clickers in large class settings for which Clicker Systems are intended
Could not negotiate to have a site license trial of a Virtual Clicker
Trial Kits allow for familiarity with a Clicker System before purchase but does not allow for real class feasibility experimentation
Experimentation Issues: Trial Duration
Vendor Trials Limited in Time
Typically one to three month duration
Limited time for experimentation after learning curve
Only enough time to learn how to use the Clicker System leaving little time for significant feasibility experimentation
Experimentation Recommendation
Collaborate with Professors Already Using Clickers
Simply visit their classroom and use their Clicker System Requires familiarity with their Clicker System
Students have Clickers and know how to use them No learning curve for Students taking away instruction time
No cost ideal solution to experimentation issues Real classroom feasibility experimentation
Further Considerations
Experiment with a Personal Response System which is widely used on campus
Which system is used most on campus?
Has your campus adopted a standard system?
If your campus is in the process of adopting a campus standard, try to arrange for a representative from your library system to be on the campus working group/committee
Favorable Conditions for my Experimentation with Clickers in
Library Instruction COM 101 Public Speaking Course has
requirement for library instruction
Director of COM 101 Program was on Clicker Standardization Committee
I have conducted library instruction for these sessions numerous times
Challenges for my Experimentation with Clickers
in Library Instruction COM 101 library instruction sessions have
a graded quiz assignment which is a summative assessment of information literacy learning outcomes
The quiz takes considerable time from instruction sessions already
Quiz questions are in formats not widely supported by Clicker systems
A Silver Lining
COM 101 Course Director considering making the required quiz take home for students
Future Clicker enhancements might fully support a quiz like this making the back end grading work for library instruction assessors much more efficient and sustainable without compromising effectiveness
QUESTIONS/COMMENTS
Patrick GriffisBusiness LibrarianUniversity of Nevada, Las Vegas4505 Maryland ParkwayLas Vegas, NV 89154-7014(702) [email protected]