Click here to load reader
Upload
roy-williams
View
217
Download
4
Embed Size (px)
DESCRIPTION
Can we customise e-assessment so that it supports learning, rather than just summative certification? Presentation at Portsmouth University
Citation preview
Fine Tuning Assessment for Teaching and Learning
Roy Williams
Michael McCabe
Context
Mathematics student numbers 2002 – 2010:
• 30 to 150 in first year (plus 2nd year?). • Negligible staff increases• Standard Wiley textbook• In-house assessments (Qmark)• PRS feedback (good question bank)• Link to personal tutors (most staff)• Support from Maths Cafe
Questionmark Graphics
2010 Semester 2
• Staffing Changes• New Responsibilities• Short notice for new delivery of course
• Regional Maths Seminar (Sigma-Hub)• Awareness of Wiley Plus resources
• Opportunity to rethink course delivery
QMark: Outdated for Mathematics
• No algorithmic-randomised questions• Technical, authoring and display issues• Answering in mathematical notation
(Limits questions formats)• Cumbersome editing and graphics
updating
Development
• Meeting with Wiley Plus within month• ? Redevelopment in MapleTA?• Decision to do ‘live pilot’ – book plus
e-assessment, e-book, etc. • Two week start-up• Wiley support and resources• Wiley meeting with staff• Wiley meeting with students • Wiley assistance at launch
Advantages
• E-book provided for all • Large banks of pre-prepared questions• Links from assessment to full-text, hints,
solutions, tutorials• Learning design underlying the structure• Pre-prepared lecturer ppts:
– Supplementary PRS questions– PPT Summary (2 screen options)– Applets – interactive activities – Instructor guidance and resources– Exportable MCQ’s >> Victory
Disadvantages
• Learning rather than examination focus• Pre-prepared ppts – formulaic• Limitations on customisation, and in-house Q’s
(MapleTA underlies Wiley Plus)• Error reporting, not correction• 3/7 tests in MapleTA – (different text book)
Assessment
Formative• Pre-prepared• Selected formative (10, mixed difficulty)• Unlimited attempts on algorithmic questions• Varied question types, including interactive graphics
Assessment
Summative• Replaced repeated assessments with
repeated questions • Cumulative scoring
– Builds on success– Promotes confidence– Avoids repetition
• Q. Classification – Easy/ medium/ hard.– By learning objective
• Varied Q. types, including interactive graphics
Disadvantages
• No password facility• Administrative overheads and work-arounds• Long-term continuity, control, stability
– Year-on-year use of same tests– Upward compatibility of software
• Non-uniformity/ comparability of questions– Scoring
– Length
– Amount of work required
– Recommended time-on-task, and marks
• Discriminating between formative and summative
Evaluation Victory
• One-stop shop for information
• Hand-written lecture notes– Lectures as process?– Lectures as mathematical thinking?– Digestible summary of contents
But• Unreliability and GUI Issues• Mathematical limitations
Evaluation
MapleTA
• Ease of Use• Many practice questions
But• Input format ?• Lack of partial credit, inflexible marking
Evaluation Wiley Plus
• Multiple attempts at questions
• Cumulative scoring • Many practice questions • Feedback and links: solutions, hints,
tutorials. • Invigilators calm and helpful
But
• ? Input format• GUI?
Evaluation Useability
• MapleTA very good• Wiley good• Victory: issues
MTH 204
Confirms findings, broadly
Evaluation Method
• Survey on 6 point scale (e.g. essential to learning >> Of no use at all
• Open-text questions (positive and negative)
• MTH 155: 68/150• MTH 204: 11/120
Questions Arising
• To ASP or not to ASP, if so, 100% or not? • Alternative philosophies (OS, Commercial) • Sustainability • Viable hybrid model?• Learning (L&T model – TEL&T)?• Scaffolding – lecturer, interactive resource, both?• Educational benefits
– Repeat assessments or questions?– Integrated multi-mode resources?
• Trade-off: security and admin. load.
• Customisation of resources