Embed Size (px)
Citation preview
While we are waiting to start please run through the Audio Setup Wizard (one last time )
1.Click (top left)2.Select the Audio Setup Wizard option
3.Follow on-screen instructions4.Let us know if you are having a problem by typing “Help” into the Chat box and we’ll see what we can do
Welcome to today’s Teaching and Learning Conversation Tuesday the 28th of April, 12-1pm UK time
Important NoteFor now you may be able to hear and see Rod and other colleagues in the video pod but not any of the other participants or yourself.
And you will not be able to speak to anyone at this time other than via the chat tool.
We have done this on purpose .
We will take you through the use of main communication tools when we have everyone present.
Please feel free to use the chat tool while we are getting things ready.
Today’s conversation about formative assessment and feedback will be led by
Dr W. Rod Cullen ([email protected])
Outline
• A bit of revision
• The trouble with formative assessment– Unpick a typical assessment scenario
• Pose and discuss some important questions in the context of formative assessment
• Reflect upon experiences of several small scale practice development and action research projects
Asking questions/marking comments:Normally your microphone and webcam will not be enabled during the webinar. You can do the following:
Type your answers into the polling pod and when requested
B Using the status tool
The Webinar host can then enable your microphone option
2. Select required status
3. Click on the mic icon to speak
1. Click on your status
Using audio (if you have a microphone)D
A
Type your questions and/or comments into the Chat pod
Using the Chat pod
Participating and Contributing
Using short answer poling podC
2. Select raise hand
1. Click on your status
Using the MCQ poling podC
A bit of revision• What is a learning outcome ?
– The essential measurable learning or behaviour demonstrated by a student after a specific period of study
• What is formative assessment?– “Formative assessment is designed to provide learners
with feedback on progress and inform development, but does not contribute to the overall assessment.”
QAA 2000
• What is summative assessment ?– The process by which we measure the extent to which a
student can demonstrate specific learning outcomes
designedfeedback developmentalnon-counting
Process measurementlearning outcomes
Measureable ability on completion
Assessment of learning
Assessment for learning
Impact of feedback“Feedback is arguably the most important aspect of the assessment process in raising achievement”
(Bloxham and Boyd, 2007)
“Arguably the most powerful enhancement to learning is feedback during learning”
(Biggs and Tang, 2007)
… however
“academics frequently report frustration that students fail to act on feedback or to collect it at all” (Jollands et al. 2009, Bloxham and Boyd, 2007)
Activity 1
Why do students often fail to act on feedback or to collect it at all?
Type your thoughts, suggestions, comments into the Polling pod
In the final year of his Business Management degree George is set the task of producing an individual 1500 word management report based on case study materials that he has been studying with his tutorial group. The report constitutes 50% of the final unit grade the other 50% is assessed by a 3 hour examination at the end of the semester.
The hand in date for the essay is 6 weeks before the examination. The work is marked within 3 weeks and returned to George with his mark and a detailed written feedback proforma. The assignment is also discussed in a feedback tutorial during the week the work is returned. The report tests two of the 5 learning outcomes for the unit. These learning outcomes are therefore not tested in examination (which tests the remaining three learning outcomes).
George sits the examination for the unit which is marked in time for exam board. George is given the mark awarded for the examination but receives no verbal or written feedback on his performance in the exam.
Activity 2c: How do we want George to use the feedback he receives on his report?
Type answers into the Polling Pod
Typical Assessment Scenario
Will this feedback help me get a better mark for
the report?
Will it help me to do better in the
exam?
“feedback provided by tutors focused on performance on the assignment being assessed”Orsmond et al, 2011
“students often struggle to transfer learning from one unit to another” Orsmond et al. 2011
Biggs and Tang, 2007 do not regard the feedback provided on end of unit summative assessment as formative as the feedback is provided when the unit is effectively finished and students rarely pay attention to comments provided at the end of a course.
Glover and Brown (2006) comment that in terms of written feedback students receive plenty of it, but that it is often misunderstood in relation to assessment criteria.
Orsmond et al. (2005) found that a majority of students preferred verbal feedback from tutors as it enabled questioning and discussion.
Some personal thoughts and lots of questions
Tricky relationship between formative and
summative assessment
Students are very strategic about their
engagement with assessment
The value of formative actives and feedback
in general are often not apparent to students
We tend to design assessment strategies at unit level rather than the programme level
When is feedback most useful to students?
Will students complete work if it doesn’t get a mark?
Can we better motive students to engage with formative assessment and feedback?
What is the most effective way to give feedback?
Do students understand the feedback they receive?
How does feedback enhance learning i.e. how do students use the feedback they receive?
What impact does feedback have on attainment?
Some case study experiences as food for thoughts
Three Case Studies
Tropical Land Use and Conservation (20 credits)2010/11 Level 5 cohort of 40 studentsWritten (2000 word) assignment mid-way through the autumn termDr Francis Brearley
Accounting, Finance & Economics (30 Credits)2011/12 Level 6 cohort of 251 studentsWritten (2000 word) assignment examining theory in relation to real world mergers and acquisitionsWendy and Peter Wild
Employment and People Management (30 credits) 2011/12 Level 5 cohort of 98 studentsA written (2000 word) reflective essayEd Bielinski
• All practice development projects/dissertation topics PG Cert/MA Academic Practice• Opportunistic Action Research
Targeting formative feedback
1. Set Assignment
2. DraftSubmission(Optional)
3. FormativeAssessment
4. Reflection and
Reworking
5. Complete Submission
6. Summative Assessment
“FormativeFeedback”
directly linked to current task
“SummativeFeedback”
or “Feedforward” to inform future
work
Directly linked
Indirectly linked
Not linked
Enhancing Learning Teaching and Assessment with Technology (ELTAT)
Date Unit Title
2008-11 Designing Courses for VLEs (DCVLEs)10 credits15-20 participants per cohort
2011-13 Design Effective Online and Blended Learning (DEBOL)10 credits15-20 participants per cohort
2014-15 Enhancing Learning Teaching and Assessment (ELTAT)30 credits15-20 participants per cohort
Ongoing action research as reflective practice
Formative assessment model
Online Preparation
In ClassSession 1
Onlineformativeactivity
Online Preparation
In ClassSession 2
Onlineformativeactivity
Week 1 Week 2
Feedback Feedback
ePortfolio
Assessment Strategy
•Mini-portfolio of formatively assessed activities
•Built week by week (developmental), underpinning summative assessment task
•Regular, rapid, personalised feedback
•E.g. Individual elements of portfolio provide evidence/basis for summatively assessed reflective account of learning from the unit.
Will students complete work if it doesn’t get a mark?
Can we better motive students to engage with formative assessment
and feedback?
Targeting formative feedback
Course % Non-Sub of Draft
Tropical Land Use and Conservation (20 credits)2010/11 Level 5 cohort of 40 students 37%
Accounting, Finance & Economics 2011/12 Level 6 cohort of 251 students
40%
Employment and People Management (30 credits) 2011/12 Level 5 cohort of 98 students 50%
Formative assessment model
Per
cen
tag
e
DCVLE & DEBOL (10 credits) attendance & formative submission 2008-13
1. Reasonably high overall attendance
2. Generally high (70% +) submission rates for formative tasks
4. Consistently lower submission rates for the last week 4
MASummative Assessment
3. Use of the feedback is embedded in the online prep and F2F activities
ELTAT (30 credits) formative submission 2014-2015
Take home messages
Students need to see the “value” in formative assessment and feedback
Holistic assessment strategy that integrates formative and summative assessment
Clear signposting required
More attention on learning how to learn
What impact does feedback have on attainment?
How does feedback enhance learning i.e. how do students use the
feedback they receive?
Targeting formative feedback
Consistently higher attainment for those who submitted a draft – 1 grade band
Tropical Land Use and Conservation
Brearley F.Q. and Cullen W.R. (2013) Providing formative audio feedback on a written assignment. Bioscience Education eJournal Vol 20, pp 22-36
http://journals.heacademy.ac.uk/doi/abs/10.11120/beej.2012.20000022
Complete4 (16%)
Work in Progress15 (60%)
Outline6 (24%)
Submitted 25 (63%) Non-submitted 15 (37%)
60.8% 52.6%
Sum
mat
ive
gra
de
69.8% 61.6% 51.8%
Fine tuning Reiteration & EvidenceFine tuning & Content
Hand-in
Feedback length
Feedback content
66.6% 61.3% 46.2% 46.2%
Complete4 (16%)
Work in Progress15 (60%)
Outline6 (24%)
Submitted 25 (63%) Non-submitted 15 (37%)
60.8% 52.6%
Sum
mat
ive
gra
de
69.8% 61.6% 51.8%
Fine tuning Reiteration & EvidenceFine tuning & Content
Hand-in
Feedback length
Feedback content
66.6% 61.3% 46.2% 46.2%
1. Time on task2. Better organised/more
strategic3. Improved performance4. More feedback, better
understood, easier to respond to
5. Is this simply making better students better?
1
2
4
3
5
Accounting, Finance and Economics
Take home messages
Quality of submission influences the quality of the feedback
The “timing” of the feedback is maybe more important than the “speed of the feedback” and
“method” of feedback
Where participation is optional the effect may be to make the “best” students better rather than helping
those who most need it academically
More attention on learning how to learn
What is the most effective way to give feedback?
Rationale for audio and video feedback
Student perspectives: Text v Audio v Video
Prefer audio and video to written feedback
Personal and Engaging
Understandable
Annotations important
50:50 split between audio and video
Utilised differently
Video more impact
Audio more reflective
Cullen W. R. (2011) A multi-technology formative assessment strategy, Media-Enhanced Feedback case studies and methods, Proceedings of the Media-Enhanced Feedback event, Sheffield, 27 October 2010 pp 28-33http://ppp.chester.ac.uk/images/4/43/Middleton-Media-enhanced_feedback_proceedings-final.pdf
Take home messages
Preference for audio and video feedback – influenced by the type of task and/or novelty value
Annotations seem to be important for “navigation” purposes
Preference may be influenced by learning styles
Choice of format should be determined by the task and the purpose of the feedback
Thank you for taking part
https://assessmentinhe.wordpress.com/
Assessment in Higher Education Open CourseRachel Forsyth & Rod Cullen
Six weeks from Friday 8 May 20151 Topic, 1 Formative Task and 1 Webinar per week
Just in case section
Where does formative assessment fit into the assessment life cycle
Involving students - How?
SelfAssessment
PeerAssessment
AssignmentBriefings
SettingCriteria
ChoosingTopics
DiscussingCriteria
AssessmentTutorials
StagedAssessments
Procedural Personal Critical
Autonomy
McAtominey & Cullen 2002, Ecclestone 2000
FeedbackTutorials
Linked to:Predefined outcomes,Knowledge and Process
Learners need direction to carry out proceduresand meet outcomes
Able to discuss learning,plan and organise work
Decide when to work alonecollaboratively, seek advice
Self managed + directed
Formulate own learning objectives
Select learning strategies
Reflect on progress and decide next step
Mode and Method of Assessment
2
Methods and Delivery of Teaching and Learning
3
Aims and Intended Learning
Outcomes
1
Exam, Essay, Report, Portfolio, Poster, Reflection, Presentation, MCQs, etc
F2F Online Blended
Underlying Pedagogye.g. PBL, Collaborative learning,
Reflective practice, etc
Lecture, Seminar, Practical, Fieldwork, Lab work
EQAL
Diagnostic Formative Summative
MMU Assessment Framework
Blooms Taxonomy
Assessment Criteria
Marking Process
Feedback Strategy
(ECF) MMU Graduate Outcomes
Requirements of Prof Bodies
Employers requirements
Subject Benchmarks
Programme Learning
Outcomes
EQAL Requirements
MMU Threshold Standards
Constructive Alignment
Why are they important to us?
• Fundamental principle of outcomes-based education (Constructive alignment)
• Advocated since Dearing (1997) by the QAA• Entrenched in Bologna process (1999)• Statutory requirement for programme validation
(programme and unit/module level)• Way we communicate to students what they are
expected to do and achieve• Metrics against which we are required to assess students• Underpin our learning, teaching and assessment
strategies• EQAL is very prescriptive
Acceptable assignments?Essay
Newsletter
Magazine article
Written ReportData Analysis
Event
Oral presentation
Artefact
Web site
Audio reportVideo report
Competition entry Problems to solve
Job Application
Critique
Case Study
Poster
Unseen Writtenexam
Photo Essay
“Dragon’s Den™”
Seen writtenexam
“The Apprentice”
MCQs
Computer based assessment
Practice Observation
Wiki contribution Blogg/learning log
Research Article
Lab Practical
Reflective account
Annotated Map
Annotated Reference ListPortfilio
Translation
DissertationGroup work
Interview
Work based assessment
A Performance Roll play
ReferencesBoud, D. (1995). Assessment and Learning: contradictory or complementary? Assessment for Learning in
Higher Education. P. Knight. London, Kogan Page: 35-48.
Biggs, J. B. and Tang, C. (2007). Teaching for quality learning at university. Open University Press/Mc Graw-Hill Education.
Bloxham, S. and Boyd, P. (2007). Developing Effective Assessment in Higher Education: A Practical Guide. Berkshire, UK. Open University Press.
Cullen W. R., (2011) A multi-technology formative assessment strategy, Media-Enhanced Feedback case studies and methods, Proceedings of the Media-Enhanced Feedback event, Sheffield, 27 October 2010 pp 28-33
Ecclestone, K. (2000) Assessment and Critical Autonomy in Post Compulsory Education in the UK, in, Journal of Education and Work, Vol. 13, No. 2.
Glover, C. and Brown, E. (2006) Written Feedback for Students: too much, too detailed or too incomprehensible to be effective? Bioscience Education Journal (7)
Jollands, M., McCallum, N., Bondy, J. (2009) If students want feedback why don’t they collect their assignments? Proceedings of 20th Australasian Association for Engineering Education Conference University of Adelaide, 6-9th December 2009
McAtominey, D. & Cullen, W.R. (2002) Effective e-Learning with VLE’s, Netskills Workshop Materials
Orsmond, Paul , Merry, Stephen and Reiling, Kevin(2005) 'Biology students' utilization of tutors' formative feedback: a qualitative interview study', Assessment & Evaluation in Higher Education, 30: 4, pp 369 - 386
Orsmond, Paul and Merry, Stephen(2011) 'Feedback alignment: effective and ineffective links between tutors' and students' understanding of coursework feedback', Assessment & Evaluation in Higher Education, 36: 2, pp 125-136.
