Formal Appraisal of Undergraduates – worth the effort? Deborah Murdoch-Eaton Professor of Medical Education Leeds

Formal Appraisal of Undergraduates –worth the effort?

Deborah Murdoch-EatonProfessor of Medical Education

Leeds

http://www.ltsn-01.ac.uk

The workshop

• Who are we?

• What do you want to get out of the day?


Proposed Outcomes

• Explored whether formal appraisal might be of benefit for students &/or faculty

• developed a model for undergraduates

• Considered whether it might be applicable in your institution ie “worth the effort”?


What is appraisal?

• 1 to 1 interview

• Purpose?

(Sam Scallan review of educational appraisal)


Purpose of Appraisal

• Review performance

• Provide opportunity to discuss progress

• Develop personal and professional development needs

• (contribute to University requirements for PDP)


Is appraisal different for UG and PG and CPD?

• Purpose ?

• Type of goals set?

• Skills of reflection ?• Students?• Appraiser

• Appraiser skills


Group work

• Benefits to student

• Benefits to faculty

• What stage of UG career might it be considered most valuable?

• Who should do the appraisals?


• Feedback from groups


Benefits to student?

• Individuality• Reassurance re progress and grades• Life at university• Feedback – on more than academic

criteria

• Achieve potential• Model for future ? Reflective practice?


“A more positive feeling about how I am doing. I truly believed I was doing badly, and without this appraisal I would have maintained this disheartening and negative feeling”


Benefits to faculty?

• Detection

• Alternative view on course

• Enhance student support system?


Who gets appraised

• Does it depend on the medical school

• Type of curriculum?

• All students?

• Selected?• Performance• Year• Self-need


Who are the appraisers?

• Different from PG and / or CPD model?

• Chosen• Knowledge of curriculum• Interest / understanding

• trained


Coffee break


methodology

• When do you do it?

• Preparation

• Documentation of outcomes?


methodology

• How and when

• Preparatory material• Student• Appraiser

• outcomes


Leeds Model

• When• After Easter break

• Who• All 1st and 2nd years, “selected” 3rd years• Is it “optional”?

(Murdoch Eaton & Levene Med Teacher 2004)


Student preparation

• Written guidelines

• Preparatory questions

• Summative and formative assessment marks• Individual

• (Study habits survey – data from 2002 + 2003

available)


Preparatory questions for students – work / course

• do you think the results you obtained were a fair reflection of the work you put in?

• Think about how you worked – was it always appropriate and efficient? Give an example of where it went well, what you are pleased about and why.


Preparatory questions for students work / course

• Is there anything over the year you didn’t do as well in or learn as much from as you could have? How might you approach something like this differently again?


Preparatory questions for students – career choice

• Has anything this year changed your views on what being a doctor is about?

• Either made you less or more enthused about your career choice?


Preparatory questions for students – personal change

• How would you measure a successful next year? What would you like to do to ensure this happens (both personally and in your UG career)?

• Is there anything else you would like to discuss – personal or professional development?


Study Habits Survey

• Who knows how hard a student works?

• What is the “norm”?


Study Habits Survey

• One representative week in term 2• Document

• Number of hours private study(all course related activities which are unsupervised)

• Where, with whom

• Employment?• Hours/week on sport / social activities?

(Whittle S R & Murdoch Eaton D G. Study Habits audit. Medical Education 2004 (38) 566-567)


Study Habits SurveyNumber of Hours spent on self-directed learning

0

5

10

15

20

25

30

0-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 >40

Hours per week

Nu

mb

er

of

stu

den

ts

Series1


Study Habits SurveyHours spent sports / societies

0

5

10

15

20

25

30

0 1-2 3-4 5-6 7-8 9-10 11-12 >12

Hours per week

Nu

mb

er

of

stu

den

ts


Use of Study Habits Survey Results

• Not the absolute values for comparison

• Relative in relation to• Academic results• Outcomes• Personal objectives

• i.e. A diagnostic tool??


Personal Tutor involvement

• Sent copy of students results

• Form to complete• Relevant information• Not breaking confidentiality


For the Appraisers

• Environment• Information for appraisers

• Student specific: Individual results appraisal record – (yr 2

+ 3)

PT report• General : preparatory guidelines prev results study habits survey

• Appraisal record form to complete with student


methodology

• How and when

• Preparatory material• Student• Appraiser

• Recording the outcomes


APPRAISAL RECORD FORM – year 1

STUDENT NAME APPRAISER DATE TOPIC OUTCOME NOTES Overview of the year’s academic progress, including assessment outcomes

Positive evaluation: What went well during the year, what achieved of last years’ objectives

Personal learning needs identified

Activities planned / to be considered over next academic year

Student signature Appraiser signature Refer to Dean


Qns on bottom of Appraisal record form

• Option to refer to Dean / Sub-Dean of students

• Use of progress file

• Contact / met with Personal Tutor

• Consider for 3rd year appraisal


Appraisal record form

What happens to the information?

• Student

• Personal tutor

• Student file

• Signed by both appraiser and student


Lunch break


Issues raised in appraisal

2 main outcome themes

• altered perception on personal progress

• Altered perception of faculty

…….feel like a person, not just a statistic – the Med school

does care!


Video clips


Tea break


So is appraising undergraduates worth the

effort?Cost effectiveness ?

Students ~ 600Appraisers ~ 28

84 X 3 hour sessions = 252 hours senior academics time

Administration - complex


Appraisers perspective

• all would do it again

• Personally rewarding

• Timing about right

• Training of future appraisers?


Effect on Students?

Student Feedback

• Questionnaire• Quantitative and qualitative questions

Student Outcomes

• Analysis of Record of Appraisal Outcomes


Outcomes – student perspective

Questionnaire – quantitative analysis

response rate (86%)

• 96% well informed about process

• 99% they could be honest in answers• 99% topics covered were relevant• 96% feedback was useful and accurate


Outcomes – student perspectiveQuestionnaire – quantitative analysis

(2)

• 94% able to discuss academic problems• 98% appraisal record fair representation

of the points discussed

• 62% appraisal feedback would alter their study habits

• 33% appraisal feedback would alter attitude towards medical career


Outcomes – student perspectiveQuestionnaire – qualitative

analysis

• Majority very satisfied with appraisal session

• Perspective of medical school changed• “feel like I’m a person, not just a statistic”• “the Med School does care”


“2 things you will take away”

• Personal• Confidence, reassurance

• Work/social balance• Healthy balance

• Study skills advice• Time management, efficiency of working

• Academic progress• C grade is fine


“Two things you will take away”

“A more positive feeling about how I am doing. I truly believed I was doing badly, and without this appraisal I would have maintained this disheartening and negative feeling”


But does it change anything?

Longer term study

To identify from Appraisal Record Forms• Pattern of advice and areas discussed

• Year 1 appraisal• Year 2 appraisal

• What of year 1 goals are achieved by year 2• What are new goals for year 2, and what

goals are carried forward from year 1 to 2• Correlation between factors


Analysis of Appraisal Outcomes

2 cohorts of students completed both 1st and 2nd year appraisals N=511

1. Anonymous student feedback questionnaires(88 % response rate)

2. Appraisal record forms (100% completion)


Feedback questionnaire - key points

Overwhelming positive perception of the value of appraisal in both years

• 59.2% overall considered issues covered would alter study habits

• 51% of 2nd years considered they had made changes based on previous years appraisal


Practice Points – key advice required early in undergraduates’ career

• Study skills guidance was the commonest need

• Some significant correlations were seen between types of advice required

• Evolution of personal goals - changing course requirements and maturation

• Early opportunity to address healthy work-leisure balance


Undergraduate appraisal

• Worth the effort?

• We think so

• Data would suggest that it is, from both students and faculty perspective

• The future?• Later years????


• Any questions / comments

Thank you for coming and participating! I hope you found it valuable.

[email protected]

Documents

Formal Appraisal of Undergraduates – worth the effort? Deborah Murdoch-Eaton Professor of Medical Education Leeds