HEARing Student Voices Self Portraits and Perpetual Motion: The Student Experience of Informed Choice and Feedback Jennie Blake, Research Associate Patricia

HEARing Student Voices

Self Portraits and Perpetual Motion: The Student Experience of Informed Choice and Feedback

Jennie Blake, Research Associate

Patricia Clift, Teaching and Learning Manager

University of Manchester

Val Wass

University of Keele

HEARing Student Voices: developing the pedagogy to reflect achievements across the student experience

This project has been funded by the 2009 NTFS Projects Funding strand


Project Background

• The University of Manchester is a HEAR trial institution

• Increasing emphasis on personalisation of learning

• New structures for advising students

• Poor NSS results on assessment and feedback

• Emphasis on volunteering, citizenship and leadership development

• Lack of pedagogy for the HEAR at a formative stage


Research Questions

• The project aims to research how:

– Students make educational choices within differently structured curricula

– Employment aspirations influence choice

– Student centred strategies can develop the HEAR process

– Formative student assessment can influence career management


Project Outline

• Stage one:

– Pilot focus group (~10 students, mixed disciplines, PASS leaders)

– Main focus groups in three contrasting disciplines: English and American Studies, Pharmacy, and Geography (~ 10 students per discipline)

– Transcription and analysis

• Stage two:

– Mixed discipline focus groups

– Transcription and analysis

– Questionnaire to verify emerging trends with wider population

– Staff focus group/consultation

– Theoretical framework


Project Outline

• Stage 3: Action research with students as co-researchers:

– To develop models for formative HEARs

– To develop feedback models

• Project Academic Lead – Prof. Val Wass, Manchester Medical School, National Teaching Fellow

• Project Manager – Patricia Clift, Teaching and Learning Support Office

• Research Associate – Jennie Blake, Teaching and Learning Support Office

• External Evaluator – Rob Ward, Centre for Recording Achievement


The Methodology

• Questions• Structure of questions allowed for definition of topics to be community based• After definition was established, personal examples were solicited, helping keep everyone on the

same page.• Definitions, because explicit, could be compared• Compare definition (or ideal) to reality.

• Focus Groups• Can allow for a “safe” space for conversation to happen (Kitzinger, 1994)• Aids in recruitment for action research as it bonds the group together with a common goal *(Chiu

2003)• Allowed for observation of dialogue between students • Mixed discipline and pure discipline focus groups allow us to compare internal degree

conversations

• Individual Interviews• Used to triangulate data and enrich themes• Used to help deal with recruitment issues**• Used to allow greater student participation


The Questions*

1. When you hear the words “informed choice”, what comes to mind?a. Tell the story of the last curricula choice you made.b. What resources do you use to make curricula choices?c. What resources you need to make an informed curricula choice?

2. Describe what you think of when you hear the words “feedback”.a. Do you ever use feedback from a prior event to make a choice?b. What would make feedback a more useful tool for choice?

3. Where do you see yourself a year/two years after leaving university?1. Do you make curricula choices based on career goals?

a. Do you feel your experiences are typical of someone in your degree programme?

b. What resources do you wish were provided for you?a. Which of these is the most important?b. Is there anything we missed?


The Participants

Programme Pure Discipline

Mixed Discipline1

Mixed Discipline2

Interview Discipline Totals

English 4 3 3* 5 15*

Pharmacy 10 2 4 3 19

Geography 8 2 2 3 15

22 7 9 11 49


Results Outline

The areas of choice, resources, and feedback seemed to be high anxiety areas for most of the participants, very few were completely comfortable, and all wanted an opportunity to work with others to improveo We’re going to speak about the student experience as a journey, one that might work

better with a map, some sort of engine to combat inertia and someone to help figure out where to go next.

• The issues are complex and interwoveno However, by focusing on a few areas (structure of feedback, academic advisor roles,

personal development) we can hope to make an impact

Most of the results are differences of degree as opposed to presence or absenceo Which means that the implications of the study can apply to all areas, as there is

always another step to take

There does seem to be a place for a formative document based on the HEARo And genuine interest in helping the University and this project to create one


BLANK MAP AND A BROKEN COMPASS

• Independent choice

• Resources

• Communication

• Goals

“No one’s ever said, ‘Look, here’s the underlying structure of what we want to give you, this is why we’re doing all of this...’ That’s

never happened.” (efg1)


Resources—What do students use to make decisions?

• Students tend to use non-University resources first—

– “Google!” (Sphai)

– “[The internet’s] the first step.” (Pfg1)

– You have to rely on friends that you’ve made. (Efg1)

This could imply that they are not benefiting from the expertise the degree programmes offer. In addition, they are spending extensive amounts of time searching for resources instead of using them.

• Difficulty deciding which resources are most trustworthy

– Other than [friends], I think I’d be struggling, really, to work out what’s good and what’s not. (Reengi)

– ...about a million articles came up...it would be really useful if there were more direction (Reengi)

– I think sometimes you’re um..maybe bombarded with too much information (mfg1g)

In this case, students have to go through a process of guessing, or trial and error, to figure out the best sources—unless they reach outside of the process and go for “extra” meetings or assistance


Communication-the route

• Who to ask/Where to look--

• I think sometimes you don’t actually know who to talk to...because if your academic advisor isn’t useful..it’s like, then who do I go to? (mfg2g)

• I was told to email my academic advisor, and he emailed back and told me he wasn’t actually working for the university... (mfg2e)

No university-based starting point—which throws them back to non-University sources

• Reliance on an underlying structure—

• You’ve got to email about ten different people to get something. (mfg1p)

• Or even communication between modules, because at the moment we just get all three deadlines in a week, so how are we meant to give each one our best? (efg2)

If there is no consistency between modules, nothing the students learn is generalizable—each seminar leader, tutor, etc must be treated as brand new.

• The unanswerable question:

• No. This is what we get in our seminars, ‘Is everyone okay with the essays then?’...Because we have to be, don’t we? What else are we going to say? (efg1)


What is the goal?—clarifying the destination

• Understanding the goal—– We’re not really aware of what we are supposed to have or what we’re

supposed to know, because we don’t know where to access or know all of these things (pfg1)

– ...When you’re on a training course at a new job, and they train you... [they tell you] If you qualify you should be able to do this...but..it’s not really established [at university] so we don’t know, if this is what we should have gained or..it would be nice to know. (pfg1)

Without being aware of what the goal is—the wider goal (so not just assessment but skills to be taken away from a specific course/module/event) it is difficult for the students to internalize what the expectations are—and what they should take away from the experience. They have no “end point” in mind.

• Feedback• A possible way to mark out the goal and the route. A way to facilitate

communication as long as the purpose of feedback is not merely to explain a mark but also gives an indication of “what next”.


Perpetual Motion—how feedback stalls out

Definitions

A Process

A Purpose

I mean, I know it’s quite preliminary, ‘cause they haven’t done feedback forever at uni, but, um... I think at the

moment we’re getting...what was wrong and less of what you could do (mfg2p)


Definitions

• An academic debrief (gfg1)

• I always think of exams, and it’s like an evaluation of your performance on exams (gfg1)

• We’re tested on something, and they mark it, and we receive it back. (pfg1)

• They tell you, like, whatever you did wrong and how you could improve it (hpharmi)

• Someone having read whatever you’ve written and critically assessed it and given it back to you. (efg1)

• Essay feedback, so to help improve on your next essay (efg1)


Process

• There’s nothing I can do I can do, if that’s the mark I’ve got, so... (mfg1e)

• And that’s it, [the feedback] just disappears and you never see it again. (pfg1)

• I think it’s hard, you know, because, you know different people mark your work...you think you’re correct in it, to what someone wants it, and the... Someone else wants something different. (mfg1e)

• And it means, I feel like as I’m writing these essays that are going towards my degree this year, I’m writing them with this blind idea of what’s a good essay.(efg1)

• Because after you’ve done the essay, you’ve already got your mark, even if you get feedback after that, if you’ve done really badly, there’s nothing you can do about it. (efg1)

• We don’t have the module again—can’t really use [the feedback from it] (hphai)

• You’ve passed-we have no feedback. We don’t care, ‘cause you’ve passed. (mfg1p)

Here, the feedback is *stopping* at the mark—which means that it is neither carrying towards the next assignment nor being internalized and moved on. Some of this can be attributed to the idea that feedback is a reflection on the past—although the definitions do include implications of using feedback forward to the next assignment, few students mention doing so.


Engine—powering the loop

• Yeah, and there’s no one to tell you how well you’re doing in general..because no one talks to each other, and no one has all of your grades. (efg2)

• We need someone to go... Look here, I can see that you keep doing this, let’s do some practice on this. Something that you only get through one on one. (efg2)

• I think it would be good also to have..what you are doing right. (gfg1)

• When you’re working somewhere and you have...set short term, long term milestones. (pfg1)

• We never know if we’ve reached [the skills] (pfg1)

• that's really useful because he's highlighted the skills that we've obviously learned from last year and we're learning now, to put on our CV and things so that does give you feedback on how you've developed those skills. so that's good for that.“ (gfg1)

To help feedback move into a more powerful resource, it seems as if it is important to explicitly state what students need to do with the feedback they get and to ensure that positive aspects of papers are pointed out. In addition, there is a place for a conversation between students and advisors on their broader skills and weaknesses. This would help in the areas of personal development as well.


Self-Portrait

• Isolation

• Independence

• Community and Clarity

So if someone who just knew, someone who knows who you are and kind of, I know it’s much too big, there are too many people at university...

(efg1)

Yeah, we don’t really get... It would be possible to do the whole..go through the whole semester, and no one would have a clue whether you were doing well, or anything. You could go through the whole semester and

not... know anything at the end of it.” (mfg1p)


Isolation

• You can just feel completely anonymous (efg1)• Yeah, but so I know it’s unrealistic for someone to keep an eye out for you, but if you

just had someone who was in whatever kind of way just tracking your progress a bit and chatting to you. (efg1)

• My first year, personal tutor was really easy to get along with... my one for the second year...he didn’t know anything about me...so it was just a waste of time. (mfg2)

• My personal tutor changed from year one to year two, but I wasn’t informed that it had changed. And then...I got an email saying “You’re supposed to have scheduled a meeting with your tutor”...and so I did...and she said “Oh I’m sorry, I’ve got no idea who you are; I didn’t know you were supposed to schedule a meeting”...So there seems to be a complete breakdown of communication...and I’ve got a lot of friends who said the same. (Reengi)

This sense of isolation (and a sensitivity about bothering busy staff members) effectively pushes students away from University resources and staff. In addition, whether accurate or not, a perception is being created that places barriers between students and staff.


Independence

• But, I think, like, the more you put in yourself, the more you get out of it. (mfg1g)• I think there’s, like, a bit of an expectancy, like, because we’re at university… might be

like, well do it off your own back – like, not spoon-feed you… (hipharmi)• I think it's striking a balance between sort of still doing our own things rather than

spoon feeding (pfg1)• the university… it’s more hands-off, like, there’s more responsibilities you… on you,

like, to do that. So there’s not as much… Although they do provide information you’ve got to find it yourself - they’re not just giving it to you straight away. (mfg1p)

• I think it’s less so, that, in university, like in school it was obviously pretty… socially unacceptable to… be bugging the teachers all the time, but… in university, I think it’s, kind of, honourable, to really, like… kind of, take control of things yourself – but then… you don’t want to be bugging them too much, because that’s definitely not honourable. (mfg2)

The students have definitely embraced the concept of independence and do not wish to be “spoon fed”. However, there is tension between this need for independence (and expectation that it is the proper way to go through university) and the students’ ability to seek out resources and assistance.


Community and Clarity

• Yeah, and if we’re aware then you can start making answers and know how to target and if you’ve not got the skill you can go and actively do something to get that skill and improve yourself. (pfg1)

• It feels like you need to go with something to discuss (efg1)• Yeah, but we don’t really have… we’re not really encouraged to go to lecturers, I think,

‘cause there’s so many of us. We… It’s just the tutors [inaudible] get our information from. Or course representatives. That’s about it, really. (mfg1)

• But I think - we have a lot - the resources we have are quite good. It's just knowing what we actually need to find. So again it falls back onto the weaknesses and strengths in your own resources, so once you know what your weaknesses are you have pretty much most of the resources to go yourself and dig in (gfg1) So, there is a lack of clarity in the purpose and roles of people in the degree community

and in the goals of the degree as a whole. If there is no clear path and a lack of connection, students feel uncomfortable reaching out. However, if there is a clear path or indication of goals/places for help then the students feel comfortable enough to seek it out on their own.


Indications for improvement/Areas for further investigation

• Use the formative HEAR as a platform for conversation and planning between academic advisors and students

• Create “plans” between and inside modules that allow students to see the skill set they are expected to learn and the feedback to expect.

• Devise ways to move feedback “beyond” the mark.

• Further investigation into structures that encourage conversation and community building between students and staff

• Explicitly react to student feedback and support their use of feedback to improve skills

Documents

HEARing Student Voices Self Portraits and Perpetual Motion: The Student Experience of Informed Choice and Feedback Jennie Blake, Research Associate Patricia