Undergraduates as Teaching Assistants Reducing the marks-based culture of learning Tony Field Department

Undergraduates as Undergraduates as Teaching AssistTeaching Assistaantsnts

http://www.doc.ic.ac.uk/lab/firstyear/envision/

Reducing the marks-based culture of learningTony Field

Department of Computing

Joint work with Esat Alpay, Peter Cutler, Susan Eisenbach

The “marks-based” The “marks-based” cultureculture• Use the weighting of the work to

determine the time spent

• Something that carries little or no weight gets little or no time

• Concerns for marks rather than the knowledge/skills attained

2

The ideaThe idea• Zero-weight first year tutorial work in

Programming and Mathematics • 6/7 students• Weekly exercises (previously assessed by

academic)

• Pay A-grade 3rd/4th year undergraduates (UTAs) to help run the tutorials• UTAs mark exercises (feedback only)• Feedback, problem-related and general discussion

Academic’s role: Mentor the UTA, check the marking, help plan meeting and guide discussions…

3

Some questionsSome questions1. Does zero weighting affect student

motivation and participation? 2. How do students react to having tutorials

run by peers? 3. What are the benefits for the UTA?4. How do the academics perceive the

change? 5. Is there a significant difference in student

performance?6. (Does it reduce the instances of

plagiarism?)

4

EvaluationEvaluation

• Cost: Around £60-£75 per student per term (one tutorial/week, 2-3hrs marking)

• Pilot study sponsored by Envision for Maths tutorials (PMTs)

• 2-year study, with qualitative and quantitative evaluation:

Changing the marks-based culture of learning through peer-assisted tutorials, European Journal of Engineering Education, Volume 35, Issue 1 March 2010 , pages 17 - 32

5

QuestionnairesQuestionnaires

• One set for first years, the other for both UTAs and academics

• Five-point Likert scale• Open-ended questions at the end

Main theme of student Main theme of student questionsquestions

How the first years compare the UTAs and the academics

ME – motivation and engagement SD – skills development ECF – explanation, contextualisation, and

feedback

Evaluation of PMT and UTA. UTA PMT1) Understanding my needs as a first-year student 1 2 3 4 5 1 2 3 4 52) Getting me to participate in the PMT 1 2 3 4 5 1 2 3 4 53) Making the PMT work interesting 1 2 3 4 5 1 2 3 4 54) Making it easy for me to talk about things I don’t understand 1 2 3 4 5 1 2 3 4 55) Having a difficult concept explained to me 1 2 3 4 5 1 2 3 4 56) Giving useful feedback on my work 1 2 3 4 5 1 2 3 4 57) Clarifying/correcting misunderstanding 1 2 3 4 5 1 2 3 4 58) Helping me improve my logic skills 1 2 3 4 5 1 2 3 4 59) Helping me improve my communication skills 1 2 3 4 5 1 2 3 4 510) Exploring a problem/concern thoroughly 1 2 3 4 5 1 2 3 4 511) Motivating me to want to do well on the course 1 2 3 4 5 1 2 3 4 512) Giving me reassurance about my learning progress 1 2 3 4 5 1 2 3 4 513) Clarifying why the material will be useful later on 1 2 3 4 5 1 2 3 4 514) Helping me to manage my time so that I can finish the exercises 1 2 3 4 5 1 2 3 4 515) Making me feel part of the department 1 2 3 4 5 1 2 3 4 516) Exploring related topics not covered directly by the course 1 2 3 4 5 1 2 3 4 5

Getting me to participate in the tutorialMaking the tutorial work interestingMotivating me to want to do well in the courseetc.

having a difficult concept explained to megiving useful feedback on my workclarifying why the material will be useful later onetc.

helping me to…improve my logic skills...improve my communication skills…manage my time so that I can finish the exercisesetc.

ME

ECF

SD

Summary of resultsSummary of results

Three key questionsThree key questions

The most discriminatory questions (UTA>Academic):Getting me to participate in the PMT (ME)Giving useful feedback on my work (SD)Helping me to improve my <maths> skills

(ECF)

Main theme of UTA/tutor Main theme of UTA/tutor questionsquestions

How UTAs and Academics judge the PMT system

UPE – UTA performance and effectivenessUPE1 (teaching qualities)UPE2 (social/affective aspects as

peer) UB – UTA personal benefits US – Support received by the UTA

A: Evaluation of PMT.1) The UTA gets good support for his/her role 1 2 3 4 52) The UTA’s communication skills are improving over time 1 2 3 4 53) The UTA prepares well for the PMT meeting 1 2 3 4 54) The UTA is a role model for the first year students 1 2 3 4 55) The UTA has many interests in common with the students 1 2 3 4 56) The UTA takes the lead role at the PMT meeting 1 2 3 4 57) The UTA uses initiative in running the PMT meeting 1 2 3 4 58) The UTA gets good support from the academic (PMT) 1 2 3 4 59) The UTA gives thorough feedback to the students 1 2 3 4 510) The UTA explains the PMT material clearly 1 2 3 4 511) The students find it easy to talk openly to a UTA 1 2 3 4 512) The students find it easy to talk freely with the academic (PMT) 1 2 3 4 513) The students want the UTA to think well of them 1 2 3 4 514) The students want the academic (PMT) to think well of them 1 2 3 4 515) The students find it easy to ask a UTA when they don’t understand 1 2 3 4 516) Most students are more concerned with marks than understanding 1 2 3 4 517) I suspect that some of my students copy their PMT submissions 1 2 3 4 518) The students take the feedback on their work seriously 1 2 3 4 519) The UTA has a clear idea about how to run the PMT meeting 1 2 3 4 520) The UTA and academic (PMT) discuss teaching strategy before the meeting 1 2 3 4 521) I enjoy seeing the students develop their skills 1 2 3 4 522) The UTA is a student so it easier for him/her to explain concepts to the PMT students 1 2 3 4 523) The students have improved in the PMT meetings over the year 1 2 3 4 524) The mentoring role of academic (PMT) is very important for a UTA 1 2 3 4 525) The UTA finds it hard to get the students to participate 1 2 3 4 526) The academic (PMT) finds it hard to get the students to participate 1 2 3 4 527) The UTA finds it helpful to revisit the first year material 1 2 3 4 528) The academic (PMT) gives the UTA useful advice about running the PMT 1 2 3 4 529) The UTA learns useful transferable skills by helping with the PMT 1 2 3 4 530) Having a UTA helps enhance the department’s intellectual culture 1 2 3 4 531) The payment is the most important motivator for a UTA 1 2 3 4 532) The experience is the most important motivator for a UTA 1 2 3 4 5

The UTA prepares well for the PMT meetingThe UTA uses initiative in running the meetingThe UTA has a clear idea about how to run the meetingetc.

The UTA gets good support from the academicThe mentoring role of the academic is importantThe academic gives the UTA useful adviceetc.

The students find it easy to talk openly to the UTAThe UTA is a student so it’s easier for them to explain…The UTA is a role model etc.

UPE1

US

UPE2

The UTA learns useful transferable skillsThe experience is the most important motivating factorThe UTA finds it useful to revisit the materialEtc.

UB

Mean Standard deviation

UPE1UTA 4.02 0.45PMT 4.34 0.56UPE2UTA 3.85 0.69PMT 3.67 0.80UBUTA 4.08 0.64PMT 3.90 0.57USUTA 3.42 0.72PMT 3.86 0.79

Summary of results (17/23 Summary of results (17/23 returns)returns)

Four key questionsFour key questions

Statistically significant disagreements (UTA vs Academic score):

The UTA prepares well for the PMT meeting (4.06 vs 4.59)

The UTA takes the lead role at the PMT meeting (4.24 vs 4.88)

The UTA and academic discuss teaching strategy before the meeting (2.45 vs 3.47)

The UTA is a student so it is easier for him/her to explain concepts to the students (3.88 vs 2.94)

Quantitative MeasuresQuantitative Measures

2005-2006(%)

2006-2007(%)

2007-2008(%)

student attendance at PMT sessions

89.2 92.3 92.7

submission rate (PMT work)

93.5 98.3 96.8

average mark (PMT work)

80.0 81.4 71.2

test result (Logic + Discrete Maths)

48.9 53.0 57.0

Discrete Mathematics exam average

64.3 59.8 57.1

Logic exam average 55.0 61.7 60.4

Reasoning exam average

53.7 55.1 53.8

UTAs mark rigorously!

Open-ended questionsOpen-ended questions

• The UTA scheme is the best part of the whole non-programming oriented part of the course!

• Excellent scheme, that a lot of other courses do not have, has helped a lot in studies, and to settle in

• overall extremely helpful...don't know what I would have done without it!

• Best method of study by far

• I think the PMT sessions is the most helpful hour of the week.

• very helpful, can't think of any way to gain significant improvement.

• The UTA scheme was the most helpful learning tool in my course this year.

• Excellent idea. It both encourages us to aspire to work hard and helps the UTA in furthering his/her skills in teaching.

• I really appreciated that the student who is still studying at the Universrity and has very recent experience with the functioning of the course is very helpful.

• Execllent, everybody wins.

• I think it's fantastic. Great job. Really appreciate the "anytime" approach.

• I think it is better than the tutorials.

• Its an excellent system that allows people to fully grasp concepts on the course.

• Personally I find it very helpful and my UTA is very keen to explain anything we ask. Thumbs up for UTA programme.

• It is very helpful as unlike lectures, the UTA understands what areas we are unsure on, since he is student after all!

• In my specfic case, my UTA is outstanding. He is a superb student, and has a lot of time and effort for me.

• My UTA was very helpful in this course.

• Excellent. Great help. I've spoken to first year maths degree kids and they are jealous of the amount of help I get in smaller groups. I have found the PMT very very helpful indeed and would like even more of this time set aside for us. We have many hours of contact time each week for programming related help, yet only this one hour PMT slot for maths (which I and others think is the more difficult of the sections).

• Good idea, provides some useful feedback

Observation 1: Students really like it

• Excellent scheme, that a lot of other courses do not have, has helped a lot in studies, and to settle in

• I think it's fantastic. Great job. Really appreciate the "anytime" approach.

• The UTA scheme is the best part of the whole non-programming oriented part of the course!

• Best method of study by faretc.


• It's a "family" atmosphere• The environment feels a bit more comfortable• He is also a student, so he can give tips of what exactly is expected from us • Down to earth, jokes, knows the situation that we are in and can relate to it. • He's friendly and helpful and gives good explanations of the material• The opportunity to have somebody who is more on our wavelength• Relaxed learning• It is personal, small group is good for learning, get to know everyone in your group, don't feel scared by results,

can discuss work with all members• That it is mainly run by student - in the way that only person who is still studying can do • They know exactly what they found difficult when they did the course and guide according to that.• Very friendly and straight to the point • It's very casual and he not only understands my problems, but also anticipates them. • That the UTA is run by students of approximately my age• UTA's are closer to our age and I think are better often at communicating with us than the lecturers• I believe its helpful, since sometimes you can approach the UTA in a more open manner than you would to with

the tutor and that is sometimes helpful.

Observation 2: They like students as TAs

• It's a "family" atmosphere• They know exactly what they found difficult when

they did the course and guide according to that• The opportunity to have somebody who is more on

our wavelength• UTA's are closer to our age and I think are better

often at communicating with us than the lecturersetc.


• More UTA involvement in the tutorial if possible• The UTA should be commanded to make students/tutees participate in the session• A more lively atmosphere would be nice• MAKE IT MORE FUN!!! • Better communication • More passionate uta's • Train them more in terms of dealing with students beforehand • It is a good system, my UTA just happens to be a little too shy for the job • Train the UTA • Possibly a more careful choice of UTA candidates • It is good if the UTA doesn't keep canceling the meetings/ not turning up to them. • My UTA for the PMT course was incredibly bad at 'public speaking', incredibly nervous

etc.• It works fine, but some people get a poor UTA, which gives them a disadvantage

Observation 3: Some UTAs need vetting/better training

• Train them more in terms of dealing with students beforehand

• Possibly a more careful choice of UTA candidates • More UTA involvement in the tutorial if possible• A more lively atmosphere would be niceetc.

• PMT should do more explaining, not just UTA• Acedemic teachers should involve more in the meeting.• Lecturers that are assigned to the PMT groups could be more involved with

the PMT• My PMT did not turn up to most of my tutorial classes, which was

disappointing• The academic staff should get more involved• Generally, the member of academic staff doesn't really do anything.• The main PMT tutor never participates on the scheme. Only the UTA is

involved.• We end up with pmt sessions where UTAs do all the work while tutors are

not participating.

Observation 4: Staff need better training(!)


• PMT is not always there, and when is is usually working and not joining in

• It was working fine until he stopped showing up a month ago

• The academic staff should get more involved• My PMT did not turn up to most of my tutorial

classes, which was disappointingetc.


• The meetings are fun and it is good talking with the students and getting them involved in the subjects.• Other first years ask me questions in the labs - I like helping people and it is good to know you've done it.• The experience and confidence it has given me, and watching how some of the students take the sessions on board and

really improve. • Enjoyable experience, learned a lot myself, good contact to PMT.• Interacting with first year students, The experience of teaching.• I enjoy working with my PMT (Dr. X) very much.• Good teaching experience. • Experience gained - teaching, assessing. Seeing students develop/overcome learning barriers. • To try out teaching - A chance to improve my presentation skills - Be able to explain topics clearly - Watching students learn • Many people benefit (student, UTA, PMT). Good atmosphere. • The UTA program is beneficial to all of the three parties: • Students have the opportunity to participate in teaching roles which would be a valuable experience and it also offloaded

some workload for PMTs. • The UTAs gain valuable experience (and some extra money which is always useful!).

Observation 5: UTAs benefit substantially

• Enjoyable experience, learned a lot myself, good contact to PMT.

• Other first years ask me questions in the labs - I like helping people and it is good to know you've done it.

• The experience and confidence it has given me, and watching how some of the students take the sessions on board and really improve.

• The UTAs gain valuable experience (and some extra money which is always useful!).

etc.

Observation 6: Academics love it!


QUIZ: Why?


• I no longer have to mark the homework.


• I no longer have to mark the homework



• They do the marking!


• I no longer have to mark the homework• They do the marking!




• No more marking.


• I no longer have to mark the homework• They do the marking!• No more marking





• Not doing the marking.


• I no longer have to mark the homework• They do the marking!• No more marking• Not doing the marking






• Avoiding the marking!


• I no longer have to mark the homework• They do the marking!• No more marking • Not doing the marking • Avoiding the marking!






• Avoiding the marking!

• No marking.


• I no longer have to mark the homework• They do the marking!• No more marking• Not doing the marking• Avoiding the marking! • No marking


• I no longer have to mark the homework. • They do the marking!• No more marking. • Not doing the marking. • Avoiding the marking! • No marking. • I appreciate not having to do the marking or prepare in detail for the tutorial. • Avoid the marking of PMT on weekly basis. • It saves a lot of time that I can more profitably spend on other activities - research for example. • More fun, less work, not so much boring exercises. • Not having the tedium of explaining this material for the 20th time. • It takes some of the teaching burden away from me, and made it easier to interact with the students on a personal

level.


• I no longer have to mark the homework• They do the marking!• No more marking • Not doing the marking• Avoiding the marking! • No marking• It takes some of the teaching burden away from me,

and made it easier to interact with the students on a personal level

etc.

ConclusionsConclusionsFrom the student perspective, the UTA

performance exceeds that of the academicDespite individual problems, students value

the system and see UTAs as role modelsThe system fosters a sense of community;

UTAs become mentors (and ‘buddies’?)Academics universally praise the UTA

systemThere is no significant change in student

exam performance, attendance etc.Needs UTAs to be carefully chosen & trained

and academics to engage fullySuccessfully instils a culture of learning

based on the intrinsic value of material, rather than credit

Documents

Undergraduates as Teaching Assistants Reducing the marks-based culture of learning Tony Field Department