Upload
centre-of-higher-education
View
310
Download
0
Tags:
Embed Size (px)
Citation preview
‘Enhancing the First Year Experience – A Case Study From Biomedical Sciences’
Paul HaganStephen McClean
University of Ulster
Evaluation• There are many, many variables to be
considered when evaluating student performance.
• What works for one individual may not offer a solution for another.
• How do we assess the efficacy of all the ‘fixes’ we offer and how do they impact on each of the many experience groups which we have to accommodate.
• We need to use some medium-weight statistics to get at the detail of the ‘fixes’ versus student performance with prior educational experience factored in.
EVALUATION
• This evaluation step is CRUCIAL in the whole exercise and it is actually the detail of the EVALUATION which I am communicating, and what it can offer.
• We all need to justify what we do in terms of committing additional resources to remedial classes.
• These robust analyses give us the ammunition we require to promote and justify these activities.
Normal Distribution fits are for the two groups of students
1. Those with previous ‘A’ level experience
2. Those without ‘A’ level experience
2005-2006 - 62% 82%
2006-2007 – 68% 81%
2007-2008 65% 84%
EX_Mark_2
Frequency
105907560453015
14
12
10
8
6
4
2
0
Mean StDev N62.37 23.23 7681.79 14.38 59
A_OR_AS_201
Histogram of EX_Mark_2Normal
BOXPLOTS
Outlier an unusually large or smallobservation. Values beyond thewhiskers are outliers.
By default, the top of the box is thethird quartile (Q3). 75% of the datavalues are less than or equal to thisvalue.
By default, the upperwhisker extends to thisadjacent value thehighest data valuewithin the upper limit. Upper limit = Q3 + 1.5(Q3 - Q1)
Median the middle of thedata. Half of theobservations are less thanor equal to it.
By default, the bottom of thebox is the first quartile (Q1)25% of the data values areless than or equal to thisvalue.
By default, the lower whiskerextends to this adjacent value thelowest value within the lowerlimit.Lower limit = Q1- 1.5 (Q3 - Q1)
Mean
CA_Mark
EX_M
ark
9080706050403020100
100
80
60
40
20
0
Scatterplot of EX_Mark vs CA_Mark
Good correlation between coursework and examination mark. Three students with low coursework and > pass marks in the examination obviously defied all attempts to improve their coursework submissions.
CA_Mark
EX_M
ark
90807060504030
100
80
60
40
20
0
Scatterplot of EX_Mark vs CA_Mark
2005-2006
2006-2007
Final_CA
Exam
_To
tal
90807060504030
100
80
60
40
20
0
Scatterplot of Exam_Total vs Final_CA
Possibility of producing a ‘tailor made’ report for individual student.
Position in cohort can be highlighted from registration number.
ABSENCES
EX_M
ark
50403020100
100
80
60
40
20
0
Scatterplot of EX_Mark vs ABSENCES
The expected correlation between attendance and examination score, increased absence
from lectures, practicals and tutorials leads to depressed examination marks. Outliers suggest that some poor attendees have
sufficient prior ‘knowledge’ or alternative access to course materials, to enable a pass.
2005-2006 2006-2007
ABSENCES
EX_M
ark
403020100
100
90
80
70
60
50
40
30
20
10
Scatterplot of EX_Mark vs ABSENCES
NUMBER OF EXTRA TUTORIALS ATTENDED
EX_M
ark
543210
100
80
60
40
20
0
Boxplot of EX_Mark by EXTRA-TUTS
STUDENTS HAVING STUDIED TO 'A' ORTHE EQUIVALENT OF 'AS' CHEMISTRY
In all 6 extra ‘voluntary’ tutorials were offered in semester one. All students attending at least one
extra tutorial passed the exam.
2005-2006
NUMBER OF EXTRA TUTORIALS ATTENDED
EX_M
ark
210
100
90
80
70
60
50
Boxplot of EX_Mark vs EXTRA_TUTS
STUDENTS HAVING STUDIED TO 'A' OR THEEQUIVALENT OF 'AS' LEVEL CHEMISTRY
2006-2007
A total of two examination revision sessions were offered immediately prior to the examination after the Semester one examinations.
NUMBER OF EXTRA TUTORIALS ATTENDED
EX_M
ark
_1
6543210
100
80
60
40
20
0
Boxplot of EX_Mark_1 by EXTRA-TUTS_1
STUDENTS WITH NO 'A' LEVEL CHEMISTRY
10 RECORDED ABSENCES
Students with no ‘A’ level chemistry experience (‘A’ or ‘AS’ level) or with no chemistry at Irish Higher or Irish Ordinary level improve their probability of passing by attending at least one extra ‘voluntary tutorial’
It can be shown that the students who failed to achieve at least a pass grade were not sufficiently motivated to attend extra tutorial classes and this lack of motivation would be reflected in their level and quality of examination preparation
2005-2006 2006-2007
NUMBER OF EXTRA TUTORIALS ATTENDED
EX_M
ark
_1
6543210
100
90
80
70
60
50
40
30
20
10
Boxplot of EX_Mark_1 vs EXTRA_TUTS_1
STUDENTS WTH NO 'A' LEVEL CHEMISTRY
BOTHEX_1
EX_M
ark
_1
10
100
80
60
40
20
0
Boxplot of EX_Mark_1 by BOTHEX_1
MEAN = 74
MEAN = 53
STUDENTS WITH NO PREVIOUS 'A' LEVELCHEMISTRY AT ANY LEVEL
Attendance at both pre-examination revision tutorials improved the mean examination mark of students without chemistry ‘A’ levels by 21% on average. None of these students gained lower than a pass grade.
BOTHEX_1
EX_M
ark
_1
10
100
90
80
70
60
50
40
30
20
10
Boxplot of EX_Mark_1 vs BOTHEX_1
MEAN = 64MEAN = 71
STUDENTS WITH NO PREVIOUS 'A'LEVEL CHEMISTRY AT ANY LEVEL
BOTHEX_1
Exam
_To
tal_
1
10
100
90
80
70
60
50
40
30
20
10
Boxplot of Exam_Total_1 vs BOTHEX_1
MEAN = 60
MEAN = 76
STUDENTS WITH NO PREVIOUS 'A'LEVEL CHEMISTRY AT ANY LEVEL
Picture is broadly similar in 2007 - 2008
BOTHEX
EX_M
ark
10
100
90
80
70
60
50
40
30
Boxplot of EX_Mark by BOTHEX
STUDENTS WITH PREVIOUS 'A' LEVELCHEMISTRY
MEAN = 81 MEAN = 83
36 RECORDED ABSENCES
Students with previous ‘A’ level experience generally do not exhibit a significantly improved examination performance by attending both pre-examination revision tutorials. It is however not
detrimental to their performance and may be viewed as a ‘confidence-building’ exercise.
2005-2006 2006-2007
BOTHEX
EX_M
ark
10
100
90
80
70
60
50
Boxplot of EX_Mark vs BOTHEX
MEAN = 81
MEAN = 84
STUDENTS WITH PREVIOUS'A' LEVEL CHEMISTRY
Multivariate analysis 2006-2007
First Component
Seco
nd C
om
ponent
0.50.40.30.20.10.0-0.1-0.2-0.3
0.50
0.25
0.00
-0.25
-0.50
EX_Mark
HOME_EC_A
PHYSICS
BIOLOGY
SCIENCE_A
VCE_SCIENCE
MATHS
CHEM_ASCHEM_A
ABSENCES
Loading Plot of ABSENCES, ..., EX_Mark
A_OR_AS
EX_M
ark
10
100
80
60
40
20
0
Boxplot of EX_Mark by A_OR_AS
STUDENT WITH 36 ABSENCES
MEAN = 82
MEAN = 62
Students with ‘A’, ‘AS’, or equivalent level chemistry, on average, score least 15% higher in the examination.
A_OR_AS
EX_M
ark
10
100
90
80
70
60
50
40
30
20
10
Boxplot of EX_Mark vs A_OR_AS
MEAN = 67
MEAN = 81
2005-2006 2006-2007
WIFT
A_OR_AS
EXA
M/100
10
90
80
70
60
50
40
30
Boxplot of EXAM/ 100 vs A_OR_AS
MEAN = 56
MEAN = 66
WHAT’S IN IT FOR THEM?
2005 - 2006
Examination marks for second semester BIOCHEMISTRY module
A_OR_AS
Exam
_To
tal
10
100
90
80
70
60
50
40
30
20
10
Boxplot of Exam_Total vs A_OR_AS
MEAN = 63
MEAN = 84
2007 - 2008
BIOCHEMISTRY
EXAM MARK
NO ’A’ LEVELS
%PRIOR ‘A’ LEVELS %
2005/6 56 662006/7 57 612007/8 63 84
Conclusions
• Attendance is a crucial factor• Specific groups can be
identified as being deficient in chemistry and mathematics
• Targeted tutorials are very effective
• Practice MCQ’s appreciated by students
• Some ‘A’ levels better than others.
Pass Rate /%
2003/4 642004/5 732005/6 912006/7 962007/8 91