Measuring and monitoring study progress

Measuring and monitoring study progress ExpandO studiedag, Brussel 19 september 2013, [email protected] 1

Measuring and monitoring study progress

A longitudinal study into study progress of first-time enrolled studentsA first step towards structural and evidence based student guidance


It’s astonishing that study program professionals have so little insight into study progress

of their students(Hans van Hout, 2006, in ‘Vernieuwing in het Hoger Onderwijs: Onderwijskundig Handboek’)


Actions of about 40 FTE’s

Actions of about 40 FTE’s

Student performance

Student performance


First step: getting insight into study progress

• Monitoring of the student population• How are students progressing?

• Indicators for study progress• Outcome indicators:

• Study completion rate after one year in a standard learning track • Study success rate/year• Bachelor completion rate within the foreseen period• Master completion rate within the foreseen period

• Process indicators: • Learning track type per year of enrolment• Learning path to the bachelor and master degree !


• Study completion rate after one year in a standard learning track:

The proportion of first time enrolled students taking a standard track of ECTS Credits that

successfully completed the first year of this track after one year of study.

• Study success rate/year:

The proportion of ECTS Credits completed with respect to the total amount of ECTS credits enrolled

for by a student.

• Bachelor completion rate within the foreseen period:

The proportion of first-time enrolled students that accomplish a bachelor degree in the foreseen

three years of time.

• Master completion rate within the foreseen period:

The proportion of first-time enrolled students that accomplish a master degree in the time foreseen

for obtaining the specific master degree.

Study progress outcome indicators: definitions


• Type of learning track by year of enrolment:

Track types:• Standard Learning Track (Standard package i.e., 60 ECTS Cr./year)• Standard Learning Track Repeaters• Personalized Learning Track (any deviation from SLT within the same program)• Reoriented Learning Track (enrolled for a new study program)• Drop out from our institution

• Type of learning path towards a bachelor/master degree:

Typology based upon the succession of specific Learning Tracks/year

Note that:

• ECTS credits which a student has to take again after failing are added to the total amount of ECTS

credits enrolled for.• Although a student did not succeed for an exam, he/she can still be allowed to pass for the first

Standard Learning Track package when his/her results were within the specified tolerance boundaries

for passing that package. In this case, the ECTS credits for that course were not obtained.

Study progress process indicators: definitions


Modelling study progress, in theoryTinto,1987, 1993 (US); Prins, 1993 (The Netherlands); Lacante 2001, 2007 (Flanders/Belgium)


Where to start

?


Modelling study progress, in theory

Student Characteristics

Context Characteristics

Study Progress

Social / Academic Integration

Not included


Data of 20,650 studentsGathered between 2005/6 -2008/9

First stepFocus on main effects

Student Characteristics

Study Progress

Program choice

Learning track typology

Moment of choice

Motivation

Background

Socio Demographics

Expectations (Questionnaire)

Pre-College schooling

Type Secundary Education

Study delay Forced track change

Performance Positioning (self assessment)

Complex Interactions

Modelling study progress, in practice


Lots of data

Some interesting

findings


Migrations in the student population due to flexibilization

Study completion rate after one year in a standard learning track

2005

2008

After 1st year exams After Bachelor exams

Need for view on all students

At start 1st year

Repeaters

Repeaters


Throughput

Starters1st enrollment

(2005-06 + 2006-07)(n=10,119)

Bachelor(n=3468)

After 3rd year

Personalised Learning Track

(n=4626)

Bachelor(n=746)

Drop out(n=2025)

Bachelor(n=592)

After 4th yearAfter 1st years

Drop out(n=1224)

Still enrolled(n=2064)

34.2%

7.3% 5.9%

20.0%

13.2% 45.8%

12.0%

20.0%


Succes in function of Secundary Education

Secundary education

Study succes rate

Success rate Categorised

0 % 100 %

Vocational 8,6 % 67,3 % 0,0 %

Technical 24,9 % 39,9 % 5,3 %

Art 30,3 % 29,5 % 7,6 %

General 67,8 % 6,8 % 32,1 %


Distribution of Succes in function of

Secundary Education


Self Positioning inSecundary Education

Secundary Education Performance postioning

(self assesment)Average

success rate

Success rate categorized

0 % 100 %

Best group (31%) 77,4 % 4,7 % 46,3 %

Better group (35%) 65,0 % 8,8 % 28,2 %

Average group (30%) 53,3 % 13,1 % 17,2 %

Weak group (3%) 50,6 % 12,5 % 13,8 %

Weakest group (1%) 46,6 % 21,7 % 15,9 %

Delay and/or

track changesAverage

success rate

Success rate categorized

0 % 100 %

No delay 71,3 % 5,4 % 34,9 %

Track change 41,8 % 20,0 % 11,1 %

Delay 37,3 % 26,8 % 10,2 %

Daley + Track change 37,1 % 26,4 % 10,8 %

Delay & Track changesIn Secundary Education

Engineer students 46% BestBio Engeneering 38% Best


Throughput After1st Bachelor

SLT

2nd BachelorSLT

Pers. LT2Ba+3Ba

Reoriented LT

2 Study Programs

DropOut

3rd BachelorSLT

2nd BachelorSLT

Pers. LT 2Ba+3Ba

Reorinented LT

DropOut

98.7% 0.4%<0.1%0.7%0.1%

86.0% 0.8%0.7%12.2%0.3%

After1st Bachelor

SLT


Throughput

1ST BachelorSLT

Pers. LT1Ba+2Ba

ReorientedLT

DropOut

1st BaSLT

2nd BaSLT

Pers. LT 1Ba+2Ba

Reor.LT

DropOut

11.8% 8.9%37.1%42.1%

8.0% 53.4%5.8%23.4%9.6%

SLT orPers. LT

Reor.LT

DropOut

71.2% 4.8% 24.0%

After1st Bachelor

SLT


Throughput

1ST BachelorSLT

Pers. LT1Ba+2Ba

ReorientedLT

DropOut

2nd BaSLT

3rd BaSLT

Pers. LT 1Ba+2Ba+3Ba

Reor.LT

DropOut

11.8% 8.9%37.1%42.1%

8.7% 13.9%22.2%14.7%

Pers. LT 2Ba+3Ba

2.3%38.2%

After1st Bachelor

SLT


Throughput

1ST BachelorSLT

Pers. LT1Ba+2Ba

ReorientedLT

DropOut

1st BaSLT

2nd BaSLT

Pers. LT 1Ba+2Ba

Reor.LT

DropOut

19.1% 66.3%12.2%2.4%

10.4% 59.4%6.7%19.6%3.9%

1st Bachelor SLT <25% success rate

Pers. LT

Reor.LT

DropOut

6.5% 32.9%

2nd BaSLT

1st BaSLT

26.7%11.9%21.7%


Type of learning paths towards a Bachelor degree in 3-years time

Learning path

1st time enrolled1st time enrolled

Total2005‑06 2006‑07

Standard Learning Track, =180 ECTS Credits study succes rate =100% 44,6 % 54,3 % 49,3 %

Standard Learning Track, =180 ECTS Credits study succes rate <100%

40,0 % 25,7 % 33,0 %

Personalised Learning Track, >180 ECTS Credits study succes rate <100%

14,4 % 18,0 % 16,2 %

Reoriented Learning; Variable amount of ECTS Credits

1,0 % 2,0 % 1,5 %

Total 2133 2081 4214

Bachelor degree in 3-years time


Type of learning paths towards a Bachelor degree in 3-years time

Learning pathType Secundary Education

General Technical Art Voc.

Standard Learning Track, =180 ECTS Credits study succes rate =100% 55.6% 46.7% 50.0% 0%

Standard Learning Track, =180 ECTS Credits study succes rate <100% 36.9% 40.0% 28.6% 0%

Personalised Learning Track, >180 ECTS Credits study succes rate <100% 6.1% 13.3% 21.4% 0%

Reoriented Learning; Variable amount of ECTS Credits

1.4% 0% 0% 0%

Total n 3690 30 14 0

Bachelor degree in 3-years time in function of SE


Study Completion Rate in function of PLT


Next steps


Monitoring study progress shows

• Realistic learning paths towards success i.e., obtaining a diploma.• Learning paths that are more likely going to result in drop out.

Now we can start figuring out what goes wrong.

Zooming in on the nature of these paths, i.e., asking ourselves ‘what’s

happening here?’ will enable us to formulate evidence based

hypotheses about what’s the underlying problem and how to deal with it

effectively.

From longitudinal study to systematic monitoring

Monitoring: Enrollment Bachelor Master

Conclusions & Next Steps


Some students take paths with very large study loads others seem to

take an ‘easy’ year. Large number of students coming from Technical

and Art Secundary Education programs end up in Personalized Learning

tracks. Why?

Some students seem to lack the insight of what it really takes to

succeed. And even after a first unsuccessful year, quite a lot still lack

that insight.

How can we help students to become more realistic? How can we

support them in learning what it takes to succeed?

Conclusions & Next Steps


Half day Higher Education Experience

Interactive group sessions

Recorded Authentic Lectures• Anathomy• Mathematics• Sociology

Experiencing teaching, learning, and examination in higher education.

Guided reflection on experiences and additional actions needed to acquire lacking skills.


Advisory Centre for StudentsNetwork study & learning track councellors

Documents

Measuring and monitoring study progress