8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
1/50
Contribution to the WorkshopGraduates on the Labour Market. Questionnaire Development at the
National Level in Sinaia, June 2-4 2010
Harald SchomburgInternational Centre for Higher Education Research
(INCHER-Kassel) University of Kassel, Germany
Concept and
Methodology ofTracer Studies International
Experiences
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
2/50
Schomburg:Concep
tandMethodol
ogyof
Trac
erStudies
2
Overview1. Institutional Background who is
INCHER?2. Theoretical Framework
3. Methodology of Tracer Studies
4. The New GRADNETApproach: Network of Graduate
Surveys5. Conclusion
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
3/50
Schomburg:Concep
tandMethodol
ogyof
Trac
erStudies
1.
Institutional
Background
who is
INCHER?
3
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
4/50
University City Kassel:
in the Heart of Germany, in the State of Hesse
Located in the center of
Germany and Europe
Unique unity of urban
culture and landscape
City of the documenta
Parks and recreational areas
Center for industry, services
and civil administration
of nothern Hesse
4
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
5/50
Schomburg:Concep
tandMethodol
ogyof
TracerStudies
University of Kassel
Location: in the centre of Germanyand Europe (200 Km to Frankfurt = 1.5hours by train)
City of Kassel: 200,000 habitants
University of Kassel
18,000 students
2,000 graduates per year Broad study programs
5
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
6/50
Campus Hollndischer Platz -
1986
Hollndischer PlatzHollndischer Platz
Humanities, Social Science, Economics,Architecture, City Planning and LandscapePlanning, Civil and Mechanical Engineering
Humanities, Social Science, Economics,Architecture, City Planning and LandscapePlanning, Civil and Mechanical Engineering
6
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
7/50
Campus Witzenhausen - 1886
7
WitzenhausenWitzenhausen
Ecological AgricultureEcological Agriculture
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
8/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
INCHER-Kassel International Centre of Higher Education
Research (INCHER-Kassel)
The centre is 32 years old Research Centre of the University of Kassel
About 20-30 researchers
About 15 doctoral students Study programme: Master of Higher Education
Research focus: Higher education and work; HE
systems and structures; (international) graduatesurveys (CHEERS; REFLEX; PROFLEX; HEGESCO)
International network of graduate surveys
8
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
9/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
INCHER
www.uni-kassel.de/incher
9
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
10/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
International Tracer Study
Cooperation
10
INCHERKassel
Central AmericaCosta Rica, El Salvadore,
Guatemala, Mexico,Nicaragua, Panama
EuropeAustria, Belgium, Czech Republic, Estonia, Finland, France,
Italy,Netherlands, Norway, Portugal, Romania, Spain, Sweden,
Switzerland, United Kingdom
Asia
Indonesia, Iran,Japan, Oman,
Philippines
AfricaCameroon, Ghana,
Kenia, Malawi, Namibia,Nigeria, South Africa,
Uganda, Tanzania
South AmericaArgentina, Brasil, Chile,
Colombia, Peru
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
11/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
2
Theoretical
Framework
11
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
12/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
12 Conceptual Framework for
Graduate Surveys
InputStudents biodata,
experiences,motives
OutputKowledge,skills,
motivation,grades
InputStudy conditionsand provisions
ProcessTeaching and
learning
Outcomes
Transitionemployment,
work
service tosociety
Graduate Survey
Context: Labour market Region Country; Socio-economicdevelopment and personal development
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
13/50
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
14/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
Key Theoretical Problems:
Causal Relevance of Study Programs
14
1. Which indicators do we need? (search duration,employment status, income, position, level match, field of
study match, work autonomy, job satisfaction, etc.)
2. How to measure the labour market signals?
3. Which factors are relevant to explain the success
of the graduates besides higher education?(labour market, regional mobility, individualmotivation, etc.
4. To what extent is the professionalsuccess/career caused by HE?
5. Which elements of HE (study conditions and
provisions, etc) have an impact?
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
15/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
3.
Methodology
of Tracer
Studies
15
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
16/50
Sch
omburg:ConceptandMethodologyof
Tra
cerStudies
What are (Graduate) Tracer
Studies? Similar terms
Graduate survey (40,700 Google hits) Tracer study (98,000 Google hits)
Alumni research (U.S.; 99,000 hits)
Follow-up study (graduates) (179.000hits)
Graduate career tracking (Hungary; 6 hits)
Target population Graduatesof an institution of higher
education (regardless the kind of degree)
Surveyed aftergraduation
16
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
17/50
Sch
omburg:ConceptandMethodologyof
TracerStudies
17
Typical Objectives of Institutional
Graduate Surveys (Tracer Studies)
1. To get valueable
information for thedevelopment of theuniversity
2. To evaluate the
relevance of highereducation
3. To contribute to
the accreditationprocess
4. To inform students,
parents, lecturers andadministrators
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
18/50
Sch
omburg:ConceptandMethodo
logyof
Tra
cerStudies
Different Design of Graduate
SurveysExamples Time after
graduationCohorts Length of
questionnaire
UnitedKingdom
Six months OneShort
(one page)
Costa Rica(CONARE)
One yearafter
OneMedium
(10 pages)
EuropeOne to tenyears after
One ordifferent
Medium tolong
(10-20 pages)
18
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
19/50
Sch
omburg:ConceptandMethodologyof
Tra
cerStudies
Scope of Graduate Tracer Studies
and Target Population
D. Type ofdegree
C. Subject
B. Institution
A. Country Country A
UniversityA
Subject A
BA MA
Subject B
UniversityB
Subject A
19
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
20/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
20 Objectives of Institutional
Graduate Surveys in GermanyMulti-functional surveys with broad focus Employment, labour market signals, professional
success Competences and work
Retrospective rating/evaluation of study
Curricular development
Continuing education (marketing)
Networking, alumni databank
Evaluation quality assurance accreditation
Multi-institutional surveys Benchmarking
Comparison with relevant others not with the
average
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
21/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
Multifunctional survey the
Netherlands
Quality assurance (and increasingly
accreditation) Alumni-tracking
National monitoring of transition from higher
education to work Study choice information for school-leavers
(both at national and institution level)
Input for labour market forecasting
Scientific and policy analyses
21
Source: Jim Allen (ROA, University of Maastricht, Netherlands), Kassel 2008
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
22/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
Multifunctional survey -
Switzerland
information for political authorities
(education/employment sectors) benchmarking (System of indicators of
Swiss higher education institutes)
evaluation and information tool forhigher education institutes
information for career/study counselling
services
22
Source: Andrea Witmer (Swiss Federal Statistical Office), Kassel 2008
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
23/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
Graduate Survey Systems in
Europe (1) (2009)Regular national
surveysInstitutional
surveys
Germany, France, Italy Yes Frequent
United Kingdom Yes Seldom
Netherlands, Norway.
Switzerland
Yes Seldom
Greece, Spain, Portugal,Austria
No Seldom
Finland, Sweden No Frequent
Poland, (Hungary), Estonia,Romania, Czech Republic,etc.
No Seldom
23
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
24/50
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
25/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
4.
The New GRADNETApproach: Networkof Graduate
Surveys
25
Th T St d R l ti i26
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
26/50
Sch
omburg:Conce
ptandMethodo
logyof
Tra
cerStudies
The Tracer Study Revolution in
Germany
Until 2007: only a few institutions of
HE conducted regular tracer studies
2008/2009 almost all relevantinstitutions of HE are partners indifferent regional or national systems
of regular tracer studies Coverage: about 80% of all graduates
in Germany
26
27
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
27/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
27 Kind of Graduate Surveys in Germany
until 2007 International Surveys: INCHER-Kassel (CHEERS
and REFLEX)
Regular nationwide surveys: HIS (Higher EducattionInformation Centre)
Ranking of HE institutions, selected fields ofstudy: CHE (Centre for Higher Education Development)
Regional surveys: Bavaria
Regular surveys of single institutions: seldom
TU Dresden, Uni Rostock
FH Wiesbaden, FH Aachen
Ad hoc surveys of single faculties/institutions:frequent
28
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
28/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
28
The Nationwide HIS Surveys
Every forth cohort (1989, 1993, 1997, 2001, 2005, 2009)
Only first degree
Paneldesign
First wave: about 12 months after graduation
Second wave: about 5 years after graduation
Sample: About 12,000 graduates in firstwave
Paper questionnaire
No comparisons between single HEinstitutions
29
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
29/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
The Classical Approach: die University
Delivers Data (e.g. Addresses)
Research institute
University A University B University C
Researcher
Researcher
Researcher
Ministry of HE
29
Data
Data
Data
30
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
30/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
The German Network Approach
INCHER-Kassel
Uni 1
Uni 2
Uni 3
Uni 4
Uni 5
Uni 6
Uni
Uni60
30
2010: About 60 institutions of HE in Germany are cooperating inconducting regular graduate tracer studies
31
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
31/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
Key Aspects of Cooperation Universities and INCHER-Kassel are
developing the core questionnaire
together Every university has an own
questionnaire INCHER gives advice
Central data processing joint database
Every university gets customized tablereports (about 500 pages)
Benchmarking according the needs of
the universities
31
32 O i ti f GRADNET t
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
32/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
32 Organisation of GRADNET at
University LevelUniversityProject-
coordinator
University
leader
ProjectCoordinator
Faculty
Project
CoordinatorFaculty
Task force Task force
Task force
Career, Alumni,
INCHER
Team
33 The German Graduate Survey
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
33/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
33 The German Graduate Survey
Network ProjectProject-coordinators from 35 universities and
fachhochschulen during a workshop in Kassel; Sept. 2008
The Bridge Concept HE34
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
34/50
Schomburg:Conce
ptandMethodo
logyof
Tra
cerStudies
The Bridge Concept HE
Research and Application
34
35 GRADNET: Network of Graduate
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
35/50
Schomburg:Conce
ptandMethodo
logyof
TracerStudies
35 GRADNET: Network of Graduate
Surveys
Established 2006 by INCHER-Kassel
2008: about 250 members German language
Individual membership (email list)
Two conferences per year
Professionalistion of tracer studyresearchers
Core group
Initiative for the RESEARCH PROJECTStudy Conditions and Professional Success
36
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
36/50
Schomburg:Conce
ptandMethodo
logyof
TracerStudies
36
Background: New Challenges
Growing competition of HE institutions
Growing international competition
New study structure (Bologna process)
Employability
Evaluation and Accreditation(Accountability; outcomeassessment)
Growing service demand from students New HE management philosophy
quality management
37 (Future) Design of Graduate
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
37/50
Schomburg:Conce
ptandMethodologyof
TracerStudies
(Future) Design of Graduate
Surveys
Exit Survey Graduate
Survey I
Graduate
Survey II
1- 2 years 4 - 5 years
Time After Graduation
38 New Survey System in Germany:
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
38/50
Schomburg:ConceptandMethodologyof
TracerStudies
New Survey System in Germany:
the GRADNET SURVEYS (INCHER)Year ofgraduation
Year of Graduate Survey
2007/2008Dec.-March
2008/2009Oct.-Feb.
2009/2010Oct.-Feb.
2010Oct.-Dec
2011
2006(9 HEinstitutions,15,000graduates)
1.5 yearsafter
graduation(done)
4.5 yearsafter
graduation
2007(48 HEinstitutions;80,000graduates)
1.5 yearsafter
graduation(done)
4.5 yearsafter
graduation
2008(46 HEinstitutions;75,000graduates)
1.5 yearsafter
graduation(running)
Panel
Panel
Panel
The GRADNET HE Institutions (Jan39
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
39/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
The GRADNET HE Institutions (Jan.
2010)1. A 27. 2. B 28.
3. B 29.
4. B B* 30. A
5. B C 31.
6. CA * 32. ()
A
7. 33. B
8. A 34.
9. B/ 35. B*
10. D 36. D
11. 37. D
12. 38.
13. 39. B
14. B 40. B15. B 41.
16. 42. ()
17. A 43.
18. D 44.
19. A 45.
20. B 46. 21. 47. 22. B B 48.
23. 49. *
24. 50.
25. B 51.
26. 52.
* Did not participate in the 2009 survey
Design and Conduction of the40
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
40/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
Design and Conduction of the
GRADNET Surveys 2008-2010
C ( )
20062007 2008
D. 2007
2008. 2008 . 2009
. 2009 . 2010
9 48 46
15.200 86.800 76.000
12.800 75.000 70.000
INCHER-Kassel: German Graduate Surveys Network Project (GRADNET); survey 2009; cohort 2007,1.5 years after graduation
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
41/50
42
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
42/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
Cooperation for development
Every university has an individual questionnaire(2009: 160 surveys were administrated at thesame time by INCHER-Kassel)
Every university has a project coordinator
At least 6 training workshops per survey
Every university received the cleaned data andready to print table reports of about 500 pages
Many universities published the results in theinternet
A lot of internal and external disseminationactivities
2009 survey: dataset with 36,000 graduates
43 Implementation of the German
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
43/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
Implementation of the German
Network Approach
Address database of the HE institution
Update of addresses by HE institution
Four contacts/send outs
Contacts by letters and/or email Combination of paper and online
questionnaire
Response rate should be about 50%
Selected Results of the German GRADNET44
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
44/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
Selected Results of the German GRADNETSurvey 2009 (participants: 36,000 graduates)
1. Successful cooperation for development among 50 HEinstitutions in Germany
2. Graduates of the year 2007: optimal labour market
conditions in Germany a success story3. Negative scenarios do not find empirical support no trend
towards a Generation Trainees
4. Differences by field of study are less important
5. Remarkable good employment conditions for Bachelorgraduates
6. Growing relevance of soft skills
7. Reputation of the HE institution seems to be not important
8. BA graduates have not less international experiences
9. Transition to first job depends on own initiatives
10. Master the winners of the Bologna process
45
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
45/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
5.
Conclusion
Status of Research on Causal46
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
46/50
Sc
homburg:ConceptandMethodologyof
TracerStudies
Relevance of Higher Education
How college affects students?
(Pascarella/Terrenzini, mostcompehensive synthesis report, 2005)
Little evidence
Little net effects of programme quality
Still in search of usable knowledge
(Peter Ewell, National Center forHigher Education ManagementSystems (NCHEMS))
47
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
47/50
Sc
homburg:ConceptandMethodologyof
Tr
acerStudies
Many Open Questions
How to measure study conditions and studyprovisions
Does program quality matters for employmentoutcomes?
Which student characteristics must be controlled?
What are the best indicators of employmentoutcome and professional success?
How to conduct surveys efficiently with highquality? What are the standards for quality?
How should the data be analyzed to gain usableknowledge?
48
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
48/50
Sc
homburg:ConceptandMethodologyof
Tr
acerStudies
Outlook
The success of cooperation should beestablished on an international level(European Higher Education Area - The Bologna Process 46countries)
(International) exchange of concepts
and experiencesINCHER cooperates with Indonesia, Oman, Romania, and manyother countries
(International) cooperation between
universities International data base
International network
International projects
49
Further information
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
49/50
Sc
homburg:ConceptandMethod
ologyof
Tr
acerStudies
Further information
http://www.uni-
kassel.de/incher/absolventThe German Graduate Survey Network
http://www.his.de
The HIS Graduate Surveys
http:// http://www.ihf.bayern.de
The Bavarian Graduate Panel Survey
50
8/9/2019 Concept and Methodology of Tracer Studies Harald Schomburg
50/50
Sc
homburg:Conc
eptandMethod
ologyof
Tr
acerStudies
THANK YOU