Embed Size (px)
Citation preview
Professionalization of evaluation –recent activities within DeGEval
Dr. Jan Ulrich HenseDepartment Psychologie, Empirische Pädagogik und Pädagogische [email protected]
Jahreskongress SEVAL 2012, Workshop 2 Freitag, 7. September 2012Université de Fribourg
# 2Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
(1) Some comments and questions on the
professionalization of evaluation
# 3Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Two uses of „professional“
1. Referring to a job or designation (“a professional painter”)• Is evaluation developing into a full-blown profession?
2. Referring to performance on a task (“a professional repair”)• How to carry out evaluations professionally?
# 4Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
(Mieg, 2010, p. 16)
Evaluators?
# 5Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Two Classes of Professions (Mieg, 2010)
1. Care and relationship orientation Clients or patients trust professionals’ expertise and integrity
Readiness to work cost free in emergency situation
e.g. Doctors, Lawyers, Psychotherapists
2. Market orientation Price competition
Income orientation
e.g. Engineers, Consultants
# 6Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Professionalization as regulation of autonomy
Definition of education and training demands Control of market entry Determination of fees and charges Monopoly of performance evaluation
(Mieg, 2010)
# 7Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Conditions for the development of a new profession
1. Combination of a problem relevant for society and
knowledge on how to explain and deal with the problem
2. Relation to central values of society (e.g. health, justice)
3. (Mostly) academic education and training4. Professional societies
(Mieg, 2003)
# 8Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Phases of development of a profession (Wilensky, 1964)
1. Some are working in a new area (mostly) full time
2. Education and training programs are developed
3. Professional societies emerge
Shaping profile by delegating “lower” tasks
Conflict between established and graduates of new training programs
Competition with neighboring professions
4. Professional Codes und Standards
5. Move to formal closure
Rules and conditions for working in the new profession
Protection of designation
Declaring non licensed work illegal
# 9Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
(2) Recent DeGEval activities in the context of
professionalisation
1. Competencies for evaluators2. Didactics of Evaluation3. Further projects and plans
# 10Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
1. Competencies for evaluators
# 11Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Why develop "Recommendations on Education and Training in Evaluation"?
Growing demand for evaluation services
Lack of qualified personnel Lack of a professional profile
Guidelines as contribution to the professionalization of evaluation
# 12Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
A bit of history...
2004: First German edition• Authors: A. Caspari, M. Hennen,
D. Scheffler, U. Schmidt & O. Schwab
• Published by the DeGEVal
working group "Education
and Training in Evaluation“
2008: English translation
download/order on www.degeval.de
# 13Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Goals and intended uses
Standards for• Training programmes
– Direction for programme developers and teachers– Evaluation of existing programmes
• Assessing evaluators– Choice of an evaluator by clients– Self-evaluation of evaluators
# 14Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
"A competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context."
(OECD/OCDE DeSeCo-Project)
Which knowledge, skills and attitudes are needed for conducting professional evaluations?
# 15Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
# 21Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Development of the recommendations
Theoretical approach Based on DeGEval standards of evaluation
Practical procedure Expert panel Feedback cycles from DeGEval members Officially adopted by DeGEval conference
# 22Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Who uses the guidelines?
Usage in advanced study programmes• Berne
• Saarbrücken
• Bonn
Further education courses ? Evaluation courses within
domain-specific curricula ?
# 23Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Soellner, R., Hapkemeyer, J. & Scheibner, N. (2010). Evaluationsausbildung in der Psychologie – eine Bestandsaufnahme. Zeitschrift für Evaluation, 9 (2), 297-308.
Coverage of competencies in psychology diploma programmes
# 24Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Need for revision (to be discussed)
Concept of competencies (KSA) Field-specific interpretation and differentiation Elaboration of methodological competencies Compatibility with international models of
evaluator competencies, e.g.• King et al. (2001; 2005); Mertens (1994)
• Zorzi, McGuire & Perrin (2002)
• Russ-Eft et al. (2008)
# 25Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
2. Didactics of Evaluation
# 26Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
htt
p:/
/ww
w.d
egeval.d
e/a
rbeit
skre
ise/
au
s-u
nd-w
eit
erb
ildung
-in-d
er-
evalu
ati
on/
pub
likati
onen-u
nd-p
rodukt
e
# 27Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Why didactics of Evaluation?
Competency profiles• Contents and learning goals of training programmes
• What to teach
Didactics of evaluation• Practical guidelines for teachers and trainers
• How to teach
# 28Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Goals
Provide guidance for• training and education programmes
• courses in evaluation
• self-learning materials (textbooks etc.)
Based on current state of theories on learning and instruction
Supplement existing recommendations on competency profiles
# 29Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
0 Executive Summary1. Introduction2. Learning contents and goals:
Competence profiles for evaluators
3. Programmes4. Types of programmes and target
groups5. Didactical foundation6. Learning and teaching methods7. Requirements for teachers/trainers8. Outlook
Condensed“Cookbook” guidelines
# 30Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Important Messages
Cover whole range of competencies (KSA) Apply a mix of construction and instruction
• Emphasize active, self-directed learning
• Allow for learners’ own practical experiences
• Provide instructional support
• Embed all contents in authentic evaluation contexts
Match method and learning goals Adapt to learners‘ individual needs and
preconditions
# 31Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Examples
Chocolate cookie exercise• Exercise on the logic of evaluation
Reflection exercise „An evaluator is like a ...“• Discuss different evaluator roles and responsibilities
Role play „stakeholder resistance“• Awareness for possible sources of resistance to evaluation
# 32Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
3. Further projects and plans
# 33Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan HenseFuture projects
Other topics and activities of the DeGEval working group “Education and Training in Evaluation“
Overview of training courses and seminars (online-database)
Textbook overview and review Evaluation capacity building Certification and credentialing of evaluators
# 34Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan HenseFuture projects
Overall DeGEval efforts
Work group on ethics in evaluation (“ethical codes”)
Revision of evaluation standards (until 2014)
# 35Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)
Dr. Jan Hense
Thank you for your interest!
contact:[email protected]
