Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
MBA Program Workshop
“Linking MBA Program and Course Learning Outcomes to QFEmirates Level 9 Descriptors”
Thursday 28th May, 2015
0930 – 1230
Hosted by the University of Sharjah
1
Facilitators
Professor James Jordan-Wagner Dean, W Frank Barton School of Business, Wichita State
University, USA
Professor Mostafa Sarhan Dean, College of Business Administration, Savannah State
University, USA
Dr Wil Williams Commission for Academic Accreditation
2
Purpose
To improve the writing of MBA program (PLOs) and course learning outcomes (CLOs) and the design of assessment regimes and instruments that should be aligned to QFEmirates Level 9 Descriptors.
3
Learning Outcomes
Participants in this workshop will gain: – an overview of key issues that the CAA have
identified in MBA program initial, full and renewal accreditation events;
– an in-depth understanding of the relationship between QFEmirates and the CAA’s Standards;
– a critical appreciation of writing MBA PLOs and CLOs at an appropriate level;
– an understanding of how to design MBA assessment regimes appropriate for the level of study; and
– a critical appreciation of MBA assessment instruments that are set at an appropriate level.
4 H
Agenda
Time Activity
0900 - 0930 Registration & coffee
0930 - 0945 Welcome & introduction
0945 - 1015 The nature of the MBA in the UAE - Issues with Initial and Renewal of MBA Accreditation
1015 - 1045 QFEmirates Level 9 and the CAA’s Standards
1045 - 1115 Guide to writing PLOs and CLOs
1115 - 1130 Coffee
1130 - 1200 Linking PLOs and CLOs to assessment
1200 - 1230 Plenary
1230 Depart
5
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA Accreditation
6
7
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
• 32 MBA programs in the UAE • Earliest: 2001 • Latest: 2015 • Number of students - Fall 2014: 6,403 • Initial Accreditation • Renewal of Accreditation • CAA Standards (2011) • QFEmirates (2012)
8
9
MBA: Private Institutions
10
MBA: Private Institutions
11
14
4
2
2
11
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
Fall 2010 Fall 2011 Fall 2012 Fall 2013 Fall 2014
MBA Total Number of Students By Gender
Female
Male
12
0 1,000 2,000 3,000 4,000 5,000 6,000 7,000
Fall 2010
Fall 2011
Fall 2012
Fall 2013
Fall 2014
MBA Total Number of Students By Nationality
UAE
Expat
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues –
• Institutional Planning (2.2): Too few programs evidence ‘closing the loop’
• Credit-bearing Program (3.1):
– Lack of market intelligence and analysis (3.1.1)
– Poor alignment of PLOs, CLOs and assessment to QFEmirates Level 9 Descriptors (3.1.3)
– Full-time programs being completed within 12 months – Too few contact and learning hours (3.1.5)
13
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues (Cont.) –
• The Curricula (3.2)
– Structured more like specialist MSc programs (3.2.1)
• Academic Courses (3.3)
– Lack of evidence of student active engagement (3.3.2)
– Poor alignment of PLOs, CLOs and assessment to QFEmirates Level 9 Descriptors (3.3.3)
– Cont.…
14
Issues (Cont.) –
• Academic Courses (Cont.) (3.3) – Use of inappropriate assessment tools, e.g., quizzes, short (very)
essay type questions and ‘participation’ (attendance) (3.3.4)
– Syllabi (3.3.5; Stipulation 5)
• Poor rationale for course selection (2)
• The lack of grading rubrics/ guides
• Insufficient information on course topics (N.B. Rarely explicitly linked to CLOs and assessment) and assessment;
• Out of date texts; and
• Lack of references to journals (5)
– Cont.…
15
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues (Cont.) –
• Academic Courses (Cont.) (3.3) – Condensed programs/ courses: too little time allowed for
preparation, reflection and analysis (3.3.6; also see 3.1.5)
– Lack of evidence of the use of adequate plagiarism prevention safeguards (3.6.8)
– Insufficient infrastructure to support blended learning provision (3.3.9)
• Graduate Programs: – Lack of research training (3.7.1)
– Lack of evidence of sufficient rigor, and courses being distinctly more challenging than undergraduate courses in the same discipline (3.7.2)
16
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues (Cont.) –
• Course Delivery:
– Poor alignment of PLOs, CLOs and assessment to QFEmirates Level 9 Descriptors (3.8.2)
– Course files stored and presented in poor order (3.8.3)
• Class sizes – too large (3.9)
• Program Effectiveness - Too few programs evidence ‘closing the loop’ (3.10)
17
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues (Cont.) –
• Faculty
– Lack of evidence of professional development - research and scholarship (4.8.1)
– High workloads (4.9)
– Too many part-time faculty (4.10)
• Student Recruitment:
– Students lack business and management experience (5.3.1.1)
– High number of conditional entrants (5.3.1.8)
18
The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA
Accreditation
Issues (Cont.) –
• Public Disclosure – Poor presentation of accreditation (self studies) documentation (9.2; also see 3.8.3)
• Research and scholarship – weak faculty research outputs (10)
19
QFEmirates Level 9 and the CAA’s Standards
20
QFEmirates Level 9 and the CAA’s Standards
QFEmirates Strands
21
1 2 3 4 5 6 7 8 9
10
Knowledge
Leve
ls
Skills Competence
22
QFEmirates Level 9 and the CAA’s Standards
• QFEmirates Level 9 – Annexure C: Grid of Level Descriptors
– Annexure D: Summary of Level Descriptors
– Annexure G: Appendix 18 – Generic Qualification Profile 9.1 (Master)
– Annexure F: Employability Indicators of QFEmirates Levels
23
Level Knowledge Skill
Competence
Autonomy &
responsibility
Role in
context
Self-
development
9 comprehensive, highly specialized knowledge
in a field of work,
discipline and/or professional practice,
and at the interface
between different fields,
including frontier
concepts and recent
developments
advanced knowledge of
applicable research principles and methods
critical awareness of
knowledge issues, as the
basis for original
thinking; encompassing appropriate processes of
enquiry and current
processes of knowledge production
detailed body of
knowledge of recent
developments in a field
of work, and/or discipline
advanced skills required in research,
analysis, evaluation
and/ or innovation of complex ideas,
information, concepts
and/or activities
skills to develop new knowledge and
procedures and to
integrate knowledge from different fields
using highly developed
cognitive and creative
skills and intellectual
independence to the
field of work or discipline
advanced problem-solving skills to analyze
highly complex issues
with incomplete data
and develop innovative
solutions and proposals
relevant to an academic/ professional
field, field of work or
discipline
planning skills to
develop and execute a major project or
comparable activities
(that includes a significant range of
variables and
complexity) with appropriately selected
research methodologies
producing sound conclusions
highly developed specialist
communication and information technology
skills to present,
explain and/or critique highly complex matters
can function autonomously and/
or take responsibility
for managing professional
practices, work,
processes or systems,
or learning
contexts that are
highly complex, unpredictable and
unfamiliar, and
require new strategic approaches
and/or intervention
or conceptual
abstract solutions
can account for high
level governance of
processes and
systems can analyze and
reflect on
sociocultural
norms and
relationships
and act to build and transform
them
can initiate and manage
professional
activities that may include a
highly complex
environment
can take
responsibility for leading the
strategic
performance and development of
professional
teams and self
can self-evaluate and take
responsibility for
contributing to professional
knowledge and
practice including
unfamiliar learning
contexts
can develop and
implement further
learning consistently and
sensitively can
consistently and
sensitively
manage highly complex ethical
issues leading to
informed, fair and valid decisions
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors
24 H
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors • Strand: Knowledge
– comprehensive, highly specialized knowledge in a field of work, discipline and/or professional practice, and at the interface between different fields, including frontier concepts and recent developments
– advanced knowledge of applicable research principles and methods
– critical awareness of knowledge issues, as the basis for original thinking; encompassing appropriate processes of enquiry and current processes of knowledge production
– detailed body of knowledge of recent developments in a field of work, and/or discipline
25 H Grid
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors
• Strand: Skills
– advanced skills required in research, analysis, evaluation and/ or innovation of complex ideas, information, concepts and/or activities
– skills to develop new knowledge and procedures and to integrate knowledge from different fields using highly developed cognitive and creative skills and intellectual independence to the field of work or discipline
– advanced problem-solving skills to analyze highly complex issues with incomplete data and develop innovative solutions and proposals relevant to an academic/ professional field, field of work or discipline
– planning skills to develop and execute a major project or comparable activities (that includes a significant range of variables and complexity) with appropriately selected research methodologies producing sound conclusions
– highly developed specialist communication and information technology skills to present, explain and/or critique highly complex matters
26 H Grid
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors
• Strand: Competencies - Autonomy and Responsibility
– can function autonomously and/ or take responsibility for managing professional practices, work, processes or systems, or learning contexts that are highly complex, unpredictable and unfamiliar, and require new strategic approaches and/or intervention or conceptual abstract solutions
– can account for high level governance of processes and systems
– can analyze and reflect on sociocultural norms and relationships and act
27 H Grid
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors
• Strand: Competencies – Role in Context
– can initiate and manage professional activities that may include a highly complex environment
– can take responsibility for leading the strategic performance and development of professional teams and self
28 H Grid
QFEmirates Level 9 and the Standards
• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors
• Strand: Competencies – Self-Development
– can self-evaluate and take responsibility for contributing to professional knowledge and practice including unfamiliar learning contexts
– can develop and implement further learning consistently and sensitively can consistently and sensitively
– manage highly complex ethical issues leading to informed, fair and valid decisions
29 H Grid
QFEmirates Level 9 and the Standards
• Standards explicitly linked to QFEmirates: – 3.1.3
– 3.3.3
– 3.8.2
30
QFEmirates Level 9 and the Standards
• Standards linked to QFEmirates: – 3.1.3 Each academic program has well-articulated program goals
and outcomes (i.e. learning outcomes at the level of the program); the outcomes are derived from the goals and are defined in measurable terms; all program goals and outcomes are accurately described in appropriate publications. Goals and outcomes are consistent with the level of qualification awarded as defined in the UAE Qualifications Framework.
– 3.3.3 provides learning outcomes for each course that are consistent with current international norms in the discipline and the level of the course and the qualification awarded; and align with, and demonstrably meet, the program outcomes of each program in which the course appears
– 3.8.2 ensures that the academic assessment of students is fair, accurate, aligned with learning outcomes and program goals, and is undertaken at an appropriate level
31 H
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to: – Understand social, professional and/or ethical responsibilities in business
environment
– Communicate business issues effectively
– Critically examine the role of management accounting information and related ethical issues in planning, organizing, and controlling activities of a firm
– Implement a strategic plan that takes into account the key functional resources of a firm
– Develop advanced skills in business research application, analysis, and evaluation as well as the ability to innovate new ideas, concepts and business practice
32 E
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to:
–Understand social, professional and/or ethical responsibilities in business environment
33 E
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to: – Understand social, professional and/or ethical responsibilities in business
environment
–Communicate business issues effectively
34 E
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to: – Understand social, professional and/or ethical responsibilities in business
environment
– Communicate business issues effectively
– Critically examine the role of management accounting information and related ethical issues in planning, organizing, and controlling activities of a firm
35 E
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to: – Understand social, professional and/or ethical responsibilities in business
environment
– Communicate business issues effectively
– Critically examine the role of management accounting information and related ethical issues in planning, organizing, and controlling activities of a firm
– Implement a strategic plan that takes into account the key functional resources of a firm
36 E
QFEmirates Level 9 and the Standards
• Exercise: Card Sort
– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the
graduate will be able to: – Understand social, professional and/or ethical responsibilities in business environment
– Communicate business issues effectively
– Critically examine the role of management accounting information and related ethical issues in planning, organizing, and controlling activities of a firm
– Implement a strategic plan that takes into account the key functional resources of a firm
–Develop advanced skills in business research application, analysis, and evaluation as well as the ability to innovate new ideas, concepts and business practice
37 E
Guide to writing PLOs and CLOs
38
Guide to Writing PLOs and CLOs
Definition of a learning outcome
“Learning outcomes are statements of what a student is expected to know, understand and/or be able to demonstrate after completion of a process of learning.”
(Directorate-General for Education, 2015, p.47)
39
Guide to writing PLOs and CLOs
• Aims, goals, objectives and learning outcomes
– Applicable to both programs and courses (module)
– An aim is a broad general statement of the teaching intention
– Goals are the broad educational expectations for each degree program (AACSB)
– An objective is a specific statement about the intended result of teaching (Instructor perspective)
– An outcome is a specific statement about the achieved result of teaching (Student perspective)
40
Guide to Writing PLOs and CLOs
• Objectives and outcomes:
– Objectives – intended results
– Outcomes – achieved results (what you will measure)
41
“[A] general lack of agreement as to the format of objectives is a complication, and justifies the abandonment of the use of the term ‘objective’ in the description of modules or programmes.” (Moon, 2002)
Guide to writing PLOs and CLOs
42
Program - Mission/ Aim - Goals - Outcomes
Course - Mission/ Aim - Outcomes
Guide to Writing PLOs and CLOs
• Learning outcomes
– A student-centered approach
– Closely linked to the competency approach to learning
43
Guide to Writing PLOs and CLOs
Why use learning outcomes? Because they:
• help students learn more effectively through transparency;
• help instructors to design their teaching and learning approach and materials;
• help instructors more precisely to tell their colleagues and students what a particular activity is designed to achieve; and
• ensure that appropriate assessment approaches and instruments are employed.
(Source: Adapted from - SOAS, 2015)
44
Guide to Writing PLOs and CLOs
45
LO
Simple
Measurable Achievable
Realistic
Timely
“On completion of this program/course/module the student should be able to …”
Action Verb
Scope
Guidance
Guide to Writing PLOs and CLOs
• To achieve the required level focus on :
– The verbs used to describe the learning outcome;
– The scope of the task and quality of the guidance given to the students; and
– Quality of guidance: How much guidance the students will be given by the instructors and how much they are capable of doing things independently.
46
Guide to Writing PLOs and CLOs
47
1 per LO
Limit the number
Action Verb
LO Minimum standard to pass
Guide to Writing PLOs and CLOs
• Program Learning Outcomes (PLOs):
– An institution must specify PLOs for each separate degree program and concentrations within a degree
– PLOs will tend to be broader in nature
– PLOS assessed during, after or not at all
– Some PLOs maybe overarching
– Map PLOs to courses (and even CLOs)
48
Guide to Writing PLOs and CLOs
• Program Learning Outcomes (PLOs):
– An institution must specify PLOs for each separate degree program and concentrations within a degree
– PLOs will tend to be broader in nature
– PLOS assessed during, after or not at all
– Some PLOs maybe overarching
– Map PLOs to courses (and even CLOs)
49
Guide to Writing PLOs and CLOs
• Course Learning Outcomes (CLOs):
– Each course must specify CLOs
– They are specific
– Fulfil the requirements of one or more of the program learning outcomes;
– The CLOs should be mapped to the PLOs to ensure that the PLOS are met
50
Guide to Writing PLOs and CLOs
• Course Learning Outcomes (CLOs):
– Each course must specify CLOs
– They are specific
– Fulfil the requirements of one or more of the program learning outcomes;
– The CLOs should be mapped to the PLOs to ensure that the PLOS are met
51
Guide to Writing PLOs and CLOs
• A useful construct to link QFEmirates qualification levels to PLOs and CLOs is Bloom’s Taxonomy
• Bloom’s original taxonomy (1956) has three parts (or domains): 1. Cognitive – knowing
2. Affective – thinking
3. Behavioral (psychomotor)
– able to do
52
Guide to Writing PLOs and CLOs
• A useful construct to link QFEmirates qualification levels to PLOs and CLOs is Bloom’s Taxonomy
• Bloom’s original taxonomy (1956) has three parts (or domains): 1. Cognitive – knowing
2. Affective – thinking
3. Behavioral (psychomotor)
– able to do
53
Guide to Writing PLOs and CLOs
Bloom’s Taxonomy - Cognitive
1. Knowledge
2. Comprehension
3. Application
4. Analysis
5. Synthesis
6. Evaluation
Leve
ls o
f K
no
win
g
54
Guide to Writing PLOs and CLOs
• Action Verbs: – Analysis
• Analyze, appraise, arrange, break down, calculate, categorize, classify, compare, connect, contrast, criticize, debate, deduce, determine, differentiate, discriminate, distinguish, divide, examine, experiment, identify, illustrate, infer, inspect, investigate, order, outline, point out, question, relate, separate, sub-divide, test.
– Synthesis • Argue, arrange, assemble, categorize, collect, combine, compile, compose,
construct, create, design, develop, devise, establish, explain, formulate, generalize, generate, integrate, invent, make, manage, modify, organize, originate, plan, prepare, propose, rearrange, reconstruct, relate, reorganize, revise, rewrite, set up, summarize.
– Evaluation • Appraise, ascertain, argue, assess, attach, choose, compare, conclude,
contrast, convince, criticize, decide, defend, discriminate, explain, evaluate, grade, interpret, judge, justify, measure, predict, rate, recommend, relate, resolve, revise, score, summarize, support, validate, value.
55 H
Guide to Writing PLOs and CLOs
Table 3.5 Checklist for writing learning outcomes Have I focused on outcomes not processes, i.e. have I focused on what
the students are able to demonstrate rather than on what I have done in my teaching?
Have I begun each outcome with an active verb? Have I used only one active verb per learning outcome? Have I avoided terms like know, understand, learn, be familiar with, be
exposed to, be acquainted with, and be aware of? Are my outcomes observable and measurable? Are my outcomes capable of being assessed? Have I included learning outcomes across the range of levels of Bloom’s
Taxonomy? Do all the outcomes fit within the aims and content of the module? Have I the recommended number of outcomes (maximum of nine per
module)? Is it realistic to achieve the learning outcomes (Source: Kennedy, 2006 )
56 H
Guide to Writing PLOs and CLOs
• Exercise 2: Grid to Action Verbs
– Examine QFEmirates : Annexure C: Grid of Level Descriptors
Identify key action verbs in the Level 9 Descriptors that relate to Bloom’s Levels:
• 4. Analysis
• 5. Synthesis
• 6. Evaluation
57 E
Linking PLOs and CLOs to Assessment
58
Linking PLOs and CLOs to Assessment
• Assessment
– Formative: assessment for learning
– Summative: summarizes student learning at a point in time
– Continuous assessment
– Grading rubrics
– Must be related to PLOs, CLOs, course content
– Must be set at the correct level, i.e., QFEmirates Level 9
59
Linking PLOs and CLOs to Assessment
• Assessment
– Formative: assessment for learning
– Summative: summarizes student learning at a point in time
– Continuous assessment
– Grading rubrics
– Must be related to PLOs, CLOs, course content
– Must be set at the correct level, i.e., QFEmirates Level 9
60
Linking PLOs and CLOs to Assessment
• Linking learning outcomes, teaching and learning activities and assessment:
1. Clearly define the CLOs
2. Select appropriate teaching and learning methods
3. Design an assessment regime that is appropriate to test the CLOs
4. Assess the learning outcomes and check to see how well they match with what was intended
5. Evaluate the research instruments in terms of the program
Source: Adapted from - SOAS, 2015)
61
Linking PLOs and CLOs to Assessment
• Measuring PLOs and CLOs
– Assessment is the only measure?
– Direct measures, e.g. examination and coursework results
– Indirect measures e.g., surveys of employers, comparison with peer institutions, surveys of past graduates, retention rates, analyses of curricula, etc.
62
Guide to Writing PLOs and CLOs
• Exercise 3: Assessment Questions
– Using the action verbs presented devise examination or coursework questions for a MBA course (of your choice) that test a student’s ability to (one question per level):
• Analyze
• Synthesize
• Evaluate
63 E
Plenary
64
Aligning PLOs, CLOs and Assessment
Source: Adapted from Moon
Program Aim/s QFEmirates Level Descriptors
Course Learning Outcomes (CLOs)
Program Learning Outcomes PLOs)
Development of assessment method to criteria (criteria implied by test
achievement of assessment criteria)
Write threshold assessment criteria (criteria implied by test achievement
of assessment criteria learning outcomes)
Course Delivery Approach Feedback loop to program and course development
Course Review
65
Aligning PLOs, CLOs and Assessment
Source: Adapted from Moon
Program Aim/s QFEmirates Level Descriptors
Course Learning Outcomes (CLOs)
Program Learning Outcomes PLOs)
Development of assessment method to criteria (criteria implied by test
achievement of assessment criteria)
Write threshold assessment criteria (criteria implied by test achievement
of assessment criteria learning outcomes)
Course Delivery Approach Feedback loop to program and course development
Course Review
66
PROCESS TO DEMONSTRATE PROGRAM ALIGNMENT WITH QFEMIRATES
67
Next CAA Program
Accreditation Review
Report no changes
CAA Substantive Change
Submission
Report changes
Institutional/College Program
Review Committee
Oversight and
guidance from
Institutional
QA Office
Negative alignment and need
for curriculum revision
Positive alignment with
appropriate QFEmirates level
Program team to modify
curriculum
Outcomes Matrix Map against
QF level descriptors
Plenary
Examples of good practice
68
Alignment is more than a paper exercise
69
71
Thanks to Prof. Ian Cumbus in the development of this material