Course Embedded Assessment of Graduate Learning: An Innovative Approach Based on The Degree Qualifications Profile Presented by William M. White, Ed. D

Course Embedded Assessment of Graduate Learning: An Innovative Approach Based on The Degree

Qualifications Profile

Presented by William M. White, Ed. D. and Indre Cuckler, M. A.

School of Graduate Studies

Who We Are

Who We Are

Who We Are

Who We Are

Outline

Levels of Assessment Definition of Assessment Theoretical Framework Program Goals and Outcomes Competency-Based Assessment Course Embedded Assessment Model Assessment Instruments Curriculum Mapping Resources Questions/Discussion

Levels of Assessment

Levels of Assessment:

Level 1. Assessing individual student learning within courses

Level 2. Assessing individual student learning across courses

Level 3. Assessing courses

Level 4. Assessing programs

Level 5. Assessing the institution(Miller & Leskes, 2005)

Definition of AssessmentDefinition of Assessment

Assessment is the systematic collection , review, and use of information about educational programs undertaken for the purpose of improving student learning and development.

(Palomba & Banta,1999)

Assessment Cycle

Establish

criteria for

success

Report

assessment results

Theoretical Framework

Theoretical Framework for MSU’s School of Graduate Studies Assessment

based on:

• Course Embedded Assessment model

• The Lumina Foundation’s Degree Qualifications Profile (DQP)

The Degree Qualifications Profile model allows our graduate programs to frame clear expectations of what students should be expected to know and be able to do at the associate, bachelor’s, and master’s level regardless of their academic discipline.

The Degree Qualifications Profile (2011)

Five Areas of Learning:1.Specialized Knowledge2.Broad, Integrative Knowledge3.Intellectual Skills Analytic Inquiry Use of Information Resources Engaging Diverse Perspectives Quantitative Fluency Communication Fluency

4. Applied Learning5. Civic Learning

Graduate Level Learning

• How do I know graduate level learning when I see it? • What are its characteristics?

The Degree Profile provides guidance to the above questions.Example:

Analytic inquiry

At the associate level, the student

Identifies, categorizes and distinguishes among elements of ideas, concepts, theories and/or practical approaches to standard problems.

At the bachelor’s level, the student

Differentiates and evaluates theories and approaches to complex standard and non-standard problems within his or her major field and at least one other academic field.

At the master’s level, the student

Disaggregates, adapts, reformulates and employs principal ideas, techniques or methods at the forefront of his or her field of study in the context of an essay or project.

Programs at MSU’s School of Graduate Studies

Mission

The School of Graduate Studies believes in the transformative power of lifelong inquiry to enhance lives. We seek to develop and maintain innovative graduate programs that fosteropportunities for personal and professional success in an increasingly multidisciplinary world.

Master of Health Science (GHSC), Master of Arts or Science in Interdisciplinary Studies (GIDS), Master of Criminal Justice Administration (GCJA), and Master of Arts in Psychology (GPSY)

Program GoalsProgram Goals Student Learning Outcomes(program level)

Student Learning Outcomes(program level)

G1: Provide an affordable and high quality graduate health science program.

G2: Foster student achievement of core competencies appropriate for health science professionals.

G3: Prepare students to serve the community and become leaders of change.

G4: Prepare graduates for careers in health administration and health services related areas.

1. Communication Skills: Students will be able to demonstrate the ability to create logical oral and written arguments, explanations and reflections based on their individual or collaborative efforts to both general and specialized audiences.

2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of the health science field in the context of an essay, a thesis or a project.

3. Knowledge, Sensitivity, and Awareness of Individual and Cultural Diversity: Students will demonstrate knowledge, sensitivity and awareness of individual and cultural diversity demonstrated through a project, paper or performance.

4. Values and Ethics: Students will assess and justify their engagement with respect to civic, social and global responsibilities.

5. Application of Knowledge and Skills in Health Science Professional Settings: Students will be able to create a project, paper, exhibit, performance or other appropriate artifacts reflecting the integration of knowledge acquired in a practicum, work community and /or research activities with knowledge and /or skills gained from at least two academic disciplines in different segments of the curriculum.

6. Qualitative and Mixed Methods Fluency: Students will demonstrate knowledge and ability to interpret, evaluate design and implement qualitative and mix-methods research appropriate to health science studies in a formal research document or by performance a project.

Note: student learning outcomes (1,2,&5) are adopted from the Lumina Foundation’s Degree Qualifications Profile

Master of Health Science (GHSC)



G1: Provide an affordable and high quality interdisciplinary studies program.

G2: Foster student achievement of core competencies appropriate for graduate level learners and professionals.


G4: Prepare graduates for careers in interdisciplinary fields according to students’ academic concentrations.


2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of their field of study in the context of an essay, a thesis or a project.


4. Broad, Integrative Knowledge: Students will articulate how their field (academic concentration) has developed in relation to other major domains of inquiry and/or practice.

5. Specialized Knowledge in an Academic Concentration Area: Students will be able to elucidate major theories, research methods, and approaches to inquiry and/or schools of practice in their field (academic concentration).

6. Research and Evaluation in Academic Concentration Area: Graduates will be able to utilize the scientific approach to knowledge generation and apply the appropriate research methods to practice.

Note: student learning outcomes (1,2,&5) are adopted from the Lumina Foundation’s Degree Qualifications Profile

Master of Arts or Science in Interdisciplinary Studies (GIDS)



G1: Provide an affordable and high quality graduate criminal justice administration program.

G2: Foster student achievement of core competencies appropriate for criminal justice administration professionals.


G4: Prepare graduates for careers in criminal justice administration field and related areas.




4. Broad, Integrative Knowledge: Students will articulate how criminal justice administration field has developed in relation to other major domains of inquiry and/or practice.

5. Engaging Diverse Perspectives: Students will address core issues in criminal justice administration from the perspective of either a different point in time, culture, language, political order, or technological context.

6. Application of Knowledge and Skills in Criminal Justice Administration Professional Settings: Students will be able to create a project, paper, exhibit, performance or other appropriate means reflecting the integration of knowledge acquired in a practicum, work community and /or research activities with knowledge and /or skills gained from at least two academic disciplines in different segments of the curriculum.

Note: student learning outcomes (1,2, 4,&6)are adopted from the Lumina Foundation’s Degree Qualifications Profile

Master of Criminal Justice Administration (GCJA)



G1: Provide an affordable and high quality graduate psychology program.

G2: Foster student achievement of core competencies appropriate for mental health and human services professionals.


G4: Prepare graduates for careers in mental health, human services, and related areas.

G5: Prepare students for doctoral and post graduate studies in psychology.




4. Information Literacy: Students will be able to provide evidence (through papers, projects, notebooks, computer files or catalogues) contributing to, expanding , assessing and/or refining either a broadly recognized information resource or an information base within the field of psychology.

5. Research and Evaluation in Psychology: Graduates will be able to utilize the scientific approach to knowledge generation in psychology and apply the appropriate research methods to practice.

6. Quantitative Fluency: Students will apply mathematical, formal logic and/or statistical tools to problems appropriate to the field of psychology in a project, paper or performance.

Note: student learning outcomes (1,2,4, & 6) are adopted from the Lumina Foundation’s Degree Qualifications Profile

Master of Arts in Psychology (GPSY)

Competency-Based Assessment

Competency - based assessment measures an individual’s performance against a predetermined criterion.

There are three levels of competency achievement throughout a program of study:

competency INTRODUCED (I) competency PRACTICED (P) competency REFINED (R)

Note: the competency achievement expectations are established by faculty in each course.

Competency-Based Assessment Chart

Yes

NoYes

No

Review courses and

revise curriculum

Evaluate student work at the

beginning of a program in the

pre-selected courses

Review courses

and revise curriculum

Write summary

report

Write summary

reportWrite summary

report

Write summary

report

Write summary

report

Write summary

report

Arecompetencies

PRACTICED (P)in a course?

Review courses

and revise curriculum No

Are competencies INTRODUCED (I)

in a course?

Yes

Evaluate student work in the middle of a

program in the pre-selected

courses

Are competencies

REFINED (R) ina course?

Evaluate student work

toward the end of a program

in the pre-selected courses

Arecompetencies

achieved?(Does the program do

what is intended?)

Curricular Alignment

Figure 1. Relationship among standards/objectives, instructional activities and materials, and assessments/tests. Adopted from Anderson (2002) Curricular Alignment: A Re-Examination.

A signature assignment to measure achievement of student learning is required in courses where assessment data are being collected for program level assessment.

Course Embedded Assessment

Standards/Objectives(MSU: Core Competencies)

Assessments/Tests(MSU: Rubrics)

Instructional Activities/ Materials

(MSU: Assignments)

Learning Outcome Assessment (2011-2012)

Three student learning outcomes (SLOs) are assessed in 2011-2012 cycle in all four graduate programs:

1. Communication Skills2. Critical Thinking3. Cultural & Social Awareness(Note: other student learning outcomes will be assessed in 2012-2013)

How are these outcomes assessed? By utilizing direct measures:• Written Assignments, and• Theses or Projects

andindirect measures: • Student Reflection Journals • Course Evaluations

Assessment Instruments

Rubrics are utilized to assess learning achievement.

Written Communication Skills Rubric

Critical Thinking Rubric

Cultural and Social Awareness Rubric

*Note: rubrics must meet graduate level learning expectations.

Student Learning Outcomes

Program level student learning outcomes must be reflected in individual courses.

Review your program’s outcomes and align them with your course outcomes.

All syllabi must include the following statement—

“Upon successful completion of this course, students will have the following competencies…” [list competencies]

Student Learning Outcomes (example)

IMPORTANT: Student learning outcomes have to be measurable!

course level

Curriculum Mapping

Curriculum maps

Shows when, what, and on what level learning is delivered throughout the program.

Why curriculum mapping is important?

Provides clarity and transparency of what is going on in the program and in specific courses.

Curriculum mapping is essential to program and course level assessment.

Program level student learning outcomes should be closely aligned with course level student learning outcomes and this alignment should be depicted in a curriculum map.

Core Courses GPSY 523 Professional Ethics

GPSY 545 History & Systems

GPSY 563 Research Methods

GPSY 505 Statistics

GPSY 517 Adv.

Develop. Psychology

GPSY 527 Social

Psychology

GPSY 520 Physiological Psychology

GPSY 525 Cognitive

Psychology

GPSY 610 Adv.

Principles of Learning

GPSY 620 Psychopathol

ogy

GPSY 630 Psychologic

al Testing

GPSY 670 Portfolio

GPSY 680 Thesis or Project

Program Outcomes

1.Communication Skills I

AP P P

AP P P P R R

A2.Critical Thinking

IA

P

P P PA

P P P P R RA

3.Information Literacy I

AP P P P P

AP P P P R R

A

4..Quantitative Fluency I

APA

R RA

5.Knowledge sensitivity,

awareness of individual and

cultural diversity

IA

P PA

P R RA

6.Research and evaluation in psychology

IA

P P PA

P P P RA

Master of Arts in Psychology Program (GPSY) Curriculum Map

Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.

Core Courses GHSC 501

Focus & Design

GHSC 563 Research Methods

GHSC 616 Health Care

Systems

GHCS 621 Health Care

Financial Mngm

GHSC 622 The Health

Care Executive

GHSC 503 Behavior in

Organizations

GHSC 504 Transforming Organizational

Cultures

GHSC 580 Concentrat.

area and field

experience

GHSC 581 Concentratio

n area and field

experience


area and field

experience

GHSC 583 Concentratio area and

field experience

GCJA 670 Portfolio

GCJA680 Thesis or Project

Program Outcomes


AP P P P P

AP P P P R R

A

2.Critical Thinking

IA

P P P P P PA

R RA

3.Application of HSC I

A P A

P P R RA

4.Values&

EthicsIA

PA

P R R RA

5.Knowledge sensitivity, and awareness of individual and

cultural diversity

IA

PA

P P P P R RA

6.Qualitative and Mixed Methods

FluencyIA

PA

R RA


Master of Health Science Program (GHSC) Curriculum Map (Concentration: Health Care Executive)

Core Courses

GHSC 501 Focus & Design

GHSC 563 Research Methods

GHSC 511 Multidsp.

Aspects of Aging

GHCS 515 Psyc. Aspects

of Aging

GHSC 516 Soc.

Aspects of Aging

GHSC 572 Working Across

Disabilities

GHSC 575 Public Policy


area and field

experience


area and field

experience

GHSC 582 Concentrat

area and field

experience

GHSC 583 Concentrat

area and field

experience

GCJA 670 Portfolio


Program Outcomes


AP P P P

AP R R

A

2.Critical Thinking I

AP P P P P

AP R R

A

3.Application of HSC I

AI P

A P P P R R

A

4.Values&

EthicsIA

PA

P P P R RA

5.Knowledge sensitivity,

and awareness of

individual and cultural

diversity

IA

PA

P P P R RA

6.Qualitative and Mixed Methods Research Fluency

IA

PA

R RA


Master of Health Science Program (GHSC) Curriculum Map (Concentration: Aging Studies)

Core Courses GIDS 501 Focus & Design

GIDS 550 Professional

Writing

GIDS 557 Interdisciplinary

Context and Theory

GIDS 563 Research Methods

GIDS 567 Statistical Methods

GIDS 580Concentration

area


area


area


area

GIDS 670 Portfolio

GIDS 680 Thesis or Project

Program Outcomes


AP P

AP P P R R

A

2.Critical Thinking

IA

P P P PA

P P P R R A

Specialized 3.knowledge in an

academic concentration

IA

PA

P P R R RA

4.Broad Integrative Knowledge

(interdisciplinary)

IA

PA

R RA

5. Knowledge sensitivity, and awareness of individual and

cultural diversity

IA

PA

P P P R RA

6.Research and evaluation in an

academic concentration

IA

PA

P P P R RA


MA/MS Interdisciplinary Studies Program (GIDS) Curriculum Map

Core Courses GCJA 501 Focus & Design

GCJA 531 History of Criminal Justice

GCJA 563 Research Methods

GCJA 580Concentration

area


area


area


area

GCJA 584 Concentration area and field

research

GCJA 585 Concentration area and field

research

GCJA 670 Portfolio


Program Outcomes


AP P P

AP P P R R

A

2.Critical Thinking

IA

P PA

P P P R RA

3. EngagingDiverse

PerspectivesIA

PA

P P P R RA

4.Broad Integrative Knowledge

IA

PA

P P P R RA

5.Application of CJA I

PPA

P P R RA

6. Knowledge sensitivity, and awareness of individual and

cultural diversity

IA

PA

P P P R RA


Master of Criminal Justice Administration Program (GCJA) Curriculum Map

Summary

“ By acting on the calls for reform in assessment, graduate education can truly lead the way in fulfilling its mission of excellence in teaching, research, and service to its students and to the broader community.” (Lydell, 2008)

References

Association of American Colleges and Universities (AAC&U). Value Rubrics. Retrieved fromhttp://www.aacu.org/value/rubrics/index.cfm

Anderson, L.W. (2002). Curricular alignment: A re-examination. Theory into Practice, 41, (4). Retrieved from Academic Search Premier.

Harden, R. M. (2001). AMEE Guide No 21: Curriculum mapping: a tool for transparent and authentic teaching and learning. Med Tech, 23, (2).

Lydell, L. (2008). Assessing outcomes in graduate education. On the Horizon, 16, (2). Retrieved from Proquest data base.

Lumina Foundation. (2011). The Degree Qualifications Profile. Retrieved from http://www.luminafoundation.org/publications/The_Degree_Qualifications_Profile.pdf

Miller, R., & Leskes, A. (2005). Levels of assessment: From the student to the institution. Washington, DC: AAC&U. Retrieved from http://www.aacu.org/pdf/LevelsOfAssessment.pdf

Questions/DiscussionQuestions/Discussion

For more information please contact:

Dr. William M. White, DeanOffice: (304) 929-1438; Email: [email protected] of Graduate Studies or

Indre Cuckler, M. A.Psychology Program SpecialistOffice: (304)-929-1401; Email: [email protected] of Graduate Studies

Thank you !

Documents

Course Embedded Assessment of Graduate Learning: An Innovative Approach Based on The Degree Qualifications Profile Presented by William M. White, Ed. D