2017-2018 CACREP COMPREHENSIVE EVALUATION REPORT

COUNSELING AND HUMAN DEVELOPMENT PROGRAM

2017-2018 CACREP COMPREHENSIVE EVALUATION REPORT

M.S. SCHOOL COUNSELING M.S. COMMUNITY MENTAL HEALTH COUNSELING Ph.D. COUNSELING AND COUNSELOR EDUCATION

Ed.D. COUNSELING

Table of Contents

COUNSELING PROGRAM MISSION ..................................................................................................... 2

COUNSELING PROGRAM OBJECTIVES .............................................................................................. 3

Core Program Objectives ................................................................................................................... 3

Clinical Mental Health Counseling ................................................................................................. 4

School Counseling ................................................................................................................................. 5

Doctoral Program .................................................................................................................................. 6

Administrative and Evaluative Processes .................................................................................. 6

INTRODUCTION TO THE YEARLY SYSTEMIC PROGRAM EVALUATION REPORT ........ 7

THE DATA SOURCES USED IN THIS REPORT ................................................................................ 8

STRUCTURE OF THE REPORT .............................................................................................................. 9

1. Annual Admissions and Across Program Demographic Data ....................................... 9

2. Individual Student Assessment of Key Performance Indicators .............................. 11

3. Annual Clinical Formative and Summative Assessments ............................................ 21

4. Student Annual Milestone Assessments .............................................................................. 22

5. Annual Student Reviews ............................................................................................................ 23

6. Adjunct and Affiliated Faculty Surveys ................................................................................ 24

7. Annual Employer Surveys ......................................................................................................... 25

8. Annual Site Supervisor Surveys .............................................................................................. 25

9. Annual Alumni Survey................................................................................................................. 27

10. Individual Class Instructor Evaluations ............................................................................ 29

11. Assessment of Clinical Experience and Clinical Site .................................................... 29

12. Assessment of Clinical Site Supervisor .............................................................................. 29

13. University Tape Supervisor Assessment .......................................................................... 30

14. Small Group Supervision Assessment ............................................................................... 31

15. Graduating Student Survey .................................................................................................... 31

IMPLEMENTATION OF ACTION ITEMS ......................................................................................... 33

Entry-Level Programs ...................................................................................................................... 33

Doctoral Programs ............................................................................................................................. 37

MOVING FORWARD ............................................................................................................................... 42

Appendices ................................................................................................................................................. 43

Table 1 Summary of CACREP Course Evaluations .......................................................... 44

Table 2 Summary of Student Feedback-Site and Site Supervisor Evaluation ..... 45

Table 3 Summary of Student Feedback-Small Group Supervision .......................... 46

Table 4 Summary of Student Feedback-University Tape Supervision ................... 47

Table 5 CACREP Rubrics, By Measure .................................................................................. 48

Table 6, Survey Results-Alumni, Master and Doctoral Students .............................. 52

Table 7, Survey Results-Master’s Site Supervisors ......................................................... 55

Table 8, Survey Results-Doctoral Site Supervisors ......................................................... 56

Table 9, Survey Results-Master’s Employers .................................................................... 57

Table 10, Survey Results-Doctoral Graduate Employers ............................................. 58

Table 11, Summary of Adjunct Survey ................................................................................. 59

Demographics Table 1: Admissions Data ........................................................................... 60

Demographics Table 2: Current Student Demographic Data ..................................... 61

Demographics Table 3: Graduating Student Demographic ......................................... 62

Demographics Table 4: Gender Across Program ............................................................. 63

Demographics Table 5: Race/Ethnicity Across Program ............................................. 64

Table of Contents Warner School 2017-2018 Comprehensive Evaluation Report 2

COUNSELING PROGRAM MISSION The mission of the Counseling Program at the Warner School of Education and Human Development is two-fold:

To educate counselors who will empower their clients and students to create more rewarding and meaningful lives and relationships; become more self-reflective, caring and compassionate persons; and contribute to reform of the systems in which their lives are imbedded. To prepare counselor educators, researchers, practitioners and leaders in the counseling profession who will educate the next generation of counselors to do all of the above; contribute new knowledge in the fields of counseling, counselor education and human development; and be leaders in reforming educational and social systems to promote social justice, healthy human development, and quality relationships.

This mission is based on an ecological perspective of professional counseling that acknowledges the importance of personal development, but also recognizes that individual lives are imbedded in a variety of personal and extended relationships and social systems. Within this perspective, we:

See human development, counseling and education in a historical, cultural and social context

Work to foster healthy development across the life course and across a diversity of capabilities

See the integration of biological, psychological and social intervention as an essential foundation for enduring and just human development and change.

Recognize the counselor’s role as an agent of institutional change and social justice as well as personal empowerment

Recognize the diversity of traditions, disciplines, practices and cultures that generate the wealth of counseling resources and welcome difference and innovation

Use the latest research to guide best practice and intervention strategies


UNIVERSITY OF ROCHESTER COUNSELING PROGRAM OBJECTIVES

Below you will find a detailed list of the Program Objectives that constitute what we, as a program, consider to be the most essential points of knowledge for our students. Because we consider these Program Objectives to be centrally important in guiding our programming, we have also made every effort to ensure that they are reflected in our assessment of students. Core Program Objectives**

1. Students understand how the historical roots of the profession are reflected

in the philosophical foundations of present day counseling practices. This

understanding extends to a working knowledge of how counseling’s

orientation to practice is distinguishable from other professions striving to

provide services that foster psychological health.

2. Students have a working knowledge of the ethical codes and associated

ethical decision-making processes espoused by the range of organizations

that regulate professional counseling practices. In addition, students are

aware of the various sets of competencies, promulgated by the American

Counseling Association and its divisions. Students understand the ways in

which knowledge of these competencies governs ethical practice within a

multicultural and pluralistic society.

3. Students are aware of their own cultural heritage, including attitudes,

beliefs, understandings and acculturative experiences, and how those

impact their views of others. They are aware of various dimensions of client

cultural diversity, including various dimensions of power and privilege, that

impact intervention success. In the context of their awareness of power and

privilege, students are familiar with advocacy processes related to

identifying and eliminating barriers, prejudices, and processes of oppression

and discrimination.

4. Students have gained understanding of a wide range of theories and models

of intervention strategy and have developed their own personal approach to

counseling, which will be informed by knowledge of theory and best

practices.

5. In devising intervention strategies, students have the knowledge and skill to

integrate systemic, interpersonal, intrapersonal, and biological approaches

to healing.

* * Core program objectives reflect knowledge and practice competencies across all master’s and doctoral programs.


6. Students demonstrate basic counseling skills and processes necessary to

establish rapport, assess problems, identify client goals, and begin to

implement and assess prevention and intervention strategies.

7. Students have knowledge of the theories and models of therapeutic group

work and an understanding of group process and development. Students

also have the ability to function as effective group leaders.

8. Students will have an understanding of the professional roles and functions

of counselors across specialty areas, and will have an understanding of

inter-agency and inter-organizational collaboration and consultation.

9. Students engage in self-reflective practices that sustain and build their

competence in the profession of counseling and support engagement in

continuous professional learning.

10. Students are able to identify emergent theories of career development and

counseling and apply theoretically-relevant strategies to facilitate client

career development and navigate career transitions.

11. Students have a knowledge of psychological, psychosocial, and socio-

structural theories of human development and are able to apply this

knowledge to aid in their understanding of common counseling issues (e.g.

addictions, anxiety, grief and loss).

12. Students recognize and demonstrate knowledge of basic concepts of

standardized and non-standardized testing, norm-referenced and criterion-

referenced assessments, and group and individual assessments. They will

also be able to identify and describe statistical concepts, including scales of

measurement, measures of central tendency, indices of variability, shapes

and types of distributions, and correlations.

13. Students understand the importance of membership in professional

counseling organizations, professional development, and advocacy for the

profession.

Clinical Mental Health Counseling 1. Students are proficient in the processes of current DSM diagnosis and are

able to accurately formulate a diagnosis of problems presented by clients;

distinguish between common forms of psychological suffering that are

generated by contextual challenges and mental disorder, and master the

vocabulary and knowledge necessary to discuss diagnosis with collaborating

professionals across the helping professions.

2. Students understand the principles, models and documentation formats of

biopsychosocial case conceptualizations and treatment planning, including

the bio-psychosocial-spiritual model of addiction.


3. Students have the necessary skills to apply principles, models and

documentation formats of biopsychosocial case conceptualizations and

treatment planning, including the bio-psychosocial-spiritual model of

addiction.

4. Developing counselors will demonstrate how to theoretically conceptualize,

treat, and assess addictive disorders from an ecological perspective

(including but not limited to family, systems of care, societal norms and

stigma, and legislative and ethical components) while recognizing the history

of addiction treatment, different models of recovery, and different counselor

roles across all levels of care.

5. Students have gained knowledge of etiological theories/models of addictions,

including research and information on environmental factors that affect both

normal and abnormal behavior.

6. Students can skillfully employ counseling and psychotherapy interventions,

including evidence-based protocols for phased treatment across individual,

dyadic, and group modalities.

School Counseling

1. Students have a vision of school counseling that fosters conditions ensuring

educational equity, access and academic success for all students.

2. Students have developed a perspective on school counseling that is based on

the National (ASCA) standards and New York State (NYSSCA) model for the

development of a comprehensive school counseling program K-12. Using

this perspective, students have the skills to design and implement

comprehensive school programs.

3. Students understand the characteristics, risk factors and warning signs of

students at risk for mental health and behavioral disorders commonly seen

in school settings.

4. Students understand and employ school counselor roles and responsibilities

in relation to school emergency management plans, crises, disasters and

trauma.

5. Students understand and utilize principles and counseling skills of a

comprehensive, developmental model to shape their practice, which consists

of service and advocacy for all students in the areas of evidence-based school

counseling and school counselor leadership.

6. Students are able to articulate a comprehensive, developmental strategy for

implementation in school counseling programs.


Doctoral Program 1. Students have an interdisciplinary understanding of the connection between

human psychological suffering and exposure to adversity across the life

course.

2. Students understand the basic principles of quantitative and qualitative

research designs.

3. Students have knowledge and skill to apply advanced research methods and

program evaluation to their research agenda.

4. Students have the ability to utilize an advanced understanding of counseling

theory to formulate integrative, holistic approaches that lead to

developmental, strengths-based counseling practices, and support a healthy

developmental trajectory.

5. Students have knowledge and demonstrated competency in models of

advocating for clients at the individual, system and policy levels.

6. Students have knowledge and demonstrated competency regarding

classroom and digital pedagogical practices, roles, responsibilities and ethics

attendant to the training and education of counselors.

7. Students demonstrate the writing and presentation skills necessary to

communicate scholarship with academics and practitioners in the field of

counseling.

8. Students have advanced understanding of practitioner roles in group

counseling.

9. Students demonstrate understanding, ethicality and capability as counseling

supervisors.

Administrative and Evaluative Processes 1. Input regarding student evaluation, as it pertains to both knowledge and

practice, reflects the perspective of core faculty, non-core faculty, students,

field supervisors, and clinical supervisors.

2. Evaluation of programs more broadly reflects the perspectives of core

faculty, non-core faculty, current students, former students, clinical

supervisors from cooperating agencies and clinics and field supervisors from

school settings.


INTRODUCTION TO THE YEARLY SYSTEMIC PROGRAM EVALUATION REPORT

The Counseling Program at the University of Rochester’s Margaret Warner Graduate School of Education and Human Development, in its striving to maintain excellence, is engaged in ongoing comprehensive program evaluation activities. These activities aim to do the following in regard to evaluating programmatic and curricular activities:

Capture the wisdom of the full range of program stakeholders including

those whom we have the honor of teaching as well as those to whom we owe a debt of gratitude for your generosity and commitment to joining us in training the next generation of counselors and counselor educators.

Collect data that ensures that our instruction and clinical experiences provide students with the tools that they need to development the full range of competencies required by the Council for the Accreditation of Counseling and Educationally Related Programs (CACREP).

Provide a sound platform for remediation of current practices and polices while building a vision for the future that is informed by the lived experiences of dedicated clinical professionals and the students that we serve.

Offer prospective students and the public a window into the operation of our department and the successes and challenges we have faced in training future counseling professionals.

In this spirit, this document is comprised of data that has been gathered from multiple sources over the course of approximately 12 months. It is our intention to keep the data in this report current through our ongoing program evaluation activities. Thus, this report should be considered a snapshot of our program structure and function, which we view as a dynamic organism that is constantly evolving in response to environmental changes and challenges. It is our hope that as we move toward the future, those of you who have generously given your time to provide the valuable feedback that has made this evaluation possible, will kindly continue to provide us with guidance. It is also our hope that you will use the faculty phone numbers and email addresses on our website to stay in communication with us on an on-going basis. For those of you who are new to the Warner School or just interested observers, we welcome you. Sincerely,

The Warner School Counseling Faculty


THE DATA SOURCES USED IN THIS REPORT The following is a general representation of the sources of data used for this report.

COMPREHENSIVE EVALUATION OVERVIEW

PLEASE NOTE….. As we share our data with you, we will use utmost precaution to protect the identity of any of our contributors. Likewise, we will not be reporting on any data that pertain to individual faculty or students—these are matters that are deliberated and resolved on an individual basis and in a manner that maintains confidentiality.

Aggregate Student Assessment Data

1. Admissions and Across Program Demographic Data

2. Individual Student Annual Assessment of Key Performance Indicators

3. Annual Clinical Formative and Summative Assessments

4. Student Annual Milestone Assessments

5. Annual Student Reviews

Student Satisfaction Assessments

10. Individual Class Instructor Evaluations

11. Assessment of Clinical Experience and Clinical Site Supervisor

12. Student Assessment of Site Supervisor

13. University Supervisor Assessment

15. Graduating

Student Survey

14. Small Group Supervision Assessment

Stakeholder Assessments

7. Annual Employer Surveys

8. Annual Site Supervisor Surveys

6. Adjunct Surveys 9. Annual

Alumni Survey


STRUCTURE OF THE REPORT

This report is divided into three sections. In the first section, we describe the assessment tools and activities, as outlined in the illustration above, which we use to collect our data followed by a report of the key findings. In the second section we outline the program changes that we have already implemented in response to the feedback and other assessment data that we have gathered. In the third section we provide an accounting of those areas in need of attention that have been addressed or will become the focus of our remediation efforts in the coming months. Finally, in the last section, we will highlight some of the basic principles that will guide us as we move forward. 1. Annual Admissions and Across Program Demographic Data The annual admissions data includes data gleaned from our master’s and doctoral level admissions form over four admissions cycles (July, October, February, and April). The admissions forms; which represent an integration of interview data, transcripts, professional references, a personal statement, and a writing sample; are completed by core faculty members who are involved in the interview process. The data that is collected on these forms reflects our efforts to maintain the highest standards in relation to CACREP-informed parameters regarding academic aptitude, adequacy of preparation for the program, disposition, and respect for cultural differences. These data are not published, but are used idiographically to compile a retrospective profile of student characteristics that give us insight into admissions profiles associated with both student success and challenge. We discovered, for example, that students most likely to struggle in the program were, not surprisingly, given among the lowest ratings relative to other admits. One aspect of admissions data that is particularly valuable in our comprehensive program evaluation is the demographic information that we are able to glean to assess the success we are having in attracting, matriculating and retaining a diverse student body. Comparing demographic data at multiple points, e.g., admissions acceptance, current student body, and graduation; gives us an understanding of our patterns in relation to diversity. The attention to demographics is a part of our conversion to 2016 CACREP Standards, and as will be described in the section on future initiatives, will, be expanded to capture more nuanced definitions of diversity. We hope, example, to expand gender and to get feedback from students on what they consider to be dimensions of diversity that are important to their social and cultural identities. FINDINGS FROM DEMOGRAPHIC DATA As alluded to above, much of the data coming from admissions materials are suitable primarily for internal use. Primarily, as we become better acquainted with students, if there are issues that arise (e.g., academic, dispositional), we are able to refer back to the admissions data and ascertain whether our admissions process


needs to be amended. One aspect of the admissions process that we do feel it is important to consider involves demographic data, which gives us a sense of the degree of diversity we are attracting in the application process as well as the diversity among those who eventually matriculate. The data reported here will provide a baseline for a more longitudinal comparative analysis. Thus, much of what is described here is a snapshot of one small moment in time. The following tables are provided as means to understand descriptively our current across program race/ethnicity and gender profile. For the sake of this report, “across program” data collection is defined as capturing data from 1) students admitted to our programs, 2) those current students who matriculated into a program, and 3) those who graduated. Table D1: Admissions Demographic Data 2017-2018 reveals the following:

The overall male to female gender ratio of our admits was 20% male to 80% female.

The split in doctoral admits is reflective of the overall ratio at 24% male and 76% female (i.e., 4M, 13F).

Master’s students are split at 20% male and 80% female (i.e., 14M, 57F). Our admits for 2017-18 identified as 61% White, 13% Black, 10%

International, 9% Hispanic, 3% Multiracial, and 3% Asian. During these admissions cycles no Black, Hispanic or Asian males were

admitted to the doctoral programs. Table D2: Current Student Demographics 2017-2018 reveals the following:

The overall male to female gender ratio of our current students is 17% male to 83% female, with a slightly lower male representation than is reflected in the admit pool.

The gender split in current doctoral students is reflective of the overall ratio at 16% male and 84% female (i.e., 7M, 39F).

Master’s student admits are split at 19% male and 81% female (i.e., 11M, 48F).

Our admits for 2017-18 identified as 59% White, 13% Black, 11% International, 9% Hispanic, 4% Multiracial, and 3% Asian.

In general, there was considerably more racial diversity among current doctoral student than was observed among admits, including; 6% Black, 2% Hispanic, Asian, and International; and 1% Multiracial.


Table D3: Graduating Student Demographic 2017-2018 reveals the following: The overall male to female gender ratio of our graduates was 19% male to

81% female. The split in doctoral graduates was 50% male versus 50% female (i.e., 1M,

1F), however, the number of graduates, N=2, makes any interpretation of these data highly suspect.

Master’s students were split at 17% male and 83% female (i.e., 4M, 20F). Our graduates for 2017-18 identified as 58% White, 12% Black, 8%

International, 4% Hispanic, 12% Multiracial, and 8% Asian.

Table D4: Gender Across Program 2017-2018 reveals the following: The gender distribution across program, i.e., figures for2017-2018 admits,

current students, and graduates, was consistently in the 20M:80F range Table D5: Race/Ethnicity Across Program 2017-2018 reveals the following:

Race/ethnicity distribution across program, i.e., figures for 2017-2018 admits, current students, and graduates reveals consistency among White students and Black students, while the patterns among the remaining race/ethnicity categories were difficult to interpret due to the low numbers.

Overall, as stated above, the demographic data presented here are being used as a foundation for comparative data analysis in future years. 2. Individual Student Assessment of Key Performance Indicators As many individuals reading this report are aware, CACREP has a comprehensive set of standards that all programs seeking accreditation must implement. These standards, in addition to requiring that programs meet certain expectations concerning how they are structured internally and supported by the larger institutions in which they are embedded, also have highly specified standards that relate to the curriculum that shapes the coursework at the master’s and doctoral levels of our training program. The assessments in this section relate directly to the standards governing curriculum. The data related to this assessment point is collected throughout the academic year as part of individual class assessments. The particular nature of data that is collected is predicated on our collection of key performance indicators (KPI’s) that were derived from our Program Objectives. The KPI’s, which were derived from the Program Objectives, can be understood as key points that are targeted for assessment as students progress through their programs. These KPI’s are divided into 5 categories that correspond to data collection mandated by the 2016 CACREP Standards (the 2016 CACREP Standards can be accessed at www.cacrep.org): Entry-level Counseling Core, Clinical Mental Health Counseling, School Counseling, Doctoral Counselor Education and Supervision, and Administrative. For each of the


KPI’s listed, a minimum of 2 independent and contrasting measures of the KPI are collected. The KPI’s are as follows:

KEY PERFORMANCE INDICATORS (KPI’S) COUNSELING CORE 1) Professional Counseling Orientation KPI CC1: The student understands legal and ethical considerations specific to clinical

mental health counseling 2) Social and Cultural Diversity KPI CC2: Students understand how their own social and cultural diversity impacts

their views of other and self. 3) Human Growth and Development KPI CC3a: Students have gained a knowledge of human development theories. KPI CC3b: Students have learned to apply theories of human development to better

understand counseling-related issues. 4) Career Development KPI CC4a: Students have knowledge of career development theories, particularly as

the theories pertain to their own personal development. KPI CC4b: Students will exhibit the ability to practice career counseling 5) Counseling and Helping Relationships KPI CC5: Students have developed their own personal approach to counseling, which

will be informed by knowledge of theory and best practices. 6) Group Counseling and Group Work KPI CC6a: Students understand the dynamics associated with group process and

development KPI CC6b: Students understand the characteristics and functions of effective group

leaders 7) Assessment and Testing KPI CC7: Students understand the responsibilities and obligations of test

administrators

8) Research and Program Evaluation KPI CC8: Students understand the basic principles needed to successfully utilize

assessment data in the counseling process.


CLINICAL MENTAL HEALTH COUNSELING KPI CMHC 1: The student demonstrates an understanding of how to conceptualize a

case through a bio-psychosocial lens. KPI CMHC 2: The student demonstrates the ability to construct a counseling

intervention plan that is appropriate for DSM disorders that have been identified in the assessment process..

KPI CMHC 3: Students have the applied theoretical knowledge to inform their

treatment of clients in an addictions setting. KPI CMHC 4: Student demonstrates the procedures for assessing and managing

suicide risk. KPI CMHC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and

knowledge that suggest the ability to competently and ethically provide mental health counseling services to clients, commensurate with current level of training

SCHOOL COUNSELING KPI SC 1: The student understands legal and ethical considerations regarding the

work of the school counselor KPI SC 2: Students understand the counselor’s role as leaders and systems change

agents in P-12 schools. KPI SC 3: The student has knowledge of current methods of using data to inform

decision-making & accountability. KPI SC 4: Student demonstrates the procedures for assessing and managing suicide

risk. KPI SC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and

knowledge that suggest the ability to competently and ethically provide school counseling services to clients, commensurate with current level of training.

DOCTORAL COUNSELOR EDUCATION AND SUPERVISION Counseling KPI DCES 1: Students are able to critically reflect upon approaches to counseling

including strengths and weaknesses, theoretical bases, the research base for existing counseling theory, and the range of methods for evaluating efficacy


Supervision KPI DCES 2: Students have gained understanding of clinical supervision theory and

ethics, and demonstrated effective supervision practice. Teaching KPI DCES 3: The student demonstrates proficiency in course design, delivery and

evaluation methods appropriate to course objectives. Research and Scholarship KPI DCES 4: The student demonstrates the skills needed in professional writing for

journal and newsletter publication Leadership and Advocacy KPI DCES 5: The student understands models and applications of advocacy and

consultation. SCORING KPI DATA The KPI-related data is recorded by individual course instructors at the end of each semester. Rubrics exist for each of the minimum of two measures for each KPI, with each rubric detailing how a score of 1-4 is derived. The 1-4 scoring scale is used across all of the KPI measures and is interpreted as follows:

1= Insufficient 2=Emergent 3=Meets Expectations 4= Exceeds Expectations

As always, professors who rate a student below a 3 on any given standard are expected to provide feedback to the student on how the student can improve in this core area. Concerns around how individual students may be failing CACREP core standards are also discussed at regular faculty meetings and in depth at our annual student review. Overall trends in ratings, especially standards where scores are consistently rated below a 3, are also discussed at our faculty meeting in order to discern proper interventions for increasing student proficiencies in those areas.

FINDINGS FROM KPI MEASURES The following are the outcomes, based on a minimum of 2 contrasting measures, at two distinct points in time, of all of the KPI’s listed above. The data presented includes scores generated for Summer 2017, Fall 2017, and Spring 2018. Click here for Table 5, KPI Outcomes, School, Mental Health, Doctoral Some General Thoughts About Key Performance Indicator Measures


In interpreting the results of our KPI measures, there are several considerations that should be applied. First, the measures that have been devised reflect a diversity of knowledge and skill-based assessments. While diversity in measures yields a more comprehensive understanding of student progress, it is notoriously difficult to ensure score reliability across measures. Thus, it is with caution that we make claims concerning overall score improvement. Also adding some uncertainty of the consistency of rating across measures is the problem of interrater reliability. The interrater conundrum is particularly problematic because the large number of raters relative to the population students being assessed. In devising this KPI assessment system we attempted as much as possible to create measures that be completed by Core Counseling Faculty and closely supervised counseling doctoral students as opposed to faculty outside of our program. With those caveats in mind, we report on the outcomes of our KPI measures. CORE COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES The results of the Core Counseling KPI measures yielded the following insights:

1) Professional Counseling Orientation KPI CC1: The student understands legal and ethical considerations specific to clinical mental health counseling

In general, students readily assimilated an understanding of professional counseling orientation principles. There is some question as to whether School Counseling students were slower to grasp some of the professional counseling orientation material, as a smaller percentage of the School Counseling students exceeded expectations relative to the Mental Health Counseling students. This possibility would comport with feedback that we have gotten through other assessments we have done where School Counseling students have repeatedly underscored that much of the material in Core Counseling classes uses exemplars that are more applicable to mental health students, hence leaving School Counseling students with the extra burden of not getting enough exposure to real-world applications of principles.

2) KPI CC2: Students understand how their own social and cultural diversity impacts their views of other and self.

The two measures in Social and Cultural Diversity had considerable fidelity with a large number of students achieving top ratings in both measures.

3) Human Growth and Development KPI CC3a: Students have gained a knowledge of human development theories. KPI CC3b: Students have learned to apply theories of human development to better understand counseling-related issues.

Outcomes for Human Growth and Development indicate that this is an area of study where students enter the program with less preparedness to negotiate the curriculum in this course. That being said, after transitioning in


time from Measure 1 to Measure 2, all students scored within a range that reflective satisfactory assimilation of material.

4) Career Development KPI CC4a: Students have knowledge of career development theories, particularly as the

theories pertain to their own personal development.

KPI CC4b: Students will exhibit the ability to practice career counseling

Students consistently scored within a satisfactory range in all measures related to Career Counseling and Development.

5) Counseling and Helping Relationships KPI CC5: Students have developed their own personal approach to counseling, which will be informed by knowledge of theory and best practices.

The KPI measures in Counseling and Helping Relationships showed the greatest improvement from Measure 1 to Measure 2, with the outcomes for Measure 1 being lower than is typical for other measures. This is not entirely surprising as the knowledge and skills in this category often requires negotiating personal challenges and adjusting to a new professional culture. That being said, cross-referencing these outcomes with student class evaluations, it is clear that students find this challenge to be personally rewarding.

6) Group Counseling and Group Work KPI CC6a: Students understand the dynamics associated with group process and development KPI CC6b: Students understand the characteristics and functions of effective group leaders

Much like what we observed for Counseling and Helping Relationship, the outcomes for Group Counseling and Group Work, relative to most other categories showed more improvement from Measure 1 to Measure 2 than is typical, with Measure 1 being lower than is typical of most measures. The class in which these assessments are performed, Counseling and Facilitating Small Groups, has a major practice component that would be quite unfamiliar to many students, thus creating a lower starting point with a steeper learning curve.

7) Assessment and Testing KPI CC7: Students understand the responsibilities and obligations of test administrators.

Assessment and Testing has some of our highest scores. In cross-referencing with other assessments (e.g., Graduating Student Assessment, Teaching Evaluations), students have repeatedly expressed their appreciation for the


instructor in this class, in spite of the fact that the content is often not embraced by students. The design of this class is one that has changed in response to student feedback in the last several years in which projects in the class have become tied to students’ personal interests and much effort has gone into ensuring that students understand the relevance of this course to clinical practice.

8) Research and Program Evaluation

KPI CC8: Students understand the basic principles needed to successfully utilize assessment data in the counseling process.

Research and Program Evaluation has had a pattern that is similar to Assessment and Testing. In spite of the fact that students struggle with the relevance of some of the material they are required to negotiate, the instructor responsible for Measures 1 and 2 has gone to great lengths to bring relevance to the material and has succeeded in creating an atmosphere in which students clearly assimilate the material.

CLINICAL MENTAL HEALTH COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES 1) KPI CMHC 1: The student demonstrates an understanding of how to conceptualize a case through a bio-psychosocial lens.

Within the Mental Health Counseling program, ensuring that students have a biopsychosocial vision of mental health challenges is among our top priorities. We have found over time that this is often a difficult concept for students to grasp and to subsequently implement in their case conceptualization. We have thus gone to great lengths to include biopsychosocial case conceptualization into many aspects of the curriculum, and in particular, have given it a central role, both in terms of knowledge and skill, in our course, EDE 449 Pre-Practicum. Prior to our recent bolstering of biopsychosocial implementation requirements in Pre-Practicum, considerable attention was given to implementation in our two semester Problem Identification and Intervention in Counseling sequence, and it was our hope that the increased attention to the biopsychosocial model in Pre-Practicum would work synergistically with the Problem Identification and Intervention in Counseling course objectives. The outcomes for the two measures of KPI CMHC 1 indicate that students were quick to grasp the biopsychosocial framework in two separate settings. Cross referencing surveys administered to site supervisors at the end of program in, EDF 458 Internship in Mental Health Counseling II class, and from Mental Health Counseling employers, we hope to see the grasp of the biopsychosocial models reflected in the feedback from these data sources.


2) KPI CMHC 2: The student demonstrates the ability to construct a counseling intervention plan that is appropriate for DSM disorders that have been identified in the assessment process.

The outcomes for this KPI were spread across two semesters. Interestingly, more students scored in the “exceeds expectations” range in the first measure when compared to outcomes for the second measure. This unexpected outcome could be attributed to either an issue with interrater reliability or with the transferability of the skills acquired in the Pre-Practicum class into the Problem Identification and Intervention II class. Also, the emphasis on DSM knowledge is greater in the latter course and testing in this course undoubtedly has as a standard a greater level of diagnosis and treatment planning skills. We will consider implementing a third measure for this skill in Internship class to ensure that the skills are ultimately transferable to the clinical setting.

KPI CMHC 3: Students have the applied theoretical knowledge to inform their treatment of clients in an addictions setting.

The outcomes for these two KPI measures indicated that students successfully transitioned from 85% exceeding expectations to 100% exceeding expectations. The Mental Health Program recently had a complete revamping of our Addictions course in preparation for our application to become providers of a curriculum leading to a New York State level credential in addictions counseling (Credentialed Alcoholism and Substance Abuse Counselor Trainee, CASAC-T). The increased attention to addictions work may have contributed to the high level of performance on addictions-related assessments.

KPI CMHC 4: Student demonstrates the procedures for assessing and managing suicide risk.

The outcomes for KPI CMHC 4 raises some level of concern regarding assessing and managing suicide risk. A downward trend in Measure 2 from 100% exceeding standards to 73% exceeding standards, including 2 students (13%) scoring in the emergent range, raises questions about the reliability of one of the two measures. Measure 2 may have required a more nuanced integration of knowledge and skill than Measure 1, thus accounting for the drop in score. Going forward, we will consider adding a Measure 3, embedded in Internship, to ensure that all students are meeting or exceeding expectations.

KPI CMHC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and knowledge that suggest the ability to competently and ethically provide mental health counseling services to clients, commensurate with current level of training.

The outcomes on the three measures of this KPI showed steady improvement throughout the program. Measure 1 showed that 32% of students were exceeding expectations, while Measures 2 and 3 showed 63% and 79% of


students exceeding expectations respectively. It should be noted that Measure 3 was a skill-based measurement occurring at the end of the two-semester Internship experience, thus indicating that all students were at or above expectations upon graduation.

SCHOOL COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES

KPI SC 1: The student understands legal and ethical considerations regarding the work of the school counselor.

The outcomes of KPI SC1 indicate that by the time School Counseling students complete Internship 66% exceed expectations regarding legal and ethical considerations in the work of school counselors and that 100% meet or exceed expectations at that time. This Measure 2 outcome is up from 50% exceeding expectations for Measure 1, which occurs in the Introduction to School Counseling class at the beginning the School Counseling course sequence.

KPI SC 2: Students understand the counselor’s role as leaders and systems change agents in P-12 schools.

While the percentage of students exceeding expectations remained steady between Measures 1 and 2 (63% and 67% respectively), one student dropped to emergent status in Measure 2 (occurring later in the program) raising the notion that leadership and systems change is perhaps not being reinforced in Core Counseling Courses. This will be addressed as we go forward.

KPI SC 3: The student has knowledge of current methods of using data to inform decision-making & accountability.

For both Measure 1 and Measure 2 of this KPI, all students met or exceeded expectations. A small jump in the number of students exceeding expectations was observed in Measure 2, which occurred at the end of the second semester of Internship. It should be noted that Measure 1 was largely knowledge, while Measure 2 was skill-based and reflects impressions of this skill in the Internship site.

KPI SC 4: Student demonstrates the procedures for assessing and managing suicide risk.

Similar to our outcomes related to Mental Health Counseling, School Counseling students scored higher on Measure 1 when compared with Measure 2. While Measure 1 showed 100% of students exceeding expectations, Measure 2 showed 77% exceeding expectations and 32% meeting expectations. As we speculated above in the Mental Health Counseling section, Measure 2 may have required a more nuanced integration of knowledge and skill than Measure 1, thus accounting for the drop in score. As with Mental Health, going forward, we will consider adding


a Measure 3, embedded in Internship, to ensure that all students are meeting or exceeding expectations.

KPI SC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and knowledge that suggest the ability to competently and ethically provide school counseling services to clients, commensurate with current level of training.

This KPI was measured three times over the course of the School Counseling curriculum, including during both semesters of Internship. While students meet or exceeded expectations across all three measures, a steady shift towards exceeded expectations was observed over time.

DOCTORAL KEY PERFORMANCE INDICATOR OUTCOMES Counseling KPI DCES 1: Students are able to critically reflect upon approaches to counseling including strengths and weaknesses, theoretical bases, the research base for existing counseling theory, and the range of methods for evaluating efficacy.

Student outcomes, although strong for Measure 1 (67% exceeding expectations and 33% meeting expectations), were stronger for Measure 2 (100% exceeding expectations). As the curriculum content upon which these measures requires good analytical skills and interpretation of difficult literature, it follows that increased exposure over time resulted in an uptick in the outcomes for the latter measurement.

Supervision KPI DCES 2: Students have gained understanding of clinical supervision theory and ethics, and demonstrated effective supervision practice.

The transition from Measure 1 to Measure 2 showed no change in score. In both cases, 57% of students met expectations while 43% exceeded expectations. While these outcomes indicate that our students are gaining the knowledge and skill required for clinical supervision practice, they are a bit lower than we observe in most courses. It is likely that this difference is due to reliability issues related to either the measure or the instructor generating the assessment score.

Teaching KPI DCES 3: The student demonstrates proficiency in course design, delivery and evaluation methods appropriate to course objectives.

Due to our course sequencing and the fact that our doctoral course, EDU 552, Counselor Education contains both of the measures for this KPI, we will not have data available until the end of Fall 2018. Retrospectively, we have observed in past years that this is a very strong part of our curriculum; hence


we expect that outcomes for this KPI will be in the meets expectation to exceeds expectation range.

Research and Scholarship KPI DCES 4: The student demonstrates the skills needed in professional writing for journal and newsletter publication.

The outcomes for KPI DCES 4 show a decline from Measure 1 to Measure 2. The assignment tied to Measure 1 asks students to write, in a highly scaffolded context, a literature review related to a topic of their choosing. The outcomes of Measure 1 put all of the students meeting or exceeding expectations. Contrastingly, Measure 2, which is tied to the literature review portion of the student’s comprehensive examination performance, showed some backsliding in the case of one student, to an emergent status. While the backsliding is not dramatic, it does suggest that student’s need more guided literature review writing. This has been the topic of many of our Core Counseling Faculty meetings, but a satisfactory solution has not yet been reached and will continue to be the focus of our faculty discussions.

Leadership and Advocacy KPI DCES 5: The student understands models and applications of advocacy and consultation.

The outcomes of the Leadership and Advocacy measures show an increase in proficiency over time. In the initial measure, 1 of 6 students was rated emergent while 5 out of 6 were meeting or exceeding expectations. In Measure 2, all students were at or above expectations, with 50% meeting expectations and 50% exceeding expectations. This Measure 2 does contrast some with other Measure 2’s in our assessment system in that it is more typical that a larger percentage of students would be exceeding expectations. This could be attributable to the notion that as a faculty we do not put enough emphasis on Leadership and Advocacy issues. This will become a topic of conversation for our Core Counseling Faculty meetings.

3. Annual Clinical Formative and Summative Assessments A large bank of formative and summative assessments are part of the annual data collection process related to entry-level clinical requirements, i.e., practicum and internship experiences. Data is collected for both master’s and doctoral-level students. The data from these assessments supplements KPI-related data and is used to guide individual remediation needs of students in their clinical practicum and internship settings. Occasionally, the data gleaned from these assessments reveal patterns that allow us to consider more general needs for program modification. These data are not presented in aggregate form as the conditions around the evaluations are also unique to each student experience and do not lend themselves to cross-student reliability.


When data emerge, through faculty discussions related to these assessments, findings are presented in this report. This year, no specific finding emerged among these assessments that are not reflected in data reported elsewhere in this document. 4. Student Annual Milestone Assessments ENTRY-LEVEL Students in our master’s programs have one milestone assessment, which is an evaluation of our capstone process. These forms provide a medium for assessing the final task in the entry-level program. In this capacity, these forms document the students’ technical and conceptual writing skills and their ability to connect their topic of study to the larger counseling literature. Due to the varying nature of capstone projects, these assessments are best understood in relation to individual student performance as opposed to aggregate consideration. FINDINGS FROM ENTRY-LEVEL MILESTONE ASSESSMENTS Although we have experienced considerable success in addressing some of the challenges of the master’s level capstone project, some challenges remain. Based on data from the milestone assessment and from discussion that has been generated in relation to this assessment, it is clear that some students are not developmentally ready to successfully execute an original, positioned literature review (one of several options for completing the capstone requirement). Attempts to scaffold this experience have proved helpful in recent years, but the capstone remains a focus of many of our discussions and object of feedback from students. We will revisit this feedback and possible action steps later in this report. DOCTORAL The doctoral student annual milestone assessments include four assessments: 1) a portfolio assessment is collected concerning the student’s development in relation to writing mechanics, concept development, argumentation, and progress being made toward developing their individual research trajectory; 2) comprehensive exam assessment to determine the student’s readiness to enter the dissertation phase of their program. 3) a proposal assessment to measure the student’s success in forming a clearly articulated research question, identifying a suitable method, and generating a literature review that exposes a suitable gap in the literature for dissertation study and 4) a dissertation assessment to record the quality of the student’s writing, the success of their analytic plan, and their ability to clearly articulate a contribution to the existing literature in the field. FINDINGS FROM DOCTORAL MILESTONE ASSESSMENTS In any given year, the outcome of any one of the doctoral milestones, involves a very small number of doctoral students making presentation of results in this document traceable to individual students. For this reason, we refrain from disseminating


results numeric data from any given year. Nonetheless, the outcomes of these measures are an important source of data for faculty. We report more generally that the outcomes from our doctoral milestone assessments, based both on the measures designed to assess the above competencies, and the discussions that have occurred around these measures, suggest that as a program, the steps that we have taken to address issues related to writing mechanics, concept development, argumentation, developing a literature review and executing an analytic plan, while improving steadily, still require more systematic implementation of interventions aimed at bolstering these skills. We have, in the past, taken numerous steps toward improving the outcomes of these competencies and have met some success in doing so. However, it is clear that these interventions still need to be retooled and applied in a more systematic way. Further discussion of these outcomes, particularly in relationship to our implementation of remediation efforts, and further efforts going forward, are discussed in the latter sections of this report. 5. Annual Student Reviews Data is formally collected at the end of every spring semester and is used to assess the progress of all master’s-level and doctoral-level students. The data collected attempts to give an overview of academic progress, clinical skill development, and emergent dispositional issues. FINDINGS FROM ANNUAL STUDENT REVIEWS While we rely heavily on our KPI measures to create a detailed assessment of students’ curricular competency, the annual student reviews, as mentioned above, look at general academic trends, clinical performance, disposition and, particularly in the case of doctoral students, success in building an identity as a scholar in the field of counseling. These data are primarily used to craft individual remediation plans for students but are also used by the counseling faculty, in our discussion of students, to consider patterns that sometimes emerge from these assessments that are relevant to program modification. ENTRY LEVEL At the entry-level, we have not had major collective issues emerge from this assessment. That being said, this assessment has played a major role in bringing individual student dispositional issues to the fore, as well as clinical difficulties, and general patterns relating to academic struggle. The discussions that emerge from this assessment process have been the basis for creating remediation plans, and, in more difficult cases, assessing the advisability of having a student continue in the program. DOCTORAL LEVEL In contrast, at the doctoral level, we have had emergent issues at the group level. The most pressing issue to arise, which corroborates some of our findings from our KPI assessments, relates to students’ professional writing success, both in relation


to dissertation writing and publication in professional venues. While many of our students are performing quite well in this arena (our newly submitted self-study documents student participation in publishing opportunities), we have grappled with mechanisms for fostering development of doctoral students who have not had great success in this arena. Some of the steps that we have taken include increasing scaffolding around required class literature reviews and intensifying evaluation efforts related to our milestone assessments. This issue will be considered in greater detail below. More generally, these annual doctoral student reviews form the basis of a systematic, formal feedback that often includes detailed requirements related to milestone completion or attention to incompletes.

Stakeholder Assessments 6. Adjunct and Affiliated Faculty Surveys Adjunct faculty are surveyed regularly to ensure that they are feeling personally supported in their faculty role, integrated into the life of the program, and are receiving adequate resources relative to their job expectations. Data is collected biannually. OUTCOMES FOR ADJUNCT AND AFFILIATED FACULTY In our continuous systematic program evaluation of the counseling program strives to maintain professional and supportive contact with our adjuncts (local clinical professionals with particular expertise) and affiliated faculty (advanced doctoral students whose development has been fostered to a level beyond Teaching Assistants/University Clinical Supervisors). We take both formal and informal occasions to garner their input and opinions concerning the ways we can support their teaching and professional growth. Due to the busy lives of our adjunct and affiliated faculty, we attempt to gather their input through on-line surveys. We currently conduct biannual evaluation of our adjunct faculty. Our last survey was in 2015-16, and showed that overall the outlook and participation of the Adjunct Faculty in the Counseling and Human Development had improved since the last time we surveyed them in 2013; however, our 2017-18 survey (see table 11) still suggests room for improvement. Adjuncts, on average, scored modestly higher than equivocal in regard to their overall job experience in their teaching roles (3.67 out of 5.0). In the context of this leaning toward job satisfaction, however, there are several areas that are targeted for improvement. Most notable among these were the average score of 2.33 in response to the question: How would you rate the feeling of inclusion into the life and work of the department (were you given the opportunity to take an active part in the department’s decision-making process)? and the average score of 2.83 in response to the question: How would you rate the support you received from the department in preparing you to teach (examples of syllabus, provide you with copy of textbook, etc.)? Both of


these issues are addressed in the part of this report that details improvement plans that are in progress. Plans are also in place to address bolster supports around technology and faculty support more generally. 7. Annual Employer Surveys Each year, we send an electronic survey to individuals from whom our graduates have secured employment. These data give us a sense of how our graduates are functioning as counseling professionals, and more specifically, the degree to which employers perceive that former students have mastered various competencies valued by our program and reflected in CACREP Standards. These surveys were administered to employers identified by alumni and was disseminated during the Spring 2018 semester. All items in the assessment were ranked on a scale of 1 through 4, where 1= Does not meet standards, 2= Emergent, 3= Meets Expectations, and 4= Exceeds expectations. FINDINGS FROM EMPLOYER SURVEYS A total of 2 employers completed surveys about the quality of our alumni as professional counselors (this was down from 5 respondents the previous year). On average, employers rated our MS graduates 2.91 out 4 (see table 9), which was also down slightly from the previous year when the average score was 3.15 out of 4. As was the case with our 2016-2017 survey, we received no responses for our doctoral student employer survey for the 2017-2018. Of greatest concern at the master’s level, is the students’ ability to integrate biological, psychological and social interventions in their work with clients (average score 2.50); and the second relates to students’ “commitment to serving as an agent of organizational change (social justice) or client empowerment”, which received a score of 2.5. Of note was the score of 3.0 in relation to master’s level employer’s ratings of graduates’ use of research, ethics and critical thinking to guide best practice, and graduate’s ability to generate new knowledge for the profession, which was up from the previous survey. While some of these areas for concern have already been addressed (see the section of this document entitled “Implementation of Action Items”), others will become the subject of faculty discussion early this fall. Overall, even with the areas of concern, the findings from site supervisors and employers suggest our students are well prepared for their internships and professional counseling positions. 8. Annual Site Supervisor Surveys Data that will be collected Each year, we send an electronic annual Site Supervisor Survey to individuals serving in the field as our site supervisors. These data give us a sense of how our students are functioning as counselor trainees in the field, and more specifically, the degree to which employers perceive that our students are demonstrating a


developmentally appropriate level of various competencies valued by our program and reflected in CACREP Standards. FINDINGS OF ANNUAL SITE SUPERVISOR SURVEYS The data presented here come from two surveys, one sent to master’s-level site supervisors (see table 7) and the other sent to doctoral site supervisors (see table 8). These surveys were administered to site supervisors engaged in supervision during the Fall 2017 and Spring 2018 semesters. All items in the assessment were ranked on a scale of 1 through 4, where 1= Does not meet standards, 2= Emergent, 3= Meets Expectations, and 4= Exceeds expectations. Entry Level We received 24 completed surveys from site supervisors ranking the preparation and performance of our master’s student interns. Site supervisors, on average, rated our MS students as 3.2 out of 4, which is precisely where they ranked our students the previous year. The outcome of greatest concern was a response to the question assessing students’ “use of research, ethics and critical thinking to guide best practice”, where students on average scored 2.96. Of particular concern here is that these data do not comport with the findings from our KPI measures related to best practice, where problem areas did not emerge. Strengths that emerged from the survey were in the extent to which our students “act with sensitivity to diversity and social justice dimensions of counseling in their work with clients”, and the extent to which they are “caring, compassionate and self-reflective people”, where in both cases students scored an average of 3.42 out of 4.0. Other areas where our students averaged 3.25 or greater were in their competency around use of technology, their openness to innovations in counseling that address diversity, and their commitment to serving as agents of change and client empowerment. Doctoral Students Only one individual responded to the doctoral site supervisor survey. It should be noted that the number of doctoral students who participated in practicum or internship during the survey period was exceptionally low (N=3), making survey work challenging. Nonetheless, the average of all of the ratings from this supervisor was 3.21, with no ratings below a 3.0. A score of 4.0 was earned for the extent to which our students “act with sensitivity to diversity and social justice dimensions of counseling in their work with clients”, the extent to which they are “caring, compassionate and self-reflective people”, and the extent to which they are “competent with the use of technology in practice”. No areas of great concern emerged from this survey, and one area, which had raised concern in the survey from the previous year, was not rated as problematic; namely, the ability of our students to “function as leaders and advocates within the organization and profession”. It is clear that some other means of assessment may have more utility if the number of doctoral students involved in clinical practice during a given assessment period is excessively low.


9. Annual Alumni Survey Data that will be collected Our Alumni Survey is a lengthy instrument that includes the following multiple domains: 1) general satisfaction with the Warner School in relation to faculty support, curriculum, and negotiating administrative tasks (e.g., registration, bursar); 2) assistance regarding employment and licensure, and 3) perceived proficiency regarding skills reflected in our program objectives and issues related to disposition, and 4) demographic data. FINDINGS FROM THE ALUMNI SURVEY Each year we send a survey to our alumni to query their employment status and their perspectives about how well the program prepared them for the work of a professional counselor. Much of these data about licensure and employment status are included in our yearly vital statistics survey, which is posted annually in early in the Fall semester. Therefore, we refer readers to our summary report of our vital statistics for information about job placements and licensure pass rates (click here

for a link to the most recent summary data from our vital statistics survey). In this section, we include data from alumni that focuses more on their experiences in the program and their perceptions of how well the program prepared them for professional counseling Click here to see table 6 for summary data of our alumni survey for 2017-2018. In this 2017-18 survey, 9 student alumni responded to our survey. Out of the 9 respondents, 5 (56%) said they were either “satisfied” or “very satisfied” with their Warner experiences. This is down from 71.4% in the 2017-18 survey. Likewise, 4 of 9 alumni (44.4%) in the 2017-18 survey said that if they had to do it over again, they would either “definitely” or “probably” attend Warner, also reflecting a drop from the score of 71.5% of the respondents in the 2016-2-17 survey. This reflects a two-year decline in that the responses to these two latter questions achieved scores of 100% in the combined very satisfied/satisfied range for the former question and the combined definitely/probably range for the latter question. While it is clear that the small number of respondents make the data very sensitive to divergent responses from one or two participants, these results necessitate further exploration. While faculty have considered possible reasons for the observed pattern, a committee assigned to explore this problem will intensify previous efforts to uncover the underlying factors that are contributing to this problem, and to suggest remediation strategies going forward. The Alumni Survey, Table 6, also asks a number of questions related to respondents’ experiences and the degree to which they are satisfied with their training in a number of CACREP standard-related areas. Entry-level students had satisfaction ratings of 3.5 or above (on a scale of 1-4) in relation to their training in multicultural counseling, ability to generate responses in novel situation, communication and relationship skills, and areas related to professional identity, self-reflection, and

https://www.warner.rochester.edu/about/accreditation

https://www.warner.rochester.edu/about/accreditation


professional behavior. Areas for improvement, identified by a score of less than 3.0 (range 2.44 to 2.89), included 7 out of the 26 items and were issues related to:

Completing a treatment plan based on a DSM diagnosis, mental status examination and biopsychosocial clinical intake interview

Monitoring and documenting the progress being made in the implementation of a treatment plan

Assessing the needs of a client in crisis Counseling process evaluation Providing session structure and focus through methods for furthering

counseling goals. Using flexibility and ability to generate novel responses to unexpected or

unfamiliar counseling situations Multicultural, consultation and advocacy skills

Of these 7 areas, the first five have been addressed since this group of alumni graduated, thus the feedback of future alumni will be critical in evaluating our success in this regard. Further, site supervisors of recent graduates did not rate performance below a 3 in any of these areas that were assessed, nor did recent graduates themselves cite these areas as needing attention. The doctoral student competency self-ratings on this survey are not reported due to the low number of responses (1) and the possibility of identifying the respondent. One item that was concerning, particularly in light of the interventions that we have implemented in response to previous dissatisfaction, is the desire at the doctoral level for greater guidance in preparing for the comprehensive examination and more opportunities to teach, publish, and participate in research. While this may also be a matter timing in that the respondent in this survey may not have experienced program changes in regard to past feedback at the doctoral level, the issues of this respondent will nonetheless be addressed in the section related to “Implementation of Action Items”. Also, as we go forward, we need to understand how alumni self-perception regarding CACREP competencies is so divergent from the outcomes we observed: 1) in KPI measures, 2) site supervisor ratings of student competencies, and 3) graduating student perceptions of their own competencies, and whether this is a function of alumni going through their program before programmatic changes were implemented. One recurrent issue emerges with each administration of the alumni survey is the fact that Warner tuition creates considerable hardship and strain for counseling students, which undoubtedly colors students’ view of the Warner experience. As faculty meet this year, we will be trying to devise strategies to potentially ease the financial burden on students. Possible avenues would include increasing student aid and attempting to acquire training grants that would cover student tuition.

Student Perception Assessments


10. Individual Class Instructor Evaluations The Instructor Evaluation Form assesses the student experience in individual classes, particularly in terms of the skill of the instructor in creating a course delivery that is engaging, organized, relevant, and has fair and clear grading policies. Students are also afforded a space to freely comment about any aspect of their experience in the course, related to content, delivery, and and any salient points regarding the instructor’s professionalism and ability to relate to students. FINDINGS RELATED TO INDIVIDUAL COURSE EVALUATIONS See the following link for a compilation of the individual class instructor evaluations for the academic year, 2017-2018: (click here to see Table 1, Summary of CACREP Course Evaluations). We are proud to report that students continue to rate our counseling classes very highly. For the Summer 2017-Spring 2018 academic year, students in our program, on average, rated statements about the courses and instructors as 4.53 on a 5-point scale. This score exceeds slightly the score from the previous year, and generally is consistent with the high ratings from the last three years. Furthermore, consistent with previous ratings, no notable patterns or trends emerge when the data are disaggregated by course and program. These quantitative findings corroborate the evidence our faculty have garnered from their conversations with students about their classroom experiences. Given the high teaching ratings, we plan to continue with much of our current approach to teaching, but will also consider new ways to improve delivery of instruction. For example, we continue to increase our efforts regarding pedagogical approaches. More specifically, we are striving to offer a diversity of didactic and experiential learning opportunities and to increase the use of creative technologies. We have, for example, implemented dramatic changes in our Pre-Practicum course that gives students more time engaged in experiential activities that simulate real-world clinical practices. We are also striving to ensure that our doctoral students, who are often given teaching responsibilities, have exposure to cutting edge classroom technologies and reflective teaching practices. Many of our doctoral students continue to imbed coursework from a certificate program offered by the Warner School in online learning. This semester, we are looking toward integrating online techniques specifically designed for the needs of the counseling classroom. In our striving to integrate cutting edge research and best practices into our curriculum, we have 1) dramatically revised our master’s-level course in addictions 2) created a master’s-level psychopharmacology course, and 3) included, at both the master’s and doctoral-levels, information on neurobiological phenomena that can augment counseling practice. Thus, although our evaluations have remained positive through this entire accreditation period, we are continually monitoring evaluations and student feedback for areas in need of improvement. 11. Assessment of Clinical Experience and Clinical Site 12. Assessment of Clinical Site Supervisor


Several forms are utilized to give students an opportunity to offer an assessment of their clinical site, clinical experience, and the site supervisor. An Interim Assessment Form gives students a chance to offer feedback regarding experiences at the clinical site midway through each semester in which they have a clinical field placement, while end-of the-semester data is collected using a Site and Site Supervisor Evaluation Form. FINDINGS FOR SITE, SITE SUPERVISOR, AND TAPE SUPERVISOR EVALUATIONS (click here to see Table 2, Summary of site and site supervisor evaluations). These ratings overall, were very positive with an average score of 3.72 out of 4.0. Our survey from last year signaled that we should consider how students can get more access to group work and to assessment-related experiences. This year, although these scores were quite positive (3.15 and 3.32 out of 4 respectively), they remained the lowest scores on the survey. These outcomes suggest that more attention to these two areas is warranted. Tape supervisors were also rated quite highly at 3.87 out of 4 (Click here to see table 4, Summary of Tape Supervisor Evaluations). This average reflects an increase from last year, where tape supervisors again were rated quite highly at 3.79 out of 4.0. Again, as was true for the Small Group Supervisor Evaluation, and the Site and Site Supervisor evaluation, no areas of concern requiring immediate improvement emerged from the findings. We note that each of the supervisor evaluations also include qualitative data from students, which are also carefully evaluated by program faculty. In order to protect student and supervisor confidentiality, we do not post the qualitative data. Overall, we are pleased with the high level of supervisee satisfaction with all their supervisors and will continue to implement the same process for training and supporting our supervisors. We will also continue to monitor our sites with the intent of ensuring that our placements remain high quality and that students are matched with placements that best suit their individual interests and plans for future employment. 13. University Tape Supervisor Assessment In light of the fact that we do not have an onsite clinic, we provide university-based individual supervisors for all of our entry-level students. The Student Counselor Evaluation of University Supervisor Assessment has a dual purpose, the first being providing both quantitative and qualitative data to the course instructor concerning the performance of a particular university-based supervisor, and the second being a mechanism for having our university-based supervisors, who are almost exclusively current doctoral students, important feedback on their supervision skill development. FINDINGS FROM UNIVERSITY TAPE SUPERVISOR SURVEY


Tape supervisors were rated quite highly at 3.86 out of 4, and is up from 3.79 out of 4 in 2016-2017 (Click here to see table 4, Summary of Tape Supervisor Evaluations). No areas for immediate improvement emerged from the findings. Overall, the already high scores for supervisors did not change significantly from last year. We note that each of the three supervisor evaluations also includes qualitative data from students, which is also carefully evaluated by program faculty. In order to protect student and supervisor confidentiality, we do not post the qualitative data. Overall, we are pleased with the high level of supervisee satisfaction with all their supervisors and will continue to implement the same process for training and supporting our supervisors. We will also continue to monitor our supervisors with the intent of fine tuning our current system. 14. Small Group Supervision Assessment All counseling students who are enrolled in practicum or internship, at the entry or doctoral-level, receive small group supervision. Thus, at the end of each semester of clinical experience, students are asked to complete a Small Group Supervision Evaluation that provides feedback regarding the students experience of processing field-related issues in a small group setting while it also provides feedback for the individual supervisor running the group. FINDINGS FROM SMALL GROUP SUPERVISION ASSESSMENT Small group supervisors were rated, on average, 3.77 out of 4 (click here to see Table 3, Summary of Small Group Supervisor Evaluations), and no areas for immediate improvement emerged from the findings. Students uniformly found their supervisors to be supportive, responsive, knowledgeable, caring and able to stimulate critical reflection. 15. Graduating Student Survey Data that will be collected The data collected from all graduating entry-level students is contained in the Graduating Student Survey. This survey asks students to share what they considered to be the strengths and weaknesses of their respective programs, suggestions going forward, and a sense of short-term career goals. FINDINGS FROM GRADUATING STUDENT SURVEY The Graduating Student Survey is comprised primarily of qualitative responses to a set of structured questions. What follows is a thematic analysis of the responses to each of the questions. Learning in Classes In response to the questions: “How would you summarize your learning through coursework in your program at Warner? and What were the highlights and what


would you like to see improve?” the following data were generated:

There was a desire for more engagement with theory, especially in internship.

Doctoral students who taught classes were generally not as well received as regular faculty.

Electives with more advanced content would be desirable. Core classes are more generally too oriented to mental health students at the

expense of school counseling students. There was a huge emphasis on growth and development. More attention to evidence-based practice is needed. It is a real positive that the program is CASAC-T eligible. The program fostered self-development. Students learned cultural competency. Students would like more information on licensure process.

Learning Through Field-Based Experiences In response to the questions: “How would you summarize the impact of your learning through field- based experiences in either clinical practice or research? and What would you consider the highlights of these forms of instruction and what were the challenges or areas for improvement?”, the following data were generated:

Warner prepared students for real world situations. Most learning came from field-based experiences. In the school program, coursework needs to coordinate better with the field

experience. Courses were generally very mental health-based. The program should add more trauma skills and knowledge.

Interaction with the Faculty In response to the questions: “How would you characterize your interaction with Warner faculty and/or staff?, Were your educational or professional goals advanced by relationships with these individuals?, If so, how? and If not, what would you prefer?”, the following data were generated:

Most faculty were welcoming, encouraging, warm, student centered, supportive, and always present.

Pivotal to students’ education. Students preferred regular faculty rather than doctoral students. Some faculty did not do enough to update curriculum and teaching style. Relationships with doctoral students were beneficial and helpful. Some students had a closer relationship with the doctoral TA’s.


Constructive Feedback In response to the question: “Is there any constructive feedback on your Warner experience as a whole that you would like to provide in service of enhancing the program?”, the following data were generated:

The program needs to be mindful of the needs of part-time students. The capstone project is overwhelming to students who are trying to manage

Internship class simultaneously. More second year electives would be desirable. More courses for school counselors would be desirable. A credit-bearing thesis course would help to manage the process. It would be helpful to have an alumni pipeline to employment opportunities

and a network plan for after graduation. The program needs more attention to be given to evidence-based manualized

treatment. The addition of CBT to the curriculum would be helpful. The program would be enhanced by the addition of a human sexuality

course. The course load for a full-time student is too much with a 20-hour internship. Students in mental health need more skills-based learning in relation to

activities such as notetaking, diagnosis, and assessment. Providing practice for the NCE would be desirable. More guest speakers in classes would be desirable.

IMPLEMENTATION OF ACTION ITEMS

This section highlights the ways in which the members of the Counseling and Human Development program, and in some cases the Warner School more broadly, have addressed concerns and suggestions gleaned from our program evaluation activities. All of the implemented changes in this section were predicated on information gleaned from one or several sources of data collection. While some of these changes are well established and have been permanently adopted, other implementation efforts are newly introduced and are still under considerable scrutiny. Because it often takes considerable time to make the journey from an idea that emerges from evaluation data to full implementation, the action items presented here are often part of multi-year efforts. Entry-Level Programs Improvements in Mental Health and School Counseling Internship Records The use of Blackboard continues to expand as a resource library for documents and professional resources for site supervisors and tape supervisors. We now have articles available on Blackboard on topics related to clinical supervision and special


issues in counseling. This year, we kept all the practicum and internship assessment tools (weekly hours logs, supervisor assessments, session tape evaluations) and all supervisor assessments of students electronically on Blackboard. These materials are now fully on-line and have been updated to reflect the CACREP 2016 Standards. Expanding Curriculum in Psychopharmacology and the Biological Basis of Counseling We have, for several years, received feedback from various stakeholders stating that students would be better prepared to enter clinical mental health sites if they were more knowledgeable concerning issues related to psychopharmacology. Combining this request for more psychopharmacology with the need to modify curriculum to include CACREP and CASAC standards referencing biological and neurobiological issues related to disorder etiology and intervention, we have devised a new course that will be offered for the first time in the Spring 2018 semester that encompasses basic psychopharmacology, brain anatomy and physiology, principles of neurocounseling, and an array of cutting edge topics in biology including fetal programming, stress epigenetics, psychoneuroimmunology, and the brain-gut axis. It is our hope that this course, in addition to requests for more instruction in psychopharmacology, will in part address feedback from site supervisors who indicated that our students needed additional exposure to material that would help them to integrate biological, psychological and social interventions in their work with clients. Efforts to Expand Curriculum to Include Optional Certificate in Mind-Body Intervention Based on feedback that includes 1) students’ interest in mind-body work, 2) students’ interest, particularly in Mental Health Counseling, of expanding employment and internship outlets, and 3) a call, by students and supervisors, to offer more in-depth study of evidence-based practice, we are introducing a new certificate that features mind-body interventions in a context that introduces students to an area of practice that addresses, in part, all of these issues. Mind/body

approaches to healing and wellness are gaining in popularity in the U.S., and in certain

forms, have been a central part of practice in Eastern and indigenous approaches to

psychological and physical wellness. Further, and of significance to us, is that research

supports their efficacy in treating a number of psychological issues. We have recently

submitted a proposal for curriculum in Advanced Certificate Program in Mind/Body

Healing and Wellness. In the context of student and supervisor feedback, we hope that

implementation of this program will address feedback in the following ways:

Offer students a path to greater understanding of mind/body work

Strengthen students understanding of the biological basis of mental health

Increase their knowledge base regarding mind-body interventions, including

various emotion-focused therapies and CBT

Offer students the opportunity to take coursework in an interprofessional

environment and establish contacts with personnel from the University of


Rochester Medical Center, thus potentially increasing their attractiveness for

employment or internship opportunities in integrated care settings.

CASAC-T Credential This year we implemented a process that is particularly relevant to clinical practice with the Licensed Mental Health Counselor (LMHC) credential in New York State. In New York State, where there are restrictions on the LMHC scope of practice that often hinder employment, the addition of the CASAC-T (i.e., Credentialed Alcohol and Substance Abuse Counselor Trainee) allows LMHC’s more flexibility and greater success in procuring employment. After considerable time invested in making changes to our curriculum and in gathering necessary supporting data, we received approval from the New York State Office of Alcoholism and Substance Abuse Services of our CASAC-T curriculum, which gives students the option of simultaneously pursuing the limited permit for the LMHC and CASAC-T. Re-Visioning Pre-practicum Preparation In response to feedback from alumni, who indicated that areas for improvement include issues related to organization of counseling session data, monitoring and documenting progress in the treatment plan, and conducting a biopsychosocial evaluation, we engaged in a complete revision of our Pre-Practicum course. The course now includes more emphasis on active biopsychosocial case conceptualization, multiple ongoing requirements to actively generate treatment plans, documentation practices that include the use of technology, and the use of client data to monitor progress and inform intervention strategies. While not a direct result of program evaluation feedback data, the revisioning of Pre-practicum also attempts to increase exposure of Community Mental Health Counseling students to technology that they are likely to encounter in clinical placements. It should be noted that the cohort of alumni and others who provided feedback asking for more exposure to the types of things that were added to the Pre-practicum course went through our program before these additions were implemented. Addressing Workload in the Final Semester of the Mental Health Counseling Program In response to the issue of workload in the final semester of the Community Mental Health Counseling curriculum, our last Annual Report detailed our remediation strategies. First, a series of scheduled workshops became available to help students structure their thesis work. Second, thesis work became embedded in the Internship course so that a more scaffolded process guides students through their respective capstone projects. Third, our advising for Mental Health Students became the purview of a single program advisor, Dr. Linnenberg, who is able to take a uniform, proactive stance working with students around the notion of completing coursework and internship hours during the summer. While these steps have been of some help to students in managing their capstone projects, feedback from graduating students underscores that there is still considerable distress over the limited time available to complete capstone work, particularly in light of having to


manage a 20-hour weekly clinical commitment and additional coursework. The faculty will be revisiting this issue, which may result in changing the fundamental nature of the required project. Some discussion has led to the possibility of having a culminating portfolio that could be amassed more incrementally rather than putting so much emphasis on the final semester. Ongoing Effort to Increase Time at Mental Health Internship Sites Based on feedback from our internship site supervisors in the community mental health counseling program, we have continued our efforts to expand the time that our students spend at internship sites. Rather than allowing students to complete their practicum in one site and their internship in another, we are increasing our efforts to have students remain in a single site for the duration of practicum and internship. We decided that while this will limit the breadth of their training experiences, it will also provide students with more depth in a particular area. This effort diminishes the amount of site-specific orientation and training activities that need to occur when students change sites. We continue to monitor how this decision to have a continuous practicum-internship experience impacts student learning and eases the strain at clinical sites. Addition of Opportunity for Expanded Training in Urban Education for School Counselors As one response to School Counseling students’ desire for more school-related course work, we have encouraged students in the School Counseling program to consider adding courses from our Urban Teaching and Leadership Program (UTL), which was designed to prepare counselors, teachers, and leaders committed to addressing the inequalities plaguing public education in urban centers. The clinical coordinator and program advisor for school counseling, Dr. Bonnie Rubenstein, who plays a central role in delivery of this program, has been instrumental in expanding participation in this program of students in the school counseling program. As an invaluable experience, these students are exposed to the challenges of urban school counseling in an inter-professional environment that includes input from counselors, teachers, and administrators. Opportunities to Expand Intervention Capability for Clients with Neuro-developmental Disabilities We reported in our 2015-2016 annual report that in response to comments from students, alumni, and supervisors who emphasized the need for school counselors to be better prepared to work with students with disabilities, Dr. Bonnie Rubenstein, who designs much of the curriculum in school counseling, has expanded her coverage of disabilities issues in the school settings. In addition to Dr. Rubenstein’s efforts, as of Fall 2017, we are able to offer school counseling students the opportunity to add to their school counseling degree a license in Applied Behavior Analysis (ABA). Students who choose this option acquire a repertoire of evidence-based skills germane to work with children and adolescents diagnosed with Autism Spectrum and related disorders. In addition to doubling down on our


response to the student, alumni and supervisor call for more work in disabilities, it is our hope that students availing themselves of the ABA program offerings will bolster their employability by setting them apart from other individuals competing for school counseling jobs. Expanded Child and Adolescent Counseling Instruction Over the past several years we have had numerous requests from a variety of stakeholders to offer more coursework in child and adolescent counseling. This past summer, our child and adolescent course was expanded to include a number of important features including developmental psychopathology, infant mental health, and emerging child and adolescent treatments, both individual and dyadic, for children experiencing early life trauma. This course is one that should address some of the concerns of school counseling students wanting more exposure to strategies for addressing the needs of the child-adolescent population, particularly in the arena of trauma-informed care. In addition to expanding the child and adolescent course, we have added a course in infant mental health and are considering offering mental health students the opportunity to build a concentration in this area. There are currently opportunities opening up in this area in New York State, and it is our hope that our attempt to be proactive in offering the infant mental health course will help to bolster employability of mental health students. Augmentation of School Counseling Faculty In response to need expressed in the past by current students and graduates in school counseling to increase the number of faculty with an expertise in school counseling, we attempted to augment the school counseling affiliated faculty by intentionally reaching out to experienced school counselors who are interested in our doctoral program and have offered them funding opportunities. This resulted in the addition of several highly qualified and experienced school counseling doctoral students who are able to teach and supervise our school counseling students. It is clear from the latest feedback from school counseling stakeholders, that these efforts are still not meeting the needs of this constituency. The faculty will be considering ways in which curriculum in core classes can be more intentionally focused on school counseling applications. One possibility we will consider is to look at how some of our courses that are offered multiple times each academic year might be divided into Mental Health and School-related sections. Doctoral Programs Increasing Doctoral Student Research Opportunities

In response to doctoral students and doctoral alumni voicing their desire for more research and publishing experiences, faculty have continued to expand research opportunities for students. We mentioned in our last Annual Report the addition of Dr. Silvia Sorensen to the Counseling and Human Development faculty as an addition to our research profile. Dr. Sorensen is the principal investigator or co-


principal investigator on an array of research projects and is an outstanding mentor. Her many years of experience working as a researcher gives her the means to provide students the opportunity to make contact with an array of researchers in the University Medical Center. With the intention of expanding doctoral student methods skills, we are now offering a course, EDE 481 Practical Knowledge and Skills for Research in Health and Education, that is designed to make our students attractive to University researchers looking for laboratory apprentices or research coordinators. Additional courses that have been added to bolster research opportunities include Designing Intervention Research in Applied Settings; Studying Human Development, Counseling, and Education in Context: Advances in Design and Method; Creativity in Qualitative Methods; and Advanced Applications of Qualitative Research Methods in Social Science. In addition to providing research opportunities, these course offerings address some of what we have observed in our monitoring of student milestones and in discussions emerging from our annual review of students, namely, that students would benefit from additional training in qualitative methods. Also in response to doctoral students having expressed the desire for additional research opportunities the faculty decided to expand our on-going research groups to include more students. More specifically, professors Lynch, Douthit, Swanson, Sorensen and Guiffrida are now conducting regularly scheduled scholarship support groups in their respective areas of interest. Working Towards Improving Leadership and Advocacy Skills Feedback from the field indicates that our doctoral students do not identify strongly with their roles as advocates and leaders. In response to this issue we are putting additional emphasis on including advocacy and leadership activities in the accumulation of doctoral internship hours. This emphasis is consistent with the 2016 CACREP Standards for doctoral programs and will hopefully make students more comfortable and invested in leadership roles. Improvements in Doctoral Internship Records The use of Blackboard continues to expand as a resource library for documents and professional resources for site supervisors and tape supervisors. We now have articles available on Blackboard on topics related to clinical supervision and special issues in counseling. This year, we kept all the practicum and internship assessment tools (weekly hours logs, supervisor assessments, session tape evaluations) and all supervisor assessments of students electronically on Blackboard. These materials are now fully on-line and have been updated to reflect the CACREP 2016 Standards. A Revised Comprehensive Examination for Doctoral Students Based on doctoral student feedback and faculty perceptions, we talked in our last Annual Report about how our comprehensive examination structure was modified. This examination, while allowing us a thorough assessment of students’ readiness


for completing their dissertations, was also described by some students as too lengthy and laborious. Faculty concurred adding that this process detracted from doctoral students’ ability to engage in dissertation research in a timely manner and that it often delayed their becoming immersed in the process of developing, through presentation and publication, their identity as scholars. As a result of this feedback and deliberation, we decided to scale back the exam and provide a more compacted, 6-month schedule for completing the three required papers. We have now had approximately two years of feedback on this process, and it is not clear that it has significantly changed student’s views, nor has it increased the quality of the products we are receiving. We will be discussing, as a faculty, whether this examination can be managed in a more scaffolded way as opposed to having students writing without faculty support. Having a more scaffolded process could compromise the objective of this examination in terms of its value in predicting student readiness for dissertation work. Alternatively, it can be argued that the comprehensive examination has inherent developmental value that could be enhanced with some degree of mentoring. Trying to resolve this debate will be the main charge of our faculty deliberations regarding this topic. Portfolio Modifications Last year, we talked about a programmatic change at the doctoral level that came from discussions that we have had in both monthly meetings and in our annual retreat regarding our current “Portfolio” process. The Portfolio is a doctoral milestone that occurs after doctoral students have successfully progressed through 18 credits of doctoral work, including several of the notably challenging methods and core counseling courses. In the Portfolio process, students are asked to provide a narrative in which they articulate a coherent plan that shows how past and future doctoral work will lead to successful completion of their dissertation. Students are expected to have formulated some rudimentary ideas about the focus of their dissertation question, and it is our hope that the narrative will give them a roadmap that will facilitate the process of moving through their dissertation work. Another purpose of the Portfolio is to ensure that before students move forward in the program that they are developmentally ready, as scholars, to do so. Thus, another requirement of the Portfolio is to complete a well-crafted literature review on a topic of the student’s choice. This literature review must follow strict guidelines and adhere to APA Style. Students who are unable to generate an acceptable literature review are asked to repeat the process the following year, and if they are still unable to successfully complete the steps they may be dismissed from the program. In the previous iteration of the Portfolio process: 1) students were allowed to submit co-authored works, 2) materials were adjudicated by faculty outside the counseling program, and 3) rules concerning the construction of the literature review were not strictly articulated. In the new process, emanating from faculty discussions: 1) students are only allowed to submit single-authored works, 2) materials are adjudicated entirely by Counseling faculty, and 3) rules concerning the construction and content of the literature are strictly articulated and students are expected to adhere to these revised guidelines.


Discussions among core faculty after observing the outcome of this new Portfolio process indicates that more work is needed in this area. Faculty hoped that the new emphasis on literature review writing might cause students to be more engaged in the nuances of this task. Numerous resources are made available to students to help in the literature review process including detailed scaffolding in at least two required courses, a course on academic writing, and a writing support services that provides a workshop on writing a literature review. Possible remediation efforts being considered by the faculty include: 1) increasing the number of courses that we scaffold literature review writing, 2) requiring all students to take our course in academic writing, 3) encouraging greater engagement with Writing Services, and 4) putting greater emphasis on the writing portion of the admissions process. Designing Online Courses In response to requests by doctoral students and others for more exposure to the principles of on-line learning and the skills required to teach an on-line learning course, a series of three courses have been designed that are devoted exclusively to giving students an understanding of how consumers of on-line coursework learn in a virtual environment and how to design on-line coursework, including use of contemporary tools for on-line course construction. Students can take individual courses in online teaching and learning or may opt for a full certificate.

Dissertation Proposal Seminar Expansion In response to difficulties that faculty have observed and students have voiced regarding the dissertation process, we created a dissertation proposal seminar class for students who would prefer a scaffolded experience in their dissertation proposal writing. Additional feedback underscoring the realization that many of our doctoral students have family responsibilities, while others have had to relocate in the midst of their doctoral program has led to our decision to offer the proposal seminar in a highly interactive, synchronous hybrid format. Predicated on the success of the dissertation proposal seminar class, we are currently considering how we might extend the scaffolding process in a way that encompasses the entire dissertation process. We envision a series of courses, structured much like the current dissertation proposal class, that provides support for students at each phase of their dissertation work. New Efforts to Support Adjunct and Affiliated Faculty Even though we have instituted several new practices that were designed to increase support for adjunct and affiliated faculty, it is clear from our survey, that many have not felt the impact of those efforts (e.g., offering a comprehensive adjunct and affiliated faculty orientation every semester). As a further step to address some of the specific concerns of these members of our faculty, we have begun a program in which each of our core counseling faculty members is in essence responsible for mentoring the adjunct and affiliated faculty associated with 2 to 3 courses. As part of the mentoring process, the core faculty are expected to ensure that their mentee


1) acquires needed syllabi, 2) is familiar with the CACREP standards embedded in their courses, 3) understands the concept of a key performance indicator and is instructed in how to complete and record key performance indicator measures, 3) understands the process for ordering textbooks and acquiring desk copies, 4) has any questions answered concerning the use of Blackboard, and 5) understands the process of assigning grades. Faculty mentors will also be available as questions arise during the semester concerning student problems or classroom management issues. These efforts by faculty mentors will supplement the steps, mentioned above, that we have taken to improve the experiences of our adjunct and affiliated faculty This includes providing a school-wide orientation for adjunct faculty that covers a number of curricular (e.g., use of Blackboard) and logistical issues (building access, photocopying, etc.). Likewise, per the adjunct faculty’s request, we established:

A data file of sample syllabi for all of the courses taught in our program (Dr. Linnenberg, one of our core faculty members, maintains this file).

A template to create new syllabi. A complete set of measurement rubrics that cover all of the CACREP

standards associated with each course (Dr. Linnenberg provides direction to the AAF concerning CACREP requirements).

Vehicles for assisting AAF’s with CACREP data input.

Additional Items Emerging from Program Evaluation in Need of Attention

Although we have made many efforts to improve program delivery, there are a number of things that still need attention going forward. Some of the things that we would like to consider in the coming year include:

Additional scaffolding of our doctoral dissertation process beginning with the proposal process and continuing through the data collection and analysis, and the final dissertation write-up.

Improving ways to capture assessment data in relation to employers and alumni. This is particularly important in relation to former doctoral students and doctoral student employers, neither of whom are well represented in our data collection efforts.

Attempting to reconcile the tension that occurs between doctoral students wanting more teaching opportunities and master’s students wanting fewer doctoral student instructors.

Expanding the inclusivity of our demographic data collection by moving away from a binary gender identification scheme to one that is more inclusive of trans, queer, and intersex options.

Considering additional ways to promote doctoral student publication opportunities.

Considering more carefully the needs of part-time


Assessing the need for expansion of curriculum regarding human sexuality; trauma skills and knowledge; work with challenging populations (e.g., the intersection of poverty and chronic severe mental health issues); and the use of research, ethics and critical thinking to inform practice.

Bolstering the relationship between students and alumni beyond our current efforts (e.g., our annual Black Tie Social, active Chi Sigma Iota chapter, annual Counseling Symposium).

MOVING FORWARD

As we move forward in our ongoing efforts to improve the quality of our programs, we will do so guided by the following principles:

In shaping our programs, we are committed to showcasing the core

principles of the counseling profession. This means that all that we do will be driven by a dedication to social justice, life-long healthy development, client empowerment, respect for diversity, understanding individuals in context, and the need for evidence-based practice.

Program development is best achieved through the input of a community

of dedicated stakeholders. Changes to our programs must be informed by the wisdom of those clinicians who have dedicated their lives to the direct service of clients; scholars who are devoted to understanding best practices and ways of applying cutting edge information emerging from research in counseling, psychology, psychiatry, human development, and the biological sciences; the experiences reported by the students who we hope to teach; and the voices of the clients that we serve.

The quality of our curriculum is ultimately judged by the degree to which

our students have assimilated its contents. Thus, we need to continually devise more effective ways to evaluate our students’ depth of understanding of curriculum, ability to transform theory into practice, ability to turn practice into theory, facility in regard to serving their clients justly, ethically and effectively, and capacity for modeling for their clients the tenets of lifelong healthy development.

Appendices


Table 1 Summary of CACREP Course Evaluations

Academic Year: Fall 2017-Summer 2018

Total Responses: 450

Scale:

1 = strongly disagree (SD)

2 = disagree (D)

3 = neither/neutral (N)

4 = agree (A)

5 = strongly agree (SA)

item#

item description AY18 AVG.

1 the course and course activities were well organized 4.46

2 written assignments advanced the goals of the course 4.42

3 reading assignments contributed to the goals of the

course 4.46

4 the instructor showed a genuine concern for the

students 4.78

5 the instructor was available to students 4.66

6 classes were interesting and stimulating 4.35

7 students in this course were free to disagree and ask

questions 4.68

8 I learned a great deal from this course 4.42

Average 4.53

Less than other

courses

Equal to other courses

More than Other

Courses

9 compared to other courses of comparable credit

hours, the work required for this course was: 25.78% 66.44% 7.78%

Yes no

10 The method for student evaluation was clearly

defined 96.00% 4.00%

11 the student evaluation(s) were a fair assessment of

my performance in this course 96.67% 3.33%


Table 2 Summary of Student Feedback-Site and Site Supervisor Evaluation

Master and Doctoral Students


Total Responses: 48 Scale:

1 = Not Satisfactory 2 = Emerging, Needs Improvement

3 = Basic Proficiency 4 = Outstanding Performance

Question AY18

Rating Average

My site supervisor was helpful in orienting me to the work site. 3.85

My site supervisor was helpful in arranging access to individual clients and groups/families for me to work with. 3.79

My site supervisor communicated clear expectations for my work. 3.79

My site supervisor was accessible to me. 3.85

My site supervisor was responsive to my questions or concerns. 3.94

My site supervisor consistently kept his/her weekly supervisory meetings with me. 3.66

My site supervision provided me with a helpful balance of direction and autonomy. 3.81

My site supervisor was a good professional role model. 3.91

I would recommend this internship site to others. 3.64

I had the opportunity to innovate at this site if I cared to. 3.68

I would recommend this site supervisor to others. 3.87

I had a good deal of individual counseling practice at my site. 3.55

I had a good deal of group counseling practice at my site. 3.15

I improved my individual counseling skills. 3.72

I improved my group counseling skills. 3.28

I was able to improve my assessment skills at my site. 3.32

I learned new ways of intervening with clients (e.g. family work, consultation, advocacy) during this internship experience. 3.68

I learned a lot about the work of a counselor in the real world. 3.83

My confidence as a counselor increased. 3.79

What I learned and did at the internship site affirmed my choice to become a counselor. 3.79

I have become more aware of the relationship between my personal values and professional decisions and actions. 3.81

I am able to describe my decision-making processes regarding my counseling work. 3.68

I am motivated to continue professional development beyond academic requirements. 3.85

My ability to situate client experiences and perspectives in multiple contexts, such as social class, ethnicity, gender and family, to inform my interpretations and practices

has improved. 3.81

My ability to systematically evaluate my work from various perspectives, such as client's viewpoint, my expectations and from supervision has improved. 3.81

I experience myself as a continuing to improve as a counselor. 3.83

Average 3.72


Table 3 Summary of Student Feedback-Small Group Supervision



Total Responses: 47

Scale:

1 = Not Satisfactory

2 = Emerging, Needs Improvement

3 = Basic Proficiency

4 = Outstanding Performance

Question AY18 Rating

Average

Peer interactions provided important ideas and support. 3.66

The small groups were good forums for exploring issues and problems. 3.64

The small groups became an important resource for empathic support and learning. 3.7

I felt safe enough in group supervision to disagree with my peers about issues or interpretations. 3.7

Student case presentations were a good learning experience. 3.68

The interactions in group supervision challenged my presumptions about my work and motivated me to try alternative

ways of understanding and practice. 3.62

My small group supervisor listened well to our comments, experiences and concerns. 3.96

He/she showed a genuine concern for me and my development as a counselor. 3.94

He/she was supportive and encouraging. 3.94

He/she had helpful ideas and suggestions for me in my counseling work. 3.77

He/she facilitated our discussions or used activities in a way that was helpful to our exploring issues and problems in our counseling. 3.79

He/she offered resources that were helpful to me now or probably will be helpful in the future. 3.79

He/she facilitated discussion in a way that made me feel comfortable participating. 3.83

Average 3.77


Table 4 Summary of Student Feedback-University Tape Supervision

Master Students


Total Responses: 47

Scale:

1 = Not Satisfactory

2 = Emerging, Needs Improvement




Average

Gives time and energy in observations, tape processing and case conferences. 3.87

Accepts and respects me as a person. 4.00

Recognizes and encourages further development of my strengths and capabilities. 3.85

Gives me useful feedback when I do something well. 3.83

Provides me the freedom to develop flexible and effective counseling styles. 3.94

Encourages and listens to my ideas and suggestions for developing my counseling skills. 3.89

Provides suggestions for developing my counseling skills. 3.87

Helps me understand the implication and dynamics of the counseling approaches I use. 3.81

Encourages me to use new and different techniques when appropriate. 3.83

Is spontaneous and flexible in the supervisory sessions. 3.87

Helps me define and achieve specific concrete goals for myself during the practicum/internship experience. 3.72

Gives me useful feedback when I do something wrong. 3.81

Allows me to discuss problems I encounter in my practicum/internship setting. 3.94

Pays appropriate amount of attention to both me and my clients. 3.91

Average 3.87


Table 5 CACREP Rubrics, By Measure

0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 01

0 0 0 01

0

6

0

23 3

45

4 4 4

8

18

3

15

7

20 20

16 16 16 16 16

14

0

5

10

15

20

25

Nu

mb

er

of

stu

de

nts

-av

era

ge

sch

ore

on

Me

asu

re

Core Curriculum Measures Program Objectives 1-4

Summer 2017-Spring 2018

1-1.49

1.50-2.24

2.25-3.49

3.50-4.00

0 0 0 0 0 0 0 0 0 001 1

01

0 0 0 0 0

20

11

15

1214

12

02

0

3

7

15

11

15

12

15

2018

20

17

0

5

10

15

20

25

Nu

mb

er

of

stu

de

nts

-av

era

ge

sch

ore

on

M

ea

sure

Core Curriculum Measures Program Objectives 5-8

Summer 2017-Spring 2018

1-1.49

1.50-2.24

2.25-3.49

3.50-4.00


0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0

2

0 0 01

2 2

54

0 0

2

13

10

4

1716 16

13

22

26

15

11

6 6

11

0

5

10

15

20

25

30

CMHCKPI #1,

M1

CMHCKPI #1,

M2

CMHCKPI #2,

M1

CMHCKPI #2,

M2

CMHCKPI #3,

M1

CMHCKPI #3,

M2

CMHCKPI #4,

M1

CMHCKPI #4,

M2

CMHCKPI #5,

M1

CMHCKPI #5,

M2

CMHCKPI #5,

M3

Nu

mb

er

of

stu

de

nts

-ave

rage

sco

re o

n M

eas

ure

CMHC MeasuresSummer 2017-Spring 2018

1-1.49

1.50-2.24

2.25-3.49

3.50-4.00



0 0 0 0 0 0 0 0 0 0 00 0 0

1

0 0 0 0 0 0 0

4

3 3

2

3

2

0

3

4

3

2

4

6

5

6 6

7

13

10

4

6

7

0

2

4

6

8

10

12

14

SC1,M1 SC2,M1 SC3,M1 SC4,M1 SC5,M1 SC5,M3

Nu

mb

er

of

stu

de

nts

-ave

rage

sco

re o

n M

eas

ure

School Counseling MeasuresSummer 2017-Spring 2018

1-1.49

1.50-2.24

2.25-3.49

3.50-4.00



0 0 0 0 0 0 0 00 0 0 0 0

1 1

0

2

0

4 4 4

1

2

3

4

6

3 3

4

3 3 3

0

1

2

3

4

5

6

7

DCES1,M1DCES1,M2DCES2,M1DCES2,M2DCES4,M1DCES4,M2DCES5,M1DCES5,M2

Nu

mb

er

of

stu

de

nts

-ave

rage

on

Me

asu

re

Doctoral MeasuresSummer 2017-Spring 2018

1-1.49

1.50-2.24

2.25-3.49

3.50-4.00



Table 6 Survey Results-Alumni



Total Responses: 9

Scale:

1 = Insufficient

2 = Emergent



Rating

Average

Relationship and Communication Skills

Use interpersonal strengths (e.g., caring, integrity, confidence, sense of

self) to build a working alliance with client 3.33

Use basic communication skills to listen and respond to client with

accurate empathic understanding 3.33

Average 3.33

Rating

Average

Assessment and Appraisal Skills

Organize session data into multiple meaningful frameworks that enhance

understanding of the client (e.g., developmental, cognitive, affective,

behavioral, systemic, sociocultural) 3.33

Recognize developmentally appropriate from problematic behaviors 3.33

Evaluate and adjust the counseling process to affect successful

counseling outcomes (process evaluation) 2.78

Complete a thorough DSM diagnosis 2.67

a) Complete a mental status examination 2.89

b) Complete a biopsychosocial intake 3.00

c) Assess the needs of a client in crisis 2.78

Average 2.97




Rating

Average

Intervention Skills

Use counseling and human development theories and concepts as part

of counseling practice 3.11

Provide session structure and focus through questions, interpretations,

experiential activity, problem solving strategies, and other appropriate

methods for furthering counseling goals 2.56

Use flexibility and ability to generate novel responses when faced with

unfamiliar or unexpected situations in the counseling process 2.78

Execute a variety of counseling modes, including individual counseling,

group counseling, career counseling, and small systems consultation 3.00

Complete a treatment plan based on a DSM diagnosis, mental status

examination, and biopsychosocial clinical intake interview 2.56

Monitor and document the progress being made in the implementation

of a treatment plan 2.67

Intervene on behalf of a client who is in crisis 2.44

Average 2.73

Rating

Average

Contextual Competencies

Advocacy - Provided me with knowledge of vehicles of client

empowerment, methods of advocacy, and legal liabilities of advocacy

work 2.67

Multicultural Skills - Developed my sociocultural self-knowledge,

awareness of social and political barriers to counseling, knowledge of

liberatory counseling theory, and ability to implement culturally congruent

counseling practices with diverse populations 2.89

Consulting - Provided knowledge of consultation models and processes

for integrating family, peer, and community support systems in counseling

and supporting clients 2.56

Systems Change - Developed my understanding of systems and

organizational change for the purpose of improving systems to enhance

human development 2.67

Average 2.70

Rating

Average

Technological Skills

Developed my competency with technology, including word processing,

email, library databases, and locating counseling related resources on

the Internet

3.00

Rating

Average

Disposition

Professional Identity - Familiarized me with history of the profession, roles of

professional counselors among the various human service professions, 3.11




Rating

Average

professional associations and journals in counseling, and issues surrounding

counseling licensure and accreditation

Professional Development - Developed my capacity for self- reflection,

self-understanding, openness to ongoing supervision, and commitment to

further personal and professional growth 3.67

Legal and Ethical Issues - Instilled legal and ethical standards of the

profession in my counseling practices 3.33

Professional Behavior - Developed my responsibility toward fulfilling

professional commitments and acting as a counseling professional 3.44

Average 3.39


Table 7 Survey Results-Master’s Site Supervisors


Total Responses: 24

Scale:

2 = Emergent

3 = Meets Expectations

4 = Exceeds Expectations

Rating Average

Relative to the student's level of development, how would you characterize our students as counseling practitioners?

3.13

To what extent do our students demonstrate the dispositions of a 21st century collaborative professional?

3.17

To what extent do our students act with sensitivity to diversity and social justice dimensions of counseling in their work with clients?

3.42

To what extent are Warner students competent with the use of technology in practice? (Emerging=hesitant)

3.38

To what extent do our students meet your expectations in each of the following dimensions?

Their use of research, ethics and critical thinking to guide best practice? 2.96

Their openness to developing innovations in counseling practice that meet diversity concerns?

3.29

Their commitment to serving as an agent of organizational change (social justice) or client empowerment?

3.25

Their ability to integrate biological, psychological and social interventions in their work with clients?

3.08

Their interest in fostering life-long healthy development? 3.13

Their holding of an ecological view of people in context (history, culture, social life)?

3.04

Their being caring, compassionate and self-reflective people? 3.42

Average 3.20


Table 8 Survey Results-Doctoral Site Supervisors


Total Responses: 1

Scale:

2 = Emergent



Rating Average


3.00


3.00


4.00


4.00




3.00


3.00


3.00



3.00


Their ability to function as leaders and advocates within the organization and profession?

3.00

Their ability to supervise and teach other staff members or students in training?

3.00

Their ability to generate new knowledge for the profession? 3.00

Average 3.21


Table 9 Survey Results-Master’s Employers


Total Responses: 2

Scale:

2 = Emergent



Rating Average


3.00


3.00


3.00


3.00




3.00


2.50


2.50



3.00


Average 2.91


Table 10 Survey Results-Doctoral Graduate Employers


Total Responses: 0

Scale:

2 = Emergent



Rating Average

There were no responses from the doctoral graduate employers from the 2018 survey cycle.


Table 11 Summary of Adjunct Survey


Total Responses: 6

Scale:

1 = Very Dissatisfied

2 = Dissatisfied

3 = Neither Satisfied nor Dissatisfied

4 = Satisfied

5= Very Satisfied


Average

How would you rate your overall “job satisfaction” experience as an Adjunct Faculty Member of the Warner Graduate School? 3.67

How would you rate the overall support you received as an Adjunct Faculty Member from the Counseling and Human Development Department? (Note: Department means the Chair, Faculty and Staff) 3.33

How would you rate the support you received from the department in preparing you to teach (examples of syllabus, provide you with copy of textbook, etc.)? 2.83

How would you rate the instruction/support you received from the department in educational technology (instruction on the use of Blackboard, instruction of classroom technology, etc.)? 3

How would you rate the support you received from the department in classroom management (dealing with difficult students, answering questions, etc.)? 3.33

How would you rate the feeling of inclusion into the life and work of the department (were you given the opportunity to take an active part in the department’s decision-making process)?

2.33

What is your opinion of the remuneration provided by the Warner Graduate School as compared to other institutions? 3.33

Average 3.12


Demographics Table 1: Admissions Data 2017-2018

Female Male No Response

Total Excluding NR

MS 57 (65) 14 (16) 9 71 (80)

Doctoral 13 (15) 4 (4) 3 17 (20)

Total 70 (80) 18 (20) 12 88 (100)

2+ Races

Asian Black Hispanic

Int’l White Unknown

No Respons

e

Total Excludin

g NR

MS 3 (3) 3 (3) 11 (13) 8 (9) 8 (9) 38 (43) 1 (1) 9 72 (82)

Doctoral 1 (1) 0 (0) 0 (0) 0 (0) 1 (1) 14 (16) 0 (0) 3 16 (18)

Total 4 (4) 3 (3) 11 (13) 8 (9) 9 (11)

52 (59) 1 (1) 12 88 (100)

2+ Races

Asian Black Hispanic Int’l White Unknown No Response

Female 3 (3) 2 (2) 7 (8) 7 (8) 8 (9) 30 (34) 0 (0) 0

MS Male 0 (0) 1 (1) 4 (4) 1 (1) 0 (0) 8 (9) 0 (0) 0

No Response

0 0 0 0 0 0 0 9

Female 1 (1) 0 (0) 0 (0) 0 (0) 0 (0) 11 (11) 1 (1) 0 Doc Male 0 (0) 0 (0) 0 (0) 0 (0) 1 (1) 3 (3) 0 (0) 0

No Response

0 0 0 0 0 0 0 3

Admissions: Gender x Program (By Percentage)

Admissions: Race/Ethnicity x Program (by percentage)

Admissions: Race/Ethnicity x Program (by percentage)


Demographics Table 2: Current Student Demographic Data 2017-2018

Female Male Total

MS 48 (40) 11 (10) 59 (54)

Doctoral 42 (39) 8 (7) 50 (46)

Total 90 (83) 19 (17) 109 (100)

2+ Races

Asian Black Hispanic Int’l White Unknown Total

MS 2 (2) 1 (1) 8 (7) 8 (7) 9 (8) 31 (28)

0 (0) 59 (54)

Doctoral 1 (1) 2 (2) 6 (6) 2 (2) 2 (2) 36 (33)

1 (1) 50 (46)

Total 3 (3) 3 (3) 14 (13)

10 (9) 11 (10)

67 (61)

1 (1) 109 (100)

2+ Races


Female 2 (2) 1 (1) 5 7 8 (7) 25 0 (0) 48

MS Male 0 (0) 0 (0) 3 (3) 1 (1) 1 (1) 6 (6) 0 (0) 11

Total 2 (2) 1 (1) 8 (7) 8 (7) 9 31 0 (0) 59

Female 1 (1) 1 (1) 5 2 1 (1) 31 1 (1) 42 Doc Male 0 (0) 1 (1) 1 (1) 0 (0) 1 (1) 5 0 (0) 10

Total 1 (1) 2 (2) 6 (6) 2 (2) 2 (2) 36 1 (1) 50

Current Students: Gender x Program (by percentage)

Current Students: Race/Ethnicity x Program (by percentage)

Current Students: Race/Ethnicity x Program (by percentage)


Demographics Table 3: Graduating Student Demographic 2017-2018

Female Male Total

MS 20 (77) 4 (15) 24 (92)

Doctoral 1 (4) 1 (4) 2 (8)

Total 21 (81) 5 (19) 26 (100)

2+ Races


MS 3 (12) 1 (4) 3 (12) 1 (4) 2 (8)

14 (54)

0 (0) 24 (92)

Doctoral 0 (0) 1 (4) 0 (0) 0 (0) 0 (0)

1 (4) 0 (0) 2 (8)

Total 3 (12) 2 (8) 3 (12) 1 (4) 2 (8)

15 (58)

0 (0) 26 (100)

2+ Races

Asian Black Hispanic Int’l White Total

Female 3 (12) 1 (4) 3 (12)

2 (8) 2 (8) 10 (38) 21 (81)

MS Male 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 4 (15) 4 (15)

Total 3 (12) 1 (4) 3 (12)

2 (8) 2 (8) 14 (54) 25

Female 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 1 (4) 1 (4) Doc Male 0 (0) 1 (4) 0 (0) 0 (0) 0 (0) 0 (0) 1 (4)

Total 0 (0) 1 (4) 0 (0) 0 (0) 0 (0) 1 (4) 2 (8)

Graduates 2017-2018: Gender x Program (by percentage)

Graduates 2017-2018: Race/Ethnicity x Program (by percentage)

Graduates 2017-2018: Race/Ethnicity x Program (by percentage)


Demographics Table 4: Gender Across Program 2017-2018

Admitted Current 2017-18 Graduate

Female 80 83 81

Male 20 17 19

0

10

20

30

40

50

60

70

80

90

Female Male

Gender Across Program (By Percentage)


Gender Across Program (By Percentage)


Demographics Table 5: Race/Ethnicity Across Program 2017-2018


2+ Races 4 3 12

Asian 3 3 8

Black 13 13 12

Hispanic 9 9 4

International 11 10 8

White 59 61 58

0

10

20

30

40

50

60

70

2+ Races Asian Black Hispanic International White

Race/Ethnicity Across Programs (In Percentage)


Race Ethnicity Across Program 2017-2018 (In Percentage)

Documents

2017-2018 CACREP COMPREHENSIVE EVALUATION REPORT