2019-2020 CACREP ANNUAL COMPREHENSIVE EVALUATION …

COUNSELING AND HUMAN DEVELOPMENT PROGRAM AREA

2019-2020

CACREP ANNUAL COMPREHENSIVE EVALUATION REPORT

M.S. SCHOOL COUNSELING M.S. MENTAL HEALTH COUNSELING

Ph.D. COUNSELING AND COUNSELOR EDUCATION Ed.D. COUNSELING

Table of Contents COUNSELING PROGRAM MISSION ...................................................................................................... 2

COUNSELING PROGRAM OBJECTIVES ................................................................................................. 3

Core Program Objectives ................................................................................................................... 3

Clinical Mental Health Counseling .................................................................................................. 4

School Counseling ................................................................................................................................ 5

Doctoral Program ................................................................................................................................. 6

Administrative and Evaluative Processes ..................................................................................... 6

INTRODUCTION TO THE YEARLY SYSTEMIC PROGRAM EVALUATION REPORT ...................... 7

THE DATA SOURCES USED IN THIS REPORT ...................................................................................... 9

STRUCTURE OF THE REPORT .............................................................................................................. 10

1. Annual Admissions and Across Program Demographic Data .......................................... 10

2. Individual Student Assessment of Key Performance Indicators..................................... 13

3. Annual Clinical Formative and Summative Assessments ................................................. 24

4. Student Annual Milestone Assessments ............................................................................... 24

5. Annual Student Reviews ............................................................................................................ 25

6. Adjunct and Affiliated Faculty Surveys .................................................................................. 27

7. Biannual Employer Surveys ....................................................................................................... 28

8. Annual Site Supervisor Surveys ................................................................................................ 29

9. Annual Alumni Survey ................................................................................................................. 30

10. Individual Class Instructor Evaluations ................................................................................ 31

11. Assessment of Clinical Experience and Clinical Site ........................................................ 32

12. Assessment of Clinical Site Supervisor ................................................................................ 32

13. University Tape Supervisor Assessment ............................................................................. 32

14. Small Group Supervision Assessment .................................................................................. 33

15. Graduating Student Survey ..................................................................................................... 33

IMPLEMENTATION OF ACTION ITEMS ............................................................................................. 39

Entry-Level Mental Health and School Counseling Programs ............................................. 39

Doctoral Programs ............................................................................................................................ 44

MOVING FORWARD .............................................................................................................................. 48

Appendices .............................................................................................................................................. 50

Table 1 Summary of CACREP Course Evaluations ......................................................... 51

Table 2 Summary of Student Feedback-Site & Site Supervisor Evaluation ........ 52

Table 3 Summary of Student Feedback-Small Group Supervision ......................... 53

Table 4 Summary of Student Feedback-University Tape Supervision .................. 54

TABLE 5 KPI Outcomes, 2019-2020 ................................................................................... 55

Table 6 Survey Results-Alumni, Master and Doctoral Students ............................. 56

Table 7 Survey Results-Master’s Site Supervisors ........................................................ 59

Table 8 Survey Results-Doctoral Site Supervisors ........................................................ 60

Table 9 Survey Results-Master’s Employers ........................................................................ 61

Table 10 Survey Results-Doctoral Graduate Employers ................................................... 62

Table 11 Summary of Adjunct and Affiliated Faculty Survey ................................... 63

Demographics Tables 1: Admissions Data ....................................................................... 64

Demographics Tables 2: Current Student Demographic Data .................................. 65

Demographics Tables 3: Graduating Student Demographic ..................................... 66

Demographics Table 4: Gender Across Program ........................................................... 67

Demographics Table 4: Gender Across Program ................................................................ 67

Demographics Table 5: Race/Ethnicity Across Program .................................................. 67

Table of Contents Warner School 2019-20 Annual Comprehensive Evaluation Report 2

COUNSELING PROGRAM MISSION The mission of the Counseling Program at the Warner School of Education and Human Development is two-fold:

To educate counselors who will empower their clients and students to create more rewarding and meaningful lives and relationships; become more self-reflective, caring and compassionate persons; and contribute to reform of the systems in which their lives are imbedded. To prepare counselor educators, researchers, practitioners and leaders in the counseling profession who will educate the next generation of counselors to do all of the above; contribute new knowledge in the fields of counseling, counselor education and human development; and be leaders in reforming educational and social systems to promote social justice, healthy human development, and quality relationships.

This mission is based on an ecological perspective of professional counseling that acknowledges the importance of personal development, but also recognizes that individual lives are imbedded in a variety of personal and extended relationships and social systems. Within this perspective, we:

• See human development, counseling and education in a historical, cultural and social context

• Work to foster healthy development across the life course and across a diversity of capabilities

• See the integration of biological, psychological and social intervention as an essential foundation for enduring and just human development and change.

• Recognize the counselor’s role as an agent of institutional change and social justice as well as personal empowerment

• Recognize the diversity of traditions, disciplines, practices and cultures that generate the wealth of counseling resources and welcome difference and innovation

• Use the latest research to guide best practice and intervention strategies


UNIVERSITY OF ROCHESTER COUNSELING PROGRAM OBJECTIVES

Below you will find a detailed list of the Program Objectives that constitute what we, as a program, consider to be the most essential points of knowledge for our students. Because we consider these Program Objectives to be centrally important in guiding our programming, we have also made every effort to ensure that they are reflected in our assessment of students. Core Program Objectives**

1. Students understand how the historical roots of the profession are reflected in the philosophical foundations of present-day counseling practices. This understanding extends to a working knowledge of how counseling’s orientation to practice is distinguishable from other professions striving to provide services that foster psychological health.

2. Students have a working knowledge of the ethical codes and associated ethical decision-making processes espoused by the range of organizations that regulate professional counseling practices. In addition, students are aware of the various sets of competencies, promulgated by the American Counseling Association and its divisions. Students understand the ways in which knowledge of these competencies governs ethical practice within a multicultural and pluralistic society.

3. Students are aware of their own cultural heritage, including attitudes, beliefs, understandings and acculturative experiences, and how those impact their views of others. They are aware of various dimensions of client cultural diversity, including various dimensions of power and privilege, that impact intervention success. In the context of their awareness of power and privilege, students are familiar with advocacy processes related to identifying and eliminating barriers, prejudices, and processes of oppression and discrimination.

4. Students have gained understanding of a wide range of theories and models of intervention strategy and have developed their own personal approach to counseling, which will be informed by knowledge of theory and best practices.

5. In devising intervention strategies, students have the knowledge and skill to integrate systemic, interpersonal, intrapersonal, and biological approaches to healing.

* * Core program objectives reflect knowledge and practice competencies across all master’s and doctoral programs.


6. Students demonstrate basic counseling skills and processes necessary to establish rapport, assess problems, identify client goals, and begin to implement and assess prevention and intervention strategies.

7. Students have knowledge of the theories and models of therapeutic group work and an understanding of group process and development. Students also have the ability to function as effective group leaders.

8. Students will understand the professional roles and functions of counselors across specialty areas, and will understand inter-agency and inter-organizational collaboration and consultation.

9. Students engage in self-reflective practices that sustain and build their competence in the profession of counseling and support engagement in continuous professional learning.

10. Students are able to identify emergent theories of career development and counseling and apply theoretically-relevant strategies to facilitate client career development and navigate career transitions.

11. Students have a knowledge of psychological, psychosocial, and socio-structural theories of human development and are able to apply this knowledge to aid in their understanding of common counseling issues (e.g. addictions, anxiety, grief and loss).

12. Students recognize and demonstrate knowledge of basic concepts of standardized and non-standardized testing, norm-referenced and criterion-referenced assessments, and group and individual assessments. They will also be able to identify and describe statistical concepts, including scales of measurement, measures of central tendency, indices of variability, shapes and types of distributions, and correlations.

13. Students understand the importance of membership in professional counseling organizations, professional development, and advocacy for the profession.

Clinical Mental Health Counseling 1. Students are proficient in the processes of current DSM diagnosis and are able to

accurately formulate a diagnosis of problems presented by clients; distinguish between common forms of psychological suffering that are generated by contextual challenges and mental disorder, and master the vocabulary and knowledge necessary to discuss diagnosis with collaborating professionals across the helping professions.

2. Students understand the principles, models and documentation formats of biopsychosocial case conceptualizations and treatment planning, including the bio-psychosocial-spiritual model of addiction.


3. Students have the necessary skills to apply principles, models and documentation formats of biopsychosocial case conceptualizations and treatment planning, including the bio-psychosocial-spiritual model of addiction.

4. Developing counselors will demonstrate how to theoretically conceptualize, treat, and assess addictive disorders from an ecological perspective (including but not limited to family, systems of care, societal norms and stigma, and legislative and ethical components) while recognizing the history of addiction treatment, different models of recovery, and different counselor roles across all levels of care.

5. Students have gained knowledge of etiological theories/models of addictions, including research and information on environmental factors that affect both normal and abnormal behavior.

6. Students can skillfully employ counseling and psychotherapy interventions, including evidence-based protocols for phased treatment across individual, dyadic, and group modalities.

School Counseling

1. Students have a vision of school counseling that fosters conditions ensuring educational equity, access and academic success for all students.

2. Students have developed a perspective on school counseling that is based on the National (ASCA) standards and New York State (NYSSCA) model for the development of a comprehensive school counseling program K-12. Using this perspective, students have the skills to design and implement comprehensive school programs.

3. Students understand the characteristics, risk factors and warning signs of students at risk for mental health and behavioral disorders commonly seen in school settings.

4. Students understand and employ school counselor roles and responsibilities in relation to school emergency management plans, crises, disasters and trauma.

5. Students understand and utilize principles and counseling skills of a comprehensive, developmental model to shape their practice, which consists of service and advocacy for all students in the areas of evidence-based school counseling and school counselor leadership.

6. Students are able to articulate a comprehensive, developmental strategy for implementation in school counseling programs.


Doctoral Program 1. Students have an interdisciplinary understanding of the connection between

human psychological suffering and exposure to adversity across the life course. 2. Students understand the basic principles of quantitative and qualitative research

designs. 3. Students have knowledge and skill to apply advanced research methods and

program evaluation to their research agenda. 4. Students have the ability to utilize an advanced understanding of counseling

theory to formulate integrative, holistic approaches that lead to developmental, strengths-based counseling practices, and support a healthy developmental trajectory.

5. Students have knowledge and demonstrated competency in models of advocating for clients at the individual, system and policy levels.

6. Students have knowledge and demonstrated competency regarding classroom and digital pedagogical practices, roles, responsibilities and ethics attendant to the training and education of counselors.

7. Students demonstrate the writing and presentation skills necessary to communicate scholarship with academics and practitioners in the field of counseling.

8. Students have advanced understanding of practitioner roles in group counseling. 9. Students demonstrate understanding, ethicality and capability as counseling

supervisors.

Administrative and Evaluative Processes 1. Input regarding student evaluation, as it pertains to both knowledge and

practice, reflects the perspective of core faculty, non-core faculty, students, field supervisors, and clinical supervisors.

2. Evaluation of programs more broadly reflects the perspectives of core faculty, non-core faculty, current students, former students, clinical supervisors from cooperating agencies and clinics and field supervisors from school settings.


INTRODUCTION TO THE YEARLY SYSTEMIC PROGRAM EVALUATION REPORT

The Counseling Program at the University of Rochester’s Margaret Warner Graduate School of Education and Human Development, in its striving to maintain excellence, is engaged in ongoing comprehensive program evaluation activities. These activities aim to do the following in regard to evaluating programmatic and curricular activities:

• Capture the wisdom of the full range of program stakeholders, including those

whom we have the honor of teaching as well as those to whom we owe a debt of gratitude for your generosity and commitment to joining us in training the next generation of counselors and counselor educators.

• Collect data that ensures that our instruction and clinical experiences provide students with the tools that they need to development the full range of competencies required by the Council for the Accreditation of Counseling and Educationally Related Programs (CACREP), particularly those reflected in the 2016 Standards.

• Provide a sound platform for remediation of current practices and polices while building a vision for the future that is informed by the lived experiences of dedicated clinical professionals and the students that we serve.

• Offer prospective students and the public a window into the operation of our department and the successes and challenges we have faced in training future counseling professionals.

In this spirit, this document is comprised of data that has been gathered from multiple sources over the course of approximately 12 months. It is our intention to keep the data in this report current through our ongoing program evaluation activities. Thus, this report should be considered a snapshot of our program structure and function, which we view as a dynamic organism that is constantly evolving in response to environmental changes and challenges. It is our hope that as we move toward the future, those of you who have generously given your time to provide the valuable feedback that has made this evaluation possible, will kindly continue to provide us with guidance. It is also our hope that you will use the faculty phone numbers and email addresses on our website to stay in communication with us on an on-going basis. For those of you who are new to the Warner School or just interested observers, we welcome you. Sincerely, The Warner School Counseling Faculty


Meet our new faculty member Dr. Scott McGuinness, Director of Counseling

Education: PhD, University of Rochester (counseling and counselor education) MS, Southern Connecticut State University (clinical mental health counseling) BA, Green Mountain College (psychology)

Director of Counseling, Scott McGuinness, began teaching courses in the Warner School in 2016, and in 2019, he became a member of the clinical faculty within counselor education, as well as the advisor of the mental health counseling program. He is a licensed mental health counselor (LMHC), a master credentialed alcoholism and substance abuse counselor (CASAC), an approved clinical supervisor (ACS), and a nationally certified counselor (NCC). At Warner, he teaches courses in addictions counseling, psychopharmacology, group counseling, and ethics, and he prepares mental health counseling students for their first clinical experiences in Pre-practicum. Dr. McGuinness has been a licensed counselor since 2013, and he has worked, across various levels of care, with clients who have co-occurring mental health and substance use disorders, including detoxification, partial-hospitalization, and outpatient settings. His research interests include the impact technology has on human behavior, harm-reduction approaches to substance use counseling, pedagogical methods for applying the counselor-educator curriculum in an online environment, and clinical supervision. Dr. McGuinness has presented his work at regional and national counseling conferences. As the advisor of mental health counseling students, he works with advisees to ensure they are meeting the course requirements of the mental health counseling curriculum, while also providing counsel on what they would like to pursue in their future careers as counselors. Dr. McGuinness also serves as the counseling department’s representative for the Office of Alcoholism and Substance Abuse Service’s Credentialed Alcoholism and Substance Abuse Counselor curriculum. He has additional training in holistic approaches that can be integrated into counseling, including clinical hypnosis and Reiki.


THE DATA SOURCES USED IN THIS REPORT The following is a general representation of the sources of data used for this report.

PLEASE NOTE….. As we share our data with you, we will use utmost precaution to protect the identity of any of our contributors. Likewise, we will not be reporting on any data that pertain to individual faculty or students—these are matters that are deliberated and resolved on an individual basis and in a manner that maintains confidentiality.


STRUCTURE OF THE REPORT

This report is divided into three sections. In the first section, we describe the assessment tools and activities, as outlined in the illustration above, which we use to collect our data, and follow each assessment by a report of the key findings for the year. In the second section we outline the program changes that we have already implemented in response to the feedback and other assessment data that we have gathered. In the third section we provide an accounting of those areas in need of attention that have been addressed or will become the focus of our remediation efforts in the coming months. Also, in the third section, we will highlight some of the basic principles that will guide us as we move forward. 1. Annual Admissions and Across Program Demographic Data The annual admissions data includes data gleaned from our master’s and doctoral level admissions form over four admissions cycles (July, October, February, and April). The admissions forms; which represent an integration of interview data, transcripts, professional references, a personal statement, and a writing sample; are completed by core faculty members who are involved in the interview process. The data that is collected on these forms reflects our efforts to maintain the highest standards in relation to CACREP-informed parameters regarding academic aptitude, adequacy of preparation for the program, disposition, and respect for cultural differences. These data are not published, but are used ideographically to compile a retrospective profile of student characteristics that give us insight into admissions profiles associated with both student success and challenge. We discovered, for example, that students most likely to struggle in the program were, not surprisingly, given among the lowest ratings relative to other admits. While being somewhat anecdotal, these data do impart some degree of validity on our admissions metrics. One aspect of admissions data that is particularly valuable in our comprehensive program evaluation is the demographic information that we are able to glean to assess the success we are having in attracting, matriculating and retaining a diverse student body. Comparing demographic data at multiple points, e.g., admissions acceptance, current student body, and graduation; gives us an understanding of our patterns in relation to diversity. The attention to demographics is a part of our conversion to 2016 CACREP Standards, and for the first time this year, for purposes of trend analysis, we will be presenting a compilation of three years of data. FINDINGS FROM DEMOGRAPHIC DATA As alluded to above, much of the data coming from admissions materials are suitable primarily for internal use. As we become better acquainted with students, if there are issues that arise (e.g., academic, dispositional), we are able to refer back to the admissions data and ascertain whether our admissions process needs to be amended. One aspect of the admissions process that we feel it is important to consider more


publicly involves demographic data, which gives us a sense of the degree of diversity we are attracting in our application process as well as the diversity among those who eventually matriculate and graduate. The demographic data set reported here for 2019-2020 provides the third in an annual series. We begin with the 2019-2020 demographics tables as a means to understanding, descriptively, our current across program race/ethnicity and gender profiles. For the sake of this report, “across program” data collection is defined as capturing data from 1) students admitted to our programs, 2) those current students who matriculated into a program, and 3) those who graduated. The following represents highlights from the demographic tables. Tables 1a-1c Admissions Demographics 2019-2020 reveal the following:

• The overall male to female gender ratio of our admits was 24% male to 76% female and reflects a slight increase in the percentage of males.

• The split in doctoral admits is reflective of the overall ratio at 29% male and 71% female and also indicates a slightly higher percentage of males.

• Master’s student admits, with a slightly higher percentage of males relative to the previous year, are split at 24% male and 76% female.

• Our admits for 2019-20, identified as 71% White, 6% Black, 10% International, 4% Hispanic, 2% Multiracial, and 2% Asian.

• During these admissions cycles, at the doctoral level no multiracial, Asian, or Hispanic, international or Black men were admitted.

Tables 2a-2c: Current Student Demographics 2019-2020 reveal the following:

• The overall male to female gender ratio of our current students is 23% male to 77% female, an increase for males of 1 percentage point from the previous year.

• Master’s students, currently matriculated, are split at 27% male and 73% female. • The gender split in current doctoral students reflects a lower percentage of

males than at the master’s level with a ratio at 17% male to 83% female. • Our currently matriculated students for 2019-2020 identified as 69% White, 8%

Black, 8% Hispanic, 4% Multiracial, 2% Asian, and 7% International. This reflects a 5% increase in White matriculated students and an increase in 1% in matriculated Black students.

• In general, the racial diversity among current doctoral student paralleled that of the master’s and doctoral combined racial/ethnic diversity figures. The most noteworthy change was an increase in 4% of Black doctoral students relative to the prior year


Tables 3a-3c: Graduating Student Demographics 2019-2020 reveal the following: • The overall male to female gender ratio of our graduates was 12% male to 88%

female. These figures underscore a drop of 8% in male graduates. • Master’s students were split at 9% male and 91% female, pointing to a decrease

of 13% among male graduates. • The split in doctoral graduates was 33% male versus 67%, N=3. While this

represents a substantial rise of 19% in relation to male doctoral graduates, the small N calls into question the overall value of the calculation.

• Our graduates for 2018-19 identified as 48% White (down from 57%), 17% Black (up from 8%), 8% International (down from 20%), 12% Hispanic (down from 13%), 4% Multiracial (same as the prior year), and 4% Asian (same as the prior year).

Demographics Table 4: Gender Across Program 2019-2020 This is the third summary gender distribution report, with the first being in 2017-2018. These figures reveal the following:

• Our admitted students and our current students reflect a greater gender balance than the gender balance in our graduating class (F/M 76%/24%, 77%/23%, and 89%/11% respectively).

Demographics Table 5: Race/Ethnicity Across Program 2019-2020 reveals the following:

• A larger percentage of students identifying as 2+ races matriculated and graduated (4% and 4% respectively) than the percentage actually admitted (2%).

• The number of students identifying as Asian indicates that the percentage admitted and matriculated (2% each respectively) has declined some in that the 2019-20 graduating class was 4% Asian.

• The Percentage of Black students was 6% at admission and 8% for both matriculated and graduating students. In the 2018-2019 Annual Report, the percentage of students identifying as Black at admission was 11%, indicating a drop in Black admits. The percentage of Black students who were matriculated and graduated in 2019-2020 was roughly comparable to the percentages in the previous year. These figures reveal that our strategies regarding recruitment of Black students needs to be reconsidered as there clearly is not a consistent level of recruitment over time.

• Latin(x) admits comprised 4% of the pool of admitted students, 8% of matriculated students, and 12% of graduates. These figures point to spike in matriculated students in 2017-2018, which has not been replicated since this point.

• International students comprised 10% of the admitted applicants in 2019-2020 (10%), while the current matriculated students are comprised of 7% and the graduation percentage was an all-time high of 20%. Future calculations are expected to vary somewhat dramatically in light of COVID and immigration


• Admission of White students was 71%, higher than the previous year when the White students represented 54% of the admits. The matriculation pool for 2019-2020 was 69% White, which is similar to the 64% White representation in the matriculation pool. In contrast, Whites comprised 49% of the graduation pool indicating that the level of heterogeneity has declined across a two-year period.

2. Individual Student Assessment of Key Performance Indicators As many individuals reading this report are aware, CACREP has a comprehensive set of standards that all programs seeking accreditation must implement. These standards, in addition to requiring that programs meet certain expectations concerning how they are structured internally and supported by the larger institutions in which they are embedded, also have highly specified standards that relate to the curriculum that shapes the coursework at the master’s and doctoral levels of our training program. The assessments in this section relate directly to the standards governing curriculum. The data related to this assessment point is collected throughout the academic year as part of individual class assessments. The particular nature of data that is collected is predicated on our collection of key performance indicators (KPI’s) that were derived from our Program Objectives. The KPI’s, which were derived from the Program Objectives, can be understood as key points that are targeted for assessment as students’ progress through their programs. These KPI’s are divided into 5 categories that correspond to data collection mandated by the 2016 CACREP Standards (the 2016 CACREP Standards can be accessed at www.cacrep.org): Entry-level Counseling Core, Clinical Mental Health Counseling, School Counseling, Doctoral Counselor Education and Supervision, and Administrative. For each of the KPI’s listed, a minimum of 2 independent and contrasting measures of the KPI are collected. The KPI’s are as follows:

KEY PERFORMANCE INDICATORS COUNSELING CORE 1) Professional Counseling Orientation KPI CC1: The student understands legal and ethical considerations specific to

professional counseling 2) Social and Cultural Diversity KPI CC2: Students understand how their own social and cultural diversity impacts their

views of other and self. 3) Human Growth and Development


KPI CC3a: Students have gained a knowledge of human development theories. KPI CC3b: Students have learned to apply theories of human development to better

understand counseling-related issues. 4) Career Development KPI CC4a: Students have knowledge of career development theories, particularly as the

theories pertain to their own personal development. KPI CC4b: Students will exhibit the ability to practice career counseling 5) Counseling and Helping Relationships KPI CC5: Students have developed their own personal approach to counseling, which

will be informed by knowledge of theory and best practices. 6) Group Counseling and Group Work KPI CC6a: Students understand the dynamics associated with group process and

development KPI CC6b: Students understand the characteristics and functions of effective group

leaders 7) Assessment and Testing KPI CC7: Students understand the responsibilities and obligations of test administrators 8) Research and Program Evaluation KPI CC8: Students understand the basic principles needed to successfully utilize

assessment data in the counseling process. CLINICAL MENTAL HEALTH COUNSELING KPI CMHC 1: The student demonstrates an understanding of how to conceptualize a

case through a bio-psychosocial lens. KPI CMHC 2: The student demonstrates the ability to construct a counseling

intervention plan that is appropriate for DSM disorders that have been identified in the assessment process.

KPI CMHC 3: Students have the applied theoretical knowledge to inform their

treatment of clients in an addictions setting.


KPI CMHC 4: Student demonstrates the procedures for assessing and managing suicide risk.

KPI CMHC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and

knowledge that suggest the ability to competently and ethically provide mental health counseling services to clients, commensurate with current level of training

SCHOOL COUNSELING KPI SC 1: The student understands legal and ethical considerations regarding the work of

the school counselor KPI SC 2: Students understand the counselor’s role as leaders and systems change

agents in P-12 schools. KPI SC 3: The student has knowledge of current methods of using data to inform

decision-making & accountability. KPI SC 4: Student demonstrates the procedures for assessing and managing suicide risk. KPI SC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and

knowledge that suggest the ability to competently and ethically provide school counseling services to clients, commensurate with current level of training.

DOCTORAL COUNSELOR EDUCATION AND SUPERVISION Counseling KPI DCES 1: Students are able to critically reflect upon approaches to counseling

including strengths and weaknesses, theoretical bases, the research base for existing counseling theory, and the range of methods for evaluating efficacy

Supervision KPI DCES 2: Students have gained understanding of clinical supervision theory and

ethics, and demonstrated effective supervision practice. Teaching KPI DCES 3: The student demonstrates proficiency in course design, delivery and

evaluation methods appropriate to course objectives. Research and Scholarship KPI DCES 4: The student demonstrates the skills needed in professional writing for

journal and newsletter publication


Leadership and Advocacy KPI DCES 5: The student understands models and applications of advocacy and

consultation. SCORING KPI DATA The KPI-related data is recorded by individual course instructors at the end of each semester. Rubrics exist for each of the minimum of two measures for each KPI, with each rubric detailing how a score of 1-4 is derived. The 1-4 scoring scale is used across all of the KPI measures and is interpreted as follows:

1= Insufficient 2=Emergent 3=Meets Expectations 4= Exceeds Expectations

As always, professors who rate a student below a 3 on any given standard are expected to provide feedback to the student on how the student can improve in this core area. Concerns around how individual students may be failing CACREP core standards are also discussed at regular faculty meetings and in depth at our annual student review. Overall trends in ratings, especially standards where scores are consistently rated below a 3, are also discussed at our faculty meeting in order to discern proper interventions for increasing student proficiencies in those areas. FINDINGS FROM KPI MEASURES The following are the outcomes, based on a minimum of 2 contrasting measures, at two distinct points in time, of all of the KPI’s listed above. The data presented includes scores generated for Summer 2019, Fall 2019, and Spring 2020. Click here for Table 5, KPI Outcomes Some General Thoughts About Key Performance Indicator Measures In interpreting the results of our KPI measures, there are several considerations that should be applied. First, the measures that have been devised reflect a diversity of knowledge and skill-based assessments. While diversity in measures yields a more comprehensive understanding of student progress, it is notoriously difficult to ensure score reliability across measures. Thus, it is with caution that we make claims concerning overall score improvement. Also adding some uncertainty to the consistency of rating across measures is the problem of interrater reliability. The interrater conundrum is particularly problematic because the large number of raters relative to the population of students being assessed. In devising this KPI assessment system we


attempted as much as possible to create measures that can be completed by Core Counseling Faculty and closely supervised counseling doctoral students as opposed to faculty outside of our program. With those caveats in mind, we report on the outcomes of our KPI measures. An Important Caveat Regarding the 2019-2020 Analysis

The academic year 2019-2020 is the first in which all KPI-related data was captured, analyzed, and reported using the Taskstream platform. Unfortunately, the reporting options offered by Taskstream do not allow us to duplicate the same kinds of reporting formats we have used in the past. Prior to this year, all of the KPI data were reported showing the distribution, for any given measure, of student average rubric scores. Taskstream allows us to calculate the average score for the collective student group being assessed in relation to that measure. In more concrete terms, this means that unlike past years when we have reported, for any given measure, the precise number of students performing, on a 4-point scale, in each of the ranges of 3.5-4, 2.25-3.49, 1.5-2.24, and 1-1.49; this was not possible for this report. In an attempt to maintain some uniformity, we will, continue to use these four ranges to characterize the overall performance of the cohort on a given measure as follows:

1-1.49 Insufficient 1.5-2.24 Emergent 2.25-3.49 Meets Expectations 3.5-4.0 Exceeds Expectations

CORE COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES The results of the Core Counseling KPI measures yielded the following insights: 1) Professional Counseling Orientation KPI CC1: The student understands legal and ethical considerations specific to professional counseling Mental health and School Counseling were assessed using contrasting measures

from specialization-specific classes. Mental Health students scored in the Exceeds Expectations range for both Measures 1 (3.76) and 2 (4.00). This is consistent with the high-performance level from the previous year. Also consistent with the previous year is that School Counseling students scored slightly lower on Measure 1 than Mental Health Students, in this case Measure 1 = 3.35, Meets Expectations, but scored in the Exceeds Expectation range for Measure 2.

These outcomes seem to point to the notion that School Counseling students began with less of an understanding of legal and ethical issues. This possibility would align with feedback that we have gotten through other assessments we have done where School Counseling students have repeatedly underscored that much of the material in Core Counseling classes uses exemplars that are more


applicable to Mental Health students, hence leaving School Counseling students with the extra burden of not getting enough exposure to real-world applications of principles. Importantly, for Measure 2, all School Counseling students scored Exceeds Expectations, which may reflect some of the measures we have taken (discussed later in this report) to accommodate requests made by School Counseling students to have School Counseling exemplars play a more central role in our pre-clinical curriculum.

2) Social and Cultural Diversity KPI CC2: Students understand how their own social and cultural diversity impacts their

views of other and self. The outcomes of Measures 1 and 2 indicate a slight decline in student performance over time, where students average score Exceeds Expectations in Measure 1 (3.59) and Meets Expectations in Measure 2 (3.43). Since we have now seen this decline more than once, we would suspect that this could be related to the nature of the two measures rather than being an actual reflection of a decline in performance. Hence, we will be considering possible modifications of these measures in upcoming assessments.

3) Human Growth and Development KPI CC3a: Students have gained a knowledge of human development theories. The outcomes for Measures 1 and 2 are identical for this KPI (3.79), and in both

cases students scored in a range that Exceeds Expectations. KPI CC3b: Students have learned to apply theories of human development to better understand counseling-related issues.

The outcomes for Measures 1 and 2 reflected an increase in average over time of 0.16 points. It is worth noting that for both measure students scored in a range that Exceeds Expectations (3.71 and 3.87 respectively).

4) Career Development KPI CC4a: Students have knowledge of career development theories, particularly as the theories pertain to their own personal development.

For Measure 1 of CC4a, the average score was in the Exceeds Expectations range (3.73). Measure 2 (3.93), which was 0.22 points higher than Measure 1, also scored in the Exceeds Expectations range. While the scores from 2018-2019 were pulled downward by one student scoring in the lower range, in general, the scores for this KPI reflect a high degree of competency.

KPI CC4b: Students will exhibit the ability to practice career counseling

For this KPI, students scored in the Exceeds Expectations range on both Measures 1 and 2 (3.83 and 3.70 respectively). These results point to two


conclusions. First, students overwhelmingly met or exceeded expectations related to the ability to practice career counseling. Second, the reliability between the two measures is questionable. The same discrepancy existed last year and our conclusion is that the slight downward trend is primarily related to the complexity of the Measure 2 assignment.

5) Counseling and Helping Relationships KPI CC5: Students have developed their own personal approach to counseling, which will be informed by knowledge of theory and best practices.

The KPI measures in Counseling and Helping Relationships remained fairly steady from Measure 1 to Measure 2, with the outcomes for Measure 1 being 3.59 and Measure 2 being 3.67. In both cases, students operated at an Exceeds Expectations level. Cross-referencing these outcomes with student class evaluations and our Graduating Student Survey, it is clear that students find this challenge to be personally rewarding.

6) Group Counseling and Group Work KPI CC6a: Students understand the dynamics associated with group process and development

For Measure 1 (3.54) and Measure 2 (3.93) of this KPI, students scored in the Exceeds Expectations range. These numbers indicate that students have a sound understanding of group dynamics, which points to a solid foundation for approaching more complex group work.

KPI CC6b: Students understand the characteristics and functions of effective group leaders

Measure 1 for this KPI indicated that on average students scored in the Exceeds Expectations range (3.54). Measure 2, which is also in the Exceeds Expectations range (3.93), reflects an increase in the average score on this measure and an indication that student understanding of group work improved over time. In the two previous annual reports, we commented on a concern regarding the outcomes for these measures as they indicated that there may be some gaps in learning. The scores for this year, however, point to the notion that this potential problem has been remedied as the groups course has now adequately reinforced the elements of the course reflected in this KPI.

7) Assessment and Testing KPI CC7: Students understand the responsibilities and obligations of test administrators.

As was the case in the last two years, Assessment and Testing has some of our highest scores. For both Measure 1 and Measure 2, 100% of students scored in the Exceeds Expectations range. In cross-referencing with teaching evaluations, students have repeatedly expressed their appreciation for the instructor in this class, in spite of the fact that the content is often not embraced by them. The


design of this class is one that has been quite dynamic in its response to student feedback, and much effort has gone into ensuring that students understand the relevance of this course to clinical practice and to having considerable experiential learning opportunity.

8) Research and Program Evaluation KPI CC8: Students understand the basic principles needed to successfully utilize assessment data in the counseling process.

The outcomes for Measures 1 & 2 in Research and Program Evaluation were 3.87 and 3.70 respectively, both in the Exceeds Expectations range. Again, the instructor responsible for Measures 1 and 2, who is the same instructor assessing KPI CC7, has gone to great lengths to bring relevance to the material and has succeeded in creating an atmosphere in which initially reluctant students clearly assimilate the material.

CLINICAL MENTAL HEALTH COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES 1) KPI CMHC 1: The student demonstrates an understanding of how to conceptualize a case through a bio-psychosocial lens.

Within the Mental Health Counseling program, ensuring that students have a biopsychosocial vision of mental health challenges is among our top priorities. We have found over time that this is often a difficult concept for students to grasp and to subsequently implement in their case conceptualization. We have thus gone to great lengths to include biopsychosocial case conceptualization in many aspects of the curriculum, and in particular, have given it a central role, both in terms of knowledge and skill, in our course, EDE 449 Pre-Practicum. In addition, considerable attention is given to implementation in our two-semester Problem Identification and Intervention in Counseling sequence. It is our hope that the increased attention to the biopsychosocial model in Pre-Practicum will work synergistically with the Problem Identification and Intervention in Counseling course objectives. For the 2019-2020 assessment period, Measure 1, at 2.87, was in the mid-range of Meets Expectations, and was the lowest average score for any of our assessments. While this was somewhat concerning in light of our success in other KPI measures, Measure 2 (3.94) allayed our concern that students were able to assimilate this concept.

2) KPI CMHC 2: The student demonstrates the ability to construct a counseling intervention plan that is appropriate for DSM disorders that have been identified in the assessment process.

The outcomes for this KPI are spread across two semesters. For Measure 1, the average score Exceeds Expectations at 3.75, and Measure 2, shows a 0.18-point increase in the average score at 3.93. These outcomes are a departure from the


previous year in which there was a downward trend between Measure 1 and Measure 2. Hopefully, the previous year was an aberration in spite of the fact that the emphasis on DSM knowledge is greater in the Measure 2 course and testing in this course undoubtedly has a greater level of diagnosis and treatment planning skill that is required, hence making a slight downward turn understandable.

KPI CMHC 3: Students have the applied theoretical knowledge to inform their treatment of clients in an addictions setting.

The outcomes for these two KPI measures indicate that students successfully built knowledge as they transitioned from Measure 1 to Measure 2. For Measure 1, the average score of 3.27 was in the Meets Expectations range and Measure 2, with an average score of 3.71, indicated that on average students scored in the Exceeds Expectations range. The Mental Health Counseling Program recently had a complete revamping of our Addictions course in preparation for our application to become providers of a curriculum leading to a New York State level credential in addictions counseling (Credentialed Alcoholism and Substance Abuse Counselor). It is our hope that the increased attention to addictions will contribute to a high level of performance on addictions-related assessments.

KPI CMHC 4: Student demonstrates the procedures for assessing and managing suicide risk.

The Measure 1 and Measure 2 scores for this KPI showed substantial improvement relative to the two previous years. While the 2017-2018 outcomes for KPI CMHC 4 raised some level of concern because of a downward drift regarding assessing and managing suicide risk, the outcomes for 2018-2019 showed some improvement in the back-sliding between Measures 1 and 2. However, both measures in 2019-2020 indicated that 100% of students scored in the range of Exceeds Expectations.

KPI CMHC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and knowledge that suggest the ability to competently and ethically provide mental health counseling services to clients, commensurate with current level of training.

The outcomes on the three measures of this KPI showed steady improvement throughout the program. Measure 1 showed that on average students scored in the Meets Expectations range (3.10), while Measures 2 and 3 showed scores in the range of Exceeds Expectations (3.57 and 3.65 respectively). It should be noted that Measure 3 was a skill-based measurement occurring at the end of the two-semester Internship experience, thus indicating that student performance was highest just prior to graduation.


SCHOOL COUNSELING KEY PERFORMANCE INDICATOR OUTCOMES

KPI SC 1: The student understands legal and ethical considerations regarding the work of the school counselor.

This KPI showed a decline of 0.26 points between Measure 1 and Measure 2 (3.40 and 3.14 respectively). It is noteworthy that Measure 1 is knowledge-based while Measure 2 is skill-based. In any case, this is a metric that warrants faculty discussion around possible disparities between the knowledge and skill domains. It is also discussion-worthy that while acceptable, these scores are generally lower than what we see for most of our outcome measures.

KPI SC 2: Students understand the counselor’s role as leaders and systems change agents in P-12 schools.

For this measure, the average score was 0.49 points higher for Measure 1 (3.89, Exceeds Expectations) when compared to Measure 2 (3.4 Meets Expectations). As these are knowledge-based assessments, this disparity may be due to interrater reliability and a more rigorous grading scheme for the Measure 2 assignment.

KPI SC 3: The student has knowledge of current methods of using data to inform decision-making & accountability.

This KPI showed a decline of 0.17 points between Measure 1 and Measure 2. It is noteworthy that Measure 1 is knowledge-based while Measure 2 is skill-based. In any case, this is a metric that warrants faculty discussion around possible disparities between the knowledge and skill domains.

KPI SC 4: Student demonstrates the procedures for assessing and managing suicide risk.

The outcomes for this KPI were quite encouraging. Both Measure 1 and Measure 2 indicate that all students scored a 4.0 on the assessment rubric. These scores are an improvement over scores from the two prior years.

KPI SC 5: The student demonstrates the attitudes, behaviors, dispositions, skills and knowledge that suggest the ability to competently and ethically provide school counseling services to clients, commensurate with current level of training.

This KPI was measured three times over the course of the School Counseling curriculum, including during both semesters of Internship. While the 2018-2019 Measures 1, 2, and 3 indicated that 100% of students exceeded expectations, the 2019-2020 Measures 1 (3.35), 2 (3.51), and 3 (3.53) indicated a steady increase in performance, which for Measures 2 and 3 were in the Exceeds Expectations range.


DOCTORAL KEY PERFORMANCE INDICATOR OUTCOMES Counseling KPI DCES 1: Students are able to critically reflect upon approaches to counseling including strengths and weaknesses, theoretical bases, the research base for existing counseling theory, and the range of methods for evaluating efficacy.

Outcomes for Measure 1 and Measure 2 indicate that on average students scored in the Meets Expectations range (3.31and 3.42 respectively), and while acceptable, do represent some of our lower scores.

Supervision KPI DCES 2: Students have gained understanding of clinical supervision theory and ethics, and demonstrated effective supervision practice.

The transition from Measure 1 (3.70) to Measure 2 (3.60) showed a slight decrease of 0.10. Both measures had an average score in the Exceeds Expectations range. These outcomes, as in past years, indicate that our students are gaining the knowledge and skill required for clinical supervision practice.

Teaching KPI DCES 3: The student demonstrates proficiency in course design, delivery and evaluation methods appropriate to course objectives.

The course used for these KPI measures is taught in odd Fall Semesters only, hence we present the scores for 2018-2019: The outcomes for DCES 3 were strong for both Measure 1 and Measure 2.

For Measure 1, 70% of students exceeded expectations while 30% met expectations, and for Measure 2, 50% exceeded expectations and 50% met expectations. The slight disparity in the outcomes for Measures 1 and 2

equates to a difference of 2 students going from a score of 4 to a score of 3 and is likely related to the nature of the measure rather than a backsliding from the Measure 1 assessment.

Research and Scholarship KPI DCES 4: The student demonstrates the skills needed in professional writing for journal and newsletter publication.

For Measure 1 of this KPI, students, on average, scored 3.61, which places them in the Exceeds Expectations range. Measure 2, which is tied to the literature review portion of the student’s comprehensive examination performance, showed that 100% of students scored a 4, Exceeds Expectations. In past years we had witnessed backsliding on Measure two, thus making this dilemma the topic of many of our Core Counseling Faculty meetings. The scores this year are hopefully the result of those discussions, which have led to our increased emphasis on literature review writing in our classes and in having more of our


students take our doctoral-level course in professional writing. Leadership and Advocacy KPI DCES 5: The student understands models and applications of advocacy and consultation. While Measures 1 and 2 for this KPI were only separated by 0.10, it was noteworthy that Measure 1 (3.50, Exceeds Expectations) was higher than Measure 2 (3.40, Meets Expectations). This is likely due to the nature of the Measure 1 and 2 assignments rather than a reflection of a small decline in competency. This is the first time that this competency is being assessed since the course instructor role was filled by a core counseling faculty member. It is our hope that the counseling applications offered by the new instructor will help students to understand the relevancy of the course material to counseling practice. 3. Annual Clinical Formative and Summative Assessments A large bank of formative and summative assessments is part of the annual data collection process related to entry-level and doctoral clinical requirements, i.e., practicum and internship experiences. The data from these assessments supplement KPI-related data and is used to guide individual remediation needs of students in their clinical practicum and internship settings. Occasionally, the data gleaned from these assessments reveal patterns that allow us to consider more general needs for program modification. These data are not presented in aggregate form as the conditions around the evaluations are also unique to each student experience and do not lend themselves to cross-student reliability. When data emerge, through faculty discussions related to these assessments, findings are presented in this report. This year, no specific finding emerged among these assessments that are not reflected in data reported elsewhere in this document. 4. Student Annual Milestone Assessments ENTRY-LEVEL Students in our master’s programs have one milestone assessment, which is an evaluation of our capstone process. The assessment rubrics for this project provide a medium for assessing the final task in the entry-level program. In this capacity, these rubrics document the students’ technical and conceptual writing skills and their ability to connect their topic of study to the larger counseling literature. Due to the varying nature of capstone projects, these assessments are best understood in relation to individual student performance as opposed to aggregate consideration. FINDINGS FROM ENTRY-LEVEL MILESTONE ASSESSMENTS


Although we have experienced considerable success in addressing some of the challenges of the master’s level capstone project, some challenges remain. Based on data from the milestone assessment and from discussion that has been generated in relation to this assessment, it is clear that some students are not developmentally ready to successfully execute an original, positioned literature review (one of several options for completing the capstone requirement). Attempts to scaffold this experience have proved helpful in recent years, but the capstone remains a focus of many of our discussions and object of feedback from students. We will revisit this feedback and possible action steps later in this report. DOCTORAL The doctoral student annual milestone assessments include four components: 1) a portfolio assessment is collected concerning the student’s development in relation to writing mechanics, concept development, argumentation, and progress being made toward developing their individual research trajectory; 2) comprehensive exam assessment to determine the student’s readiness to enter the dissertation phase of their program. 3) a proposal assessment to measure the student’s success in forming a clearly articulated research question, identifying a suitable method, and generating a literature review that exposes a suitable gap in the literature for dissertation study and 4) a dissertation assessment to record the quality of the student’s writing, the success of their analytic plan, and their ability to clearly articulate a contribution to the existing literature in the field. FINDINGS FROM DOCTORAL MILESTONE ASSESSMENTS In any given year, the outcome of any one of the doctoral milestones, involves a very small number of doctoral students making presentation of results in this document traceable to individual students. For this reason, we refrain from disseminating results of numeric data from any given year. Nonetheless, the outcomes of these measures are an important source of data for faculty. We report more generally that the outcomes from our doctoral milestone assessments, based both on the measures designed to assess the above competencies, and the discussions that have occurred around these measures, suggest that as a program, the steps that we have taken to address issues related to writing mechanics, concept development, argumentation, developing a literature review and executing an analytic plan, while improving steadily, still require more systematic implementation of interventions aimed at bolstering these skills. We have, in the past, taken numerous steps toward improving the outcomes of these competencies and have met some success in doing so. However, it is clear that these interventions still need to be retooled and applied in a more systematic way. Further discussion of these outcomes, particularly in relationship to our implementation of remediation efforts, are discussed in the latter sections of this report. 5. Annual Student Reviews


Data is formally collected at the end of every spring semester and is used to assess the progress of all master’s-level and doctoral-level students. The data collected attempts to give an overview of academic progress, clinical skill development, and emergent dispositional issues. FINDINGS FROM ANNUAL STUDENT REVIEWS While we rely heavily on our KPI measures to create a detailed assessment of students’ curricular competency, the annual student reviews, as mentioned above, look at general academic trends, clinical performance, disposition and, particularly in the case of doctoral students, success in building an identity as a scholar in the field of counseling. These data are primarily used to craft individual remediation plans for students but are also used by the counseling faculty, in our discussion of students, to consider patterns that sometimes emerge from these assessments that are relevant to program modification. ENTRY LEVEL At the entry-level, we have not had major collective issues emerge from this assessment. That being said, this assessment has played a major role in bringing individual student dispositional issues to the fore, as well as clinical difficulties, and patterns relating to academic struggle. The discussions that emerge from this assessment process have been the basis for creating remediation plans, and, in more difficult cases, assessing the advisability of having a student continue in the program. Because of the degree to which the faculty regularly meet to discuss student academic and dispositional issues, it is actually unusual for our annual reviews to reveal issues that were not already known to us. DOCTORAL LEVEL In contrast, at the doctoral level, we have had emergent issues at the group level. The most pressing, ongoing issue to arise (i.e., the issue was also discussed in the 2017-2018, and 2018-2019 Annual Reports), relates to students’ professional writing success in relation to comprehensive exams, proposal writing, dissertation writing and publication in professional venues. While many of our students are performing quite well in this arena (our current self-study documents student participation in publishing opportunities), we have grappled with mechanisms for fostering development of doctoral students who have not had great success in this realm. Two years ago, we reported that most of our discussions in this regard related to some of the steps that we have taken including increasing scaffolding around required class literature reviews and intensifying evaluation efforts related to our milestone assessments. Our thoughts in the 2018-2019 report, similar to the previous year, were focused on adding our doctoral-level professional writing course as a requirement for students as one course of action, and secondly to become more stringent about what is expected of students at our Portfolio milestone. In relation to the latter, doctoral students in the earlier part of


their program are required to submit, among other items, a writing sample of what that they consider to be their best work. In the past, we had accepted co-authored writing samples and class assignments that did not entail a literature review. We then became more stringent about only accepting single-authored literature reviews and raising the bar on what is considered to be work that meets standards. We can report at this juncture that in our most recent round of 2012020 Portfolios we discovered that this more stringent approach to Portfolio may itself have developmental significance in that we encountered fewer writing issues. While we have experienced some resolution to this early-program writing proficiency, our year-end reviews still suggest that our students’ writing proficiency needs to be further bolstered for them to be able to meaningfully participate in publication opportunities Although we avoid specific information regarding dispositional issues for fear of violating student privacy/anonymity, we can say more generally that our dispositional assessment of doctoral students has been fruitful and has fostered constructive mid-course corrections. More generally, these annual doctoral student reviews form the basis of a systematic, formal feedback that often includes detailed requirements related to milestone completion or attention to incompletes. We continue to experience bottlenecking in the area of comprehensive examinations despite stricter limits being put into place regarding time limits. Discussions regarding changes in the comprehensive exam process are ongoing but to date, no acceptable alternative has emerged. Other discussions regarding establishment of cohort experiences for the comprehensive examination/dissertation proposal/dissertation process are on-going.

Stakeholder Assessments 6. Adjunct and Affiliated Faculty Surveys Adjunct faculty are surveyed to ensure that they are feeling personally supported in their faculty role, integrated into the life of the program, and are receiving adequate resources relative to their job expectations. Data is collected biannually. OUTCOMES FOR ADJUNCT AND AFFILIATED FACULTY Our continuous systematic program evaluation of the counseling program strives to maintain professional and supportive contact with our adjuncts (local clinical professionals with particular expertise) and affiliated faculty (advanced doctoral students whose development has been fostered to a level beyond Teaching Assistants/University Clinical Supervisors). We take both formal and informal occasions to garner their input and opinions concerning the ways we can support their teaching and professional growth. Due to the busy lives of our adjunct and affiliated faculty (AAF), we attempt to gather their input through on-line surveys.


We currently conduct biannual evaluation of our AAF. Our 2016-18 survey, as noted in the previous annual report suggested room for improvement. Most notable among these were the average score of 2.33 in response to the question: How would you rate the feeling of inclusion into the life and work of the department (were you given the opportunity to take an active part in the department’s decision-making process)? and the average score of 2.83 in response to the question: How would you rate the support you received from the department in preparing you to teach (examples of syllabus, provide you with copy of textbook, etc.)? Our most recent 2018-2020 survey was taken after the department adopted a more stringent mentorship policy for all of our AAF. Each core faculty member is given the task of overseeing adjunct teaching within specifically assigned program courses. Also, our adjuncts now receive an extensive orientation to technology and other Warner processes. That being said, the scores in the most recent survey were appreciably higher than in the 2016-18. More specifically, the average of all questions, on a scale of 1-5, was 3.67 and was up from 3.12 in the previous survey. The previous score of 2.33 in response to the question: How would you rate the feeling of inclusion into the life and work of the department (were you given the opportunity to take an active part in the department’s decision-making process)? was increased to 3.2, and the question How would you rate the support you received from the department in preparing you to teach (examples of syllabus, provide you with copy of textbook, etc.)? went from a 2.83 to 3.70. 7. Biannual Employer Surveys Each year, until this year, we have sent an electronic survey to individuals from whom our graduates have secured employment. These data give us a sense of how our graduates are functioning as counseling professionals, and more specifically, the degree to which employers perceive that former students have mastered various competencies valued by our program and reflected in CACREP Standards. Because the number of returned surveys has been in low single digits, we decided to begin conducting this survey biannually. Thus, the survey included in this report were administered to employers identified by alumni and was disseminated during the Spring 2019 semester. All items in the assessment were ranked on a scale of 1 through 4, where 1= Does not meet standards, 2= Emergent, 3= Meets Expectations, and 4= Exceeds expectations. FINDINGS FROM SPRING 2019 EMPLOYER SURVEYS The number of respondents regarding this survey is ongoingly quite low. A total of 3 employers completed surveys about the quality of our alumni as professional counselors (this was up from 2 respondents the previous year). On average, employers rated our MS graduates 3.38 out 4 (see table 9), which was up from the previous year when the average score was 2.9 out of 4. As was the case with our 2017-18 survey, we received no responses for our doctoral student employer survey for the 2018-2019. At the master’s level, no major concerns emerged. Our major concern from the previous survey was the average score of 2.50 regarding the students’ ability to integrate


biological, psychological and social interventions in their work with clients. In this current survey, that score increased to 3.33. A second concern in the previous survey related to students’ “commitment to serving as an agent of organizational change (social justice) or client empowerment”, which received a score of 2.5. In this most recent survey, the score rose to 3.3. In this year’s survey, none of the scores dropped below 3.0. We are continuing to monitor previous areas of concern. 8. Annual Site Supervisor Surveys Data that will be collected: Each year, we send an electronic annual Site Supervisor Survey to individuals serving in the field as our site supervisors. These data give us a sense of how our students are functioning as counselor trainees in the field, and more specifically, the degree to which supervisors perceive that our students are demonstrating a developmentally appropriate level of various competencies valued by our program and reflected in CACREP Standards. FINDINGS OF ANNUAL SITE SUPERVISOR SURVEYS The data presented here come from two surveys, one sent to master’s-level site supervisors (see table 7) and the other sent to doctoral site supervisors (see table 8). These surveys were administered to site supervisors engaged in supervision during the Fall 2019 and Spring 2020 semesters. All items in the assessment were ranked on a scale of 1 through 4, where 1= Does not meet standards, 2= Emergent, 3= Meets Expectations, and 4= Exceeds expectations. Entry Level We received 13 completed surveys from site supervisors ranking the preparation and performance of our master’s student interns. Site supervisors, on average, rated our MS students as 3.38 out of 4, which is up from 3.22 the previous year. The outcome of greatest concern in the 2017-2018 report was a response to the question assessing students’ “use of research, ethics and critical thinking to guide best practice”, where students on average scored 2.96. In the 2018-19 survey, this score increased to 3.19, and in the present survey it increased again to 3.38. This year, for the first time none of the survey outcomes had scores below 3.0. Strengths that emerged from the survey were that our students are “caring, compassionate and self-reflective people” (average score = 3.69) and that they are competent in using technology (average score = 3.54). Other areas where our students averaged 3.25 or greater were in their openness to innovations in counseling that address diversity, their commitment to serving as agents of change and client empowerment, their sensitivity to diversity and social justice dimensions of counseling in their work with clients, and relative to the student's level of development, how supervisors would characterize our students as counseling practitioners. The lowest scores, both at 3.15, were in relation to biopsychosocial conceptualization and ecological conceptualization. These lower scores corroborate


with our KPI measures related to biopsychosocial case analysis, which has been challenging for some of our students to fully grasp. Doctoral Students Only one individual responded to the doctoral site supervisor survey. It should be noted that the number of doctoral students who participated in practicum or internship during the survey period was exceptionally low, making survey work challenging. Nonetheless, the average of all of the ratings from this supervisor was 3.86, with no ratings below a 3.0. The modal score, 4.0, was achieved in 12 out of 14 questions. No areas of concern emerged from this survey, and one area, which had raised concern in a previous survey was rated as a 4.0; namely, the ability of our students to “function as leaders and advocates within the organization and profession”. It is clear that some other means of assessment may have more utility if the number of doctoral students involved in clinical practice during a given assessment period is excessively low. 9. Annual Alumni Survey Data that will be collected Our Alumni Survey is a lengthy instrument that includes the following multiple domains: 1) general satisfaction with the Warner School in relation to faculty support, curriculum, and negotiating administrative tasks (e.g., registration, bursar); 2) assistance regarding employment and licensure, 3) perceived proficiency regarding skills reflected in our program objectives and issues related to disposition, and 4) demographic data. FINDINGS FROM THE ALUMNI SURVEY Each year we send a survey to our alumni to query their employment status and their perspectives about how well the program prepared them for the work of a professional counselor. Much of these data about licensure and employment status are included in our yearly vital statistics survey, which is posted annually in early in the Fall semester. Therefore, we refer readers to our summary report of our vital statistics for information about job placements and licensure pass rates (click here for a link to the most recent summary data from our vital statistics survey). The Alumni Survey, Table 6, asks about 26 items related to respondents’ experiences and the degree to which they are satisfied with their training in a number of CACREP standard-related areas. On a scale of 1-4, alumni had an average satisfaction rating of 3.93 in the area of relationship and communication skills, 3.53 for assessment and appraisal skills, 3.55 in the area of intervention skills, 3.54 on contextual competencies, 3.43 on technological skills and 3.93 on dispositions. A number of individual items emerged below a score of 3.5, although they were at a level that meets expectations; namely, completing a treatment plan based on a DSM diagnosis, mental status exam, and biopsychosocial clinical intake interview (score = 3.14), completing a mental status examination (score = 3.43), assessing the needs of a client in crisis (score = 3.43), monitoring and documenting the progress being made in the implementation of a

https://www.warner.rochester.edu/about/accreditation

https://www.warner.rochester.edu/about/accreditation


treatment plan (score = 3.29), developing sociocultural self-knowledge, awareness of social and political barriers to counseling, knowledge of liberatory counseling theory, and ability to implement culturally congruent counseling practices with diverse populations (score = 3.29), and developing competency with technology, including word processing, email, library databases, and locating counseling- related resources on the Internet (score = 3.43). It is noteworthy that just two years ago, 7 areas for improvement emerged, identified by a score of less than 3.0 (range 2.44 to 2.89), including issues related to:

• Completing a treatment plan based on a DSM diagnosis, mental status examination and biopsychosocial clinical intake interview

• Monitoring and documenting the progress being made in the implementation of a treatment plan

• Assessing the needs of a client in crisis • Counseling process evaluation • Providing session structure and focus through methods for furthering counseling

goals. • Using flexibility and ability to generate novel responses to unexpected or

unfamiliar counseling situations • Multicultural, consultation and advocacy skills

Of these 7 areas, none were rated in 2019-2020 as areas for improvement. We would speculate that for many of these items, changes in curriculum; namely, in our Pre-practicum in Community Mental Health Counseling, and Crisis Counseling and Disaster Mental Health courses.

Student Perception Assessments

10. Individual Class Instructor Evaluations The Instructor Evaluation Form assesses the student experience in individual classes, particularly in terms of the skill of the instructor in creating a course delivery that is engaging, organized, relevant, and has fair and clear grading policies. Students are also afforded a space to freely comment about any aspect of their experience in the course, related to content, delivery, and any salient points regarding the instructor’s professionalism and ability to relate to students. FINDINGS RELATED TO INDIVIDUAL COURSE EVALUATIONS See the following link for a compilation of the individual class instructor evaluations for the academic year, 2019-2020: (click here to see Table 1, Summary of CACREP Course Evaluations). We are proud to report that students continue to rate our counseling classes very highly. For the Summer 2019-Spring 2020 academic year, students in our program, on average, rated statements about the courses and instructors as 4.45 on a 5-


point scale. This score is up slightly from the score from the previous year (4.40), and is generally consistent with the high ratings from the last three years. As would be expected with the increase in average, the large majority of scores on individual items were also up from the prior year. The only score that declined from the previous year showed a mere 0.04 point drop and was in response to the statement “Students in this class were free to disagree and ask questions. These quantitative findings corroborate the evidence our faculty have garnered from their conversations with students about their classroom experiences. Given the high teaching ratings, we plan to continue with much of our current approach to teaching, but will also consider new ways to improve delivery of instruction that are discussed later in this document. 11. Assessment of Clinical Experience and Clinical Site 12. Assessment of Clinical Site Supervisor Several forms are utilized to give students an opportunity to offer an assessment of their clinical site, clinical experience, and the site supervisor. An Interim Assessment Form gives students a chance to offer feedback regarding experiences at the clinical site midway through each semester in which they have a clinical field placement, while end-of the-semester data is collected using a Site and Site Supervisor Evaluation Form. FINDINGS FOR SITE, SITE SUPERVISOR, AND TAPE SUPERVISOR EVALUATIONS (click here to see Table 2, Summary of site and site supervisor evaluations). These ratings overall, were very positive with an average score of 3.63 out of 4.0, down 0.09 from the previous year. Our survey from the last two years signaled that we should consider how students can get more access to group work and to assessment-related experiences. This year, although these scores were quite positive (3.11 and 3.14 out of 4 respectively), they were slightly lower relative to 2018-2019 and remained the lowest scores on the survey. These outcomes suggest that more attention to these two areas is warranted. 13. University Tape Supervisor Assessment In light of the fact that we do not have an onsite clinic, we provide university-based individual supervisors for all of our entry-level students. The Student Counselor Evaluation of University Supervisor Assessment has a dual purpose, the first being providing both quantitative and qualitative data to the course instructor concerning the performance of a particular university-based supervisor, and the second being a mechanism for having our university-based supervisors, who are almost exclusively current doctoral students, important feedback on their supervision skill development. The tape supervisor assessment is based on a 4-point rating scale, with 4 being the highest.


FINDINGS FROM UNIVERSITY TAPE SUPERVISOR SURVEY A total of 37 respondents rated tape supervisors quite highly at 3.82 out of 4, which is consistent with the findings from the previous year (Click here to see table 4, Summary of Tape Supervisor Evaluations). No areas for immediate improvement emerged from the findings. We note that each of the three supervisor evaluations also includes qualitative data from students, which is also carefully evaluated by program faculty. In order to protect student and supervisor confidentiality, we do not post the qualitative data. Overall, we are pleased with the high level of supervisee satisfaction with all their supervisors and will continue to implement the same process for training and supporting our supervisors. We will also continue to monitor our supervisors with the intent of fine tuning our current system. 14. Small Group Supervision Assessment All counseling students who are enrolled in practicum or internship, at the entry or doctoral-level, receive small group supervision. Thus, at the end of each semester of clinical experience, students are asked to complete a Small Group Supervision Evaluation that provides feedback regarding the student’s experience of processing field-related issues in a small group setting while it also provides feedback for the individual supervisor running the group. The small group supervisor assessment is based on a 4-point rating scale, with 4 being the highest. FINDINGS FROM SMALL GROUP SUPERVISION ASSESSMENT Small group supervisors were rated, on average, 3.73 out of 4 (click here to see Table 3, Summary of Small Group Supervisor Evaluations), and no areas for immediate improvement emerged from the findings. Students uniformly found their supervisors to be supportive, responsive, knowledgeable, caring and able to stimulate critical reflection. Overall, we are pleased with the high level of supervisee satisfaction with all their supervisors and will continue to implement the same process for training and supporting our supervisors. We will also continue to monitor our sites with the intent of ensuring that our placements remain high quality and that students are matched with placements that best suit their individual interests and plans for future employment. 15. Graduating Student Survey Data that will be collected


The data collected from all graduating entry-level students is contained in the Graduating Student Survey. This survey asks students to share what they considered to be the strengths and weaknesses of their respective programs, suggestions going forward, and a sense of short-term career goals. FINDINGS FROM GRADUATING STUDENT SURVEY The Graduating Student Survey is comprised primarily of qualitative responses to a set of structured questions. What follows is a thematic analysis of the responses to each of the questions. Learning in Classes In response to the questions: “How would you summarize your learning through coursework in your program at Warner? and What were the highlights and what would you like to see improve?” the following data were generated:

• The faculty is so knowledgeable and I enjoyed the Courses very much. I especially valued Group Therapy class and Multicultural.

• The highlights of the work at Warner were the interactive assignments that allowed us to put into practice what we were learning. I really enjoyed doing peer-counseling and other forms of interactive practice that helped us prepare for fieldwork.

• I very (sic) loved my program at the Warner School. I like that we were able to

practice during our first year of the program. I enjoyed almost all of my classes and professors.

• I appreciate the attention and help from my academic advisors that supported me and provided me with advice and guidance during difficult times.

• I believe that the highlights of the program are that it feels like learning and communication is a two-way street, where everyone is willing to listen and help you.

• I would say that the general coursework for the CMHC Master's program is

manageable. I enjoy that classes in (sic) in the evening, but I believe that some classes can be shorter. I believe that some classes can function well online compared to in-person (which sometimes seems excessive).

• I really enjoyed the majority of my coursework at Warner. Personally, I found my

first-year courses to be more focused on preparing me for counseling whereas


second-year courses felt more like support, which was good timing because my main priority shifted to field experience at that time.

• I learned the most from my Counseling Theories classes, Small Groups,

Integrating Expressive Arts into Counseling and Addictions Counseling. I appreciated that these courses incorporated multiple modalities, which made the learning fresh and engaging.

• I thoroughly enjoyed the breadth of topics that were covered in detail in this

program, from theories to counseling, to addictions, I feel that overall this program provided a wealth of knowledge in many pertinent areas of counseling.

• The greatest highlight for me was learning about the theories of counseling in Theories and Practice I and II, and having the opportunity to take an elective that focused on diving deeper into emotion-focused and psychodynamic theories.

• I think that I have gained a lot of insight into the counseling profession at Warner. I have met very helpful and open-minded professors whom have aided

me in finding my way as a person and as a counselor. I have felt supported in exploring who I am in the respect of counseling and fostering that exploration no matter where it took me.

• I think that I have gained a lot of insight into the counseling profession at Warner. I have met very helpful and open-minded professors whom have aided me in finding my way as a person and as a counselor. • I have felt supported in exploring who I am in the respect of counseling and fostering that exploration no matter where it took me. • My learning through coursework to Warner was impeccable. I really enjoyed

class interactions and engaging assignments. I especially liked were the final projects of each class that needed a collective and cumulative effort throughout the semester.

• The areas of this program that could improve are including more practical

knowledge throughout the program. For instance, in the Pre-Practicum course we learned about treatment plans, progress notes, and various terms used in counseling. These practical pieces I think could be better integrated into other classes throughout the courses to build up this knowledge and skill.

• The highlights were that most of the coursework was meaningful, and many

assignments helped me reflect on who I am as a counselor, what my personal


therapeutic approaches are, and my own personal beliefs and biases that may exist.

• I will say a few classes…had a huge assignment load, which was really difficult to manage on top of three other classes, internship, and personally having a part time job. Being mindful of the amount or time involved in coursework is something that I think should be taken into consideration.

Learning Through Field-Based Experiences In response to the questions: “How would you summarize the impact of your learning through field- based experiences in either clinical practice or research? and What would you consider the highlights of these forms of instruction and what were the challenges or areas for improvement?” the following data were generated:

• I had a great experience in my field placements except neither site gave me more one on one opportunities

• I truly enjoyed being connected with various doctoral supervisors for supervision

that gave additional perspective to the process. I made great connections with my supervisors that really helped solidify my counseling skills. With that, I would encourage allowing supervision time to occur potentially during class time, as it was often difficult to schedule meeting times outside of class.

• One of the things I appreciate the most about our program is that we start

practicing our counseling skills during our first semester. I know it could be overwhelming and challenging. Nonetheless, the only way to improve those necessary skills is by practicing and feeling confident. Also, I valued the different levels of supervision we have during the program. It gives you various opportunities to grow and appreciate different points of view during your learning process.

• I think I learned a lot through being in the field and interacting with clients and

other environments. One important highlight is that I learned a lot about myself and how my counseling method is affected. I feel like in-class instruction is presented very academically-based, rather than clinical.

• The majority of my second-year learning came from my field experience (and

through reinforcement and support in our internship


• My clinical practice and research experiences in this program have been instrumental in developing my counseling skill and my professional identity. I loved my practicum and internship at xxxxxx where I was able to engage in many different opportunities that expanded my concept of what counseling is.

• I think the Master’s thesis project was helpful in allowing me to synthesize my learning into a cohesive project that I can now use to enhance my practice.

• I feel like I have learned the most while out in the field. I am a very hands on

learner so I found my footing when I was allowed to take counseling into the real world. I felt that it let me really grow and learn. Wonderful staff also supported me and supervisors who always had my back and made me feel like I had what it takes to be a great counselor.

• The coursework was broad and had depth. The highlights were interactive

discussions and engagement with classmates, doctoral supervisors and professors.

• My field-based experiences were all positive and really helped me excel in my

abilities and professionalism as a counselor. My site supervisors were wonderful.

Faculty and Staff In response to the questions: How would you characterize your interaction and Warner faculty and/or staff? Were your educational or professional goals advanced by relationships with these individuals? The following responses were generated:

• They all have our best interests in mind, I enjoyed all of their classes!

• Most of my interaction with Warner faculty/staff was positive. I felt as though there was an open-door policy and I could always email any professor with questions.

• I do feel as though my professional goals could be advanced by my relationships with these individuals, as I hope they see me as someone they would recommend to employers or other networking connections.

• I think, as a department, we need to find ways to connect different programs

and the value of learning from other departments.

• I feel like the Warner staff is friendly, helpful, experienced, and thoughtful. I feel like I did not develop a close relationship with any staff member.


• The Warner faculty and staff are all wonderful – helpful, supportive,

understanding and really great people in general. My educational goals were certainly advanced by them – I wouldn’t have made it through this program without support from my professors, advisor and staff. I absolutely love our career services representative but also wish… here was a team of people or dedicated career advisors.

• My interactions with Warner faculty and staff have been overwhelmingly

positive. I feel I have learned a lot from each faculty member and have been supported throughout this learning process.

• I always had very good interactions with the staff at Warner. I felt connected to

them and they made me feel like I had a say in how I wanted to interact with the material.

• Warner faculty and staff were always professional. Some more supportive than

others. My professional and educational goals have been advanced by relationships with these individuals.

• My interaction with Warner faculty has been really amazing! Absolutely positive

from stopping by for advice and direction to learning, to setting appointments to deal (sic) with academic and personal issues throughout the program.

• My interactions with Warner staff was overall positive and definitely helped me

advance in my professional goals and development as a school counselor. Constructive Feedback In response to the question: “Is there any constructive feedback on your Warner experience as a whole that you would like to provide in service of enhancing the program?”, the following data were generated:

• Overall, I really enjoyed my experience at Warner. I would respectfully critique the Capstone project/thesis as its purpose was never fully explained to students, and enough time could not be dedicated to it as full-time students…I would advise the conversation surrounding the project to be started earlier in the program and have students secure their topics/advisors earlier on to get them thinking about it, before internship begins and their schedules become too packed.

• Overall, I am happy with my program. I wish we could create activities that we

enhance relationships and connections with students from our class and years


above and below. Also, to have events that connect us more with doctoral students.

• It feels like the only activities that are organized comes from the honor society. We need to create a sense of belonging to our school and our program.

• Overall, I would just repeat that a career services team would be helpful and

reducing the course workload somewhat during the second year so students can focus on internship.

• I would say that the workload for those in practicum and internship could be

very overwhelming to keep up with. I think if the classes were more geared to just making sure all paper work was done weekly and maybe giving the students less to accomplish would be helpful.

• More timely feedback from professors in answering questions and with feedback

from homework. Possibly, fewer writing assignments.

IMPLEMENTATION OF ACTION ITEMS

This section highlights the ways in which the members of the Counseling and Human Development program, and in some cases the Warner School more broadly, have addressed concerns and suggestions gleaned from our program evaluation activities. All of the implemented changes in this section were predicated on information gleaned from one or several sources of data collection. While some of the changes discussed here in the past are well established and have been permanently adopted, other implementation efforts are newly introduced and are still under considerable scrutiny. Because it often takes considerable time to make the journey from an idea that emerges from evaluation data to full implementation, the action items presented here are often part of multi-year efforts. Entry-Level Mental Health and School Counseling Programs Curriculum in Biological Basis of Counseling and Biopsychosocial Assessment As we reported last year, responding to student and stakeholder feedback, and to meet CACREP and CASAC standards referencing biological and neurobiological issues related to disorder etiology and intervention, we devised a new course that encompasses basic psychopharmacology, brain anatomy and physiology, principles of neurocounseling, and an array of pertinent topics in biology including fetal programming, stress epigenetics, psychoneuroimmunology, and the brain-gut axis. This course was offered for the first time the Spring 2019 semester. We are pleased to report that feedback from students in the course has been very positive.


While it was our hope that this year we would see significant increases in the ability of our students to engage in biopsychosocial assessments, there is clearly still more work to be done in this regard. It has become evident that our efforts to bolster confidence in biopsychosocial assessment by substantially modifying our Pre-Practicum course and offering our course on the biological basis of mental health challenges, which includes a large volume of information of biosocial interactions, will require further thinking. Improvements in Mental Health and School Counseling Internship Records The use of Blackboard continues to expand as a resource library for documents and professional resources for site supervisors and tape supervisors. We now have articles available on Blackboard on topics related to clinical supervision and special issues in counseling. In the last two years, we moved to keeping all the practicum and internship assessment tools (weekly hours logs, supervisor assessments, session tape evaluations) electronically on Blackboard. These materials are now fully on-line and have been updated to reflect the CACREP 2016 Standards. Although major steps had been achieved regarding this transition in the past few years, the 2019-2020 year saw continued improvement in this regard. That being said, there are remaining records to be added to the online library. Growing our Optional Certificate in Mind-Body Intervention Based on feedback that includes students’ interest in mind-body work, and students’ interest, particularly in Mental Health Counseling, of expanding employment and internship outlets, last year we launched a new certificate that features mind-body interventions, which current students or alumni can pursue. We continue to market this program and are considering ways that it might be expanded. Mind/body approaches to healing and wellness are gaining in popularity in the U.S., and in certain forms, have been a central part of practice in Eastern and indigenous approaches to psychological and physical wellness. Further, and of significance to us, is that research supports their efficacy in treating a number of psychological issues. In the context of student and supervisor feedback, we hope that implementation of this program will address feedback in the following ways:

• Offer students a path to greater understanding of mind/body work • Strengthen students understanding of the biological basis of mental health • Increase their knowledge base regarding mind-body interventions, including

various emotion-focused therapies • Offer students the opportunity to take coursework in an interprofessional

environment and establish contacts with personnel from the University of Rochester Medical Center, thus potentially increasing their attractiveness for employment or internship opportunities in integrated care settings.

A Diversity of Teaching Methods


Given our high teaching ratings, we plan to continue with much of our current approach to teaching, but will also consider new ways to improve delivery of instruction that addresses the outcome indicating that classes were less interesting and stimulating than in the past (although still in an acceptable range of 4.07 out of 5). For example, we continue to increase our efforts regarding pedagogical approaches. More specifically, we are striving to offer a diversity of didactic and experiential learning opportunities and to increase the use of creative technologies. We have, for example, implemented dramatic changes in our Pre-Practicum course that gives students more time engaged in experiential activities that simulate real-world clinical practices. We are also striving to ensure that our doctoral students, who are often given teaching responsibilities, have exposure to cutting edge classroom technologies and reflective teaching practices. As our classes have transitioned to fully online during the COVID crisis, many faculty have taken advantage of rigorous professional development opportunities offered by highly accomplished professors in our On-line Teaching and Learning programs. This has helped us to offer quality online instruction taking advantage of best practices in this realm. As we have noted in past years, many of our doctoral students have had the opportunity to imbed this kind of coursework into their doctoral programs. Other examples of our striving to integrate cutting edge research and best practices into our curriculum have been discussed in previous reports. Previously cited examples include: 1) we dramatically revised our master’s-level course in addictions 2) we created a state-of the art master’s-level psychopharmacology course, 3) we are offering a course in Infant Mental Health and 4) we are offering a child and adolescent course being taught by a member of a dynamic research team working in the field of pediatric developmental psychopathology. Thus, although our evaluations have remained positive for many years, we are continually monitoring evaluations and student feedback for areas in need of improvement. CASAC-T Credential This year, for the second time, Mental Health students entered their program with an imbedded curriculum, approved by the New York State Office of Alcoholism and Substance Abuse Services, leading to the Credentialed Alcohol and Substance Abuse Counselor Trainee (CASAC-T) designation. In New York State, where there are restrictions on the Licensed Mental Health Counselor (LMHC) scope of practice that often hinder employment, the addition of the CASAC-T allows LMHC’s more flexibility and greater success in procuring employment. Re-Visioning Pre-practicum Preparation In our last report, we talked at length about how we were re-visioning the preparatory work for students entering Mental Health Practicum sites in the spring. At that time, we reported that in response to feedback from alumni, who indicated that areas for


improvement included issues related to organization of counseling session data, monitoring and documenting progress in the treatment plan, and conducting a biopsychosocial evaluation, we engaged in a complete revision of our Pre-Practicum course. The course now includes more emphasis on active biopsychosocial case conceptualization, multiple ongoing requirements to actively generate treatment plans, documentation practices that include the use of technology, and the use of client data to monitor progress and inform intervention strategies. We are pleased to say that this year, assessments in these areas from alumni or site supervisors did not indicate any weakness, although we are still interested in shoring up confidence in biopsychosocial assessment and treatment planning as this is CACREP mandate. Addressing Workload in the Final Semester of Entry-Level Programs We have attempted to respond to this particular issue for several consecutive years. Our last Annual Report detailed our remediation strategies. First, a series of scheduled workshops became available to help students structure their thesis work. Second, thesis work became embedded in the Internship course so that a more scaffolded process guides students through their respective capstone projects. Third, our advising for Mental Health Students became the purview of a single program advisor who was able to take a uniform, proactive stance toward working with students around the notion of completing coursework and internship hours during the summer. With these changes in place we are still receiving feedback indicating that the burden of the capstone project is very difficult to manage during the final semester. To further address this problem, we have taken the following steps. First, rather than having students write full-length literature reviews, which for many at the entry-level seem too developmentally advanced, we have instituted a new approach to the capstone project that we will be able to assess this coming spring. Students will be given the opportunity to scale down the literature-review requirement for their project and to put more focus on application. In addition, a PowerPoint slide will take the place of the poster required for all projects. Expanded Child and Adolescent Counseling Instruction Over the past several years we have had numerous requests from a variety of stakeholders to offer more coursework in child and adolescent counseling. In the previous Annual Report, we described how our Child and Adolescent course had been expanded to include a number of important features including developmental psychopathology and emerging child and adolescent treatments, both individual and dyadic, for children experiencing early life trauma. This course is one that should address some of the concerns of both School Counseling and Mental Health Counseling students wanting more exposure to strategies for addressing the needs of the child-adolescent population, particularly in the arena of trauma-informed care. Curiously, in spite of the large demand for this course content, it was sorely under-subscribed and was consequently converted to a biannual offering. The same is true for another course


that we added to the curriculum, Infant Mental Health. Considering the disconnect between enrollment numbers and the feedback we have received indicating a need for this course material, we are going to consider ways to restructure the current course schedule to make these courses easier for students to access. Also, as the School Counseling curriculum expands to 60 credits, we anticipate that this course will become a requirement. Augmentation of School Counseling Faculty In our last Annual Report, we discussed our response to the need expressed by current students and graduates in School Counseling to increase the number of faculty with an expertise in school counseling. In short, we attempted to augment the school counseling affiliated faculty by intentionally reaching out to experienced school counselors who are interested in our doctoral program and offering them funding opportunities. This resulted in the addition of several highly qualified and experienced school counseling doctoral students who are able to teach and supervise our school counseling students. It is clear from the latest feedback from School Counseling stakeholders, that these efforts are still not meeting the needs of this constituency. As a result of this feedback, we made the decision to divide the incoming classes into School Counseling and Mental Health Counseling sections in our showcase first-year, two-semester course sequence, Counseling Theory and Practice 1 and 2. The School Counseling section is being taught by a veteran school counselor and emphasis is being given to theories that are most germane to school counseling. Use of School Counseling Exemplars in Core Classes In response to requests from School Counseling students desire for more exposure to school counseling related materials in core counseling classes, a battery of case studies germane to school counseling practice was written by one of our faculty members and was piloted in a class this past summer. Responses to the material were very positive and the faculty member hopes to expand this collection of materials so that they can be utilized in other core counseling classes. Hiring Scott McGuinness As described in previous years of this report, we were receiving feedback that students in the field needed more exposure to contemporary digital modalities around intake, mental status examination, case documentation and other routine practices in mental health clinic settings. At the same time, as also reported in previous years, we have made a concerted effort to contemporize our Addictions course, expand addictions-related clinical training site options, and embed New York State addictions credentials (see the aforementioned CASAC-T credential) into our entry-level mental health curriculum. Careful consideration led us to the hiring of our newest faculty member, Scott McGuinness, whose considerable experience in both of these realms have allowed him to contribute to our program in ways that have already improved our entry-level assessment scores in relation to technology and have opened the door to a new


collaboration with the addictions clinic at the University of Rochester’s Strong Memorial Hospital. Dr. McGuinness has also become our liaison to the New York State Office of Addictions Services and Supports, the grantor of the CASAC-T credential. Doctoral Programs Ongoing Efforts to Increasing Doctoral Student Research Opportunities Doctoral students and doctoral alumni continue to voice their desire for more research and publishing experiences. In this regard, faculty have continued to expand research opportunities for students. The majority of our faculty now hold regular “lab meetings” where students learn from each other’s research interests, and where there are opportunities for collaboration around publication and presentation. Many of these collaborations are evident in faculty vitas where many collaborative efforts are detailed. That being said, there are those who still feel disengaged from the research process. Going forward, we believe it would be beneficial to have a Research and Publishing Orientation in which students receive more detail about what is available and how to take advantage of existing opportunities. What follows is an excerpt from the 2017-2018 Annual Report, which details some of the additional steps we have taken: In an effort to increase research and publication opportunities we have implemented the following:

• Added to our curriculum EDE 481 Practical Knowledge and Skills for Research in Health and Education, which is designed to make our students attractive to University researchers looking for laboratory apprentices or research coordinators.

• Additional courses that have been added to bolster research opportunities include Designing Intervention Research in Applied Settings; Studying Human Development, Counseling, and Education in Context: Advances in Design and Method; Creativity in Qualitative Methods; and Advanced Applications of Qualitative Research Methods in Social Science.

In addition to providing research opportunities, these course offerings address some of what we have observed in our monitoring of student milestones and in discussions emerging from our annual review of students, namely, that students would benefit from additional training in methods. Increased Accountability in the Doctoral Research Apprenticeship Another response to doctoral student requests for more research experience has come in the form of us attempting to bolster the quality of experiences that are already in place. In discussions with doctoral students who had availed themselves of our credit-bearing Research Apprenticeship course (required for PhD’s and optional for EdD’s), it became apparent that students’ experiences in this course varied considerably from one placement to the next. In an attempt to ensure that all students are getting quality


research experiences, we have created a form that students are required to complete, which contains acceptable categories of research experience to complete the Apprenticeship requirements. By using the mandatory form to structure the experience, students will hopefully more reliably emerge from the Apprenticeship with a bolstered research skill set. Working Towards Improving Leadership and Advocacy Skills In response to perceived weakness in advocacy and leadership, we have revamped our doctoral course, EDU 563, Advocacy, Consultation and Systems Change, to put more emphasis on counseling-related leadership and advocacy. In the past, this course was taught by a very talented instructor whose primary identification was with Human Development. With the departure of this faculty member from the Warner School, the decision was made to move this course from the purview of a human development faculty to the purview of a core counseling faculty member, Dr. Karen Mackie, who brings her very rich background in both Mental Health and School Counseling leadership and advocacy into the classroom. Based on her experience, Dr. Mackie is able to model and discuss leadership and advocacy roles that are relevant to counseling students and it is our hope that this more intensive focus on counseling-specific advocacy and leadership issues will give students the impetus to increase the number of hours that they engage in advocacy and leadership-related doctoral internship hours.

Addition of Technology Component in Doctoral Program. In response to the site supervisors’ observations reflected in our Doctoral Site Supervisor Surveys over the past several years that there is some weakness among doctoral students in the area of technology, we introduced a technology unit into our course EDU 564, Contemporary Trends in Mental Health Appraisal, Intervention and Research. This unit includes topics such as telemental health and use of mental health apps among others. We are pleased to report that on this year’s site supervisor survey, a score of Meets Expectations was given to the assessed doctoral student.

Doctoral Student Coffee Hour Recent feedback, which is bears some relationship to COVID restrictions around interpersonal contact, indicated that doctoral students feel the need for a venue that allows them to foster supportive relationships in a noncompetitive, interactive environment. In response to this feedback, we have instituted a monthly virtual coffee hour that allows students to interact with each other and to interact with faculty. We are happy to report that our first meeting was very successful and that our second meeting is coming up soon. Continued Emphasis on Improving Professional Writing Skills This topic is part of an ongoing discussion among core faculty. As stated in a past Annual Report, faculty hoped that our bolstered emphasis on literature review writing


within the curriculum might result in students being more engaged in the nuances of this task. Although numerous resources are made available to students to help in the literature review process, including a detailed scaffolding in at least two required courses, a course on academic writing, and a writing support services that provides numerous workshops to aid in the literature review process, we are still contemplating solutions to this problem. Possible remediation efforts being considered by the faculty include: 1) increasing the number of courses that scaffold literature review writing, 2) requiring all students to take our course in academic writing, 3) encouraging greater engagement with Writing Services, and 4) putting greater emphasis on the writing portion of the admissions process. New Tiered Expansion of Dissertation Proposal Class As reported last year, in response to difficulties that faculty have observed, and students have voiced, regarding the dissertation process, we created a dissertation proposal seminar class for students who would prefer a scaffolded experience in their dissertation proposal writing. Additional feedback underscoring the realization that many of our doctoral students have family responsibilities, while others have had to relocate in the midst of their doctoral program has led to our decision to offer the proposal seminar in a highly interactive, synchronous/hybrid format. Although we have had conversations considering how we might extend the scaffolding process in a way that encompasses the entire dissertation process, this has not yet materialized. We have, however, taken a first step in this direction by creating a tiered expansion of the Dissertation Proposal Class. Realizing that the combination of work obligations, developmental readiness, and lingering responsibilities around Comprehensive Exams creates an environment where progress through the proposal writing process varies widely, the instructor for this class has created two tiers in this class that meet separately. One tier consists of individuals taking the class for the first time who receive detailed instruction concerning elements of a sound dissertation proposal. These individuals simultaneously engage in the proposal writing process. The second tier of students consists of those who have previously received the highly scaffolded instruction in tier one, but who need further support in order to complete the proposal process. Administrative Adoption of Taskstream This is the first year during which all of our student assessment data collection and reporting has been processed through a well-established data collection and analysis platform known as Taskstream. This system allows us to more easily cross reference data sources to complete complex KPI and Program Objective (PO) measures. It also allows for direct student input of KPI and PO-related assignments that can be accessed by the instructor who completes rubrics that are stored in the Taskstream system. One of the main benefits of the system is its ability to integrate classroom and clinical data.


Ongoing Efforts to Support Adjunct and Affiliated Faculty Even though we have instituted several new practices that were designed to increase support for adjunct and affiliated faculty, it is clear from our survey, that many have not felt the impact of those efforts (e.g., offering a comprehensive adjunct and affiliated faculty orientation every semester). It is important to note that the Adjunct Survey that is part of this report does not reflect changes that we have made in the last year. In any case, as a further step in addressing some of the specific concerns of these members of our faculty, we have begun a program in which each of our core counseling faculty members is in essence responsible for mentoring the adjunct and affiliated faculty associated with 2 to 3 courses. As part of the mentoring process, the core faculty are expected to ensure that their mentee 1) acquires needed syllabi, 2) is familiar with the CACREP standards embedded in their courses, 3) understands the concept of a key performance indicator and is instructed in how to complete and record key performance indicator measures, 3) understands the process for ordering textbooks and acquiring desk copies, 4) has any questions answered concerning the use of Blackboard, and 5) understands the process of assigning grades. Faculty mentors will also be available as questions arise during the semester concerning student problems or classroom management issues. These efforts by faculty mentors will supplement the steps, that we have already taken to improve the experiences of our adjunct and affiliated faculty (AAF). As outlined in last year’s Annual Report, this includes providing a school-wide orientation for adjunct faculty that covers a number of curricular (e.g., use of Blackboard) and logistical issues (building access, photocopying, etc.). Likewise, per the adjunct faculty’s request, we established:

• A data file of sample syllabi for all of the courses taught in our program (one of our core faculty members, maintains this file).

• A template to create new syllabi. • A complete set of measurement rubrics that cover all of the CACREP standards

associated with each course (one of our core faculty members provides direction to the AAF concerning CACREP requirements).

• Vehicles for assisting AAF’s with CACREP data input.

Additional Items Emerging from Program Evaluation in Need of Attention

Although we have made many efforts to improve program delivery, there are a number of things that still need attention going forward. Mindful of the notion that many changes may take years to implement, some of the things that we would like to consider in the coming year and beyond include the following. Please note that 1) this list contains some items from previous years that are still problematic, and 2) some of the efforts that might have gone into curricular change were consumed in the previous year by management of both the pandemic and reaccreditation:


• Trying to provide entry-level students more access to group work. • Working to integrate skills learned in Pre-practicum such as treatment planning,

progress notes, mental status examination etc. into subsequent coursework. • Working to expand curriculum related to the desire for increased exposure to

empirically-supported therapies targeting specific diagnoses. • Continuing to find new ways to present School Counseling students with

increased opportunities for more reality-based, experiential learning. • To incorporate more school counseling exemplars into our counseling courses. • Expand our doctoral student coffee hour to create opportunities for doctoral and

entry-level students to build relationships. • Working toward more comprehensive, developmental work on trauma. In

particular, developing an understanding of trauma, and its impacts across the life course and across a multiplicity of contexts.

• Additional scaffolding of our doctoral dissertation process beginning with the proposal and continuing through the data collection and analysis, and the final dissertation write-up—potentially through a new class offering.

• Expanding our doctoral methods course offerings to include methodologies that are conducive to studying counseling practice, e.g. action research, interpretive description.

• Improving ways to capture assessment data in relation to employers and alumni. This is particularly important in relation to former doctoral students and doctoral student employers, neither of whom are well represented in our data collection efforts.

• Expanding the inclusivity of our demographic data collection by moving away from a binary gender identification scheme to one that is more inclusive of trans, queer, and intersex options.

• Considering expansion of our mind-body offering to include creative arts, mindfulness, and other related therapies.

• Rethinking our program objectives so that the outcomes are more easily measured.

• Increasing the validity and inter-rater reliability of our KPI measures.

MOVING FORWARD

As we move forward in our ongoing efforts to improve the quality of our programs, we will do so guided by the following principles:

• In shaping our programs, we are committed to showcasing the core

principles of the counseling profession. This means that all that we do will be driven by a dedication to social justice, life-long healthy development, client empowerment, respect for diversity, understanding individuals in context, and the need for evidence-based practice.


• Program development is best achieved through the input of a community of

dedicated stakeholders. Changes to our programs must be informed by the wisdom of those clinicians who have dedicated their lives to the direct service of clients; scholars who are devoted to understanding best practices and ways of applying cutting edge information emerging from research in counseling, psychology, psychiatry, human development, and the biological sciences; the experiences reported by the students who we hope to teach; and the voices of the clients that we serve.

• The quality of our curriculum is ultimately judged by the degree to which our

students have assimilated its contents. Thus, we need to continually devise more effective ways to evaluate our students’ depth of understanding of curriculum, ability to transform theory into practice, ability to turn practice into theory, facility in regard to serving their clients justly, ethically and effectively, and capacity for modeling for their clients the tenets of lifelong healthy development.

Appendices

Table of Contents Warner School 2019-2020 Comprehensive Evaluation Report 51

Table 1 Summary of CACREP Course Evaluations Academic Year: Fall 2019-Summer 2020

Total Responses: 997

Scale: 1 = strongly disagree (SD)

2 = disagree (D) 3 = neither/neutral (N)

4 = agree (A) 5 = strongly agree (SA)

item

# item description AY20 AVG.

1 the course and course activities were well organized 4.33

2 written assignments advanced the goals of the course 4.42

3 reading assignments contributed to the goals of the course 4.43

4 the instructor showed a genuine concern for the students 4.68

5 the instructor was available to students 4.63

6 classes were interesting and stimulating 4.16

7 students in this course were free to disagree and ask questions 4.62

8 I learned a great deal from this course 4.35 Average 4.45

Less than other

courses

Equal to other courses

More than Other

Courses

9 compared to other courses of comparable credit hours, the work required for this course was: 31.70% 61.48% 8.93%

Yes no

10 The method for student evaluation was clearly defined 95.29% 6.82%

11 the student evaluation(s) were a fair assessment of my performance in this course 97.19% 4.11%


Table 2 Summary of Student Feedback-Site & Site Supervisor Evaluation

Master and Doctoral Students Academic Year: Fall 2019-Summer 2020

Total Responses: 38 Scale: 1 = Not Satisfactory 2 = Emerging, Needs Improvement 3 = Basic Proficiency 4 = Outstanding Performance

Question AY20

Rating Average

My site supervisor was helpful in orienting me to the work site. 3.7 My site supervisor was helpful in arranging access to individual clients and

groups/families for me to work with. 3.7

My site supervisor communicated clear expectations for my work. 3.67 My site supervisor was accessible to me. 3.73

My site supervisor was responsive to my questions or concerns. 3.89 My site supervisor consistently kept his/her weekly supervisory meetings with me. 3.67 My site supervision provided me with a helpful balance of direction and autonomy. 3.7

My site supervisor was a good professional role model. 3.81 I would recommend this internship site to others. 3.73

I had the opportunity to innovate at this site if I cared to. 3.51 I would recommend this site supervisor to others. 3.84

I had a good deal of individual counseling practice at my site. 3.27 I had a good deal of group counseling practice at my site. 3.11

I improved my individual counseling skills. 3.54 I improved my group counseling skills. 3

I was able to improve my assessment skills at my site. 3.14 I learned new ways of intervening with clients (e.g. family work, consultation,

advocacy) during this internship experience. 3.43

I learned a lot about the work of a counselor in the real world. 3.73 My confidence as a counselor increased. 3.68

What I learned and did at the internship site affirmed my choice to become a counselor.

3.78

I have become more aware of the relationship between my personal values and professional decisions and actions.

3.73

I am able to describe my decision-making processes regarding my counseling work. 3.73 I am motivated to continue professional development beyond academic requirements. 3.97 My ability to situate client experiences and perspectives in multiple contexts, such as social class, ethnicity, gender and family, to inform my interpretations and practices

has improved.

3.68

My ability to systematically evaluate my work from various perspectives, such as client's viewpoint, my expectations and from supervision has improved.

3.76

I experience myself as a continuing to improve as a counselor. 3.92 Average 3.63


Table 3 Summary of Student Feedback-Small Group Supervision


Total Responses: 36

Scale: 1 = Not Satisfactory 2 = Emerging, Needs Improvement 3 = Basic Proficiency 4 = Outstanding Performance

Question AY20 Rating

Average Peer interactions provided important ideas and support. 3.69

The small groups were good forums for exploring issues and problems. 3.75

The small groups became an important resource for empathic support and learning. 3.61

I felt safe enough in group supervision to disagree with my peers about issues or interpretations. 3.72

Student case presentations were a good learning experience. 3.78 The interactions in group supervision challenged my

presumptions about my work and motivated me to try alternative ways of understanding and practice. 3.53

My small group supervisor listened well to our comments, experiences and concerns. 3.78

He/she showed a genuine concern for me and my development as a counselor. 3.81

He/she was supportive and encouraging. 3.81 He/she had helpful ideas and suggestions for me in my counseling

work. 3.75 He/she facilitated our discussions or used activities in a way that was

helpful to our exploring issues and problems in our counseling. 3.72 He/she offered resources that were helpful to me now or probably will

be helpful in the future. 3.67 He/she facilitated discussion in a way that made me feel comfortable

participating. 3.81 Average 3.73


Table 4 Summary of Student Feedback-University Tape Supervision

Master Students Academic Year: Fall 2019-Summer 2020

Total Responses: 37

Scale: 1 = Not Satisfactory 2 = Emerging, Needs Improvement 3 = Basic Proficiency 4 = Outstanding Performance


Average Gives time and energy in observations, tape processing and case

conferences. 3.81 Accepts and respects me as a person. 3.92

Recognizes and encourages further development of my strengths and capabilities. 3.86

Gives me useful feedback when I do something well. 3.84 Provides me the freedom to develop flexible and effective counseling styles. 3.81

Encourages and listens to my ideas and suggestions for developing my counseling skills. 3.92

Provides suggestions for developing my counseling skills. 3.76 Helps me understand the implication and dynamics of the counseling

approaches I use. 3.76 Encourages me to use new and different techniques when appropriate. 3.84

Is spontaneous and flexible in the supervisory sessions. 3.81 Helps me define and achieve specific concrete goals for myself during the

practicum/internship experience. 3.68 Gives me useful feedback when I do something wrong. 3.73

Allows me to discuss problems I encounter in my practicum/internship setting. 3.86 Pays appropriate amount of attention to both me and my clients. 3.84

Average 3.82


TABLE 5 KPI Outcomes, 2019-2020

Entry-level Counseling Core (CC) KPI’s School Counseling (SC)

Clinical Mental Health Counseling (CMHC)

KPI # Measure 1 Measure 2 Measure 1-Measure 2 Change Over Time

CC1 (CMHC) 3.76 4.00 +0.24 CC1 (SC) 3.35 3.75 +0.40 CC2 (CMHC & SC) 3.59 3.43 -0.16 CC3a (CMHC & SC) 3.79 3.79 No change CC3b (CMHC & SC) 3.71 3.87 +0.16 CC4a (CMHC & SC) 3.73 3.93 +0.22 CC4b (CMHC & SC) 3.83 3.70 -0.13 CC5 (CMHC & SC) 3.59 3.67 +0.08 CC6a (CMHC & SC) 3.54 3.93 +0.39 CC6b (CMHC & SC) 3.54 3.93 +0.39 CC7 (CMHC & SC) 4.00 4.00 No change CC8 (CMHC & SC) 3.87 3.70 -0.17 CMHC1 2.87 3.94 +1.07 CMHC2 3.75 3.93 +0.18 CMHC3 3.27 3.71 +0.44 CMHC4 4.00 4.00 No change CMHC5 3.10 3.57** 3.65 +0.47, +0.80 SC1 3.40 3.14 -0.26 SC2 3.89 3.40 -0.49 SC3 3.31 3.14 -0.17 SC4 4.00 4.00 No change SC5 3.35 3.51 3.53 +0.16, +0.02 DCES1 3.31 3.42 +0.11 DCES2 3.70 3.60 -0.01 DCES3 EYC* EYC* N/A DCES4 3.61 4.00 +0.39 DCES5 3.50 3.40 -0.10

*Even-year course—no data for this cycle **Due to COVID-related circumstances, this score is short of the total N.


Table 6 Survey Results-Alumni


Total Responses: 7

Scale:

1 = Insufficient 2 = Emergent 3 = Basic Proficiency 4 = Outstanding Performance

Rating Average

Relationship and Communication Skills Use interpersonal strengths (e.g., caring, integrity, confidence, sense of self) to build a working alliance with client 3.86 Use basic communication skills to listen and respond to client with accurate empathic understanding 4.00

Average 3.93

Rating Average

Assessment and Appraisal Skills Organize session data into multiple meaningful frameworks that enhance understanding of the client (e.g., developmental, cognitive, affective, behavioral, systemic, sociocultural) 3.57

Recognize developmentally appropriate from problematic behaviors 3.86 Evaluate and adjust the counseling process to affect successful counseling outcomes (process evaluation) 3.57

Complete a thorough DSM diagnosis 3.29

a) Complete a mental status examination 3.43

b) Complete a biopsychosocial intake 3.57

c) Assess the needs of a client in crisis 3.43

Average 3.53



Master and Doctoral Students Rating

Average Intervention Skills

Use counseling and human development theories and concepts as part of counseling practice 3.71 Provide session structure and focus through questions, interpretations, experiential activity, problem solving strategies, and other appropriate methods for furthering counseling goals 3.71 Use flexibility and ability to generate novel responses when faced with unfamiliar or unexpected situations in the counseling process 3.71 Execute a variety of counseling modes, including individual counseling, group counseling, career counseling, and small systems consultation 3.71 Complete a treatment plan based on a DSM diagnosis, mental status examination, and biopsychosocial clinical intake interview 3.14 Monitor and document the progress being made in the implementation of a treatment plan 3.29 Intervene on behalf of a client who is in crisis 3.57

Average 3.55 Rating

Average Contextual Competencies

Advocacy - Provided me with knowledge of vehicles of client empowerment, methods of advocacy, and legal liabilities of advocacy work 3.57 Multicultural Skills - Developed my sociocultural self-knowledge, awareness of social and political barriers to counseling, knowledge of liberatory counseling theory, and ability to implement culturally congruent counseling practices with diverse populations 3.29 Consulting - Provided knowledge of consultation models and processes for integrating family, peer, and community support systems in counseling and supporting clients 3.57 Systems Change - Developed my understanding of systems and organizational change for the purpose of improving systems to enhance human development 3.71

Average 3.54 Rating

Average Technological Skills

Developed my competency with technology, including word processing, email, library databases, and locating counseling- related resources on the Internet

3.43

Rating Average

Disposition Professional Identity - Familiarized me with history of the profession, roles of professional counselors among the various human service professions, 3.86



Master and Doctoral Students Rating

Average professional associations and journals in counseling, and issues surrounding counseling licensure and accreditation Professional Development - Developed my capacity for self- reflection, self-understanding, openness to ongoing supervision, and commitment to further personal and professional growth 4.00 Legal and Ethical Issues - Instilled legal and ethical standards of the profession in my counseling practices 3.86 Professional Behavior - Developed my responsibility toward fulfilling professional commitments and acting as a counseling professional 4.00

Average 3.93


Table 7 Survey Results-Master’s Site Supervisors Academic Year: Fall 2019-Summer 2020

Total Responses: 13 Scale: 2 = Emergent 3 = Meets Expectations 4 = Exceeds Expectations

Rating Average Relative to the student's level of development, how would you characterize our students as counseling practitioners?

3.38

To what extent do our students demonstrate the dispositions of a 21st century collaborative professional? 3.23

To what extent do our students act with sensitivity to diversity and social justice dimensions of counseling in their work with clients? 3.46

To what extent are Warner students competent with the use of technology in practice? (Emerging=hesitant) 3.54

To what extent do our students meet your expectations in each of the following dimensions?

Their use of research, ethics and critical thinking to guide best practice? 3.31 Their openness to developing innovations in counseling practice that meet diversity concerns? 3.46

Their commitment to serving as an agent of organizational change (social justice) or client empowerment? 3.38

Their ability to integrate biological, psychological and social interventions in their work with clients? 3.15

Their interest in fostering life-long healthy development? 3.38 Their holding of an ecological view of people in context (history, culture, social life)? 3.15

Their being caring, compassionate and self-reflective people? 3.69 Average 3.38


Table 8 Survey Results-Doctoral Site Supervisors Academic Year: Fall 2019-Summer 2020



4.00









Their being caring, compassionate and self-reflective people? 4.00 Their ability to function as leaders and advocates within the organization and profession? 4.00

Their ability to supervise and teach other staff members or students in training? 4.00

Their ability to generate new knowledge for the profession? 4.00 Average 3.86


Table 9 Survey Results-Master’s Employers

Academic Year: Fall 2018-Summer 2020



3.50









Their being caring, compassionate and self-reflective people? 3.67 Average 3.38


Table 10 Survey Results-Doctoral Graduate Employers

Academic Year: Fall 2018-Summer 2020


Rating Average There were no responses from the doctoral graduate employers from the 2018 survey cycle.


Table 11 Summary of Adjunct and Affiliated Faculty Survey

Academic Year: Fall 2018-Summer 2020 Total Responses: 10

Scale: 1 = Very Dissatisfied 2 = Dissatisfied 3 = Neither Satisfied nor Dissatisfied 4 = Satisfied 5= Very Satisfied


Average How would you rate your overall “job satisfaction” experience as an Adjunct Faculty Member of the Warner Graduate School? 3.80 How would you rate the overall support you received as an Adjunct Faculty Member from the Counseling and Human Development Department? (Note: Department means the Chair, Faculty and Staff) 3.70 How would you rate the support you received from the department in preparing you to teach (examples of syllabus, provide you with copy of textbook, etc.)? 3.70 How would you rate the instruction/support you received from the department in educational technology (instruction on the use of Blackboard, instruction of classroom technology, etc.)? 4.10 How would you rate the support you received from the department in classroom management (dealing with difficult students, answering questions, etc.)? 3.80 How would you rate the feeling of inclusion into the life and work of the department (were you given the opportunity to take an active part in the department’s decision-making process)?

3.20

What is your opinion of the remuneration provided by the Warner Graduate School as compared to other institutions? 3.40

Average 3.67


Demographics Tables 1: Admissions Data 2019-2020

1a) Admissions: Gender x Program 2019-2020

Female Male Total

MS 32 76% 10 24% 42 Doctoral 5 71% 2 29% 7

Total 37 76% 12 24% 49

1b) Admissions: Race/Ethnicity x Program 2019-2020

2+ Races Asian Black Hispanic Int’l White Unknown Total

MS 1 2% 1 2% 1 2% 2 5% 4 10% 31 74% 2 5% 42 Doctoral 0 0% 0 0% 2 29% 0 0% 1 14% 4 57% 0 0% 7

Total 1 2% 1 2% 3 6% 2 4% 5 10% 35 71% 2 4% 49

Admissions: Race/Ethnicity x Program

2+ Races Asian Black Hispanic Int’l White Unknown No Response

Total Excluding

NR MS 2 0 3 3 1 11 0 0 20 Doctoral 2 0 0 0 0 4 2 0 8 Total 4 0 3 3 1 15 2 4 28

1c) Admissions: Race/Ethnicity x Program

2019-2020 2+ Races Asian Black Hispanic Int’l White Unknown Total

MS Female 1 3% 0 0% 1 3% 1 3% 4 13% 23 72% 2 6% 32 Male 0 0% 1 10% 0 0% 1 10% 0 0% 8 80% 0 0% 10 No Response 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0

Doc Female 0 0% 0 0% 2 40% 0 0% 1 20% 2 40% 0 0% 5 Male 0 0% 0 0% 0 50% 0 0% 0 0% 2 100% 0 0% 2 No Response 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 49


Demographics Tables 2: Current Student Demographic Data 2019-2020

2a) Current Students: Gender x Program

2019-2020 Female Male No

Response Total Excluding NR

MS 46 73% 17 27% 29 46% 63 Doctoral 35 83% 7 17% 5 12% 42 Total 81 58% 24 17% 34 24% 105

2b) Current Students: Race/Ethnicity x Program 2019-2020


Total Excluding

NR MS 2 3% 1 2% 4 6% 7 11% 4 6% 42 67% 3 5% 29 46% 63 Doctoral 2 5% 1 2% 4 10% 1 2% 3 7% 30 71% 1 2% 5 12% 42 Total 4 3% 2 1% 8 6% 8 6% 7 5% 72 52% 4 3% 34 24% 105

2c) Current Students: Race/Ethnicity x Program 2019-2020


Total Excluding NR

MS Female 2 4% 0 0% 3 7% 5 11% 4 9% 29 63% 3 7% 0 0% 46 Male 0 0% 1 6% 1 6% 2 12% 0 0% 13 76% 0 0% 0 0% 17

No Response 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 29 0% 29 Doc Female 2 6% 1 3% 4 11% 1 3% 2 6% 26 72% 0 0% 1 3% 36

Male 0 0% 1 5% 4 18% 0 0% 1 5% 15 68% 1 5% 1 5% 22 No Response 0 0% 1 0% 4 0% 0 0% 0 0% 11 0% 0 0% 5 0% 16

105


Demographics Tables 3: Graduating Student Demographic 2019-2020

3a) Graduates: Gender x Program

2019-2020 Female Male Total

MS 20 91% 2 9% 22 Doctoral 2 67% 1 33% 3 Total 22 88% 3 12% 25

3b) Graduates: Race/Ethnicity x Program 2019-2020

2+ Races Asian Black Hispanic Int’l White Unknown Total

MS 1 5% 1 5% 2 9% 3 14% 5 23% 10 45% 0 0% 22 Doctoral 0 0% 0 0% 0 0% 0 0% 0 0% 2 67% 1 33% 3 Total 1 4% 1 4% 2 8% 3 12% 5 20% 12 48% 1 4% 25

3c) Graduates: Race/Ethnicity x Program 2019-2020

+ Races Asian Black Hispanic Int’l White Unknown Total

MS Female 1 5% 1 5% 0 0% 3 15% 5 25% 10 50% 0 0% 20 Male 0 0% 0 0% 2 100% 0 0% 0 0% 0 0% 0 0% 2 No Response 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0

Doc Female 0 0% 0 0% 0 0% 0 0% 0 0% 1 50% 1 50% 2 Male 0 0% 0 0% 0 0% 0 0% 0 0% 1 67%

33% 1

25


Demographics Table 4: Gender Across Program 2019-2020

Demographics Table 4: Gender Across Program

2019-2020

Admitted Current 2019-20 Graduates

Female 37 76% 81 77% 22 89%

Male 12 24% 24 23% 3 11%

Demographics Table 5: Race/Ethnicity Across Program 2019-2020

Demographics Table 5: Race/Ethnicity Across Program 2019-2020

Admitted Current 2019-20 Graduates

2+ Races 1 2% 4 4% 1 4% Asian 1 2% 2 2% 1 4% Black 3 6% 8 8% 2 8%

Hispanic 2 4% 8 8% 3 12% International 5 10% 7 7% 5 20%

White 35 71% 72 69% 12 49% Unknown 2 4% 4 4% 1 4%

Documents

2019-2020 CACREP ANNUAL COMPREHENSIVE EVALUATION …