Program Evaluation 2018-2019 - Milligan College · 2018 Association for Counselor Education and Supervision Webinar: Courageous Conversations: Unraveling the Taboo Dr. Rebecca Sapp:

Milligan College Counselor Program Evaluation

YEAR IN REVIEW: 2018‐2019

COMPLETED: JANUARY, 2020

PREPARED BY COUNSELOR EDUCATION PROGRAM FACULTY

Table of Contents

Table of Contents ......................................................................................................................... 1

COUNSELOR EDUCATION PROGRAM OVERVIEW ............................................................. 3

Program Evaluation Overview ................................................................................................ 3

Evaluation Timeline ................................................................................................................. 5

EVALUATION OF PROGRAM INPUTS .................................................................................... 7

RESULTS ................................................................................................................................... 7

Faculty, Adjunct Faculty and Staff ...................................................................................... 7

Internship Sites, Supervision, and Advisory Board .......................................................... 9

Budget and Resources......................................................................................................... 9

EVALUATION OF PROGRAM OUTPUTS (ACTIVITIES) ...................................................... 10

results ..................................................................................................................................... 10

Assessment and Evaluation Process ................................................................................. 10

Enrollment and Diversity .................................................................................................. 10

Course Offerings ................................................................................................................. 11

Mission and Professional Standards .................................................................................. 11

Student Satisfaction ........................................................................................................... 12

EVALUATION OF PROGRAM OUTCOMES .......................................................................... 13

results ..................................................................................................................................... 13

Student learning: Counselor Preparation Comprehensive Examination (CPCE) ......... 13

Student Professional Competencies (PC) as Rated by Faculty and Site Supervisors .... 13

Student Self‐Assessments .................................................................................................. 13

Student Grades, Key Assignments, and GPA ................................................................... 14

Site Supervisor Program‐level feedback ........................................................................... 15

Retention and advancement to candidacy ....................................................................... 15

Key Performance Indicators (KPIs) .................................................................................. 17

Employment and Licensure .............................................................................................. 17

Alumni Survey .................................................................................................................... 17

Employer Survey and Advisory Board Recommendations ............................................. 18

USE OF FINDINGS TO INFORM PROGRAM MODIFICATIONS ......................................... 19

Programmatic Decisions Based on Key Assessments ............................................................. 19

APPENDIX A .............................................................................................................................. 23

2018‐2019 COURSES BY CORE, NON‐CORE fT, AND ADJUNCT FACULTY ................... 23

APPENDIX B .............................................................................................................................. 25

Site Supervisor program‐level feedback ............................................................................... 25

COUNSELOR EDUCATION PROGRAM OVERVIEW

The Clinical Mental Health Counselor Education Program (CEP) is a two‐year, 60 credit

hour curriculum that prepares graduates to pursue licensure as a Licensed Professional

Counselor (LPC). The curriculum consists of 42 credit hours of required coursework in

human behavior and development, evidence‐based practice, effective counseling

strategies, ethical practice, and other core knowledge areas; 9 credit hours of clinical

experience (practicum and internships); and 9 elective hours from selected topics.

Because Milligan is a Christian Liberal Arts College, part of the required coursework

includes an ongoing discussion and examination of the theoretical and practical aspects of

the integration of faith and learning.

PROGRAM EVALUATION OVERVIEW

The annual program evaluation is the culmination of regular and systematic review of

program objectives and performance measures involving all relevant stake‐holders,

including current faculty, staff, adjunct instructors, students, site supervisors, employers,

and an Advisory Board. The evaluation process is guided by a Logic Model, which outlines

the resources, activities, and outcomes as well as the multiple performance measures used

to evaluate the Milligan College Counselor Education Program. The results of the 2017‐

2018 evaluation questions are provided in this report.

Abbreviations used in Program Evaluation defined:

CACREP Council for Accreditation of Counseling and Related Educational Programs

CEP Counselor Education Program

CPCE Counselor Preparation Comprehensive Examination

CMHC Clinical Mental Health Counseling

FTE Full‐time Equivalency

KPI Key Performance Indicators

MSC Master of Science in Counseling

PC Professional Competencies

PIE Practicum and Internship Experiences Coordinator

Figure 1. Logic Model

Figure 1. Counselor Education Program Logic Model

INPUTS OUTPUTS (PROGRAM ACTIVITIES) OUTCOMES (PROGRAM OBJECTIVES)

CEP Faculty Administrative support Adjunct instructors Advisory Board Site supervisors Resources

Offer MSC

degree and

courses matching

professional

standards

MSC Students

ACTIVITIES RECIPIENTS SHORT TERM MID‐TERM LONG TERM

Students demonstrate

excellence in

professional

counseling knowledge

and skills and advance

to candidacy.

Students complete the

program, begin

professional careers,

and obtain professional

licensure.

Alumni are successful as

evidenced by being LPCs

in good standing,

becoming site

supervisors, and taking

on leadership roles in

professional and at‐large

communities

# FTE

Core faculty advocacy, identity and PD

# Staff

# of Adjuncts

# of internships

Budget

ANNUAL REVIEW OF THE FOLLOWING PERFORMANCE MEASURES

Mission &

program

objectives

Curriculum map

Syllabi review

Student

evaluations of

courses and sites

Enrollment

Student demographics

Retention rates

CPCE

PC ratings by supervisors & CEP faculty

Student self‐assessments

Grades, KPIs, GPA

Site Supervisor feedback

Retention rates by semester

KPIs

Graduation Rates

Employment Rates

Licensure Rates

Alumni survey

Employer survey

Advisory Board reflections

EVALUATION QUESTIONS RELATED TO PROCESS AND OUTCOMES

1. Did we have enough qualified faculty, adjuncts and staff?

2. Did we have enough internships sites for our students?

3. Was the budget adequate?

4. Were all assessments collected and reviewed annually?

5. Did we enroll a qualified and diverse cohort?

6. Were all courses offered as planned?

7. Is the program consistent with the mission of the college and aligned with

professional standards?

8. Were students satisfied with the program?

1. To what extent did student knowledge and skills increase?

2. Did students advance to candidacy?

3. Were retention and graduation rates acceptable?

4. Did students find employment in the field if they wanted it?

5. Did students obtain licensure?

6. Are graduates prepared for their work as counselors?

7. Are employers satisfied with the graduates of our program?

8. How many of our graduates are now site supervisors?

EVALUATION TIMELINE

Table 1. Evaluation Timeline

Process Evaluation

Assessment Measure Responsible Party Schedule

# of FTE, staff, and adjuncts

MSC Director‐ Dr. Sapp Annual‐ August

Core faculty advocacy, identity and professional development activities

Core CEP faculty‐ Dr. Browning, Dr. Nefos‐Webb, Dr. Sapp

Annual‐ August

# of supervised internship sites

Practicum & Internship Experiences (PIE) Coordinator‐ Dr. Sapp

End of each semester Dec & May

Funding sources (budget) MSC Director‐ Dr. Sapp Annual‐ August

Review of Mission, Goals, and Objectives

All MSC faculty Annual‐ January

Curriculum map and syllabus review

Counseling faculty Every 2 years‐ Spring, even years

Student evaluations of courses and sites

All MSC faculty End of each semester Dec & May

Student enrollment and demographics

Dr. Nefos‐Webb Annual‐ Sept

Outcome Evaluation

Assessment Measure Responsible Party Schedule

CPCE results MSC Director‐ Dr. Sapp Annual‐ August

Professional competencies as rated by faculty

MSC Director‐ Dr. Sapp Annual‐ January

Professional competencies as rated by supervisors

PIE Coordinator‐ Dr. Sapp Annual‐ January

Growth Edges as rated by students

PIE Coordinator‐ Dr. Sapp Annual‐ January

Site Supervisor Program‐ level Feedback

PIE Coordinator‐ Dr. Sapp Annual‐ June

Retention & graduation rates

MSC Director‐ Dr. Sapp Annual‐ January

Grades, KPIs and GPA Core CEP faculty (as advisors)‐ Dr. Browning, Dr. Nefos‐Webb, Dr. Sapp

End of each semester Dec & May

Advisory Board recommendations &

reflections

MSC Director‐ Dr. Sapp

(Semi‐)Annual meeting

(Jan & Aug)

NCE and/or Praxis passing rates

MSC Director‐ Dr. Sapp As students take them

Licensure rates Administrative Assistant Annual‐ August

Employment rates Administrative Assistant Annual‐ August

Alumni Survey Administrative Assistant First, Third, and Fifth year after graduation‐

January

Employer Survey Administrative Assistant Every 2 years‐ December, odd years

EVALUATION OF PROGRAM INPUTS

Evaluation of program inputs was guided by the following questions:

1. Did we have enough qualified faculty and adjuncts?

2. Did we have enough internships sites for our students?

3. Was the budget adequate?

RESULTS

Faculty, Adjunct Faculty and Staff

Core faculty in the Counselor Education Program (CEP) included Dr. Christine Browning

(Program Director), Dr. Rebecca L. Sapp (Practicum and Internship Experiences (PIE)

Coordinator, and Dr. Shauna Nefos Webb. In addition, full‐time faculty from the related

area of psychology also taught core courses: Dr. Lori Mills, Dr. John Paul Abner, and Dr.

Joy Drinnon. Furthermore, the program used three adjunct instructors to cover core and

elective courses during this year in review: Dr. Greg Wallace, Mr. David Sapp, and Mrs.

Colleen Weems. The ratio of FTE students to FTE faculty for the fiscal year 2018‐2019 was

4.92 to 1; this ration is calculated using Summer, 2018 through Spring, 2019 data. The

program was able to cover all the required courses and one elective and stay under the

maximum ratio of 12:1 with these faculty and adjuncts.

To keep abreast of the current counseling profession’s best practices in today’s

multicultural and pluralistic society, the Milligan College administration supports

financially and with time to attend professional conferences. In 2018‐2019 the core

Counseling faculty were able to attend and/or participate in the following:

Dr. Christine Browning:

(2019) Mastering Differential Diagnosis with the DSM‐5. PESI Continuing

Education training in preparation for teaching COUN 530 Child and Adult Mental

Health Diagnosis, Spring 2020. (6.25 CE)

(2019, July). McDonald, J.D., & Browning, C.M. When it Hurts to Help: A Review of

Secondary Trauma in Helpers. Presented at COUN 680 Research Conference.

Graduate Research/Faculty Mentor

(03/29‐31/19) ACA 2019 Conference and Expo, New Orleans, LA (11.50 CE/1.50

Ethics)

(2019, March 13). Worthington, E., Understanding Forgiveness and Using REACH

Forgiveness in Your Practice. [Organized 6.25 hour seminar and arranged to offer

CE credits through NBCC]

(2018, November). Browning, C.M. How Childhood Trauma Impacts Development.

Lunch & Learn, Milligan College, TN Presenter

(09/27/18) PESI Conference, Knoxville, TN (Live In‐person), Ethical Principles in

the Practice of Tennessee Mental Health Professionals, Allan M. Tepper, J.D.,

Psy.D. (6.25 CE Ethics)

(2018, September). Nefos‐Webb, S., Sapp, R.L., & Browning, C.M. Climbing

Together: Training Future Counselors While Serving Current Students. Presented

at 2018 Appalachian College Association Annual Summit, Kingsport, TN

Dr. Shauna Nefos Webb:

2019 Association for Counselor Education and Supervision Webinar: Human

Sexuality as a Counseling Competency

2019 Association of Multicultural Counseling and Development Webinar: Focus on

the Family:

An Integrative Framework for Counseling Athletes Across the Lifespan

2018 International Association of Addictions and Offender Counselors Webinar:

Let's be

Blunt: Providing Compassionate Care to Individuals Who Use Cannabis

2018 Southern Association for Counselor Education and Supervision fall

conference,

Myrtle Beach, SC

2018 National Board for Certified Counselors Webinar: Adolescent Minorities and

Suicide Risk: 15 Things we often Miss

2018 International Association of Addictions and Offender Counselors Webinar:

How

Counselors Can and Should Respond to the Opioid Crisis

2018 Appalachian College Association: The Summit, Kingsport, TN

2018 Association of Multicultural Counseling and Development Webinar:

Microaggressions and the Intersection of Persons of Color, LGBTQ Identities, and

People with Disabilities: Implications for Counseling

2018 Highlands Community Services Summer Adverse Childhood Experiences

Workshop, Abingdon, VA

2018 Association for Counselor Education and Supervision Webinar: Courageous

Conversations: Unraveling the Taboo

Dr. Rebecca Sapp:

Attended and presented at the National Conference on School Mental Health in

Las Vegas, NV. First, Dr. Sapp presented collaboratively an oral presentation

entitled, “Betting on the Emotional Health of Our Students.” In addition, she also co-led an intensive 3-hour workshop entitled, “Creating Connection as the Sure Bet Against Bullying.”. She attended other sessions about mental health

counseling in the school environment.

Currently in certification track for Gottman certification, receiving supervision by master-trainer Carrie Cole, LPC-S.

Attended the American Counseling Association annual conference in New Orleans, LA. Attended workshops on providing supervision, ethics, and marital and family techniques. March, 2019.

Internship Sites, Supervision, and Advisory Board

In the 2018‐2019 academic year (Fall, 2018 through Summer, 2019), we placed eight

Practicum and eight Internship students at sites both on and off campus. Practicum

students completed a portion of the required 100 hours at the on‐campus Counseling

Center with one of the core faculty providing live supervision and consultation. Four

Interns also completed a portion of their 600 hours providing counseling at the on‐

campus Counseling Center. In addition to the Counseling Center, students completed

hours at ten additional sites off campus. Off‐campus sites included Frontier Health,

Camelot Care Centers, Highlands Community Services, Woodridge Psychiatric Hospital (a

division of Ballad Health), Tri‐cities Behavioral Therapy, and two private practices. This

year, we added ETSU Counseling Center, Vaya Health MCO in NC, and Helen Ross

McNabb to our placement options.

The number of sites was adequate to meet the CEP needs at this time. However, we are

actively seeking more site supervisors to accommodate our growing program and the

addition of the Addictions Counseling concentration. Some of our current sites offer great

opportunities for Addictions Counseling practice, including Frontier Health and Helen

Ross McNabb. We also entered into an agreement with the Veteran’s Administration

Hospital locally to place a Counseling Intern there in the future.

Budget and Resources

The CEP budget was adequate for the 2018‐2019 year. The budget was sufficient to cover

faculty salaries (full‐time) and adjunct faculty needed and a half‐time Counseling Center

assistant. In addition, the budget met faculty professional development opportunities

(attending professional conferences) as well as student learning needs (adequate space

and technology). One of those student learning needs is the campus Counseling Center,

where all practicum and some internship students develop their skills; the budget allowed

for all necessary equipment and supplies to be purchased.

EVALUATION OF PROGRAM OUTPUTS (ACTIVITIES)

Evaluation of program outputs was guided by the following questions:

1. Were all assessments collected and reviewed annually?

2. Did we enroll a qualified and diverse cohort?

3. Were all courses offered as planned?

4. Is the program consistent with the mission of the college and aligned with

professional standards?

5. Were students satisfied with the program?

RESULTS

Assessment and Evaluation Process

In the fall of 2019, we sent out alumni surveys to the 1st, 3rd, and 5th year graduates. We did

not send out the Employer Survey in December of 2019; we will revise this survey and send

it out this Spring semester (2020). We did collect all other assessment data, reviewed, and

decided no major changes are needed at this time.

Enrollment and Diversity

Our goal in the CEP is to recruit as diverse a cohort as possible each year. The cohort

assessed in this report entered Fall, 2018. They consisted of 3 males and 10 females (for

both the CMHC and the SC concentrations). We had little racial/ethnical diversity in this

cohort. The CEP is committed to continue the efforts to enroll qualified candidates who

represent a wide array of underrepresented identities. We are working with the

Admissions office to find new places to recruit for more diversity. Stacy Shankle plans to

be more involved in diverse communities both on and off campus travel (i.e., Goah

Diversity Scholars Program at Milligan, Chamber of Commerce meetings) send a mailing

to churches with diverse backgrounds, visit colleges/universities that have more diversity

on their campuses (i.e., Appalachian State, UTK, lots of community colleges), attend the

juvenile justice conference in Nashville and meet with agency workers there.

MSC Program Student Demographics 2012‐2018

COHORT YEAR of ENTRY

TOTAL

ADMITTTED

MALE

FEMALE

RACE

STATE/COUNTRY OF

ORIGIN

2012 9 3 6 9 C Tennessee/USA

2013 19 3 16 16 C, 1 AA, 1 Bi,

1 H

Kentucky, North Carolina, Tennessee, Virginia/USA; The

Netherlands

2014 10 1 9 10 C Tennessee/USA

2015 10 1 9 8 C, 1 AA, 1 B

Florida, Michigan, North Carolina,

Tennessee/USA; Africa

2016 12 3 9 10 C, 1 AA, 1 Bi

Colorado, North Carolina, Tennessee/USA

2017 12 5 7 10 C, 2 AA

Tennessee; Illinois

2018 13 3 10 12 C, 1 AA

Tennessee, Michigan, NC/USA

C = Caucasian; AA = African American; B = Black; Bi = Biracial; H = Hispanic

Course Offerings

A table listing all the courses offered in the 2018‐2019 calendar year is located in Appendix

A. All courses were offered as planned. Core CEP faculty offered 66% of the courses. This

meets a goal of core faculty teaching the majority of classes in the program. A core faculty

member began teaching COUN 620M in the summer, 2018, giving core faculty a larger

percentage of required courses.

Faculty reviewed the syllabi this spring (2019) for the CACREP addendum. We will review

them again Spring, 2020, and report the review and subsequent changes in the 2019‐2020

Program Evaluation.

Mission and Professional Standards

The CEP faculty reviewed the program in relation to the mission of the college. We see

the program still aligning well with the scholarship, faith, and community goals of

Milligan College.

In February, 2019, we had our site visit from the Council for Accreditation of Counseling

and Related Educational Programs. At the end of July, 2019, we received our letter stating

our Clinical Mental Health Counseling program received a two‐year accreditation status.

We need to collect data, analyze data, and utilize data from our Key Performance

Indicators and submit a report to CACREP no later than April, 2021. The report will be

reviewed in July, 2021, and if it is approved, Milligan will be accredited for an additional 6

years to be on an 8‐year cycle. This accreditation demonstrates our CEP meets the highest

professional standards. We will also continue to seek input from our site supervisors and

MSC Advisory Board to stay abreast of developments and needs in the profession.

Student Satisfaction

After every semester course, students complete course evaluations in which they provide

input regarding their satisfaction with course objectives, course content, the instructor’s

teaching, helpfulness, and use of technology. For the year in review, response rates varied

from 30‐75% of students in each course completing evaluations. The overall response rate

across the institution was low; a committee is considering ways to increase the response

rate. The students who did respond reported overall satisfaction with instructors and the

material covered. Some general comments made included the students wanting quicker

feedback on assignments and some negative feedback around specific assignments. No

major changes were made in courses. However, the faculty made adjustments in course

assignments, dates a summer course met, added more options of assignments meeting

objectives, and brought in more guest speakers.

One change we made based on student and feedback from one of our Adjunct Instructors

who is also a site supervisor. We now require student videos demonstrating general

counseling skills to be completed in COUN 520 (Theory and Practice of Counseling)

instead of COUN 510 (Intro to the Counseling Profession). This allows for core CEP

faculty to evaluate the skills of both our CMHC and SC candidates and also better

prepares them for their Practicum experience in our campus Counseling Center.

In addition to course evaluations, students also complete an evaluation on their Practicum

or Internship Experience and site supervisor. In 2018‐2019, 100% of the students rated

their sites and site supervisors as good or excellent. Several students had very positive

comments specific to their supervisors (‘very kind in providing feedback”; “showed she

really cared for me”) and reported good development of their counseling skills. If a

student rated a site supervisor as poor, the PIE Coordinator would meet with the student

to address specific concerns; in most situations of this low of a rating, future students

would not be placed with that site supervisor again.

EVALUATION OF PROGRAM OUTCOMES

Evaluation of program outcomes was guided by the following questions:

1. To what extent did student knowledge and skills increase? 2. Did the student advance to candidacy? 3. Were retention and graduation rates acceptable? 4. Did students find employment in the field if they wanted it? 5. Did students obtain licensure? 6. Are graduates prepared for their work as counselors? 7. Are employers satisfied with the graduates or our program? 8. How many of our graduates are now site supervisors?

RESULTS

Student learning: Counselor Preparation Comprehensive Examination (CPCE)

Program faculty have decided to use the CPCE as one of the ways to assess that our

students have the knowledge needed to be well‐educated counselors. The Fall 2018 cohort

will be the first to take this examination; they will take the exam Summer, 2020.

Student Professional Competencies (PC) as Rated by Faculty and Site Supervisors

Faculty and site supervisors completed the Professional Competencies Rating Forms (PC)

for each student. Consistently, students are achieving the goal of 3s (meets expectations)

and 4s (exceeds expectations) on the professional competencies. Two students were rated

below the goal; both of these students’ advisers met with them to address the behavior

that needed improvement. For both of these students, the problem identified was in

accepting feedback; both made improvements the next semester. In the Spring and

Summer semesters, 100% of the students met or exceeded expectations.

This was the first year (2018‐2019) we used an electronic version of the PC Form with

electronic signatures as an option. In addition, the form requires an explanation if the

student receives any rating other than a 3; this helps provide more specific feedback to

students and faculty. Site Supervisors still struggle with technology, and we have had to

assist them in completing this form electronically. We also hand enter the data on an

Excel sheet to then aggregate our information for Program Evaluation. We plan to try to

use Qualtrics after the Fall, 2019, semester to see if the collection and aggregation of data

is simplified.

Student Self‐Assessments

Students complete self‐assessments at mid‐term and end of the semester using questions

posted by their instructors that allow for reflection and open‐ended narration. Students

provide insight into their strengths and growth edges; in addition, they also have a place

for providing input as to what they need from the practicum/internship class and

instructor’s supervision (i.e., topics they may need more information regarding, personal

triggers to be discussed in supervision). Students reported this year an appreciation for

this style of self‐assessment. They reported they paused more to consider their progress in

skill development and confidence in their abilities. The self‐assessment posted in Canvas

is:

Pause, reflect, and self‐monitor/assess. Answer these questions:

1. Looking at the Professional Competencies and considering the level I should be,

on a scale of 0‐10, I think my overall level of competency is a ....... because......

2. Thinking of my rating above, I can improve my overall level of competency by......

3. Two specific skills I find are my strengths are..... as evidenced by.....

4. One or two specific skills I find are my "growth edges" are .... as evidenced by .....

5. One thing I have needed and/or appreciate about my site supervisor at this time

is.....

6. One thing I am missing from my site supervisor at this time is....

7. One thing I have received from and/or appreciated about this class is.....

8. One thing I have needed but not received from this class is.....

9. My biggest take‐away from this semester at this point is....

10. What else would you like us to talk about or me know about you and your

practicum/internship at this point?

Answers have been mostly positive in their skill development. The students reported

needing more confidence and practice, more techniques, and more knowledge around

certain populations (i.e., addictions, very young children, families). Overall, students

reported they see evidence of growth in themselves such as being more directive when

needed, being able to conceptualize a problem with clients according to chosen theories,

and feeling more comfortable staffing cases.

Student Grades, Key Assignments, and GPA

All of the students progressed through the courses successfully for the year in review,

earning 3.0 and better GPAs each semester with cumulative GPAs being 3.5 and above. In

addition, all students completed satisfactorily the Key Assignments as identified by

faculty. The CEP faculty have identified per CACREP standards Key Performance

Indicators that demonstrate 1) individual student‐level to assess students’ knowledge and

review progression throughout the program as well as 2) program‐level information of

where we excel and where we need improvement. CACREP reviewers provided feedback

that they approve the system we have in place for collecting data on Key Performance

Indicators; however, as this system was newly in place Spring, 2019, we have not had a

chance to begin collecting nor analyzing this information. We will begin this collecting

process Fall, 2019.

Site Supervisor Program‐level feedback

At the end of each semester, one of the CEP faculty meet with each site supervisor to

review both students’ individual progress as well as elicit feedback regarding the MSC

program. Site Supervisors consistently tell us Milligan College students are well‐prepared

for the field experience, being well‐versed in professional theories and language, and

having a strong articulation of professional identity. We received no feedback on

deficiencies or areas to improve this past year. We will continue to seek feedback in this

same manner (see questions used to solicit site supervisor feedback in Appendix B).

Retention and advancement to candidacy

We now have created an Excel document to help us better track and aggregate

information about how students progress through the program. We have identified

“Points of Progression”, and CEP faculty review the student information (i.e., GPA, PC

ratings, etc.) and mark the decision of whether the student is approved, provisional, or

denied progression in the program. These points of progression are defined in the

Student Handbook.

This year we had our lowest retention rate‐ 64%. The Program faculty met to discuss

reasons and solutions to improve the retention rate to at least 85%. One plan we are

implementing is addressing any weaknesses either in skills, academics, or dispositions

very early. We are documenting them in a file on a FERPA‐compliant drive that allows us

each to see where problems are in each course and catching patterns early. When a

problem is identified, the adviser meets with the student to brainstorm together resources

that may help. In 2018‐2019, we referred students to writing tutors, broke down

assignment deadlines to assure on‐time completion, and connected students to outside

mental health counselors. At this time, we have had 2 students need remediation in the

above manner, and we have been able to retain all of the Fall 2018 cohort. We will

continue to monitor and look for ways to identify 1) in advance who will succeed in the

program (refining our admission criteria accordingly) and 2) additional supports we can

put in place after acceptance into the CEP.

Retention, Remediation, Graduation Chart by Cohort

Cohort

Year

Admitted

into

CEP/CMHC

Withdrew

Personal

Reasons

Remediation Failed

Out of

Program

Dropped

to Part‐

time

Withdrew

or

Dismissed

from

Program

for PC or

Academic

Integrity

Re‐

Entered

Program

Completed

(%)

2012‐

2014

9 9 (100%)

2013‐

2015

19 4* 1 (did not

pass

COUN

520 twice)

(appealed;

denied)

1

(completed

Dec., 2016)

2 add on

SC;

*1 re‐

entered

(completed

Dec., 2017)

15 (79%)

2014‐

2016

10 2 8 (80%)

2015‐

2017

10 1* 1 (PC; took

Spring, 2017

off;

completed,

Dec., 2017)

1

(Practicum‐

remediation;

internship

PC‐

withdrew

before

dismissed‐

Fall, 2017)

*1 re‐

entered

Fall, 2018

9 (90%)

2016‐

2018

12‐ CMHC 1 (retook

COUN 510;

completed)

1 11 (92%)

2017‐2019

11‐ CMHC 1 3 7 (64%)

2018‐2020

10‐ CMHC 2 (personal; retook COUN 690)

Key Performance Indicators (KPIs)

In order to assess students’ mastery of knowledge required to be successful counselors,

core faculty identified Key Performance Indicators in January, 2019. These were not in

place in the year under review in this report. We have created an e-Portfolio course in Canvas as a place for students to upload the KPI assessment measures. The CEP faculty will use this course to aggregate data, report results, and use to make program adjustments as needed. The CACREP site visit team reviewed this plan in February, 2019, and commented they were impressed with the plan. Unfortunately, we do not have data yet, and therefore cannot demonstrate we use the results to evaluate the Program. We will begin requiring students in both cohorts to upload assignments in the Fall, 2019, semester. We will report results in the 2019-2020 Program Evaluation report.

Employment and Licensure

As of December, 2019, the MSC program has had 59 CMHC graduates and 20 School

Counseling graduates; 5 of those were dual enrollments. School Counseling graduates

have 100% licensure rate and 100% job placement rate (for those actively seeking

employment in their field). Of the CMHC students who have reported taking the NCE,

100% passed (2 reported passing on 2nd attempt), and of those desiring a job in the

professional field of mental health counseling, 97% are employed at this time.

Alumni Survey

We revised and sent two versions of the alumni survey during the Fall of 2019. The first

one was to those who graduated in 2018, and the second was to those who graduated 3 and

5 years prior. From those who graduated in December, 2018, 11 of them began their CEP in

the Fall, 2016, and one began her CEP in the Fall, 2015. Unfortunately, only 3 of these

completed the survey, all from the 2016 cohort. One suggestion made by an alumnus was

for all students to have at least one course in Substance Abuse as this is often a co‐

morbidity in mental health treatment. The respondents did not rate any classes as not

being useful, and they strongly agreed that their professors helped them integrate their

faith with their professional practice as a counselor. We have already discussed ways to

improve our response rate, including adding an incentive such as a gift card or chance at a

prize.

On our revised Alumni survey for 3rd and 5th year post‐graduates, we included questions to

answer our long‐term goals of 1) being LPCs in good standing, 2) becoming site

supervisors, and 3) taking on leadership roles in the profession and community. Out of 20

alumni surveyed, only 7 returned completed surveys. Of these responding, the following

were notable results addressing 2) and 3) of the long‐term goals:

3 have received LPC designation

1 respondent is a member of a professional counseling association

3 respondents are members of a community organization

1 holds a leadership role in a community organization

1 is a program manager in their workplace

2 have presented at professional conferences or workshops

3 have given mental health presentations in their workplace

3 have given mental health presentations in the community

2 supervise practicum or internship students

2 provide training for other counselors

1 is enrolled in or has completed a doctoral program

Overall, these 7 alumni are meeting or working toward our long‐term goal of being

successful individuals.

Employer Survey and Advisory Board Recommendations

The Counseling faculty and Advisory Board decided in January to only send the employer

survey in odd years. We will send out a revised Employer Survey before the 2019‐2020

Program Evaluation. Advisory Board recommendations have been explained throughout

this document, including improvements in training site supervisors, student self‐

assessments, and ways to contribute to professional development for area professionals.

USE OF FINDINGS TO INFORM PROGRAM

MODIFICATIONS

Programmatic Decisions Based on Key Assessments

This table highlights change points in programmatic or instructional design and identifies

either the policy or the data sets associated with those decisions.

Date Change in Program or Instructional Design

Policy or Data Set Associated with Change

Aug., 2019 Group Counseling (COUN 610) will be moved back to May term beginning May, 2020

Student feedback‐ course overlaps with 2 other courses; students report feeling overwhelmed with the material in the summer semester of their 1st year

Aug., 2019 NCE to be offered as a choice of assignment in COUN 648; to begin Summer, 2020

MSC Advisory Board members suggestion as a way to encourage students/graduates to take this next step toward licensure

July, 2019 Received CACREP accreditation status for two‐year period. Report addressing deficit areas to be submitted by April, 2021

CACREP Board decision

May, 2019 Milligan Cabinet approved Addictions Counseling track in MSC to begin August, 2019

Feedback from community leaders that need for Addictions Counselors in our area is high; Milligan Cabinet request of faculty

May, 2019 Counseling skill videos to be submitted and reviewed in COUN 520 instead of COUN 510 to enhance skills development of both CMHC and SC candidates

Student feedback; Site Supervisor/Adjunct Instructor feedback

Jan., 2019 Built e‐Portfolio course in Canvas where students will house KPI information; Canvas allows for evaluation of mastery level and aggregating information assisting in Program‐level evaluation

CACREP reviewers’ feedback; input from all CEP faculty

Jan., 2019 Defined Key Performance Indicators using multiple measures over multiple times for each KPI

CACREP feedback; core CEP faculty reviewed; input from all CEP faculty

Dec., 2018 Revised Logic Model CACREP feedback; faculty review

July, 2018 Change in MSC Program Director‐ Dr. Sapp became new director

Several factors

Summer, 2018

Advisors make students aware Addiction Counseling course is

Site Supervisor program‐level feedback

required to be hired in VA. Faculty will be sure this elective course will not conflict with other required courses.

Summer, 2018

COUN 610‐ Group Dynamics and Group Counseling offered in 8‐week session instead of 3‐ week session

Student feedback

Summer, 2018

COUN 620M‐ Career Counseling will be offered by core CEP faculty

Alumni feedback as well as Program Evaluation for % of courses taught by core faculty/other faculty

Jan., 2018 Systematic Program Evaluation put in place

Response to accreditation application

Jan, 2018 Revised Professional Competencies Rating Form‐ rubric, dispositions, skill levels

Site supervisor feedback; Advisory Board recommendations; student feedback

Jan, 2018 Revised Self‐Assessment Ratings‐ narrative responses instead of scales

Faculty and Advisory Board recommendations

Jan., 2018 Established and convened first MSC Advisory Board comprised of current students, former students, and area counseling professionals

In response to self‐study and reviewers’ comments‐ need for input into CEP from current students, former students, and area counseling professionals

Feb., 2017 Dropped COUN 553‐ Theories of Personality from CMHC required courses and made it an elective course; Added COUN 510‐Introduction to the Counseling Profession to the MSC/CMHC list of required courses; Changed course title of COUN 550 back to Legal and Ethical Issues

In response to the self‐study, CEP faculty discussed need to add a separate course to address professional standards instead of covering too much in COUN 550

Jan., 2017 Revised interview process for pre‐admission candidates

In response to retention rate review, decision was made by CEP faculty to adjust admission requirements and process to select better qualified students who would be successful in the program

May, 2016 Proposed to change course name, description, and SLOs of COUN 550 from Legal and Ethical Issues to Professional Orientation and Ethical Practice

As a way to better address Professional Standards

April, 2016 Proposed Course Substitutions including CMC 7070 for COUN 650; CMC 6030 for COUN 540; and CMC 6210 for COUN 625; Expanded Program Learning Outcomes from the original 4 to 11

Provide a greater range of options for students in the MSC program or students already holding a MDiv to transfer into the program; Modified Program Outcomes to address Professional Standards for CEP programs

Mar., 2016 Proposed new electives to offer MSC students more options‐ CMC 6010‐ Ministry to the Aging and Their Families; CMC 6020‐ Human Sexuality; CMC 6030‐ Counseling and Multiculturalism; CMC 6200‐ The Church and Family Formation; CMC 6210‐ The Church and Marriage and Family Therapy; CMC 7070 Suffering and Christian Care and Counseling

Response to student requests for more electives

Jan., 2016 New elective course proposed‐ COUN 660‐ Creative and Expressive Arts in Counseling

Response to student feedback requesting another elective and specifically one exploring “non‐traditional” counseling techniques

Aug., 2015 Began self‐study in order to apply for CACREP accreditation

In order for CEP to be more marketable and keep up with Professional Standards

Aug., 2015 Dr. Browning‐ Director of CEP; New hire: Shauna Nefos Webb/ CACREP Ph.D. Counseling & Student Personnel Services

Dr. Mills (Licensed Clinical Psychologist) returned to Director of Undergraduate Psychology program

July, 2015 Annual Campus Poster Presentation Conference for CEP graduate students launched

In response to need for more CEP student involvement in research‐related and counseling profession‐promoting activities

Aug., 2014 Updated CEP retention & remediation policy

In response to review of current policies in comparison to Professional Standards

Aug., 2014 On‐campus Counseling Center began‐ used for Practicum and Internship Counseling training‐ Dr. Browning‐ Director

Student feedback; faculty recommendations‐ needed a way to observe student counseling skills and provide live supervision per Professional Standard; also service to campus community

April, 2014 Established role of Coordinator of Practicum and Internship Experiences (PIE) with job description; appointed Dr. Sapp as PIE Coordinator

Professional standard to have this position

Sept., 2013 Modified Practicum hours‐ decreased the required number of Practicum hours from 150 to 100

Professional standard requirement for Practicum hours is 100; to not overburden students with Practicum level hours, program decreased to allow students to move to Internship‐level hours sooner

Aug., 2013 New hire: Dr. Browning /Ph.D. in CACREP Counselor Education and Supervision

Needed full‐time CEP faculty to replace Dr. Schnyders, who took a faculty position in OH

Jan., 2013 Modified Internship hours‐ raised the required number of internship hours from 400 to 600.

Professional standard requirements‐ began working toward CACREP accreditation

Fall, 2012 Counselor Education Program launched‐ Dr. Lori Mills (Program Director); Dr. Rebecca Sapp (Practicum and Internship Experiences Coordinator); Dr. Christina Schnyders; original number of required practicum hours was 150 & internship hours was 400

Met State of TN minimum practicum & internship hours required (required was 500 total)

June, 2012 MSC Catalog description submitted to Academic Committee

Full‐time faculty and prospective faculty recommendations after reviewing other Counselor Education Programs

Sept., 2011 MSC Course Descriptions and Course Learning Outcomes proposed to Academic Committee; SACS prospectus submitted

Full‐time faculty and prospective faculty recommendations after reviewing other Counselor Education Programs

APPENDIX A

2018‐2019 COURSES BY CORE, NON‐CORE FT, AND ADJUNCT FACULTY

Summer 2018

Course Offered Core Faculty (Hours) Full‐Time, Non‐Core

Faculty (Hours)

Adjunct Faculty

(Hours)

Summer‐

Required

COUN 500M Abner (3)

COUN 600 Sapp (.5)

COUN 610 Browning (3)

COUN 648 Abner (3)

COUN 680 Drinnon (3)

COUN 692 R. Sapp (3)


Summer‐Elective

COUN 660 Nefos Webb (3)

Total Credit

Hours Offered =

21.5

12.5

9

0

Fall 2018


Faculty (Hours)

Adjunct Faculty

(Hours)

Fall‐ Required




COUN 550M Nefos Webb (3)

COUN 600 Nefos Webb (.5)


COUN 692 D. Sapp (3)


Fall‐ Elective


COUN 645 Weems (3)

Total Credit

Hours Offered =

27.5

18.5

9

Spring 2019


Faculty (Hours)

Adjunct Faculty

(Hours)

Spring‐Required

COUN 530 Abner (4)


COUN 600 Drinnon (.5)






Spring‐ Elective

COUN 527 Drinnon (3)




Total Credit

Hours Offered =

34.5

24

7.5

3

FTE calculated= 4.64; core Counseling faculty offered 66% of course credit hours

APPENDIX B

SITE SUPERVISOR PROGRAM‐LEVEL FEEDBACK

Some site supervisors have seen only one student; some site supervisors have worked with many students from the Counselor Education Program at Milligan College. We are interested in your OVERALL experience with our students.

1. On a scale of 0-5 where 0 is not at all and 5 is completely agree, which describes how much you agree with the following statement: Overall, Milligan College MSC students are well prepared to enter the professional field of Counseling in today’s multicultural and pluralistic society?

2. With what techniques do you wish our students would have been more familiar?

3. With which theories do you wish our students would have been more familiar?

4. What populations or projected needs do we need to address more in our CEP?

5. On the whole, what professional competencies do our students need to improve upon that program faculty should emphasize?

6. What additional professional competencies do you suggest we add to our evaluation?

7. In addition to the comment above, how can we improve the evaluation instrument we

provide for you?

8. In what ways do you suggest we improve communication from program faculty?

9. Any other comments you can add to help improve the Master of Science in Counseling

Program at Milligan College is greatly appreciated.

Documents

Program Evaluation 2018-2019 - Milligan College · 2018 Association for Counselor Education and Supervision Webinar: Courageous Conversations: Unraveling the Taboo Dr. Rebecca Sapp: