University of Kentucky College of Health Sciences ... · practicing physician assistant. Students will demonstrate the medical knowledge needed to effectively practice as an entry-level

MSPAS Assessment Plan 1

University of Kentucky College of Health Sciences Department of Clinical Sciences

Division of Physician Assistant Studies (PAS)

Assessment Plan for the Masters of Science in Physician

Assistant Studies (MSPAS) Degree

College of Health Sciences

Spring 2016

Assessment Plan for the Masters of Science in Physician Assistant Studies (MSPAS) Degree 2

Physician Assistant Studies Assessment Plan - MSPAS

1. Introduction and Program Overview

The Physician Assistant Studies Program at UK was established in 1973 and is one of two Physician Assistant

programs in the Commonwealth of Kentucky fully accredited by the Accreditation Review Commission on

Education for the Physician Assistants (ARC-PA), which is recognized by the Council for Higher Education

Accreditation (CHEA). The Master of Science in Physician Assistant Studies degree is conferred by the UK Graduate

School for students at the UK Lexington campus and at the rural campus located at Morehead State University.

Graduates are eligible to take the Physician Assistant National Certifying Examination (PANCE). After successful

completion of the examination, they are eligible for state certification and licensure to practice as certified

physician assistants. Each year, an average of 40 students are admitted to the Lexington campus and 16 to

Morehead campus.

1.1. Mission Statement

The mission of the UK Physician Assistant program is to educate competent physician assistants committed to

the delivery of high quality and compassionate health care in the Commonwealth of Kentucky and beyond.

1.2. Basic Assessment Approach

Assess 3-5 learning outcomes within a three year cycle, using direct methods for representative courses in the

curriculum. Please see the Curriculum Map and Learning Outcomes Matrix included in this document.

1.3. Definition of Key Terms

PAS – Division of Physician Assistant Studies

CHS – College of Health Sciences

DD – Division Director – Administrative chief of the PAS program

ARC-PA – Accreditation Review Commission on Education for the Physician Assistants

NCCPA – National Committee on Certification of Physician Assistants

PANCE – Physician Assistant National Certification Examination

Curriculum Map: A visual depiction of how learning outcomes and/or professional standards are translated

into individual courses taught within a program

Direct Evidence: Students show achievement of learning goals through performance of knowledge and skills.

Includes, but is not limited to: capstone experiences, score gains between entry and exit, portfolios, and

substantial course assignments that require performance of learning

Indirect Evidence: Data from which you can make inferences about learning but do not demonstrate actual

learning, such as perception or comparison data. Includes, but is not limited to: surveys, focus groups, exit

interviews, grades, and institutional performance indicators

Student Learning Outcomes (SLOs): Statements of learning expectations

2. Assessment Oversight & Resources

2.1 College Learning Outcomes Assessment Coordinator

College of Health Sciences (CHS) Office of Assessment.



2.2. Unit Assessment Coordinator

Program Academic Coordinator (AC) in consultation with the program’s Director of Graduate Studies and DD.

The AC is responsible for coordination of efforts with the College’s Office of Assessment.

2.3. Other Assessment Resources

The PAS program administrative assistants provide clerical support to implement this Assessment Plan. In

addition, PAS faculty members provide input and support during planned PAS faculty meetings in the early

segment of the summer/early fall semester at which time the assessment data and improvement action plan

are reviewed and discussed.

3. Program-Level Learning Outcomes

3.1 Learning Outcomes

1. Students will demonstrate competence in patient evaluation, management and treatment at a level of a

newly practicing physician assistant. These skills will be evidenced by their ability to: a) Elicit an

appropriate targeted patient history, b) Perform a patient-appropriate comprehensive physical

examination, c) Construct an appropriate differential diagnosis, d) Formulate an appropriate assessment

and treatment plan, and e) Effectively interact with the standardized patient using appropriate

examination and communication skills.

2. Students will demonstrate the medical knowledge needed to effectively practice as an entry-level physician

assistant.

3. Students will apply the concepts of systems-based practice to improve healthcare delivery.

3.2 Accreditation Standards/Outcomes by Program (if applicable)

The graduate program in Physician Assistant Studies is accredited by Accreditation Review Commission on

Education for the Physician Assistant (ARC-PA). While the assessment plans may be used as supporting

documentation for the accreditation self-study, the processes are conducted separately.

4. Curriculum Mapping - (I= introduce, R = reinforce, E = emphasize)

SLO #1 SLO #2 SLO #3

Students will demonstrate

competence in patient

evaluation, management

and treatment at a level of

a newly practicing

physician assistant.

Students will

demonstrate the

medical knowledge

needed to effectively

practice as an entry-

level physician

assistant.

Students will apply

the concepts of

systems-based

practice to improve

healthcare delivery.

Course Course Type

HSM 601 Didactic I I I

PGY 412G Didactic I I

ANA 611 Didactic



PAS 651 Didactic I I I

PAS 610 Didactic I I

PAS 678 Didactic I I, R


PAS 654 Didactic I, R I, R




PAS 645 Didactic R

PAS 658 Didactic R, E I, R

PAS 656 Didactic R, E I, R

PAS 673 Didactic R I, R


PAS 646 Didactic R, E


CNU 503 Didactic I I

PAS 660 Clinical R, E R, E










PAS 680 Clinical R, E

5. Assessment Methods & Measures

SLO #1 SLO #2 SLO #3

Students will demonstrate

competence in patient

evaluation, management and

treatment at a level of a newly

practicing physician assistant.

Students will demonstrate the

medical knowledge needed to

effectively practice as an entry-

level physician assistant.

Students will apply the concepts

of systems-based practice to

improve healthcare delivery.

Standardized Patient Clinical

Assessment

PANCE Examination

Final Exam Healthcare Vignette



5.1 Direct Methods/Measures

SLO #1 – Standardized patient

a) Elicit an appropriate targeted patient history,

b) Perform a patient-appropriate comprehensive physical examination

c) Construct an appropriate differential diagnosis

d) Formulate an appropriate assessment and treatment plan,

e) Effectively interact with the standardized patient using appropriate examination and communication skills.

Individual students are evaluated both by a faculty member as well as by standardized patients (SP) using a

standardized rubric for a checklist of items students are expected to cover. The rubric evaluates interpersonal

communication, professionalism as well as medical care. This tool is administered during the SP interview.

Students are tested in the standardized patient facility housed on campus. This facility has 11 standardized patient

rooms where 11 students can be tested simultaneously. The rooms are equipped with video observation and

recording capabilities so that encounters can be monitored, recorded for review by faculty and students. The

training of the standardized patients is done in a uniform and consistent basis by the same team for all specialties.

Subjective, Objective, Assessment and Planning (SOAP) notes are graded by faculty as part of the standardized

patient exam. Standardized patients are specialty trained to evaluate our students on the physical exam skills as

well as professional behavior and communication expected during a patient encounter.

SLO #2 – Physician Assistant National Certifying Examination (PANCE)

The PANCE exam is a computer-based, timed test comprised of 300 multiple-choice questions assessing medical

and surgical knowledge required to become a certified physician assistant. The PANCE exam is administered in 5

blocks of 60 questions each, with 60 minutes provided for the completion of each block. Exam questions fall into

two categories: (1) organ systems and diseases, disorders and medical assessments that relate to those systems;

and (2) knowledge and skills needed to confront those diseases, disorders and assessments. The PANCE exam is

administered by computer year round.

SLO #3 – Healthcare Clinical Vignette

As part of the HSM 601-Overview of the United States Health Care Delivery course, students are asked to review

and successfully complete a written examination question via a clinical vignette that evaluates the systems-based

practice competency. Systems-based practice encompasses the societal, organizational and economic

environments in which health care is delivered. The clinical vignette centers on communication deficits that occur

between the patient, a variety of providers as well as between health services. For successful completion of this

exercise, 1st year PA students must be able to demonstrate an awareness of the communication deficits woven

throughout the vignette, the ’person-centered’ errors that led to the clinical misadventure, as well as the ‘systems-

based’ solutions that could have prevented harm to the patient. This experience is situated during the first

semester of PA school for our students and is a critical foundational component for student learning. The UKPAS

program is committed to evaluating this competency longitudinally throughout the program as a way to for our

students to formatively expand their knowledge and skills relative to the important competency of systems-based

practice.



5.2 Indirect Methods/Measures

All student learning outcomes use direct measures of student learning.

6. Data Collection & Review

6.1 Data Collection Process/Procedures

6.1.1 When will data be collected for each outcome?

6.1.2 How will data be collected for each outcome?

6.1.3 What will be the benchmark/target for each outcome?

Learning

Outcome Cycle 1 Cycle 2

SLO #1 2017-2018 2020-2021

SLO #2 2015-2016 2018-2019

SLO #3 2016-2017 2019-2020

Learning

Outcome Collection of data

SLO #1 Standardized patient clinical assessment in the terminal month of the

program (June)

SLO #2 Compilation of PANCE scores in the Fall (July – November)

SLO #3 Final Exam in HSM 601 – Introduction to the Healthcare Delivery System

Learning Outcome Collection of data Rationale

SLO #1

Students are expected to score

>60% on the standardized patient

(SP) checklist as well as the faculty

assessment.

If a student chose the examination

option, our target is that all

students must pass the exam with

a 60% or higher score based on the

passing score of the SP and faculty

scoring checklist and rubric.

The benchmark of >60% has been the

standard the program has been using on

clinical rotation examinations as well as the

summative/ masters examination.

SLO #2

Benchmark is to meet/ exceed the

national pass rate for first-time test

takers

This benchmark is used by almost all PA

programs nationally as the surrogate

measure of student/ program success

SLO #3

Students are to perform at a level

of 70% or greater on the final

vignette examination

>70% is accepted by the graduate school as

the benchmark value for passing



6.1.4 What individuals/groups will be responsible for data collection

SLO #1 - DGS and AC DGS and Assessment Coordinator

SLO #2 – AC Assessment Coordinator

SLO #3 – Course instructor and the Assessment Coordinator

7. Assessment Cycle and Data Analysis

7.1 Assessment Cycle (3 years)

The PAS Program follows a three year assessment cycle, with 1 outcome assessed each year. Data are

compiled and analyzed in the early summer/fall semester each year for the outcome being assessed. Drafts

of the results are analyzed and interpreted by the Unit Assessment Coordinator in conjunction with the

College Learning Outcomes Assessment Coordinator in late summer/early fall semester. Assessment

reports will be completed no later than October 1st of every year and turned in to the College Learning

Outcomes Assessment Coordinator for review. Final reports are sent to the university’s assessment office

no later than October 31st of every year.

7.2 Data Analysis Process/Procedures

1. SLO #1: This learning objectives utilizes standardized patients as well as faculty to assess medical

knowledge, professionalism and interpersonal communications. Standardized patient testing

information is collected immediately after the test is performed in the ‘clinical testing & simulation

center’ by faculty & staff. Rubrics are completed on each student by both one faculty member and

one standardized patient. The areas of testing include: history taking & physical exam skills, clinical

reasoning, presentation skills, professionalism, oral & written communication. Trained

standardized patients complete a rubric evaluating student (clinical) skills, and the faculty

completes the rubric on other areas (as described above). It is important to note that the UKPAS

program will be moving towards one examination for the graduate/ summative examination

utilizing a standardized patient clinical encounter. For future examinations, and in the spirit of

using one schema for assessment, the Program is moving towards a sole SP examination. The

program will be able to evaluate the professional competencies of professionalism, medical care as

well as elements of verbal/non-verbal interpersonal communication in a way more closely

resembling clinical practice.

SLO #2: Students who successfully satisfy all required elements to confer the MSPAS degree are

eligible to sit for the PANCE examination. Results from each class cohort are not completed in its

entirety until the 4-6 months from graduation date. Once individual results are sent to the

program, a comparison of the class cohort to the established first time pass rate of all national

takers is performed. The UKPAS program’s benchmark for this outcome is to meet or exceed the

national average. Student performance is analyzed based on the organ systems and the diseases,

disorders and medical assessments physician assistants encounter within those systems. Student

performance relative to their knowledge and skills that physician assistants should exhibit when

confronted with those diseases, disorders as well as their assessment of these clinical systems is

analyzed.



SLO #3: This learning objective is designed to assess individual student development of the system-

based practice competency. This competency is centered on concepts such as advocacy germane to

quality patient care, assisting patients with the system complexities of health care, recognizing

health system deficiencies that promote inefficiency and waste. As part of the final examination in

the HSM 601 course, a complicated clinical vignette is constructed in which students will need to

identify system based flaws, complexities, as well as identify potential systems-based solutions

promoting care that is safe, timely and efficient. Students will be assessed in their ability to

correctly identify the person and systems-based flaws contained within the vignette, but also their

ability to offer appropriate system-based solutions.

7.2.1 How will the data and findings be shared with faculty?

Results are presented to the PAS faculty members during the summer/early fall of each year. This

opportunity allows for faculty members to reflect on the outcomes of the annual assessment of the courses

inspected. Faculty members develop strategies for addressing concerns raised during the assessment

process.

7.2.2 Who was involved in analyzing the results?

The Unit Assessment Coordinator is responsible for the initial data analysis. The assessment is conducted

in consultation with the Curriculum Committee and course instructors, DGS and Program Director (where

applicable).

7.2.3 How are results aligned to outcomes and benchmarks/targets given?

When assessment results are presented to faculty for review and reflection, they are shared in comparison

to the benchmark/target for the learning outcome under review. As part of this process, the

benchmark/target is also reviewed to determine whether it should be altered or remain the same based on

the assessment results.

7.2.4 How will the data be used for making programmatic improvements?

Feedback reports are shared with the faculty in the spring. During a mid-spring faculty meeting, the faculty

identify strategies to address identified deficiencies. Faculty identify if strategies can be universally

implemented to affect more than just the course that was analyzed in the preceding summer/fall semester.

7.3 Data Analysis Report Process/Procedures

Each degree-granting academic program within the College submits a six-year assessment plan as part of

the UK periodic review process. The assessment plan template is provided by the UK Office of Assessment.

The assessment plans contain two three-year assessment cycles of student learning outcomes, curriculum

maps and assessment tools. The completion of the annual assessment reports and improvement action

plans is a priority for the College and has been included as a metric in previous versions of the college

strategic plan.



8. Teaching Effectiveness

8.1 Identify Measures of Teaching Effectiveness

All instructors will use the University Teacher Course Evaluation (TCE) process to be evaluated by their

students each semester. Through the annual and biennial review processes of faculty performance, each

instructor will be asked to provide a self-reflection which will include areas of improvement. Program

Director and Department Chair review the TCE results, any available peer review forms, and the self-

reflection with the instructors and provides feedback to the instructor.

8.2 What efforts to improve teaching effectiveness will be pursued based on these measures?

A key measure of teaching effectiveness is the performance on the annual faculty performance review. The

review includes evaluation of the teaching portfolio, including the teacher course evaluation data. This

information is used to develop individualized faculty intervention plans and/or College-wide faculty

development initiatives as needed.

9. Plans to evaluate students’ post-graduation success

The UKPAS program has recently constructed an evaluation tool that is deployed once students have

graduated from the program. This tool assesses recent graduates, collects demographic information

(practice site, practice details, etc.) as well as asks general questions relative to how prepared the graduate

feels having gone through the UKPAS program. The program also sends a similar survey to employers of

our graduates asking questions regarding their satisfaction on the preparation of our graduates.

10. Appendices

See pages 10-14.



1. HSM 601 Clinical Vignette Rubric

Exemplary (5 points) Satisfactory (3 points) Unsatisfactory (0 point)

Identify

Communication

Issues

Clearly identifies all communication

issues:

-provider to patient

-health care professional to health

care professional

-technical

Identifies only a limited amount of

communication issues:

-provider to patient

-health care professional to health

care professional

-technical

Does not determine

communication issues

specified in the clinical

vignette

Identify Patient-

Centered Errors

Clearly identifies all patient-centered

errors:

-provider to patient (no plan)

-provider to pharmacist (illegible Rx)

-pharmacist error (misreading

Rx/dosing error)

Identifies only a limited amount of

patient-centered errors:

-provider to patient (no plan)

-provider to pharmacist (illegible Rx)

-pharmacist error (misreading

Rx/dosing error)

Does not determine patient-

centered errors specified in

the clinical vignette

Systems-based

Solution: Societal

-Comprehensive insight into

Improving communication between

system medical partners to deal with

system complexities/enhance patient

care

-Provides limited insight into

Improving communication between

system medical partners to deal with

system complexities/patient care

Response does not provide

an adequate amount of

insight into domain solutions

Systems-based

Solution:

Organizational

- Comprehensive insight into

Improving Information technology to

support Patient education

- Provides limited insight into

Improving Information technology to

support Patient education

Response does not provide

an adequate amount of

insight into domain solutions

Systems-based

Solution: Economic Need to add to vignette/rubric Need to add to vignette/rubric

Need to add to

vignette/rubric

Note: The HSM 601 Course Instructor is in the process of revising the clinical vignette to reflect an additional economic component.

The rubric will be revised subsequently.



2. Summative/Masters: Standardized Patient Assessment Checklist

Objective Yes No

Greets the patient by name

Appropriate & effective communication Introduces self by giving name, title and role

Washes hands before beginning exam

History

Objective Yes No

Asked open-ended questions about chief complaint

Asked about character /quality, chronology (onset), duration

Asked about alleviating / aggravating factors

Asked about severity of dyspnea

Asked about presence of chest pain

Asked about similar symptoms in the past

Listens to patient &provides eye contact

Asked about progression of symptoms (worse with

exertion)

Asked about cough

Asked about hemoptysis (blood in sputum)

Asked about details of sputum (color, amount,

consistency)

Asked about fatigue / weakness

Asked about palpitations or racing heart beats

Asked about fever / chills or symptoms of infection

Asked about smoking history

Asked about weight loss or gain

Asked about previous heart & lung problems

Asked about exercise or current tolerance of physical activity

Asked about medication allergies

Asked about home medications

Asked about past medical problems



Asked about past surgical problems, hospitalizations

Asked about routine health maintenance (for HTN or DM)

Asked about Family History

Physical Exam

Objective Yes No

Rechecked Blood Pressure correctly(not over clothing, cuff tight, arm correct relaxed position, etc.)

Head/Neck Yes No

Looked in mouth / throat with light

Palpated frontal sinuses

Palpated head & neck lymph nodes (bilateral)

Cardiac Yes No

Inspect and palpate chest for lifts, heaves, thrills (palpate apex and LSB) Palpation of apical impulse, if not palpable roll patient to left lateral decubitus position

Auscultated heart in four places with the diaphragm (on skin, sitting or lying)

Auscultated carotids for bruits bilaterally

Evaluated Jugular vein for JVD (at 30 -45 degrees)

Pulmonary Yes No

Inspects chest anteriorly and posteriorly for respiratory pattern

Auscultated the lungs anteriorly and posteriorly correctly (3 areas auscultated on each side against skin)

Percussed the chest anteriorly and posteriorly correctly (3 areas auscultated on each side against skin) Patient instructed deep slow breaths, mouth open

Peripheral Vascular Yes No

Inspect lower extremities bilaterally for symmetry

Palpated posterior tibial and dorsalispedispulses bilaterally Palpation pitting edema bilateral lower extremities. (5 sec, with pressure, over distal tibia) Assessed capillary refill hands and feet bilateral Homan’s Sign Removed socks and inspect feet bilaterally

Professionalism

Yes No Student was dressed professionally with good hygiene

Stays on task



Polite with appropriate affect I was comfortable being examined by this student

Communication

Yes No

Explains physical exam techniques to patient using plain and understandable language

Explains diagnosis and treatment plan without overuse medical vernacular

Uses appropriate eye contact with patient

Listens to patient without cutting her/him off

Faculty Checklist Presentation

Yes No

Begins presentation with appropriate patient identifiers

Encapsulates the patient history & physical with a summative synopsis at the start

Presentation follows a logical progression in thought

Summarizes the HPI with reasonable depth & accuracy

Highlights pertinent historical facts

Highlights pertinent examination findings

Develops a logical / reasonable differential diagnosis

Communicates a working likely diagnosis, assessment & diagnostic plan

Clinical Problem Solving

YYess No

Develops Appropriate Differential Diagnosis

Primary & secondary diagnoses developed

Soap Note follows correct form

Ordered appropriate tests for chief complaint

Understands level of acuity (patient needs aggressive treatment of COPD exacerbation with

possible admission to hospital?)

Conveys an initial assessment & plan consistent with the case findings: (ie. Includes COPD /Obstructive lung disease with hypoxia as primary and Triages HTN & NIDDM as secondary problems to be addressed as well)

Patient Education

Yes No



Discusses working diagnosis (COPD) & hypoxia

Explains test results to patient (in layman terms)

Discusses secondary diagnosis (Hypertension): either discusses need to take blood pressure medicine

or increases dose of Accupril, or similar measure

Discusses secondary diagnosis (Diabetes): discusses compliance with medication or adds a second

agent

Explains initial treatment plan for COPD: including B2 agonist nebulizer, IV steroids, need for oxygen,

possible need for admission

Discusses need for compliance to medications

Explains need to stop smoking (set a date) offers medication

Discusses diabetic diet and need to adhere to ADA diet

Documents

University of Kentucky College of Health Sciences ... · practicing physician assistant. Students will demonstrate the medical knowledge needed to effectively practice as an entry-level