Competency-Based Holistic Evaluations for Prehealth Applicants

'The Ad1-isor

I Competency-Based Holistic Evaluation of Prehealth Applicants

Ei'i'ziI T. Chuck, PbD

Introduction

Let-III assisfnntpi.@rsor of' biolqgi ut George

i\,fc~.io~~ U11i~'et:iifjl. He a~elromes co~x~?/ents ci/~d jedbcirk om hir r~tb/-lc

fi-o/ll nlili~oi.s, an~~/issio~ls denlrs lid dil-ecto/:r, 01.

health ~ ~ ~ o f e s s ~ o i ~ . r .schools .ctt~dellt .ffC~its

clenns. if dd1.e.r.r

C ommunicating the qualifications of an applicant through a reference let-

ter is always a challenge. hlany references take a position of strongly recommending candidates based on the academic record and accomplishments in the contest of specific classes, but often the humanistic elements of an applicant are not easily discussed or deciphered.

\With the emphasis on an applicant's ability to del-elop the professional s l d s and insights necessary to becoming an escel- lent health care provider, it is unfortunate how little these s M s and insights are evaluated within the contest of academic letters of recommendation. \T:llile it is important for professors to describe the context of a student's grade in a lecture or small-group discussion-based course, there is often very little insight offel-ed to admissions committees (and institutional evaluation committees) on the non-academic qualities. Yet, nriting about an applicant's character and/or service ori- entation is often a complete1~- subjective assessment. If an applicant did not have a modicum of compassion for others, XI-ould we be willing to write a letter on behalf of the applicant? In the current application system, all applicants hare that compassion which unfortunatel!. fails to materialize in patient assessments.

hle&cal education most recentlj- has adopted competent!--based el-aluations to guide professional development to irr- pro\-e the qualin- of patient-doctor inter- actions and of patient care. Competen- cies are best described as the masteri- of sMls that express specific characteristics or criteria that are d u e d by a profession.

In other n~ords, it is designed as a de\-ei- opmental model and standard against \r-hich all are measured. hl!- goal in developing a competency-based eraluation rubric was to emphasize the lifelong development of these professional traits to applicants, references, and eralu- ators by providing a framenrork for discussion and assessment of the individual applicant without resorting to arbitrary "class rank" or subjectil-e, non-specific characterizations.

The Process and Research Rationale

At Virgnia Commonwealth Unirrersity, the competencies articulated by the Ac- credtation Council for Graduate hfedical Education (ACGhlE) have been applied to prehealth students'. But the literature reveals few efforts where a system of advising focused on competency-based evaluations is utilized. At the time I be- gan advising at George hlason Uniwrsit!; no evaluation rubric for committee letters or professional letters s~~pport ing prehealth students could be found online. ,It the same time, my own interest in professional career del-elopment for graduate students and postdoctoral scholars in science and engineering fields brought me to participate as a committee member and Professionalism section au- thor for the National Postdoctoral Association's Core Competencies.

Eventuallj; my search for guidance brought me to the Technical Standards for admitted students outhned b! man!- medical, dental, pharmac!; optometry, podiatry, 1-etennary, and other health professional programs. Regardless of

The Advisor

the degree offered, all the Technical Standards at these schools n-ere fairly uniform in the expectations for students and graduates of their programs. Fur- thermore, these standards paralleled the basic tenets of Harold Gardner's multiple intelhgence theory From these data, I identified ten areas of competency that characterize the holistic quabties desired in in- coming students and future professionals.

Academic acconrplishments. the scientific grade point average, coursework rigor, overall coursework variety and difficult); depart- mental or school-wide honors, scholarships and recognition.

I~~tellectualabifi~: competency with, success in and interest in lifelong learning.

S o d intelligence: demonstration of interper- sonal sldls, empathy, effective teamwork, and cultural awareness (especially outside one's own identity).

Practicalintelligence: leadership, project and time management, fiscal management.

Emotionalintelligence: profession&sm, ethics, morality, emotional stability, and competency to self-assess and improre.

Ae~thetic intelligence: observational and diagnostic skills.

Killesthetic intelligence manual dexterity and fine motor shlls.

Written commtttticatiol~ skills, as evidenced by pre-application essays, personal statement, and general communication.

Oral commrrnicotion skills, as eridenced by interviews and general communication.

h%to~vleAge ~ t h e p r d e s ~ i o ~ ~ : awareness of challenges that WILI face health care and the profession, with sensitirit!. to adjust to those challenges on a personal, professional, and societal level.

The one other area that is also critical for admissions decisions is the question of "fit with the institution." This criteria is not a competency, but it is critical to

kno\\- whether the applicant's career goals can be fa- cllitated througll the mission, resources, and experiences offered b!- a particular institution. \Y'hile t h s piece ma!- not bc easily addressed in generic letters of recommendation, I hare evaluated earl!--admission sophomore applicants n-ho vant to gain admission through articulation agreements \x-ith this element in mind.

Competencies in each area are ewluated based on evidence that demonstrates the derelopment of that area. In general, the levels of competencies are best described as follo\vs.

A'azic There is little mention or support- ing evidence of interest or development.

KOLIZCE: Evidence and references suggests that one passi~el!; accepts the importance of those s l d s and is "learning" new tasks.

Adt~a~icedNol~ice: While the skill has been taught, the individual needs more practice on his/her on7n.

PrOfinent: The individual can perform the task satisfactorily.

Couqeterrr: The individual considers ways to improre the task or outcome and makes adjustments.

E.~.er-r: The individual tests improvements to tasks in a systematic manner.

ilrla~fe~: The individual is rccognized by others for contributions that reflect masten- of the area.

Examples of holistic evaluation

For this article, I draw upon some fictionalized es- amples of comments in solicited letters of recommendation or feedback from committee interviews that could help me to assess an applicant's competencies. All of these examples are composite partial pro- files, and the intent of this document is to provide a platform for discussion with frequent letter-writers, intervie\v evaluators, co~nlnittee letter writers, and adn~issions officers to determine whether these examples mould provide better insight for an adrnis- sions decision. \Vhile the possible examples can be infinite, I have deliberately tried to provide diverse experiences as models, noting that many of the examples can also touch more than one competency area. Feedback is always elcom come, and a competency

The r\dvisor

Competency-Based Holistic Evaluation of Prehealth Applicants ~ ~ o / I / ~ ; I I I ~

rating should al\va!-s be rien-ed with possible error of one rating category hgher or lo\ver than assigned. Alost of these examples corer undergraduate and recent graduate applicants instead of postbaccalaureate or career-changing applicants.

Academic accomplishments

A7c7it 'e: The applicant has just begun to take science-rigorous course\t-ork. As mentioned in the personal statement, the applicant has occasionally \vithdra\~:n from courses n-hen his/her n-orliload outside the class- room seems to get heal-\; but st111 believes that s/he can handle the challenges of a health professional education.

Ad~lamced X o ~ ~ i c e : The applicant's science GPA stands at 3.07 because of transitional issues that occurred in the semester after changing from sociology to biophysics as a major. So far the applicant has only taken prereclui- site courses with a couple of up- per-level biomedical courses in progress. In the last year, the applicant's semester-b!.-semester GPA has risen to 3.6 in science course\vork. The applicant's difficulties freshman year contribute to the 3.17 science GPA. Sophomore-year and be- yond grades ha\-e been above 3.50 each semester.

(Io//lpele/,fi The applicant has escelled in chal- lenging science course\vork (science GPLI 3.42) despite a C in first-semester physics (calculus-based). In fact, \\-hle the applicant is an art hlstorp major, s/he has shon-n sur- psisingly little difficulty with science. The applicant's outstanding perfor- mance in science-rigorous course-m-ork ( G P L ~ 3.62) nas recent$ acknov-1- edged with the "outstanding or- ganic chemistry student" award. Representing our school, the applicant was finished the top score on the team in the recent Putnam mathematics competition.

Intellectual ability, including willingness to succeed in lifelong acquisition of knowledge

ATc/ji ,e: The applicant apparentlj- did really XI-ell in my class, though I hardly remember llun/her connecting \I-ith others during class dscussions.

S o ~ , i c e : The applicant's lab reports were grammatically correct but reflect ven- little adclitional insight or interest in the main topics COI-ered with the exercise.

A d ~ ~ n n c e d N o ~ i c e : In a club meeting \\-here \\-e brought in a cardiologist to talk about his/her career journey, the applicant askeci questions which reflect a desire to clarify topics that were apparently confusing.

Proficie~zt The applicant \\-as engaged in the course I teach on the homeless community. The questions s/he asked were probative and thoughtful.

Co71pete11f: The applicant mas inspired from taking my class on cancer biology epidemiology. hlany of the questions s/he asked revealed intellectual engagement in the topic, so I +as pleased when the applicant asked me to mentor him/her in an independent study project.

E3,pe/per-t: The applicant has been verj- help- ful in designing surveys for our psychology esperiment on interra- cial dating and sexual behavior, and s/he has further decided to take language courses in Farsi to develop the project further.

Ll 1~1.rter The applicant was instrumental in giving 1-ital feedback 11-hile writing a peer-rerien-ed journal article on more effective health comrnunica- tion strategies for Nspanic youth, \~-Ilich was submitted one month ago.

Social intelligence

A-023e: The applicant keeps remarlcing ho\r- because I n-as not raised in his/her famil!- that I would not understand the pressure to become a doctor.

The Advisor

Aiolice: \Yhen describing his/her role as - president, the applicant's authority and indi~idual contributions is ap- parent. \Yrllat is not obvious is the applicant's abhty to consider the opinions of hs/her collaborators, and he/she gets very defensive when it comes to constructi~~e criticism.

Ad~uncedAro~'ice: f hile the applicant claims to enjoy diverse company when it comes to

Competent

Expert:

group projects, the applicant pre- sents very few definitive examples that she knows what constitutes a good team versus a bad one. The applicant has been volunteer- ing at a local nursing home. \YMe the supervisor's letter re~eals that the applicant seems to "work well with the staff" without providing much detail, the applicant does articulate his/her duties at the nursing home. Through this exchange program, the applicant exchanged emails with a pen pal in an Iranian science-oriented school for girls. Her description of what she learned about Iranian culture articulates great respect for the hstory and scientific contribution of Persia. The applicant's dedication to build large student support for the club's Relay for Life team helped him/ her attract over 30 students and friends and collect over 92000 in pledges.

Practical intelligence

hTai'ue: The applicant fails to understand the importance of deadlines when submitting assignments.

hrozice: The applicant consistently arrives late to advising appointments and forgets to bring the appropriate materials that would make the meeting productive.

AdvancedhTouice: The applicant occasionally needs and expects attention, al\vays ask- ing others to show him/her how to use the spectrophotometer.

PrOfiir~~t: The applicant attended the re- quired meetings for club treasurers but did \-en- little to help the club request new funds for programs it intended to run.

Co/llpete~tt: The applicant created a notebook (\\:hich we never had before) that documented the steps taken to or- ganize the annual awards banquet. The notes included comments on how to improve the process for the next person assigned to orga nize the event.

E.ybe?-t The applicant's answers indicate a vcrg \veil-planned process of se- lecting a career in the health pro- fessions n~hile maintaining personal interests outside the health profes- sions. illternate timelines in case the applicant were to reapply were considered and evaluated.

A.laste7: The applicant was cited by references repeatedly for hls/her abkty to multitask and address contin gencies and emergencies calmly and reliably. The applicant received a leadership award from the Office of Student Activities.

Emotional intelligence

1Yaz1x The applicant is solely focused on doing well in class and on stan- dardized exams that s/he sees little reason to consider others' opinions on alternatives.

Nouicp. The applicant responds to stressful situations by avoidmg discussion or changng the subject. \Vhile the applicant appears remorseful, s/he tends to obsess over all the faults of his/her application and has not sought additional support to handle the stress.

Ah~ancedNouire: The applicant is au7are of some of the resources at the college that \\?ill help him/her with stress management. Holverer, the applicant appears conrinced that successful practitioners would never use these resources because one should never look weak to one's peers.

I l ~ e Advisor

Competency-Based Holistic Evaluation of Prehealth Applicants ro~it i~i l red

Projcitnk The applicant confides often with the University chaplain \rrhenever s/he finds hlrn/herself doubting his/her abilities. \B?llie reference may express confidence in the applicant, none of the references reflects a strong role as a confidant or a mentor. References express confidence in the applicant's integrity and character in general terms. When asked about ways that s/he \\,ill manage the stress of a professional career, the applicant identifies a few issues nrith some specific resources that s/he could access. References enthusiastically pro- mote the applicant. They articulate with specific examples their support for thc apphcant as a mentee. For example, one of the professors' letters recalls how the applicant disclosed his/her absences from class due to family problems. The applicant received the Goldman award, honoring a student who best emulates the core values of integrity and service to- wards others.

Aesthetic or observational intelligence

Nazue: The applicant lacks any listening s U s when it came to the intervieni.

Arovice: The applicant repeatedly needed guidance in operating the labora- tory equipment.

Adva~~cediVovice: The applicant's description of the mission trip to Nicaragua mas de- void of details about the village served, the people cared for, or the diseases being treated. The applicant describes the way the physician s/he shadowed me- ticulousl?- look at a mammogram to point out the location of the benign growths. I was amazed hon. I was drawn into the applicant's enthusiasm for model trains. The applicant taught me some of the subtle differences and the histories each of his favor- ite trains had. .

E q e r t The applicant's description of the patients s/he saw and the emo- tions he experienced by watching how the phj-sicians consoled them mere vivid and sensitive.

.\ laster: The applicant produced a series of tea kettles for a local art sho~v which n-on a "people's choice" award.

Kinesthetic intelligence or dexterity

Xak~e: There is no mention of the applicant's dexterity.

A70tire: The applicant cites texting as evidence of dexterity.

Adrwced Kol~ire: I t is peculiar how the microbiol- ogy lab instructor does not express as much confidence in the applicant'sdexterity. There is little mention of the lab exercises that the applicant had to complete whch requircd delicate handling.

Pi-ojhe~zt: The applicant has been washing dishmare and handling small tissue samples.

Cotltpe tent: As a co-recreational softball pitcher, the applicant describes ho\v he/she has practiced impart- ing spin on the ball while throwing to get more batters to strike out.

Eqerr: The professor discusses the applicant's output in a circuit de- sign class as not only being func- tional but also economically artistic.

it lastet: Describing his discipline and focus approaching the campus "garage band" competition, the applicant plays lead guitar for a rock band which recently finished in the Final Four.

Written and oral communications skills

Xuzi .t: Responses are extremely brief, despite the opportumq- for the applicant to esplore his/her thoughts.

L2 'o~ ire: Responses reflect estrcme difficulty in language skills. It is difficult to comprehend the applicant's thoughts or their organization.

The Advisor

Ad~sa~ l r rd fi~nice: The applicant can respond to questions briefly though some difficulty in language slillls is discernible. Responses tend to provide little in- teresting information of relevance.

Profici~n~ The applicant's responses shon- satisfactor!- communications shlls. Grammar, spelling, and organization are generally clear despite an!- adbtional difficulties.

Competeni: The applicant's responses show a relatively sophisticated level of depth, detail, and focus when an- swering questions. The applicant's responses appro- priately acknon-ledge the evaluator and convey confidence as a future professional. The applicant has received a com- mendation for submitting and de- fending a senior project/thesis.

Knowledge of the profession

Nazitp: The applicant reveals little knowledge of the realities of the profession and shows no interest in other health care careers.

Nouice: The applicant is able to articulate the appeal of a health professional career in very generic terms.

AduanredAToz~zce: The applicant has begun to volunteer in clinical or community set- tings to observe ho\v health professionals work with patients wvith various dnesses or disabilities. The applicant has a broad set of experiences that describe how health professionals work with patients. Applicant generally articulates the challenges patients and caregivers hare in the clinic or community center.

Conpetelrt: The applicant recognizes the challenges of a health professional lifestyle in the career of choice and in other associated careers in the field. There is substantial knon-1- edge of current challenges facing health care practitioners.

Discussion

The applicant realisticall!- describes the challenges facing the health care system and has developed an-are- ness and sensitix-ity to how practitioners manage careers. The applicant's reference letter es- presses great tlust in the applicant as a future health care practitioner and provides extraordinary support of the applicant's contribution to the profession.

Using this rubric to create my evaluation letters allows me to seek specific evidence in the applicant's submitted materials - pre-application, letters of recommendation, interview feedback. This assists in a holistic evaluation of the applicant against standards that are set forth by health professional programs themselves. The articulated criteria are based on the schools' poli- cies on admission and grounded in ps)-chological research and theory Additional considerations, such as fit with the institution or an individual's background and journey to the profession, provide additional context to vien~ the application. Secondly, the standards set forth in the rubric prevent me from subjec- tively changng rn! standards based on the q p e of program to \vhich'the applicant is seehng entry or the date when I write the letter. While I do have a tem- plate to present an applicant in my institutional eraluation letter, I am sure to address all of the ten competency areas in my final eraluation.

Another advantage this rubric has is the opportunity to engage students on the concepts of competency self- assessment wvhich the!- must become accustomed to in their professional practice. I present thls rubric to in- coming students as a n7ay for each potential applicant to understand how their overall character and accomplishments can be viewed. Presenting their skills in &us n7aj* avoids the "checl&st" mentality to a certain degree and focuses attenrion more on the personal and professional development of the applicant. In a retrospective study of graduates from the University of Michigan, students who have had opportunities to become comfortable with self-assessment tools demonstrate a predictive ten- dency to exhbit professionally acceptable behavior'. By having h s clear set of standards and goals, self-analysis becomes possible for an indnidual seeking to pursue a successful career in health care ser~lce and establish appropriate goals to address an!- areas of deficiency

The Advisor j t~ne 2009

Competency-Based Holistic Evaluation of Prehealth Applicants co/iti//ued

Acknowledgements

hluch of the work that I did for this rubric mould not have been possible without the help of Dr. Laurie Fathe (former director of the GAIU Center for Teaching Excellence) and Ms. Mary Zamon (Office of Institutional Assessment). Much of the discussion was sparked by the National Postdoctoral Asso- ciation Subcommittee for Core Competencies, with many valuable discussions chaired by Dr. Joan Chesney (St. Jude's Medcal Center) and other mem- bers of the committee. Discussions and informal reviews of this rubric were made by Gina Moses

(XACOhl), AIelinda hIaris (formerly Johns Hopkins University), and Michael Druitt (Hampton Univer- sity). I thank the admissions administrators and committees who may have reviewed the rubric online when evaluating GhfU-associated candidates.

References

' Harduarsingh, Liebonltz, and Rodurtha. Prehealth students' re- ceptivit!- to el-aluation b!- ACGAIE competencies. Tl~e Adl,isor27 (2): 21-26, June 2007.

' Stern, Frohna, and Gruppen. The prediction of professional behavior. ,\iediraJEdticatio~r 2005; 39: 75-82.