DO

vb(ccrim

dpnsor

ORIGINAL REPORTS

Surgery Clerkship Evaluations DriveImproved Professionalism

Frances E. Biagioli, MD,* Rebecca E. Rdesinski, MPH,* Diane L. Elliot, MD,†Kathryn G. Chappelle, MA,* Karen L. Kwong, MD,‡ and William L. Toffler, MD*

epartments of *Family Medicine; †Medicine; and ‡Surgery, Oregon Health & Science University, Portland,

regon

Lldwiiltc

o

wapafd

PURPOSE: To determine whether a brief student survey candifferentiate among third-year clerkship student’s professional-ism experiences and whether sharing specific feedback with sur-gery faculty and residents can lead to improvements.

METHODS: Medical students completed a survey on profes-sionalism at the conclusion of each third-year clerkship spe-cialty rotation during academic years 2007-2010.

RESULTS: Comparisons of survey items in 2007-2008 re-ealed significantly lower ratings for the surgery clerkship onoth Excellence (F � 10.75, p � 0.001) and Altruism/RespectF � 15.59, p � 0.001) subscales. These data were shared withlerkship directors, prompting the surgery department to dis-uss student perceptions of professionalism with faculty andesidents. Postmeeting ratings of surgery professionalism signif-cantly improved on both Excellence and Altruism/Respect di-

ensions (p � 0.005 for each).

CONCLUSIONS: A brief survey can be used to measure stu-ent perceptions of professionalism and an intervention as sim-le as a surgery department openly sharing results and commu-icating expectations appears to drive positive change intudent experiences. (J Surg 70:149-155. © 2012 Associationf Program Directors in Surgery. Published by Elsevier Inc. Allights reserved.)

KEY WORDS: codes of professional ethics, clinical clerkship,education, faculty, medical, surveys

COMPETENCIES: Professionalism, Interpersonal and Com-munication Skills, Practice-Based Learning and Improvement

INTRODUCTION

Professionalism is a concept frequently discussed in medicaleducation,1 including its importance in surgical training.2-4 Ingeneral, professional attitudes and behaviors in medicine are

Correspondence: Inquiries to Frances Biagioli, MD, OHSU Family Medicine, 3181 SW

Sam Jackson Park Road, Mail Code FM, Portland, OR 97239; fax: 503-494-4496; e-mail:biagioli@ohsu.edu

Journal of Surgical Education • © 2012 Association of Program DirecPublished by Elsevier Inc. All rights res

described according to 3 domains: (1) provider characteristics(eg, respect, integrity, and accountability), (2) ethical integrity,and (3) sensitivity to the unique issues of each patient.5 The

iaison Committee on Medical Education requires that studentearning experiences be evaluated to maintain appropriate stan-ards of professionalism.6 Two challenges face those taskedith ensuring that medical student experiences include training

n professionalism. First, a brief, easily administered instruments needed to assess specific aspects of professionalism in studentearning environments. Second, effective methods are neededo improve professionalism among clinical faculty and residentsontributing to medical student education.

A variety of instruments have been used to index componentsf professionalism,7-13 however, most of these have been ap-

plied at either the institutional or training program level andhave not been used to assess discrete medical student educationexperiences (ie, clerkships within an institution). The Profes-sionalism Survey developed by Arnold, Blank, Race, and Cip-parrone is a brief evaluation tool validated for postgraduatetraining which has reliable dimensions corresponding to themain attributes of professionalism.14 We adapted this survey tomeasure the climate of professionalism in third-year medicalstudent rotations.

Studies indicate medical professionals learn and come to in-ternalize professionalism primarily through role modeling15 so

e used a validated instrument to measure whether attendingsnd residents across required clerkships consistently modeledrofessional behaviors and attitudes. Because this instrumentllows identification of specific behaviors associated with pro-essionalism, we planned to also use the results to inform facultyevelopment.

METHODS

Survey Instrument

The Clerkship Professionalism Survey (Table 1) lists 12 state-ments, each with a 6-point Likert agreement response scale(strongly disagre [1], moderately disagree [2], somewhat dis-

agree [3], somewhat agree [4], moderately agree [5], strongly

tors in Surgery 1931-7204/$30.00erved. http://dx.doi.org/10.1016/j.jsurg.2012.06.020

149

agree [6]). This survey is identical to the original Professional-ism Survey14 except that the phrases “residency training” and“residency colleagues” were deleted and the attribution of “res-idents” was changed to read “residents or attending physicians.”Survey items are grouped in 3 subscales: Excellence, Honor/Integrity, and Altruism/Respect. The initial validated Profes-sionalism Survey14 phrases some questions in a manner suchthat a score of 1 reflects a more professional behavior than a 6.To alter the validated tool as little as possible, the ClerkshipProfessionalism Survey keeps the original wording and reversescoring is used to report results for those items. The reversescoring translates the results so that higher scores consistentlyreflect more professional behaviors.

Survey Administration

Third-year students at Oregon Health & Science University(OHSU) rotate through 7 required clinical rotations: childhealth, family medicine, internal medicine, obstetrics/gynecol-ogy, psychiatry, rural primary care, and surgery. For more thana decade, OHSU has required students to complete a standard-ized evaluation of their learning experiences across clerkshipsbefore receiving their grades. During the 2007-2008 academicyear, OHSU augmented this online evaluation with the Clerk-ship Professionalism Survey questions at the conclusion of everyrequired clerkship; students were thus obliged to also complete

TABLE 1. Clerkship Professionalism Survey*: Survey Items‡ and

Excellence• During this clinical rotation, I have met individuals whom I• My colleagues who are residents or attending physicians h

patients I am seeing.• I observed that the residents or attending physicians I have• During my most recent clinical rotation, I have encountered• I have observed resident or attending physician colleaguesHonor/integrity†• I have been urged by my resident colleagues or attending

than gather my own information from the patient.• I have been instructed to withhold data from a patient’s cha

attending physician.• I have observed a resident colleague or attending physicia• The residents or attending physicians I have worked with h

to them.Altruism/Respect†• I have observed residents or attending physicians making d

groups or other health care workers.• I have observed residents or attending physicians referring

frequent flyers” or other terms.• I have observed residents or attending physicians schedulin

for themselves than for the patient.*Adapted from Arnold EL, Blank LL, Race KEH, Cipparrone N. Can proenvironment. Acad Med. 1998;73:1119–1121.14

†For reporting, these survey items are reverse scored so that a higher‡Student rating scale: strongly disagree [1], moderately disagree [2], so

[6].§OHSU: Oregon Health & Science University.

the questions focused on professionalism to receive their grades.

150 Journal of Surgi

This resulted in a nearly 100% return rate. Data were collectedfor the 2007-2008, 2008-2009, and 2009-2010 academicyears. Responses were anonymous, and only survey completionwas tracked at an individual level. The OHSU InstitutionalReview Board approved this study under the exemption forresearch involving normal educational practices conducted inestablished or commonly accepted educational settings. Clerk-ship Professionalism Survey questions to be added to studentevaluations of clerkship learning experiences were shared withand approved by clerkship directors before being instituted.There was no general announcement of the addition of thesequestions to any clerkship faculty, thus faculty were not awarethey were being rated in these areas until after 2007-2008 sur-vey results were shared.

Survey Analyses

Statistical analyses were carried out using SPSS, version 19 (Ar-monk, New York, USA). To facilitate interpretation, certainitems were reverse-scored so that higher scores consistently in-dicated positive professional behaviors. Comparisons of scoresfrom the first and second half of the 2007-2008 academic yearrevealed no seasonal differences or maturation effects across theyear. Thus, all surveys for each academic year were combinedand used for the analyses. The subscale scores of Excellence,Honor/Integrity, and Respect/Altruism were calculated by

ale Reliability

Original* OHSU§

0.72 0.63er role models.sisted me in attaining educational materials pertaining to

d with educate their patients about their illness.duals who display and promote professional behavior.the needs of their patients ahead of their own self-interest.

0.60 0.57ians to copy their history and physical examination rather

hout being given an explanation from a senior resident or

a patient.ked me to write orders or fill out forms and sign their names

0.59 0.71tory comments about other medical/surgical specialty

ients as “hits, gomers, real citizens, walkie-talkies, players,

or performing procedures at times that are more convenient

alism be measured? The development of a scale for use in the medical

desirable and reflects more professional behavior.t disagree [3], somewhat agree [4], moderately agree [5], strongly agree

Subsc

considave as

workeindiviplace

physic

rt wit

n lie toave as

eroga

to pat

g tests

fession

score ismewha

summing items and dividing by the number of items included

cal Education • Volume 70/Number 1 • January/February 2013

in each subscale. Reliability was determined using Cronbach’s� scores. Subscale reliability measures of the survey used withthird-year medical students were comparable with those of theoriginal survey of postgraduate trainees. We compared meanratings across clerkships for the subscales and individual itemswith analysis of variance, using post hoc Scheffé tests. A similaranalysis was used to assess subscale scores across years withinclerkships. Because of the multiple comparisons assessed, theBonferroni correction adjusted the � levels to the p � 0.01level.

Changes Driven by Survey Findings

Findings from the 2007-2008 academic-year surveys were pre-sented to all clerkship directors during one of their monthlymeetings held in late 2008. The survey findings indicated thatalthough overall mean subscale scores (range 4.2-5.9) indicatedthat students had observed professionalism to some degree onall clinical rotations, the surgery clerkship received lower ratingsthan any other clerkship on all subscales. In addition, signifi-cant differences were found between surgery and other clerk-ships on 2 subscales and several individual items.

In response to these survey findings, the Department of Sur-gery undertook specific activities. The surgery clerkship direc-tor discussed the survey results with the surgery departmentchair and education committee, which prompted faculty dis-cussions and raised departmental awareness of student percep-tions of surgery faculty and professionalism. Subsequently, thesurgery education committee shared the survey results and awritten summary of faculty discussions about its findings withall surgery faculty and residents. This written communicationacknowledged the positive attitudes of the surgery faculty andthe department’s reputation for its friendly atmosphere andnoted some possible explanations for the survey’s lower ratings.However, the communication also stressed that, whatever thereason for the lower ratings, changes were necessary to improveclerkship students’ perceptions of surgery’s professionalism.

After this, a portion of a department-wide meeting agendawith faculty and residents was dedicated to a discussion of theprofessionalism survey results. The group addressed specificsurvey items, subscale scores, survey validity, potential explana-tions for the scores, and possible student biases regarding thesurgery clerkship because of specialty stereotypes. Issues dis-cussed included the hierarchical structure of surgical training,how comments may be interpreted negatively from less experi-enced viewpoints, maintaining professional behavior in stress-ful situations, and exhibiting consistent respect when speakingof other physicians and specialties. Department leaders explic-itly stated that they expected to see improvements in studentperceptions of professional behavior during the coming aca-demic year.

This first professionalism meeting was held during the latterpart of the 2008-2009 academic year with surgery faculty andnearly 90 residents during their weekly protected education

time. The discussion is now repeated annually during the sur-

Journal of Surgical Education • Volume 70/Number 1 • January/Febr

gical resident orientation, with additional time devoted to howresidents might best model professional behavior and interac-tions while teaching, evaluating, and working with medical stu-dents. Each year the 2007-2008 study results are used to em-phasize the importance of student perceptions. In addition,professionalism issues are now explicitly addressed by the sur-gery clerkship director with third-year students at the beginningof every surgery clerkship orientation.

RESULTS

A total of 734 surveys were assessed for the 2007-2008 aca-demic year and were compared with results from 868 surveysfrom 2008-2009 and 432 surveys during the 6 months of datacollection during 2009-2010. All students completed surveys atthe conclusion of each clerkship because survey completion wasrequired to obtain a grade.

The 2007-2008 mean survey scores for the subscales of Excel-lence, Honor/Integrity, and Altruism/Respect for each clerkshipare presented in Figure 1. Although subscale comparisons revealedno significant differences among the 7 clerkships on the Honor/Integrity subscale, significant differences were found between sur-gery and the other clerkships on the Excellence (F � 10.75, p �0.001) and Altruism/Respect (F � 15.59, p � 0.001) subscales.On the Excellence subscale, surgery was rated significantly lowerthan the family medicine, internal medicine, and rural primary-care clerkships (p � 0.001 for each). On the Altruism/Respectsubscale, at baseline the variance was even more striking with sur-gery rated significantly lower than all 6 other clerkship experiences(p � 0.001 for each).

Surveys continued to be collected in 2008-2009, in whichthe surgery clerkship continued to be an outlier in terms ofstudent perceptions of professionalism. However, during thethird academic year of the Clerkship Professionalism Survey,2009-2010 (see Fig. 2), the mean scores of the surgery clerkshipsignificantly improved while there were no significant changesin mean scores for any of the other clinical rotations. On boththe Excellence and Altruism/Respect subscales, differences ob-served between surgery and other clerkships in 2007-2008 wereno longer present. In 2009-2010 the mean scores on both theExcellence and Altruism/Respect subscales for the surgeryclerkship were significantly higher (p � 0.005 for each, see Fig.3); this change occurred after the surgery department meetingon professionalism.

DISCUSSION

This study has informed our ability to improve medical studenteducation in professionalism in several ways. A clerkship pro-fessionalism survey can be used to identify particular aspects ofprofessionalism and to discriminate among students’ clinicalexperiences on different clerkship rotations. Sharing results ofsuch a survey can identify specific areas for faculty and resident

growth in professionalism. Finally, increasing awareness of stu-

uary 2013 151

wd

ionalism

dent perceptions of faculty and resident professionalism, in thecontext of supportive departmental leadership, can be an effec-tive way of promoting faculty self-reflection of their personalbehaviors, how they are perceived by others, as well as how theycan improve their role modeling of professionalism during stu-dents’ clinical experiences.

The results of this study also confirm the psychometric prop-erties of the Professionalism Survey and demonstrate its ability

† p<0.001 The mean Surgery Clerkship Excellence subscale raand Rural Health Clerkship ra ngs The Honor/Integrity subscale showed no significant differen

* p<0.001 The mean Surgery Clerkship Altruism/Respect subsc

4.0

4.2

4.4

4.6

4.8

5.0

5.2

5.4

5.6

5.8

6.0

Excellence(5 items)

Honor / Integrity items)

A

Maa

n Sc

ore

on P

rofe

ssio

nalis

m S

urve

y

Third Year Clerkship Subscale

†

FIGURE 1. Third-year clerkship Profess

‡ There were no significant differences between clerkship

4.0

4.2

4.4

4.6

4.8

5.0

5.2

5.4

5.6

5.8

6.0

Excellence(5 items)

Honor / Integ(4 items)

Mea

n Sc

ore

on P

rofe

ssio

nalis

m S

urve

y

Third Year Clerkship Su

FIGURE 2. Third-year clerkship Professionalism

152 Journal of Surgi

to be adapted to assess the professional climate of clerkship aswell as residency experiences. Our findings also provide poten-tial calibration for expected scores within required third-yearclinical rotations. Evaluations of professionalism are typicallyconducted at the institutional level or within single trainingprograms.7-13 We found that this instrument can be used

ithin a medical school to discriminate different clerkship stu-ent experiences of professionalism but also can drive depart-

ignificantly less than the Family Medicine, Internal Medicine

een clerkships g was significantly less than each of the other six clerkships

espects)

gs 2007 - 2008

Child Health

Family Medicine

Internal Medicine

OB/GYN

Psychiatry

Rural Health / Primary Care

Surgery

*

Survey subscale ratings, 2007-2008.

or any of the subscales.

Altruism / Respect(3 items)

le Ra ngs 2009 - 2010‡

Child Health

Family Medicine

Internal Medicine

OB/GYN

Psychiatry

Rural Health / Primary Care

Surgery

ng was s

ces betwale ra n

ltruism / R(3 item

Ra�n

scores f

rity

bsca

Survey subscale ratings, 2009-2010.

cal Education • Volume 70/Number 1 • January/February 2013

ahe

e and A

mental improvements in faculty and resident professionalism.In addition, we found the instrument to be useful in document-ing professionalism both within and across clerkships over time.Because student professionalism issues need to be addressed andcarefully documented by faculty, this anonymous instrumentwould not be useful for use in the individual evaluation ofstudent professionalism. However, the Clerkship Professional-ism Survey could be adapted to a variety of clinical and educa-tional settings where the intent is to measure the overall climateof professionalism experienced by learners.

While the survey itself was valuable, we believe the manner inwhich the survey data were presented to clerkship directors waskey to the improvements that followed. In addition to identi-fying differences among experiences, the individual surveyitems provided specific feedback regarding professional behav-iors observed by students during their clerkship experiences.We found that openly presenting clerkship directors with dataon their own and other clerkships’ ratings on specific behaviorsprovided ample impetus for positive change. Because of thepride teaching physicians commonly take in their own special-ties, no department wishes to be perceived as “less professional”than other specialties. Although actual differences (though sta-tistically significant) between ratings were small, the open shar-ing of data prompted, in this case, faculty and resident aware-ness of student perceptions as well as behavioral changes whichsignificantly improved subsequent evaluations of the profes-sional climate of the surgery clerkship. Whether these behav-ioral changes were due more to physician self-reflection and asincere desire to practice more professionally, by departmentalpride and directives, or simply by the knowledge that studentswere evaluating them on professionalism is not known—likelymost faculty and resident behavior changes were due to a com-

* p< 0.005 The Surgical Clerkship showed a significant incSubscales in 2010 as compared to prior years.

3.5

4.0

4.5

5.0

5.5

6.0

07/08 08/09 09/10 07/08

Mea

n Sc

ore

on P

rofe

ssio

nalis

m S

urve

y

Su

*

FIGURE 3. Comparison of the surgical clerkship Excellenc

bination of all the above.

Journal of Surgical Education • Volume 70/Number 1 • January/Febr

Based on significantly higher surgery scores on the 2009-2010 subscales, the awareness generated by sharing survey datafrom students with surgical faculty and residents appeared toinfluence student perceptions and experiences of surgery clerk-ship professionalism. While formal faculty development work-shops are often used to address professionalism,16,17 we foundthat a single meeting, simple in scope and brief in duration,proved to be an effective intervention in this case. This meetingallowed surgery faculty and residents to dialog about possiblemethods to dispel surgical stereotypes, address how workloadstresses can affect professional attitudes and behaviors, and re-view how relatively inexperienced learners may view commentsand behaviors differently than colleagues.

The original development of the survey on professionalismwas prompted partially by findings that learners frequentlyheard derogatory comments regarding patients, colleagues, andother health professionals.18 In our study, the Altruism/Respectsubscale of the survey indexed those behaviors, and it had thegreatest initial difference across clerkships. Its items includedbehaviors that have been termed “bad mouthing” or “bash-ing.”19,20 While perhaps used as a means to vent frustrationsnd cope with stress, such remarks or “gallows” humor mayave unintended adverse effects on trainees and others in thenvironment.21-23

Improving student experience of and, thus, training in pro-fessionalism may be accomplished most effectively throughdepartment-wide awareness and priorities. This is the dimen-sion most associated with the “hidden curriculum” in whichenacted social norms can supersede written objectives.24 Thesurgery department’s combined department-wide discussion,to acknowledge challenges and establish shared responsibilityfor upholding new behavioral expectations, was an important

the mean scores for the Excellence and Altruism/Respect

09/10 07/08 08/09 09/10

*

ltruism/Respect subscale scores across 3 academic years.

rease in

08/09

rgery

component of improving student experiences in the area of

uary 2013 153

sr

srmdt

professionalism. Clearly, a leadership that regularly emphasizesand demonstrates the importance of professionalism is key tosuccessful, sustained improvement. Students are generally moreinfluenced by what they observe than by stated professionalstandards,18 and thus altering the day-to-day culture of theurgical clerkship was implicit to the improvement in studentatings.

LIMITATIONS

Although the survey items describing specific behaviors of in-terest25 have subscale reliability26 and face validity, survey re-ponses are based upon student recollections and estimationsather than a direct audit of experiences. In addition, studentsay begin the third year having little direct experience with

ifferent specialties and may be influenced by cultural stereo-ypes.27 As a result, findings may have been affected by confir-

mation bias because of student expectations of different experi-ences.28 Directly addressing and putting professional issues incontext during the surgery clerkship student orientation mayalso have positively impacted surgery professionalism ratings.However, improvement in scores reflecting observations of spe-cific behaviors by faculty and residents suggests that the profes-sional environment of the surgery clerkship was indeed favor-ably altered.

CONCLUSIONS

The interest in physician professionalism is growing because ofits relationship to quality of care and health outcomes.29 Re-cently the challenges of providing training in professionalismand the need for new methods of training have been high-lighted.30 Other findings suggest learners can provide formativeinformation that can guide targeted faculty development inprofessionalism.31 We found that anonymous assessment ofstudent perceptions cannot only identify specific professionalbehaviors observed in different learning contexts, but that shar-ing survey results can also drive improvements in medical edu-cation and practice. Effecting change may not require muchmore than motivating faculty and residents through brief dis-cussions and clear messaging from leadership. The findings ofthis study are useful for those responsible for the professionaleducation of medical students, residents, and faculty.

ACKNOWLEDGEMENTS

The authors thank the Oregon Health & Science UniversityClerkship Directors and the Department of Surgery for theirassistance with this study, and Ms LeNeva Spires for editorialassistance.

Funding was provided by the National Institutes of Health,Office of Behavioral and Social Sciences Research K07 grantRFA-OD-001, Strengthening Behavioral and Social Science in

Medical Schools and this site’s 5K07CA121457.

154 Journal of Surgi

Aspects of this work were presented as a poster at NorthAmerican Primary Care Research Group Annual Meeting,Banff, Alberta, November 2011: Rdesinski RE, Chappelle KG,Biagioli FE, Toffler BL, Elliot DL. Can Student EvaluationsDrive Improved Professionalism?

REFERENCES

1. Lucey C, Souba W. Perspective: The problem with theproblem of professionalism. Acad Med. 2010;85:1018-1024.

2. Kavic MS. Professionalism, passion, and surgical educa-tion. JSLS. 2010;14:321-324.

3. McCawley N, Leahy AL. The challenges of surgical pro-fessionalism. Surgeon. 2011;9:1-2.

4. D’Cunha J. Professionalism in medicine: are we closer tounifying principles? Semin Thorac Cardiovasc Surg. 2010;22:111-112.

5. Kirk LM. Professionalism in medicine: definitions andconsiderations for teaching. Proc (Bayl Univ Med Cent).2007;20:13-16.

6. Liaison Committee on Medical Education. Functions andstructure of a medical school. Standards for accreditation ofmedical education programs leading to the M.D. degree.Available at: http://www.lcme.org/functions2011may.pdf.

7. Cavanaugh S, Simmons P. Evaluation of a school climateinstrument for assessing affective objectives in health pro-fessional education. Eval Health Prof. 1997;20:455-478.

8. Ells C, Downie J, Kenny N. An assessment of ethicalclimate in three healthcare organizations. J Clin Ethics.2002;13:18-28.

9. Haidet P, Kelly PA, Bentley S, et al. Not the same every-where. Patient-centered learning environments at ninemedical schools. J Gen Intern Med. 2006;21:405-409.

10. Quaintance JL, Arnold L, Thompson GS. Developmentof an instrument to measure the climate of professional-ism in a clinical teaching environment. Acad Med. 2008;83(10 suppl):S5-S8.

11. Scott T, Mannion R, Davies H, Marshall M. The quan-titative measurement of organizational culture in healthcare: a review of the available instruments. Health Serv Res.2003;38:923-945.

12. Stern TD. Measuring Medical Professionalism. New York,NY: Oxford University Press; 2006.

13. Jha V, Bekker HL, Duffy SR, Roberts TE. A systematicreview of studies assessing and facilitating attitudes to-wards professionalism in medicine. Med Educ. 2007;41:

822-829.

cal Education • Volume 70/Number 1 • January/February 2013

14. Arnold EL, Blank LL, Race KE, Cipparrone N. Can pro-fessionalism be measured? The development of a scale foruse in the medical environment. Acad Med. 1998;73:1119-1121.

15. Park J, Woodrow SI, Reznick RK, Beales J, MacRae HM.Observation, reflection, and reinforcement: Surgery fac-ulty members’ and residents’ perceptions of how theylearned professionalism. Acad Med. 2010;85:134-139.

16. Steinert Y, Mann K, Centeno A, et al. A systematic reviewof faculty development initiatives designed to improveteaching effectiveness in medical education: BEME GuideNo. 8. Med Teach. 2006;28:497-526.

17. Steinert Y, Cruess S, Cruess R, Snell L. Faculty develop-ment for teaching and evaluating professionalism: Fromprogramme design to curriculum change. Med Educ.2005;39:127-136.

18. Feudtner C, Christakis DA, Christakis NA. Do clinicalclerks suffer ethical erosion? Students’ perceptions of theirethical environment and personal development. AcadMed. 1994;69:670-679.

19. Holmes D, Tumiel-Berhalter LM, Zayas LE, Watkins R.“Bashing” of medical specialties: students’ experiences andrecommendations. Fam Med. 2008;40:400-406.

20. Hunt DD, Scott C, Zhong S, Goldstein E. Frequency andeffect of negative comments (“badmouthing”) on medicalstudents’ career choices. Acad Med. 1996;71:665-669.

21. Bosk CL. Occupational rituals in patient management.N Engl J Med. 1980;303:71-76.

22. Maxwell W. The use of gallows humor and dark humorduring crisis situations. Int J Emerg Ment Health. 2003;5:

93-98.

Journal of Surgical Education • Volume 70/Number 1 • January/Febr

23. Van Wormer K, Boes M. Humor in the emergencyroom: a social work perspective. Health Soc Work.1997;22:87-92.

24. Snyder BR. The Hidden Curriculum. New York, NY:Knoph, 1971.

25. Messick S. Validity. In: Linn RL, ed. Educational Mea-surement, 3rd edition. New York, NY: American Councilon Education and Macmillan, 1989.

26. Cook DA, Beckman TJ. Current concepts in validity andreliability for psychometric instruments: theory and appli-cation. Am J Med. 2006;119:e7-16.

27. Martin FM, Mayo PR, McPherson FM. Professional ste-reotypes of first-year medical students. Br J Med Educ.1967;1:368-373.

28. Nickerson RS. Confirmation bias: a ubiquitous phenom-enon in many guises. Rev Gen Psychol. 1998;2:175-220.

29. Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND.Unprofessional behavior in medical school is associatedwith subsequent disciplinary action by a state medicalboard. Acad Med. 2004;79:244-249.

30. Bryden P, Ginsburg S, Kurabi B, Ahmed N. Professingprofessionalism: Are we our own worst enemy? Facultymembers’ experiences of teaching and evaluating profes-sionalism in medical education at one school. Acad Med.2010;85:1025-1034.

31. Sullivan C, Arnold L. Assessment and remediation in pro-grams of teaching professionalism. In: Cruess RL, CruessSR, Steinert Y, eds. Teaching Medical Professionalism.

Cambridge, UK: Cambridge University Press, 2009.

uary 2013 155