Supplemental Digital Appendix 1

Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 1

Supplemental Digital Appendix 1

Rationale, Detailed Methods, Protocol, and Supplemental Results

Rationale

This protocol examines the potential diagnostic value that lies in the nonverbal information provided by

patient facial affect, as interpreted by physicians. It is postulated that patients’ facial affect contributes to

the initial evaluation of patients with symptoms that raise suspicion of an immediate threat to life. Acute

pulmonary embolism (PE) represents a prototype disease on this issue.[2,3] Because PE can present with

complaints caused by numerous other conditions (e.g., chest pain, shortness of breath, fatigue), and

because PE is the third leading cause of cardiovascular death in the US, and can kill suddenly, clinicians

in emergency departments (EDs) tend to over test patients with possible PE.[4] For example, using a

National database, Feng et al examined outcomes of ED patients undergoing computerized tomographic

pulmonary angiography (CTPA) scans—the primary imaging modality used to diagnose and exclude

PE—only 3.7% went on to have a diagnosis of PE.[5] Over use of CTPA has negative consequences,

including radiation exposure, injury to kidneys related to iodinated contrast, false positive testing and

cost.[6-8] Reasons for over testing include medicolegal concerns, perceived normative behaviors of peers,

patient expectations and the belief in a zero tolerance for error culture [4,9-11]

Within this context, the overarching hypothesis of this study is that the quality of diagnostic testing in

emergency care is strongly influenced by the strength of the clinician’s initial belief that a patient has an

immediate threat to life. This form of metacognition has also been referred to as fast, or system one

decision-making.[12-14] Here, we refer to the quality of diagnostic testing generally as less testing,

especially with CTPA scanning, among patients who have no discernable diagnosis on longitudinal

follow-up.[2]



In the emergency care setting, the initial belief about illness acuity is based on a small set of objective

data, including the patients’ chief complaint, age, gender past medical history, vital signs, and is also

determined by the clinicians’ interpretation of the patients’ overall appearance, which includes the

patient’s affect.[15,16] In emergency care, 87% clinicians use implicit clinical judgment over evidence to

make final dispositions, and this number increases to 93% in low-risk patients.[17]

Prior literature has suggested that patients with acute illness portray different emotions from healthy

subjects: disgust, fear and increased frontalis muscle tone in response to airway resistance. [18] In

previous work, we found decreased facial affect variability in patients with significant cardiopulmonary

disease, compared to those without disease. Additionally, clinicians who perceived that their patients had

smiled, rated them as having a lower probability of acute PE than patients they believed not to have

smiled.[1,19] Although the design of this previous study prevents us from making any inferences about

the diagnostic utility of facial expression analysis in clinical practice, it nevertheless, laid the groundwork

for further studies to explore the concept of using patients’ facial expressions as an additional objective

tool to assess pretest probability and improve clinician-patient communication. This is relevant given that

70% clinicians have identified communication with patients as the most important factor to reduce

unnecessary advanced imaging[20].

The overarching goal of minimizing unnecessary testing is also shared by low risk patients with chest

pain or dyspnea; one study revealed that these patients have communicated to their providers that they

desire less testing for PE than clinicians.[21] To our knowledge, no study has tested if viewing a patient’s

facial affect changes the belief of a clinician in the probability of significant disease or desire to order a

diagnostic test. Accordingly, we designed this protocol to test the hypothesis that patient affect can

change physician pretest probability of significant cardiopulmonary disease in a population of emergency



patients previously selected for CT pulmonary angiography. Another aim is to compare this effect

between results of a standardized test of affect recognition, training level, specialty, and gender of the

clinicians.

Methods and Protocol

A. Overall design:

This was a pilot study conducted at Indiana University affiliated hospitals (IU Health University

Hospital, IU Health Methodist Hospital, Sidney and Lois Eskenazi Hospital). The protocol was

designed with awareness of domains used to assess effectiveness of instruments to assess

physician decision-making: 1. The conceptual framework (emergency clinicians must use limited

information to make rapid decisions), 2. The theoretical framework (the patient’s faces affects

decisions), 3. Content validity (faces of real patients recorded in real time watching a standardized

stimulus), 4. Construct validity (visual analogue scale pre and post assessment of physician global

belief of patient illness and desire to order a specific test) and 5. Internal consistency, reliability

and reproducibility (multiple observers, novice to expert of either gender from two

specialties).[22] The study has two parts: 1. Collection of patient videos and 2. Presenting the case

histories and videos to clinicians to determine the effect of affect on their belief in presence or

absence of significant CP disease. This study had approval from the institutional review board

(IRB) and all patients and physicians signed an informed consent form.

B. Patient videos

To systematically elicit changes in patients’ facial expressions under standardized conditions,

patients were presented with five visual stimuli on the laptop (Figure 1). Four were standardized

still photos from the International Affective Picture Set (IAPS): three of which have been shown



to elicit strong positive Valence responses from normal persons (baby, embrace, puppies) and one

neutral to serve as a baseline (cup).[23] Patients also viewed a 26 s video clip from a “Best of”

America’s Funniest Home Videos (a cat that flips after being taunted by a bird and an excited dog

falling in a pool); We added this video as a stimulus since it had won a funniest video award in a

nationwide voting and we therefore expected it would be likely to elicit a strong (positive)

emotional response that would notably alter facial expressions. The point of interest was when the

dog falls in the pool at approximately 25 s into the video. Each of the standardized IAPS images

was projected for 4 s, separated by 1 s of a black screen. The patients’ faces were recorded only

when the image was projected. Patients were placed in semi-Fowler’s position, and the camera of

a small laptop computer (MacBook Air®, Inc., Apple Inc. Cupertino, CA) is positioned

approximately 18 inches in front of the subject. The computer’s webcam recorded participants’

facial expressions. The computer was programmed using Mac OS X to demonstrate the six-slide

presentation shown above. The cup, typically associated with a neutral response, is the control

stimulus used to establish a baseline facial expression, and was always presented first. The

order of the subsequent four slides was automatically shuffled by the operating program to present

a different order for each patient. We pre-tested, revised and refined the image collection methods

in 15 consenting patients undergoing CTPA. We found that optimal lighting conditions required a

portable photography umbrella-reflecting hood to provide uniform indirect lighting (LimoStudio-

Photography Photo Portrait Studio 600W Day Light Umbrella Continuous Lighting Kit, Rancho

Cucamonga, CA). This setup produced reliable image quality.

We obtained video images of 75 subjects, of whom 11 had the criterion standard definition of

acute CP disease. All subjects were called for additional approval before showing their videos to

physicians. We reasoned that at least 10 patients with disease positive (in this case, the definition

of significant CP disease) would be necessary to generate a meaningful receiver operating



characteristic curve for each clinician tested. Table 1 shows the clinical characteristics of the

enrolled patient population. The IRB asked the investigators to reconsent all patients, during

which 2 patients withdrew, leaving 73 videos for the final work.

C. Criterion standard for significant cardiopulmonary disease

Using previously defined methods, patient disease outcomes were determined from usual medical care

processes by 1) structured evaluation of the medical record including all radiographic and laboratory

results as well as physician documentation, 2) supplemented by results of a telephone call to ask the

patient if he or she had any new diagnosis 3) Data from these two sources were then reviewed by three

independent clinicians who were blinded to each other’s opinions and had unfettered access to the entire

research database (for further clarification as needed).The three clinicians used a previously defined

explicit definition of a cardiopulmonary emergency (CPE, (Table 2) including any emergent thoracic

diagnoses that can be detected on CTPA requiring immediate treatment to prevent imminent deterioration:

this includes PE, defined as a filling defect in a segmental or larger pulmonary artery on enrollment

CTPA or within 90 days or high probability V/Q scan or ultrasound-proven DVT resulting in treatment

according to published guidelines;[24-26] pneumonia, aortic dissection or aneurysm, pneumothorax, new

thoracic mass or cardiac tamponade. Clinicians indicated a binary Yes/no determination regarding the

presence or absence for each of the 75 patients.[2] The final outcome of CPE required agreement of 2/3

clinicians and was found in 11/75 patients.

D. Experimental protocol

The experiment was conducted as a “Survey” in REDCap electronic data collection system that included

the Diagnostic Assessment of Nonverbal Accuracy (DANVA) test of facial affect recognition, followed

by standardized case reports of 75 patients shown to clinicians. [27]



Physicians filled their personal information, including their specialty and years of training (Figure 2).

Then, they were shown a presentation in PowerPoint that displayed a series of 24 slides of faces with

different emotional states. The pictures were from the Diagnostic Analysis of Nonverbal Accuracy Scale

(DANVA: Nowicki & Duke, 1994)[28]. Each slide depicted a specific facial expression and was shown

for two seconds. Physicians viewed each slide and answered the dominant emotion shown among four

options being displayed: Happy, Sad, Angry, or fearful. The entire test took about 2 minutes. The

DANVA served as a measure to determine physicians’ innate ability to recognize emotions from facial

expressions. The initial part of the affect recognition test is shown in Figure 3.

After submitting the affect recognition test, physicians were directed to the main part of the survey

comprising 75 videos of patients’ faces and corresponding questions and visual analogue scales (VAS)

before and after watching the video. Physicians could watch the videos all at once or in batches. Each

subject video was logged in a spreadsheet for that physician until the physician had completed all eligible

subject videos. Physicians first read a description of subject medical history (age, chief complaint, past

medical history and vital signs) that was prepared by consensus of two authors. All case histories are

given as an addendum in this supplement. The content and importance of the case history is justified by

prior evidence showing that emergency clinicians generate the majority of diagnostic hypothesis on these

criteria, and that novice clinicians tend to over-rely on these objective data in hypothesis

generation.[29,30] Then, physicians answered a set of 2 questions based on their interpretation of subject

medical history using a visual analogue scale. The first VAS asked, “What is the probability that this

patient has a life threatening disease process” (e.g., myocardial infarction pulmonary embolism, aortic

dissection, infection with sepsis, pneumothorax, etc…). The second (#2) VAS asked, “What is your

certainty that you will order a computerized tomographic (CT) scan of the chest with intravenous (IV)



contrast?” Physicians then watched the video of the patient’s face then answered the same set of questions

after viewing the video. This process is shown in Figure 5.

E. Recruitment of clinicians

Residents of postgraduate years 1-3 were eligible, as were fellows and faculty with any year of

experience. Clinicians were approached by one of the study authors and were given a verbal description

of the project and instructions on how to use the test instrument. In the process of obtaining informed

consent, authors attempted to distribute forms equally between residents, fellows, faculty with equal

representation of emergency medicine and internal medicine. Each completed the DANVA with a study

associate present, but they were able to take the remaining portion of the survey at their leisure.

Clinicians were enrolled while working in the emergency department, intensive care setting, on the wards,

or by appointment through email. Enrollment of physicians took place at any time that they were

available. (Emergency, and Internal Medicine) They had control of when to start and stop the viewing the

patient videos.

F. Data analysis and sample size

The primary analysis was the measurement of the clinician marks on the two sets of visual analogue

scales (Set #1 given after the case history and set #2 given after the video), 0-10 cm with 0 representing

0% and a 10 representing 100% measured in two sets (#1 and #2). The videos belonged to a subgroup of

patients enrolled in a larger pulmonary embolism registry and they all underwent CTPA scanning. Their

disposition was home, observation, or admission to the medicine wards or intensive care unit. Three

physician observers examined the patients’ medical records to determine whether they had a serious

cardiopulmonary emergency (CPE) diagnosed within 45 days. The explicit definitions of CPE were Acute

Coronary Syndrome (ACS), Pulmonary Embolism (PE), or other significant cardiopulmonary diagnosis



as highlighted in table 2. Additionally, we will calculate a receiver operating characteristic curve for each

VAS and report the mean and standard deviation area under the ROC for all clinicians before and after

viewing the patient videos. Recognizing gender differences in decision-making style, we planned in

advance to stratify responses according to physician gender. [31]

For the first aim of the study, testing the extent to which clinicians change their opinion about patient

prognosis and need for diagnostic testing based upon the patient’s facial affect, we sought to test for a 10

mm difference between set #1 and set #2 for the first VAS using a paired t-test, representing a 10%

difference in the clinician’s belief that the patient is experiencing a potentially life-threatening illness.

This difference is justified as significant on the basis that the minimum change in diagnostic certainty for

CT scanning of emergency department patients with dyspnea is approximately 10%.[32] Similarly, we

will examine the change in mean area under the ROC for the two VAS scales with a paired t-test. Prior

work with similar patients undergoing CTPA scanning using clinician entered VAS data has found that

clinicians will mark approximately a 16±15% degree of certainty that patient has a life threatening

condition[33]. The following analysis, as shown below, indicated need for 20 clinicians from each

specialty (emergency medicine and internal medicine including intensive care) at each training level. Thus

at least 30 clinicians from each specialty were enrolled (PGY1, PGY3 and staff), assuming a worst-case

of 30% drop-out.)

Probability of type I error, Alpha = 0.05, probability of type II error (Beta) = 0.2 or 80% power to detect a

10 mm difference from zero with a standard deviation = 15 mm requires an estimated minimum sample

size = 20. Because we want to have 20 pairs of either gender, we estimate a minimum sample size of 50

physicians with complete data

Project Administration



The project was administered by the Indiana University School of Medicine. The research team

comprised of emergency and internal medicine attending physicians and residents, a research professor, a

clinical research specialist, and a medical student. A biostatistician was also involved in the study design

and data management. The residents and medical student that were part of the research team conducted

physician enrollment and data collection.

Ethical Considerations

Ethical approval was granted by the IRB at Indiana University. All patients gave written informed consent

to shoot a film of their faces for the purpose of research. Similarly, all clinicians selected for the survey

were required to sign a consent form before starting the survey. A copy of the informed consent was

offered to every physician. Clinicians may have experienced temporary anxiety in trying to formulate a

decision about the patient but this was temporary. There were no potential risks for physical discomfort

involved in participating in this study, as it was observational and for data collection only. To reduce

anxiety, clinicians were assured that their results were anonymous and that they were going to be

informed about patient outcomes if they wish. Clinicians, as well as patients, could opt out of the study at

any time. All physicians and patients were adults at least 21 years of age.

Data Management and Data Quality

Research electronic and data capture (REDCap) software was used for entering, editing, and charting

survey records.



RESULTS SUPPLEMENTAL TO MAIN PAPER

Overview of All Clinicians

We recruited 179 clinicians, who collectively completed 4,524 patient modules. Seeing the patients’ faces

on video caused an increase in VAS reported probability of CPE in 35%, and decrease in 48% and no

change in 17%. The video increased desire for CTPA in 35%, decreased it in 43% and caused no change

in 22%. To determine the absolute magnitude of effect (increase or decrease) of the patients’ faces on

physician pretest probability, we subtracted the first VAS from the second VAS (VAS after seeing face –

VAS before seeing face) for each of 4,524 case encounters for the change probability of a CPE and

change in desire to order a CTPA. Negative values were converted to positive to yield the absolute VAS

change caused by viewing the patient faces: for pretest probability of CPE this was 10.2 (SD 3.8) and for

desire for CTPA this was 10.0 (SD3.7). Seeing the video caused a >10% change in VAS probability of

emergency in 1,695/4,524 (37%) and >10% change in VAS need for CT in 1,624/4,524 (36%).

We compared features of the 50 physicians who completed the module to the 129 who started but did not

complete the module. Completers were older than non-completers (34 vs. 31 years, ∆=3, 95% CI for

difference in means 0.1 to 5.7) faculty [(24/50 (48%) vs. 22/129 (17%) ∆=31%, 95% CI 16 to 46%], and

more likely to be from emergency medicine [34/50 (68%) vs. 33/129 (26%) ∆=42%, 95% CI 27 to 56%.

Female gender distribution was relatively similar (17/50 (34%) vs. 48/129 (37%) ∆=3%, 95% CI -13 to

18%. Completers had a slightly higher DANVA score (18.9 vs. 18.0, ∆ = 0.9, 95% CI 0.1 to 1.7). For the

50 completers, the mean first and second VAS score for CPE were 45 (14) and 43 (16) and for CT were

36 (13) and 35 (14). For the 13 non-completers who scored more than half of the 73 videotaped patients,

the mean first and second VAS score for CPE was 48 (12) and 46 (12) and for CT was 43 (13) and 43



(13). For the 113 non-completers who scored less than half of the patients, the mean first and second VAS

score for CPE were 53 (12) and 48 (13) and for CT was 47 (11) and 43 (13).

Discussion

Here, we present a methodology to assess the contribution of patient affect to clinical judgment, which we

believe to be conceptually similar to gestalt reasoning, system I (rapid) reasoning, and expert

intuition.[12-14,17] Rapid and accurate pretest probability assessment can improve appropriateness of

diagnostic test ordering.[34] Pretest probability assessment is a major factor that guides clinicians to make

decisions about diagnostic testing and prognosis. One tool that clinicians use to assess pretest probability

of disease is the overall appearance of their patients. This protocol will allow insight into the magnitude

and direction of looking at the patients’ faces on physician pretest probability assessment.

Reference List

(1) Kline JA, Neumann DM, . Relationship of patient affect to pretest probability for pulmonary

embolism. Journal of Thrombosis and Haemostasis 13[S2], 978. 2015.

(2) Kline JA, Shapiro NI, Jones AE, Hernandez-Nino J, Hogg MM, Nelson RD, Troyer JL. Outcomes

and radiation exposure of emergency department patients with chest pain and shortness of breath

and ultralow pretest probability. Ann Emerg Med 2014;63:281-8.

(3) Schuur JD, Carney DP, Lyn ET, Raja AS, Michael JA, Ross NG, Venkatesh AK. A top-five list

for emergency medicine: a pilot project to improve the value of emergency care. JAMA Intern

Med 2014;174:509-15.



(4) Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, de FS, Despres JP,

Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH,

Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, et al. Heart disease and stroke

statistics--2015 update: a report from the American Heart Association. Circulation 2015;131:e29-

322.

(5) Feng LB, Pines JM, Yusuf HR, Grosse SD. U.S. trends in computed tomography use and

diagnoses in emergency department visits by patients with symptoms suggestive of pulmonary

embolism, 2001-2009. Acad Emerg Med 2013;20:1033-40.

(6) Mitchell AM, Kline JA, Jones AE, Tumlin JA. Major Adverse Events One Year After Acute

Kidney Injury After Contrast-Enhanced Computed Tomography. Ann Emerg Med 2015;66:267-

74.

(7) Adams DM, Stevens SM, Woller SC, Evans RS, Lloyd JF, Snow GL, Allen TL, Bledsoe JR,

Brown LM, Blagev DP, Lovelace TD, Shill TL, Conner KE, Aston VT, Elliott CG. Adherence to

PIOPED II investigators' recommendations for computed tomography pulmonary angiography.

Am J Med 2013;126:36-42.

(8) Schissler AJ, Rozenshtein A, Kulon ME, Pearson GD, Green RA, Stetson PD, Brenner DJ,

D'Souza B, Tsai WY, Schluger NW, Einstein AJ. CT pulmonary angiography: increasingly

diagnosing less severe pulmonary emboli. PLoS One 2013;8:e65669.

(9) Studdert DM, Mello MM, Sage WM, DesRoches CM, Peugh J, Zapert K, Brennan TA. Defensive

medicine among high-risk specialist physicians in a volatile malpractice environment.[see

comment]. JAMA 2005;293:2609-17.



(10) Lucas FL, Sirovich BE, Gallagher PM, Siewers AE, Wennberg DE. Variation in cardiologists'

propensity to test and treat: is it associated with regional variation in utilization? Circ Cardiovasc

Qual Outcomes 2010;3:253-60.

(11) Rothberg MB, Class J, Bishop TF, Friderici J, Kleppel R, Lindenauer PK. The Cost of Defensive

Medicine on 3 Hospital Medicine Services. JAMA Intern Med 2014.

(12) Norman G, Sherbino J, Dore K, Wood T, Young M, Gaissmaier W, Kreuger S, Monteiro S. The

etiology of diagnostic errors: a controlled trial of system 1 versus system 2 reasoning. Acad Med

2014;89:277-84.

(13) Croskerry P, Petrie DA, Reilly JB, Tait G. Deciding about fast and slow decisions. Acad Med

2014;89:197-200.

(14) Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009;84:1022-8.

(15) Pelaccia T, Tardif J, Triby E, Ammirati C, Bertrand C, Charlin B, Dory V. Insights into

emergency physicians' minds in the seconds before and into a patient encounter. Intern Emerg

Med 2015;10:865-73.

(16) Wiswell J, Tsao K, Bellolio MF, Hess EP, Cabrera D. "Sick" or "not-sick": accuracy of System 1

diagnostic reasoning for the prediction of disposition and acuity in patients presenting to an

academic ED. Am J Emerg Med 2013;31:1448-52.

(17) Calder LA, Arnason T, Vaillancourt C, Perry JJ, Stiell IG, Forster AJ. How do emergency

physicians make discharge decisions? Emerg Med J 2015;32:9-14.



(18) Ritz T, Claussen C, Dahme B. Experimentally induced emotions, facial muscle activity, and

respiratory resistance in asthmatic and non-asthmatic individuals. Br J Med Psychol 2001;74 Part

2:167-82.

(19) Kline JA, Neumann D, Haug MA, Kammer DJ, Krabill VA. Decreased facial expression

variability in patients with serious cardiopulmonary disease in the emergency care setting. Emerg

Med J 2014;32:3-8.

(20) Kanzaria HK, Hoffman JR, Probst MA, Caloyeras JP, Berry SH, Brook RH. Emergency physician

perceptions of medically unnecessary advanced diagnostic imaging. Acad Emerg Med

2015;22:390-8.

(21) Geyer BC, Xu M, KABRHEL C. Patient preferences for testing for pulmonary embolism in the

ED using a shared decision-making model. Am J Emerg Med 2014;32:233-6.

(22) Legare F, Moher D, Elwyn G, LeBlanc A, Gravel K. Instruments to assess the perception of

physicians in the decision-making process of specific clinical encounters: a systematic review.

BMC Med Inform Decis Mak 2007;7:30.

(23) Lang PJ, Bradley MM, Cuthbert BN. International affective picture system: Affective ratings of

pictures and instruction manual.

(IAPS): Affective ratings of pictures and instruction manual. Gainesville, FL: University of Florida; 2008.

Report No.: Technical Report A-8.

(24) PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism.

JAMA 1990;263:2753-9.



(25) Courtney DM, Miller CD, Smithline HA, Klekowski N, Hogg MM, Kline JA. Prospective multi-

center assessment of interobserver agreement for radiologist interpretation of multidetector CT

angiography for pulmonary embolism. Journal of Thrombosis & Haemostasis 2010;8:533-40.

(26) Kearon C, Akl EA, Comerota AJ, Prandoni P, Bounameaux H, Goldhaber SZ, Nelson ME, Wells

PS, Gould MK, DENTALI F, Crowther M, Kahn SR. Antithrombotic therapy for VTE disease:

Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest

Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012;141:e419S-e494S.

(27) Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture

(REDCap)--a metadata-driven methodology and workflow process for providing translational

research informatics support. J Biomed Inform 2009;42:377-81.

(28) Nowicki S, Duke MP. Individual differences in the nonverbal communication of affect: The

Diagnostic Analysis of Nonverbal Accuracy Scale. J Nonverbal Behav 1994;18:9-35.

(29) Pelaccia T, Tardif J, Triby E, Ammirati C, Bertrand C, Dory V, Charlin B. How and when do

expert emergency physicians generate and evaluate diagnostic hypotheses? A qualitative study

using head-mounted video cued-recall interviews. Ann Emerg Med 2014;64:575-85.

(30) Schubert CC, Denmark TK, Crandall B, Grome A, Pappas J. Characterizing novice-expert

differences in macrocognition: an exploratory study of cognitive work in the emergency

department. Ann Emerg Med 2013;61:96-109.

(31) Calder LA, Forster AJ, Stiell IG, Carr LK, Brehaut JC, Perry JJ, Vaillancourt C, Croskerry P.

Experiential and rational decision making: a survey to determine how emergency physicians make

clinical decisions. Emerg Med J 2012;29:811-6.



(32) Pandharipande PV, Reisner AT, Binder WD, Zaheer A, Gunn ML, Linnau KF, Miller CM, Avery

LL, Herring MS, Tramontano AC, Dowling EC, Abujudeh HH, Eisenberg JD, Halpern EF,

Donelan K, Gazelle GS. CT in the Emergency Department: A Real-Time Study of Changes in

Physician Decision Making. Radiology 2015;150473.

(33) Kline JA, Stubblefield WB. Clinician gestalt estimate of pretest probability for acute coronary

syndrome and pulmonary embolism in patients with chest pain and dyspnea. Ann Emerg Med

2014;63:275-80.

(34) Roy PM, Durieux P, Gillaizeau F, Legall C, rmand-Perroux A, Martino L, Hachelaf M, Dubart

AE, Schmidt J, Cristiano M, Chretien JM, Perrier A, Meyer G. A computerized handheld

decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann

Intern Med 2009;151:677-86.

(35) Lucassen W, Geersing GJ, Erkens PM, Reitsma JB, Moons KG, Buller H, van Weert HC. Clinical

Decision Rules for Excluding Pulmonary Embolism: A Meta-analysis. Ann Intern Med

2011;155:448-60.

(36) Runyon MS, Richman PB, Kline JA. Emergency medicine practitioner knowledge and use of

decision rules for the evaluation of patients with suspected pulmonary embolism: variations by

practice setting and training level. Acad Emerg Med 2007;14:53-7.

(37) Kline JA. Making the black box more useful. Emerg Med J 2014.

(38) Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS. Physicians' empathy

and clinical outcomes for diabetic patients. Acad Med 2011;86:359-64.



(39) Steinhausen S, Ommen O, Antoine SL, Koehler T, Pfaff H, Neugebauer E. Short- and long-term

subjective medical treatment outcome of trauma surgery patients: the importance of physician

empathy. Patient Prefer Adherence 2014;8:1239-53.



Table 1: Clinical characteristics of enrolled patients (N=73)

Female gender 45 61.64 %

White race 38 52 %

Triage code >2 %

Admitted to hospital 23 31.5%

Age (years) 49 SD 17

Heart rate (beats/min) 95.1 SD 17.48

Systolic BP (mm Hg) 142.96 SD 30.82

SpO2 97.5 SD 2.62

Respiratory rate (breaths/min) 19.1 SD 4.3

Temperature (degrees C) 36.84 SD 0.55



Table 2: Definitions of cardiopulmonary emergency

Criterion standard for ACS Definition was consistent with published guidelines. This included:

myocardial infarction, defined by either > 0.1 uV of ST segment

elevation in two contiguous leads on 12 lead electrocardiogram, or a

troponin I concentration that was above the 99th percentile with <10%

coefficient of variability (standard deviation/mean) with no prior history

of prior chronic troponin elevation., death thought to be from ACS

(autopsy not required), need for revascularization (angioplasty, stent

placement, or surgical bypass grafting), or a coronary stenosis on cardiac

catheterization >60% not amenable to stent placement and requiring new

medical management. A positive stress test or nuclear cardiology test

followed by new medical management in the absence of any other

criteria was not considered ACS.

Criterion standard for PE Diagnosis of PE required the medical record to demonstrate evidence of

a filling defect in a pulmonary artery on pulmonary vascular imaging

together with a clinical plan to treat, or evidence of a compression

ultrasound positive for deep vein thrombosis in the setting of

nondiagnostic pulmonary vascular imaging

Criterion standard for other

significant cardiopulmonary

diagnosis

Major adverse cardiac events (dysrhythmia requiring treatment, acute

heart failure, or pericarditis with effusion), aortic aneurysm or dissection,

other pulmonary diagnosis (pneumothorax, pneumonia), dangerous

mediastinal processes, or hemorrhage related to treatment.



Figure 1: Visual stimuli shown to patients



Figure 2: Physicians’ personal information



Figure 3: Affect Recognition Test



Figure 4: Example of case history



Figure 5: Example of what clinicians viewed. On the left is the survey with the standardized history at the top. The

patient’s video appeared after the clinician recorded his or her initial VAS estimates from the standardized history.

The clinician then revised the VAS estimates after seeing the patient face. (Although all patients gave written

informed consent to have their faces shown for teaching and publication, this patient’s eyes are obscured to help

protect privacy).



STANDARDIZED CASE HISTORIES—ALL PATIENTS

Faces 1

Age: 59

Primary symptom: Mental Status Change

PMH: Asthma, hypertension, meningioma and seizures

Temp: 36.7

HR: 91

Resp: 21

SpO2: 97

BP: 163/88

HT(cm): 175.26

WT(kg): 109.8

Faces 2

Age: 74

Primary symptom: Dyspnea

PMH: Hypertension, anemia, GI bleed, and anxiety

Temp: 36.0

HR: 88

Resp: 14

SpO2: 100

BP: 160/67

HT(cm): 157.5

WT(kg): 90.7

Faces 3



Age: 57


PMH: PE and DVT on anticoagulation therapy, CAD, Hypertension, prior stroke, and HIV

Temp: 36.6

HR: 108

Resp: 18

SpO2: 99

BP: 121/83

HT(cm): 182.9

WT(kg): N/A

Faces 4

Age: 38


PMH: Pulmonary embolism, CAD, diabetes, hypertension and sleep apnea

Temp: 36.7

HR: 95

Resp: 20

SpO2: 99

BP: 189/102

HT(cm): 185

WT(kg): 179.3

Faces 5

Age: 49

Primary symptom: substernal chest pain



PMH: Pulmonary embolism and DVT, CAD, hypertension, sleep apnea, depression, GERD, personality disorder

unspecified

Temp: 37.0

HR: 110

Resp: 24

SpO2: 97

BP: 126/110

HT(cm): 185.4

WT(kg): 147.7

Faces 6

Age: 35

Primary symptom: Substernal chest pain

PMH: Smoker and pulmonary embolism

Temp: 37.4

HR: 95

Resp: 18

SpO2: 95

BP: 134/79

HT(cm): 160.0

WT(kg): 113.7

Faces 7

Age: 85


PMH: Hypertension, atrial fibrillation, and HLD

Temp: 38

HR: 93



Resp: 20

SpO2: 95

BP: 179/58

HT(cm): 188.0

WT(kg): 102.3

Faces 8

Age: 48


PMH: Prior history of pulmonary embolism and DVT, hypertension, and CHF

Temp: 36.8

HR: 108

Resp: 17

SpO2: 100

BP: 219/146

HT(cm): 195

WT(kg): 181.9

Faces 9

Age: 82


PMH: Asthma and hypertension

Temp: 36.8

HR: 114

Resp: 21

SpO2: 99



BP: 168/81

HT(cm): 167.6

WT(kg): 79.3

Faces 10

Age: 49


PMH: COPD, diabetes, and hypertension

Temp: 36.4

HR: 91

Resp: 14

SpO2: 89

BP: 134/87

HT(cm): 167

WT(kg): 104.04

Faces 11

Age: 54


PMH: Diabetes, Hypertension, and GERD

Temp: 36.8

HR: 83

Resp: 18

SpO2: 97

BP: 214/104



HT(cm): 180.32

WT(kg): 138.6

Faces 12

Age: 39

Primary symptom: Pleuritic chest pain

PMH: Hypothyroidism, peptic ulcer disease, eating disorder unspecified, and anxiety disorder

Temp: 36.6

HR: 92

Resp: 15

SpO2: 100

BP: 111/78

HT(cm): 157.5

WT(kg): 38.1

Faces 13

Age: 31

Primary symptom: Weakness

PMH: Paraplegia, hypertension, and rheumatoid arthritis

Temp: 38.0

HR: 126

Resp: 27

SpO2: 96

BP: 73/50

HT(cm): 182.9



WT(kg): 95.5

Faces 14

Age: 19


PMH: GERD, current oral contraception

Temp: 36.7

HR: 122

Resp: 18

SpO2: 100

BP: 137/95

HT(cm): 160.0

WT(kg): 74.6

Faces 15

Age: 40

Primary symptom: Substernal chest pain,

PMH: Asthma and diabetes

Temp: 36.7

HR: 105

Resp: 26

SpO2: 100

BP: 112/76

HT(cm): 157

WT(kg): 112.9



Faces 16

Age: 74


PMH: CAD, pulmonary embolism, hypertension, CHF, hyperlipidemia, atrial fibrillation, and history of lung

nodule

Temp:

HR: 75

Resp: 16

SpO2: 94

BP: 126/50

HT(cm): 162.6

WT(kg): 98.4

Faces 17

Age: 45

Primary symptom: Syncope or Seizure

PMH: Prothrombin mutation, on anticoagulation therapy for DVT, history of TIA, and gastric ulcer

Temp: 36.4

HR: 108

Resp: 12

SpO2: 98

BP: 116/46

HT(cm): 173

WT(kg): 105.2



Faces 18

Age: 77


PMH: COPD

Temp: 36.2

HR: 85

Resp: 17

SpO2: 95

BP: 129/54

HT(cm): 175

WT(kg): 71.9

Faces 19

Age: 58


PMH: CAD, hypertension, CHF, and hyperlipidemia

Temp: 36.7

HR: 71

Resp: 14

SpO2: 97

BP: 133/75

HT(cm): 182

WT(kg): 92.4

Faces 20

Age: 49



Primary symptom: Pleuritic Chest Pain

PMH: On anticoagulation therapy for PE and DVT, asthma, COPD, and fibromyalgia

Temp: 37.0

HR: 100

Resp: 20

SpO2: 99

BP: 139/73

HT(cm): 165.4

WT(kg): 100.8

Faces 21

Age: 50

Primary symptom: Hemoptysis

PMH: COPD

Temp: 36.8

HR: 59

Resp: 16

SpO2: 97

BP: 157/97

HT(cm): 177.8

WT(kg): 66

Faces 22

Age: 33




PMH: On anticoagulation therapy for pulmonary embolism, Gitelman Syndrome, and Uterine/Cervical Cancer s/p

treatment

Temp: 37.3

HR: 99

Resp: 16

SpO2: 99

BP: 111/68

HT(cm): 167.6

WT(kg): 77.3

Faces 23

Age: 32


PMH: Lupus, history of DVT, Addison’s disease, autoimmune hepatitis, anemia, vitamin D deficiency

Temp: 36.1

HR: 108

Resp: 16

SpO2: 99

BP: 117/74

HT(cm): 165.1

WT(kg): 86.4

Faces 24

Age: 60

Primary symptom: Substernal Chest Pain



PMH: COPD, CAD, diabetes, hypertension, paroxysmal atrial fibrillation, hypothyroidism, CHF and history of

endometrial carcinoma s/p treatment

Temp: 37.8

HR: 134

Resp: 14

SpO2: 98

BP: 202/82

HT(cm): 172

WT(kg): 99.1

Faces 25

Age: 64


PMH: Hypertension

Temp: 36.2

HR: 111

Resp: 30

SpO2: 98

BP: 179/114

HT(cm): 167.6

WT(kg): 69.6

Faces 26

Age: 67




PMH: CAD s/p CABG and multiple PCIs, diabetes, hypertension, hyperlipidemia, recently diagnosed

adenocarcinoma of colon, left BKA, and history of renal cell carcinoma s/p nephrectomy

Temp: 36.8

HR: 98

Resp: 20

SpO2: 97%

BP: 123/66

HT(cm): 176

WT(kg): 72.7

Faces 27

Age: 60


PMH: Hypertension, hyperlipidemia, hypothyroid, and glioblastoma undergoing chemotherapy

Temp: 36.5

HR: 68

Resp: 18

SpO2: 97%

BP: 117/64

HT(cm): 162.6

WT(kg): 78.4

Faces 28

Age: 29


PMH: Asthma and Hypertension



Temp: 37.2

HR: 126

Resp: 23

SpO2: 96

BP: 149/73

HT(cm): 180

WT(kg): 91

Faces 29

Age: 20


PMH: Smoker

Temp: 36.9

HR: 135

Resp: 18

SpO2: 100%

BP: 143/94

HT(cm): 156

WT(kg): 76.5

Faces 30

Age: 41


PMH: CAD, diabetes, hypertension, PVD with right below knee amputation, CKD stage 3

Temp: 36.4

HR: 104



Resp: 18

SpO2: 100%

BP: 229/93

HT(cm): 185.4

WT(kg): 88.2

Faces 31

Age: 33


PMH: Pulmonary embolism, obstructive sleep apnea

Temp: 36.8

HR: 87

Resp: 17

SpO2: 100%

BP: 158/89

HT(cm): 165

WT(kg): 105.2

Faces 32

Age: 36


PMH: On anticoagulation therapy for PE and DVT, Factor 5 Leiden variation, other genetic induced clotting

disorders

(PAI-1, MTHFR), asthma, hypertension, GERD

Temp: 36.6

HR: 77



Resp: 23

SpO2: 97

BP: 121/76

HT(cm): 162.6

WT(kg): 72.7

Faces 33

Age: 28


PMH: On anticoagulation for pulmonary embolism, C-section within last month, and asthma

Temp: 37.0

HR: 81

Resp: 16

SpO2: 100%

BP: 102/82

HT(cm): 165

WT(kg): 80

Faces 34

Age: 59


PMH: COPD, chronic interstitial PNA, prior pulmonary embolism, HTN, CKD, depression

Temp: 37.7

HR: 135

Resp: 24

SpO2: 100%

BP: 190/69



HT(cm): 170.2

WT(kg): 116.1

Faces 35

Age: 81


PMH: Hypertension

Temp: 36.4

HR: 73

Resp: 24

SpO2: 96%

BP: 127/67

HT(cm): 193

WT(kg): 88.7

Faces 36

Age: 18


PMH: No past medical history

Temp: 36.2

HR: 88

Resp: 18

SpO2: 97

BP: 136/70

HT(cm): 160

WT(kg): 73.8



Faces 37

Age: 35


PMH: Smoker, on anticoagulation therapy for DVT

Temp: N/A

HR: 101

Resp: 18

SpO2: 98%

BP: 138/86

HT(cm): 170.2

WT(kg): 114.1

Faces 38

Age: 76


PMH: Hypertension, history of DVT, metastatic prostate cancer with current oral chemotherapy, and anxiety

Temp: 36.4

HR: 77

Resp: 20

SpO2: 98

BP: 152/91

HT(cm): 173

WT(kg): 69.4

Faces 39



Age: 53


PMH: Breast cancer s/p mastectomy and current chemotherapy

Temp: 36.6

HR: 80

Resp: 17

SpO2: 97%

BP: 125/58

HT(cm): 157

WT(kg): 64

Faces 40

Age: 21


PMH: Smoker and prior history of pulmonary embolism

Temp: 37.2

HR: 79

Resp: 16

SpO2: 96%

BP: 139/75

HT(cm): 175.3

WT(kg): 181.8

Faces 41

Age: 32




PMH: Asthma, personality disorder unspecified, anxiety disorder

Temp: 37.1

HR: 80

Resp: 20

SpO2: 98%

BP: 125/80

HT(cm): 167.6

WT(kg): 102.3

Faces 42

Age: 53


PMH: COPD, smoker, and history of pericarditis

Temp: 36.8

HR: 107

Resp: 17

SpO2: 95%

BP: 123/76

HT(cm): 170

WT(kg): 153.2

Faces 43

Age: 50


PMH: Diabetes, Hypertension, GERD, Hepatitis C, schizophrenia, and prior history of PE per patient



Temp: 37.0

HR: 96

Resp: 18

SpO2: 100%

BP: 157/87

HT(cm): 182

WT(kg): 78.6

Faces 44

Age: 66


PMH: PVD s/p LE angioplasty within one week, hypertension, CHF, COPD, smoker, and anxiety disorder

Temp: 36.7

HR: 93

Resp: 20

SpO2: 97%

BP: 136/59

HT(cm): 174

WT(kg): 62.7

Faces 45

Age: 34


PMH: Lupus, antiphospholipid antibody, and smoker

Temp: 36.8

HR: 91



Resp: 16

SpO2: 100%

BP: 105/65

HT(cm): 265

WT(kg): 77.8

Faces 46

Age: 91


PMH: History of DVT, hypertension, PVD, smoker, CKD, NSCLC, and history of bladder cancer

Temp: 36.1

HR: 81

Resp: 22

SpO2: 96%

BP: 168/93

HT(cm): 152.4

WT(kg): 59.1

Faces 47

Age: 51


PMH: COPD, hypertension, CREST syndrome, interstitial lung disease

Temp: 37.0

HR: 106

Resp: 20



SpO2: 96%

BP: 176/97

HT(cm): 170.2

WT(kg): 90.4

Faces 48

Age: 34

Primary symptom: Hemoptysis

PMH: History of pulmonary embolism, diabetes, and hypertension

Temp: 36.8

HR: 85

Resp: 16

SpO2: 99

BP: 118/68

HT(cm): 154.9

WT(kg): 84.2

Faces 49

Age: 69

Primary symptom: Cough

PMH: Leukemia with ongoing therapy, DVT on anticoagulation

Temp: 37.1

HR: 95

Resp: 18

SpO2: 95%



BP: 125/74

HT(cm): 165

WT(kg): 60.4

Faces 50

Age: 70


PMH: CAD, history of pulmonary embolism, anxiety disorder

Temp: 37.5

HR: 74

Resp: 20

SpO2: 98%

BP: 158/93

HT(cm): 177

WT(kg): 69.1

Faces 52

Age: 35


PMH: HIV, smoker, and non-cirrhotic portal hypertension with recent TIPS revision under conscious sedation

Temp: 37.2

HR: 77

Resp: 14

SpO2: 98%

BP: 128/48

HT(cm): 172



WT(kg): 59.1

Faces 53

Age: 43


PMH: orthopedic surgery within last month

Temp: 36.7

HR: 90

Resp: 16

SpO2: 98%

BP: 144/85

HT(cm): 165

WT(kg): 81.9

Faces 54

Age: 27


PMH: Asthma

Temp: 36.7

HR: 102

Resp: 20

SpO2: 97%

BP: 149/85

HT(cm): 152.4

WT(kg): 88.2



Faces 55

Age: 57


PMH: HIV, rheumatoid arthritis, smoker, anxiety disorder

Temp: 36.8

HR: 88

Resp: 16

SpO2: 94%

BP: 113/69

HT(cm): 162

WT(kg): 80

Faces 56

Age: 51


PMH: Diabetes, Hypertension, Depression, and history of pericarditis

Temp: 36.8

HR: 110

Resp: 26

SpO2: 98%

BP: 142/102

HT(cm): 160

WT(kg): 91.9

Faces 57



Age: 57


PMH: History of multiple DVTs with recent cessation of anticoagulation, hypertension, hyperlipidemia, and

anxiety/depression

Temp: 36.2

HR: 121

Resp: 28

SpO2: 100%

BP: 166/113

HT(cm): 160

WT(kg): 59.1

Faces 58

Age: 34


PMH: Smoker

Temp: 36.7

HR: 86

Resp: 16

SpO2: 98

BP: 146/72

HT(cm): 169.5

WT(kg): 75.7

Faces 59

Age: 60




PMH: History of PE and DVT, COPD, CAD, diabetes, hypertension, PVD with right foot amputation, ESRD on

dialysis, atrial fibrillation, GERD, Hepatitis C, and obstructive sleep apnea

Temp: 36.2

HR: 76

Resp: 12

SpO2: 100%

BP: 182/94

HT(cm): 172.7

WT(kg): 100

Faces 60

Age: 55


PMH: Anxiety and personality disorder unspecified

Temp: 36.4

HR: 104

Resp: 28

SpO2: 98%

BP: 123/73

HT(cm): N/A

WT(kg): 72.6

Faces 61

Age: 74




PMH: CAD

Temp: 36.2

HR: 72

Resp: 17

SpO2: 98%

BP: 122/80

HT(cm): 177.8

WT(kg): 76.2

Faces 62

Age: 24


PMH: Smoker, elective abortion 1 month ago

Temp: 36.6

HR: 88

Resp: 22

SpO2: 99%

BP: 105/56

HT(cm): 149.9

WT(kg): 117.5

Faces 64

Age: 32


PMH: Anxiety disorder

Temp: 36.9



HR: 107

Resp: 24

SpO2: 96%

BP: 159/99

HT(cm): 167.6

WT(kg): 77.2

Faces 65

Age: 50


PMH: On anticoagulation therapy for DVT and PE, antithrombin deficiency, asthma, COPD, and hypertension

Temp: 36.9

HR: 89

Resp: 14

SpO2: 100%

BP: 154/119

HT(cm): 167.6

WT(kg): 154.5

Faces 66

Age: 54


PMH: CAD, hypertension, lung cancer with associated brain and skeletal metastasis undergoing chemotherapy and

radiation, smoker

Temp: 36.8

HR: 95



Resp: 16

SpO2: 100%

BP: 116/91

HT(cm): 165.1

WT(kg): 64.3

Faces 67

Age: 59


PMH: Hypertension, obstructive sleep apnea, depression

Temp: 37.5

HR: 83

Resp: 19

SpO2: 97

BP: 145/91

HT(cm): 188

WT(kg): 157.8

Faces 68

Age: 46


PMH: Asthma, hypertension, obstructive sleep apnea, GERD, depression, personality disorder

Temp: 37.2

HR: 66

Resp: 18

SpO2: 97%



BP: 107/61

HT(cm): 162.5

WT(kg): 109

Faces 69

Age: 45


PMH: Hypertension, DVT on anticoagulation therapy

Temp: 36.5

HR: 70

Resp: 21

SpO2: 98

BP: 137/95

HT(cm): 160

WT(kg): 74.3

Faces 70

Age: 61


PMH: Diabetes, hypertension, HLD, stroke, Rheumatoid arthritis, and Wolff Parkinson White

Temp: 36.4

HR: 112

Resp: 21

SpO2: 99

BP: 232/137

HT(cm): 167.64



WT(kg): 86.4

Faces 71

Age: 28


PMH: Smoker, history of pulmonary embolism

Temp: 36.7

HR: 95

Resp: 16

SpO2: 99%

BP: 143/82

HT(cm): 170.2

WT(kg): 71.7

Faces 72

Age: 62

Primary symptom: Cough

PMH: COPD, CAD, hypertension, smoker, anxiety

Temp: 39.0

HR: 130

Resp: 18

SpO2: 83%

BP: 101/71

HT(cm): 169.6

WT(kg): 56.7



Faces 73

Age: 44


PMH: COPD, hypertension, and anxiety

Temp: 37

HR: 117

Resp: 36

SpO2: 97%

BP: 136/89

HT(cm): 152.4

WT(kg): 76.6

Faces 74

Age: 52


PMH: CAD, diabetes, hypertension, paroxysmal atrial fibrillation, obstructive sleep apnea, factor 8 deficiency

Temp: 38.8

HR: 94

Resp: 21

SpO2: 96%

BP: 155/73

HT(cm): 167.6

WT(kg): 118.2

Faces 75

Age: 46




PMH: Hypertension and smoker

Temp: 36.4

HR: 85

Resp: 18

SpO2: 99%

BP: 142/96

HT(cm): 172.7

WT(kg): 89.5




Figure 1A



Figure 1B

Plot of the physician visual analogue scale (0-100) result for the estimate of the desire to order a

computerized tomographic pulmonary angiogram after reading the standardized case history (Figure 1A),

followed by the score after seeing the patient’s face (Figure 1B). Plots show the median and 1st-3rd

quartile (X axis) for 50 physician completers for each of 73 patients (Y axis).




Figure 2A



Figure 2B

Plots (boxes) of visual analogue (VAS) data from 50 completer physicians for the probability of

cardiopulmonary emergency (Figure 2A) and desire for CTPA (Figure 2B). The boxes are organized from

the least experienced physicians in the upper left corner) to the most experienced in the lower right hand

corner in terms of diagnostic accuracy caused by looking at the patient face. In each box, the X-axis

represents patient number, and the left to right direction indicates increasing initial VAS value after

reading the case report. The vertical bars represent the deflection in VAS caused by seeing the patient



face. Red lines are PE+ and black bold are other CPE+ patients; light gray vertical lines represent CPE-

patients.

Documents

Supplemental Digital Appendix 1