48
WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope of research and scholarship Subha Ramani MBBS, MMEd, MPH Gail Sullivan, MD, MPH Matthew Tuck, MD, MEd Katherine Chretien, MD SGIM National Meeting 2016 Hollywood, Florida May 12, 2016 from 1:30 PM to 3:00 PM 1

WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Embed Size (px)

Citation preview

Page 1: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

WD07

Moving beyond numbers: Applying qualitative methodology to expand the scope of research and scholarship

Subha Ramani MBBS, MMEd, MPH

Gail Sullivan, MD, MPH Matthew Tuck, MD, MEd Katherine Chretien, MD

SGIM National Meeting 2016 Hollywood, Florida 

May 12, 2016 from 1:30 PM to 3:00 PM 

1

Page 2: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

WORKSHOP GOALS

•Understand the applications of qualitative methods in health services and educational research

•Contrast qualitative and quantitative research methodology

•Discuss 3 main approaches to qualitative research

•Describe and practice key steps involved in qualitative study design

•Discuss strategies to ensure rigor in qualitative research

2

Page 3: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

QUALITATIVE RESEARCH

1. Natural settings are the source of

data and the investigator is an

instrument

2. Data are words rather than numbers

3. Concerned with process as well as

product

4. Analysis is inductive… mostly

5. Researchers are interested in the

perspectives of subjects

3

Page 4: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Research method

Qualitative Quantitative

Logic Inductive Deductive

Hypothesis Describe, understand people/groups, subsequently develop/test theory

Test hypotheses, isolate/manipulate variables to test their validity

Design Emergent, flexible; findings affect subsequent stages of research

Controlled; variation in design and data is not desirable

Sampling Purposive More random

Data Words , observed Numbers, measured

Data collection & analysis

Non-linear; concurrent data collection and analysis

Linear; analysis begins after data collection is completed

4

Page 5: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STUDY QUESTIONS

Quantitative

•What percentage of diabetic patients take their insulin daily?

•What percentage of teachers provide constructive feedback to trainees in your ... course?

•What percentage of patients with chronic pain achieve satisfactory pain control?

•What are student scores after implementation of the new curriculum?

Qualitative

•What factors prevent diabetic patients from taking their insulin daily?

•What barriers prevent teachers from providing constructive feedback to students?

•What are the factors that contribute to poor pain control in patients?

•What are students’ opinions about the new curriculum?

5

Page 6: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

ESSENTIAL STEPS IN QUALITATIVE STUDY DESIGN

6

Page 7: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STEPS

PhenomenologyGrounded theoryEthnography

•Topic

•Study question/s

•Framework

•Design•Population (teachers, trainees, leaders)•Sampling•Data•Outcome

•Analysis

•Quality and rigor

7

Page 8: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

APPROACHES TO QUALITATIVE RESEARCH

EthnographyStudy of social interactions and cultural patterns in their natural settings

PhenomenologyStudying the reactions to or perceptions of a phenomenon or lived experience

Grounded theory Developing a theory that can be tested or studied further

8

Page 9: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STEPS

•Topic

•Study question/s

•Framework

•Design•Population •Sampling•Data•Outcome

•Analysis

•Quality and rigor

Typical, Purposeful, Convenience, Extreme, Critical, Diverse, Snowball“Data Saturation”

9

Page 10: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STEPS

Observations in natural settingsConversations (one on one, focus groups)Narratives (reflections, open ended comments)Archival documents (mission statements, course objectives)

•Topic

•Study question/s

•Framework

•Design•Population •Sampling •Data•Outcome

•Analysis

•Quality and rigor

10

Page 11: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STEPS

•Topic

•Study question/s

•Framework

DesignPopulation SamplingDataOutcome

Analysis

Quality and rigorOpinionsReflectionsSocial interactionsCultural beliefs BarriersReasons for beliefs or behaviors

11

Page 12: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

THE FINAL STEP

Credibility (Internal validity): Triangulation, Prolonged observation, Skillful interviews

Transferability (External validity): Detailed descriptions

Dependability (Reliability): Multiple observations, Systematic sampling data collection and analysis, Respondent validation

Confirmability (Objectivity): Detailed recording of notes, Grounded theory, Include negative cases

•Topic

•Study question/s

•Framework

Design Population

Sampling

Data

Outcome

Analysis

Quality and rigor

12

Page 13: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

SUMMARY

•Qualitative methods can •Bridge the gap between scientific evidence and practice

•Provide opinions of participants in everyday context•Help us understand barriers to using evidence & guidelines and their limitations in informing health care or educational decisions

•Selection •Be clear on the research question(s), and select those participants who can best and most broadly inform the question(s)

•Recruit participants purposefully until saturation or thorough understanding of the phenomenon is achieved.

13

Page 14: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

STEPS IN ANALYSIS

• Raw text (notes, transcripts, recordings)

• Relevant text (codes)

• Repeating ideas (coding categories)

• Themes (groups of repeating ideas )

• Theoretical constructs (abstract grouping of themes by researcher)

• The narrative

14

Page 15: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

BIBLIOGRAPHY

1. Creswell, J. 2003. Research design: Qualitative, quantitative, and mixed methods approaches.

2. Creswell, J. 2007. Qualitative inquiry and research design: Choosing among five approaches (2nd ed).

3. Maxwell, J. 2005. Qualitative research design: An interactive approach.(2nd ed).

4. Miles, M. & Huberman, A. 1994. Qualitative data analysis.

5. Varpio, Artino Jr, and The Qualitative Collaborative (2015) Answering the Mail: Replying to Common Questions About Qualitative Inquiry. Journal of Graduate Medical Education: December 2015, Vol. 7, No. 4, pp. 667-668.

6. Teherani et al (2015) Choosing a Qualitative Research Approach. Journal of Graduate Medical Education: December 2015, Vol. 7, No. 4, pp. 669-670.

7. Wright et al (2016) Research Design Considerations. Journal of Graduate Medical Education: February 2016, Vol. 8, No. 1, pp. 97-98.

8. Nimmon et al. Integrating theory into qualitative medical education research. Journal of Graduate Medical Education. 8(2): 2016, in press.

9. Sullivan, Sargeant, (2011) Qualities of Qualitative Research: Part I. Journal of Graduate Medical Education: December 2011, Vol. 3, No. 4, pp. 449-452.

10.Sargeant, (2012) Qualitative Research Part II: Participants, Analysis, and Quality Assurance. Journal of Graduate Medical Education: March 2012, Vol. 4, No. 1, pp. 1-3.

11.Ramani and Mann. Introducing medical educators to qualitative study design: Twelve tips from inception to completion. Medical Teacher 2015 Apr 21:1-8. [Epub ahead of print] DOI: 10.3109/0142159X.2015.1035244.

15

Page 16: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Applying Qualitative Methodology to Expand the Scope of Research and Scholarship

May 12, 2016

Subha Ramani, Gail Sullivan, Katherine Chretien and Matthew Tuck

You are Dean Dumbledore of HMS (Hogwarts Magical School). HMS is the premier institution to educate

students with potential wizarding traits and transform them into outstanding wizards who can protect

the world from evil wizards like Lord Voldemort. Lord Voldemort and his team, the death eaters, are

highly intelligent wizards with outstanding skills in dark arts. The Dean thinks that in order to defeat the

death eaters, his young wizards need to understand the theory of the dark arts (knowledge), perform

the spells that can shield or protect the world from evil spells (skills), and believe firmly in the value of

good magic (attitudes).

With this in mind, he wants to pilot a course in school called “Defense of the dark arts”. Since this is a

brand new course, a new curriculum needs to be designed and a course director appointed. The Dean

wishes to explore what this curriculum should consist of and whom to appoint as Professor of this

course. He convenes a subcommittee to help him design a study to answer both questions and decides

that qualitative research design would be the most appropriate.

Using the worksheet provided and in small groups:

Identify an area of interest from the case

Please develop a qualitative study question for either or both of the following issues:

o Curriculum design

o Selection of course director

Please choose one of 3 approaches for this study and justify your choice:

o Ethnography

o Phenomenology

o Grounded theory

Who is your study population?

What sources of data might help answer your study questions?

What data collection methods would you use?

16

Page 17: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Applying Qualitative Methodology to Expand the Scope of Research and Scholarship

May 12, 2016

Subha Ramani, Gail Sullivan, Katherine Chretien and Matthew Tuck

Worksheet

A. Define a Research Question

What do I want to study?

_____________________________________________________________________________________

_____________________________________________________________________________________

What about this topic makes me curious?

_____________________________________________________________________________________

_____________________________________________________________________________________

What key findings about this area of study appear in the literature?

a)

b)

What is missing from the literature?

a)

b)

What about this area of inquiry may provide a good fit with qualitative methods? Why might

quantitative methods be suboptimal for this inquiry?

_____________________________________________________________________________________

_____________________________________________________________________________________

Therefore, the purpose of this study is:

_____________________________________________________________________________________

_____________________________________________________________________________________

One central research question to address this is:

Possible sub-questions:

1. ___________________________________________________________________________2. ___________________________________________________________________________3. ___________________________________________________________________________

B. Approach

Which approach (e.g. phenomenology, grounded theory, ethnography) would best address the research

question and why?

17

Page 18: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Applying Qualitative Methodology to Expand the Scope of Research and Scholarship

May 12, 2016

Subha Ramani, Gail Sullivan, Katherine Chretien and Matthew Tuck

_____________________________________________________________________________________

_____________________________________________________________________________________

C. Design and Data Collection

Who is your study population? How will you sample the population (e.g. typical, purposive, convenience,

extreme, critical, diverse, snowball)?

_____________________________________________________________________________________

_____________________________________________________________________________________

What method(s) of data collection would best help answer my question (e.g. observations,

conversations, narratives, archival documents)?

_____________________________________________________________________________________

_____________________________________________________________________________________

What study outcomes are of interest?

_____________________________________________________________________________________

_____________________________________________________________________________________

What could be done specifically within my context/limitations/institution?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

18

Page 19: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

A Digital Ethnography of Medical Students who Use Twitterfor Professional Development

Katherine C. Chretien, MD1,2, Matthew G. Tuck, MD1,2, Michael Simon, BA2, Lisa O. Singh, PhD3,and Terry Kind, MD, MPH2,4

1Washington DCVeterans AffairsMedical Center, Washington, DC, USA; 2School of MedicineandHealth Sciences, GeorgeWashington University,Washington, DC, USA; 3Georgetown University, Washington, DC, USA; 4Children’s National Health System, Washington, DC, USA.

BACKGROUND:While researchers have studied negativeprofessional consequences of medical trainee social me-dia use, little is known about howmedical students infor-mally use social media for education and career develop-ment. This knowledge may help future and current phy-sicians succeed in the digital age.OBJECTIVE: We aimed to explore how and why medicalstudents use Twitter for professional development.DESIGN: This was a digital ethnography.PARTICIPANTS:Medical student Bsuperusers^ of Twitterparticipated in the studyAPPROACH: The postings (Btweets^) of 31medical studentsuperusers were observed for 8 months (May–December2013), and structured field notes recorded. Through pur-posive sampling, individual key informant interviews wereconducted to explore Twitter use and values until thematicsaturation was reached (ten students). Three faculty keyinformant interviews were also conducted. Ego networkand subnetwork analysis of student key informants wasperformed. Qualitative analysis included inductive codingof field notes and interviews, triangulation of data, andanalytic memos in an iterative process.KEY RESULTS: Twitter served as a professional tool thatsupplemented the traditional medical school experience.Superusers approached their use of Twitter with purposeand were mindful of online professionalism as well as ofbeing good Twitter citizens. Their tweets reflected a mix ofpersonal and professional content. Student key infor-mants had a high number of followers. The subnetworkof key informants was well-connected, showing evidenceof a social network versus information network. Twitterprovided value in two major domains: access and voice.Students gained access to information, to experts, to avariety of perspectives including patient and public per-spectives, and to communities of support. They alsogained a platform for advocacy, control of their digitalfootprint, and a sense of equalization within the medicalhierarchy.CONCLUSIONS: Twitter can serve as a professional toolthat supplements traditional education. Students’ prac-tices and guiding principles can serve as best practices forother students as well as faculty.

KEY WORDS: social media; undergraduate medical education; internet;

twitter; professional development.

J Gen Intern Med 30(11):1673–80

DOI: 10.1007/s11606-015-3345-z

© Society of General Internal Medicine 2015

BACKGROUND

Social media has transformed the way we communicate as asociety. Medical students, who have grown up using thesetechnologies, are adept at using social media for personalpurposes. Ample attention has been paid to health profes-sionals’ unprofessional use of social media, including privacyviolations, conflicts of interest, and inappropriate relationshipswith patients;1–4 egregious lapses of professionalism haveeven resulted in medical school dismissal and state medicalboard sanctions.2,3 However, social media has power thatcould be harnessed for positive professional purpose. Sharingcredible health content, advocacy, career networking, andstaying up to date with one’s field can all be achieved throughuse of social media.5,6

Twitter is a social media platform with characteristics thatcan make it useful for professional networking. Its mission is,BTo give everyone the power to create and share ideas andinformation instantly, without barriers.^7 Twitter has 271 mil-lion active users monthly, and 500 million tweets are sent eachday.7 Among US online adults ages 18–29 years, 37 % useTwitter, and the majority of users have public accounts.8 Userswrite 140 character-limited messages called Btweets^ that cancontain links to webpages or other content. Users can interactwith other users by following their accounts, and replying to,or mentioning other users in their tweets, thereby creating anetwork of connections. Many medical organizations andmedical journals have Twitter accounts and share relevantinformation and news.9 Physicians on Twitter often post med-ical or health-related tweets and share information.10 Twitterhas been identified as a potential learning tool in medicaleducation.11,12 Medical education faculty can regularly partic-ipate in Bchats^ on Twitter, and others host Twitter journalclubs to discuss medical articles of interest. Hashtags helpcategorize tweets for indexing. There are currently activehashtags for medical education (#meded, #GME, #CME)and medical students (#medstudent).Medical educators are challenged with guiding medical

students along their professional development and giving

Electronic supplementary material The online version of this article(doi:10.1007/s11606-015-3345-z) contains supplementary material,which is available to authorized users.

Published online May 8, 2015

1673

19

Page 20: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

them the tools to succeed in their future careers. Little researchhas addressed how to leverage social media use for education-al and career development.1 The goal of this research was todescribe how and why medical students are using Twitter fortheir professional development.

METHODS

We conducted a digital ethnography with the research question,BWhat is the culture of medical students who use Twitter foreducation?^ Sub-questions were, BHow do they use Twitter?^,BWith whom do they interact?^, and BWhat do they value?^

An ethnography is a qualitative research framework thatseeks to provide a description and interpretation of a culturalgroup or system.13,14 This typically involves prolonged obser-vation of the group, interviews with key informants who canprovide insights into the group, and examination of culturalartifacts and documents. Digital ethnography (also called In-ternet, virtual, or online ethnography) is a research methodused to examine the culture of online groups, modifyingtraditional ethnographic approaches and theory and applyingthem to the online space.15,16

Our target group was medical student Bsuperusers^ whoused Twitter for educational or professional purposes. Wesought to describe this cultural group—their online behavior,their digital networks, and their values.

Identification of Target Group

We identified potential subjects by: 1) searching key wordsBmedical student^ and Bmed student^ on Twitter.com; 2)searching existing public lists of medical students on Twittercollated by individual users; 3) searching relevant hashtags(#medstudent, #meded); 4) soliciting referrals by current sub-jects (chain sampling); and 5) hosting a medical educationBtweetchat.^ A Btweetchat^ is a public real-time discussion onTwitter run by a moderator. For this study, two authors (KCand TK) hosted a 1 h tweetchat in April 2013 on how medicalstudents use Twitter for educational or professional purposes,using the established #meded tweetchat. This helped to iden-tify students in our target group, and also helped structure ourinterview guides.All identified students were added to a general medical

student Twitter list shared among the investigators. The num-ber of students on this list was fluid, as students were contin-ually added or removed (if not actually a student, for example),with a total of 293 at study closure. From the general medicalstudent list, students were added to a superuser list if they wereobserved to post professional content (for instance, sharing ajournal article or live-tweeting a medical conference), interactwith other medical students and faculty, and/or participate inprofessional Twitter Bchats.^ The research teammet weekly toestablish consensus on who should be added to the superuserlist. To be included, users had to be current medical students inthe US. Current students at our medical schools were

excluded. The final superuser list as of May 2013 included31 subjects.

Structured Observations

Four investigators, including two expert Twitter users (KC andTK) and two novice users (MS and MT), observed tweets ofthe 31 superusers and of the general list of medical students for8 months (May–December 2013), taking structured fieldnotes, guided by key dimensions of descriptive observations(Space, Actors, Activities, Objects, Acts, Events, Time, Goals,Feelings).17,18

Semi-Structured Interviews with Key Informants

Using purposive sampling, key informant superusers wereidentified for interviews. We used a semi-structured interviewguide designed to explore their use of Twitter, the nature oftheir interactions with other users, and what they valued fromtheir use (Appendix). Subjects were invited for interview viaemail and provided an institutional reviwe board (IRB)-ap-proved information sheet on study procedures. Their agree-ment to participate served as consent. Interviewswere digitallyaudio-recorded, and with the exception of personal identifiers,transcribed by an external service. At the conclusion of eachinterview, major themes were reviewed with the participant formember-checking. Interviews continued until saturation ofthemes was reached (n=10). For data triangulation, three fac-ulty key informants, identified through student key informantinterviews, were also interviewed. Faculty informants wereasked to describe their interactions with students on Twitter,how they felt students used Twitter for educational purposes,and student best practices. All subjects interviewed were of-fered $25 for their participation. Interviews lasted between 30and 60 min.

Network Analysis

In order to understand students’ use of Twitter, we extractedtotal number of tweets, number of followers, number follow-ed, and for a subset of tweets, analyzed content for URLs, re-tweets, and hashtags for each key informant. Further, to un-derstand and describe the virtual community in which studentsBlive^ and Binteract,^ we conducted two forms of networkanalysis: 1) an ego network (local network of an individual)analysis of key informants, and 2) a subnetwork analysis ofkey informants who were connected to each other. For the egonetwork analysis, we computed the centrality (ego behaviormetrics) degree and clustering coefficient. The degree is thenumber of connections the individual (node) has, or the size ofthe local neighborhood. Clustering coefficient (value range 0–1) is a measure of the connectivity a local neighborhood.Higher clustering coefficients indicate a well-connected neigh-borhood and potential for rapid information dissemination.Subnetwork analysis focuses on the characteristics of the

network itself. The metrics we computed were subnetworkdensity, number of triangles, and subnetwork diameter. These

1674 Chretien et al.: Students on Twitter JGIM

20

Page 21: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

metrics help describe the connections between the key infor-mants and the ability to transmit information within this group.We used R, Node XL (an Excel-based tool) and Gephi (net-work analysis and visualization software) for the networkanalysis.

Qualitative Analysis and Trustworthiness

The team met monthly to discuss observations and reviewinterview transcripts. Memos captured emerging themes andanalytic insights from comparing and contrasting the variousforms of data (triangulation). We compiled a list of codes thatemerged inductively from the data and applied these to all fieldnotes and interview transcripts. Each transcript was codedindependently by two investigators; in addition, to add con-sistency, a single investigator (KC) coded all transcripts.Codes were compared and any discrepancies were resolvedthrough discussion; subsequent modification of codes result-ed. Representative and exemplary quotes were highlighted foreach code. Following principles of inductive analysis19 andincorporating network analysis results, we constructed a de-scription of the target group.We bolstered trustworthiness in several ways: 1) triangula-

tion of data sources and methods; 2) inclusion of expert andnovice Twitter users and a medical student on the researcherteam; 3) member checking of main themes at the end of theinterviews; 4) creation of an audit trail to document teamdecisions and discussion; 5) peer-review of transcripts withfeedback given to interviewers; and 6) use of multiple inter-viewers. Qualitative data was managed with NVivo10 (Cam-bridge, MA). The Washington DC Veterans Affairs MedicalCenter Institutional Review Board approved the study.

RESULTS

Of the 31 superusers, 17 were female, 13 male, and for one,gender was not specified. Thirteen (41 %) linked to theirblogs. The majority (25, 81 %) appeared to include their realname on their Twitter prolife. They represented all regions ofthe US and all years of medical school. For the ten student keyinformants, seven were female and three were male. Theywere in their first year (2), third year (2) and fourth year (6)of medical school. They lived in large cities on the East Coast,West Coast, Midwest, and Southern regions of the US.Results are organized around how students use Twitter

(practices), their interactions (network analysis), and why theyuse it (value).

Twitter Practices of Superusers

Overall, students were active Twitter users who were oftenearly adopters of Twitter use among their medical schoolclassmates. Many also had a personal blog. They followedother medical students, medical faculty, medical journals,medical and news organizations, and other health profes-sionals, among others.

Medical student superusers used Twitter thoughtfully andwere guided by the purpose of their use. They were mindful oftheir professionalism online and were careful not to mentionspecific patients to protect patient privacy. Many mentionedthinking twice before tweeting to avoid potential misinterpre-tation of what they meant to say. In general, they avoidedventing and possible inflammatory statements and were awareof their public image. Their tweets included both professionaland personal content.

BI’m a real-life person. I’m not a made-up person. Idon’t think that that detracts from my professionalinteraction—that’s just me.^ (Key informant 8)BI try to go about my social media as if I was actuallyinteracting with a patient. So if I were to see a patient inthe office…you do share a little bit about your person-ality so that’s where I post things about myself and mypersonal life.^ (Key informant 9)

Superusers also practiced Bgood Twitter citizenship^ bycontributing to conversations, sharing helpful informationwith others, and being cognizant of posting etiquette.Students identified faculty role models on Twitter who

helped guide their own use in terms of how to share informa-tion and how to respond to challenging situations with otherusers. We did not observe any unprofessional content posted.See Box 1 for a summary of superusers’ Twitter practices.

Box 1. Twitter Practices of Medical Student Superusers

Network Analysis of Key Informants

As of July 2014, the student key informants followed a medianof 489 (range 56–6425) Twitter users. They, in turn, werefollowed by a median of 1770 (range 403–11169) users. Thekey informants posted a median 7371 tweets (range 4172–21057) since their Twitter debuts (which predated our study inall cases). Analysis of a subset of 2074 tweets (Twitter im-poses restrictions on howmuch data can be imported) revealedthat 553 contained URLs, 423 were re-tweets, and 647contained one or more hashtags (Table 1).

Know purpose for use

Mindful of online professionalismAvoid mentioning specific patientsThink twice, tweet onceAvoid ventingAvoid inflammatory statementsAware of public image

Not strictly medical content, show “personality”

Practice “good Twitter citizenship”Contribute to conversationsShare informationAware of posting etiquette

Identify faculty role models for use

1675Chretien et al.: Students on TwitterJGIM

21

Page 22: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Figure 1 shows the full followers network on the keyinformants. The ten key informants connect to 40,091 otherindividuals on Twitter. Based on the average directed degree(2) and the low clustering coefficients (0.054 directed, 0.102undirected), key informant networks have limited overlap.

They are not the center of a broader community structure,although they may participate with multiple separate overlap-ping communities. This suggests an opportunity for extensiveinformation dissemination (access to different groups to shareinformation with).Figure 2 visually highlights the connectivity between key

informants from the ego and subnetwork analysis. Overall, theinformant network is well connected (clustering coefficient of0.851, average directed degree 9). None of the males are con-nected to each other, but the majority of females follow eachother. The overall subnetwork density is 0.46, high for a Twittersubnetwork. Subnetwork analyses support that students main-tain large, diverse networks that have other key informants inthem, but otherwise interact with different subgroups of people.

Value for Medical Students

Twitter served as a professional tool, something that supple-mented their medical school education.

BI just think it’s helped me grow as a professionalbecause…there is more to who you become as a phy-sician that can enable better patient care that doesn’thappen just with textbooks.^ (Key informant 4)

Students derived value fromTwitter in twomain domains ofAccess and Voice (Fig. 3).Access. Medical student superusers gained access toinformation, to experts, to a variety of perspectives includingpatient/public perspectives, and to communities.Twitter is a way that these students accessed information.

They used it to stay up to date with general news and theirfields of interest.

Figure 1 Full followers network of the key informants. The size and color of the node correspond to the degree (number of connections) of thenode. (Average degree =2; clustering coefficient, directed=0.054; clustering coefficient, not directed=0.102; number of nodes=40,101; number of

edges=61,602).

Table 1 Hashtags Used by Medical Student Key Informants in aSubset of 2074 Tweets

Hashtag Frequencycount

Comment

#socaphealth 69 Socap Health conference; goalis to examine the roles ofentrepreneurs, investors, andfunding bodies in the largerpublic health ecosystem.

#meded 63 Medical education; weeklyTwitter chat.

#hcsm 31 Healthcare communicationsand social media; weeklyTwitter chat.

#fmrevolution 26 Family medicine revolution;Re-branding campaign startedby family med residents topromote family medicinecareers.

#ase2013 25 Association for ScienceEducation 2013 conference

#sdoh 23 Social determinants of health#fitstats 20 Fitness statistics derived from

popular fitness app#disruptinginequality 16 Disrupting inequality;

advocacy for social change#medschool 16 Medical school#medstudent 15 Medical student#residency 11 Residency#usavsbel 11 USA versus Belgium World

Cup game#commdev 10 Community development#obgyn 10 Obstetrics/Gynecology

*Tweets occurred between January 2013 and July 2014. No hashtagswere excluded

1676 Chretien et al.: Students on Twitter JGIM

22

Page 23: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

BIt helps keep me up to date on what’s going on in theworld, in the world of medicine in this country, withhealthcare reform, and everything.^ (Key informant 3)BI definitely know about various articles that tend to beof importance in my field before other people.^ (Keyinformant 7)

Students on Twitter find value in the ability to accesscontent experts in medicine with whom they otherwisewould not be able to connect. One key informant asked aquestion on Twitter during a lecture and had an interna-tional expert respond with the answer within 2 min—beforethe class instructor could look up the answer. Other keyinformants described having access to medical contentexperts during Twitter chats or following experts whotweeted clinical pearls.Twitter provided some students with opportunities for pro-

fessional collaborations, allowing them to connect with othermedical students with similar interests.Access to individuals with different perspectives was of

particular value for students, including others’ opinions abouthealth policy, medical education, and the medical professionin general. For many key informants, being able to gain thepatient perspective was invaluable to their professional devel-opment.

B…looking at their feeds and hearing them talk abouttheir experiences has really changed the way that I seethe personal side of patients– how the medical system

has worked for them, and that’s really not somethingthat we get [in training].^ (Key informant 1)B[I] ask them a lot of questions about how they’ve beentreated by their doctors…It’s kind of like an opportu-nity to just interview a lot of patients without having towait until third year.^ (Key informant 6)

Beyond the patient perspective, Twitter also allowed them abroader context by which to understand medicine, outside ofthe patient–physician dynamic.Through Twitter, students gained access to communities of

learning and support. One key informant shared, BThe mostvaluable thing for me is a sense of community, because as anolder medical student there’s just not a lot of community.^Students described having a community of medical students

that congratulate each other when milestones are reached, helpeach other study for exams and experience medical schooltogether.

BWe all kind of support each other and…are there tocheer each other on as we go through medical school.^(Key informant 10)BWe share information all the time. We share articles,we share study techniques.^ (Key informant 8)

Students interacted with faculty on Twitter who providedadvice, encouragement and virtual mentorships.

BWe had basically a public discussion where I justasked her questions about what is the right way to talk

Figure 2 Key Informant Network. The size of the node corresponds to the number of key informant followers the node has. A large nodeindicates that many key informants follow that node. The color of each node maps to the clustering coefficient. A red node indicates a lowclustering coefficient a green node indicates a mid-range clustering coefficient, and a blue node indicates a high clustering coefficient. It is notsurprising that nodes with a higher degree have a lower clustering coefficient. Finally, a white dot in the middle of a node indicates that the keyinformant represented by that node is male. (Average degree=9; clustering coefficient=0.851; number of triangles, no direction=42; number of

triangles, with direction=69; network diameter=3).

1677Chretien et al.: Students on TwitterJGIM

23

Page 24: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

to patients about this, and what is the right word choicethat won’t be offensive?^ (Key informant 6)B…phenomenal advice from people who are very ac-complished…I got to shadow one of them.^ (Keyinformant 5)B…being able to follow older physicians [is] the mostvaluable, just getting a glimpse into what my life couldbe like when I’m a doctor.^ (Key informant 3)

Students also connected with communities of like-minded others, such as communities centered on personalinterests and hobbies or those who carried similar views.For some, these communities served as sources ofinspiration.For instance, one student commented on the impact of a

Twitter community on her decision to pursue a primary carespecialty.

^ …coming from my institution… the attendingsweren’t always the most supportive in trying to

persuade people to go into primary care so having thatonline group of people that was really supportive andgiving advice to each other and patting each other onthe back— I think that was super inspiring.^ (Keyinformant 9)

These communities of like-minded others have helped stu-dents maintain their ideals.

BI want to make a difference …Twitter gave me acommunity where I could like not like feel alone indoing that. How Twitter has influenced me is that itreally connectedme to other people that believe that wecan make things better. B(Key informant 8)

Voice. Twitter also provided students value by giving them avoice in the domains of advocacy and an enduring digitalfootprint, and was an equalizer of power.Some students used Twitter as a platform to support polit-

ical issues and to share their health policy opinions. It helped

Figure 3 Value for medical students.

1678 Chretien et al.: Students on Twitter JGIM

24

Page 25: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

them develop their own voice and engage with others outsideof their comfort zones.

BI think it’s important for everyone to have a voice andhonestly I think a lot of times medical students reallyneed to have the loudest voice of all, because they’rethe people in the transition zone between being apatient and a receiver of healthcare to being ahealthcare provider.^ (Key informant 8)

Twitter allowed medical students a way to craft their digitalfootprint in a positive way.

BI want them to search for me and find a consistentpresence that is positive.^ (Key informant 7)

Finally, Twitter served as an equalizer and a Bleveling forcein terms of power hierarchy^ (Key informant 5).

BA lot of times I don’t even know if… the person thatI’ve talked to is faculty or student or resident… it’soften just a conversation… the great equalizer.^ (Keyinformant 7)BIt kind of disrupts who gets a voice, like who gets totalk about what.^ (Key informant 4)

DISCUSSION

For this group of students, Twitter served as a professionaltool, giving them access and voice. It provided added valuefrom the traditional medical school curricula; they gainedsupport, networking opportunities, mentorship, and learning.These students do not represent the majority of medical stu-dents on Twitter, but a subculture who were often pioneers intheir use of Twitter for professional purpose.We were particularly moved by students’ interactions with

patients. Note that these were not patients the students hadcared for clinically, but people on Twitter who self-identifiedas patients with certain disease conditions. Students seemed togreatly value hearing patient perspectives, gaining greatercultural sensitivity and understanding of what it is like forpatients living with their conditions. Social media, with itsmany virtual patient communities, could be a resource formedical students to better understand the patient experience.The posting practices of these superusers align with guiding

principles of online professionalism derived from nationalconsensus statements and guidelines,20,21 research on stake-holders’ perspectives of what is appropriate and what is inap-propriate to post,22–25 and expert opinion.6,26 These includesafeguarding patient privacy, Bpause before posting,^ avoidinginflammatory material, and carefully considering material thatis to be posted for public consumption. The one area wheresuperusers deviated from published guidance is the advice tomaintain separate personas or accounts for personal and

professional purposes. Superusers included parts of their per-sonal lives on their accounts and felt this added to theirauthenticity. Of note, others have disagreed with recommen-dations to separate professional and personal online identities,arguing that this is simply not possible on the Internet, isinconsistent with the concept of professional identity, andcould be potentially harmful.27

In comparison to the average Twitter user, the key infor-mants in this study had more followers. The majority ofTwitter users have fewer than 50 followers whereas the medi-an number of followers for student key informants was1770.28 In Twitter, the amount of community structure iscorrelated with the number followers a person has.30 Networkanalysis metrics of the key informant subnetwork reflect asocial network community as opposed to an information net-work community.29 In other words, the students we studiedtend to have social, reciprocal relationships on Twitter more sothan simple information retrieval, and this may account forsome of its added value.In a systematic review of social media use in medical

education that included 14 studies incorporating social mediain formal curricula, Cheston et al. cited opportunities of pro-moting learner engagement, feedback, collaboration and pro-fessional development.30 Some medical educators have advo-cated for the use of Twitter, specifically, as a professionaltool.11,12 In their pilot studies, George and Dellasega notedthat Twitter, as one of several social media platforms incorpo-rated into two humanities courses for fourth-year students,augmented learning and collaboration and allowed real-timecommunication between learners and instructors outside theclassroom.31 Little research exists about how medical traineesor physicians actually use social media for informal learningand professional development. The present study thus fills agap in the literature.Connectivism is a learning theory that supports the use of

Twitter for learning.32 Connectivism relates learning as aprocess of connecting information sources; nurturing theseconnections is needed for continual learning. A diversity ofperspectives and opinions fuel learning and knowledge, andthe capacity to increase one’s knowledge is more importantthan what one currently knows.32 Managing and filtering theinformation on Twitter and other social media platforms inorder to obtain the most current information in students’ fieldsof interest may be a key competency for today’s learners that isnot regularly addressed in undergraduate medical curricula.33

Future research could explore how students critically appraiseinformation on Twitter or other social media sites, as well asthe quality of students’ learning.The ability and desire to use Twitter as a professional tool

may be specific to certain students. In one study of osteopathicmedical students, students with learning styles self-assessed asactive, global, intuitive and/or visual were more likely toaccess online educational materials than those whose learningstyles were reflective, sensing, sequential, and/or verbal.34 Inaddition, individual student preferences and characteristics

1679Chretien et al.: Students on TwitterJGIM

25

Page 26: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

may dictate whether students choose to use Twitter for profes-sional development. Many in our sample also had a blog; thiscould indicate particular comfort in sharing ideas publicly?This study has potential limitations. Digital ethnography is

a newer research method; diverse approaches have been usedto study the Internet with an ethnographic perspective.16 Keyinformants may not have disclosed fully in interviews if con-cerned about anonymity, though efforts were made to reducethis possibility. Expert Twitter user faculty investigators mayhave carried pre-existing assumptions about student use; totest these assumptions, the team also included novice usersand a medical student. Also, we limited network analysis tokey informants (instead of the larger group of superusers)given we had only the informants’ expressed consent to beincluded in our study.In conclusion, medical students who regularly used Twitter

as a professional tool were doing so with thoughtfulness andpurpose. Twitter allowed them access and voice that supple-mented their medical school experience. Their Twitter prac-tices can serve as best practices for other students as well asfaculty, in their career lifelong learning and professionaldevelopment.

Acknowledgements: This work was funded by the CDIM-iInTimeSmall Grants Program and presented in poster form at AcademicInternal Medicine Week on 12 September 2014 in Washington, DC.

Conflicts of Interest: The authors declare that they do not have aconflict of interest.

Corresponding Author: Katherine C. Chretien, MD; Washington DCVeterans Affairs Medical Center, 50 Irving Street, NW, Washington, DC20422, USA (e-mail: [email protected]).

REFERENCES1. Fenwick T. Social media and medical professionalism: rethinking the

debate and the way forward. Acad Med J Assoc Am Med Coll.2014;89(10):1331–1334.

2. Greysen SR, Chretien KC, Kind T, Young A, Gross CP. Physicianviolations of online professionalism and disciplinary actions: anational survey of state medical boards. JAMA. 2012;307(11):1141–1142.

3. Chretien KC, Greysen SR, Chretien J-P, Kind T. Online posting ofunprofessional content by medical students. JAMA. 2009;302(12):1309–1315.

4. Chretien KC, Tuck MG. Online professionalism: A synthetic review. Int RevPsychiatry Abingdon Engl. 2015:1–12.

5. Greysen SR, Kind T, Chretien KC. Online professionalism and the mirrorof social media. J Gen Intern Med. 2010;25(11):1227–1229.

6. Chretien KC, Kind T. Social media and clinical care: ethical, professional,and social implications. Circulation. 2013;127(13):1413–1421.

7. Twitter, Inc. About Twitter. 2015. https://about.twitter.com/company.Accessed March 23, 2015.

8. Duggan M, Ellison NB, Lampe C, Lenhart A, Madden M. Demographicsof Key Social Networking Platforms. Pew Res Cent Internet Sci Tech. 2015.

http://www.pewinternet.org/2015/01/09/demographics-of-key-social-networking-platforms-2/. Accessed March 24, 2015.

9. Mishori R, Singh LO, Levy B, Newport C. Mapping physician twitternetworks: describing how they work as a first step in understandingconnectivity, information flow, and message diffusion. J Med Internet Res.2014;16(4), e107.

10. Chretien KC, Azar J, Kind T. Physicians on twitter. JAMA.2011;305(6):566–568.

11. Forgie SE, Duff JP, Ross S. Twelve tips for using twitter as a learning toolin medical education. Med Teach. 2013;35(1):8–14.

12. Kind T, Patel PD, Lie D, Chretien KC. Twelve tips for using social mediaas a medical educator. Med Teach. 2014;36(4):284–290.

13. Cresswell J. Qualitative inquiry and research design: choosing among fivetraditions. Thousand Oaks: Sage Publishing; 1998.

14. Reeves S, Peller J, Goldman J, Kitto S. Ethnography in qualitativeeducational research: AMEE Guide No. 80. Med Teach. 2013;35(8):e1365–e1379.

15. Murthy D. Digital ethnography: an examination of the use of newtechnologies for social research. Sociology. 2008;42:837.

16. Dominguez D, Beaulieu A, Estalella A, Gomez E, Schnettler B, Read R.Virtual Ethnography. Forum Qual Soc Res. 2007;8(3).

17. Spradley J. Participant observation. New York: Holt, Rinehart & Winston;1980.

18. Robson C. Real world research. Malden: Blackwell; 2002.19. Miles M, Huberman A. Qualitative data analysis. 2nd ed. Thousand Oaks:

Sage; 1994.20. Farnan JM, Snyder Sulmasy L, Worster BK, et al. Online medical

professionalism: patient and public relationships: policy statement fromthe american college of physicians and the federation of state medicalboards. Ann Intern Med. 2013;158(8):620–627.

21. American Medical Association. Opinion 9.124 - Professionalism in the Useof Social Media. AMA Code Med Ethics. 2011. http://www.ama-assn.org/ama/pub/physician-resources/medical-ethics/code-medical-ethics/opin-ion9124.page. Accessed March 24, 2015.

22. Chretien KC, Farnan JM, Greysen SR, Kind T. To friend or not to friend?social networking and faculty perceptions of online professionalism. AcadMed J Assoc Am Med Coll. 2011;86(12):1545–1550.

23. Kind T, Greysen SR, Chretien KC. Pediatric clerkship directors’ socialnetworking use and perceptions of online professionalism. Acad Pediatr.2012;12(2):142–148.

24. Jain A, Petty EM, Jaber RM, et al. What is appropriate to post on socialmedia? Ratings from students, faculty members and the public. Med Educ.2014;48(2):157–169.

25. Greysen SR, Johnson D, Kind T, et al. Online professionalism investiga-tions by state medical boards: first, do no harm. Ann Intern Med.2013;158(2):124–130.

26. Mostaghimi A, Crotty BH. Professionalism in the digital age. Ann InternMed. 2011;154(8):560–562.

27. DeCampM, Koenig TW, ChisolmMS. Social media and physicians’ onlineidentity crisis. JAMA. 2013;310(6):581–582.

28. Beevolve Inc. An Exhaustive Study of Twitter Users Across the World. 2012.http://www.beevolve.com/twitter-statistics/. Accessed March 24, 2015.

29. Myers S, Sharma A, Gupta P, Lin J. Information network or socialnetwork? The structure of the Twitter follow graph. 2014;493–498.

30. Cheston CC, Flickinger TE, Chisolm MS. Social media use in medicaleducation: a systematic review. Acad Med J Assoc Am Med Coll.2013;88(6):893–901.

31. George DR, Dellasega C. Use of social media in graduate-level medicalhumanities education: two pilot studies from penn state college ofmedicine. Med Teach. 2011;33(8):e429–e434.

32. Siemens G. Connectivism: a learning theory for the digital age. J InstrTechnol Distance Learn. 2005;2(1):3–10.

33. Sandars J. Developing competences for learning in the age of the internet.Educ Prim Care Off Publ Assoc Course Organ Natl Assoc GP Tutors WorldOrgan Fam Dr. 2009;20(5):340–342.

34. Halbert C, Kriebel R, Cuzzolino R, Coughlin P, Fresa-Dillon K. Self-assessed learning style correlates to use of supplemental learning materialsin an online course management system. Med Teach. 2011;33(4):331–333.

1680 Chretien et al.: Students on Twitter JGIM

26

Page 27: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Moral Distress Amongst American Physician TraineesRegarding Futile Treatments at the End of Life:A Qualitative Study

Elizabeth Dzeng, MD, MPH, MPhil, MS1,2,3,4, Alessandra Colaianni, M.D., M.Phil.1,Martin Roland, B.M., B.Ch., D.M.3, David Levine, M.D., M.H.S., Sc.D.1, Michael P. Kelly, Ph.D.3,Stephen Barclay, B.M., B.Ch., M.D.3, and Thomas J. Smith, M.D.2,5

1Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, USA; 2Program in Palliative Care, Johns Hopkins School ofMedicine, Baltimore, USA; 3Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK;4Department of Hospital Medicine, University of California San Francisco, San Francisco, CA, USA; 5Department ofOncology, Johns Hopkins Schoolof Medicine, Baltimore, MD, USA.

BACKGROUND: Ethical challenges are common in end oflife care; the uncertainty of prognosis and the ethicallypermissible boundaries of treatment create confusionand conflict about the balance between benefits and bur-dens experienced by patients.OBJECTIVE: We asked physician trainees in internalmedicine how they reacted and responded to ethical chal-lenges arising in the context of perceived futile treatmentsat the end of life and how these challenges contribute tomoral distress.DESIGN: Semi-structured in-depth qualitativeinterviews.PARTICIPANTS: Twenty-two internal medicine residentsand fellows across three American academic medicalcenters.APPROACH: This study uses systematic qualitativemethods of data gathering, analysis and interpretation.KEY RESULTS: Physician trainees experienced signifi-cant moral distress when they felt obligated to providetreatments at or near the end of life that they believed tobe futile. Some trainees developed detached anddehumanizing attitudes towards patients as a copingmechanism, which may contribute to a loss of empathy.Successful coping strategies included formal and infor-mal conversations with colleagues and superiors aboutthe emotional and ethical challenges of providing care atthe end of life.CONCLUSIONS: Moral distress amongst physiciantrainees may occur when they feel obligated to providetreatments at the end of life that they believe to be futileor harmful.

J Gen Intern Med 31(1):93–9

DOI: 10.1007/s11606-015-3505-1

© Society of General Internal Medicine 2015

BACKGROUND

Ethical challenges are common in end of life care; the uncer-tainty of prognosis and the ethically permissible boundaries oftreatment create confusion and conflict about balance of ben-efits and burdens experienced by patients. Embedded in end-of-life care are ethical dilemmas that are punctuated by con-flicts between differing ethical obligations such as respecting apatient’s autonomy and the duty to do no harm.Case studies in the ethical literature have described and

hypothesized that physicians experience moral angst regardingoverly aggressive or Bfutile^ care.1,2 The definition of futility iscontroversial, and no one definition is universally accepted. Onedescribes futility as an Beffort to provide a benefit to a patientthat is highly likely to fail and whose rare exceptions cannot besystematically produced.3^ In one study, nearly 70% of housestaff reported acting against their conscience in the care theyprovided at the end of life, with four times as many respondentsconcerned about over-treatment than under-treatment.4 Surveyshave shown that clinicians sometimes perceive care in theintensive care unit (ICU) as inappropriate or futile, resulting inhigh costs and resource utilization.5,6

Moral distress occurs when individuals believe they areunable to act in accordance with their ethical beliefs due tohierarchical or institutional constraints.7 Mobley et al. hypoth-esized that the intensity and frequency of moral distress in-creased with exposure time to futile care, resulting in burnoutand emotional exhaustion.8 This can have significant negativeeffects on job satisfaction, psychological and physical well-being, and self-image, with consequent burnout and thoughtsof quitting.6,9–11

The vast majority of the literature on moral distress focuseson the nursing profession.12,13 Several qualitative studies onnurses have demonstrated that moral distress is associated withprovision of treatments perceived to be overly aggressive andnon-beneficial to patients.10,14,15 While there have been theo-retical discussions on moral distress experienced by physi-cians, there have only been a small number of empiricalstudies demonstrating moral distress in U.S. physicians, themajority of which have focused on physicians as members of

Electronic supplementary material The online version of this article(doi:10.1007/s11606-015-3505-1) contains supplementary material,which is available to authorized users.

Received December 6, 2014Revised May 1, 2015Accepted August 14, 2015Published online September 21, 2015

93

27

Page 28: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

larger interdisciplinary teams.12,16,17 To our knowledge, thereare no studies that describe the physician or physician traineeexperience surrounding moral distress associated with end oflife care in the United States.This report is part of a larger study investigating physician

and trainee views on resuscitation orders.18 During the courseof these research interviews, moral distress emerged as a majortheme amongst trainees. In this paper, we examine how med-ical physician trainees perceive and respond to ethical chal-lenges arising in the context of treatments at the end of life thatthey perceive to be futile and how these challenges maycontribute to moral distress.

METHODS

Design

Semi-structured in-depth interviews were conducted to inves-tigate physician trainees’ experiences and attitudes regardingmedical practices and treatments at the end of life.

Hospital Sample

We purposively sampled three academic medical centers withaccredited internal medicine residency and fellowship pro-grams in medium to large cities in the United States. Thesehospitals were chosen based on variations among them in endof life care.19 All three hospitals had palliative care and ethicsconsultation services.

Physician Sample

Participants were sampled by stage of training to provide awide range of perspectives. Physicians were excluded if theyhad not attended medical school and residency in the UnitedStates. Our sampling strategy was opportunistic and non-probabilistic. Recruitment occurred through e-mail solicita-tions to residency list serves, announcements before housestaff conferences, individual solicitations of physiciantrainees, and referrals from respondents of colleagues whomight be interested.Interviews were conducted in person, with the exception of

two interviews that were conducted via Skype. An interviewguide used across all sites provided thematic continuity (seeappendix, available online). However, the interview formatwas open-ended, encouraging participants to explore issuesthey considered most relevant. Interviews lasted between 45and 120 min and were audiotaped and transcribed verbatim.Data collection concluded when we reached theoretical satu-ration, a point where no new themes arose from theinterviews.20

Analysis

This qualitative study was exploratory in nature,intended to deepen conceptual understanding of

underlying phenomena that drive physician attitudesand behavior. Themes and patterns emerged from initialinterviews and analysis, and were refined and validatedin subsequent interviews through questions added to theinterview guide and probing of key themes during theinterviews.Our qualitative approach was grounded in a framework that

acknowledges that multiple perspectives are intrinsic to theresearch process and the importance of the perspectives thatthe researchers bring to the fieldwork and analysis.21–23

Throughout the analyses, we drew upon our own clinicalexperiences in a reflexive manner, understanding how it wouldboth inform and potentially bias our interpretation of theinterview data. Data were analyzed and theories developedas more interviews were conducted and coded. Hypothesesand themes developed became the subject of questions insubsequent interviews to further confirm the trustworthinessof the data. Disconfirming cases were recognized and ana-lyzed in light of their effect on the emerging theory.Two independent readers (ED, AC) identified initial

key themes and concepts that occurred through a subsetof the interviews using an editing analysis style, anddeveloped a codebook through an iterative process.They subsequently coded 20% of the interviews withrare disagreement, meeting to discuss emerging themesand patterns. One researcher (ED) then analyzed andcoded the remaining interviews using the codebook,adding additional themes and adapting categories asneeded. Data were coded by hand and managed inExcel. Member checking was conducted through regulardiscussions with individual physicians and in-groupmeetings where findings were discussed.Informed consent was obtained from all interviewees and

interview data were anonymized during transcription. Thestudy was approved by the Johns Hopkins University Institu-tional Review Board.

RESULTS

Over a 9-month period, one investigator (ED) recruited andinterviewed 22 internal medicine residents and medicine sub-specialty fellows or physicians with less than 6 years ofexperience (Table 1). Because similar themes and patternsemerged in trainee responses across all sites, we based our

Table 1 Demographic Characteristics of Study Participants (n=22)

Hospital A(n=7)

Hospital B(n=7)

Hospital C(n=8)

Years of experience,range (mean)

1–6 (3.57) 1–5 (3.17) 2–6 (4.37)

Male/female 3/4 2/4 3/5Level of trainingFellow, n (%) 3 (43) 3 (43) 4 (50)Resident, n (%) 4 (57) 4 (57) 4 (50)

94 Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile Treatments JGIM

28

Page 29: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

assessment of theoretical saturation on the aggregate inter-views among all three sites. Themes that emerged relating tomoral distress included language of torture and suffering,practitioner suffering, powerlessness, hierarchy, and dehu-manization. Physician trainees at one of the three hospitalsalso discussed institutionally organized coping mechanismssuch as conversations about patients who died.

TREATMENTS PERCEIVED TO BE FUTILE

One hospital had a futility policy in place (Hospital B), anotherhad a futility clause incorporated into their DNR policy (HospitalC), and the last did not have a futility policy at all (Hospital A).Trainees were generally not aware of futility policies or DNRpolicies, and rather appeared more aware of the culture of theirinstitution regarding these policies than the policies themselves.This study focused on trainee physicians’ attitudes towards

treatments that they perceived to be futile. Respondents wereasked to describe relevant cases. While we were primarilyinterested in their perceptions and reactions to clinical situa-tions that they personally felt were futile and distressing, themajority of these cases appeared to fulfill the standard defini-tions of futility described earlier in the paper. One example islisted here; others can be found in the online supplement:

This person with advanced dementia had been in andout of the ICU multiple times that month at baseline,and had very poor cognitive functioning. She had noquality of life. She was septic. I forget how many othercomorbidities on board. Just kind of a remote familymember was making the decisions, and had spent aweek in the ICU remaining full code despiteeverybody’s efforts, and ultimately coded again anddidn’t survive. But I think that’s a pretty commonscenario, especially in the ICUs and everything (Hos-pital B, PGY-2).

BTORTURE^ AND BSUFFERING^

Trainees sometimes felt obligated to provide end of life carethat was not in the patient’s best interest. They frequently usedthe words Btorture^ and Bsuffering^ to describe this treatment:

It felt horrible, like I was torturing him. He was tellingus wewere torturing him. I did not think wewere doingthe right things (A, PGY-3).

We spend a lot of time at the end of life in the ICUtorturing our patients and so, I can’t in good consciencesay that our current system really seems to serve the bestinterests of the patient because, we torture them beforethey die, even though we know that they are going to die(B, PGY-4).

A common source of moral angst among respondents ap-peared to relate to situations where surrogate decisions ap-peared to go against prior patient wishes. Physiciansquestioned whether the families made decisions in the pa-tient’s best interests. For example, one said:

It is infuriating when the family is not there and theycannot see. I feel like it’s morally wrong. When peoplesee their family members suffering and they are theresuffering with them, I am more understanding of theirdecision…I agree with giving the patients choice, butoftentimes it’s the family member. If the patient says,BTorture me, I want everything done.^ Fine. The fam-ily member is doing it for other reasons. Like guilt;they can’t let go (A, PGY-3).

PRACTITIONER SUFFERING

Many of the trainees expressed practitioner suffering andemotional angst over treatments they or their colleagues pro-vided at the end of life:

At this point, the staff felt so much moral distresscaring for this person. They just feel like they’ve beenprolonging suffering as opposed to providing care (C,PGY-6).

I thought maybe we should involve ethics because thehouse staff team was very, very demoralized by thisgentleman’s care (B, PGY-4).

Trainees felt particularly distressed providing what theybelieved to be overly aggressive treatments such as resuscita-tion that was unlikely to work:

A lot of traumatic things happen when you’re a resi-dent. There was this tiny 90-year-old lady. We had tocode her and it was one of the worst experiences of mylife…I had a lot of moral distress when I kept codingher for an hour (B, PGY-6).

PERCEIVED POWERLESSNESS

One theme that frequently emerged from these interviews wasa perceived powerlessness over physicians’ ability to preventharmful and futile treatments:

You know there’s no good outcome. You just continue tocode them and at some point they’re going to die. You’vewasted time and resources and you’ve just provided futilecare and tortured somebody for however much moretime. Then there’s the whole disassociation where youwant what’s best, but what you can do? And what do youhave ability to affect? You just do your job (B, PGY-3).

95Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile TreatmentsJGIM

29

Page 30: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

HIERARCHY

Physician trainees attributed some of their powerlessnessto a clear hierarchy in academic institutions. Traineesfelt unable to question the decisions of their attendingeven when those decisions seemed contrary to what thetrainee believed was right. The trend was often towardsmore aggressive care:

I was taken aback. I had multiple patients where thepatient and families were on board with comfort care.They had the goal of decreasing suffering and pain, butthe attending was not on board with comfort care andDNR/DNI. That can be very difficult as a resident (A,PGY-2).

Another reflected on the overall hierarchy, with lessmoral distress the more removed one was from patientcare:

It’s very significant moral distress. There are def-initely patients that disturb the nursing staff be-cause they are the ones who have to carry out thedoctor’s orders and who are at bedside seeing theeffects of our treatment—seeing patients suffer.That translates to the interns who are seeing thepatients suffer, then the residents, fellows, andsometimes even attendings. So it goes up thechain, but I feel that each step is slightly furtherremoved from the patient so they’re seeing less(C, PGY-6).

DEHUMANIZATION AND RATIONALIZATION

This resident appeared to employ a process of dehumanizationto detach himself as a coping mechanism:

We’re abusing a body, and I get that, but as long as Iremember I’m only abusing a body and not a person,it’s okay. Frequently when it’s an inappropriate code,that’s what’s happening (C, PGY-3).

One fellow (A, PGY-4) remarked that she had becomeBnumb to it^ and that to not reflect upon these ethical di-lemmas was the only way to make it through training. Anotherphysician remarked:

We do a lot of terrible things to critically ill patients andat the end of life. It’s routine care, and I feel prettynumb to having done those things…it seems like thereis no benefit and only risk. Yet I am accepting thepatient to have these procedures done to them. I’m inthat situation all the time. I’m pretty powerless to doanything about it (B, PGY-4).

This physician described the self-interest aspect of medicallearning as a justification for futile resuscitation for the sake ofmedical education:

We are torturing this poor gentleman; that is really allwe are doing. I do vaguely feel uncomfortable aboutthe general gestalt of what we do in the ICU to peopleat the end of life. I feel morally sick to my stomachabout it of course. Some of what we do is awful, butsome of those things have also given me theskills to resuscitate [others]. I don’t mean tojustify the torture that we put our elderly critical-ly ill and dying through, but it did provide mewith many learning opportunities to help peoplewho then could be saved (B, PGY-4).

Another resident worried that his cynicism would affect hisbehavior and attitudes towards patient care:

I have grown increasingly cynical about what medicinehas the capacity to do. That has shaped how I conversewith patients. I think cynically through residency Istarted to wish this personwould be DNR/DNI becausethey are totally unfixable. The danger is that you get abit sloppy and you’re looking for DNR as a wayto off-burden your work and labor and not bemeticulous (B-PGY-4).

SUCCESSFUL COPING STRATEGIES AGAINST MORALDISTRESS

The most common coping strategies described involvedformal and informal open forum discussions. One of thethree hospitals had a culture that actively promoted suchconversation:

When I was a med student, a patient I was takingcare of died. I didn’t find out for two daysbecause I had left. I felt hurt by that. I havenoticed that whenever a patient dies here, whoev-er is taking care of them is notified, whether it’sby a quick text message or whatever. The firsttime an intern has a patient who dies, I talk aboutit with them before, how to approach the familyand talk about with them afterwards about howthey felt it went to the family. I feel like there isa lot of space for emotions here (C, PGY-3).

The culture in this hospital seemed to be influenced in part bya palliative care-friendly environment, especially a programcalled Bdeath rounds.^ This weekly session facilitated opendiscussion and normalization of emotional issues, providingtime and space for reflection within a busy resident schedule:

96 Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile Treatments JGIM

30

Page 31: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

We have death rounds once a week and talk about ouremotions around making these decisions. It gives ustime to slow down and everybody can say their storyabout a patient that touched them, or a concern thatthey had, or that made them feel a little uncomfortable.It usually ends with people crying. The program, thestaff, and the residents tend to talk about emotions a lot.I think death rounds helps facilitate that (C, PGY-3).

CONCLUSIONS

Our study sheds light on a significant cause of moral distressamongst physician trainees when they feel obligated to pro-vide treatments at the end of life that they believe to be futile orharmful. Their words—Btorture,^ Bgruesome,^ Babuse,^Bmutilate,^ and Bcruel^—evoke images more befitting penalregimes than hospitals. The moral toll exacted upon thesephysicians is evident in descriptions such as feeling Bviolated^and Btraumatized.^These findings are consistent with a study by Solomon et al.

suggesting that trainees Bacted against their conscience inproviding care to the terminally ill.^4 The vast majority ofrespondents in the present study appeared more concernedabout providing overly burdensome treatment than under-treatment. Their attitudes may reflect inexperienced doctors'feelings of being trapped by expectations and policies thatprioritize patient autonomy,24 as they have yet to develop theexperience or confidence to cope with these ethical conflicts.Among physicians, trainees are particularly vulnerable to

moral distress because they are subordinate but on the frontline,2 which reinforces a perceived powerlessness to act asindependent moral agents against treatments that they believemay domore harm than good. This study provides evidence thatsimilar to nurses, trainees may experience moral distress fromperceived futile care,8 thus highlighting the need for greaternuance and attention to differences between trainees and attend-ing physicians. One study noted that decreased autonomy wasassociated with increased frequency and intensity of moral dis-tress amongst nurses.25 Because the hierarchy of control overdecision-making descends from attending to resident to nurse,the consequent degree of moral distress experienced by traineesmay be more similar to nurses than attending physicians.25

In light of this perceived helplessness, physician traineescan become emotionally detached and cynical, and may de-humanize their patients in order to protect themselves. Priorreports have highlighted the negative effects of cynicism andburnout on empathy, patient care, and the culture of medi-cine.26,27 Cynicism can alienate young physicians from theirprofession, as they begin to wonder whether their efforts aremeaningless or harmful.Trainee distress may be a root cause of a decline in empa-

thy,26,28–34 and can be related to experiences such as perceivedethical and professional dilemmas and exposure to death and

human suffering.26,31 Contradictions between the ethics taughtin medical school and practices on the wards may contribute tothe ethical erosion that can occur during medical training,35

which results from an inability to address the moral distressand ethically unjustified treatments they are asked to pro-vide.36 Ethical erosion and empathy decline may reflect mech-anisms of self-preservation through detachment anddehumanization.37

The dispensation to inflict pain is a necessary professionalduty, but it also engenders moral vulnerabilities.38 Steppingover the fine line between inflicting necessary and unnecessarypain, for example during futile treatments, may contribute to thedecline in empathy during medical training, as demonstrated byprevious U. S. studies.31,32,39,40 These experiences have signif-icant impacts on physicians’ professional identity and moralpersonhood during their most formative years.37,41,42

The limitations of this study include social desirability biasas well as a concern among respondents that their answersmight affect their evaluations or be relayed to their superiors.However, participants were assured that interviews would beconfidential and that the interviewer had no affiliation withtheir residency program. Another limitation is the exclusion ofcommunity hospital-based residency programs.Interventions that remind physicians of the humanity of

their patients and reconnect them to their own humanity andprofessional purpose can help mitigate moral distress andcounteract loss of empathy. Programs such as Schwartz CenterRounds and death rounds (at Hospital C) serve as importantcoping strategies for dealing with these difficult issues.43,44

Providing a safe space where emotions and compassion areencouraged helps to counteract the culture of stoicism inmedicine. In the harried life of a resident, encouraging oppor-tunities to stand back and reflect, even as simple as a textmessage or a short time-out, gives permission to acknowledgethe inherently challenging emotional and humanistic aspectsof patient care. These conversations also promote the impor-tance of physician self-care, a prerequisite to one’s ability tocare for others,11,45 which may in turn help foster the empathyneeded to remain a compassionate physician. A crucial com-ponent of fostering open dialogue and awareness of issuessurrounding death and dying is establishing palliative care-friendly environments through palliative care and ethics ser-vices and consultation.Strategies for addressing moral distress explored in the

literature thus far include re-calibration of emotional re-sponses, resilience and mindfulness training, and promotionof inquiry and reflection.46–49 Medical education should con-tinue to recognize the importance of addressing these issuesthrough focus groups, didactic sessions, and awareness of thetraining environment and culture. Root cause analyses andother systematic methods to understand structural and organi-zational factors can also help in recognizing and addressingsources of moral distress.50 An example of a policy-orientedintervention includes a statement issued by the American

97Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile TreatmentsJGIM

31

Page 32: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Thoracic Society that addresses situations to which a clinicianmight morally object and seeks to establish institutional normsthat allow for practitioners to personally excuse themselvesfrom morally problematic situations.51

These strategies, likely underutilized at many institutions,could be used to mitigate moral distress and loss of empathyduring medical residency and fellowship training. This studydescribes experiences of moral distress amongst physiciantrainees in response to treatments administered at the end oflife that they perceive to be futile. It provides insight into thescope and character of moral distress amongst physicians, andphysician trainees in particular. These findings underscore theneed to enhance support and training for physician trainees inthe context of end of life care in order to address morallydistressing situations.

Acknowledgments:We would like to thank the residents and fellowswho generously gave their time to participate in the interviews. Theauthors are grateful to J. Randall Curtis, MD, MPH (Division ofPulmonary and Critical Care Medicine, University of Washington), andJimmy Hoard (Palliative Care Center of Excellence, University ofWashington) for providing institutional support and assistance withparticipant recruitment. We would also like to thank Cynda Rushton,PhD, RN, FAAN (Johns Hopkins Berman Institute of Bioethics), GailGeller, ScD, MHS (Johns Hopkins Berman Institute of Bioethics),Joann Bodurtha, MD, MPH (Department of Pediatrics, Johns HopkinsUniversity), and Anna Jo Bodurtha Smith, MPH, MPP (HarvardMedical School), for helpful feedback on drafts of the manuscript.

Author Contributions: Dr. Dzeng had full access to all of the data inthe study and takes responsibility for the integrity of the data and theaccuracy of the data analysis.Study concept and design: Dzeng, Levine, Smith, Roland, Barclay.Acquisition, analysis, or interpretation of data: Dzeng, Colaianni.Drafting of the manuscript: Dzeng.Critical revision of the manuscript for important intellectual content:Dzeng, Levine, Smith, Colaianni, Roland, Kelly, Barclay.Procurement of funding: Dzeng.Study supervision: Dzeng.

Conflict of Interest Disclosures: All authors have completed andsubmitted the ICMJE form fordisclosure of potential conflicts of interest.There are no disclosures or conflicts of interest to report.

Funding/Support: This study was funded by the Health ResourcesandService Administration T32HP10025-20 TrainingGrant, theGatesCambridge Scholarship, Society of General InternalMedicine FoundersGrant, and the Ho-Chiang Palliative Care Research Fellowship at theJohns Hopkins School of Medicine.

Role of the Funders/Sponsors: None of the funders/sponsors hadany role in the design or conduct of the study; collection, management,analysis, or interpretation of the data; preparation, review, or approvalof the manuscript; or decision to submit the manuscript for publication.

Corresponding Author: Elizabeth Dzeng, MD, MPH, MPhil, MS;Department of Hospital Medicine, University of California SanF r a n c i s c o , S a n F r a n c i s c o , C A , U S A(e-mail: [email protected]).

REFERENCES1. Schneiderman LIJN. Futility in Practice. Arch Intern Med.

1993;153:437–441.

2. Winkenwerder WJ. Ethical Dilemmas for House Staff Physicians. J AmMed Assoc. 1985;254(24):3454–3457.

3. Schneiderman LJ, Jecker NS. Wrong Medicine: Doctors, Patients, andFutile Treatment. 2nd ed. Baltimore: JohnsHopkinsUniversity Press; 2011.

4. Solomon MZ, Donneu LO, Jennings B, et al. Decisions Near the End ofLife: Professional Views on Life-Sustaining Treatments. Am J PublicHealth. 1993;83(1):14–23.

5. Huynh TN, Kleerup EC, Wiley JF, et al. The frequency and cost oftreatment perceived to be futile in critical care. JAMA Intern Med.2013;173(20):1887–94. doi:10.1001/jamainternmed.2013.10261.

6. Piers RD, Max A, Michalsen A, Maia PA, et al. Perceptions ofAppropriateness of Care Among European and Israeli IntensiveCare Uni t Nurses and Phys ic i ans . J Am Med Assoc .2011;306(24):2694–2703.

7. Jameton A. Nursing practice: the ethical issues. Englewood Cliffs: PrenticeHall; 1984.

8. Mobley MJ, Rady MY, Verheijde JL, Patel B, Larson JS. The relationshipbetween moral distress and perception of futile care in the critical careunit. Intensive Crit Care Nurs. 2007;23(5):256–63. doi:10.1016/j.iccn.2007.03.011.

9. Dyrbye LN, Thomas MR, Power DV, et al. Burnout and serious thoughtsof dropping out of medical school: a multi-institutional study. Acad Med.2010;85(1):94–102. doi:10.1097/ACM.0b013e3181c46aad.

10. Elpern EH, Covert B, Kleinpell R. Moral Distress of Staff Nurses in aMedical Intensive Care Unit. Am J Crit Care. 2005;14:523–530.

11. Sanchez-Reilly S, Morrison LJ, Carey E, et al. Caring for oneself to carefor others: physicians and their self-care. J Support Oncol. 2013;11(2):75–81. doi:10.12788/j.suponc.0003.

12. Oberle K, Hughes D. Doctors’ and nurses’ perceptions of ethical problemsin end-of-life decisions. J Adv Nurs. 2001;33(6):707–715.

13. Hefferman PAM, Heilig S. Giving BMoral Distress^ a Voice: EthicalConcerns among Neonatal Intensive Care Unit Personnel. Cambridge QHealthc Ethics. 1999;8:173–178.

14. Austin W, Kelecevic J, Goble E, Mekechuk J. An overview of moraldistress and the paediatric intensive care team. Nurs Ethics.2009;16(1):57–68. doi:10.1177/0969733008097990.

15. Ferrell BR. Understanding the moral distress of nurses witnessingmedically futile care. Oncol Nurs Forum. 2006;33(5):922–30. doi:10.1188/06.ONF.922-930.

16. Houston S, Casanova M a, Leveille M, et al. The intensity and frequencyof moral distress among different healthcare disciplines. J Clin Ethics.2013;24(2):98–112.

17. Bruce CR, Miller SM, Zimmerman JL. A Qualitative Study ExploringMoral Distress in the ICU Team: The Importance of Unit Functionality andIntrateam Dynamics. Crit Care Med. 2014;3:1–9. doi:10.1097/CCM.0000000000000822.

18. Dzeng E, Colaianni A, Roland M, et al. Influence of Institutional Cultureand Policies on Do-Not-Resuscitate Decision Making at the End ofL i f e . JAMA Intern Med. 2015;21287:1–8. do i :10.1001/jamainternmed.2015.0295.

19. Dartmouth Atlas Project. End of Life Care, Inpatient days per decedentduring the last six months of life. 2007.

20. Guest G, Bunce A, Johnson L. How Many Interviews Are Enough?: AnExperiment with Data Saturation and Variability. Field methods.2006;18(1):59–82. doi:10.1177/1525822X05279903.

21. Barbour RS. Checklists for improving rigour in qualitative research: a caseof the tail wagging the dog? Br Med J. 2001;322:1115–1117.

22. Mays N, Pope C. Assessing quality in qualitative research. Br Med J.2000;320:50–52.

23. Golafshani N. Understanding Reliability and Validity in QualitativeResearch. 2003;8(4):597–606.

24. Dzeng E, Colaianni A, Roland M, et al. Institutional Culture and Policies’Influence on Do-Not-Resuscitate Decision-Making at the End of Life. J AmMed Assoc Intern Med. 2015;In Press.

25. Papathanassoglou BEDE, Karanikola MNK, Kalafati M,Giannakopoulou M, Lemonidou C, Albarran JW. Professional Auton-omy, Collaboration with Physicians, and Moral Distress amongEu r o p e an I n t e n s i v e C a r e Nu r s e s . Am J C r i t C a r e .2012;21(2):e41–e52.

26. Dyrbye LN, Thomas MR, Shanafelt TD. Medical student distress: causes,consequences, and proposed solutions. Mayo Clin Proc. 2005;80(12):1613–22. doi:10.4065/80.12.1613.

27. Brazeau CMLR, Schroeder R, Rovi S, Boyd L. Relationships betweenmedical student burnout, empathy, and professionalism climate. AcadMed. 2010;85(10):S33–S36. doi:10.1097/ACM.0b013e3181ed4c47.

98 Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile Treatments JGIM

32

Page 33: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

28. Sheehan D V, White K, Leibowitz A, Baldwin DC. Study of MedicalStudent’ Abuse’ and Misconduct in Medical School Perceptions of Mis-treatment. 2014.

29. Lomis KD, Carpenter RO, Miller BM. Moral distress in the third year ofmedical school; a descriptive review of student case reflections. Am J Surg.2009;197(1):107–12. doi:10.1016/j.amjsurg.2008.07.048.

30. Thomas MR, Dyrbye LN, Huntington JL, et al. How do distress and well-being relate to medical student empathy? A multicenter study. J GenIntern Med. 2007;22(2):177–83. doi:10.1007/s11606-006-0039-6.

31. Neumann M, Edelhäuser F, Tauschel D, et al. Empathy decline and itsreasons: a systematic review of studies with medical students andresidents. Acad Med. 2011;86(8):996–1009. doi:10.1097/ACM.0b013e318221e615.

32. Hojat M, Mangione S, Nasca TJ, et al. An empirical study of decline inempathy in medical school. Med Educ. 2004;38(9):934–41. doi:10.1111/j.1365-2929.2004.01911.x.

33. Patenaude J, Niyonsenga T, Fafard D. Changes in students’ moraldevelopment during medical school: a cohort study. CMAJ.2003;168(7):840–4. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=151989&tool=pmcentrez&rendertype=abstract.

34. Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, GonnellaJS. Physicians’ empathy and clinical outcomes for diabetic patients. AcadMed. 2011;86(3):359–64. doi:10.1097/ACM.0b013e3182086fe1.

35. Hafferty F, Franks R. The Hidden Curriculum, Ethics Teaching, and theStructure of Medical Education. Acad Med. 1994;69:861–871.

36. Feudtner C, Christakis DA, Christakis NA. Do Clinical Clerks SufferEthical Erosion? Students’ Perceptions of Their Ethical Environment andPersonal Development. Acad Med. 1994;69(8):670–679.

37. Jennings ML. Medical Student Burnout : Interdisciplinary Explorationand Analysis. J Med Humanit. 2009;30:253–269. doi:10.1007/s10912-009-9093-5.

38. Colaianni A. A long shadow: Nazi doctors, moral vulnerability andcontemporary medical culture. J Med Ethics. 2012;38(7):435–8. doi:10.1136/medethics-2011-100372.

39. Hojat M, Vergare MJ, Maxwell K, et al. The Devil is in the Third Year: ALongitudinal. 2009;84(9):1182–1191.

40. Bellini LM, Shea JA. Mood Change and Empathy Decline Persist Training.2005;80(2):164–167.

41. Lemonidou C, Papathanassoglou E, Giannakopoulou M, Patiraki E,Papadatou D. Moral professional personhood: ethical reflections duringinitial clinical encounters in nursing education. Nurs Ethics.2004;11(2):122–137. doi:10.1191/0969733004ne678oa.

42. Epstein EG, Hamric AB. Moral Distress, Moral Residue, and theCrescendo Effect. J Clin Ethics. 2009;20(4):330–342.

43. Hough CL, Hudson LD, Salud A, Lahey T, Curtis JR. Death Rounds:end-of-life discussions among medical residents in the intensivecare unit. J Crit Care. 2005;20(1):20–25. doi:10.1016/j.jcrc.2004.09.006.

44. Penson RT, Schapira L, Mack S, Stanzler M, Lynch TJ. Connection:Schwartz Center Rounds at Massachusetts General Hospital CancerCenter. Oncologist. 2010;15(7):760–4. doi:10.1634/theoncologist.2009-0329.

45. Rushton CH, Sellers DE, Heller KS, Spring B, Dossey BM, Halifax J.Impact of a contemplative end-of-life training program: being with dying.Pa l l i a t Suppor t Care . 2009 ;7 ( 4 ) : 405–14 . do i :10 .1017/S1478951509990411.

46. Rushton CH, Kaszniak AW, Halifax JS. Addressing moral distress:application of a framework to palliative care practice. J Palliat Med.2013;16(9):1080–8. doi:10.1089/jpm.2013.0105.

47. Rushton CH. Principled moral outrage: an antidote to moral distress?AACN Adv Cri t Care. 2013;24(1) :82–9. doi :10.1097/NCI.0b013e31827b7746.

48. Back AL, Rushton CH, Kaszniak AW, Halifax JS. B‘Why Are We DoingThis?’^: Clinician Helplessness in the Face of Suffering. J Palliat Med.2015;18(1):26–30. doi:10.1089/jpm.2014.0115.

49. Krasner MS, Epstein RM, Beckman H, et al. Association of aneducational program in mindful communication with burnout, empathy,and attitudes among primary care physicians. JAMA. 2009;302(12):1284–1293. doi:10.1001/jama.2009.1384.

50. Rushton CH. Defining and Addressing Moral Distress Tools forCr i t i ca l Care Nurs ing Leaders . AACH Adv Cr i t Care .2006;17(2):161–168.

51. Lewis-Newby M, Wicclair M, Pope T, et al. An Official American ThoracicSociety Policy Statement: Managing Conscientious Objections in IntensiveCare Medicine. Am J Respir Crit Care Med. 2015;191(2):219–227. doi:10.1164/rccm.201410-1916ST.

99Dzeng et al.: Moral Distress Amongst Physician Trainees Regarding Futile TreatmentsJGIM

33

Page 34: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

RESEARCH ARTICLE Open Access

Collaborating in the context of co-location:a grounded theory studyPamela Wener1* and Roberta L. Woodgate2

Abstract

Background: Most individuals with mental health concerns seek care from their primary care provider, who may lackcomfort, knowledge, and time to provide care. Interprofessional collaboration between providers improves access toprimary mental health services and increases primary care providers’ comfort offering these services. Building andsustaining interprofessional relationships is foundational to collaborative practice in primary care settings. However,little is known about the relationship building process within these collaborative relationships. The purpose of thisgrounded theory study was to gain a theoretical understanding of the interprofessional collaborative relationship-buildingprocess to guide health care providers and leaders as they integrate mental health services into primary care settings.

Methods: Forty primary and mental health care providers completed a demographic questionnaire and participated ineither an individual or group interview. Interviews were audio-recorded and transcribed verbatim. Transcripts werereviewed several times and then individually coded. Codes were reviewed and similar codes were collapsed to formcategories using using constant comparison. All codes and categories were discussed amongst the researchers and thefinal categories and core category was agreed upon using constant comparison and consensus.

Results: A four-stage developmental interprofessional collaborative relationship-building model explained theemergent core category of Collaboration in the Context of Co-location. The four stages included 1) Looking forHelp, 2) Initiating Co-location, 3) Fitting-in, and 4) Growing Reciprocity. A patient-focus and communicationstrategies were essential processes throughout the interprofessional collaborative relationship-building process.

Conclusions: Building interprofessional collaborative relationships amongst health care providers are essentialto delivering mental health services in primary care settings. This developmental model describes the process of howthese relationships are co-created and supported by the health care region. Furthermore, the model emphasizes thatall providers must develop and sustain a patient-focus and communication strategies that are flexible. Applying thismodel, health care providers can guide the creation and sustainability of primary care interprofessional collaborativerelationships. Moreover, this model may guide health care leaders and policy makers as they initiate interprofessionalcollaborative practice in other health care settings.

BackgroundIndividual Canadians seeking mental health services aremost often seen by their primary care provider (PCP).Watson et al. reported that 30–40 % of Canadians whovisit their PCP have symptoms of a mental illness [1].Individuals with mental illness make up at least 20 % ofprimary care patient visits [2] and take up approximately25–50 % of the PCP’s practice time [3]. PCPs treat more

than 50 % of Canadians who are seeking mental healthservices [4–6], while mental health specialists treat only25 % of these individuals [4, 7]. Given these statistics,PCPs make a significant contribution to the overallCanadian mental health system.Although PCPs provide most of the mental health

services, their knowledge, skills, and comfort workingwith those who have mental illness varies. Some authorsdiscuss family physician’s (FP) feelings of discomfortworking with patients with depression [8–11]. Otherauthors, discuss the lack of PCPs’ knowledge and experi-ence as a barrier to treating patients with depression.For example, Henke et al. describe a qualitative study

* Correspondence: [email protected] of Occupational Therapy, College of Rehabilitation Sciences,University of Manitoba, R125-771 McDermot Ave., Winnipeg, MB R3E 0T6,CanadaFull list of author information is available at the end of the article

© 2016 Wener and Woodgate. Open Access This article is distributed under the terms of the Creative Commons Attribution4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Wener and Woodgate BMC Family Practice (2016) 17:30 DOI 10.1186/s12875-016-0427-x

34

Page 35: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

using semi-structured interviews to gather informationabout the barriers to working with patients with depres-sion. These authors collected data from 23 FPs who arepracticing throughout the United States. In describingthe study, the authors include their methods for creatingthe interview guide, the interview process and details ofhow they used a grounded theory approach to analyzethe data. These authors reported six barriers to workingwith patients with depression including, difficulty diag-nosing and a lack of experience. Anthony et al. con-ducted a mixed methods study of 40 PCPs includingFPs, NPs, and general internists from one large urbancentre in the United States [10]. These authors sought tounderstand PCPs’ decision to refer patients for depres-sion care. The authors provide a thorough description ofstudy process including, methodology, data collectioninstruments, and the specifics of the data analysis. Thereported results of this study described the participantsdiscomfort treating patients with depression. Prescribingmedication is an important aspect of evidence-basedtreatment for depression and anxiety [12]. However,FPs report moderate levels of comfort prescribingmedications for these patients [13]. For example,Craven and Bland [14] who conducted a comprehensiveliterature review reported that PCPs are comfortable treat-ing individuals with mental illness who are responsive tomedication that the provider is familiar with prescribing.Goossen et al. conducted a mixed methods evaluation of anexisting CMHC program reported that PCPs, are lesscomfortable when medications need to be changed orcombined [15]; a practice outlined in Canadian practiceguidelines as an important part of improving a patientresponse [16].In addition to prescribing medications, PCPs are aware

of the effectiveness of evidence-based counseling.Grenier et al. surveyed 118 FPs in one Canadian prov-ince and found that 95 % of FPs knew of evidence-basedcounseling for depression and anxiety such as, cognitivebehavioural or interpersonal therapy [17]. These authorsnote that a lack of time and training make it difficult forPCPs to implement counseling within their practices[17]. While individuals with mental illness are mostlikely to be treated by a PCP, the practitioner may notpossess the comfort, training or time to implementevidence-based treatment, leaving patients with less thanoptimal mental health services.PCPs believe that their ability to deliver mental health

services would improve if they had support from mentalhealth specialists [18, 19]. Acknowledging that most ofthe mental health services in Canada are provided byPCPs, physician leaders recognized the need to increasePCPs’ access to mental health specialists in primary caresettings. In 1997, the Canadian Psychiatric Associationand the College of Family Physicians of Canada together

developed a position paper calling on PCPs and psychia-trists to work together [20]. In this paper, Kates, et al.declared that primary and mental health care providerswere joining together to improve access to mental healthservices in what is referred to as shared or collaborativemental health care (CMHC), two terms that are usedsynonymously in this paper [14]. Furthermore, these twoprofessional groups agreed that:

family physicians and psychiatrists work morecooperatively to integrate their respective skills andexpertise in a complementary and cost effectivemanner ([21], p 1785).

Although it was agreed that generalists, PCPs and spe-cialists, mental health providers would work together, lit-tle was known about how to develop the collaborativerelationship and the importance of relationship buildingto the overall interprofessional collaborative process.Today, well over 100 CMHC programs exist in

Canada, each reporting successes [22]. For example,Kates discussed CMHC that were integrated into Ontar-io’s family health teams and who saw symptom reduc-tion and improved functionality for 50 % of the patientswith mental health concerns [23]. Bower et al. examinedoutcomes of CMHC for depression and concluded thatpartnering with case managers who receive supervisionfrom a mental health specialist improved outcomes [24].In terms of system changes, researchers report thatCMHC results in increased access to timely psychiatriccare [25–29], decreased referrals to outpatient psychiatryclinics [27], earlier detection of mental illness, reducedutilization of specialized mental health services [30, 31],and increased continuity of care [27, 28, 32, 33].Researchers also found that individuals who participated ina CMHC program reported decreases in symptomatology,[32, 34–41]; less interference with social activities [32, 33],and increased satisfaction [28]. Furthermore, researchersreport that implementation of CMHC increases PCPs’capacity to work with individuals with mental illness.Several researchers found that subsequent to the initiationof CMHC, PCPs reported having increased, mental healthcare skills and comfort [27–29, 34, 42, 43], providersatisfaction [27, 34], and physician perceived patientsatisfaction [34]. The World Health Organization (WHO)and the World Organizations of Family Doctors (Wonca)released Integrating Mental Health into Primary Care tojustify the need to integrate mental health services intoprimary care settings. One of the key messages reported inthis document is that there is less stigma and discrimin-ation when patients with mental illness are seen in PCsettings [44].While there seems to be some agreement about the

value of CMHC for individuals diagnosed with common

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 2 of 15

35

Page 36: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

mental illness such as depression and/or anxiety, there islittle consensus about the patient outcomes of CMHCwith individuals with serious mental illness. Fitzpatrick,et al. reported that CMHC did not improve patient out-comes for those individuals with serious mental illness[45]. Brown, et al. found FPs offered those with seriousmental illness continuity of care, comfort and familiarity,and a whole person clinical approach [25]. In a chart re-view, Doey, et al. found that individuals with moderateto serious mental illness who participated in CMHC hadreduced number of hospital and emergency room visitsand patients reported high levels of satisfaction and con-tinuity of care [46]. Smith et al. explored the effective-ness of collaborative care and found that while there issome reported improvements in patients with depres-sion, the consistent finding was improved PCP prescrib-ing practices [47].Among those studying CMHC, there is some consen-

sus about the components that contribute to an effectivetreatment program [48]. For example, most CMHC pro-grams include a case manager; psychiatric consultation;brief forms of psychotherapy or counseling such as, cog-nitive behavioural approaches, motivational interviewingor interpersonal approaches; patient education; access toresources; and screening for depression and anxiety [48].While these program components are essential, theymust be developed upon an understanding of the PCP’sneed for collaboration with the mental health specialist[15, 48] and a strong collaborative interprofessional rela-tionship [25, 49–51].Historically, PCPs and mental health providers report

they have poor interprofessional relationships and a lackof mutual trust and respect [52] that seems to underpina proclivity toward poor communication [19]. Katesstated that in addition to not meeting the needs ofpatients’ with mental illness, the relationship betweenPCPs and psychiatrists was poor including, insufficientaccess, poor communication, and a lack of understand-ing and support for the role of PCPs in delivering mentalhealth services [20]. However after over a decade ofCMHC, the Joint Working Group on Shared Care re-ported on the strides made in offering increased accessto mental health services [48]. More recently, Goossenet al. [15] and Benzer et al. [11] recognized and reportedthat the interprofessional relationship is integral toshared care between primary care generalists and mentalhealth care specialists. Although CMHC has been in placesince the late 1990s, the development and sustainment ofthe interprofessional collaborative relationship aspect ofthe shared care model, has not been well developed. Thuswhile the shared care model has been widely imple-mented, we have little knowledge about how generalistand specialists build and maintain their interprofessionalcollaborative relationship. An increased understanding of

how to build and maintain interprofessional collaborativerelationships will provide much needed guidance to thosehealth care providers attempting to navigate this complexprocess.To date, there is little understanding of the relation-

ship building process providers use to support the on-going engagement to work together to provide primarymental health services. Understanding the providers’perspective is essential to developing best practices thatwill ensure patients with mental illness receive the fullbenefits of the interprofessional primary mental healthcare team. Accordingly, we used a qualitative approachto explore the following study question: How do primarycare providers and mental heath care providers collabor-ate to provide mental health care in primary care set-tings. More specifically the research objectives included:

1. To detail the need for IPC in the delivery of mentalhealth services in primary care from the perspectiveof the primary healthcare providers.

2. To detail primary healthcare providers and mentalhealthcare providers experiences and perspectives ofIPC in the context of a primary care program,Collaborative Mental Health Care program.

3. To identify how the individual, group dynamics andsystem influence the IPC process in the context of theShared Mental Health Care program.

4. To identify the opportunities and challenges of IPC inthe context of the Shared Mental Health Careprogram.

This paper describes the grounded theory of interpro-fessional collaborative relationship building that pro-viders described developing and maintaining to delivermental health services in PC settings.

MethodsStudy designThis study was best approached from a qualitative re-search paradigm where the exploration is grounded in theproviders’ experiences of IPC [53]. The purpose of thestudy was not to deduce a single truth, but rather tounderstand the multiple realities of the participatinghealth care providers from an emic perspective [54]. Morespecifically, social constructivist grounded theory method-ology [55] was used to facilitate an inductive explorationof the interprofessional collaborative relationship buildingprocess providers use to work together to deliver mentalhealth services in primary care. Grounded theory asdescribed by Charmaz is an appropriate methodology touse when the study purpose is to understand, rather thantry to explain process. Social constructivist groundedtheory acknowledges the co-creation of the study findingsby both the researchers and participants [55].

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 3 of 15

36

Page 37: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Symbolic Interaction (SI) served as the guiding theor-etical framework for this study. As SI focuses on themeaning individuals ascribe to an interaction, thisframework helps us to explore multiple realities ratherthan to seek a single explanation [56]. In this study,using an SI lens, we focused on understanding themeaning provider participants ascribed to the interpro-fessional collaborative relationship building process asthey engaged to provide mental health services in pri-mary care settings. As SI focuses on meaning ascribedby individuals as they interact with other it is thought tobe a useful framework when one is exploring processand change [55]. Further description of the study designand conceptual framework used is available in the meth-odology paper by Wener and Woodgate [57].

Ethics, consent, and permissionThe University of Manitoba Health Research Ethics Boardprovided ethical approval for this study (H2011:003).Informed consent was obtained from participants prior tothe commencement of all interviews.

Consent to publishConsent to publish anonymized individual participant’sdata was obtained as part of the informed consent process.

ParticipantsPurposive sampling was used to recruit providers whoparticipate in one health region’s CMHC service. All 110PCPs, (100 FPs and ten nurse practitioners (NPs), 16shared care counsellors, and eight shared care psychia-trists who participate in the health region CMHC pro-gram were invited to participate through recruitmentflyers. We sought to achieve diversity in terms of geo-graphical location of practice, physician remunerationmodel, and practitioner’s gender in the sample throughmaximum variation sampling [58]. There are 11 identi-fied communities within the urban centre, seven ofwhich have a CMHC service. Recruitment occurredfrom all seven communities that offered CMHC. Ingeneral, family physicians within this urban centre areremunerated using a fee-for-service model or receive ayearly salary. We sought to ensure that we recruited arelatively equal number of family physicians from eachof the remuneration models. Previous studies haveshown that the average socioeconomic status, educationand health care needs vary among these communities(MCHP). We assumed that the patients living in each oftheses communities are most apt to attend health carepractices located within their communities and thatthese differences in income, education and health careneeds, may contribute to the health providers’ interpro-fessional collaboration experience. Literature suggeststhat females are more apt to collaborate than males,

therefore we attempted to ensure that we had represen-tation of both male and female FPs, NPs, psychiatristsand counsellors [59, 60]. Sampling continued until cat-egories could account for new data and theoretical suffi-ciency was achieved ([61], p 117).

Data collectionDemographic information was collected to obtain a profileof the participants. Information about how the providerscollaborate to provide mental health services in primarycare was gathered using semi-structured in-depth individ-ual interviews and focus groups that took place in a privateroom in the participant’s place of work. Data was collectedfrom three groups of participants: 1) PCPs, 2) groups ofproviders that included FPs, NPs, psychiatrists, and coun-sellors, and 3) health authority regional leaders. First, PCPswere interviewed individually. The initial interview guidewas created based on the results of a literature reviewand a previously completed program evaluation [15].The interview guide for the individual PCP interviewsincluded open-ended questions about the patient popu-lation served, experiences providing mental health ser-vices, need for collaboration with mental healthspecialist and their experiences of collaboration.Second, interprofessional focus group interviews in-

cluding PCPs, and mental health care providers wereconducted. The focus group interview guide was basedon the data analysis of the PCP interviews and the litera-ture, and focused on understanding the details of theproviders’ experiences of interprofessional collaborationto provide mental health services to patients. Focusgroup interview questions were created based on theemergent themes from the PCP individual interviewsand the literature, and included asking providers aboutthe meaning of interprofessional collaboration, processof collaborating, strengths and challenges of interprofes-sional collaboration, process of resolving conflicts amongteam members, influence of co-location on the interpro-fessional collaboration process, and the role of the healthregion in interprofessional collaboration. Questionsabout interprofessional conflict were added to the inter-view guide when it was noticed that participants did notdiscuss this issue, although it is reported in the litera-ture. Third, interviews with the regional leaders wereconducted. The Regional leaders’ and decision-makers’interview guide was created based on the emergentfindings from the previous interviews. Although theseinterview guides were used for all interviews, the inter-viewer (PW) was responsive to participants’ invitingthem to further discuss issues raised. As well, the inter-viewer encouraged the participants to raise any issuesthat the participants wanted to discuss prior to endingeach interview. A sample of interview questions from allthree guides is included in Table 1.

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 4 of 15

37

Page 38: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

Data analysisAll demographic questionnaires were analyzed usingdescriptive statistics. Individual and group interviews wereaudio recorded and transcribed verbatim. Prior to initiatingcoding, the transcripts were read several times to gain anunderstanding of the whole. In keeping with groundedtheory, the coding process consisted of initial and focusedcoding phases [55]. We analyzed the data, assigning initialcodes for each transcript and writing memos to form initialdefinitions [55]. Using focused codes as preliminary cat-egories, we wrote more in-depth memos from the firstseven interviews and used constant comparison, remainingopen to new and emerging categories as we analyzed theremaining interviews [62]. Authors met to discuss theoverarching theme and categories to achieve consensus.Interview transcripts and a newsletter describing thepreliminary findings were mailed to all study participantsfor feedback prior to the finalization of the overarchingtheme, categories and developmental model however, noparticipants suggested changes to the proposed categories.We included several methods to ensure study rigour

[63]. The credibility and dependability of this study wasestablished by aligning data collection methods with thestudy questions [57]. Data was collected over a longperiod of time and included participants from differentgeographical locations and from practices with differentremuneration models. We kept an audit trail and reflex-ive journal to establish confirmability [63]. Transferabil-ity was explored by sharing the overarching theme,categories and developmental model with study partici-pants and solicitation of feedback at conferences,presentations, and from peers [62–64].

ResultsDescription of participantsHealth care providers (n = 32) and health region leaders(n = 8) participated in this study and completed thedemographic questionnaire. Of the health care providersthat participated in the study, there were 16 (50 %) FPs,8 (25 %) nurse practitioners (NP), 3 (9.4 %) psychiatrists,and 5 (15.6 %) counsellors. Of the 16 FPs, 10 (62.5 %)reported that they participate in the provincial fee-for-

Table 1 PCP Individual Interview Sample Questions

1. Tell me about your primary care practice?

2. Describe the patient population in your primary care practice?

3. Tell me about your experiences in your practice of providinghealth services to patients with mental health problems?

4. Tell me about an experience where you were asked by a patientto provide mental health services/support to a patient when youfelt comfortable or equipped to do so?

5. Tell me about an experience where you were asked to providemental health services/support to a patient when you did notfeel comfortable or equipped to do so?

6. What have been your experiences working with the psychiatrist?

7. What have been your experiences working with the counsellor?

8. What kinds of decisions were made during these collaboration?

9. How did the collaborative decisions meet your needs?

10. How did the collaborative decisions meet your patient’s needs?

PCPs and MHPs Focus Group with Sample Questions:

1. Tell me what Interprofessional collaboration means to you?

2. In your particular practice tell me who is involved in theinterprofessional collaboration process to deliver mental healthservice?

3. How does co-location influence the interprofessional collaborationprocess?

4. Tell me about your approach to patients?

5. How is information such as decisions communicated betweenhealth care providers?

6. What are your team’s strengths?

7. What have been your biggest challenges collaborating to delivermental health services?

8. Tell me what happens when there is disagreement betweenproviders? How are conflicts resolved?

9. How does the Shared Care program or the WRHA supportinterprofessional collaboration to deliver mental health services?

Regional Leaders and Decision-makers Focus Group and Individual In-terviews Sample Questions:

1. From your perspective, what is the role of the various teammembers in delivering mental health care?

2. What do you see as your role in relation to delivery mentalhealth care in primary care settings?

3. Shared care is thought to involve interprofessional collaboration,what does that mean to you?

4. Describe how interprofessional collaboration is used to delivermental health services in primary care?

5. What structures does the program or the region provide thatsupports interprofessional collaboration in Shared Care MentalHealth? Are there other structures that you think would provideadditional support or facilitate greater collaboration?

6. What processes do you think are facilitative of interprofessionalcollaboration and how does the program or region supportthese processes? Are there other processes that you think couldmake a facilitating contribution to interprofessionalcollaboration?

Table 1 PCP Individual Interview Sample Questions(Continued)

7. Describe any or how the program or region impedeinterprofessional collaboration? What kinds of things could bechanged to remove these barriers?

8. What role does this group play in developing and facilitatinginterprofessional collaboration?

9. What resources does this group access to encourage andsupport interprofessional collaboration? What kinds of resourcesare missing/unavailable that could further supportinterprofessional collaboration?

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 5 of 15

38

Page 39: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

service (FFS) remuneration program and 6 (37.5 %) ofthe FPs stated they receive a salary from the region(SFP). All NPs, psychiatrists and counsellors receive ayearly salary from the health authority, the regional bodyresponsible for health care delivery.The providers’ ages varied within each of the provider

groups from 30 years to over 60 years of age. However,within the PCP sample, FFS FPs tended to be older thaneither the SFPs or the NPs and the NPs tended to beolder than the SFPs. For example, 70 % of FFS FPs re-ported they were 50 years of age or older while none ofthe SFPs or NPs were over 50 years of age. In terms ofyears with the CMHC program only one of the 32 healthcare provider participants had been with the CMHCprogram for less than one year, while ten participantshad greater than 5 years’ experience in the program.Taken together the health care providers worked in 12different primary care clinics that varied in geographicallocation within the health region. Eleven FPs and fiveNPs participated in the initial individual interviews thattook place over a 1 year period, March 2011 to February2012. The six focus groups included 2–4 participantsand took place over a 6 month from the end of Novem-ber 2012 to May 2013. One counsellor, and two psychia-trists participated in more than one focus groupinterview because they provide service to more than oneclinic. In these cases providers were directed to talkabout their experiences in each clinic within the separatefocus groups. One family physician participated in bothan initial interview as well as a focus group and no spe-cific directions were provided by the interviewer.In addition to these health care providers, eight mem-

bers of the regional leadership group participated ineither a focus group or an individual interview based onthe individual’s ability to attend the focus group. Theseinterviews took place over a 2-month period from July2013 to August 2013. The regional health leaders in-cluded individuals who belonged to a variety of healthprofessions and had additional education in, healthsystems and administration. The members of this groupwere responsible for overall implementation and moni-toring of the CMHC program. Pseudonyms are used inthis manuscript to maintain confidentiality of studyparticipants.The findings revealed one overarching emergent

theme, Collaborating in the Context of Co-location thatincludes a four-stage developmental interprofessionalrelationship building model. The emergent categorieswere the four stages of the developmental model andincluded: Looking for Help, Initiating Co-location,Fitting-in, and 4) Growing Reciprocity. This model andfour developmental stages describe the role of the healthregion leaders and the providers in creating interprofes-sional relationships amongst the PCPs and mental health

care providers. These relationships enabled providers todeliver primary mental health care. The authors usedmember checking to confirm that the developmentalmodel and stages were an accurate representation of theparticipants’ interprofessional collaborative experiences.These developmental stages held true across professionsand gender.Collaborating in the context of co-location was the

overarching theme that describes the evolving interpro-fessional relationships between primary care and mentalhealth care providers for the purpose of meeting primarycare patients’ mental health needs. Collaborating in thecontext of co-location is how the mental health careproviders who are part of the CMHC program are situ-ated within the PCPs office to facilitate the PCP’spatient-focused provision of primary mental health ser-vices. Lisa, a nurse practitioner describes how she and aco-located psychiatrist were able to provide mentalhealth care when otherwise, this patient would not havereceived treatment. Furthermore, the psychiatrist is ableto fulfill the NP’s patient care need, being available atthe patient’s PC appointment time: I can think of atleast, well more than one time… I had someone that wasclearly very ill, with no insight. And would not agree tocome and see a psychiatrist. I needed that assessmentdone… I just had to arrange for him to have an appoint-ment with me… and then have our psychiatrist just kindof join us… being co-located allowed for that to happen.(NP, Lisa)In supporting PCPs, all providers use a variety of

communication methods with the explicit intention oflearning to work together to both provide and enhancethe capacity of primary mental health services. The pro-viders’ evolving relationship proceeds through fourstages over time that begin with looking for help to pro-vide mental health services, to a stage where providersparticipate as partners of patient care as shown in Fig. 1.During each stage of development the providers buildupon the aspects of the relationship established duringthe previous stage. The groups of providers were alwaysfocused on patient care using varied communicationstrategies that were implemented flexibly depending onthe needs of the individual practice. Overall, co-locatedgroups of providers moved through the stages at differ-ent rates of time and not all interprofessional collabora-tions develop to the stage of growing reciprocity.

Stage 1-looking for helpLooking for help is when the PCPs and regional leaderslook to mental health experts to work with PCPs to helpPCPs to deliver mental health services in their primarycare settings. Participants in this study expressed theirneed for help; access to mental health services and clin-ical experts to help them increase their mental health

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 6 of 15

39

Page 40: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

knowledge and skills. PCPs in this study, discuss howthey need timely access to mental health services andhow this access was not available prior to participatingin CMHC program.

I have worked at other places where a 3-month waitfor psychiatry and an eight-week wait for counsellingis a short wait. Usually by that time, the problem thatthe person has come in to ask for help has now fizzledin one way or the other. So you’ve missed thatopportunity. So access in a timely manner is massive.And I think that that only expedites the patient’sability to improve or get better. (NP, Evelyn)

Although PCPs are patient-focused and want to pro-vide mental health services to primary care patients, theyperceive they have a lack of time, comfort and/or expert-ise. Comfort working with patients with mild to moder-ate mental illness varied amongst the PCPs participatingin this study, with more experienced PCPs reporting thattheir comfort working with patients with a mental illnesshas grown over time and with life experiences. Sarahexpressed this growing comfort:

I think as a whole with being in practice for a longtime…I think part of it is just my own experience andmy own competence or comfort with feeling not asoverwhelmed with some of the people that come inwith those problems. (FP, Sarah)

Participants in this study all reported that patientswith mental illness that are difficult to diagnose, or that

have a personality disorder, and those that are not re-sponsive to medications require that PCPs have special-ized knowledge and skills that are beyond their ownclinical capacity. For example, this FP with many yearsin clinical practice describes the circumstances when herequires specialist help. …mild to moderate depression Ican usually handle. People with severe depression, peoplewho present with mild to moderate depression who arenot responding well to my initial approach, that’s wherethe call for help usually comes in (FP, Gary). As patient-focused PCPs, these study participants want to provideprimary mental health services, are aware of their know-ledge and skill limitations, and require help from mentalhealth specialists.At a health region administrative and clinical level,

the leaders identified and embraced the need toenhance mental health services in primary care settingsthrough interprofessional collaboration between gener-alist PCPs, and specialists mental health care providers.As another regional health leader explained, the mentalhealth service enhancement in primary care was logicalas PCPs were already playing a key role in the mentalhealth system, …the need for collaboration… primarycare physicians are providing a significant amount ofmental health services. That’s a driver. (RegionalLeader, Ralf )

Stage 2-initiating Co-locationInitiating Co-location is the regional leaders belief in theusefulness of the CMHC model and then situating themental health providers into the primary care clinics. Asthis regional leader explains, learning about collaborative

Fig. 1 Stages of Interprofessional Collaborative Relationaship Building

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 7 of 15

40

Page 41: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

mental health programs from an expert convinced herthat co-location of providers was the next step in improv-ing the mental health system: I had been to a conferencewith Nick Kates (Canadian Founder of Collaborative Care)and gone to a couple of presentations and thought, this(co-locating providers) is where we need to go as a system(Regional Leader, Leanne).Initiating co-location, that is, geographically bringing

providers together signaled to the PCPs and mentalhealth care providers that the leadership was committedto intra- and interprofessional collaboration in primarycare sites. As this counsellor and psychiatrist describe,creating the structures and processes to co-locateproviders meant the regional leaders believed in theprogram: the (health) region supports collaboration be-cause they’ve put this structure into place for us(Counsellor, Nofar); they (the health region) pay me asalary that I’m able to participate in the program(Psychiatrist, Eleni). Regional leaders secured the ser-vices of the psychiatrists, and counsellors providingyearly salary arrangements with an understanding thattheir days would include time for collaboration. An-other counsellor and psychiatrist explain how initiat-ing co-location, the regional leaders understand thatproviders need face-to-face time and value it as acritical component of the program. In this examplethe providers use the term collaboration to meanface-to-face time working together.

…if I’m spending (face-to-face) time collaborating withany of the primary care providers, I know that SharedCare sees that as a legitimate use of my time. …from aShared Care perspective, we still need to see a certainamount of people but the (face-to-face) time spentcollaborating is equally or more important even thanthat as a program. (Counsellor, Elia)

It’s (collaboration) valued. (Psychiatrist, Daniel)Unlike the PCPs on salary, initiating shared care in

FFS PC sites regional leadership needed to be more flex-ible in how and when providers were co-located. For ex-ample, regional leaders had to negotiate with providersabout the use of rooms and time for collaboration. ThisFP describes how part of bringing the providers togethermeant that providers needed to be willing to providespace for the mental health providers. While this mayinitially be perceived as negative, financial compensationalleviated the situation:

…it might actually work even a little negativelybecause Patty (counsellor) is using one of my roomsand if I have a resident then I’m short one room, butShared Care does pay us sort of a token rent so in thelong run there’s no negative (FP, Hart)

Stage 3-fitting-inFitting-in is when co-located mental health providersand PCPs begin to interact within one another to pro-vide mental health services to PC patients’. For manyPCPs, bringing providers together was about creating afamiliarity with the specialist provider that was pro-foundly different from the historical non-co-locatedgeneralist/specialist relationship. In this relationship, themental health care providers work to fit-in into the PCclinics, interacting with the PCPs as they provide mentalhealth services that the PCP identifies needing for thepatients. During this stage all PCP study participantsidentify needing mental health consultation for diagno-sis, medication management, and therapy. Essential tothis this developmental stage is the mental health careprovider being flexible with their time in order to fit inwith the unique schedule of a PC clinic and/or the PCP.For example, one counsellor purposely altered his sched-ule to stay late into the early evening, ensuring that hewas free to meet with the physicians when they wereavailable. One of the psychiatrists at another PC settingdescribes waiting outside of physician’s examining roomto be able to catch the doc between appointments.Another FP describes how the psychiatrist and counsellorhave, learned to fit with him, Because I don’t eat lunchdownstairs. So, in my office, they’ve learned that, So if theywant to find me they can. (FP, Michael)During this stage mental health care providers needed

to develop patient-focused communication strategies thatwere flexible and fit with each PCP. However, in somepractices psychiatrists and counsellors reported that notall PCPs consulted with them nor did all PCPs meet withthem about the patients seen. Both mental health pro-viders and PCPs described how fitting-in occurred withsome providers and not others but in all cases mentalhealth services were not provided and the relationshipsdid not progress. This counsellor describes how she is ableto develop a relationship with those PCPs willing to meetwith her and the challenge when PCPs are not preparedto make the time to share in the care of patients:

The challenges, that I believe that we get along reallywell but I can’t say that for every physician…Andpeople do have different willingness to meet and toshare and collaborate…. it’s like getting the maildelivered. They love having it come to the door andthey don’t want it. But they don’t want to necessarilygo to the corner to pick it up, you know. And so we’rehere. Are they willing to put in extra effort? To workwith me I would say, yea, it’s kind of a workingcollaboration. Not just the talking. (Counsellor, Lori)

For this FP it is clear to him that when a PCP is notwilling to meet with the mental health care provider

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 8 of 15

41

Page 42: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

then the PCP is declaring that they are opting out of thecollaborative relationship.

…if you’re providing a service for us and be willing totalk to us and everything else, to just say I won’t eversit down with you and talk.…fine, then you’veexcluded yourself from this group…. It’s just got to bethat way at some point. (FP, Michael)

Using patient-focused communication strategies suchas short hallway conversations or patient referral formsalong with the mental health care providers’ timely ser-vice provision, providers become more familiar with oneanother and their interprofessional relationships develop.One of the FPs describes how the face-to-face patient-focused interaction between providers is a key aspect ofcreating familiarity: we’ve said over and over again that’sbeen a huge part …you literally can talk to somebody inthe hallway … just that physical presence is helpful … ahuge part for us (FP, Adi). Collaboration was difficult forPCPs who did not fit in with providers at particularclinics. For example, when the mental health care pro-viders work on days when a PCP was not present, thePCPs did not perceive that the mental health specialistservice was available:

… maybe that is there (the ability to email or call thepsychiatrist) and I’m just not aware of it. …I’m not inevery day, she’s in on a day that I’m not here, …I don’tever see her…. (FP, Jacquie)

Most of the mental health providers discussed howthey expected PCPs to discuss their referral to thepsychiatrist or counsellor with the patient to ensurethere was an understanding and agreement from thepatient. This counsellor suggests that PCPs who do notaccept their responsibility do not fit with the CMHCprogram.

I have someone (PCP) who habitually sends me peoplethat don’t show up. That this person (PCP) kind ofdoesn’t get it or they don’t communicate to theirpatient what it’s really all about and why they have tocome or why they would benefit by coming. I wouldn’twant anybody seeing me because they have to. Becauseas you’re, some people (PCPs) just won’t fit, you know.Because they have, there’s some responsibility to dosomething. (Counsellor, Lori)

However, as Juliette describes during this third stagewhen providers fit in with the PC clinic, collaborationwithin the context of co-location moves beyond physicalproximity of providers to the receptivity providers feelamongst them: … the biggest difference is one of familiarity

cause I see Samantha (the counselor) every day that I workhere and Gretta (the psychiatrist) …she’s very approach-able, she’s happy to talk about cases. (NP, Juliette). As theproviders work together to ensure the patients’ mentalhealth needs are met, they are simultaneously creating in-terprofessional communication and service delivery strat-egies that work for their particular PC practice.Written communication is an important aspect

throughout the fitting-in stage. PCPs initiate a consult-ation to a mental health care provider and receive writ-ten consultation reports. While mental health servicesare provided to the patients, mental health care pro-viders write progress notes in a common patient chartor electronic medical records (EMR). These writtenforms of communication contribute to building PCPs’mental health knowledge, skills and comfort. FP partici-pants describe how the specifics of the written commu-nication processes are important to the PCP’s capacity totreat patients. This FP describes that because the writtenconsultation includes treatment specifics, it is facilitativeof the provider’s ability to comfortably treat the patient:

I would look to that written consult… they’re veryspecific as far as recommendations go for medications,for doses, for resources, (FP, Leslie)

In contrast, one FP describes the inconsistent commu-nication she typically experienced prior to participatingin the CMHC program:

I had a patient who has a mood disorder who wasadmitted… I worry about these people when I don’tsee them, a discharge summary may come four monthsafter they’ve been discharged from hospital, the flow ofcommunication is often lacking. (FP, Adi)

Although the written forms of communication areimportant, once the mental health consultation processwas initiated, the PCPs relied on talking directly to themental health providers for day-to-day patient-focusedservice provision. As this nurse practitioner describes,talking with the mental health provider facilitates timelytreatment planning that is perceived to be meeting thepatient’s needs.

…she was evaluated and then we had a conversationright at my desk, right after she was evaluated and wetalked about what do. (NP, Donna).

All participants discussed that during this fitting-instage, being familiar with one another facilitated directcommunication, such as quick talks before a patient isseen or after a patient leaves the visit. Most study partic-ipants describe using direct communication between the

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 9 of 15

42

Page 43: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

PCP and counsellor as an efficient and timely approachto patient care.

Stage 4- growing reciprocityThis last stage in the developing interprofessional collab-orative relationships in the context of collaboration iswhen the providers come to know and care about oneanother, value each other’s personal and professional ex-pertise, and discover shared patient care values. ThePCPs in this study appreciated when the psychiatrist andshared care counsellor shared their knowledge and sug-gested assessment and treatment approaches that en-abled the PCP to respond to patients mental healthneeds confidently and in a timely manner. PCPs whoparticipated in this study expressed an unequivocal trustin the psychiatrist and shared care counsellor. For ex-ample, Jacquie a FP, expresses appreciation for andconfidence in the medication management suggestionsprovided by the psychiatrists: …if I’m having trouble get-ting the right medication, then I’ll refer to the psychiatristand then I definitely take their opinion…(FP, Jacquie).Many study participants shared that they implementedthe treatment recommendations as suggested and thatthey would not consider changing what was recom-mended: …I would never alter it from what the psych-iatrist has suggested but initially make sure I follow thatexactly as they’ve suggested… (NP, Susan). On the otherhand, this FP defines the interprofessional relationshipin terms of being most responsible and acting on behalfof the patient:

I’m still quarterback, I’m still the guy that’s runningthe show for my patient and I’m ultimately responsiblefor what’s going to happen, and I have to take theadvice of the consultant and decide whether I thinkthis is appropriate or not…Sometimes knowing yourpatient or knowing a different circumstance saying thisisn’t going to work you may not follow that bit. (FP, Ira)

Participants also express relief and appreciation thatthe shared care counsellor knew of other mental healthresources that the patients could access: …knowing whatother places offer counselling cause that's one of the bigblack holes out …I have a sense of a few things just thatI’ve learned over time, but she (counsellor) knows awhole lot more than I do so. (FP, Adi). The PCPs relief iscoupled with the counsellors’ recognition of how theirability to provide assistance deepens the developinginterprofessional relationship: …once somebody sees youactually can be helpful that will go a long way in build-ing a relationship. (Counsellor, Brandon)During this stage, the interprofessional collaborative re-

lationship becomes deeper, as the valuing of one anotherprocess becomes reciprocal and providers recognize that

they have shared values such as providing holistic patientcare. This FP describes the psychiatrist or counsellor look-ing to him to ensure the specialist has a complete and hol-istic understanding of the patient…they’ll call me in andask if any other thoughts that I have [sic], cause a lot ofthese people I’ve known them for 35 years, I have the ad-vantage of experience with them. (FP, Hart). Similarly, themental health providers value and understand how thePCP’s long-time knowledge of the patient was an import-ant aspect of patient care:

There’s a lot of brainstorming too because if I just meeta client, for the first time, I’ll come back, (to the PCP)…these guys know that client well. And so I’ll say, wellthis is my impression or this is kind of my feeling, whatdo you think? And so then it’s usually we tease outkind of where we go together, you know. (Psychiatrist,Eleni)

At this stage there is an ease and comfort betweenproviders that has moved beyond a one-way valuing to amore comfortable reciprocal relationship that is basedon a shared value of providing patients with the best carepossible. As this provider describes there is an increasingcomfort that includes flexibility …sometimes I will gothere or they will go here or we’ll meet in the corridorand say I’d like to talk about so and so and it’s a verycomfortable relationship. (FP, Gary) For some groups ofproviders, a perceived non-hierarchical structure was animportant contributor to the growing reciprocity. Thiscounsellor describes the impact of perceived non-hierarchy on the providers’ sense of cohesiveness

there’s respect for the different roles that people playwithin the clinic…that has separated this clinic interms of functioning and cohesiveness in a way thatlots of clinics set up similarly haven’t really been ableto achieve. And I think that it’s really been because oftaking out that hierarchical structure. That has madethe clinic function so much better as a workplace.(Counsellor, Corey)

During this stage providers’ shared value of beingpatient focused is heightened and together they createrelationships that ensure patients have timely access tomental health services, while at the same time, retainingthe PCPs’ position as the key health care providers. ThisFP shares how the PCP and mental health specialistexpressed their joint commitment to timely patient fo-cused care:

I know myself and at least one of my other colleaguesmay call him up and saying you know I’ve got thisperson or what do you think about this medication for

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 10 of 15

43

Page 44: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

this person that you already know and being able tomake a lot of those decisions with his you know okayor with his input on a more informal and timely basis.(FP, Adi)

PCPs describe developing relationships with mentalhealth providers that are based on trust and respect, andhow this creates not only trust between providers butalso trust between PCPs and the patients. This providerdescribes how the patients benefits from the establishedrelationship among providers:

… from the patient’s perspective that’s helpful that weactually know each other. I’ve said to people there’sother specialists …I don’t know them but I thinkthey’re good… I think from the point of the view of thepatient because it’s very personal that everybody’s kindof connected. (FP, Sarah)

Many of the study participants described that the col-laborative relationship developed over time. This PCPshare the sense of ease and trusting collaborative rela-tionship that develops over time:

It’s also about establishing a relationship with them aswell…I think the more you collaborate, the more youunderstand each other and the more your thinkingtends to line up around how you deal with yourpatients or your clients. Like working with [counsellor’sname] for 8 years, I know how [counsellor’s name]thinks. I know what her patients are like. I know howshe is going to treat her patients. I’ve worked with[psychiatrist’s name] for, I don’t know (FP, Jacquie).

Another FP describes how the collaboration facilitatespatients receiving the right care at the right time:

…if the counsellor, was to see somebody and thoughtthis person needs medication, they would come outand talk to me about it or as I say if it’s somebodythat I think really needs to be seen more quickly thanaverage I will make a point of going around andtalking to the counsellor… (FP, Gary)

At this later stage of development the health careproviders anticipate that as they come together toprovide patient care, there will be different opinionsabout how best to meet the patient’s needs. Providersin this study understood that these differences eman-ate from the providers having different knowledgeand skills but that all providers are motivated to dothe best for the patient. Understanding that all pro-viders share a common interest in meeting the needsof the patient seems to help the providers reframe

interprofessional provider into a culture that wel-comes diverse perspectives:

The only times there has been somewhat of adifference has been more on the impressions that we’vehad of what’s going on because we come to it from twodifferent angles. But I don’t think there’s ever beenreally a disagreement about how to go forward fromthere because it does always involve the patient andtheir opinion…, and their preferences. And it does alsoalways come from a place of wanting to do the bestthat we can by that person. And so it’s hard toimagine conflict when you have the same ultimategoal in mind. (Counsellor, Corey)

Providers express the evolving collaborative relation-ship with mental health providers as caring about oneanother on a more personal basis. This FP explains howwhen providers work together and get to know one an-other on a more personal basis, the relationship deepensand creates a closeness between providers that enrichesthe work relationship:

…when you know somebody and you know that they’redue with their next pregnancy or who their husband isand you know what their kids do.. It’s really hard tohave a bad relationship when you know people reallywell. And it’s so much easier to have great workingrelationships when you are that intimate with people…(FP, Taryn)

DiscussionOur study describes the stages of developing interprofes-sional collaborative relationships in a CMHC program ina primary care setting which to date, has received lim-ited study. Using an SI lens allowed us to understandthe meaning that the interactions between the regionalleaders, PCPs, mental health care providers and theprimary care context contributed to provider perceivedinterprofessional collaborative relationships. The resultsof our study situate co-location as a crucial componentto developing interprofessional collaborative relation-ships in the shared care, primary care practice setting.Co-location has consistently been identified as an im-portant factor in building collaborative teams betweenthose in mental health and primary care [14, 41, 65].Allport found that interpersonal contact is an effectiveway to overcome intergroup conflict, a suggestion heput forward as the contact hypothesis [66]. In this study,co-locating providers set the stage to develop interpro-fessional collaborative relationships. Similarly, Kates etal. reported that co-location enhances communicationand eases the referral process, case discussions and im-proves continuity of care [27]. Participants in this study

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 11 of 15

44

Page 45: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

described that co-locating providers encourages inter-professional interaction that they perceive to be criticalto the developing interprofessional relationships.Hewstone and Brown agreed that interpersonal con-

tact is important, however, they state that it is not suffi-cient to increase trust among group members [67].These authors suggest that to increase trust amonggroup members there also needs to be personal inter-action, equal status, common goals, support from the in-stitution or agency, and cooperation. Mulvale et al.found that personal contact and face-to-face case confer-ences between providers is an important contributor tothe success of the CMHC program [41] and FPs whoworked with co-located counsellors and psychiatrists re-ported the highest levels of satisfaction [18]. The partici-pants in our study also emphasized the importance ofboth face-to-face interaction as well as written forms ofcommunication. Providers in this study also discussedthe importance of a non-hierarchical structure, a com-mon focus on improving patients’ mental health, andsupport from the program and health region leadership.In this study, participants from different practices de-

scribed a similar road taken to develop their relation-ships that included co-location of providers, a focus onfitting-in to the PC culture and clinic, and then a senseof having arrived at a mutually respectful and collabora-tive relationship where providers knew each other pro-fessionally and personally. However, while this studydescribes the patterns of the interprofessional collabora-tive relationship development, it falls short of helping usto understand what and how the team propels itselfforward.While the stages of the interprofessional relationship

building process in a CMHC program have not been de-scribed previously, Chidambaram and Bostrom con-ducted a review of group development models. Theseauthors described two broad types of group develop-ment, sequential and non-sequential [68]. In health care,most authors describe team development using a se-quential linear progressive model where the team ma-tures and is defined by improved performance over time[69]. Tuckman and Tuckman and Jensen’s sequentiallineal progressive model that includes five stages ofdevelopment [70, 71] is widely accepted by experts ofsmall group processes. Moreover, this team developmentaltheory has been used to describe interprofessional healthcare team development [72–74]. However, while the studyparticipants described that interprofessional collaborativerelationships develops over time, the participants in thisstudy also describe the critical role of the regional leadersin the interprofessional team development.In our model the regional leaders play an important

role in the first two stages, identifying the need for inter-professional collaboration and initiating co-location of

providers. Organizational leaders have long been recog-nized as an essential element to successful interprofes-sional collaboration. For example, San Martin-Rodriguezreviewed theoretical and empirical studies to determinethe components for successful collaboration. These au-thors found that when the organization believes in inter-professional collaboration i.e. identify and/or understandthe need for collaboration and create physical proximitybetween providers are among the important features ne-cessary for interprofessional collaboration [75]. D’Amourand Oanadasan, 2005 also suggest that the organizationalleaders or decision makers must be supportive and playan important role in implementing interprofessional col-laboration [76].The participants in this study describe fitting-in, where

the mental health care provider fulfills the PCP’s patientneeds by sharing their clinical expertise. As the PCPsrecognize that their patient needs are being met, all pro-viders begin to respect, trust and value one another,similar to the “norming” process that is Tuckman’s thirdstage of group development [70]. In a recent study, Ben-zer et al. reported that when mental health care pro-viders in PC settings attend to the PCPs identifiedpatient needs, communication between the PCPs andmental health care providers increased [11]. While Ben-zer’s work makes an important contribution to our un-derstanding of interprofessional communication, it wasnot describing developmental stages nor grounded inhealth care providers’ experiences.The fourth stage of our proposed relationship building

model, Growing Reciprocity includes descriptions of in-creased cohesion, a sense of trust, belonging, and to-getherness. Cohesion is reflected in the study participants’discussion of comfort, trust, respect, sharing of values,and valuing of differences in opinion amongst providers.Cohesion, is thought to be an essential feature of groupperformance [77, 78] and was identified as a key compo-nent of interprofessionality [76, 79]. While several partici-pants in this study discussed the importance of cohesion,further research would need to be done to understand therole of cohesion amongst the interprofessional health careproviders.In our study, participants discovered that they all

valued a patient focus and holistic care that addressedpatient and provider needs. As the participants in ourstudy worked together, they recognized that they neededto be flexible depending on the primary care contextand the unique needs of the patient and/or provider.Participants described adapting their communicationstrategies, approaches and schedules to meet each other’sand the patient’s needs.The two central components of our model, communi-

cation strategies and the patient-centred approach havebeen reported findings of several previous studies. A

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 12 of 15

45

Page 46: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

commonly reported findings is the importance of pro-viders communicating openly aiming towards reciprocaldialogue [18, 25, 32, 34–41, 46] while Lucena and Le-sage, discuss the importance of written communicationstrategies [80]. In support of the second key finding, au-thors describe how a focus on the patient may assistteams in dealing with role conflict [25, 28, 81]. Teamconflict is often a result of role boundaries, scope ofpractice, and accountability. However, in our study pro-viders focused on providing patient focused care wherethe PCP requested interprofessional collaboration basedon the patient’s identified need for mental health ser-vices. Rather than focusing on areas that are the typicalsources of conflict, such as role boundaries and scope ofpractice [82], providers in our study recognized thatconsideration of all of the perspectives may best meetthe patient’s needs. Maintaining a patient focus helpedproviders in our study to not categorize the varyingopinions as “correct” or incorrect, rather they wereunderstood as a reflection of various professional know-ledge and expertise. The Canadian InterprofessionalHealth Collaborative established interprofessional com-munication and patient-centred care as foundationalcompetencies for interprofessional collaboration [83]. Flat-tened hierarchy [81] and flexibility [49] have also beendiscussed in the shared care literature, although not con-ceptualized within a model that facilitates the interprofes-sional collaborative relationship building process.Findings from our study make an initial contribution

to our understanding of the developing interprofessionalcollaborative relationship between health care providers.More research is needed to understand how the compo-nents of the interprofessional collaborative relationshipswithin a stage of development facilitate or impede teamdevelopment. Future research may also explore the ap-plication of this interprofessional collaborative relation-ship building model to other practice settings. Thiscollaborative relationship building model highlights co-location of providers; future research may explore virtualinterprofessional collaborative teams and the processesthey use to develop their relationships. Other limitationsof this study include the possibility that only providershaving positive interprofessional relationship buildingexperiences volunteered to participate in this study thus,limiting our understanding of the role of conflict andconflict resolution. Furthermore, in this study the patientvoice was represented by the health care providers andnot by the patient themselves. Future research on the in-terprofessional collaborative relationships should includeasking patients directly for their perspective [84].

ConclusionIncreasingly health care providers are asked to workcollaboratively with their colleagues from other

professions. However, little attention has been given tohow these professionals are to initiate and maintainthese interprofessional relationships. Providers partici-pating in CMHC programs within Canada, collaborateto successfully provide mental health services in primarycare settings. Exploring and documenting how theseproviders develop and maintain their interprofessionalcollaborative relationships contributes to our overallunderstanding of the importance of the provider-to-provider relationship. Recognizing that relationshipsdevelop in stages and require time for collaboration,may guide other health care providers to consider howthey can individually and collectively maintain a patient-focus and use communication strategies that are aimedat achieving greater reciprocity within their health careteam. Ultimately, understanding the characteristics ofeach developmental stage, the importance of co-location,patient-focus, and communication strategies and theneed to be flexible may position health care providersfrom a variety of professional backgrounds to success-fully navigate the journey of developing relationshipsthat may provide improved patient care.

Competing interestsNeither of the two authors have any competing interests to report.

Authors’ contributionsPW and RLW contributed to the conception and design of the study. PWwas responsible for all data collection. PW and RLW performed the dataanalysis and contributed to the drafting, reviewing, and approving thearticle. Both authors read and approved the final manuscript.

Author’ informationPW is an Associate Professor at the College of Rehabilitation Sciences,Department of Occupational Therapy, Faculty of Health Sciences at theUniversity of Manitoba. PW is also a doctoral student, Individual InterdisciplinaryStudies (Nursing, Occupational Therapy, Education), Faculty of Graduate Studiesat the University of Manitoba.RLW is a Professor at the College of Nursing, Faculty of Health Sciences atthe University of Manitoba. RLW holds a Canadian Institutes of Health ResearchApplied Chair in Reproductive, Child and Youth Health Services and PolicyResearch.

AcknowledgementsThe authors are grateful to all who shared their experiences with us.PW was supported by the Manitoba Health Research Council, DissertationAward, Canadian Occupational Therapy Foundation Doctoral Scholarship andthe Faculty of Medicine and College of Rehabilitation Sciences, University ofManitoba.RLW is supported by a Canadian Institutes of Health Research Applied Chairin Reproductive, Child and Youth Health Services and Policy Research(Grant#: CIHR APR −126339). The funding sources had no role in the conductof the study, analysis of data or decision for publication.

Author details1Department of Occupational Therapy, College of Rehabilitation Sciences,University of Manitoba, R125-771 McDermot Ave., Winnipeg, MB R3E 0T6,Canada. 2College of Nursing, University of Manitoba, 465 Helen Glass Centre,89 Curry Place, Winnipeg, MB R3T 2N2, Canada.

Received: 8 November 2015 Accepted: 2 March 2016

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 13 of 15

46

Page 47: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

References1. Watson DE, Heppner P, Roos NP, Reid RJ, Katz A. Population-based use of

mental health services and patterns of delivery among family physicians:1992–2001. Can J Psychiat. 2005;50(7):398–406.

2. Fleury MJ, Imboua A, Aubé D, Farand L, Lambert Y. General practitioners’management of mental disorders: a rewarding practice with considerableobstacles. BMC Fam Pract. 2012;13:19.

3. Macfarlane D. Current state of collaborative mental health care. Mississauga:Canadian Collaborative Mental Health Initiative; 2005.[http://www.shared-care/page.aspx?menu=69&app=266&cat1=744&tp=2&lk=no].

4. Lin E, Goering P. The utilization of physician services for mental health inOntario. Toronto: Institute for clinical evaluative sciences; 1999 [http://www.ices.on.ca/flip-publication/the-utilization-of-physician-services/index.html].

5. Mulvale G, Abelson J, Goering P. Mental health service delivery in Ontario,Canada: How do policy legacies shape prospects for reform? Health EconPolicy Law. 2007;2:363–89.

6. Rhodes A., Bethell J., and Schultz S. In Jaakkimainen L, Upshur REG, Klein-Geltink JE, Maaten S, Schultz SE, Leong A, Wang L. ‘Primary care in Ontario’,in ICES Atlas, Toronto: Institute for Clinical Evaluative Sciences, Chapter 9,2006. [http://www.ices.on.ca/Publications/Atlases-and-Reports/2006/Primary-care-in-Ontario]

7. Health Canada. Health care system delivery, 2005. [http://www.hc-sc.gc.ca/hcs-sss/delivery-prestation/index-eng.php]

8. Fickel J, Parker L, Yano E, Kirchner J. Primary care-mental healthcollaboration: An example of assessing usual practice and potential barriers.J Interprof Care. 2007;21(2):207–16.

9. Henke RM, Chou AF, Chanin JC, Zides AB, Scholle SH. Physician attitudetoward depression care interventions: Implications for implementation ofquality improvement initiatives. Implement Sci. 2008;3(1):40.

10. Anthony JS, Baik SY, Bowers BJ, Tidjani B, Jacobson CJ, Susman J. Conditionsthat influence a primary care clinician’s decision to refer patients fordepression care. Rehabil Nurs. 2010;35(3):113–22.

11. Benzer JK, Beehler S, Miller C, et al. Grounded theory of barriers andfacilitators to mandated implementation of mental health care in theprimary care setting. Depress Res Treat, 2012;597157. Article ID 597157, 11pages doi:10.1155/2012/597157.

12. Parikh SV, Segal ZV, Grigoriadis S, Ravindran AV, Kennedy SH, Lam RW, et al.Canadian Network for Mood and Anxiety Treatments (CANMAT) clinicalguidelines for the management of major depressive disorder in adults. II.Psychotherapy alone or in combination with antidepressant medication.J Affect Disord. 2009;117:S15–25.

13. Swenson RJ, Aubry T, Gillis K, MacPhee C, Busing N, Kates N, et al.Development and implementation of a collaborative mental health careprogram in a primary care seeing: The Ottawa Shared Program. Can JCommun Ment Health. 2008;27:75–91.

14. Craven MA, Bland R. Better practices in collaborative mental health care: Ananalysis of the evidence base. Can J Psychiat. 2006;51(6 Suppl 1):7S–72S.

15. Goossen R, Jones T, Botting I, Wener P, Staley D, Gibbens M. Shared mentalhealth care program: Evaluation report. Winnipeg: Winnipeg RegionalHealth Authority; 2012 [http://www.wrha.mb.ca/professionals/familyphysicians/files/SC_Eval_May5-2012.pdf].

16. Lam RW, Kennedy SH, Grigoriadis S, McIntyre RS, Milev R, Ramasubbu R, etal. Canadian Network for Mood and Anxiety Treatments (CANMAT) Clinicalguidelines for the management of major depressive disorder in adults.: III.Pharmacotherapy. J Affect Disord. 2009;117:S26–43.

17. Grenier J, Chomienne MH, Gaboury I, Ritchie P, Hogg W. Collaborationbetween family physicians and psychologists: What do family physiciansknow about psychologists’ work? Can Fam Physician. 2008;54:232–3.

18. Kates N, Fugere C, Farrar S. Family physician satisfaction with mental healthservices: Findings from a community survey. Can Psychiat Assoc B. 2004;36(2):10–4.

19. Craven M, Cohen M, Campbell D, Williams J, Kates N. Mental health practiceof Ontario family physicians: A study using qualitative methodology. Can JPsychiatry. 1997;42:943–9.

20. Kates N, Craven M, Bishop J, Clinton T, Kraftcheck D, LeClair K, et al. Sharedmental health care in Canada [position paper]. Can J Psychiat. 1997;42(8):1–12.

21. Craven MA, Handfield-Jones R. Shared mental health care in Canada.Making a joint effort to define our roles. Can Fam Physician. 1997;43:1785.

22. Kates N, Ackerman S, editors. Shared mental health care in Canada: acompendium of current projects. Ottawa: Canadian Psychiatric Associationand College of Family Physicians of Canada; 2002.

23. Kates N. Significant achievement award – Bringing mental health servicesinto the offices of primary care physicians. Psychiatr Serv. 1999;50(11):1484–5.

24. Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care fordepression in primary care: Making sense of a complex intervention:systematic review and meta-regression. Br J Psychiatry. 2006;189:484–93.

25. Brown JB, Lent B, Stirling A, Takhar J, Bishop J. Caring for seriously mentallyill patients: Qualitative study of family physician’s experiences. Can FamPhysician. 2002;48:915–20.

26. Haggarty J, Jarva J, Cernovsky Z, Karioja K, Martin L. Wait time impact ofco-located primary care mental health services: The effect of addingcollaborative care in northern Ontario. Can J Psychiat. 2012;57(1):29–33.

27. Kates N, Crustolo AM, Farrar S, Nikolaou L. Counsellors in primary care:Benefits and lessons learned. Can J Psychiat. 2002;47(9):857–62.

28. McElheran W, Eaton P, Rupcich C, Basinger M, Johnston D. Shared mentalhealth care: The Calgary model. Fam Syst Health. 2004;22(4):424–38.

29. Sedgwick W, Washburn C, Newton C, Mirwaldt P. Shared care depressioncollaborative model: From project inception to outcome data. Can JCommun Ment Health. 2009;27(2):219–32.

30. van Orden ML, Deen ML, Spinhoven P, Haffmans J, Hoencamp E. Five-YearMental Health Care Use by Patients Referred to Collaborative Care or toSpecialized Care. Psychiatr Serv. 2015;66(8):840–4.

31. Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, BauerMS. Comparative effectiveness of collaborative chronic care models formental health conditions across primary, specialty, and behavioral healthcare settings: systematic review and meta-analysis. Am J Psychiatry. 2012;169(8):790–804. doi:10.1176/appi.ajp.2012.11111616.

32. Goossen R, Staley JD, Pearson M. Does the introduction of shared caretherapists in primary health care impact clients’ mental health symptomsand functioning? Can J Commun Ment Health. 2008;27:37–46.

33. Haggarty J, Klein R, Chaudhuri B, Boudreau D, McKinnon T. After SharedCare: Patient’s Symptoms and Functioning 3 to 6 Months Following Care AtA Rural Shared Mental Health Care Clinic. Can J Commun Ment Health.2008;27:47–54.

34. Farrar S, Kates N, Crustolo A, Nikolaou L. Integrated model for mental healthcare: Are health care providers satisfied with it? Can Fam Physician. 2001;47:2483–8.

35. Haworth H, Powell S, Burley J, Bell, P. Collaboration between communitymental health workers or counsellors and psychiatrists in a shared caresetting. Can Psychiat Assoc B 2004; (April). [https://ww1.cpa-apc.org/Publications/Archives/Bulletin/2004/april/burley-haworth-SF.asp].

36. Kates N, Lesser A, Dawson D, Devine J, Wakefield J. Psychiatry and familymedicine: the McMaster approach. Can J Psychiat. 1987;32(3):170–4.

37. Kates N, Craven MA, Crustolo AM, Nikolaou L, Allen C, Farrar S. Sharing care:The psychiatrist in the family physician’s office. Can J Psychiat. 1997;42(9):960–5.

38. Kates N, Craven M, Atkinson M, el-Guebaly N. How psychiatrists view theirrelationships with family physicians. Can Psychiat Assoc B. 2001;33(1):9–12.

39. Kates N, McPherson-Doe C, George L. Integrating mental health serviceswithin primary care settings: The Hamilton Family Health Team. J AmbulCare Manage. 2011;34(2):174–82.

40. Kates N, Crustolo AM, Farrar S, Nikolaou L. Integrating mental health servicesinto primary care: Lessons learnt. Fam Syst Health. 2001;19(1):5–12.

41. Mulvale G, Danner U, Pasic D. Advancing community-based collaborativemental health care through interdisciplinary family health teams in Ontario.Can J Comm Ment Health. 2008;27:55–73.

42. Chang E, Magnabosco J, Chaney E, Lanto A, Simon B, Yano E, Rubenstein L.Predictors of primary care management of depression in the veterans affairshealthcare system. J Gen Intern Med 2014;doi: 10.1007/s11606-014-2807-z.

43. Kisely S, Duerden D, Shaddick S, Jayabarathan A. Collaboration betweenprimary care and psychiatric services: Does it help family physicians? CanFam Physician. 2006;52(7):876–7.

44. World Health Organization, World Organization of Family Doctors.Integrating mental health into primary care: a global perspective. 2008[http://www.who.int/mental_health/resources/mentalhealth_PHC_2008.pdf].

45. Fitzpatrick NK, Shah S, Walker N, Nourmand S, Tyrer PJ, Barnes TR, et al. Thedeterminants and effect of shared care on patient outcomes and psychiatricadmissions. Soc Psych Psych Epid. 2004;39(2):154–63.

46. Doey T, Hines P, Myslik B, Leavey JE, Seabrook JA. Creating primary careaccess for mental health care clients in a community mental health setting.Can J Comm Ment Health. 2009;27(2):129–38.

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 14 of 15

47

Page 48: WD07 Moving beyond numbers: Applying qualitative ... Library/SGIM/Resource Library/Meeting Handouts...WD07 Moving beyond numbers: Applying qualitative methodology to expand the scope

47. Smith SM, Allwright S, O’Dowd T. Effectiveness of shared care across theinterface between primary and specialty care in chronic diseasemanagement. Cochrane Database of Systematic Reviews, 2007, Issue 3. Art.No.: CD004910. DOI: 10.1002/14651858.CD004910.pub2.

48. Kates N, Mazowita G, Lemire F, Jayabarathan A, Bland R, Selby P, et al. Theevolution of collaborative mental health care in Canada: A shared vision forthe future. Can J Psychiatry. 2011;56(5):I1.

49. Paquette-Warren J, Vingilis E, Greenslade J, Newnam S. What dopractitioners think? A qualitative study of a shared care mental health andnutrition primary care program. Int J Integr Care. 2006;6(9):e18.

50. Way D, Jones L, Baskerville B, Busing N. Primary health care servicesprovided by nurse practitioners and family physicians in shared practice.Can Med Assoc J. 2001;165(9):1210–4.

51. Younes N, Gasquet I, Gauderbout P, Chaillet MP, Kovess V, Falissard B, Bayle,MC. General practitioner’s opinions on their practice in mental health andtheir collaborations with mental health professionals. BMC Fam Pract 2005;6(18), doi:10.1186/1471-2296-6-18.

52. Craven M, Bland MB. Shared Mental Health Care: The need for increasedcollaboration between psychiatry and primary care. The Can J Psychiatry.2002;47 Suppl 1:1S–11S.

53. Creswell JW. Qualitative inquiry & research design: Choosing among fiveapproaches. 2nd ed. Thousand Oaks: Sage Publications Ltd; 2007.

54. Crotty M. The foundations of social research:meaning and perspective inthe research process. Thousand Oaks: Sage Publications Inc.; 1998.

55. Charmaz K. Constructing grounded theory. 2nd ed. Thousand Oaks: SagePublications Ltd; 2014.

56. Prus R. Approaching the study of human group life: Symbolic interactionand ethnographic inquiry. In: Dietz ML, Prus R, Shaffir W, editors. Doingeveryday life: Ethnography as human lived experience. Toronto: Copp,Clarke, Longman; 1994. p. 10–29.

57. Wener P, Woodgate RL. Use of a qualitative methodological scaffoldingprocess to design robust interprofessional studies. J Interprof Care. 2013;27(4):305–12.

58. Creswell JW. Educational research: Planning, conducting, and evaluatingquantitative and qualitative research. Upper Saddle River: PearsonEducation, Inc.; 2005.

59. Williams P, Domnick K, Vayda E. Women in medicine: towards a conceptualunderstanding of the potential for change. In: Coburn D, D’Arcy C, TorranceG, editors. Health and Canadian Society: Sociological Perspective. 3rd ed.Toronto: University of Toronto Press Inc; 1998.

60. Fewster-Thuente L, Velsor-Friedrich B. Interdisciplinary collaboration forhealthcare professionals. Nurs Admin Q. 2008;32(1):40–8.

61. Dey I. Grounding grounded theory: Guidelines for qualitative inquiry. SanDiego: Academic; 1997.

62. Charmaz K. Constructing grounded theory: A practical guide throughqualitative analysis. Thousand Oaks: Sage Publications Ltd.; 2006.

63. Shenton AK. Strategies for ensuring trustworthiness in qualitative researchprojects. Educ Inf. 2004;22(2):63–75.

64. Speziale HJ, Carpenter DR. Qualitative research in nursing: Advancing thehumanistic perspective. 4th ed. Philadelphia: Lippincott Williams andWilkins; 2007.

65. Knowles S, Chew-Graham C, Coupe N, Adeyemi I, Keyworth C, Thampy H, etal. Better together? A naturalistic qualitative study of inter-professionalworking in collaborative care for co-morbid depression and physical healthproblems. Implement Sci. 2013;8:110.

66. Allport GW. The nature of prejudice. Cambridge: Perseus Books; 1954.67. Hewstone ME, Brown RE. Contact and conflict in intergroup encounters.

Oxford: Basil Blackwell; 1986.68. Chidambaram L, Bostrom RP. Group development (1): A review and

synthesis of development models. Group Negot. 1996;6:159–87.69. Heinemann GD. Teams in health care settings. In: Heinemann GD, Zeiss AM,

editors. Team Performance in Health Care: Assessment and Development.New York: Kluwer Academic/Plenum Publishers; 2002.

70. Tuckman BW. Developmental sequence in small groups. Psychol B. 1965;63(6):384.

71. Tuckman BW, Jensen MA. Stages of small-group development revisited.Group Organ Manag. 1977;2(4):419–27.

72. Farrell MP, Schmitt MH, Heinemann GD. Informal roles and the stages ofinterdisciplinary team development. J Interprof Care. 2001;15(3):281–95.

73. Drinka T, Clark P. Health care teamwork: Interdisciplinary practice andteaching. Westport: Auburn House, Greenwood Publishing Inc.; 2000.

74. Hammick M, Olckers L, Campion-Smith C. Learning in interprofessionalteams: AMEE Guide no. 38. Med Teach. 2009;31(1):1–12.

75. Martin-Rodriguez LS, Beaulieu MD, D’Amour D, Ferrada-Videla M.The determinants of successful collaboration: a review of theoretical andempirical studies. J Interprof Care. 2005;19 suppl 1:132–47.

76. D’Amour D, Oandasan I. Interprofessionality: An emerging concept.J Interprof Care. 2005;19 suppl 1:8–20.

77. Evans CR, Dion KL. Group cohesion and performance: a meta-analysis. SmallGroup Res. 1991;22(2):175–86.

78. Gully SM, Devine DJ, Whitney DJ. A Meta-analysis of Cohesion andPerformance: Effects of Level of Analysis and Task Interdependence. SmallGroup Res. 1995;25(4):497–520.

79. Clement D, Dault M, Priest A. Effective teamwork in healthcare: research andreality. Healthc Pap. 2007;7(SP):26–34.

80. Lucena RJ, Lesage A. Family physicians and psychiatrists. Qualitative study ofphysicians’ views on collaboration. Can Fam Physician. 2002;48(5):923–9.

81. Mulvale G, Bourgeault IL. Finding the Right Mix: How Do Contextual FactorsAffect Collaborative Mental Health Care in Ontario? Can Public Policy.2007;33:S49–64.

82. Brown J, Lewis L, Ellis K, Stewart M, Freeman TR, Kasperski MJ. Conflict oninterprofessional primary health care teams – can it be resolved? J InterprofCare. 2011;25:4–10. doi:10.3109/13561820.2010.497750.

83. The Canadian Interprofessional Health Collaborative. A nationalinterprofessional competency framework, 2010. http://www.cihc.ca/files/CIHC_IPCompetencies_Feb1210.pdf

84. Kates N, Gagne MA, Whyte JM. Collaborative mental health care in Canada:Looking back and looking ahead. Can J Comm Ment Health. 2008;27(2):1–4.

• We accept pre-submission inquiries

• Our selector tool helps you to find the most relevant journal

• We provide round the clock customer support

• Convenient online submission

• Thorough peer review

• Inclusion in PubMed and all major indexing services

• Maximum visibility for your research

Submit your manuscript atwww.biomedcentral.com/submit

Submit your next manuscript to BioMed Central and we will help you at every step:

Wener and Woodgate BMC Family Practice (2016) 17:30 Page 15 of 15

48