Embed Size (px)
Citation preview
Rhetorical Moves and Audience
Considerations in the Discussion
Sections of Randomized Controlled Trials
of Health InterventionsJodi Schneider (University of Illinois at Urbana Champaign, USA)
Graciela Rosemblat (National Library of Medicine, USA)
Shabnam Tafreshi (The George Washington University, USA)
Halil Kilicoglu (National Library of Medicine, USA)
European Conference on Argumentation, Fribourg, Switzerland
2017-06-22
Disclaimer
The views and opinions expressed do not
necessarily state or reflect those of the U.S.
Government, and they may not be used for
advertising or product endorsement
purposes.
Why study scientific papers?
• “The interpretive face of science is
characterized by controversy,
disagreement, dissensus.” (Prelli, “A
Rhetoric of Science”, 1989, p4)
• “Scientific publications are documentary
representations of defeasible arguments,
supported by data and repeatable
methods.” (Clark, Ciccarese, Goble 2014)
• Prelli, Lawrence J. "A rhetoric of science: Inventing scientific discourse." (1991).• Clark, Tim, Paolo N. Ciccarese, and Carole A. Goble. "Micropublications: a
semantic model for claims, evidence, arguments and annotations in biomedical communications." Journal of biomedical semantics 5.1 (2014): 28.
Why study medical papers?
• Large potential impact on human health
• Input to evidence-based practice
• Well-elaborated study of evidence
Figure credit: Duke University Medical Center Library. Introduction to
Evidence-based Practice. What is Evidence-Based Practice (EBP)? http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036021
Hierachy of Evidence
Hierachy of Evidence
Randomized Controlled Trial
(RCT)
• Medical RCTs test the efficacy of medical treatments.
• Features:
– Control group (placebo or standard treatment)
– Random assignment
– Neither doctors nor patients know which group is which (“double blind”)
Image credit: Ancker, Jessica S & Quynh Pham. "Beyond the RCT
Evaluating innovations in the Learning Health System” AMIA 2016 Tutorial
Corpus
• 37 research articles reporting on medical
trials, mainly RCTs
• Full-text available from PubMed Central
• Chosen for an ongoing citation annotation
study using Xu et al.’s corpus
Xu, Jun, Yaoyun Zhang, Yonghui Wu, Jingqi Wang, Xiao Dong, and Hua Xu. "Citation sentiment analysis in clinical trial papers." In AMIA Annual Symposium Proceedings, vol. 2015, p. 1334. American Medical Informatics Association, 2015.
Other ongoing study on this corpus
• Understand citation structure of Discussion sections of Randomized Clinical Trials
• Annotate two aspects of citations:
– Citation scope
– Citation matter
• Began summer 2016 at U.S. National Library of Medicine, led by Halil Kilicoglu, with Graciela Rosemblat leading annotation.
• Ongoing study in biomedical informatics
Corpus characteristics
• Various conditions, e.g.– Brain injury
– Degenerative eye disease
– Malaria
– Postpartum back pain
• Various settings, e.g. – Critical care
– Community/Ambulatory care
– Internet-based treatment
– Laboratory treatment
– School-based treatment
Corpus characteristics
• Various locales, e.g. – Madagascar
– Malawi
– Netherlands
– Uganda
– United Kingdom
– United States
• Single site and Multisite
• Industry-sponsored and Researcher-focused
• Various stages of clinical trial
Focus: 37 discussion sections
http://europepmc.org/articles/PMC1420299
Rhetorical Moves in RCT
Discussion Sections
Rhetorical Moves in RCT
Discussion Sections• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries
Novelty
• “We report the first safety data collected in the setting of a randomized, controlled study for patients sustaining TBI with polytrauma.” (2206502)
• “This is the first double blind prospective study designed to evaluate the effects of fruit juice feedings during diarrheal disease in young children.” (1217327)
Gaps/opportunities
• "At the start of this study, little was known about the specific effect of psychosocial determinants in SCs, although interventions aiming to modify these determinants, such as cognitive behavioural programs were considered to be promising for musculoskeletal pain in general [4-8]. We developed the EAP to fill a gap in UC, which focuses mainly on biomedical determinants." (2211478)
• "We have already studied the stimulation of periodontal tissue regeneration by FGF-2 in animal models and believe that the protein represents a major candidate for a periodontal tissue-regenerating agent." (2432040)
Contextualization
• “This study is part of a series to explore effective xylitol delivery vehicles that can be used in school programs in the U.S. Results from the xylitol dose study [3] showed...” (2527560)
• “In general we did not deviate from the original study design, which was described independently of the study results [15].” (1420299)
• "The first report on this study showed that the intra-operative use of TA significantly decreased the incidence of intra-operative complications associated with PPV, and no adverse events related to surgery were found over a 3-month observation period [13]." (2413124)
Comparability to previous results
• "However, none of these 11 studies were performed on pediatric populations, making further inferences or comparisons problematic" (1876598)
• "This is in line with previous studies of internet-driven CBT for posttraumatic stress reactions [9], complicated grief [30] and CBT interventions in face-to-face studies [10,31]." (1885249)
Validity of methods
"Algometry is used to measure the sensitivity of pain or pressure [4].
Algometer instrumentation can include manual and electric models.
The use of any experimental instrument including the PGA must
be tested for validity and reliability between examiners and
between performances of the same examiner. The PGA used in
this study has been tested against itself, palpation, pressure
plates with reliable results [17,18]. However, the above is based on
the assumption that the tester is trained in the application of the PGA
otherwise issues associated with the rate of pressure application [45-
47] the determination of an end point based on a verbal patient
response [45,46] and the possible sensitisation of a selected landmark
based on repeated measures may all alter the accuracy of the
outcome." (2427032)
Validity of methods
"The technique of using the consumption of morphine during PCA treatment of postoperative pain, as a measure of the effect of the analgesic regime under study, has been used in several other studies of this kind [5,6]." (1637100)
Anticipating objections
"It might be argued that these observations of blood stage immunity (reduction in growth rates, presence of crisis forms etc.) may rather be related to the potential pre-erythrocytic action of PEV3A. However, in previous challenge studies, the number of parasites emerging from the liver of unvaccinated control volunteers has been shown to vary as much as five fold [7]. Despite this variation, rates of parasite growth in these volunteers were similar. Equally, crisis forms have never been observed historically in our studies of pre-erythrocytic vaccine candidates. Indeed, following their observation in this study, the same slide reader went back to examine a selection of blood films from vaccinated and control volunteers in two previous studies where some evidence of pre-erythrocytic efficacy has been observed (VAC021 [28] and VAC023 [29]). The slide reader was blinded to the group allocation of the volunteers. In total 72 slides were selected for re-examination (6 slides each from 6 volunteers from each study, 3 vaccinees plus 3 controls) and
no crisis forms were observed. It seems likely therefore, that the differences observed here are indeed related to vaccine induced blood stage immunity." (2204057)
Pointers to ongoing work
• "An ongoing phase III study is likely to provide a better evaluation of safety in this patient population." (2206502)
• Further analysis of the 18 months follow-up data and the examination of other potentially relevant moderators such as posttraumatic growth [Maercker & Knaevelsrud, in preparation] will hopefully enhance our understanding of online therapeutic processes." (188529)
Acknowledging limitations
• "Among the limitations of this study is the screening strategy for the recruitment of the patients. We deliberately handled strict exclusion criteria for participation in this study. We excluded 72% (n = 253) of the patients who wanted treatment but did not meet the inclusion criteria. This might limit the generalizability of our results.” (1885249)
Future work
• "New studies should be carried out, with longer follow-up times and larger samples, to evaluate the effects of NPPV on the quality of life of patients on weaning ventilation support and to understand how the cause of ARF could affect the results of different weaning ventilation
methods. Cost evaluation should also be included in these studies." (2447605)
• "Future studies directed on possible central mechanisms of this complicated chronic pain syndrome are warranted."(1815612)
Summaries
• "Based on present evidence BTX cannot be recommended as treatment for neck pain in chronic whiplash patients."(1815612)
• "The evidence from this study is that Hall PMCs allow a reliable, low-maintenance seal to be achieved by GDPs." (2265270)
• "Therefore, the mouthwash clearly demonstrated an anti-malodor effect on morning breath potential without any measurable side effects in healthy subjects." (2637235)
Rhetorical Moves in RCT
Discussion Sections• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries
Audience
Multiple audiences
“Audiences of specialists…do not comprise
the only audiences who respond to scientific
or other professional discourse. There may
be a number of interested audiences for any
particular discourse.” (Prelli,1989, p26)
Interested audiences include:
• Specialists responsible for “authorizing the
claims to knowledge” (Prelli,1989, p26)
• Non-specialists who might “judge the work
as interested witnesses” especially if there
are “broad social implications” (Prelli,1989,
p27)
Fulfilling audience expectations
“To succeed rhetorically, any rhetor must ground a
position in what the addressed audience considers
reasonable”(Prelli,1989, p27-8)
“…[draw] on standard or conventional patterns of
thinking that will be employed by one’s specific
audience”(Prelli,1989, p37)
“…choose purposes and contents likely to be
deemed pertinent and conventionally appropriate
by the specifically situated thought groups that
comprise the audience.”(Prelli,1989, p38)
Audience-specific sections from
discussion sections of reports of
Randomized Controlled Trials
Researchers are the expected
audience for RCTs
Mellis, Craig. "Evidence-based Medicine: What Has Happened in the Past 50 Years?" J Paediatr Child Health 51, no. 1 (2015): doi:10.1111/jpc.12800.
Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• Health systems
Specialist researchers in the same
area
• “The most plausible explanation for the
similar trends in BV reduction in both arms
of this study is the inherent activity of the
placebo formulation.” (1851729)
Clinicians
• “caution should be exercised in applying these findings
to the general management of dental caries without
being aware of potential pit falls.”(2265270)
• “none of the results of this particular clinical trial suggest
any clinical problems concerning the safety of
administering FGF-2 to patients with periodontitis. One
more piece of evidence supporting the high safety of
FGF-2 applied locally to periodontal tissue is that this
therapy has already been used for more than 5 years as
a remedy for intractable ulcers (Fiblast spray).”
(2432040)
Clinical practice managers
• “Our findings are particularly pertinent
to smaller clinical practices that might
not be aligned with larger organizations,
and who will have to choose EMR systems
and DSSs on their own.”(1876598)
Grant funders
• “Additional research is needed to determine whether these improvements translate into even longer-term reductions of health-related costs. Although the sample was heterogeneous on demographic factors such as income, the group was relatively ethnically and racially homogenous, consisting primarily of white women. Future replications with diverse populations are indicated to further evaluate the promise and translation potential of the MLP.” (1783667)
Grant funders
"Admittedly, overall complexity of the issue addressed by the present study clearly merits pursuit of further studies of a closely similar, multi-factorial intervention design, though preferably conducted over much longer time span, as well as best targeting the over 60s as the population standing by far the best chance of a successful outcome, in order to verify whether the present findings, clearly encouraging as they appear, might actually have sufficient potential to develop into a trend of indisputable clinical significance, especially in terms of possible application in a comprehensively designed, nationwide programme specifically aimed at addressing the problem of a steadily growing proportion of the frail seniors dependent for their activities of daily living." (2637855)
Health systems
• “Given the rising costs and growing burden of diabetes on the health-care system [57], strategies that aim to prevent or delay the onset of this disease might be beneficial [58]” (1865086)
• “AQ+AS (0.51 US$) [28] is currently four times more expensive than AQ+SP (0.13 US$). In Madagascar, with a reported 2,114,400 cases of suspected malaria (2003), the use of AQ+SP instead of AQ+AS would reduce the annual antimalarial treatments costs by 800,000 US$. In view of the recent report by Tagbor and colleagues [29] there is hope that AQ+SP could serve as a safe and effective alternative for malaria treatment in pregnancy and for intermittent preventive treatment until the safety of ACTs for pregnant women has been established.” (1887535)
Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• Health systems
Rhetorical Moves in RCT
Discussion Sections• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries
Future work
• Systematically code rhetorical moves and audiences – likely to lead to revisions
• Compare to previous work in English for Specific Purposes for medical and general scientific work
• Identify key suggestions for writing effective RCT reports, to reach their intended audiences.
• Develop Argumentation Mining algorithms to automatically detect rhetorical moves.– Could be used to automatically extract discussion
topics for evidence synthesis of RCTs.
– Could be used to give authors fast, automatic feedback about readability.
Citation examples from
discussion sections of reports of
Randomized Controlled Trials
“This” work agrees with…
• “This is in accordance with earlier studies
in the ambulatory surgical setting [3]” -
PMC1637100
Definitions and background info
• “Self-efficacy, which may relate to
motivation, is the perceived confidence in
one's ability to accomplish a specific task
[19].” - PMC2194735
Presenting a range of evidence
• “Except in one study [20], short-term
administration of GH transiently worsened
insulin resistance [19,53] and increased
fasting glucose levels [53].” - PMC1865086
Clause-level changes in meaning
• “Two of four randomised clinical trials
…have found a difference in admission
rate [12,19] and two have not [22,23].” -
PMC1142326
A single citation can support a
whole paragraph• Dutton and colleagues [8] described a series
of 81 coagulopathic trauma patients treated with rFVIIa. Of these, 20 received rFVIIa for treatment of coagulopathy related to TBI. Six of these patients had additional polytrauma. The outcome of these patients was poor and 15 of 20 patients died. The authors attributed this high mortality rate to the severity of brain injury. None of the 81 trauma patients in this series had any clinical indication of TE events.”
Discussing treatments, outcomes,
other authors’ conclusions• Dutton and colleagues [8] described a series
of 81 coagulopathic trauma patients treated with rFVIIa. Of these, 20 received rFVIIa for treatment of coagulopathy related to TBI. Six of these patients had additional polytrauma. The outcome of these patients was poor and 15 of 20 patients died. The authors attributed this high mortality rate to the severity of brain injury. None of the 81 trauma patients in this series had any clinical indication of TE events.”
Sometimes several parallel
paragraphs.
• Dutton and colleagues [8] described a series of 81 …patients treated with rFVIIa”
• “Zaaroor and Bar-Lavie [23] reported the first series of five patients …”
• “Morenski and colleagues [24] described …three pediatric … cases”
Multiple citations in a paragraph
• “Berger et al. [42] compared the efficacy of hypertonic saline and mannitol to reduce ICP after a combination of two different neuronal injuries. Initially, ….The authors demonstrated that …After …. It is remarkable that … An accumulation …These different effects … [42]. Furthermore, Prough et al. observed a higher regional cerebral blood flow in dogs with induced intracerebral hemorrhage after hypertonic saline without any increase of the CPP [43].” - PMC1297608
Avoiding a 1-sentence paragraph?
• “Berger et al. [42] compared the efficacy of hypertonic saline and mannitol to reduce ICP after a combination of two different neuronal injuries. Initially, ….The authors demonstrated that …After …. It is remarkable that … An accumulation …These different effects … [42]. Furthermore, Prough et al. observed a higher regional cerebral blood flow in dogs with induced intracerebral hemorrhage after hypertonic saline without any increase of the CPP [43].” - PMC1297608
