A clinical case definition of post COVID-19 condition by a Delphi consensus

6 October 2021

A clinical case definition of post COVID-19 condition by a Delphi consensus

6 October 2021

WHO continues to monitor the situation closely for any changes that may affect this document. Should any factors change, WHO will issue a further update. Otherwise, this document will expire 2 years after the date of publication.

© World Health Organization 2021. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO licence.

WHO reference number: WHO/2019-nCoV/Post_COVID-19_condition/Clinical_case_definition/2021.1

-iii-

Contents Acknowledgements ..................................................................................................................................... iv

Abbreviations ................................................................................................................................................. v

Abstract .......................................................................................................................................................... vi

1. Research in context .............................................................................................................................. 1

1.1 Evidence before this study .................................................................................................................. 1

1.2 Added value of this study .................................................................................................................... 1

1.3 Implications of all the available evidence ......................................................................................... 1

2. Introduction ............................................................................................................................................. 2

3. Methods .................................................................................................................................................... 2

3.1 Study design .......................................................................................................................................... 2

3.2 Study participants ................................................................................................................................. 2

3.3 Study procedures ................................................................................................................................. 2

4. Statistical plan ........................................................................................................................................ 3

4.1 Sample size and power ....................................................................................................................... 3

4.2 Primary and secondary endpoints ..................................................................................................... 3

5. Results ...................................................................................................................................................... 4

6. Discussion ............................................................................................................................................... 4

6.1 Strengths and limitations ..................................................................................................................... 4

6.2 Future implications ............................................................................................................................... 5

7. Conclusion .............................................................................................................................................. 5

Tables and figures ........................................................................................................................................ 6

Fig. 1. Agreement rules and thresholds for consensus and disagreement ................................................ 6 Fig. 2. STROBE flowchart of participation in the two Delphi rounds .......................................................... 7 Fig. 3. World map distribution of participants ............................................................................................ 8 Table 1. Demographic characteristics of participants ................................................................................ 9 Table 2. Domains that achieved consensus by participants in each Delphi stage ................................... 10 Table 3. A definition of post COVID-19 condition .................................................................................... 11

Annex 1. Repository of published/available definitions of post COVID-19 condition ............. 12

Annex 2. Score for every domain/value in Rounds 1 and 2 ............................................................ 13

References .................................................................................................................................................... 19

-iv-

Acknowledgements

World Health Organization (WHO) clinical case definition working group on post COVID-19 condition: Joan B Soriano (Senior Consultant), Maya Allan, Carine Alsokhn, Nisreen A Alwan, Lisa Askie, Hannah E Davis, Janet V Diaz, Tarun Dua, Wouter de Groote, Robert Jakob, Marta Lado, John Marshall, Srin Murthy, Jacobus Preller, Pryanka Relan, Nicoline Schiess, Archana Seahwag.

We thank all participants, and particularly the patients and patient-researchers with post COVID-19 condition who contributed their time and expertise to this Delphi exercise. We also thank Professor Paula Williamson, at the University of Liverpool, Liverpool, United Kingdom of Great Britain and Northern Ireland, for providing free access to the DelphiManager software and Bridget Griffith for her technical support in organizing data from DelphiManager. Professor Joan B Soriano was a Senior Consultant at the COVID-19 Clinical Management Team, WHO Health Emergency Programme, World Health Organization, Geneva, Switzerland from November 2020 to June 2021.

Contributions: Joan B Soriano, Janet V Diaz, John Marshall, Srin Murthy and Pryanka Relan wrote the research protocol; Joan B Soriano and John Marshall wrote the first draft of the manuscript; Pryanka Relan performed the data analysis; all authors contributed to the writing and approved the final version.

Conflicts of interest: The authors declare there are no conflicts of interest in relationship with this manuscript.

Funding: This study was funded internally by WHO. There were no payments to participants.

-v-

Abbreviations

AIDS acquired immune deficiency syndrome

CDC Centers for Disease Control and Prevention

COVID-19 coronavirus disease 2019

HIV human immunodeficiency virus

ICD International Classification of Diseases

NICE National Institute for Health and Care Excellence

PICS post-intensive care syndrome

SEIS systemic exercise intolerance syndrome

STROBE Strengthening the Reporting of Observational studies in Epidemiology

WHO World Health Organization

-vi-

Abstract

Background: A proportion of those infected with SARS-CoV-2 experience long-term symptoms. Definitions of this emerging condition vary, leading to complexities in advancing research and clinical policy development. Over the course of the pandemic, various terminology including long COVID, long-haul COVID or the WHO-recommended post COVID-19 condition have been proposed. Still, a globally standardized clinical case definition of this condition remains lacking.

Aim: We aimed to determine the most important domains and variables for inclusion into a globally relevant and standardized clinical case definition for post COVID-19 condition.

Methods: We conducted a two-round Delphi exercise, followed by a mixed, iterative consensus process. Five groups of stakeholders were engaged: patients, patient-researchers, external experts, WHO staff and others. Participants were chosen for balanced representation across age, gender, specialty, area of expertise and geography. Pre-defined statistical thresholds for consensus and disagreement were established.

Results: There were 265 participants in Round 1, with 241 complete responses and 24 incomplete responses. In Round 2 there were 195 participants, with 178 complete responses and 17 incomplete responses. From an initial list of 14 domains identified, 11 were selected in Round 1, and one was added in Round 2 for a final total of 12. Each domain consisted of multiple questions and a total of 45 items were asked in the survey. A clinical case definition was developed with those domains that reached the pre-defined thresholds and further expanded with values that reached borderline significance. Wording was trimmed in an iterative process with patients and patient-researchers.

Conclusion: Through a large global consensus process, a working clinical case definition of post COVID-19 condition, including 12 domains, is now available for use in all settings. This definition may change as new evidence emerges and our understanding of the consequences of COVID-19 continues to evolve.

-1-

1. Research in context

1.1 Evidence before this study

Most patients who suffer from coronavirus disease 2019 (COVID-19) fully recover, but some remain with long-term effects on several body systems, including pulmonary, cardiovascular and nervous systems, as well as psychological effects. These effects appear to occur irrespective of the initial severity of infection, but occur more frequently in women, middle age, and in those with more symptoms initially/ The absence of both a single terminology and a clinical case definition have been repeatedly signalled as drawbacks to advance on research and management of these patients. 1.2 Added value of this study

By means of a Delphi methodology, and in two rounds, this study identified the domains and variables to be included in a clinical case definition of post COVID-19 condition, which is the name proposed by the WHO International Classification of Diseases (ICD) – ICD-10 U09. They were scored by patients, clinicians, researchers and others, representing all WHO regions. A clinical case definition was built, and it was further expanded with those domains that reached the pre-defined thresholds and values that reached borderline significance. Wording was trimmed in an iterative process with a small group of patients and patient-researchers. Those involved in assessing the Delphi findings submitted conflict of interest forms that were reviewed and managed by the WHO technical unit. No conflicts of interest were identified. A definition with 12 domains and 88 words was identified.

Post COVID-19 condition occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, cognitive dysfunction but also others (see Table 3 and Annex 2) which generally have an impact on everyday functioning. Symptoms may be new onset, following initial recovery from an acute COVID-19 episode, or persist from the initial illness. Symptoms may also fluctuate or relapse over time. A separate definition may be applicable for children.

1.3 Implications of all the available evidence A clinical case definition of post COVID-19 condition based on existing evidence objectively assessed with a robust methodology and pre-determined statistical thresholds is presented. This definition is a first, necessary, step to optimize the recognition and care of persons experiencing post COVID-19 condition in community and health care settings, while awaiting outcomes from ongoing research exploring the use of electronic health records of representative samples of patients identified in primary care linked with hospital care, with comparator groups of individuals fully recovered after acute infection. Given the many unknowns surrounding the natural history and recovery/sequelae from COVID-19, further discussion and agreement of a definition should help future research and management of these patients.

-2-

2. Introduction As of August 2021, over 210 million confirmed cases of COVID-19 and over 4.4 million deaths have been reported to WHO (1), although estimates greatly surpass these figures (2). However, the natural history, clinical course and consequences of this new disease are still not completely understood (3). Most patients with COVID-19 return to baseline after acute infection with SARS-CoV-2, but a proportion report ongoing health issues. How many people are affected with longer term sequelae after acute COVID-19 remains unknown, but published reports indicate that approximately 10–20% of COVID-19 patients experience lingering symptoms for weeks to months following acute SARS-CoV-2 infection (4). Several organizations and societies have proposed definitions based upon the constellation of symptoms that affect people after acute SARS-CoV-2 infection (Annex 1). In September 2020, and in response to requests from Member States, the Classification and Terminologies unit at the WHO created ICD-10 and ICD-11 codes for “post COVID-19 condition” (5). However, standardization of this nomenclature and clinical case definition of post COVID-19 condition is still required to facilitate global discussion and streamline research methodologies, management strategies and policies. The objective of this study is to determine the domains and variables for inclusion into a standardized clinical case definition for post COVID-19 condition.

3. Methods 3.1 Study design The research protocol is available as pre-print elsewhere (6). This study is a prospective Delphi consensus-seeking exercise, and mixed iterative survey of internal and external experts, patients and other stakeholders. The Delphi method is a structured communication technique originally developed as a systematic, interactive forecasting method which relies on a panel of experts (7,8). Delphi has been widely used for research and has certain advantages over other structured forecasting approaches (9,10). 3.2 Study participants The primary users of a clinical case definition for the post COVID-19 condition will include patients, relatives and caregivers, clinicians, researchers, advocacy groups, policy-makers, health and disability insurance and media. We therefore aimed to have a diverse representation of participants including clinicians with expertise in a variety of specializations, quality improvement and/or research, patients who have suffered from COVID-19 and its mid- and longer term effects, researchers, policy-makers and others from countries representing all WHO regions and World Bank income levels. There were no specific exclusion criteria for participants. A statement explaining implied consent was on the title page of the survey, with consent to participate in the survey implied by answering and returning the surveys. 3.3 Study procedures Participants were identified from the list of clinician and patient researchers that attended a previous WHO webinar on post COVID-19 condition Expanding our understanding of post COVID-19 condition: Webinar 2 (who.int), members of WHO COVID-19 clinical characterization and management research working group working on post COVID-19 condition, members of WHO

-3-

COVID-19 clinical network working on post COVID-19 condition, members of LongSOS patient group and clinicians and/or patients nominated by WHO regional office case management officers. Eligible participants were invited to participate via an online recruitment letter soliciting participation and engagement, along with an explanation of study objectives, instructions and outputs. The survey contained listed options regarding domains and variables to consider in the definition and initially were kept as broad and comprehensive as possible. The agreed domains and variables were followed by a series of questions relating to these variables with eventual values/thresholds related to each (Annexes 2). Survey responses were anonymous and tabulated by groups only. Registration of panellists and the actual Delphi questionnaire were accessible via https://delphimanager.liv.ac.uk/DefiningPostCOVID/Delphi All questions were evaluated on a 9-point Likert scale, from 1 (least important) to 9 (most important) and participants were asked to choose the level of importance for each variable in the definition. Wherever there was a value in the DelphiManager rating column that was something other than the Likert scale values of 1 to 9, the system coded “-9”, the value allocated when an outcome had not been rated; or “10”, the value allocated to the “unable to rate” option. The first round of the Delphi exercise lasted 14 days, and participants were sent two reminders to complete the online survey. The second round, conducted 5 weeks later, used a modified questionnaire based on iterative feedback and consensus during Round 1, and lasted 8 days, again with two reminders. In Round 2, participants were provided with the number/percentage of respondents having chosen that answer, and a reminder of their individual answer in Round 1. During each round, participants had the opportunity to add comments for each item, and also during Round 1 only, to add variables.

4. Statistical plan 4.1 Sample size and power

We aimed to obtain a diverse sample of participants from all stakeholders. Considering that some participants could be experts in more than one category, at least 20 experts from each of the five categories were invited with the goal of a sample size of at least 75–100 participants. Allowing for a 10–15% non-response (or non-desire to participate) rate, and 10% dropout rate, at least 100 participants were invited (11). 4.2 Primary and secondary endpoints The primary outcome was to achieve consensus on the importance of variable and value inclusion in the definition. “Consensus” was achieved on a question if 70% or more of the responses fell within 7 to 9 on the 9-point Likert scale (Fig. 1). “Disagreement” was considered to occur if 35% or more of responses fell within both of the two extreme ranges of possible options on the Likert scale (1 to 3 and 7 to 9). All other combinations of panel answers were considered “partial agreement”. For each question, consensus proportions were considered based on the number/percentage of respondents (excluding the category “not my area of expertise”). Therefore, the denominator for the consensus included only participants with knowledge and expertise for that specific question. Participant responses, including baseline and demographic characteristics, were analysed using basic statistics such as mean (standard deviation), median (interquartile range) and range. Responses on all other domains were analysed in proportion and illustrated using histograms.

-4-

5. Results Initially, a total of 747 invitations were sent by email. There were 265 respondents in Round 1, with 241 complete responses and 24 incomplete responses. In Round 2 there were 195 respondents with 178 complete responses and 17 incomplete responses (Fig. 2). The demographics of the 265 participants by stakeholder group, gender, age band and country are presented in Table 1. In Round 1, there were 61 (23.0%) patients, 18 (6.8%) patient-researchers, 138 (52.1%) external experts, 33 (12.5%) WHO staff and 15 (5.7%) “other”. The gender distribution was as 115 (43.4%) female, 147 (55.5%) male, and 3 (1.2%) “other”, with ages ranging from 20 years to 90+ years old but most in their fifties and sixties. Responses were received from participants from countries representing all WHO regions and World Bank income groups (Fig. 3). There were no statistically significant differences in the subset of participants in Round 2 (Table 1). From an initial, comprehensive list of identified 14 domains, 11 were selected in Round 1, and then one more added, for a total of 12 in Round 2 (Table 2). They were expanded with thresholds, and symptoms for a total of 45 items after further group discussion, namely on those results that reached borderline significance based on pre-defined thresholds (Annex 2). During subsequent revision, two domains that did not fully reach pre-specified thresholds were included in the clinical case definition after panel discussion, namely: i) “a minimum time period from onset of symptoms”; and ii) “duration of symptoms”. Similarly, the “new onset” nature of symptoms was expanded to incorporate “relapsing and fluctuating” through patient-panel feedback. A clinical case definition was built and further expanded with those domains/thresholds/values, and wording was trimmed in a quantitative/qualitative dedicated discussion with patients and patient-researchers (Table 3).

6. Discussion We present a clinical case definition of post COVID-19 condition to be applied in community and health care setting to optimize recognition and care of persons experiencing post COVID-19 condition. This definition was obtained by a robust, protocol-based methodology (Delphi consensus), engaging a diverse group of representative patients, care givers and other stakeholders from multiple geographies. This definition is compatible and consistent with previous suggestions available elsewhere (Annex 1), but is likely to change as new evidence emerges and our understanding of the consequences of COVID-19 continues to evolve. To date there have been several attempts to define different COVID-19 related topics and outcomes (12,13,14), but existing definitions do not take into account presentations in low- and middle-income countries and often miss domains that are relevant to various groups of stakeholders. To the best of our knowledge, the one presented here is the first Delphi exercise to define post COVID-19 condition. From a historical perspective, it took years to define AIDS/HIV, with the first human immunodeficiency virus (HIV) cases identified in June 1981 and the naming of acquired immune deficiency syndrome (AIDS) in September 1982, with the WHO AIDS surveillance case definition developed in October 1985 (15). Other examples are systemic exercise intolerance syndrome (SEIS) (formerly called chronic fatigue syndrome) (16), and most recently, post-intensive care syndrome (PICS) (17). 6.1 Strengths and limitations The strengths of this study include a robust protocol-based Delphi methodology and inclusiveness and representation of participants from five diverse stakeholder groups, from countries representing all WHO regions and World Bank income groups. We aimed to surpass current

-5-

controversies on the naming of the condition by using the WHO terminology of post COVID-19 condition (beyond others such as Chronic COVID-19 Syndrome, Late sequelae of COVID-19, Long COVID, Long haul COVID, Long-term COVID-19, Post COVID syndrome, Post-acute COVID-19, Post-acute sequelae of SARS-CoV-2 infection, etc.). We acknowledge patients’ activism conducted under the “Long COVID” umbrella (18,19). Regarding the methodology of the study, a number of limitations must be considered. English language was selected for practicality issues, but subsequent Delphi exercises should include other languages. Response rates in both rounds could have been greater – this is not unexpected, given its conduct during a pandemic. Best practices for enhancing response rates were integrated throughout (20), including introductory messages and reminder emails. Responses from the African and Eastern Mediterranean Regions were especially sought and obtained, but their overall proportions were lower than from other geographies. Wording of some domains and values were modified and new items were added from Round 1 to 2, given the enthusiastic persistence of some participants. Inclusion of criteria on timing and duration were agreed after Round 2, even though consensus was not achieved (as timing was deemed important to be included). A third round was considered unnecessary and impractical. Overall, as several pathophysiological mechanisms are in place and interplay during and after acute infection (21), and different trajectories for recovery after COVID-19 exist (22), it could be considered overtly ambitious to produce a single, universal definition that might work well for clinical, research, policy and advocacy purposes, and for all care levels and severities. The definition presented here (Table 3), with 88 words, might be considered a description based on the opinions of those participating, and difficult to operationalize in practice. Not only timing and duration, but the symptoms are subject to subjectivity and bias of participants. We strongly support that its open discussion in an organized way and integrating emerging evidence, such as prospective cohort trials, should help to advance this field. 6.2 Future implications As mentioned above, this proposal of a clinical case definition is likely temporary, as new data continue to emerge. Initial reports describing post COVID-19 condition were from small patient samples, with an inherent short follow-up, and likely subject to bias (23), and will be unravelled in ongoing meta-analyses (24). New research is exploring the use of electronic health records from representative samples of patients identified in primary care and elsewhere (25). The use of comparator samples of individuals fully recovered after acute infection is envisaged. By using cluster analysis and other mathematical tools to determine specific symptoms and their minimum number, they all could be formally identified, and eventually clustered for different phenotypes. Importantly, time thresholds from onset of infection or the duration of these symptoms could be established (26,27).

7. Conclusion COVID-19 will remain a challenge for the foreseeable future (28). Many pending answers surrounding COVID-19 and its sequelae remain, with new questions constantly being formulated (29,30,31). This definition of post COVID-19 condition will help to advance both advocacy and research but will likely change as new evidence emerges and our understanding of the consequences of COVID-19 continues to evolve.

-6-

Tables and figures Fig. 1. Agreement rules and thresholds for consensus and disagreement

‘Consensus’ will be obtained on a question if 70% or more of the responses fall within the same response on a 9-point Likert scale.

‘Disagreement’ will occur if 35% or more of responses fall in both of the two extreme ranges of possible options on the Likert scale.

All other combinations of panel answers will be considered ‘partial agreement’.

(1) (3)(2) (8)(7)(6)(5)(4) (9) Not my expertise

Limited importance Important but not critical Critical to include

-7-

Fig. 2. STROBE flowchart of participation in the two Delphi rounds

Participants invited on Tuesday 11 May 7 AM n=747

Total responses in Round 1

n=265

Complete responses in Round 1

n=241

Incomplete responses in Round 1

n=24

Total responsesin Round 2

n=195

Complete responsesin Round 2

n=178

Incomplete responses in Round 2

n=17

Excluded:WHO country representatives n=153

Email address wrong n=25“away from the office” n=39

New suggestions: 15 May by WHO representatives

n=2519 May n=139

plus a second reminder

-8-

Fig. 3. World map distribution of participants

-9-

Table 1. Demographic characteristics of participants

Variable and values Round 1, n (%) Round 2, n (%)

Stakeholder group Patient Patient-researcher External expert WHO staff Other

61 (23.0) 18 (6.8)

138 (52.1) 33 (12.5) 15 (5.7)

47 (24.1) 13 (6.7)

103 (52.8) 22 (11.3) 10 (5.1)

Gender Female Male Non-binary Prefer not to say

115 (43.4) 147 (55.5)

1 (0.4) 2 (0.8)

86 (44.1) 107 (54.9)

0 2 (1.0)

Age band 20 to 29 years old 30 to 39 years old 40 to 49 years old 50 to 59 years old 60 to 69 years old 70 to 79 years old 90 years or older

16 (6.0) 53 (20.0) 86 (32.5) 73 (27.5) 32 (12.1) 4 (1.5) 1 (0.4)

11 (5.6)

42 (21.5) 63 (32.3) 52 (26.7) 22 (11.3) 4 (2.1) 1 (0.5)

WHO region African American Eastern Mediterranean European Southeast Asian Western Pacific Country not specified

9 (3.4)

53 (20.0) 7 (2.6)

94 (35.5) 10 (3.8) 19 (7.2) 73 (27.5)

8 (4.1)

36 (18.5) 4 (2.1)

70 (35.9) 8 (4.1) 18 (9.2)

51 (26.2)

World Bank income group High income Upper middle income Lower middle income Low income Country not specified

140 (52.8) 37 (14.0) 13 (4.9) 2 (0.8)

73 (27.5)

110 (56.4) 22 (11.3) 10 (5.1) 2 (1.0)

51 (26.2)

Total 265 (100) 195 (100)

-10-

Table 2. Domains that achieved consensus by participants in each Delphi stage

Domain number

Domain name

1 History of SARS-CoV-2 infection

2 SARS-CoV-2 laboratory confirmation

3 Minimum time period from onset of symptoms (or from date of positive test for asymptomatic) 3 months

4 Minimum duration of symptoms at least 2 months

5 Symptoms and/or impairments: cognitive dysfunction, fatigue, shortness of breath, others

6 Minimum number of symptoms

7 Clustering of symptoms

8 Time-course nature of symptoms: (fluctuating, increasing, new onset, persistent, relapsing)

9 Sequelae of well-described complications of COVID-19 (stroke, heart attack, etc.)

10 Symptoms cannot be explained by an alternative diagnosis

11 Application of definition to different populations: Include separate definition for children, others

12 Impact on everyday functioning

Note: Consensus achieved in Round 1, in Round 2 and after Delphi panel group discussion.

-11-

Table 3. A definition of post COVID-19 condition

Post COVID-19 condition occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, cognitive dysfunction but also others* and generally have an impact on everyday functioning. Symptoms may be new onset following initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms may also fluctuate or relapse over time. A separate definition may be applicable for children. Notes: There is no minimal number of symptoms required for the diagnosis; though symptoms involving different organs systems and clusters have been described. *A full list of described symptoms included in the surveys can be found in Annexes 2 . Definitions: Fluctuate – a change from time to time in quantity or quality. Relapse – return of disease manifestations after period of improvement. Cluster – two or more symptoms that are related to each other and that occur together. They are composed of stable groups of symptoms, are relatively independent of other clusters, and may reveal specific underlying dimensions of symptoms (32).

-12-

Annex 1. Repository of published/available definitions of post COVID-19 condition

Source Text Wellcome Symptoms persisting beyond 4 weeks after symptom onset suggestive of COVID-19 (33).

Lancet Multiorgan symptoms after COVID-19 are being reported by increasing numbers of patients. They range from cough and shortness of breath, to fatigue, headache, palpitations, chest pain, joint pain, physical limitations, depression, and insomnia, and affect people of varying ages. At the Lancet–Chinese Academy of Medical Sciences conference on 23 November 2020, Bin Cao presented data (in press at the Lancet) on the long-term consequences of COVID-19 for patients in Wuhan, and warned that dysfunctions and complications could persist in some discharged patients for at least 6 months. So-called long COVID is a burgeoning health concern and action is needed now to address it (34).

NICE Signs and symptoms that develop during or after an infection consistent with COVID-19, continue for more than 12 weeks and are not explained by an alternative diagnosis (35).

Scientific American

Individuals whose symptoms persist or develop outside the initial viral infection, but the duration and pathogenesis are unknown (36).

Royal Society

The onset of persistent or recurrent episodes of one or more of the following symptoms, within x* weeks of infection with SARS-CoV-2 and continuing for y* weeks or more: severe fatigue, reduced exercise capacity, chest pain or heaviness, fever, palpitations, cogintive impairment, anosomia or ageusia, vertigo and tinnitus, headache, peripheral neuropathy, metallic or bitter taste, skin rash joint pain or swelling (3).

* Maximum period between acquisition of the infection (if known) and the onset of symptoms, and the minimum duration of symptoms, should be specified in the definition.

Haute Autorité de santé, France

Three criteria: Having presented with a symptomatic form of COVID-19; presenting with one or more initial symptoms 4 weeks after the start of the disease; and none of these symptoms can be explained by another diagnosis (37).

CDC Long COVID: While most persons with COVID-19 recover and return to normal health, some patients can have symptoms that can last for weeks or even months after recovery from acute illness. Even people who are not hospitalized and who have mild illness can experience persistent or late symptoms (38).

Wikipedia Condition characterized by long-term sequelae – persisting after the typical convalescence period – of coronavirus disease 2019 (COVID-19) (39).

Nature Post-acute COVID-19 as persistent symptoms and/or delayed or long-term complications of SARS-CoV-2 infection beyond 4 weeks from the onset of symptoms (40).

-13-

Annex 2. Score for every domain/value in Rounds 1 and 2

Domain Round

Text n Sum 1 to 3 (%)

Sum 7 to 9 (%)

1 1 How important is “having a history of SARS-CoV-2 infection” to the clinical case definition of post COVID-19 condition?

252

4 79

2 Not asked in Round 2 - -

2 1 How important is “having a laboratory- confirmed previous history of SARS-CoV-2 infection” to the clinical case definition of post COVID-19 condition?

253

14 51

2 How important is “having a laboratory- confirmed previous history of SARS-CoV-2 infection when there is access to laboratory testing” to the clinical case definition of post COVID-19 condition?

194

10 52

3 1 How important is “the inclusion of the following proposed minimum time period (in weeks) between the onset of symptoms of acute COVID-19 (or for asymptomatic cases from date of positive test)” to the clinical case definition of post COVID-19 condition? • 4 weeks • 8 weeks • 12 weeks • 16 weeks or more

253

233

230

228

29 18 16 29

44 46 60 47

2 How important is “the inclusion of a minimum time period (in months) from the onset of COVID-19 to the presence of symptoms” to the clinical case definition of post COVID-19 condition? • 1 month • 2 months • 3 months • 4 months • 6 months or more • No time period

194

192

189

187

184

178

13 14 14 27 34 29

46 43 59 40 37 38

-14-

4 1 How important is “the inclusion of the following minimum duration of the symptoms” to the clinical case definition of post COVID-19 condition? • At least 2 weeks • At least 4 weeks • At least 6 weeks • At least 8 weeks

243

239

235

235

44 21 23 23

31 41 47 52

2 How important is “the inclusion of any of the following minimum duration of symptoms” to the clinical case definition of post COVID-19 condition? • At least 0.5 months (2 weeks) • At least 1 months (4 weeks) • At least 1.5 months (6 weeks) • At least 2 months (8 weeks) • At least 3 months • At least 6 months • No minimum time duration

184

184

183

183

178

177

171

38 17 20 17 23 37 47

28 42 42 47 40 32 27

5 1 How important is “each of the following symptoms” to the clinical case definition of post COVID-19 condition?

Abdominal pain 221

27 27

Menstrual and period problems 210

32 24

Altered smell/taste 232

9 57

Anxiety 230

16 49

Blurred vision 225

17 38

Chest pain 230

7 55

Cognitive dysfunction/brain fog 234

3 74

Cough 232

8 63

Depression 232

13 50

-15-

Dizziness 230

8 47

Fatigue 234

2 78

Intermittent fever 229

14 46

Gastrointestinal issues (diarrhoea, constipation, acid reflux)

226

17 33

Headache 226

8 56

Memory issues 230

6 65

Joint pain 231

8 52

Muscle pain/spasms 231

5 64

Neuralgias 225

9 49

New onset allergies 222

32 27

Pins and needles sensations 227

15 39

Post-exertional malaise 231

2 67

Shortness of breath 232

3 78

Sleep disorders 231

6 62

Tachycardia/palpitations 230

8 60

Tinnitus and other hearing issues 226

13 45

2 How important is “to include, in addition to FATIGUE, SHORTNESS OF BREATH AND COGNITIVE DYSFUNCTION, any of the following symptoms” to the clinical case definition of post COVID-19 condition?

Abdominal pain 178

29 24

Menstrual and period problems 172

34 22

Altered smell/taste 180

6 61

Anxiety 183

15 49

Blurred vision 177

20 30

Chest pain 182

7 56

Cough 181

8 58

Depression 180

18 46

-16-

Dizziness 181

10 39

Intermittent fever 178

17 40

Gastrointestinal issues (diarrhoea, constipation, acid reflux)

179

17 33

Headache 181

8 54

Memory issues 180

4 69

Joint pain 181

7 53

Muscle pain/spasms 180

4 63

Neuralgias 177

11 45

New onset allergies 172

36 20

Pins and needles sensations 177

14 29

Post-exertional malaise 180

2 67

Sleep disorders 179

6 65

Tachycardia/palpitations 180

6 59

Tinnitus and other hearing issues 177

12 38

6 1 How important is “having a minimum number of symptoms” to the clinical case definition of post COVID-19 condition?

239

25 38

2 Not asked in Round 2 - -

7 1 How important is “the inclusion of clusters of symptoms” to the clinical case definition of post COVID-19 condition?

237

8 55

2 How important is “the inclusion of clusters of symptoms” to the clinical case definition of post COVID-19 condition?

179

4 68

8 1 How important is “to include each of the following descriptors on the nature of symptoms” in the clinical case definition of post COVID-19 condition? • Fluctuating • Increasing • New onset • Persistent • Relapsing

231

231

233

237

17 17 7 5 9

52 49 69 78 62

-17-

230

2 How important is “to include, in addition to PERSISTENT, other descriptors of the nature of symptoms” to the clinical case definition of post COVID-19 condition? • Fluctuating • Increasing • New onset • Relapsing

184

182

183

183

10 22 5 6

55 36 69 65

9 1 How important is “to include only symptoms separate from those that are thought to be a sequela of well-described acute complications of COVID-19 (i.e. stroke, acute respiratory distress syndrome, acute kidney injury, myocarditis, thrombosis, post intensive care syndrome [PICS])” in the clinical case definition of post COVID-19 condition?

230

18 60

2 How important is “to exclude symptoms which are directly related to acute complications of COVID-19 (such as stroke, acute respiratory distress syndrome, acute kidney injury, myocarditis, thrombosis, post-intensive care syndrome)” in the clinical case definition of post COVID-19 condition?

180

14 63

10 1 How important is “that post COVID-19 condition be considered a diagnosis of exclusion determined by a health provider when symptoms cannot be explained by an alternative diagnosis” to the clinical case definition of post COVID-19 condition?

231

5 71

2 Not asked in Round 2 - - 11 1 How important is “that the post COVID-19

condition definition can be applied to all populations, including women who are pregnant or post-partum; young children; neonates; people with chronic disease; people living with HIV; older people; or individuals who are immunocompromised because of other conditions” to the clinical case definition of post COVID-19 condition?

230

7 69

-18-

2 How important is “to include a separate clinical case definition for post COVID-19 condition to apply for children”?

173

2 79

12 2 How important is “that the symptoms experienced have an impact on everyday functioning” in the clinical case definition of post COVID-19 condition?

178

2 77

-19-

References

1. WHO coronavirus (COVID-19) dashboard. Geneva: World Health Organization; 2021 (https://covid19.who.int/, accessed 31 August 2021).

2. COVID-19. World Health Statistics. Geneva: World Health Organization; 2021 (https://www.who.int/data/gho/publications/world-health-statistics, accessed 31 August 2021).

3. Long Covid: what is it, and what is needed? London: Royal Society; 23 October 2020. DES7217.

4. GBD Long COVID Collaborators. Surviving COVID-19: a global systematic analysis of long COVID disability in 2020. (submitted).

5. Emergency use ICD codes for COVID-19 disease outbreak. Geneva: World Health Organization; 2021 (https://www.who.int/standards/classifications/classification-of-diseases/emergency-use-icd-codes-for-covid-19-disease-outbreak, accessed 31 August 2021).

6. Janet V Diaz, Joan B Soriano. A Delphi consensus to advance on a clinical case definition for post COVID-19 condition: a WHO protocol. Protocol Exchange. 2021. doi:10.21203/rs.3.pex-1480/v1 (https://protocolexchange.researchsquare.com/article/pex-1480/v1, accessed 31 August 2021).

7. Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Management Science. 1963;9(3):458–467. doi:10.1287/mnsc.9.3.458. hdl:2027/inu.30000029301680.

8. Brown BB. Delphi process: a methodology used for the elicitation of opinions of experts. Santa Monica (CA): RAND Corporation; 1968 (https://www.rand.org/pubs/papers/P3925.html, accessed 31 August 2021).

9. Green KC, Armstrong JS, Graefe A. Methods to elicit forecasts from groups: Delphi and prediction markets compared. Foresight: International Journal of Applied Forecasting. 2007 (https://repository.upenn.edu/marketing_papers/157/, accessed 31 August 2021).

10. Rowe G, Wright G. The Delphi technique as a forecasting tool: issues and analysis. Intl J Forecasting. 1999;15(4):353–375.

11. Murphy E, Black N, Lamping D, McKee C, Sanderson C. Consensus development methods, and their use in clinical guideline development: a review. Health Technol Assess. 1998;2(3).

12. Shanbehzadeh M, Kazemi-Arpanahi H, Mazhab-Jafari K, Haghiri H. Coronavirus disease 2019 (COVID-19) surveillance system: development of COVID-19 minimum data set and interoperable reporting framework. J Educ Health Promot. 2020;9:203. doi:10.4103/jehp.jehp_456_20. eCollection 2020. PMID: 33062736.

13. Nasa P, Azoulay E, Khanna AK, Jain R, Gupta S, Javeri Y et al. Expert consensus statements for the management of COVID-19-related acute respiratory failure using a Delphi method. Crit Care. 2021;25(1):106. doi:10.1186/s13054-021-03491-y. PMID: 33726819.

14. Schell CO, Khalid K, Wharton-Smith A, Oliwa JN, Sawe HR, Roy N et al. Essential emergency and critical care – a consensus among global clinical experts. medRxiv preprint. doi:https://doi.org/10.1101/2021.03.18.21253191 (this version posted 25 March 2021).

15. CDC. Revision of the case definition of acquired immunodeficiency syndrome for national reporting--United States. Washington (DC): Centers for Disease Control and Prevention. MMWR Morb Mortal Wkly Rep. 1985;34(25):373–5. PMID: 2989677.

16. Komaroff AL. Advances in understanding the pathophysiology of chronic fatigue syndrome. JAMA. 2019;322(6):499–500. doi:10.1001/jama.2019.8312. PMID: 31276153.

17. Smith S, Rahman O. Post intensive care syndrome. StatPearls Publishing; 2021.

18. Alwan NA, Burgess RA, Ashworth S, Beale R, Bhadelia N, Bogaert D et al. Scientific consensus on the COVID-19 pandemic: we need to act now. Lancet. 2020;396(10260):e71–e72. doi:10.1016/S0140-6736(20)32153-X. PMID: 33069277.

19. Davis HE, Assaf GS, McCorkell L, Wei H, Low RJ, Re’em Y et al. Characterizing long COVID in an international cohort: 7 months of symptoms and their impact. EClinicalMedicine. 2021;38:101019. doi:10.1016/j.eclinm.2021.101019. PMID: 34308300.

20. Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ. 2008;179(3):245–52. doi:10.1503/cmaj.080372. PMID: 18663204.

21. Editorial. COVID-19 pathophysiology: looking beyond acute disease. Lancet Respir Med. 2021;9(6):545.

22. Sakurai A, Sasaki T, Kato S, Hayashi M, Tsuzuki SI, Ishihara T. Natural history of asymptomatic SARS-CoV-2 infection. N Engl J Med. 2020;383(9):885–886. doi:10.1056/NEJMc2013020. Epub 2020 Jun 12. PMID: 32530584.

-20-

23. Rando HM, Bennett TD, Byrd JB, Bramante C, Callahan TJ, Chute CG et al. Challenges in defining long COVID: striking differences across literature, electronic health records, and patient-reported information. medRxiv. 2021:2021.03.20.21253896. doi:10.1101/2021.03.20.21253896. Preprint. PMID: 33791733.

24. Iqbal FM, Lam K, Sounderajah V, Clarke JM, Ashrafian H, Darzi A. Characteristics and predictors of acute and chronic post-COVID syndrome: a systematic review and meta-analysis. EClinicalMedicine. 2021;36:100899. doi:10.1016/j.eclinm.2021.100899. eCollection 2021 Jun. PMID: 34036253.

25. Soriano JB, Waterer G, Peñalvo JL, Rello J. Nefer, Sinuhe and clinical research assessing post COVID-19 condition. Eur Respir J. 2021;57(4):2004423. doi:10.1183/13993003.04423-2020. PMID: 33380509.

26. Sun C, Hong S, Song M, Li H, Wang Z. Predicting COVID-19 disease progression and patient outcomes based on temporal deep learning. BMC Med Inform Decis Mak. 2021;21(1):45. doi:10.1186/s12911-020-01359-9. PMID: 33557818.

27. Lancet Digital Health. Artificial intelligence for COVID-19: saviour or saboteur? Lancet Digit Health. 2021;3(1):e1. doi:10.1016/S2589-7500(20)30295-8. PMID: 33735062.

28. Muller JE, Nathan DG. COVID-19, nuclear war, and global warming: lessons for our vulnerable world. Lancet. 2020;395(10242):1967–1968. doi:10.1016/S0140-6736(20)31379-9. Epub 2020 Jun 12. PMID: 32539935.

29. Norton A, Olliaro P, Sigfrid L, Carson G, Hastie C, Kaushic C et al. Long COVID: tackling a multifaceted condition requires a multidisciplinary approach. Lancet Infect Dis. 2021;21(5):601–602. doi:10.1016/S1473-3099(21)00043-8.

30. Lerner AM, Robinson DA, Yang L, Williams CF, Newman LM, Breen JJ et al. Toward understanding COVID-19 recovery: National Institutes of Health Workshop on Postacute COVID-19. Ann Intern Med. 2021;174(7):999–1003. doi:10.7326/M21-1043. PMID: 33780290.

31. WHO Clinical case definition working group on post COVID-19 condition. Towards a universal understanding of post COVID-19 condition. Bull World Health Organ; 2021 (in press).

32. Barsevick A. Defining the symptom cluster: how far have we come? Seminars in Oncology Nursing. 2016;32(4):334–350.

33. Perego E, Callard F, Stras L, Melville-Jóhannesson B, Pope R, Alwan NA. Why the patient-made term 'Long Covid' is needed. Wellcome Open Research. 2020;5:224.

34. Editorial. Facing up to long COVID. Lancet. 2020;396(10266):1861. doi:10.1016/S0140-6736(20)32662-3.

35. NICE guideline [NG188] COVID-19 rapid guideline: managing the long-term effects of COVID-19. Published: 18 December 2020. London: National Institute for Health and Care Excellence; 2020.

36. Barber C. The problem of ‘long haul’ COVID. Scientific American. 29 December 2020. (https://www.scientificamerican.com/article/the-problem-of-long-haul-covid/?print=true, accessed 31 August 2021).

37. Haute Autorité de Santé, France. Covid long : les recommandations de la Haute Autorité de santé. Service-Public.fr. 16 February 2021 (https://www.service-public.fr/particuliers/actualites/A14678, accessed 31 August 2021).

38. CDC. COVID-19: your health. Washington (DC): Centers for Disease Control and Prevention; 2021 (https://www.cdc.gov/coronavirus/2019-ncov/long-term-effects.html, accessed 31 August 2021).

39. Wikipedia. Long COVID. 2021 (https://en.wikipedia.org/wiki/Long_COVID, accessed 31 August 2021).

40. Nalbandian A, Sehgal K, Gupta A, Madhavan MV, McGroder C, Stevens JS et al. Post-acute COVID-19 syndrome. Nat Med. 2021;27(4):601–605. doi:10.1038/s41591-021-01283-z. PMID: 33753937.