• 1 •Chin J Integr Med

Background Chinese medicine (CM), also known as traditional

Chinese medicine, operates in parallel with Western medicine in China's healthcare system. Both of the medicines play important roles in public healthcare in mainland China. Ever since evidence-based medicine (EBM) has been introduced to mainland China from 1996, clinical research in CM has made considerable developments. However, in the past three decades, CM researchers conducted a lot of randomized controlled trials (RCTs) to evaluate the effi cacy of CM.

The first RCT of CM was published in the early 1980s and the first systematic review of CM was published in 2002.(1) The number of RCTs has doubled over the past 30 years. A literature review that analyzed RCTs in CM in twenty years from 1976 to 1996(2,3) found that the top 10 common diseases in RCTs were ischemic heart disease, stroke, chronic viral hepatitis, peptic ulcer, childhood diarrhoea, hyperlipidemia, primary hypertension, upper digestive tract bleeding, diabetes mellitus, and pneumonia. Another review reported that research on CM in the past decade increased dramatically, including systematic reviews/meta analysis, RCTs, cohort studies, case control studies, case series, case report and other research on quality of life, patient reported outcomes and biomarkers, quality control, safety evaluation and case studies.(4) Methods and technologies in clinical epidemiology and EBM signifi cantly improve the quality

of clinical research in CM. Many CM researchers realize that critical appraisal of evidence and reporting results according to international standardization are very important. The majority of CM researchers are able to conduct clinical trials according to correct rules about the design, measure and evaluation.

In 2011, a prospective, non-blinded, randomized, and controlled trial of Oseltamivir compared with the Chinese herbal decoction Maxing Shigan–Yinqiaosan (麻杏石甘–银翘散) in the treatment of H1N1 infl uenza was published in Annals of Internal Medicine, the result of which showed that Oseltamivir and MY, alone and in combination reduced the time for fever resolution in patients with H1N1 infl uenza virus infection.(5) Such a well-designed study published in an international Western medicine journal with high impact factor was encouraging for CM researchers. It has had two important signifi cances. One is that the clinical research on CM can be published in international renowned

ACADEMIC EXPLORATION

What Can Comparative Effectiveness Research, Propensity Score and Registry Study Bring to Chinese Medicine?

LIAO Xing (廖 星) and XIE Yan-ming (谢雁鸣)

©The Chinese Journal of Integrated Traditional and Western Medicine Press and Springer-Verlag Berlin Heidelberg 2013Supported by National Natural Science Foundation of China (No. 81202776), the Seventh-Science Foundation of China Academy of Chinese Medical Sciences (No. ZZ070817) and the Sixth-Science Foundation of Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences (No. Z0215)Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing (100700), ChinaCorrespondence to: Prof . XIE Yan-ming, Tel : 86-10- 64014411-3302, E-mail: datamining5288@163.comDOI: 10.1007/s11655-013-1662-9

ABSTRACTABSTRACT The impact of evidence-based medicine and clinical epidemiology on clinical research has contributed The impact of evidence-based medicine and clinical epidemiology on clinical research has contributed

to the development of Chinese medicine in modern times over the past two decades. Many concepts and methods to the development of Chinese medicine in modern times over the past two decades. Many concepts and methods

of modern science and technology are emerging in Chinese medicine research, resulting in constant progress. of modern science and technology are emerging in Chinese medicine research, resulting in constant progress.

Systematic reviews, randomized controlled trials and other advanced mathematic approaches and statistical analysis Systematic reviews, randomized controlled trials and other advanced mathematic approaches and statistical analysis

methods have brought reform to Chinese medicine. In this new era, Chinese medicine researchers have many methods have brought reform to Chinese medicine. In this new era, Chinese medicine researchers have many

opportunities and challenges. On the one hand, Chinese medicine researchers need to dedicate themselves to opportunities and challenges. On the one hand, Chinese medicine researchers need to dedicate themselves to

providing enough evidence to the world through rigorous studies, whilst on the other hand, they also need to keep providing enough evidence to the world through rigorous studies, whilst on the other hand, they also need to keep

up with the speed of modern medicine research. For example, recently, real world study, comparative effectiveness up with the speed of modern medicine research. For example, recently, real world study, comparative effectiveness

research, propensity score techniques and registry study have emerged. This article aims to inspire Chinese medicine research, propensity score techniques and registry study have emerged. This article aims to inspire Chinese medicine

researchers to explore new areas by introducing these new ideas and new techniques.researchers to explore new areas by introducing these new ideas and new techniques.

KEYWORDSKEYWORDS comparative effectiveness research, propensity score, registry study, Chinese medicine comparative effectiveness research, propensity score, registry study, Chinese medicine

• 2 • Chin J Integr Med

medicine journals, with a rigorous peer review process. The other is that RCTs could be applied in the research on herbal decoctions. Although many RCTs have been published over the years, more and more researchers realized that RCT design has distinct limitations when applied to CM. As we know CM has a very different clinical practice system and process from Western medicine, all the core of designing, measuring and evaluating in CM should focus on "CM patterns or syndromes". In other words, the effi cacy or effectiveness of CM is not only based on the diagnosed disease but also on the diagnosed pattern/syndrome. How to develop appropriate research methods to evaluate the effi cacy or effectiveness of CM has been put forward by its researchers.

At present, CM researchers at the frontiers of medical knowledge have been working hard to introduce new ideas and new technologies of modern medicine to CM in order to develop CM research.(6,7) It is worth noting that some CM researchers with sharp and incisive minds have begun to introduce "real-world study",(8) comparative effectiveness research (CER),(9) registry study and propensity score techniques to CM.

So what can these new ideas and techniques bring to CM research in the new era of medicine? This article will take some examples to illustrate the following points.

What Can CM Researchers Learn from "Real World Studies" and CER?

More and more researchers find that traditional trials, in particular the classic explanatory randomized controlled trials, have many limitations and restrictions in evaluating the effectiveness of CM. CM doctors also fi nd they have difficulty in relating the results of ERCTs to their clinical practice. In recent years, "real-world studies," also called "effectiveness studies" or "pragmatic trials," and CER have gained increasing importance in the context of EBM.(10) CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.(11) In other words, CER is a way of identifying what works for which patients under which circumstances.(7) Both "real-world studies" and CER aim to increase the generalizability of studies and advocate that the clinical trial should be conducted under conditions as close as possible to "routine" care, where effectiveness should be evaluated. Pragmatic RCT and observational study have

thereby been utilized and stressed to explore clinical relevance. As such, they could be designed to evaluate the effectiveness of the treatment in real-world and in the patient being representative of everyday clinical practice for CM.

However, the less restrictive design and inherent methodological limitations of these "effectiveness studies" have to be carefully considered. Non-randomized and non-blinded designs, including patients with comorbidity or comedication, can induce different kinds of biases and increase the variance. So researchers have suggested that they should be cautious in interpreting the results of trials with such problems.

Due to these methodological l imitat ions, effectiveness studies are on a principally lower level of evidence, adding only a complementary view to the results of traditional trials.(10) Data from "real world study" rather than traditional experiment is challenging for analysts. Actually, many statisticians committed themselves to circumvent these problems. The propensity score (PS) method is a good resolution to estimate the effect of the treatment when random assignment of interventions to subjects is not feasible. Rosenbaum and Rubin proposed "propensity scores" in 1983 as a method of controlling confounders in observational studies,(12) which could be used to deal with challenges in the real world.

How Could PS Techniques Help CM Researchers Control Confounders in "Real World Studies"?

In order to overcome the selection biases in the assessment of treatment effects from observational (non-randomized) studies for patients not randomly allocated to the treated or control group, PS is often used to adjust for differences of the characteristics between patients treated with real and control interventions at baseline. The PS allows the researchers to design and analyze a non-randomized study to mimic some of the particular characteristics of a RCT.(13) PS has been widely used recently in medical research.(14) There are 3,803 related publications in PubMed with the phrase "Propensity Score" as of 17th Jan, 2013. It showed that the popularity of PS techniques has increased considerably. The PS is defi ned as the conditional probability of a study that a patient received the treatment of interest rather than the control treatment. PS is a device to create matched pairs or matched sets or strata that balance many observed covariates, not unobserved covariates.(15) Rosenbaum

• 3 •Chin J Integr Med

and Rubin showed that conditional on this score, all observed pretreatment covariates are independent of group assignment and, in large samples, covariates will be distributed equally in both groups and will not confound estimated treatment effects.(16) Most of the examples used in the literature apply a parametric logistic regression model in the estimation of PS. There are mainly four different ways to incorporate the estimated PS in the treatment effect model such as matching, stratifying, covariate adjustment and inverse probability of received treatment weighting. Six typical steps of propensity analysis could be summarized from some research, for instance, identifying confounding variables, estimating PS as the probability of receiving experimental treatment, matching experimental to control patients, evaluating success of matching, and comparing outcomes of treatment groups and interpretation.(17,18)

To date, many statisticians have developed di f ferent methods to est imate PS, (19,20) which researchers of CM could refer to. In mainland China, some researchers of Western medicine have already begun exploring PS a few years ago.(21,22) Nonetheless, PS techniques have not yet been used widely in the research of CM.

In summary, PS techniques have become an important statistical tool, which can give the researchers of CM a way to control the confounding bias for their particular dataset and to resolve the questions in the real world clinical practice of CM.

What Benefi ts May the Registry Study Produce for the Active Safety Surveillance of Post-Marketing CM?

There is a very famous book entitled "Registries for Evaluating Patient Outcomes: A User's Guide" edited by the Agency for Healthcare Research and Quality of U.S.(23) It is a high-level practical guide for registry study. In this book, registries are defi ned as using observational study design that does not specify treatments or require any therapies intended to change patient outcomes. Also, the definition of registry study was found in the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) 14th Annual International Meeting, which is "Prospective observational study of subjects, with certain shared characteristics, that collects ongoing and supporting data over time on well defi ned outcomes of interest for analysis and reporting". Normally, few inclusion and exclusion criteria are used

in an effort to study a broad range of patients to make the results more generalizable. The data collected in registries generally reflect whatever tests and measurements clinicians customarily use.

Registry studies play an increasingly important role in providing information that will validate the safety and efficacy of interventions reported in other clinical trials. And registry study is an ideal method especially in populations where interventional studies are unethical, where large numbers of subjects are needed, or when post-approval studies are required. Registry studies can be used in epidemiology, public health and clinical research. Researchers can use it to conduct different types of research, including outcome research, safety surveillance, risk management and disease or drug research. For safety surveillance, researchers can study the nature, severity, and frequency of occurrence of adverse drug events and adverse drug reactions (ADRs). Registry studies can be either prospective, retrospective, or both. If using data from existing medical records (hospital information system), the study could be a retrospective or prospective study. Study designs for registries are still those conventional observational study models of cohort, case-control, and case-cohort. Its sampling strategies are divided into two categories, is probability sampling and the other one is non-probability sampling.

The exciting news is that we are now conducting a registry study on post marketing pharmacovigilance and safety monitoring for CM injections. To our knowledge, this is the largest cohort study that investigates new or suspected ADRs and calculates the incidence rate of CM injection ADRs in China, which is a prospective, multicenter, clinical registry. This registry study is a 3-year active surveillance of more than 10 injections of CM. For each injection, about 30,000 patients will be observed.

To sum up, registry study is a scientific tool with certain advantages, which is conducted in the real world in order to evaluate the interventions' effectiveness, and is also a useful tool that can validate whether the safety and efficacy of interventions evaluated in controlled clinical trials translate to everyday practice. The great advantage of registries is allowing large-scale/long-term data collection compared to traditional studies. The information gained from registries can be used to develop scientific questions for further study. It also serves as a valuable tool to develop better defi nitions of inclusion

• 4 • Chin J Integr Med

and exclusion criteria for clinical study by clarifying the characteristics of the population to be evaluated.

Here, we would like to remind the researchers of CM to make the method transparent to ethics boards. To be careful of all the potential biases in the observations when undertaking a registry study and to ensure that the statistical data analysis is consistent with the limitations of research in a nonrandomized setting.

ConclusionIn summary, it is increasingly recognized that

new ideas and new techniques in modern medicine represent new developing trends in medical sciences. They may facilitate the development of research in CM in many aspects. Great progress in CM can be achieved by using new research methods. We are looking forward to a new modern picture of CM.

AcknowledgementMany thanks to Prof. MA Liang-xiao (Beijing University

of CM) and Dr. Trina Ward (from the United Kingdom) with their

kind help in the English writing in this article.

REFERENCESNestler G. Traditional Chinese medicine. Med Clin North 1.

Am 2002;86:63-73.

Tang JL, Zhan SY, Ernst E. Review of randomized controlled 2.

trials of traditional Chinese medicine. BMJ 1999;319:160-161.

Zhan SY, Tang JL, Xie LY, Wu T, Bei D, Li LM. Review of 3.

randomized controlled trials of traditional Chinese medicine

from Chinese Medicine Academic journals and suggestions for

future. Chin J Integr Tradit West Med (Chin) 1999;19:568-570.

Jiang M, Yang J, Zhang C, Liu B, Chan K, Cao HX, et al. 4.

Clinical studies with traditional Chinese medicine in the past

decade and future research and development. Planta Med

2010;76:2048-2064.

Wang C, Cao B, Liu QQ, Zou ZQ, Liang ZA, Gu L, et al. 5.

Oseltamivir compared with the Chinese traditional therapy

Maxingshigan–Yinqiaosan in the treatment of H1N1 infl uenza

a randomized trial. Ann Intern Med 2011;155:217-225.

Zhang C, Jiang M, Lu AP. A traditional Chinese medicine 6.

versus Western combination therapy in the treatment

of rheumatoid arthritis: two-stage study protocol for a

randomized controlled trial. Trials 2011;12:1-8.

Wu DR, Cai YF, Cai JX, Liu QL, Zhao YQ, Cai JH, et al. 7.

Comparative effectiveness research on patients with acute

ischemic stroke using Markov decision processes. BMC

Med Res Methodol 2012;12:1-10.

Tian F, Xie YM. Real-world study: a potential new approach8.

to effectiveness evaluation of traditional Chinese medicine

interventions. J Chin Integr Med 2010;8:301-306.

Xie YM, Liao X, Wang YY. Introduce the idea of comparative 9.

effectiveness research to clinical research of Chinese

medicine. J Chin Integr Med 2011;9:813-818.

Möller HJ. Effectiveness studies: advantages and 10.

disadvantages. Dialogues Clin Neurosci 2011;13:199-207.

"IOM: Initial national priorities for comparative effectiveness 11.

research " http://www.iom.edu/~/media/Files/Report%20

Files/2009/ComparativeEffectivenessResearchPriorities/

CER%20report%20brief%. Retrieved 13 September 2008.pdf.

Rosenbaum PR, Rubin DB. The central role of the 12.

propensity score in observational studies for causal effects.

Biometrika 1983;70:41-55.

Austin PC. An introduction to propensity score methods 13.

for reducing the effects of confounding in observational

studies. Multivariate Behav Res 2011;46:399-424.

Georg H, Jüni P. An overview of the objectives of and the 14.

approaches to propensity score analyses. Eur Heart J

2011;32:1704-1708.

Joffe MM, Rosenbaum PR. Propensity scores. Am J 15.

Epidemiol 1999;150:327-333.

Rosenbaum PR, Rubin DB. The central role of the 16.

propensity score in observational studies for causal effects.

Biometrika 1983;70:41-45.

Ahmed A, Husain A, Love TE, Gambassi G, Dell' Halia LJ, 17.

Francis GS, et al. Heart failure, chronic diuretic use, and

increase in mortality and hospitalization: an observational

study using propensity score methods. Eur Heart J

2006;27:1431-1439.

Rajeev H, Dehejia, Sadek W. Propensity score-matching 18.

methods for nonexperimental causal studies. Rev Econ

Stat 2002;84:151-161.

McCaffrey DF, Ridgeway G, Morral AR. Propensity score 19.

estimation with boosted regression for evaluating causal effects

in observational studies. Psychol Methods 2004;9:403-425.

Setoguchi S, Schneeweiss S, Brookhart MA, Glynn RJ, Cook 20.

EF. Evaluating uses of data mining techniques in propensity

score estimation: a simulation study. Pharmacoepidemiol

Drug Saf 2008;17:546-555.

Wang YJ, Cai HW, Xia JL, Jiang ZW. Scores on propensity 21.

(Ⅰ) conception and measures. Chin J Epidemiol (Chin)

2010;31:347-348.

Wang YJ, Cai HW, Xia JL, Jiang ZW, Wang L. Propensity 22.

score (Ⅲ) Key issues in practice of propensity score. Chin

J Epidemiol (Chin) 2010;31:823-825.

Gliklich RE, Dreyer NA. Registries for evaluating patient 23.

outcomes: a user's guide. 2nd ed. (Prepared by Outcome

DEcIDE Center [Outcome Sciences, Inc. d/b/a Outcome]

under Contract No.HHSA29020050035I TO3.) AHRQ

Publication No.10-EHC049. Rockville, MD: Agency for

Healthcare Research and Quality. 2010.

(Received July 22, 2013)Edited by ZHANG Wen