Introduction

Doing clinical research is easy when you understand. Before conducting a clinical trial, you first need to know how to appraise a study. For different study types, you deliberately look out for bias, and that is, systematic errors affecting the truth. We call this process "Validity". Practice makes perfect. After practicing appraisal works on different types of studies, you can start to do your own research. It is because you know how to minimise bias in your own trial.

Clinical trial only deals with data in a quantitative manner. It is totally different from qualitative study. Each study can only answer one single question called PICO, namely, Population, Intervention, Comparison, and Outcome. And the result is presented as a continuous data or dichotomous (yes or no) data. It is then analyzed and presented either as "statistically significance" or "statistically non-significance", or in a clinical manner with different magnitudes of effect. Because one PICO comes with only one answer, we cannot be brave in our discussion.

Lastly, we may like to project the results to our own group of patients. This comes to applicability, namely, racial and sex differences, treatment's benefit and harm, easy accessibility of treatment, and patients' own choice and belief.

Status of clinical research in mainland China

At present, there is a scarcity of clinical trials in mainland China. Most trials are primary. And clinical studies are limited to researchers in main cities. Most are published in non-SCI journals, and the majorities are meta-analysis studies. There are only a few SCI published articles which are related to emergency medicine.

Recently, over 80% of clinical trial data submitted to support new drug registrations in China have been revealed as fraudulent or substandard by China's State Food and Drug Administration (CFDA). An investigation of data for 1622 new drugs submitted for registration said that 1308 of the applications should be withdrawn because they contained fabricated, flawed, or inadequate data from clinical trials.

Difference between clinical research and primary study

A confounder (also confounding variable or confounding factor) is a variable that influences both the dependent variable and independent variable causing a spurious association. Known confounders include variables, like gender and racial differences, age, and co-morbidity. Primary studies mainly deal with animals. And you cannot be sure about confounders in animals. For clinical studies, we deal with human. There are known and unknown confounders among us. To distribute all confounders equally into treatment arm and placebo arm, we randomize subjects into two groups. So to speak, we deliberately design our clinical trial to minimise bias (also, known as systematic errors). Bias refers to the tendency of a measurement process to over-or under-estimate the true value of a population parameter.

Know your strength

We do have strength in doing research in mainland. Mainland population comes to 1.3 billion. It is a big pool of data. The territory is wide with diversity of inland mountains, plateaus, rivers, seismic activities and long coastal line at eastern border along North Pacific Ocean. Our country is developing at a fast pace and is energetic 24 hours a day. Hence we have a lot of (a). Rare and common diseases, (b). Traumatic and medical events, (c). Man-made and natural catastrophic incidents. Hence we do not have any problem of subjects' recruitment. In contrary, researchers around the world come to us!

Before you do your own study, you should know how to appraise an article

Do not waste your effort. After knowing how to appraise a study, you start to do your own. Get smart and look out for knowledge gap. Think of PICO, which stand for Population, Intervention, Comparison, Outcome.

You need to remember the mnemonic "FINER", namely Feasibility, Interesting, Novel, Ethical and Relevant.

The most important point is whether your study is "original/novel". You must have a thorough literature searching using (a). Appropriate engines, namely, Medline, EMBASE, EBM Review and CINAHL, (b). Appropriate strategies, namely correct key words related to your topic. To improve your searching efficiency, you should use Boolean Logic. Using the words "and", "or", and "not" will help refine your search.

"Feasibility" refers to the easy access of resources to complete the study. Retrospective study is easier to perform as compared to prospective study. "Interesting" refers to whether you or your seniors are interested to conduct this study. Ethical concern is important, for example, RCT only looks for better clinical effects. RCT will not be designed to look for harm or side effects comparison. "Relevant" refers to clinical impact by readers, as well as, journal reviewers. In simple terms, after I have read your articles, does it affect my practice? Do I learn something new and important clinically?

Remember FINER before you start. Ask opinions from experts in related field.

EBM tool……literature appraisal

EBM tool is the method we learn to appraise the Validity, Results, and Applicability of a study. EBM refers to Evidence-based Medicine. EBM refers to the integration of (a). Best update evidence, (b). Physician's clinical expertise and (c). Patient's own preference. We have 3 elements of scientific rigors which can stratify different study types and label them under "Pyramid of Evidence".

That include (a). Randomization, (b). The power to manipulate the independent factor, (c) The power to control the known and unknown confounders. The highest hierarchy is RCT (Randomized Control Trial), followed by prospective and retrospective cohort studies, case control study, case series, and personal experience.

Is PICO correctly addressed?

To start appraising an article, we look at PICO. For Population, it is crucial you have a detailed explanation of inclusion and exclusion criteria. Criteria cannot be too loose or too tight. Selection of subjects must be random. Recruitment is not subjectively chosen according to your own decision. Recruitment period needs to be a week, a month, 3 months, or a year interval. For Intervention study, are we Comparing a new treatment with a standardized and acceptable control arm with fairness? Outcome measures need to be objective and blind. Usually, the writer of original article has the full set of data. He can present the data in a dichotomous manner or continuous manner. The manipulation may make the final results look totally different!

Simple way to appraise RCT

To help colleagues to remember the EBM tool, we emphasis Mnemonic. For Validity of RCT, we just remember RAMBO. "R' stands for randomization method and concealment. "A" stands for ascertainment, which is related to dropout rate. "MBO" stands for blinded and objective measurement. We follow the "CONSORT", especially the flow diagram. "CONSORT" stands for Consolidated Standards of Reporting Trials.

For the result, we teach colleagues concepts of (a). Absolute results such as Absolute Risk Reduction and Number Needed to Treat, (b) Relative results such as Relative Risk, Relative Risk Reduction.

For applicability, we teach colleagues Mnemonic "5 killer Bs", namely, Biology, Barriers, Burden, Bargain, and Beliefs. In simple terms, biology refers to racial and sex differences, co-morbidity etc. Is there any barrier to get access to treatment? Is there any financial burden? We let patients to make a decision between benefits and harm. Do your patients believe?

Simple way to appraise Diagnostic test

For a diagnostic test study, we have 2 components, namely, a diagnostic test versus a gold standard. For example, Troponin I is used to diagnose myocardial infarction. And we use WHO criteria of myocardial infarction as a gold standard. At present, sensitivity and specificity are obsolete. We use Positive Likelihood Ratio, Negative Likelihood ratio, and ROC curve. To make life easy, we introduce just 2 bias in Validity of a diagnostic test study, namely, (a). Verification bias and (b). Spectrum bias.

Verification bias refers to the process of verification of effectiveness of diagnostic test by Gold standard. Are the two components independent from one another? Are the two components acceptable by us? Do the writer perform gold standard on all subjects?

Spectrum bias refers to a spectrum of disease severity. A good study will include mixed subjects with no disease, mild, moderate and extreme severity.

Simple way to appraise a meta-analysis

Meta-analysis refers a mathematical tool to sum up studies with smaller sample size and become a large sample size study. It aims to come to a conclusive result and avoid further trials. For validity concern, we must understand that there are biases in each step of doing the meta-analysis…..from recruitments of eligible studies, pooling of data, to options of random and fixed effect models in analysis.

Simple way to appraise a case control study

Case control study has low level of evidence. We first identify subjects with disease. Then we arbitrarily recruit subjects with no disease and act as control group. We look back years ago and recall whether the subjects have been exposed or not. For example, we identify subjects with lung carcinoma, and ask them if they smoke 5 years ago.

Hence we have 2 biases, namely, control bias and recall bias.(self-explanatory).

My advice to start doing your own trial….

Know your strength.

Think of retrospective trial first.

Remember FINER before you start.

Ask experts before you start.

Minimize biases in your study design.

Before and after study….an Audit cycle

Before and after study is common and welcome by journal reviewer. It shows the beneficial effects to patients. It is related to audit cycle. But first you must have a benchmarking for comparison.

Where to learn EBM tool?

EBM tool allows you to appraise a study with 3 pillars…namely, Validity, Results, and Applicability.

It does not dig into statistics. Leave the statistics to the statisticians. To learn EBM tool, didactic lectures are of no use. We emphasis small group tutorial. Practice makes perfect. After learning, you start your own journal clubs and appraise articles with your colleagues.

I recommend Singapore Asia-Pacific EBMN workshop. You can apply and enroll yourself into Basic group, Advanced group, and "How to teach" group. It is held annually by NUHS in February, conjoint with McMaster Gordon Guyatt.

My roadmap to set up Chinese Clinical Research Centre

Lastly, I have a dream. I hope I can promulgate EBM to mainland colleagues. In coming 5 years before my retirement, I would like to locate local champions in mainland and form a teaching group. I would like to set up "Chinese Clinical Research Centre". It may serve 3 goals…, namely, (a). To teach EBM tools to new comers and also, to train the trainers, (b). To assist writers on study design, statistics support and English translation, (c). To create our own trial registry.