Clinical Trials: the architecture of experimental research

Clinical Trials: the architecture of experimental research

Clinical Trials: the architecture of experimental research


“Where the value of a treatment, new or old, is doubtful, there may be a higher moral obligation to test it critically than to continue to prescribe it year-in-year-out with the support of merely of custom or wishful thinking. ” -FHK Green

The hierarchy of epidemiological research
Epidemiological studies are traditionally classified as either Observational or Experimental. Observational designs are considered weaker and less robust Observational study designs range from purely descriptive studies to extremely analytic designs like cohort studies. Experimental studies are those where the investigatormanipulates the intervention. The major advantage of experimental studies lie in the strength of causal inference that can be made. On the other hand, it is very difficult to make causal inferences based on observational studies. Also, experimental studies offer the best design for controlling confounding variables.

When to choose an experimental design?

Experimental studies are generally reserved for relatively “mature”research questions. A lot of ground work has to be done before embarking on a clinical trial. Experimental designs are chosen when:

  • the research question cannot be answered by observational studies
  • earlier observational studies have not answered the research question
  • existing knowledge is not sufficient to determine clinical or public health policy
  • an experiment is likely to provide an important extension of this knowledge

The sequence of clinical research

The critical first step in any clinical research is choosing the research question. Hulley offers the following criteria (FINER) for choosing a good research question [Hulley 1988]:

  • Feasible: in terms of resources, expertise, etc.
  • Interesting to the investigator
  • Novel: ideally the question should generate new data or confirm or refute earlier findings
  • Ethical
  • Relevant

The Randomized Controlled Trial (RCT)

The RCT is widely held as the ultimate study design; the “gold standard” against which all other designs are compared. The sequence of an RCT (parallel) design is shown below.

The subjects are usually chosen from a large number of potential subjects. Sampling includes prescreening using a set of inclusion and exclusion criteria. After this, an informed consent is obtained from each participant. Randomization is then done to allocate subjects to either the treatment group or the placebo group Randomization achieves two important things: allocation to different treatment groups is done without bias because it is taken out of bands of the investigator, and, importantly, randomization distributes known are unknown confounders equally between the two treatment groups. Thus a good baseline comparability is ensured between the two arms.

Once randomization is done, intervention is begun. Ideally, intervention should be done in a blinded fashion. Neither the investigator nor the subject should know the nature of the treatment that is being administered. After the intervention, the key outcomes that are being studied need to be measured by a blinded investigator Analysis involves looking for differences in the outcome rates in the two arms of the trial.

Thus, the core questions in a RCT are:

  • Is the trial justified?
  • Is the control group appropriate?
  • Is the allocation landomized?
  • Is the intervention blinded?
  • Is the outcome ascertainment blinded?
  • Is analysis by intention-to-treat principle?

Is the trial justified?

The first issue in any clinical trial is whether it is appropriate to do a trial at all. It is widely held that to justify a clinical trial there needs to be a state of equipoise. Freedman defines equipoise thus: “There is no consensus within the expert clinical community about the comparative merits of the alternatives to be tested” [Freedman B 1987] In other words, if the investigator is sure that the new therapy is better than the earlier one, then he/she is not justified in doing a trial. Equipoise, in this case, is disturbed by the fact that evidence favors the new treatment when compared with the earlier one. In such a state it would not be fair or ethical to subject one group of the trial to an inferior therapy.

Equipoise is both a fascinating and difficult concept. It is the responsibility of every clinician to prove (to an ethical committee or review group) that equipoise does exist before starting a trial. At times, equipoise can be disturbed even when the trial is midway. New evidence from other studies may settle the research question and disturb equipoise that existed at the time of the launch of the trial. In such situations, the trial needs to be terminated even before completion. Early termination of a trial can be a very difficult and painful decision to the trial investigators and the trial participants. Safety of the subjects and their interests are ultimately more important than the research study.

Is the control group appropriate?

A recent controversy will serve to highlight this issue. In a recent paper titled “Unethical trials of interventions to reduce perinatai transmission of the HIV in developing countries,” Lime and Wolfe [Lurie & Wolfe 1997] managed to raise a lot of dust in the research circles! According to them, many trials currently underway, which study the effect of the drug AZT to reduce perinatal transmission of HIV are unethical. In 1994, a trial called ACTG 076 clearly demonstrated the efficacy of AZT in reducing the incidence of HIV infection among babies born to HIV-positive mothers by two-thirds. After that study, the standard of care for HflV-positive pregnant women became the ACTG 076 regimen. Despite this, several studies (including those supported by major institutions like CDC and NIH) are being conducted in developing countries where several AZT regimes are being compared against placebo arms. According to Lurie and Wolfe, this is unethical because several mothers and babies are being denied AZT even though it is known that AZT is effective. They also argue that most of these trials will never be allowed in developed countries on ethical grounds. The arguments following their paper [Angell 1997, Varumus & Satcher 1997] are equally fascinating and a must read for those who are interested in the ethical issues of a clinical trial.

Can a placebo arm be justified when there is a therapy already available? Many authors argue that it is unethical to do a placebo controlled study when some therapy is already existent. No patient should be denied some form of therapy even if it is not very effective.

Mienert offers the following requirements for the test and control treatment[Mienert 1986]:

  • They must be distinguishable from one another
  • They must be medically justifiable
  • There must be an ethical base for use of either treatment
  • Either treatments must be acceptable to study patients and to physicians administering them
  • There must be reasonable doubt regarding the efficacy of the test treatment

There should be reason to believe that the benefits will outweigh the risks of the treatment

Is the allocation randomized?

Once the eligible subject has agreed to participate in the trial, it is important that assignment to treatment or control group is done in a manner that is free of any selection bias. To avoid bias, neither the patient nor the physician should be aware of the group to which the patient will be allocated. This is done by randomizing blinded fashion.

Randomization also ensures that the baseline characteristics of the test and the control groups are more or less similar in order to provide a valid basis for comparison. If allocation is not randomized, there is always room for suspicion: it is possible that subjects with favorable characteristics may be allocated to the treatment group while those with less favorable characteristics may be allocated to the control group.

Is Blinding achieved?

The aim of blinding is to ensure that outcome ascertainment is done without any bias. Blinding is logistically difficult but essential. Some authors use the word “masking” instead of blinding. A single-blinded trial in which the patient is not informed of the treatment assignment. A double-blinded trial is one in which neither the patient nor the physician responsible for the treatment is informed of the treatment assignment:

RCTs usually report the effectiveness of blinding. Sometimes, known adverse effects of drugs may unblind the physician (e.g. bradycardia due to beta blockers). Ideally, data collection, measurement, reading and classification procedures on individual patients should be made by persons who are completely blinded. For instance, if chest radiographs have to be read, the films can be sent to another site where they are read by radiologists who have no idea about the patients or their treatment groups.

As far as possible, outcomes chosen (end points) should be objective and clinically relevant. Outcomes should be capable of being observed in a blinded fashion. For instance, pain is a very subjective outcome and difficult to measure in a blinded fashion. On the other hand, if the outcome is a biochemical parameter, then it is objective and can be easily blinded.

Is analysis by intention-to-treat principle?

This is a very important issue in the analysis ofRCT results. All patients allocated to each arm of the treatment regimen are analyzed together as representing that treatment arm, whether or not they received or completed the prescribed regimen. Failure to follow this defeats the main purpose of randomization and can invalidate the results [Last 1995]. For instance, if a patient had been originally randomized to receive placebo, and if, for some reason, he actually ended up getting the study treatment, for the purposes of analysis, this patient will still be counted as belonging to the placebo group.

Ethical issues in a clinical trial

  • Is there equipoise to justify the trial?
  • Is informed consent obtained?
  • Is confidentiality protected?
  • Is the choice of control group justified?
  • Early stopping rule specified?

It is now imperative that all clinical trials be cleared by an ethical committee or Institutional Review Board (IRB). The issue of equipoise and choice of control group has already been discussed. Informed consent is another important issue. The potential participant should be made aware of the fact that he/she could end up getting only a placebo and the risks of adverse events or even death should be explained before obtaining the consent. Confidentiality also needs to be protected.

In some situations, the trial may need to be terminated early. In several trials, independent data monitoring and safety committees periodically analyse the results of the trial. If there is a significant difference between the treatment groups during the interim analysis, a decision to terminate the trial may be made. This is to prevent the participants from being exposed to an inferior form of therapy. Early termination may also be done if the frequency of adverse events or deaths is unacceptably higher in any of the arms.

Understanding the results of a clinical trial

Consider a clinical trial comparing a new antibiotic drug ‘Bactex’ for bacterial meningitis against the conventional antibiotic therapy (control group). The outcome of the trial is the mortality rate in each arm. These are dichotomous outcomes (alive or dead). At the end of the trial we would have the death rate among those who got Bactex, .and death rate among the control group. If the death rate among those who got Bactex is much lower than the rate among the control group, that would be a result in favor of the new drug.

Let us now assign numbers for this hypothetical trial. If the death rate among those who got Bactex was 20% and among the control group it was 40%, then these results could be presented in many ways:

Absolute Risk Reduction: ARR is simply the difference between the death rates: 40% minus 20% = 20%. Bactex reduced the risk of mortality by 20%.

Relative Risk Reduction: If 40% of the control group died, what fraction of this would have been prevented if the control group had received Bactex. This percentage is called Relative Risk Reduction (RRR).

RRR =       Outcome rate in control group – Outcome rate in the treatment group
                                                Outcome rate in the control group

RRR = 40% – 20% = 50% 40%

A RRR of 50% means that Bactex reduced the risk of mortality by 50% relative to that occurring among control group. The greater the relative risk reduction, the more effective the new therapy.

Number Needed to Treat

The Number Needed to Treat (NNT) to prevent one adverse event is a novel way of expressing results of trials. It attempts to overcome an inherent weakness in expressing results as RRR. For example, if the risk was reduced from 10% to 5% by Bactex, the RRR would still be 50%. In this situation, the overall death rate is quite low and this is not taken into account by the RRR. The NNT is nothing by the inverse of the ARR [NNT=l/ARR].

In the earlier case, the ARR was 20%. I divided by ARR would be the NNT. In this case it is 5. In other words, 5 patients would have to be treated to prevent one patient from dying. Consider the alternative scenario where the event rate is very low: 10% rate among controls versus 5% rate among the Bactex arm. In this case, the NNT is 20. So, 20 patients would now have to be treated to prevent one death. The lower the NNT, the more effective the new therapy is. All the above measures of outcomes can be summarised as a table:

Risk with conventional therapy (baseline risk) X Risk with new therapy Y Absolute Risk Reduction X-Y Relative Risk Reduction X-Y/X Number Needed to Treat 1/X-Y
40%- 20% = 20% 20%/40% = 50%
10%-5% = 5% 5%/10% = 50%

Precision of rates

All the above rates are only point estimates. Along with these rates, one would have to report the 95% confidence intervals to clearly express the significance of the results. 95% Cl can be calculated for all these measures: ARR, RRR, and NNT.

 Consider the scenario where we got a RRR of 50%. If the 95% Cl for this point estimate were to be-10%- 80%, then we would infer that if the trial were to be repeated several times, we could get an extreme result of minus 10% RRR (the new therapy performs worse than the conventional therapy). If we got such a wide 95%Cl, we would have to conclude that the new therapy is no better than the conventional therapy. The P value this case would not be significant.

In the other case, we got a point estimate of 50%, and if the 95% Cl were 40% to 60%, then would infer that even the worst performing trial would still give us a 40% RRR. This result could be reported as statistically significant [P value would be < 0.05], and the new drug would be considered significantly better than the conventional drug.

It is easy to appreciate that smaller the trial, wider would be the confidence interval. In very small trials, it is virtually impossible to get a statistically significant difference in the outcomes. The trial would not have adequate ‘power’ to pick up a genuine difference even if it truly exists.

The moral obligation to design a good trial

Experimental designs pose many dilemmas. On the one band, it may be unethical to introduce into general use a therapy or drug which is totally untested or poorly tested. As Sir Austin Bradford Hill put it “The ethical problem is, indeed, not solely one of human experimentation; it can also be one of refraining from human experimentation.” [Hill AB 1991]. On the other hand, to paraphrase Hulley, a clinical trial should not be undertaken when, because of the absence of randomization, blinding, or sufficient number of subjects, it is unlikely to provide a conclusive answer [Hulley 1988]. Indeed, it is important that a researcher embarking on a clinical trial make every effort to design the trial well and pay attention to all the core issues in the trial. It is quite common to see reports concluding that no inference could be made about the efficacy of the new treatment because of inadequate sample size! Why put human lives at risk, and spend a lot of resources when the research question is unlikely to be satisfactorily answered?

References & further reading

  1. Hulley SB et al. Designing Clinical Research. Baltimore: Williams & Wilkins, 1988.
  2. Freedman B. Equipoise and the ethics of clinical research. NEJM 1987;317:141-145.
  3. Lurie P, Wolfe SM. Unethical trials of interventions to reduce p-rinatal transmission of the HTV in develo countries. NEJM 1997;337:847-849.
  4. Angell M. The ethics of clinical research in the Third World. NEJM 1997,337:847-849.
  5. Vannus H, Satcher D. Ethical complexities of conducting research in developing countries. NEJM 1997,337:1003-1005.
  6. Mienert C. Clinical Trials. New York: Oxford University Press, 1986.
  7. Hill AB.Hill ID.BradfordHill’sPrinciples of MedicalStatistics. 12thEdition.BIPublications, 1991.
  8. Pocock SJ. Clinical Trials: A practical Approach. Chichester: John Wiley & Sons, 1983.
  9. Friedman LM et al. Fundamentals of Clinical Trials. Boston: Johns Wright, 1985.
  10. Rothman KJ, Greenland S. Modem Epidemiology, 2nd Edition. Philadelphia: Lippincot-Raven, 1998.

Exercise: Clinical Trials
1.In a field trial to measure the protective effect of a new influenza vaccine, parents of all children aged 6 months to 3 years living in 5 selected villages were contacted to obtain informed consent. The children whose parents agreed to participate were stratified into two groups based on the presence or absence of a school going child in the household. Children from each strata were then randomly allocated in a double blind (double masked) fashion during August 1998.
An epidemic of influenza reached this are during November 1998 and lasted till Jan 1999. All the children were visited once a week to obtain information on incidence of influenza.

The following were the results:

    • Number of children 6 months to 3 years in the 5 villages: 460
    • Number of children who enrolled for the study : 400
  Vaccine Group (n = 200) Placebo Group (n = 200)
Number of children who developed influenza
Number of deaths




    Why was the informed consent of the parents obtained? Is it necessary?

    The children were randomized after getting the informed consent. Would it have been better to get the consent after randomization?

    What do you understand by the term randomization? Why was it done?

    Why were the children stratified according to the presence of a school going child in the family prior to randomization?

    What do you understand by the term “double blind”? Why was it done?

    What was the incidence of influenza among the vaccinated group (exposed) = a

    What was the incidence of influenza among the unvaccinated group (unexposed) = b

    Calculate the protective effect of the vaccine: b-a xlOO b

    Calculate the relative risk (RR): RR=a/b=

    Calculate: (I-RR) X 100 = (protective effect)

    What was the overall case fatality rate?

    Was it ethical to give placebo to a group of children when one knows that influenza could be a fatal disease among young children?

    Study Designs: Review Questions

    From hospital charts of 40 patients diagnosed as having dengue fever, it was noted that a large proportion had fasted during the two-week period preceding onset of fever. To determine if fasting is associated with clinical dengue, data on a series of 40 patients with non-dengue febrile illness were collected. These patients were matched for age, sex and race to the dengue patients. The hospital charts of these patients were then reviewed to determine whether they also fasted prior to their illness. Classify the study described above as the following:

      1. cross-sectional study
      2. case-control study
      3. concurrent cohort study
      4. nonconcurrent cohort study
      5. randomized clinical trial
      • Mesothelioma is an extremely rare cancer in children. Risk factors for this disease in children are largely unknown. What kind of study design, using children, would be most appropriate for investigating the etiology and risk factors of childhood mesotheliomas:
          1. Cross-sectional study
          2. Acase-control study
          3. Concuirent cohort study
          4. Nonconcurrent cohort study
          5. Ecological study
      •  Which of the following is an advantage of a case-control study?
          1. There is little or no bias in assessment of exposure to the factor
          2. It is possible to study more than one exposure at a time
          3. Multiple disease outcomes following a selected exposure can be readily studied
          4. This study design is useful for studying rare exposures
          5. Recall bias is easily avoided
      •  In a study of all cases of a disease, if the relative risk for the association between a factor and the disease is equal to or less than 1.0 then:
          1. There is no association between the factor and the disease
          2. The factor protects against the development of the disease
          3. Either matching or randomization has been unsuccessful
          4. The comparison group used was unsuitable and a valid comparison is not possible
          5. There is either no association or a negative association between the factor and the disease
      •  Of 2872 individuals who had received radiation treatment in childhood because of enlarged thymus 24developed cancer of the thyroid and 52 developed benign thyroid tumor. A comparison group consisted of 5055 children who had received no such treatment (brothers and sisters of those children who had received radiation treatment). During the follow-up period, none of the comparison group developed Ca thyroid, but six developed benign thyroid tumors. Classify  the study described above as the following:
          1. Cross-sectional study
          2. Case-control study
          3. Concurrent cohort study
          4. Nonconcurrent cohort study
          5. Randomized clinical trial

    Calculate the relative risk for Ca thyroid:

      Calculate the relative risk for benign thyroid tumors:


          Dr. Madhukar Pai MD, DNB

            Consultant, Community Medicine & Epidemiology

              Email: [email protected]