What is Non-inferiority trial (NI)?

NI is designed to see whether a new test agent is Not Unacceptably less efficacious than a standard treatment with well-established historical data. With continuous updates in primary studies and clinical uses, incremental benefits of newly developed treatments may be marginal over current treatments. To design another 3-arms RCT of new test agent to a standard treatment, as well as, placebo group may be unethical and a waste of resources. In such circumstances, there has been increasing use of NIs. NIs are more complex to design, conduct, and interpret than traditional superiority trials.

Let's illustrate with a 3-arms diagram below.

Drug A is better than Placebo in treating a disease.

Drug A is a standard treatment with well-established historical data.As time goes, one may develop a new test agent Drug B.Drug B belongs to same drug group with similar efficacy as compared with Drug A.However Drug B is less expensive with less adverse effects.Shall we replace Drug A with Drug B? It is unfair and unethical to conduct a trial comparing Drug B vs.Placebo.We do not conduct a superiority trial of Drug B vs.Drug A to decide if Drug B is better than Drug A.In contrary, we conduct NI to say that that Drug B is not worse than Drug A.If the result is positive, we may consider to replace Drug A with Drug B.During the trial, we compare Drug B vs.Drug A only.There is no direct comparison of Drug B vs.Placebo.We call Drug A "Active control group", and Drug B "Test group".

Margin of Non-inferiority (-δ)

NI is a trial designed to determine whether the effect of a new agent Drug B is not worse than a standard treatment Drug A by more than a pre-specified amount (margin of non-inferiority, also called delta "δ").It is one-tail design.Trial is used to demonstrate Drug B has better secondary outcomes, e.g.simpler administration, less expensive or fewer side effects.Δ is the difference in effect between Drug B and Drug A.The null hypothesis "H0: Δ < -δ", simply speaking, Drug B is worse than Drug A.The lower bound of 95% CI of Δ extends below -δ.Alternate hypothesis "HA:Δ > -δ", it means Drug B is not worse than Drug A.The lower bound of 95% CI does not include the line of -δ.

The margin is arbitrary.It is based on 2 factors: (a) The active control effect vs.placebo, which is estimated from historical data, and, (b) Percentage of the minimum clinical effect to be given up.Non-inferiority is established when lower bound of 95% CI of the effect of the new drug vs.active control, do not fall outside the preset margin of -δ.

Suppose we have the following results:

Placebo group is 6 % (from historical data of Drug A vs.Placebo)

Drug A is 10 %.(NI of Drug B vs.Drug A)

Drug B is 8 %.(NI of Drug B vs.Drug A)

Is Drug B acceptable to you to replace Drug A, if Drug B have better secondary outcomes?

Is Drug B not worse than Drug A?

How do we set the Margin of NI (δ)?

An example:

Before we conduct NI, we must first accept that Drug A is a standard treatment.We use historical data from studies of Drug A vs.Placebo and get a risk difference of 4 % (95% CI of 2% to 6%).This result is accepted by others.Drug A is superior to Placebo.The minimum effect/lower bound of Drug A is 2%.

To set non-inferiority margin (δ), we need to consider how much of that 2 % we are willing to give up and document that Drug B is not worse than Drug A.If outcome is mortality, we may give up less.And (δ) will be smaller, and the Lines of (-δ) and (+δ) will both move closer to Line of No effect.The benefit of lower cost, ease of administration and less side effects are other factors.Consensus by other physicians is another consideration.If we give up 100%, δ < -2% means the same effect as Placebo!

If we arbitrarily declare that 50% of minimum effect should be maintained, the margin is set at δ < -1%.So to speak, the lower bound of risk difference of Drug B vs.Drug A should be greater than -1%.

If the final result of Drug B vs.Drug A turns out to be -2% (95% C.I.-4 to 0%), the lower bound is -4%.Drug B is inferior to Drug A because -4 % extends beyond -δ of -1%.Drug B fails to demonstrate non-inferiority to Drug A.

Non-inferiority trials have a number of inherent weaknesses.

1.Assay sensitivity is the property of a clinical trial to distinguish an effective treatment from a less effective or ineffective intervention.Without assay sensitivity, a trial is not internally valid.The trial may find an ineffective intervention to be non-inferior and could lead to erroneous conclusion of efficacy.The non-inferiority trials rely on use of relevant historical data of selected active control.However, the active control may not be widely accepted.The methodology may vary among trials.Besides, assay sensitivity drops with smaller (δ).

2.Because the new agent is not directly compared with placebo, the new agent and active control may be both equally beneficial or both equally not beneficial.

3.Margin of non-inferiority is arbitrary.

4.Nowadays many classic superiority trials recruited more than 10 000 subjects to prove the result.However, most NIs usually are small sample size studies (in terms of hundreds).Theoretically, to prove non-inferiority with small effect size, it requires larger sample size.

5.Study design and validity: "Intention to treat" or "Per protocol" analysis? Intention to treat is no longer conservative: any blurring of the difference between 2 groups will increase the chance of finding non-inferiority.Recruitment of subjects unlikely to respond, protocol violations, treatment crossovers, missing data, withdrawals & loss to follow up, tend to increase risk of rejecting H0 wrongly, i.e.bias towards NI & erroneous acceptance of possibly inferior treatments.Hence editor should present both ITT and Per-protocol results together.

6.Biocreep is a cyclical phenomenon/possible risk that with more and more on-going NIs, new drugs that are no more effective than a placebo, may finally be allowed to creep to market for sale.

End.