A Review of the Use of the Number Needed to Treat to Evaluate the Efﬁcacy of Analgesics

Home , Number needed to harm, Number needed to treat

The Journal of Pain, Vol 16, No 2 (February), 2015: pp 116-123 Available online at www.jpain.org and www.sciencedirect.com

Nathaniel Katz, Florence C. Paillard, and Richard Van Inwegen Analgesic Solutions, Natick, Massachusetts.

Abstract: Standardized measures of efficacy are needed to compare analgesic efficacy across trials. The number needed to treat (NNT) is considered a statistically robust and readily interpretable measure to rank the efficacy of treatments, including analgesics. The NNT has become widely utilized to compare the efficacy of chronic pain treatments, helping physicians make treatment decisions and informing decisions for market access, reimbursement, and treatment guidelines. However, the NNT is associated with specific weaknesses in calculation and interpretation not associated with other methods for integrating trial data. These weaknesses include distortions in calculation as placebo effects approach treatment effects, with the possibility of infinite values; difficulties in estimating the NNT’s confidence interval; and difficulties in interpretation. The NNT also requires selecting cutoffs of the original variable for dichotomization, with the NNT often changing depending on the cutoff. The NNT also suffers from problems common to other placebo-adjusted endpoints, including being sensitive to study-related and external factors (eg, year of publication). Therefore, clinicians and other stakeholders need to be aware of these issues to correctly calculate, use, and interpret the NNT. Nevertheless, efficacy, as measured by any variable, is only one aspect of a treatment to be considered in determining its place in therapy. Perspective: The NNT has become widely utilized to compare the efficacy of chronic pain treatments. This article reviews the uses of the NNT and the potential problems associated with its calculation, use, and interpretation. Clinicians should be aware of these issues when interpreting clinical trial data based on the NNT. ª 2015 by the American Pain Society Key words: Number needed to treat, analgesics, efficacy, clinical trials, chronic pain treatment.

he number needed to treat (NNT) was devised in mean value for the primary endpoint. The group mean 1988 by Laupacis et al29 as a single unitary measure difference is often difficult to interpret in a clinically Tof a drug’s efficacy that was meant to provide an intuitive manner. Thus, to allow easier interpretation of intuitive means for evaluating the relative efficacy of clinical data, the NNT defines each patient as either a different drugs in order to rank them as to their effi- responder or a nonresponder (based on some predefined cacy.33 Like any other method that evaluates the relative definition of response) and compares the proportion of efficacy of treatments, the NNT is dependent on responders in each group. comparing data across randomized double-blind controlled trials, the gold standard in clinical research. The efficacy of the drug being studied is measured as in- Definition and Methods of Calculation of cremental benefit above that in the placebo group and is the NNT typically quantified by the difference between groups in The NNT is interpreted as the number of patients one would need to treat in order to get one more responder on the active treatment than one would have gotten had The authors received a grant from Janssen Pharmaceuticals to support they been treated with control. In technical terms, the independent writing of this review. The authors declare having no financial relationship to the work. N.K. NNT is the inverse of the absolute risk difference (ARD): and R.V.I. are employees and F.C.P. is a contractor of Analgesic Solutions. ¼ 1 : NNT ARD Address reprint requests to Florence C. Paillard, PhD, Analgesic Solutions, ARD is the difference in proportion of patients who 232 Pond St, Natick, MA 01760. E-mail: [email protected] 1526-5900/$36.00 manifest a response to a treatment and the proportion ª 2015 by the American Pain Society of patients who manifest a response to control. It is http://dx.doi.org/10.1016/j.jpain.2014.08.005 possible to use placebo, nontreatment, or another active

116 Katz, Paillard, and Van Inwegen The Journal of Pain 117 treatment as the control to calculate the NNT. The choice the lives of countless patients. Reliance on the NNT for of controls has a huge impact on the NNT values and this purpose is based on presumptions of methodologic their interpretations. Thus, any comparison of NNT robustness and straightforward interpretability. values must use the same controls for calculations; Herein, we conducted a qualitative review of the comparing NNT values calculated with different control advantages and limitations of the NNT to assess whether groups would not be valid. Therefore, the discussions the NNT is suitable for evaluating the comparative effi- in this review are limited to comparisons to placebo con- cacy of analgesics for chronic pain. trols, unless otherwise so stated. Most often, the response used to calculate the ARD is an improvement Critiques of the NNT (eg, reduction in pain), in which case ARD is calculated as ‘‘response with drug minus response with placebo.’’ Critiques of NNT have been grouped into 6 major Responders, patients who manifest a response, are categories (Table 1). A number of these critiques (both defined as patients who meet a predefined criterion of positive and negative) were not specifically inherent to response in an all-or-nothing fashion (ie, death/life). How- the NNT, but could be applied to any comparative mea- ever, for most uses, a nondichotomous endpoint is used sure of efficacy such as the ARD and OR. The fact that (eg, pain intensity score), and a predefined response crite- the NNT can be impacted by study design (eg, study rion is created (eg, having a $30% pain reduction or not). size, number of arms, type of comparator) and subject’s The following is an example of calculating the NNT. If characteristics (eg, indication, severity, and duration of 14,31,32,48,51 half (50%) the patients on active treatment respond disease; reviewed in references ) is not too (response rate = .5), and one quarter (25%) of the surprising as these factors influence the response rate, placebo-treated patients respond (response rate = .25), which forms the basis of the NNT definition. These then the ARD is .5.25 = .25. The NNT is 1/.25 = 4. This variables can also influence other comparative can be interpreted as 4 patients would have to be treated measures. Thus, critiques of the NNT were categorized with the treatment to get 1 more responder than with based on whether they are specific to the NNT or placebo. In other words, treating 4 patients with treat- nonspecifically applicable to any placebo-adjusted ment would yield 2 responders, whereas treating 4 efficacy measure used in meta-analysis. patients with placebo would yield only 1 responder. The NNT can be also calculated using the odds ratio Critiques Specific to the NNT (OR) or the relative risk reduction (RRR) (reviewed in33). The optimal NNT value is 1, whereby every patient has Issues Associated With Calculating the NNT a positive response to treatment and no patient responds Calculating the Combined NNT From Various Trials Can to placebo. When a drug produces an identical effect to Be Subject to Bias. Two methods have been proposed placebo, the NNT would have a value of infinity (ARD = 0 and used to calculate the NNT from several clinical trials. and NNT = 1/0). Thus, in theory, the NNT can vary from 1 One method uses a meta-analytic method wherein each to infinity. It is also possible for the NNT to have a nega- trial is handled separately, and the other treats the data tive value when the response rate for placebo is greater as if arising from a single trial.2,6 The latter method is than that for the drug. In practice, the NNT is usually used prone to bias, especially when there is an imbalance in to compare effective drugs, so negative and infinity sample size between treatment and placebo arms, a values are not usually reported because the drug is phenomenon called the Simpson’s paradox. The meta- considered ineffective. Thus, the lower (ie, the closer to analytic method is not prone to this issue. In meta- 1) the NNT, the more efficacious the drug. analyses, the use of a relative measure such as OR or risk ratio (RR) is advocated because event rates vary considerably from study to study even for the same Uses of the NNT drug.6 The method used to calculate the combined NNT The NNT is usually used to compare the efficacy of from various trials should be clearly stated. different drugs for the same indication to enable physi- The NNT Can Have an Infinite Value. Unlike other cians to make informed choices in clinical practice. To measures of efficacy (eg, ARD), the NNT is an inversion allow comparison and ranking of the efficacy of different of a response (ie, 1/ARD) so that a zero difference treatments, many systematic reviews and meta-analyses between active and placebo groups results in an unde- of randomized controlled trials provide average NNT fined number (ie, 1/0).18,26,43 This could be further values for various treatments across an indication (the complicated if the treatment in one study is less NNT being treatment- and dose-specific).33 For instance, efficacious than placebo, resulting in a negative number. a meta-analysis evaluating the efficacy of analgesics Having an infinite or negative value renders ranked the drugs’ efficacy by NNT value (the lower the comparing studies and calculating the combined NNT NNT, the higher the efficacy) as follows: efficacy of for several trials difficult.2,8,22,30,50 As stated above, ibuprofen 400 mg (NNT = 2.8) > paracetamol (600/ using a meta-analytic methodology based on the OR or 650 mg) (NNT = 5.0) > codeine 60 mg (NNT = 18).34 RR is recommended.6 The NNT has become widely utilized for comparing In some systematic reviews, the NNT is considered treatment efficacy for chronic pain and other conditions, ‘‘nonsignificant’’ when it is high or infinite,16 which which informs decisions for market access, reimburse- means that the treatment has no therapeutic value. ment, and position in treatment guidelines, which affect However, the cutoff value for ‘‘high’’ is arbitrary; the 118 The Journal of Pain Uses and Misuses of NNT to Rank Analgesics Table 1. Critiques of the NNT

TYPES OF CRITIQUES STUDIES Critiques specific to the NNT Issues associated with calculating the NNT: Suissa,49 Stang,48 Hutton,22 Hutton,23 Thabane,50 The NNT can have an infinite value, creating problems for meta-analyses and Ebrahim,13 Hutton,24 Lesaffre,30 Newcombe,37 causing a disproportionate impact of failed versus successful studies Newcombe,38 Smeeth,47 Altman1,2 The precision of the NNT is difficult to estimate The NNT has a skewed CI The NNT is difficult to interpret Christensen,8 Ebrahim,13 Smeeth47 Critiques of the NNT applicable to other endpoints used in meta-analyses The NNT is sensitive to the level of efficacy in the placebo group Moore 33 The NNT depends on the endpoint: Suissa,49 Stang,48 Citrome,9 McAlister,32 Moore,35 The NNT is dependent on the selection of outcome measured Christensen,8 Mayne,31 Tramer,51 Wu,53 Ebrahim13 The NNT is dependent on the cutoff chosen for the dichotomous outcome The NNT is dependent on the time point of outcome The NNT depends on factors internal and external to the study aside Suissa,49 Stang,48 Citrome,9 McAlister,32 Moore,35 from study drug Christensen,8 Mayne,31 Tramer,51 Wu,54 Ebrahim13 NNT does not reflect effectiveness of treatment in clinical practice Tramer,51 Smeeth,47 Black,4 Dowie10

therapeutic value or lack thereof of a therapy with a high mean differences, RRR, OR, and absolute risk reduction NNT would be dependent on the therapeutic area and (ARR).33 availability of successful treatments with lower NNT However, several studies using a randomized design values. In an indication with limited treatments (eg, to survey the understanding of NNT among doctors,21,39 visceral pain), a ‘‘high’’ number would be more accept- medical students,43 patients,44 and laypersons20,28 have able than in a pain indication with multiple options. shown that they have difficulty understanding the The Precision of the NNT Is Difficult to Estimate. A meaning of the NNT. One survey of first-year medical measure of drug effect size should have a definable students showed that treatment benefit was correctly precision, such as a 95% confidence interval (CI). When interpreted by $75% of the students when presented the NNT is very high or infinite, its 95% CI cannot be accu- as RRR or ARR, but only by 30% of students when pre- rately calculated.1,2,8,22,30,50 For instance, if the ARD is .01 sented as the NNT.43 A survey of Norwegian medical and its 95% CI is [.01 to 1.03], the NNT is 100 (1/.01) but doctors showed that many believed that only 1 of NNT its 95% CI is not [100 to 133.3]. In this example, the patients would benefit from treatment (eg, if the NNT 95% CI would be mathematically 2 intervals, one at were 4, then 1 of 4 patients would benefit).21 Patients [N, 100] and another at [133.3, 1N], because the who were presented with clinical data in terms of ARD 95% CI contains zero, which is very difficult to RRR, ARR, and NNT were best able to interpret the ben- interpret.8 Therefore, in such circumstances, it has been efits of treatment when it was presented as RRR or recommended that the NNT be given only as a point ARR.44 A study showed that a large majority of people estimate (ie, 100 in this example) without the 95% CI, would accept using the hypothetical treatment pre- which avoids the mathematical problem but does not sented to them regardless of its NNT, and that those fulfil the desire for an estimate of precision. who declined using the medication misinterpreted the The NNT Has a Skewed CI. The NNT is calculated by NNT.28 In another study, laypersons presented with the inverting the ARD and its CI by inverting the ARD’s CI benefit of a hypothetical osteoporosis intervention limits. This leads to an uneven distribution of the NNT’s and offered the therapy were sensitive to the magni- CI, whereby the upper bound of the CI is artificially ‘‘in- tude of treatment benefit when it was presented in flated.’’2,50 For instance, if the ARD is .1 times the 95% terms of postponement of hip fracture, but not in terms CI: [.05–.15], the NNT is 10 times the 95% CI: [6.7–20]. of NNT.7,20 Therefore, the NNT value is further from the upper A search for definitions of the NNT on the Internet bound of the CI than from the lower bound of the CI.2 shows that the NNT is commonly misinterpreted as the To provide a real example, the NNT for 60 mg etoricoxib number of patients one needs to treat to get one to produce at least 50% pain relief at 6 weeks compared responder. with placebo was 2.6 [95% CI: 2.0–3.9], showing an ‘‘inflation’’ of the upper bound of the 95% CI.35 However, this anomaly is not unique to the NNT and has been Critiques of the NNT Applicable to Other observed with the OR, albeit for different reasons. Endpoints Used in Meta-Analyses The NNT Is Difficult to Interpret The NNT Is Sensitive to the Level of ‘‘Efficacy’’ Since its inception in 1988, the NNT has quickly become in the Placebo Group a widely used measure of a drug’s efficacy because it was An important asset of the NNT is that it takes into thought to be a more intuitive measure of a drug’s effi- account the response observed in the placebo group, cacy than other measures of efficacy such as group thereby allowing for the adjustment for nonspecific Katz, Paillard, and Van Inwegen The Journal of Pain 119 Table 2. Impact of the Placebo Response on Table 3. Impact of the Placebo Response on NNT—Theoretical Example No. 1 NNT—Theoretical Example No. 2

%RESPONDERS %RESPONDERS %MORE %RESPONDERS IN ACTIVE IN PLACEBO RESPONDERS IN IN ACTIVE %RESPONDERS IN TREATMENT GROUP GROUP DELTA NNT ACTIVE GROUP* TREATMENT GROUP PLACEBO GROUP DELTA NNT ARD 10 0 10 10.0 100.0 40 5 35 2.9 .35 20 10 10 10.0 50.0 40 10 30 3.3 .30 30 20 10 10.0 33.3 40 20 20 5 .20 40 30 10 10.0 25.0 40 30 10 10 .10 50 40 10 10.0 20.0 40 40 0 N 0 60 50 10 10.0 16.7 70 60 10 10.0 14.3 80 70 10 10.0 12.5 The fact that the NNT varies more widely than the ARD is illustrated in examples of clinical trials noted in *Calculated as (% responders in active group) (% responders in placebo Table 4. This table shows trials from anticonvulsant group) / % responders in active group. analgesics with very different NNTs (presented in order of lowest to highest NNT) and presents the ARD. In these examples, the NNT varied from 1.4 to infinity, factors that may be associated with non–treatment-spe- whereas the ARD only varied from .704 to 0. This is cific benefits in a clinical trial, such as nonspecific again a consequence of the difference between the response, spontaneous improvements/worsening of active and placebo groups in the proportion of patients the disease, natural history, and regression to the improving. mean. To this end, the NNT is intending to serve as a measure of the pharmacologic benefit of a drug over The NNT Depends on the Endpoint and above nonspecific factors that may be associated To calculate the NNT, continuous data need to be with benefits in a clinical trial. Other placebo-adjusted converted into dichotomous data; for instance, pain measures such as group mean differences, ORs, and scores over time need to be converted into change in RRs are also commonly used to ‘‘normalize’’ the incre- pain score from baseline to endpoint, and then con- mental pharmacologic benefit of a drug in the context verted to a response rate using a predefined criterion; of a clinical trial. for example, proportion of subjects having an X% pain As with other comparative measures of efficacy that reduction from baseline to endpoint. Therefore, to calcu- rely upon the difference between drug and placebo, late the NNT of an active drug in a trial, one needs to the absolute level of placebo response can have an select 1) the derived outcome measure (eg, % re- impact on interpretation. For example, a treatment sponders); 2) the cutoff for the definition of response with a high absolute benefit, but a nearly as high (eg, 30% reduction in pain); and 3) the time at which benefit in the placebo group, will appear highly effec- the observation of response/nonresponse is made (time tive in clinical practice but will have an unfavorable point). Several articles have shown that the NNT is NNT. A treatment with a more modest absolute benefit, dependent on the outcome measured,11,28 the time but a much lower response in the placebo group, will point of measurement,6,11,44,45 and the cutoff value of appear less effective in clinical practice despite a more response.31 favorable NNT. This is illustrated in the theoretical The NNT Is Dependent on the Selection of Endpoint example illustrated in Table 2, where the NNT of Measured. Comparison of the NNT for total mortality, different treatments remains constant, but the likely mortality, and all cardiovascular events in various statin clinical interpretation and desirability of treatment trials showed that the NNT value was different for each does not. of these outcome measures.11 In an analgesic trial, it Another theoretical example to consider is where the seems obvious that the NNT for reduction in pain will not response to drug remains constant but the response to be the same as the NNT for improvement in sleep. placebo varies, as is often observed between similar Moore’s review36 of pregabalin efficacy presented the studies of the same treatment (see examples below NNT for 2 outcomes: the proportion of subjects with and reference12). In Table 3, the drug’s response was $30% or $50% pain reduction (% responders) and the held constant at 40% (a typical response rate observed level of improvement in Patient’s Global Impression of for analgesics) and the placebo response varied from Change (Fig 1). The NNT for the level of improvement in 5% to 40%. This example shows that the NNT dramat- Patient’s Global Impression of Change was systematically ically increases with increasing placebo response, higher than the NNT for the proportion of responders reaching infinity when the response in the placebo (regardless of the cutoff), although all 3 sets of NNT group is equal to that in the active group (Table 3). values yield the same conclusion.36 Although the NNT for these theoretical examples var- The NNT Is Dependent on the Cutoff Chosen for the ied from 2.9 to infinity, the ARD only varied from .35 Dichotomous Outcome. Transformation of continuous to 0, illustrating how the NNT appears to be more data into dichotomous data, a process necessary to calcu- unstable than the ARD in the face of variable placebo late the NNT, reduces the statistical estimation of out- responses. comes and results in an important loss of information. 120 The Journal of Pain Uses and Misuses of NNT to Rank Analgesics Table 4. NNT Varies More Widely Than ARD—Examples of Clinical Trials of Anticonvulsants in Neuropathic Pain

NO.(%)OF PATIENTS WHO IMPROVED* NO. (%) OF PATIENTS WHO IMPROVED* REFERENCE ANALGESIC DRUG WITH ACTIVE TREATMENT WITH PLACEBO NNT ARD Killian27 Carbamazepine 19/27 (70.4) 0/27 (0.0) 1.4 .704 Zakrzewska55 Lamotrigine 7/13 (53.8) 1/14 (7.1) 2.1 .467 Gilron19 Gabapentin 27/44 (61.4) 13/42 (31.0) 3.3 .304 Simpson45 Gabapentin 15/30 (50.0) 7/30 (23.3) 3.8 .267 Simpson46 Lamotrigine 86/150 (57.3) 30/77 (39.0) 5.4 .183 Otto40 Valproate 8/31 (25.8) 3/31 (9.7) 6.2 .161 Raskin41 Topiramate 74/214 (34.6) 23/109 (21.1) 7.4 .135 Drewes11 Valproate 6/20 (30.0) 4/20 (20.0) 10.0 .10 Serpell42 Gabapentin 32/153 (20.9) 21/152 (13.8) 14.1 .072 Finnerup17 Lamotrigine 4/21 (19.0) 4/21 (19.0) N .0

NOTE. Data are from Finnerup et al.16 *Response was based on 50% pain relief; if 50% pain relief could not be obtained directly from the publication, then the number of patients reporting at least good pain relief or reporting improvement was used to calculate the NNT.

Thus, if the NNT was to become the measure of choice in typical cutoffs of 20%, 30%, and 50% of pain relief, clinical trials, study design may evolve toward the use of respectively (Table 5). Thus, as with any comparative dichotomous measures (such as response rate) as primary measure, the selection of the cutoff chosen is critical endpoints, which would reduce studies’ power (or lead and the same exact cutoff needs to be used for to increased sample size).15 comparing data from different studies. Dichotomizing the data requires that the analyst The NNT Is Dependent on the Time Point of decide on a cutoff, which can be somewhat arbitrary; Outcome. The NNT depends on when the outcome is not all studies use the same definition for the cutoff measured. First, it is expected that the response to treat- point to determine the number of responders (most ment will vary over time: the response rate may be very studies use the 30% threshold, but others use a 20% different at 1 week and at 4 weeks of treatment, and or 50% threshold). Dichotomization separates patients so will the NNT calculated at these 2 time points. This into responders and nonresponders, losing the nu- issue renders difficult the comparison of NNT from ances of partial responders (those close to the different studies with various lengths, which is typical threshold). For instance, a subject with a 31% response in chronic indications.31 will have a preferential impact over one that has a The 3 issues mentioned above are not unique to the 29% response. NNT, and any comparative measure of efficacy is likely The example illustrated in Table 5 shows that the NNT to be affected by the type of outcome considered, the is dependent on the cutoff of the outcome: the NNT for time point of the outcome, and the cutoff chosen. the 30% pain reduction cutoff is systematically lower However, what is unique to the NNT is the mechanism than that for the 50% cutoff. In another example, the by which one has to dichotomize the selected NNT seems to increase with increasing cutoffs of percent- endpoint. Some endpoints do not lend themselves ages of response, especially for the lower doses easily to simple dichotomization into responder or (Table 5)35; indeed, for the 5-mg dose, the NNT jumps nonresponder. from 4.0 to 5.3 to 12.5 (quadruples), and for the 10-mg dose it jumps from 3.7 to 5.9 to 7.1 (doubles) for the The NNT Depends on Internal and External Factors Aside From Study Drug Like all other measures of efficacy in clinical trials, the NNT is affected by factors internal to the study (study design and conduct) and factors external to the study (real-world factors).

Table 5. NNT Varies With the Cutoff for the Dichotomous Variable (% Pain Relief)

ETORICOXIB NNT FOR 20% NNT FOR 30% NNT FOR 50% DOSE (MG) PAIN RELIEF PAIN RELIEF PAIN RELIEF 5 4.0 5.3 12.5 10 3.7 5.9 7.1 30 3.6 4.0 4.3 60 2.3 2.4 2.6 Figure 1. NNT for pregabalin calculated from various outcome measures (data from Moore et al36). NOTE. Data are from Moore et al.35 Katz, Paillard, and Van Inwegen The Journal of Pain 121

Figure 2. NNT for analgesics by year of publication (data from Finnerup et al16). Left panel: tricyclic antidepressants; right panel: anticonvulsants. Each point of data represents 1 clinical trial.

The NNT Depends on Factors Internal to the Study. A group mean differences. However, this phenomenon few articles have reviewed the study factors that can results in more distortion of the NNT over time compared impact the NNT12,14,25,26,31,32,48,51 and we refer the to alternative measures of efficacy (as discussed earlier). reader to these articles for further explanations. Briefly, Another, perhaps more obvious, reason for the increase these factors include in NNT over time is that the experimental conditions (study design and patient population) have evolved over 1. The study population, as the type of pain indication time, rendering the comparison of older and recent and severity of condition at baseline influence the studies inappropriate. Indeed, more recent trials, for response to active treatment12 reasons that are not entirely clear, have greatly increased 2. The study design, for example, number of sites12 placebo responses compared to older trials.10 3. The comparator: controls used as comparators have huge impact on numbers generated and interpreta- Use of the NNT in Clinical Practice tions; possible controls include a true placebo, an active placebo, no treatment, or another active The NNT has been used as a yardstick to compare the treatment3,53 efficacy of various drugs, to help clinicians choose 4. The time period (as discussed in the previous between active treatments, inform treatment guide- section) lines, determine market access and reimbursement, and 5. The type of outcome (as discussed in the previous support risk-benefit analyses. section) But as placebo is not given in clinical practice, placebo- adjusted metrics like the NNT have limited usefulness in The heterogeneity of trials’ data makes pooling NNT clinical practice and can be irrelevant to real-world effi- data across studies clinically irrelevant.14 Thus, if the cacy3 and misleading.43 For example, in the clinic, a physi- NNT is to be used to compare treatments, the therapies cian facing the choice between 1) Treatment A with an must have been tested in similar populations with the NNT of 10 that produces a response in 80% of patients same condition at the same stage, using the same (but a response in 70% of placebo patients) or 2) Treat- comparator, time period, and outcomes.28 Therefore, ment B with an NNT of 2.9 that yields a response in 40% comparisons of analgesics using the NNT that fail to of active-treated patients (but 5% in the placebo group) follow these guidelines may not be valid. Approaches would likely choose Treatment A because it shows greater to calculation of the NNT that adjust for such con- efficacy in a large proportion of patients—which is what founders have been described; using an NNT model to the clinician is looking for—even though it has a higher account for differences in study design and conduct NNT. Thus, if one wants to measure efficacy in a clinical allows for more meaningful comparisons.5 trial (as defined by the incremental benefit of the drug’s The NNT Depends on Factors External to the Study. pharmacology over the nonspecific effects of being in the External factors have been cited as potential influencers trial), one needs a placebo-adjusted robust measure like of observed net treatment effects. The year of publica- the NNT. However, to measure efficacy in the real world, tion is one such factor that may stand as a proxy for the absolute response rate and similar unadjusted mea- multiple other influences (eg, availability of other treat- sures may provide a more relevant perspective. ments, approval status of the treatment, expectation of Moreover, the NNT taken alone does not summarize all treatment benefit, and shifts in sources of patients). A necessary information for the clinician to make informed valid measure of treatment effects should be robust to decisions regarding treatment.10,51 Indeed, a physician factors other than the treatment itself. The NNT for anti- not only considers the efficacy of the drug but also takes depressant analgesics as well as for anticonvulsant anal- into account the rate and seriousness of adverse effects gesics has steadily increased (the efficacy had gotten of each treatment, the patient’s preference, cost, and worse) over time (Fig 2; reference18). This could be clinical experience (if any) with the treatments. Efforts attributed to the fact that in antidepressant trials, are underway to create quantitative metrics that placebo response increased more over time (year of integrate these factors. Thus, decisions for use in the real publication) than the response to active treatment.52 world need to be based on multiple pieces of Increasing placebo response over time relative to the information, not just a solitary measure of efficacy drug will distort any measure of efficacy, including the compared to placebo. 122 The Journal of Pain Uses and Misuses of NNT to Rank Analgesics Conclusions selecting cutoff points for dichotomization, with the NNT often changing depending on the cutoff. The NNT Standardized measures of efficacy are needed to suffers from problems common to other placebo- compare analgesic efficacy across trials. The NNT was adjusted endpoints, including being sensitive to study- developed to be a statistically robust and readily inter- related and external factors (eg, year of publication) pretable measure to rank the efficacy of treatments, that, when not considered, undermine the validity of including analgesics. However, the NNT is associated the NNT as a tool for comparing different treatments in with specific weaknesses in calculation and interpreta- different studies. Approaches to improving the validity tion that are not associated with other methods for inte- of the NNT (and other summary measures) by adjusting grating data from multiple trials. These weaknesses for such factors deserve further exploration. Finally, effi- include distortions in its calculation as placebo effects cacy alone, as measured by any measure, is only one part approach treatment effects, with the possibility of infin- of the profile of a treatment to be considered in deter- ite values; difficulties in estimating the precision of the mining its place in therapy. Therefore, clinicians and NNT particularly for CI calculation; and, contrary to the other stakeholders need to be aware of these issues to original intent, difficulties in interpretation. The NNT correctly calculate, use, and interpret the NNT. requires manipulation of the original variable by

References Hertz S, Jay GW, Junor R, Kerns RD, Kerwin R, Kopecky EA, Lissin D, Malamut R, Markman JD, McDermott MP, Munera C, Porter L, Rauschkolb C, Rice AS, Sampaio C, 1. Altman DG: Conﬁdence intervals for the number needed Skljarevski V, Sommerville K, Stacey BR, Steigerwald I, to treat. Br Med J 317:1309-1312, 1998 Tobias J, Trentacosti AM, Wasan AD, Wells GA, Williams J, Witter J, Ziegler D: Considerations for improving assay sensi- 2. Altman DG, Deeks JJ: Meta-analysis, Simpson’s paradox, tivity in chronic pain clinical trials: IMMPACT recommenda- and the number needed to treat. BMC Med Res Methodol tions. Pain 153:1148-1158, 2012 2:3, 2002 13. Ebrahim S: The use of numbers needed to treat derived 3. Backonja M, Wallace MS, Blonsky ER, Cutler BJ, Malan P Jr, from systematic reviews and meta-analysis. Caveats and Rauck R, Tobias J: NGX-4010 C116 Study Group: NGX-4010, a pitfalls. Eval Health Prof 24:152-164, 2001 high-concentration capsaicin patch, for the treatment of postherpetic neuralgia: a randomised, double-blind study. 14. Edelsberg J, Oster G: Summary measures of number Lancet Neurol 7:1106-1112, 2008 needed to treat: How much clinical guidance do they provide in neuropathic pain? Eur J Pain 13:11-16, 2009 4. Black HR, Crocitto MT: Number needed to treat: Solid science or a path to pernicious rationing? Am J Hypertens 15. Fedorov V, Mannino F,Zhang R: Consequences of dichot- 11:128S-134S, 1998. discussion 135S-137S omization. Pharm Stat 8:50-61, 2009 5. Caro JJ, Ishak KJ, Caro I, Migliaccio-Walle K, Klittich WS: 16. Finnerup NB, Otto M, McQuay HJ, Jensen TS, Sindrup SH: Comparing medications in a therapeutic area using an Algorithm for neuropathic pain treatment: An evidence NNT model. Value Health 7:585-594, 2004 based proposal. Pain 118:289-305, 2005

6. Cates CJ: Simpson’s paradox and calculation of number 17. Finnerup NB, Sindrup SH, Bach FW, Johannesen IL, needed to treat from meta-analysis. BMC Med Res Methodol Jensen TS: Lamotrigine in spinal cord injury pain: A random- 2:1, 2002 ized controlled trial. Pain 96:375-383, 2002

7. Christensen PM, Brøsen K, Brixen K, Andersen M, 18. Finnerup NB, Sindrup SH, Jensen TS: The evidence for Kristiansen IS: A randomized trial of laypersons’ perception pharmacological treatment of neuropathic pain. Pain 150: of benefit of osteoporosis therapy: Number needed to treat 573-581, 2010 versus postponement of hip fracture. Clin Ther 25: 2575-2585, 2003 19. Gilron I, Bailey JM, Tu D, Holden RR, Weaver DF, Houlden RL: Morphine, gabapentin, or their combination 8. Christensen PM, Kristiansen IS: Number-needed-to-treat for neuropathic pain. N Engl J Med 352:1324-1334, 2005 (NNT)—Needs treatment with care. Basic Clin Pharmacol Toxicol 99:12-16, 2006 20. Halvorsen PA, Kristiansen IS: Decisions on drug therapies by numbers needed to treat: A randomized trial. Arch Intern 9. Citrome L: Compelling or irrelevant? Using number Med 165:1140-1146, 2005 needed to treat can help decide. Acta Psychiatr Scand 117: 412-419, 2008 21. Halvorsen PA, Kristiansen IS, Aasland OG, Førde OH: Medical doctors’ perception of the ‘‘number needed to 10. Dowie J: The ‘‘number needed to treat’’ and the treat’’ (NNT). Scand J Prim Health Care 21:162-166, 2003 ‘‘adjusted NNT’’ in health care decision-making. J Health Serv Res Policy 3:44-49, 1998 22. Hutton JL: Misleading statistics: The problems surround- ing number needed to treat and number needed to harm. 11. Drewes AM, Andreasen A, Poulsen LH: Valproate for Pharm Med 24:145-149, 2010 treatment of chronic central pain after spinal cord injury. A double-blind cross-over study. Paraplegia 32:565-569, 1994 23. Hutton JL: Number needed to treat and number needed to harm are not the best way to report and assess the results 12. Dworkin RH, Turk DC, Peirce-Sandner S, Burke LB, of randomised clinical trials. Br J Haematol 146:27-30, 2009 Farrar JT, Gilron I, Jensen MP, Katz NP, Raja SN, Rappaport BA, Rowbotham MC, Backonja MM, Baron R, 24. Hutton JL: Number needed to treat: Properties and Bellamy N, Bhagwagar Z, Costello A, Cowan P, Fang WC, problems. J R Statist Soc A 163:403-419, 2000 Katz, Paillard, and Van Inwegen The Journal of Pain 123 25. Katz N: Methodological issues in clinical trials of opioids 41. Raskin P, Donofrio PD, Rosenthal NR, Hewitt DJ, for chronic pain. Neurology 65:S32-49, 2005 Jordan DM, Xiang J, Vinik AI: Topiramate vs placebo in painful diabetic neuropathy: Analgesic and metabolic effects. 26. Katz J, Finnerup NB, Dworkin RH: Clinical trial outcome Neurology 63:865-873, 2004 in neuropathic pain: Relationship to study characteristics. Neurology 70:263-272, 2008 42. Serpell MG: Gabapentin in neuropathic pain syndromes: A randomised, double-blind, placebo-controlled trial. Pain 27. Killian JM, Fromm GH: Carbamazepine inthe treatment of 99:557-566, 2002 neuralgia. Use of side effects. Arch Neurol 19:129-136,1968 43. Sheridan SL, Pignone MP: Numeracy and the medical 28. Kristiansen IS, Gyrd-Hansen D, Nexø J, Nielsen JB: student’s ability to interpret data. Eff Clin Pract 5:35-40, 2002 Number needed to treat: Easily understood and intuitively meaningful? Theoretical considerations and a randomised 44. Sheridan SL, Pignone MP, Lewis CL: A randomized com- trial. J Clin Epidemiol 55:888-892, 2002 parison of patient’s understanding of number needed to treat and other risk reduction formats. J Gen Intern Med 29. Laupacis A, Sackett DL, Roberts RS: An assessment of 18:884-892, 2003 clinically useful measures of the consequences of treatment. N Engl J Med 318:1728-1733, 1988 45. Simpson DA: Gabapentin and venlafaxine for the treatment of painful diabetic neuropathy. J Clin Neuromuscul Dis 30. Lesaffre E, Pledger G: A note on the number needed to 3:53-62, 2001 treat. Control Clin Trials 20:439-447, 1999 46. Simpson DM, McArthur JC, Olney R, Clifford D, So Y, 31. Mayne TJ, Whalen E, Vu A: Annualized was found better Ross D, Baird BJ, Barrett P, Hammer AE: Lamotrigine for than absolute risk reduction in the calculation of number HIV-associated painful sensory neuropathies: A placebo- needed to treat in chronic conditions. J Clin Epidemiol 59: controlled trial. Neurology 60:1508-1514, 2003 217-223, 2006 47. Smeeth L, Haines A, Ebrahim S: Numbers needed to treat 32. McAlister FA: The ‘‘number needed to treat’’ turns 20— derived from meta-analyses sometimes informative, usually and continues to be used and misused. CMAJ 179:549-553, misleading. BMJ 318:1548-1551, 1999 2008 48. Stang A, Poole C, Bender R: Common problems related 33. Moore RA: What Is an NNT? In ‘‘What Is. Series.’’ April to the use of number needed to treat. J Clin Epidemiol 63: 2009. Available at: http://www.whatisseries.co.uk/whatis/ 820-825, 2010 pdfs/What_is_an_NNT.pdf 49. Suissa D, Brassard P, Smiechowski B, Suissa S: Number 34. Moore RA: Pain and systematic reviews. Acta Anaesthe- needed to treat is incorrect without proper time-related siol Scand 45:1136-1139, 2001 considerations. J Clin Epidemiol 65:42-46, 2012 35. Moore RA, Moore OA, Derry S, McQuay HJ: Numbers 50. Thabane L: A closer look at the distribution of number needed to treat calculated from responder rates give a bet- needed to treat (NNT): a Bayesian approach. Biostatistics 4: ter indication of efficacy in osteoarthritis trials than mean 365-370, 2003 pain scores. Arthritis Res Ther 10:R39, 2008 51. Tramer MR, Walder B: Number needed to treat (or 36. Moore RA, Straube S, Wiffen PJ, Derry S, McQuay HJ: harm). World J Surg 29:576-581, 2005 Pregabalin for acute and chronic pain in adults. Cochrane Database Syst Rev 3:CD007076, 2009 52. Walsh BT, Seidman SN, Sysko R, Gould M: Placebo response in studies of major depression: Variable, substan- 37. Newcombe RG: Know your limitations: Not just for clini- tial, and growing. JAMA 287:1840-1847, 2002 cians: Estimation of confidence intervals is not straightforward. J Public Health Med 21:481-482, 1999 53. Webster LR, Malan TP, Tuchman MM, Mollen MD, Tobias JK, Vanhove GF: A multicenter, randomized, 38. Newcombe RG: Confidence intervals for the number double-blind, controlled dose finding study of NGX-4010, needed to treat. Absolute risk reduction is less likely to be a high-concentration capsaicin patch, for the treatment of misunderstood. BMJ 318:1765-1767, 1999 postherpetic neuralgia. J Pain 11:972-982, 2010

39. Nexø J, Gyrd-Hansen D, Kragstrup J, Kristiansen IS, 54. Wu LA, Kottke TE: Number needed to treat: Caveat Nielsen JB: Danish GPs’ perception of disease risk and emptor. J Clin Epidemiol 54:111-116, 2001 beneﬁt of prevention. Fam Pract 19:3-6, 2002 55. Zakrzewska JM, Chaudhry Z, Nurmikko TJ, Patton DW, 40. Otto M, Bach FW, Jensen TS, Sindrup SH: Valproic acid Mullens EL: Lamotrigine (Lamictal) in refractory trigeminal has no effect on pain in polyneuropathy: A randomized neuralgia: Results from a double-blind placebo controlled controlled trial. Neurology 62:285-288, 2004 crossover trial. Pain 73:223-230, 1997