Living in the Best of All Worlds. Despite Premises Approximating the Truth with Meta-Analyses

LETTER TO THE EDITOR Living in the best of all worlds. Despite premises approximating the truth with meta-analyses

International Journal of Impotence Research (2006) 18, Concerning their point about heterogeneity of 572–573. doi:10.1038/sj.ijir.3901489 trials, we report here additionally that even with their proposed threshold of P ¼ 0.10, there is no We thank Dr Rosen and Dr Shaw for the stimulating evidence of heterogeneity between primary study comments on our study. In their letter they claim results. The funnel plot (along with Higgins’ I that our analysis failed to consider several issues square) suggests the presence of moderate hetero- that might have an impact on the findings. geneity, but it is evident from the plot, that part of Performing a meta-analysis is never completely this heterogeneity is to be explained by the agent objective. One has to make some premises as anyone used. conducting a meta-analysis has to. Therefore, stating We agree with Rosen and Shaw that sensitivity that some aspects are neglected is an argument that analyses with the assumption of random effects applies to all meta-analyses and, therefore, always (REM) could be included if wanted. We certainly true. In our article, we provided the reader with all judge it obligatory in the cases with unclear real information that is needed to be able to form his 2 effect they argue with (soj proteins or passive opinion on the (internal and external) validity of our 3 smoking ) with a considerable heterogeneity of analyses. trial results. We ourselves applied this, for It is true that we included trials with different example, in case of herbal essences.4 We argue treatment durations. We are aware that disconti- that the effect of a clearly chemically defined nuation rates may increase with longer treatment, agent as sildenafil or vardenafil with an incon- although most studies adapted an intention-to- siderable heterogeneity of trial results is different treat model that accounts for this bias. To date, from those of soj proteins or hypericum extracts. there is to our knowledge no evidence from the As there is no substantial heterogeneity between literature that disease progression influences trial findings on the same agent, it is not at all logical findings in erectile dysfunction, even though it is that REM leads to increased standard errors. In possible. Although there is some evidence for post hoc analyses, we here report that random sildenafil in clinical settings that the full effect of effect procedures lead to the same results as fixed the medication is reached only after multiple effect ones (in the argued cases of sildenafil dosages,1 we had no evidence from literature that and vardenafil exactly the same effect sizes and in highest fixed dose trials the trial duration itself standard errors as in FEM), only the standard has a similar impact. error of the pooled effect size of tadalafil It is also possible that phosphodiesterase 5 (PDE- studies increases marginally. An obligatory 5) inhibitor-naive patients (as in most sildenafil report of sensitivity analyses testing different studies) may show higher response than men with a priori model assumptions (fixed vs random) PDE-5 inhibitor experience. However, speculating is not regulated in the QUality Of Reporting of one could argue that the control group (as well as Meta-analyses (QUOROM) statement either.5 the drug group) might also have been more We have to correct Rosen and Shaw, the analysis responsive to treatment, so that the relative was not ‘limited exclusively to change in EF benefit should be approximately the same. We domain scores from baseline’, rather ‘weighted mentioned this aspect in the discussion (labeling mean differences between agent and placebo it ‘treatment history’). group in the EF domain score after treatment’ We also agree with Rosen and Shaw that broad (citation from article text) were calculated. The standard deviation data imputation can distort baseline EF domain score for the study Padma- findings. However, data extraction from figures or Nathan 2003 was extracted from Figure 2 of the computation from standard error is not the same published study report.6 as data imputation. We imputed standard devia- Rosen and Shaw’s comment on failing to control tions from other studies in seven cases (only in for possible confounders of treatment effect (such tadalafil studies). As all the reported standard as baseline severity) is very well worth consider- deviations were largely homogeneous (irrespec- ing. We discussed this already briefly in the tive of agent), the performed imputation is article. In a pilot study, we analyzed the effect unlikely to bias the results. of several covariates on the treatment effect as Letter to the Editor 573 measured by post-treatment difference between References agent and placebo group.7 We found that several different factors are likely to affect the treat- 1 McCullough AR, Barada JH, Fawzy A, Guay AT, Hatzichristou ment effect. However, controlling for all of D. Achieving treatment optimization with sildenafil citrate (Viagra) in patients with erectile dysfunction. Urology 2002; these factors was impossible, and we did not 60(Suppl 2B): 28–38. have enough information to decide for one (or 2 Anderson JW, Johnstone BM, Cook-Newell ME. Meta-analysis some) of them to control. Based on this pilot of the effects of soy protein intake on serum lipids. N Engl study, we performed a more sophisticated J Med 1995; 333: 276–282. analysis in sildenafil flexible dose studies to find 3 He J, Vupputuri S, Allen K, Prerost MR, Hughes J, Whelton PK. Passive smoking and the risk of coronary heart disease – a meta- determinants of treatment effect, and, actually, we analysis of epidemiologic studies. N Engl J Med 1999; 340: found that baseline severity, among others, plays a 8 920–926. key role. 4 Linde K, Berner MM, Egger M, Mulrow C. St John’s wort for depression: meta-analysis of randomised controlled trials. Br We would like to underline once more that this J Psychiatry 2005; 186: 99–107. kind of comparisons can be biased by several 5 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. issues and that the differences may be attributed Improving the quality of reports of meta-analyses of rando- to other factors than the agents themselves. Our mised controlled trials: the QUOROM statement. Lancet 1999; meta-analysis neither can nor claims to prove that 354: 1896–1900. there is a difference in efficacy of PDE-5 inhibitors. 6 Padma-Nathan H, Stecher VJ, Sweeney M, Orazem J, Tseng L-J, deRiesthal H. Minimal time to successful intercourse after It only gives evidence at the highest possible and sildenafil citrate: results of a randomized, double-blind, clearly defined methodological standard that there placebo-controlled trial. Urology 2003; 62: 400–403. might be one. 7 Berner MM, Kriston L, Harms A. Covariates of treatment effect in trials with oral sildenafil in the treatment of erectile MM Berner and L Kriston dysfunction – a meta-analytic regression analysis. J Sex Med 2005; 2(Suppl 1): 56. Department of Psychiatry and Psychotherapy, 8 Kriston L, Harms A, Berner MM. A meta-regression analysis of University Hospital Medical Center, Freiburg, treatment effect modifiers in trials with flexible-dose oral Germany sildenafil for erectile dysfunction in broad-spectrum popula- E-mail: [email protected] tions. Int J Impot Res 2006; 18: 559–565.

International Journal of Impotence Research