Spie Charts for Quantifying Treatment Effectiveness and Safety in Multiple Outcome Network Meta- Analysis: a Proof-Of-Concept Study
Total Page:16
File Type:pdf, Size:1020Kb
Spie charts for quantifying treatment effectiveness and safety in multiple outcome network meta- analysis: A proof-of-concept study Caitlin Daly McMaster University https://orcid.org/0000-0002-8272-797X Lawrence Mbuagbaw McMaster University Lehana Thabane McMaster University Sharon E Straus Li Ka Shing Knowledge Institute, St. Michael's Hospital Jemila S Hamid ( [email protected] ) University of Ottawa Technical advance Keywords: network meta-analysis, ranking, SUCRA, spie chart, radar plot, multiple outcomes Posted Date: September 24th, 2020 DOI: https://doi.org/10.21203/rs.3.rs-36139/v2 License: This work is licensed under a Creative Commons Attribution 4.0 International License. Read Full License Version of Record: A version of this preprint was published on October 28th, 2020. See the published version at https://doi.org/10.1186/s12874-020-01128-2. 1 Spie charts for quantifying treatment effectiveness and safety in multiple 2 outcome network meta-analysis: A proof-of-concept study 3 4 Caitlin H Daly1,2, Lawrence Mbuagbaw1,3, Lehana Thabane1,3, Sharon E Straus4,5, Jemila S 5 Hamid1,6* 6 7 1 Department of Health Research Methods, Evidence, and Impact, McMaster University, 8 McMaster University Medical Centre, 1280 Main Street West, 2C Area, Hamilton, Ontario, L8S 9 4K1, Canada 10 2 Population Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 11 Whatley Road, Bristol, BS8 2PS, United Kingdom 12 3 Biostatistics Unit, Father Sean O’Sullivan Research Centre, St Joseph’s Healthcare Hamilton, 13 50 Charlton Avenue East, Hamilton, Ontario, L8N 4A6, Canada 14 4 Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 15 Victoria Street, Toronto, Ontario, M5B 1TB, Canada 16 5 Department of Medicine, Faculty of Medicine, University of Toronto, C. David Naylor 17 Building, 6 Queen’s Park Crescent West, Third Floor, Toronto, Ontario, M5S 3H2, Canada 18 6 Department of Mathematics and Statistics, University of Ottawa, STEM Complex, room 336, 19 150 Louis-Pasteur Private, Ottawa, Ontario, K1N 6N5, Canada 20 21 * Corresponding author: 22 Dr. Jemila S Hamid 23 Department of Mathematics and Statistics 24 University of Ottawa 25 STEM Complex, room 336 26 150 Louis-Pasteur Private 27 Ottawa, Ontario K1N 6N5, Canada 28 Tel +1 613 737 7600 ext 4194 29 Email [email protected] 1 1 Abstract 2 Background: Network meta-analysis (NMA) simultaneously synthesises direct and indirect 3 evidence on the relative efficacy and safety of at least three treatments. A decision maker may use 4 the coherent results of an NMA to determine which treatment is best for a given outcome. 5 However, this evidence must be balanced across multiple outcomes. This study aims to provide a 6 framework that permits the objective integration of the comparative effectiveness and safety of 7 treatments across multiple outcomes. 8 Methods: In the proposed framework, measures of each treatment’s performance are plotted on 9 its own pie chart, superimposed on another pie chart representing the performance of a hypothetical 10 treatment that is the best across all outcomes. This creates a spie chart for each treatment, where 11 the coverage area represents the probability a treatment ranks best overall. The angles of each 12 sector may be adjusted to reflect the importance of each outcome to a decision maker. The 13 framework is illustrated using two published NMA datasets comparing dietary oils and fats and 14 psoriasis treatments. Outcome measures are plotted in terms of the surface under the cumulative 15 ranking curve. The use of the spie chart was contrasted with that of the radar plot. 16 Results: In the NMA comparing the effects of dietary oils and fats on four lipid biomarkers, the 17 ease of incorporating the lipids’ relative importance on spie charts was demonstrated using 18 coefficients from a published risk prediction model on coronary heart disease. Radar plots 19 produced two sets of areas based on the ordering of the lipids on the axes, while the spie chart only 20 produced one set. In the NMA comparing psoriasis treatments, the areas inside spie charts 21 containing both efficacy and safety outcomes masked critical information on the treatments’ 22 comparative safety. Plotting the areas inside spie charts of the efficacy outcomes against measures 23 of the safety outcome facilitated simultaneous comparisons of the treatments’ benefits and harms. 2 1 Conclusions: The spie chart is more optimal than a radar plot for integrating the comparative 2 effectiveness or safety of a treatment across multiple outcomes. Formal validation in the decision- 3 making context, along with statistical comparisons with other recent approaches are required. 4 5 Keywords 6 network meta-analysis, ranking, SUCRA, spie chart, radar plot, multiple outcomes 7 3 1 Background 2 Health technology assessments and clinical guidelines are increasingly being supported by 3 evidence synthesised through network meta-analysis (NMA) [1, 2]. The main output from an NMA 4 is a coherent set of relative treatment effects, based on pooled direct and indirect evidence typically 5 contributed by randomised controlled trials (RCTs) [3, 4]. The estimated treatment effects relative 6 to a common comparator may then be used to inform a ranked list of treatments, from which 7 knowledge users may be able to deduce which treatment is best for a given clinical problem. 8 Interpreting NMA results is challenging, particularly as the number of treatments and 9 outcomes increase. Several pieces of literature have aimed to ease the interpretative burden of 10 NMA. For example, three graphical tools were developed to display key features of an NMA (i.e. 11 relative effects and their uncertainty, probabilities of ranking best, and between-study 12 heterogeneity) for a single outcome [5]. The rank heat plot has been proposed as a visual tool for 13 presenting NMA results across multiple outcomes [6]. However, knowledge users could also 14 benefit from the quantification of the overall integrated results across multiple outcomes to 15 facilitate interpretation in a more objective way. This is particularly important in situations where 16 the comparative rankings of treatments on each outcome contradict each other. 17 Radar plots are often used as a visualisation tool to communicate multivariate data [7]. 18 Recently, they have been used to visually compare the surface under the cumulative ranking curves 19 (SUCRAs) in an NMA evaluating multiple interventions for relapsing multiple sclerosis [8]. 20 Another NMA on dual bronchodilation therapy for chronic obstructive pulmonary disease has 21 compared the area within radar charts of SUCRA values to deliver a single ranking of their 22 efficacy-safety profile [9]. However, in this NMA, the quantification of the area weighed each 23 outcome equally, which may not reflect a knowledge user’s preferences. The use of radar plots for 4 1 the purpose of comparing the overall performance of treatments is also limited by the fact that the 2 area depends on the ordering of the outcomes on the plot. For this reason, the spie chart has been 3 suggested as a better alternative [10]. 4 A spie chart is a combination of two pie charts, where one is superimposed on another, 5 allowing comparisons between two groups on multiple attributes [11]. For example, in the context 6 of NMA, this could be the comparison of a treatment against a hypothetical treatment that is 7 uniformly the best across multiple outcomes. The former’s area will be a fraction of the latter’s, 8 thereby facilitating the comparison of multiple treatments in a manner similar to comparing areas 9 on a radar chart. 10 To address the aforementioned gaps and limitations, the objective of this paper is to lay the 11 groundwork for conceptualising a treatment’s likelihood of being the best overall in terms of its 12 coverage area inside a spie chart. This circular plot may be divided into segments representing a 13 treatment’s level of efficacy or safety for each outcome. We provide a methodological framework 14 and assess the feasibility of adapting the area on a spie chart to numerically integrate the efficacy 15 and safety of treatments estimated by NMAs of multiple outcomes. Since radar plots have not been 16 formally investigated and generalised for NMA, we also present the area on a radar plot and 17 compare it to that of spie charts. We illustrate how the spie chart may be used to overcome the 18 limitations of the radar plot, as well as its flexibility for further adaptations. 19 20 Methods 21 Measuring the Coverage Area inside a Spie Chart 22 Consider for example a situation where the performance of a treatment has been measured in terms 23 of J = 8 outcomes valued between 0 and 1. Simulated values are plotted on a spie chart in Figure 5 1 1. In general, the resulting shape on any spie chart is a series of J sectors, each with their own 2 radius equal to the value of the J outcome measures. The area covered by these sectors may be 3 calculated as the sum of the areas of the individual sectors. 4 In Figure 1, the shaded area, A , is the sum of the area of the 8 sectors,Ajj ,= 1,...,8 1 q 2 5 Ayjj= 2 j , q 6 wherey j and j are the radius and angle of sector j , respectively. In Figure 1, all angles are equal, 2pp 7 i.e.,q == radians, and the shaded area on the spie chart is then: j 84 8 p 2 8 Ay= å j , 8 j=1 9 which is an average of the squared values of the 8 outcomes, scaled by a factor of p . In general, 10 the shaded area within a spie chart informed byJ ³1 , outcomes for an intervention is J 1 2 11 Ay= åq jj.