Comparing approaches to univariate sensitivity analysis Conor Teljeur Patricia Harrington Máirín Ryan Introduction In the context of health technology assessment, economic evaluations are designed to support decision making regarding the investment or disinvestment in technologies. An important element of evaluations is addressing the decision uncertainty that arises due to uncertainty or imprecision in our knowledge regarding the parameters that underpin the economic model. A variety of methods are used to convey uncertainty, including Monte Carlo simulation, plotting individual simulation results on the cost- effectiveness plane, use of cost-effectiveness acceptability curves, and univariate or one-way sensitivity analyses. Introduction In this study we focus on univariate sensitivity analysis, whereby the impact on decision uncertainty of uncertainty in individual parameters is explored. A univariate sensitivity analysis is commonly applied by setting a single parameter at its lower and upper bounds, respectively, while all other parameters are fixed at their mean values. From this we can see how much the incremental costs, benefits or ICER shifts due to a change in the value of a single parameter. The outcome of a univariate sensitivity analysis is often presented as a tornado diagram, although spider plots are also used. Aim of the study At least two approaches to univariate sensitivity analysis have appeared in the literature. The purpose of the study was to explore the impact of adopting different approaches to univariate sensitivity analysis and how they might affect interpretation of decision uncertainty. Methods We tested three approaches to univariate sensitivity analysis: ▪ Set one parameter at its lower and upper bounds, respectively, while all other parameters were kept at their mean values ▪ Analysis of covariance ▪ Set one parameter at its mean and allow all others to vary according to their statistical distributions Methods We used an assessment of endovascular treatment including mechanical thrombectomy for the treatment of acute ischaemic stroke as a case study. ▪ Stroke outcomes were measured as disability using the modified Rankin Scale (mRS) and grouped into three health states: functional independence, functional dependence, and death. ▪ Clinical efficacy was derived from 9 RCTs estimating the risk of functional independence and mortality at 90 days after stroke for mechanical thrombectomy plus IV-tPA compared to IV-tPA alone. ▪ The model included 40 stochastic parameters all defined by probability distributions that were either normal, log normal or beta. Methods How the three approaches were compared: ▪ Main outcome considered was the net monetary benefit (NMB) at a willingness-to-pay threshold of €20,000/QALY. ▪ Correlations between parameters were taken into account. ▪ Pearson and Spearman's correlation were measured to explore relationship between the estimated ‘parameter influence’ generated by the different analysis methods. ▪ Tornado plots were restricted to the 15 highest ranked parameters in each approach, although all were included in the analysis. Results One parameter at bounds, all others at mean value Results Analysis of covariance Results One parameter at mean value, all others allowed to vary Results Results Correlations Spearman rank Comparison Outcome m1:m2 m1:m3 m2:m3 NMB 0.98 0.45 0.49 Cost 0.97 0.49 0.55 Benefit 0.55 0.32 0.49 Pearson Comparison Outcome m1:m2 m1:m3 m2:m3 NMB 0.95 0.89 0.84 Cost 0.94 0.89 0.93 Benefit 0.97 0.90 0.94 Discussion We explored the impact of three methods for univariate sensitivity analysis using a case study. Each method offered a different ranking of parameters in terms of their influence on uncertainty in the summary outcome. Each method addresses different questions, and hence can be of greater or lesser value in different contexts. Discussion Setting individual parameters at their bounds is effectively a systematic scenario analysis, and may be misleading to decision makers. Consideration should be given to using alternate bounds, such as the interquartile range. The ANCOVA approach may be more simply interpreted but has disadvantages. It is unidirectional and interaction between variables might need to be considered. There are issues for Dirichlet distributions. Setting a parameter at its mean while varying other parameters bears similarity to value of information analysis. Conclusions The univariate sensitivity analysis often assists both the analyst and the decision maker in understanding which parameters influence uncertainty. The analytical approach can impact on which parameters are considered most influential. As with any sensitivity analysis, it is imperative that the uncertainty associated with each parameter is adequately captured in the model. [email protected] https://www.hiqa.ie/areas-we-work/health-technology-assessment Thank you.