Comparing approaches to univariate sensitivity analysis

Conor Teljeur Patricia Harrington Máirín Ryan Introduction

In the context of health technology assessment, economic evaluations are designed to support decision making regarding the investment or disinvestment in technologies. An important element of evaluations is addressing the decision uncertainty that arises due to uncertainty or imprecision in our knowledge regarding the parameters that underpin the economic model. A variety of methods are used to convey uncertainty, including Monte Carlo simulation, plotting individual simulation results on the cost- effectiveness plane, use of cost-effectiveness acceptability curves, and univariate or one-way sensitivity analyses. Introduction

In this study we focus on univariate sensitivity analysis, whereby the impact on decision uncertainty of uncertainty in individual parameters is explored. A univariate sensitivity analysis is commonly applied by setting a single parameter at its lower and upper bounds, respectively, while all other parameters are fixed at their mean values. From this we can see how much the incremental costs, benefits or ICER shifts due to a change in the value of a single parameter. The outcome of a univariate sensitivity analysis is often presented as a tornado diagram, although spider plots are also used. Aim of the study

At least two approaches to univariate sensitivity analysis have appeared in the literature. The purpose of the study was to explore the impact of adopting different approaches to univariate sensitivity analysis and how they might affect interpretation of decision uncertainty. Methods

We tested three approaches to univariate sensitivity analysis: ▪ Set one parameter at its lower and upper bounds, respectively, while all other parameters were kept at their mean values ▪ Analysis of covariance ▪ Set one parameter at its mean and allow all others to vary according to their statistical distributions Methods

We used an assessment of endovascular treatment including mechanical thrombectomy for the treatment of acute ischaemic stroke as a case study. ▪ Stroke outcomes were measured as disability using the modified Rankin Scale (mRS) and grouped into three health states: functional independence, functional dependence, and death. ▪ Clinical efficacy was derived from 9 RCTs estimating the risk of functional independence and mortality at 90 days after stroke for mechanical thrombectomy plus IV-tPA compared to IV-tPA alone. ▪ The model included 40 stochastic parameters all defined by probability distributions that were either normal, log normal or beta. Methods

How the three approaches were compared: ▪ Main outcome considered was the net monetary benefit (NMB) at a willingness-to-pay threshold of €20,000/QALY. ▪ Correlations between parameters were taken into account. ▪ Pearson and Spearman's correlation were measured to explore relationship between the estimated ‘parameter influence’ generated by the different analysis methods. ▪ Tornado plots were restricted to the 15 highest ranked parameters in each approach, although all were included in the analysis. Results

One parameter at bounds, all others at mean value Results

Analysis of covariance Results

One parameter at mean value, all others allowed to vary Results Results

Correlations Spearman rank Comparison Outcome m1:m2 m1:m3 m2:m3 NMB 0.98 0.45 0.49 Cost 0.97 0.49 0.55 Benefit 0.55 0.32 0.49

Pearson Comparison Outcome m1:m2 m1:m3 m2:m3 NMB 0.95 0.89 0.84 Cost 0.94 0.89 0.93 Benefit 0.97 0.90 0.94 Discussion

We explored the impact of three methods for univariate sensitivity analysis using a case study. Each method offered a different ranking of parameters in terms of their influence on uncertainty in the summary outcome. Each method addresses different questions, and hence can be of greater or lesser value in different contexts. Discussion

Setting individual parameters at their bounds is effectively a systematic scenario analysis, and may be misleading to decision makers. Consideration should be given to using alternate bounds, such as the interquartile range. The ANCOVA approach may be more simply interpreted but has disadvantages. It is unidirectional and interaction between variables might need to be considered. There are issues for Dirichlet distributions. Setting a parameter at its mean while varying other parameters bears similarity to value of information analysis. Conclusions

The univariate sensitivity analysis often assists both the analyst and the decision maker in understanding which parameters influence uncertainty. The analytical approach can impact on which parameters are considered most influential. As with any sensitivity analysis, it is imperative that the uncertainty associated with each parameter is adequately captured in the model. [email protected] https://www.hiqa.ie/areas-we-work/health-technology-assessment

Thank you