Public Health & Intelligence PHI Trend Analysis Guidance Document Control Version Version 1.5 Date Issued March 2017 Author Róisín Farrell, David Walker / HI Team Comments to [email protected] Version Date Comment Author Version 1.0 July 2016 1st version of paper David Walker, Róisín Farrell Version 1.1 August Amending format; adding Róisín Farrell 2016 sections Version 1.2 November Adding content to sections Róisín Farrell, Lee 2016 1 and 2 Barnsdale Version 1.3 November Amending as per Róisín Farrell 2016 suggestions from SAG Version 1.4 December Structural changes Róisín Farrell, Lee 2016 Barnsdale Version 1.5 March 2017 Amending as per Róisín Farrell, Lee suggestions from SAG Barnsdale Contents 1. Introduction ........................................................................................................................................... 1 2. Understanding trend data for descriptive analysis ................................................................................. 1 3. Descriptive analysis of time trends ......................................................................................................... 2 3.1 Smoothing ................................................................................................................................................ 2 3.1.1 Simple smoothing ............................................................................................................................ 2 3.1.2 Median smoothing ........................................................................................................................... 2 3.2 Measuring change .................................................................................................................................... 3 3.2.1 Confidence Intervals ........................................................................................................................ 3 3.2.2 Population Proportion Tests ............................................................................................................ 3 3.2.3 Comparing standardised rates......................................................................................................... 3 3.2.4 Statistical Process Control................................................................................................................ 4 3.3 Charting data to highlight trends ............................................................................................................. 4 3.3.1 Logarithmic scales ........................................................................................................................... 4 3.3.2 Changing axis ratios......................................................................................................................... 5 4. Understanding trend data for exploratory analysis ................................................................................ 5 4.1 Univariate analysis ................................................................................................................................... 5 4.2 Multivariate analysis ................................................................................................................................ 6 5. Analytical methods for exploratory trend analysis ................................................................................. 6 5.1 Using Regression ...................................................................................................................................... 7 5.2 Methods ................................................................................................................................................... 8 5.2.1 Linear regression .............................................................................................................................. 8 5.2.2 Logistic regression ........................................................................................................................... 9 5.2.3 Polynomial regression ...................................................................................................................... 9 5.2.4 Restricted cubic splines .................................................................................................................. 10 5.2.5 Segmented regression ................................................................................................................... 11 5.2.6 Poisson regression ......................................................................................................................... 11 5.2.7 ARIMA ............................................................................................................................................ 12 5.2.8 Age Period Cohort Analysis ............................................................................................................ 13 6. Discussion ............................................................................................................................................ 13 Appendix 1: Examples from PHI outputs ..................................................................................................... 14 1.1: Simple Smoothing and Confidence Intervals ................................................................................................ 14 1.2: Linear Regression .......................................................................................................................................... 15 1.3: Population Proportion Tests................................................................................................................... 16 1.4: Age-Period-Cohort Analysis .................................................................................................................... 17 Appendix 2 Worked examples .................................................................................................................... 18 2.1: Simple Smoothing .................................................................................................................................. 18 2.2: Population Proportion Tests................................................................................................................... 19 2.3: Logarithmic Scales .................................................................................................................................. 20 2.4: Modifying axes ....................................................................................................................................... 22 2.5: Histograms ............................................................................................................................................. 23 2.6: Standard deviation ................................................................................................................................. 25 2.7: Scatterplots ............................................................................................................................................ 26 2.8: Residual plots ......................................................................................................................................... 28 2.9: Linear Regression ................................................................................................................................... 29 2.10: Logistic regression .................................................................................................................................. 31 2.11: Polynomial regression ............................................................................................................................ 34 2.12: Restricted cubic splines .......................................................................................................................... 37 2.13: Poisson regression .................................................................................................................................. 41 2.14: ARIMA ..................................................................................................................................................... 46 Appendix 3 Further Reading ....................................................................................................................... 50 1. Introduction When presenting and publishing data, it is common to provide a trend or time series analysis. A time series is simply a sequence of data points plotted over time (frequently using column or line charts). Time series analysis is used for a number of reasons: To summarise a trend and show if a measure is increasing or decreasing. By summarising data across a range of years, it may be possible to remove the ‘noise’ of a single-year analysis and expose an underlying trend. To project future trends, or estimate uncertain past events. To identify a change in trend resulting from policy change or a significant event. In many cases, a description of the observations and the magnitude of any observed change over time may be sufficient. However, when change occurs (and in some cases, when it does not) we may be required to explain why this has happened. This document provides individuals producing analytical outputs with guidance on time trend analysis methods and examples of their appropriate use. It first examines methods of describing trend data in order to aid interpretation and then discusses methods for exploring associations within data over time, observing the key principle that the best method to use is the