NATIONAL SURVEY ON DRUG USE AND HEALTH AN OVERVIEW OF TREND TESTING METHODS AND APPLICATIONS IN NSDUH AND OTHER STUDIES DISCLAIMER SAMHSA provides links to other Internet sites as a service to its users and is not responsible for the availability or content of these external sites. SAMHSA, its employees, and contractors do not endorse, warrant, or guarantee the products, services, or information described or offered at these other Internet sites. Any reference to a commercial product, process, or service is not an endorsement or recommendation by SAMHSA, its employees, or contractors. For documents available from this server, the U.S. Government does not warrant or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed. Substance Abuse and Mental Health Services Administration Center for Behavioral Health Statistics and Quality Rockville, Maryland April 2017 This page intentionally left blank NATIONAL SURVEY ON DRUG USE AND HEALTH AN OVERVIEW OF TREND TESTING METHODS AND APPLICATIONS IN NSDUH AND OTHER STUDIES RTI Project Nos. 0213757.004.107.008.003.007 & 0213405.005.002.002.003.005 Contract No. HHSS283201300001C RTI Authors: SAMHSA Authors: Project Director: Dan Liao Matthew Williams David Hunter Christopher Sroka Sarra Hedden SAMHSA Project Officer: Peter Tice For questions about this report, please e-mail [email protected]. Prepared for Substance Abuse and Mental Health Services Administration, Rockville, Maryland Prepared by RTI International, Research Triangle Park, North Carolina April 2017 Recommended Citation: Center for Behavioral Health Statistics and Quality. (2017). National Survey on Drug Use and Health: An Overview of Trend Testing Methods and Applications in NSDUH and Other Studies. Substance Abuse and Mental Health Services Administration, Rockville, MD. Acknowledgments This methodological document was prepared for the Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Quality (CBHSQ), by RTI International (a registered trademark and a trade name of Research Triangle Institute). In addition to the listed authors, significant contributors include Rachel Harter and Ralph Folsom at RTI and Jonaki Bose and Eunice Park-Lee at CBHSQ. Debbie Bond word processed and formatted the report, while Richard Straw copyedited it and Teresa Bass coordinated its web production. The authors thank the following federal personnel who graciously contributed their expertise on the trend analysis methods used by their programs. Name Affiliation Abbreviation Shannan Catalano Bureau of Justice Statistics BJS Lynn Langton Bureau of Justice Statistics BJS Joy Sharp Bureau of Transportation Statistics BTS David Raglin Census Bureau Census Nicole Scanniello Census Bureau Census Eileen O'Brien Energy Information Administration EIA Wendy Barboza National Agricultural Statistics Service NASS Nathan Cruze National Agricultural Statistics Service NASS Chris Chapman National Center for Education Statistics NCES Marilyn Seastrom National Center for Education Statistics NCES Jennifer Madans National Center for Health Statistics NCHS Donald Malec National Center for Health Statistics NCHS Cathie Alderks Substance Abuse and Mental Health Services Administration SAMHSA Elizabeth Crane Substance Abuse and Mental Health Services Administration SAMHSA Laura Milazzo-Sayre Substance Abuse and Mental Health Services Administration SAMHSA ii Table of Contents Chapter Page List of Figures ................................................................................................................................. v List of Tables ................................................................................................................................. v 1. Introduction ......................................................................................................................... 1 2. Summary of Trend Testing Methods in Current Practices in NSDUH .............................. 3 2.1 Overview ................................................................................................................. 3 2.2 t Test for Pairwise Comparison between Two Different Time Points .................... 3 2.2.1 Example of t Test for Pairwise Comparison in the 2014 NSDUH Detailed Tables ........................................................................................... 5 2.2.2 Example of t Test for Pairwise Comparison in the 2014 First Findings Reports ......................................................................................... 5 2.3 t Test for Testing Linear and Quadratic Trends across Multiple Time Points ........ 6 2.3.1 SUDAAN Code (Test of Linear Trends with DESCRIPT) ........................ 6 2.3.2 SUDAAN Code (Test of Quadratic Trends with DESCRIPT) ................... 7 2.3.3 Example of Linear Trend Testing in the 2013 NSDUH National Findings Report ........................................................................................... 7 2.3.4 Example of Linear and Quadratic Trend Testing in an Analytic Study ........................................................................................................... 8 2.4 Parametric Regression for Trend Testing ............................................................... 9 2.4.1 Example of Trend Testing in the 2014 Sample Redesign Impact Analysis..................................................................................................... 10 2.4.2 Example of Trend Testing in a Study of Trends in Cigarette Use, by Serious Psychological Distress Status, in a National U.S. Sample....................................................................................................... 11 2.5 Testing Estimates at Two Different Time Points for Small Area Estimation Documents ............................................................................................................ 12 3. Summary of Trend Testing Methods Used by Other Agencies ........................................ 17 3.1 Overview ............................................................................................................... 17 3.2 t Test for Pairwise Comparisons and for Testing Linear and Quadratic Trends with Orthogonal Contrast Matrices across Multiple Time Points ........................ 17 3.3 Regression Analysis for Trend Testing ................................................................. 18 3.3.1 Logistic Regression and Linear Regression .............................................. 18 3.3.2 Time Series Analysis ................................................................................ 19 3.3.3 Hierarchical and Multilevel Models ......................................................... 19 3.4 Testing a Nonlinear Trend and Identifying an Apparent Trend Change .............. 20 3.4.1 Three-Step Analysis in Youth Risk Behavior Survey (YRBS) ................ 20 3.4.2 Other Studies That Use Orthogonal Polynomial Trend Contrasts ............ 21 3.4.3 Other Studies That Use Joinpoint Regression .......................................... 21 3.5 Trend Testing without Considering Sample Variation at Each Time Point ......... 22 3.6 Caveats When Doing Trend Testing with Medical Expenditure Panel Survey (MEPS) Data ......................................................................................................... 22 iii Table of Contents (continued) Chapter Page 4. Literature Review of Selected Topics in Trend Testing ................................................... 25 4.1 Overview ............................................................................................................... 25 4.2 Detecting Outliers in Trends and Time Series ...................................................... 25 4.3 Nonparametric and Bayesian Methods in Time Series ......................................... 26 4.3.1 Nonparametric Methods ............................................................................ 26 4.3.2 Bayesian Methods ..................................................................................... 27 4.4 Methods Related to Joinpoint Regression ............................................................. 28 4.5 Sensitivity of Analyses to Time Window ............................................................. 29 4.6 Scanning Statistics ................................................................................................ 29 4.7 Updating Analyses in the Presence of New Data ................................................. 29 5. Summary and Future Research ......................................................................................... 31 References ..................................................................................................................................... 33 Appendix A Summary of Documents under This Review That Discussed or Used Trend Analysis............................................................................................................................. 43 B Methods Investigated through Literature Review ............................................................. 53 C Methodological Approaches Used by Other Statistical Agencies .................................... 59 iv List of Figures Figure Page 2.1 Past Month Cigarette Use among Youths in NSDUH and MTF: 2002-2013 (Figure 8.2 in the 2013 NSDUH National Findings Report) .............................................. 8 2.2 Past Year Mental Health Service Use, by Service Type: Percentages,