PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE FINAL RESEARCH REPORT Creating Recommendations on Types of Data to Present in Patient-Reported Outcome Measure Development Studies Joel J. Gagnier, PhD, ND, MSc1; Wieneke Mokkink, PhD2; Caroline B. Terwee, PhD2 AFFILIATIONS: 1Department of Orthopaedic Surgery, University of Michigan, Ann Arbor; Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor 2Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Epidemiology and Biostatistics, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands Institution Receiving Award: University of Michigan Original Project Title: Development of Reporting Guidelines for Psychometric Research on Patient- Reported Outcome Measures PCORI ID: ME-1606-35556 HSRProj ID: HSRP20181636 _______________________________ To cite this document, please use: Gagnier JJ, Mokkink W, Terwee CB. (2021). Creating Recommendations on Types of Data to Present in Patient-Reported Outcome Measure Development Studies. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/03.2021.ME.160635556 TABLE OF CONTENTS ABSTRACT ............................................................................................................................... 4 BACKGROUND ......................................................................................................................... 5 Criteria for Reporting Patient-Reported Outcomes .................................................................. 5 Purpose of This Study ................................................................................................................ 8 PATIENT AND STAKEHOLDER ENGAGEMENT ......................................................................... 10 METHODS .............................................................................................................................. 11 Phase 1: Conducting an Extensive Literature Review .............................................................. 11 Phase 2: Preparing for the Delphi Process .............................................................................. 13 Phase 3. Conducting the Delphi Rounds ................................................................................. 14 Phase 4: Developing Recommendations and Guidance Documents ....................................... 15 Departure From Original Protocol ........................................................................................... 16 RESULTS ................................................................................................................................ 17 Phase 1: Conducting the Literature Review ............................................................................ 17 Table 1. Common Items Relevant to Any Investigation of MPs Relating to 8 Report Sections ................................................................................................................ 20 Table 2. Specific Items for Each of the 9 MPs Relating to 8 Report Sections ................... 24 Phases 2 and 3: Preparing for and Conducting the Delphi Process ......................................... 30 Table 3. Characteristics of Panelists Completing Delphi Round 1 .................................... 31 Delphi Round 1 Results ........................................................................................................... 32 Figure 1. Results of Delphi Round 1 Voting: Items in the “Common” Set ........................ 33 Figure 2. Results of Delphi Round 1 Voting: Items in the “Specific Measurement Properties” Set ................................................................................................................. 34 Table 4. Summary of Voting for Common Set of Items for All Studies of PROM MPs .................................................................................................................................. 35 Table 5. Delphi Round 1 Results of Specific Items for All PROM MPs .............................. 37 Delphi Round 2 Results ........................................................................................................... 38 Table 6. Results of Delphi Round 2 Voting ....................................................................... 38 Figure 3. Results of Delphi Round 2 Voting ...................................................................... 40 Delphi Round 3 Results ........................................................................................................... 42 Table 7. Results of Delphi Round 3 Voting About Agreement to Remove the First 6 Items and Keep the Seventh Item ................................................................................. 43 Figure 4. Results of Delphi Round 3 Voting ...................................................................... 44 2 Final Items for Reporting Guidelines ....................................................................................... 45 Table 8. Final List for the Common Set of Items for Reporting of Studies of PROMs .............................................................................................................................. 46 Table 9. Final List for the Specific Set of Items, Relating to MPs, for Reporting of Studies of PROMs ............................................................................................................. 50 Phase 4: Developing Guidance Documents ............................................................................. 54 DISCUSSION .......................................................................................................................... 55 Limitations and Strengths of the Project ................................................................................. 55 Future Research ...................................................................................................................... 58 CONCLUSIONS ....................................................................................................................... 59 REFERENCES .......................................................................................................................... 60 RELATED PUBLICATIONS ........................................................................................................ 63 Submitted for Publication ....................................................................................................... 63 Published ................................................................................................................................. 63 ACKNOWLEDGMENTS ........................................................................................................... 64 APPENDICES .......................................................................................................................... 65 Appendix A. Search Strategies ................................................................................................. 65 Appendix B. Reporting Item Survey Round 1 .......................................................................... 67 Appendix C. Nonduplicate Search Results ..............................................................................107 Appendix D. Comments From Round 1 ..................................................................................373 3 ABSTRACT Background: Patient-reported outcome measures (PROMs) are key tools When performing clinical research or monitoring patient progress. One barrier to interpreting individual studies on the measurement properties (MPs) of PROMs is the frequent underreporting of important information. Although a checklist for investigating the methodological quality of studies on MPs of health status measurement instruments has been developed, guidelines for reporting these studies in the peer-revieWed literature do not exist. Objective: To develop a set of consensus and empirically based reporting recommendations for primary studies of the MPs of PROMs. Methods: This study included 4 phases: (1) conducting an extensive literature revieW of methods for performing or reporting studies testing MPs of PROMs; (2) preparing for a Delphi process to develop a consensus about reporting requirements by identifying experts to participate; (3) conducting the Delphi rounds of voting on the item lists developed in phase 1; and (4) developing recommendation and guidance documents. Results: The literature revieW resulted in our including 92 reporting items in the Delphi survey. A total of 83 individuals (from 12 countries) agreed to participate in the Delphi survey, With 47, 30, and 25 responding in rounds 1, 2, and 3, respectively. After 3 rounds, We achieved consensus on a set of 71 items separated into a set of 35 “common” items (relevant to all studies on MPs) and 36 “specific” items (exclusively relevant to 1 of the 9 MPs: (1) reliability, (2) measurement error, (3) internal consistency, (4) content validity, (5) face validity, (6) construct validity [including structural validity, hypothesis testing, and cross-cultural validity], (7) criterion validity, (8) responsiveness, and (9) interpretability). We created a manuscript for publication of the protocol, a PCORI final report, and similar to the final report, but much shorter, a manuscript also meant for publication of this Work. Conclusions: Consensus Was achieved on 2 nonduplicative sets of items, 71 in total, for inclusion in a reporting guideline for studies on MPs of PROMs. These items Will guide researchers on the necessary information