PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE FINAL RESEARCH REPORT Testing Ways to Display Patient-Reported Outcomes Data for Patients and Clinicians Principal Investigators: Claire Snyder, PhD1-3; Michael Brundage, MD, MSc4 Project Team: Katherine C. Smith, PhD2,3; Elissa T. Bantug, MHS3; Elliott E. Tolbert, PhD1,2; Emily Little1*; Amanda L. Blackford, ScM3 Stakeholder Advisory Board: Neil K. Aaronson, PhD5; Patricia A. Ganz, MD6; Ravin Garg, MD7; Michael Fisch, MD8*; Vanessa Hoffman, MPH9*; Bryce B. Reeve, PhD10; Eden Stotsky-Himelfarb3; Ellen Stovall11; Matthew Zachary12 AFFILIATIONS: 1Johns Hopkins School of Medicine, Baltimore, 8The University of Texas MD Anderson Cancer Maryland Center, Houston 2Johns Hopkins Bloomberg School of Public Health, 9Bladder Cancer Advocacy Network, Baltimore, Maryland Washington, District of Columbia 3Sidney Kimmel Comprehensive Cancer Center at 10University of North Carolina-Chapel Hill and Johns Hopkins, Baltimore, Maryland Lineberger Comprehensive Cancer Center, 4Queens Cancer Research Institute, Kingston, Ontario, Chapel Hill Canada 11National Coalition for Cancer Survivorship, 5Netherlands Cancer Institute, Amsterdam, The Washington, District of Columbia (posthumous) Netherlands 12Stupid Cancer, New York, New York 6University of California-Los Angeles and Jonsson *Institutional affiliation has changed since the Comprehensive Cancer Center, Los Angeles time of study conduct; original affiliation is 7Anne Arundel Medical Center, Annapolis, Maryland listed. Original Project Title: Presenting Patient-Reported Outcomes Data to Improve Patient and Clinician Understanding and Use PCORI ID: 323 HSRProj ID: 20142257 _______________________________ To cite this document, please use: Snyder C, Brundage M, Smith KC, et al. (2018). Testing Ways to Display Patient- Reported Outcomes Data for Patients and Clinicians. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/12.2018.CE.323 TABLE OF CONTENTS ABSTRACT .............................................................................................................................. 6 BACKGROUND ........................................................................................................................ 8 Role of the Stakeholders in Shaping the Research Design ........................................................ 9 PART 1: EVALUATING CURRENT APPROACHES FOR PRO DATA PRESENTATION ...................... 11 Methods .................................................................................................................................. 11 Figure 1a. Individual Patient Line Graphs of Scores Over Time (Adapted From Brundage et al29) ............................................................................................................... 14 Figure 1b. Tabulated Scores (Adapted From Brundage et al29) ........................................ 15 Figure 1c. Bubble Plot of Scores (Adapted From Brundage et al29) .................................. 16 Figure 1d. Heat Map of Scores (Adapted From Brundage et al29) .................................... 17 Figure 2a. Line Graphs of Mean Scores ............................................................................ 18 Figure 2b. Normed Line Graphs ........................................................................................ 19 Figure 2c. Line Graphs with Confidence Intervals (Adapted From Brundage et al29) ........ 20 Figure 2d. Bar Charts of Average Changes (Adapted From Brundage et al29) .................. 21 Figure 2e. Bar Charts of Responders (Adapted From Brundage et al29) ........................... 22 Figure 2f. Cumulative Distribution Function (Adapted From Brundage et al29) ............... 23 Results ..................................................................................................................................... 24 Table 1. Part 1 Participant Demographics (Adapted From Brundage et al29) ................... 26 Table 2. Summary of Results for Individual-Level Data Formats from Part 1 (Adapted From Brundage et al29) ..................................................................................... 27 Table 3. Summary of Results for Group-Level Data Formats from Part 1 (Adapted From Brundage et al29) ..................................................................................................... 33 PART 2: PARTNERING WITH STAKEHOLDERS TO DEVELOP IMPROVED DATA PRESENTATION FORMATS ...................................................................................................... 38 Methods .................................................................................................................................. 38 Table 4. Summary of Interpretation Challenges Addressed and Approaches Tested for Each Data Presentation Topic in Part 2 (Adapted From Smith et al31) ............ 42 Table 5. Part 2 Interview Participant Characteristics (Adapted From Smith et al31) ........ 46 Results ..................................................................................................................................... 47 PART 3: EVALUATING PRO PRESENTATION APPROACHES FROM PART 2 ................................ 51 Methods .................................................................................................................................. 51 Results ..................................................................................................................................... 54 Table 6. Part 3 Individual-Level Data Online Survey Participant Characteristics (Adapted From Snyder et al33) .......................................................................................... 55 2 Figure 3a. Green-Shaded Normal Range-“More” Directionality (Adapted From Snyder et al33) ................................................................................................................... 58 Figure 3b. Red-Circled Concerning Scores-“More” Directionality (Adapted From Snyder et al33) ................................................................................................................... 59 Figure 3c. Threshold Line for Possibly Concerning Scores-“More” Directionality (Adapted From Snyder et al33) .......................................................................................... 60 Figure 3d. “Better” Directionality (Example Using Threshold Line for Possibly Concerning Scores) (Adapted From Snyder et al33) .......................................................... 61 Table 7. Part 3 Individual-Level Data: Accuracy of Interpretation for Clinical Importance (Adapted From Snyder et al33)a ..................................................................... 63 Table 8. Part 3 Individual-Level Data: Accuracy of Interpretation for Directionality (Adapted From Snyder et al33)a ........................................................................................ 67 Table 9. Part 3 Individual-Level Data: Multivariable Model Results for Accuracy of Interpretation and Clarity Ratingsa ................................................................................... 71 Table 10. Part 3 Individual-Level Data: Clarity Ratings ..................................................... 72 Table 11. Part 3 Individual-Level Data: Proportion Selecting Each Format “Most Useful” .............................................................................................................................. 75 Table 12. Part 3 Group-Level for Patients: Online Survey Participant Characteristics (Adapted From Tolbert et al34)a ............................................................... 76 Figure 4a. Pie Charts: Group-Level Data for Patients (Adapted From Tolbert et al34) ................................................................................................................................... 79 Figure 4b. Bar Graphs: Group-Level Data for Patients (Adapted From Tolbert et al34) ................................................................................................................................... 80 Figure 4c. Icon Arrays: Group-Level Data for Patients (Adapted From Tolbert et al34) ................................................................................................................................... 81 Figure 4d. “More” Line Graphs: Group-Level Data for Patients (Adapted From Tolbert et al34) .................................................................................................................. 82 Figure 4e. “Better” Line Graphs: Group-Level Data for Patients (Adapted From Tolbert et al34) .................................................................................................................. 83 Figure 4f. Normed Line Graphs: Group-Level Data for Patients (Adapted From Tolbert et al34) .................................................................................................................. 84 Table 13. Part 3 Group-Level for Patients: Accuracy of Interpretation-Proportions ........ 85 Table 14. Part 3 Group Level for Patients: Multivariate Model Results for Accuracy of Interpretation and Clarity Ratings Proportions* (Adapted From Tolbert et al34) .................................................................................................................. 86 Table 15. Part 3 Group-Level for Patients: Clarity Ratings for Proportions ...................... 88 Table 16. Part 3 Group-Level for Patients: Proportion Selecting Each Proportion Format “Most Useful” .....................................................................................................