1 Faculty of Electrical Engineering, Mathematics & Computer Science Designing a smartphone application for supporting the COMET-program in mental health care Bob Loos Master Thesis Interaction Technology July 2020 Supervisors: prof.dr. D.K.J. Heylen (HMI) dr. R. Klaassen (HMI) dr. K.P. Truong (HMI) drs. Y.P.M.J. Derks (PHT) Human Media Interaction Group University of Twente Disclaimer This research is based on the COMET-methodology as designed by Korrelboom (2000). Apart from COMET being used, this research is not connected to any other research by Korrelboom. The emphasis of this research is laid on the design of a con- cept of a mobile application. This does not include the effectiveness of the latter on the patient's progress. On 17-02-2020, Korrelboom has given explicit permission for using the COMET-methodology in the current publication as well as for making the used prototypes available to the public. In this publication and in the prototypes, two screenshots from videos from VGCt and Gedachten Uitpluizen are used. On 23-04-2020 explicit permission for using these materials is granted. Furthermore, freely available material is used from svgrepo.com, the Spotify Branding Guideline and Pexels.com. Where possible, attribution is provided. The prototype does not contain any content that is copyrighted (unless permission is granted by the relevant party). Patients from mental healthcare organisation GGNet Doetinchem are asked to par- ticipate. During this part of the research, the researcher was following an internship at GGNet Doetinchem. Nor does this report or the prototypes contain patient information. For this research, permission of the Ethical Committee EWI of the University of Twente has been requested and granted on 07-08-2019. The following reference number has been provided: RP 2019-79. 2 Abstract The process is described of designing a concept of a smartphone application to support patients during the COMET-program in mental health care, as well as when having finished the program. Background information on COMET and the after-therapy gap is searched for, whereas the latter can be well supported by lit- erature. Inspiration for the application is drawn from the related work. Interviews have been held with mental health care professionals who have experience with (a derivative of) COMET, resulting in a list of requirements for the application. The after-therapy gap, as discussed in the background information, is also supported by the interviews. The defined requirements are used to create a first prototype of COMET-E. A user test with this prototype has been performed to discover the experiences of the patients and professionals. The latter to test if the needs of the end users would be sufficiently met. The promising results of this user test are used to create a final prototype of COMET-E. Recommendations for future (development and research) work are provided, concluding this thesis. 3 Contents 1 Introduction 11 1.1 Research context . 11 1.2 Research questions . 12 1.3 Research outline . 12 2 Background 13 2.1 COMET - training for improving one's self-esteem . 13 2.1.1 Counterconditioning stimuli . 13 2.1.2 Contents of the training . 15 2.2 The after-therapy gap . 16 2.2.1 Findings from a preliminary project on the use of technology in mental health care . 16 2.2.2 State of the art on the after-therapy gap . 17 2.3 Related work and state of the art . 18 2.3.1 Related work: COMET as self-help intervention . 18 2.3.2 State of the art: technology for treatment, aftercare and self-help 19 2.3.3 Related work: notable features of smartphone applications . 20 2.4 Conclusion . 22 3 Professionals about COMET 24 3.1 Goal . 24 3.2 Participants . 24 3.3 Method . 24 3.4 Materials . 24 3.5 Procedure . 25 3.5.1 Recruiting participants . 25 3.5.2 Introduction and signing the informed consent form . 25 3.5.3 Activity #1: the interview . 25 3.5.4 Analyses of the data . 25 3.6 Results . 26 3.7 Discussion . 28 3.7.1 Requirements based on the results . 28 3.7.2 Translation to the persuasive systems design (PSD) model . 32 3.7.3 Limitation of the current results . 33 3.8 Conclusion . 34 4 Application Design 35 4.1 Target group and personas . 35 4.2 Stakeholders . 36 4.2.1 The patient who follows the therapy . 36 4.2.2 The professional who provides the therapy . 36 4.3 Content of the application . 36 4.3.1 Inclusion of original COMET-content . 36 4.3.2 Expanding and extending COMET . 38 4.3.3 Included features and user stories . 40 4.3.4 Mapping the features and content to the requirements . 42 4.4 Realisation of the prototype . 43 4.4.1 An impression of the used prototype . 43 4 5 User test: testing with professionals and patients 46 5.1 Goal . 46 5.2 Participants . 46 5.3 Method . 46 5.4 Materials . 47 5.5 Procedure . 48 5.5.1 Recruiting participants . 48 5.5.2 Introduction and signing the informed consent form . 48 5.5.3 Activity #1: Performance of a set of predefined tasks . 48 5.5.4 Activity #2: Fill in two questionnaires . 49 5.5.5 Activity #3: Evaluative interview . 49 5.5.6 Analyses of the data . 49 5.5.7 Unexpected change of method due to COVID-19 outbreak . 50 5.5.8 Second unexpected change of method due to COVID-19 outbreak 50 5.6 Results . 51 5.6.1 Interventions, completion time and error count . 51 5.6.2 SUS-score and its meaning . 52 5.6.3 TAM-scores . 53 5.6.4 Interview . 54 5.7 Discussion . 56 5.7.1 Interpretation of the results . 56 5.7.2 Required improvements based on the results . 57 5.7.3 Limitation of the current results . 58 5.8 Conclusion . 60 6 Changes in the final prototype 61 6.1 Topic 1: changing the location of the helplines / introduction of the toolkit 61 6.2 Topic 2: adding links to make content easier to find . 62 6.3 Topic 3: text is limited and provided optionally . 63 6.4 Topic 4: introduction and tips to the available content . 64 6.5 Topic 5: improvements of the monitor . 66 6.6 Minor changes . 67 6.7 Access to the final prototype . 68 7 (General) Discussion 70 8 (General) Conclusion 71 9 Future work 73 9.1 Applying the same method for other trainings, therapies and mental dis- orders . 73 9.2 Next steps in the (technical) development of COMET-E . 73 9.3 From prototype to final version . 74 9.4 Validation of COMET-E . 75 9.5 Final notes on future work . 76 References 77 Appendices 82 A Questions for interviews with professionals 82 B Information letter for interviews with professionals 84 5 C Informed consent for interviews with professionals 87 D Information letter for the user test with patients 90 E Informed consent for the user test with patients 94 F Information letter for the user test with professionals 97 G Informed consent for the user test with professionals 100 H User test: prepared set of tasks 103 I User test: SUS-questionnaire 106 J User test: TAM-questionnaire 109 K User test: interview questions 112 L User test: all performed interventions 114 M Comparing the first and the final prototype 115 N Interviews { open coding { coding scheme 121 O Interviews { axial coding { coding scheme 128 6 List of Figures 1 COMET explained for altering conditioned stimuli. 14 2 COMET explained for coping in case of unconditioned stimuli. 14 3 Modules to work on . 23 4 Monitor by answering surveys . 23 5 Progress visually presented . 23 6 Virtual companion . 23 7 Conversational agent . 23 8 Peer support by messages . 23 9 John...................................... 35 10 Sarah . 35 11 Each of the sessions are presented individually . 38 12 A checklist is shown of the homework assignments . 38 13 Answers can be provided and will be stored in the app . 38 14 The app will be tailored by asking basic questions at first start . 40 15 The user can include interactive media to enrich their content . 40 16 A push-notification to remind the user to work on exercises . 40 17 A tailored summary about the training, including helplines . 40 18 The user can monitor himself by answering a simple question . 40 19 Introductory questions to alter the app to the situation of the user . 44 20 Start-page showing the remaining steps of the current session . 44 21 Session-screen explaining the session and showing homework . 44 22 Summary-page showing helplines to be called when necessary . 45 23 Monitor-page visualizing the current state of self-esteem . 45 24 Start-page once finished with the training . 45 25 Interactive media from other apps like Spotify can be included . 45 26 Notification about positive stories to maintain self-esteem . 45 27 Recall positive moments to maintain self-esteem . 45 28 Boxplot of each of the topics of TAM . 53 29 A new button is added in the menu bar: "hulplijnen" . 61 30 The helplines as a seperate menu item . 61 31 A soft red toolkit-button is shown on every main screen . 62 32 The toolkit provides the user a selection of tools . 62 33 Calling a helpline is one of the tools from the toolkit . 62 34 Links in the Summary-page refer to earlier followed sessions . 63 35 Links can be used across the app; not only in the summary . 63 36 Links can be stacked if multiple screens are relevant .