Master Thesis Identifying Patients in Community Pharmacies to Become

UNIVERSITY OF COPENHAGEN FACULTY OF HEALTH AND MEDICAL SCIENCES

Master Thesis Nina Scheel Andersen (vsg237)

Identifying patients in community pharmacies to become active participants in new medicines development Development and evaluation of a new recruitment technique derived from a public-private partnership

Department of Pharmacy, School of Pharmaceutical Sciences

Faculty of Health and Medical Sciences, University of Copenhagen

Internal supervisor: Susanne Kaae

External supervisor: Camilla Krogh Lauritzen

Submitted on: 31 August 2020

Name of department: Department of Pharmacy

Author(s): Nina Scheel Andersen (vsg237)

Title and subtitle: Identifying patients in community pharmacies to become active participants in new medicines development – Development and evaluation of a new recruitment technique derived from a public- private partnership.

Topic description: This thesis is written within the field of social and clinical pharmacy. The thesis concerns the development and evaluation of a new recruitment technique to recruit patients through community pharmacies to be engaged in medicines research and development in a pharmaceutical company.

Internal supervisor: Susanne Kaae

External supervisor: Camilla Krogh Lauritzen

Submitted on: 31 August 2020

Grade:

Number of study units:

£ 2 £ 3

Number of characters: 152,934

2 Preface

This master thesis had a volume of 45 ECTS credits and was conducted from 31 August 2019 to 31 August 2020, partly at the Department of Pharmacy, School of Pharmaceutical Sciences, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark; and at LEO Pharma A/S, Ballerup, Denmark. The supervisors were Susanne Kaae as the internal supervisor from the University of Copenhagen, and Camilla Krogh Lauritzen as the external supervisor from LEO Pharma A/S.

I would like to use this preface to thank all the people who supported me during my master thesis. This period has been a very interesting, educational, and exciting part of my life but occasionally also one of the hardest. However, it was all worth it, and I could not have done it without the huge help and support from the following people.

First, a huge thank you to all the patients who showed an interest and participated in my project. Your help was tremendous. My deepest gratitude to my internal supervisor Susanne Kaae who was there for me all along with unlimited support, both academically and emotionally. Further, I would like to express a huge thank you to LEO Pharma’s Patient Engagement Team, Camilla Krogh Lauritzen, Lasse Funch Jacobsen, Søren Holm, and Torben Löser, who provided their full support during the project. Especially, my deepest gratitude to Camilla for being my external supervisor and allowing me to write my thesis within your team. Further, I would like to thank the additional LEO Pharma employees who supported me.

A great thank you to Jacob Lenau and Skovlunde Pharmacy, as much as Ute Pørksen and Ballerup Pharmacy, for letting me use your pharmacies and supporting me significantly. A great thank you to Bente Buus Nielsen and Michael Koehler from FAIM for your excessive support during the project. A great thank you to Ballerup Municipality, especially Jette Rau, with your countless support during the project. Finally, a heartfelt thank you to my family and friends who supported me impressively all along the thesis period.

Nina Scheel Andersen | Copenhagen, Denmark | August 2020

3 Abstract

Aim/Background: The aim of this thesis is to concept-test and investigate how people with a rare disease diagnosis can be identified in a community pharmacy to be engaged in medicines research and development in a pharmaceutical company. To that end, a new project was established as a public-private partnership between the master thesis student, Ballerup Pharmacy, Skovlunde Pharmacy, Ballerup Municipality, the patient organisation of autoimmune diseases in Denmark (FAIM), and LEO Pharma. Methods: A Plan-Do-Study-Act cycle was used to develop the recruitment technique for recruiting patients to medicines research and development. Plan contains the preparational meetings leading up to the project start where several decisions were made. Do contains the practical, non- scientific parts of the project, which were the training of pharmacy staff, marketing campaign, recruitment process, patient introduction, patient preparation, and workshops at LEO Pharma. Study contains the scientific methods of the project, which were the following; semi-structured interviews with the project’s partners and patients; focus groups with the patients; and a quantitative overview of the patient contact. Act contains the recommendations for the further development of the project. Results: During the three-month recruitment phase, 17 patients inside the scope of the project made contact to participate whereof nine patients ended up participating in interviews, and hereof five patients in workshops as well. The community pharmacies were considered positive places of recruitment by both patients and the project’s partners. According to the patients, the workshops lived fully or almost fully up to their expectations, and the scientists from LEO Pharma were described as listening and understanding. However, the patients were unable to mention who the project’s partners were, but they found the partners’ involvement positive. Overall, the project’s partners were positive with the execution of the project and surprised by a larger number of patients being recruited than expected. The partners believed that the project has a future as a new concept of recruiting patients to medicines research and development. Conclusions: The concept-testing of recruiting people with a rare disease diagnosis through community pharmacies to be engaged in medicines research and development was accomplished. Overall, patients and partners found the project positive, and it has the potential to become a new practice of recruitment for patient engagement in new medicines development.

4 Table of contents

PREFACE ...... 3

ABSTRACT ...... 4

LIST OF ABBREVIATIONS ...... 7

LIST OF APPENDICES ...... 8

1. INTRODUCTION ...... 9

1.1 Thesis statement ...... 10

1.2 Research questions ...... 10

1.3 Purpose ...... 10

2. BACKGROUND ...... 11

2.1 History of patient engagement ...... 11

2.2 Legislation on patient engagement ...... 13

2.3 Roadmap for patient engagement in medicines R&D ...... 14

2.4 Coverage of existing literature ...... 16 2.4.1 Stakeholder expectations ...... 17 2.4.2 Best practices of patient engagement ...... 19 2.4.3 A master framework needed ...... 19 2.4.4 Recruiting patients ...... 19

2.5 Roles and responsibilities in the project ...... 20

2.6 About the partners of the project ...... 22

2.7 Conclusion ...... 24

3. METHODOLOGY ...... 25

3.1 Plan ...... 26

3.2 Do ...... 27

3.3 Study ...... 30 5 3.3.1 Similar method conditions for interviews and focus groups ...... 32 3.3.2 Ethical considerations ...... 35 3.3.3 The project’s partners ...... 35 3.3.4 Patients in the project ...... 37

4. RESULTS ...... 41

4.1 Quantitative overview of patient contact ...... 41 4.1.1 The channels of noticing and registration for patients ...... 42 4.1.2 Reasons for not participating ...... 42

4.2 Patient interviews ...... 43 4.2.1 The patients’ lives with the disease and treatment used ...... 44 4.2.2 Reflections leading up to participation in the project ...... 45 4.2.3 Evaluation of participation in the project ...... 49 4.2.4 Opinions about the partners and their cooperation in the project ...... 53 4.2.5 The patients’ trust in the public healthcare system ...... 54

4.3 Partner interviews ...... 55 4.3.1 The development of the project and the motivation behind ...... 55 4.3.2 Project evaluation ...... 57 4.3.3 Future of the project ...... 60 4.3.4 The partners’ perception of patients ...... 62

5. DISCUSSION ...... 63

5.1 Results discussion ...... 63 5.1.1 Which recruitment technique can be constructed to recruit the patients in community pharmacies? ...... 63 5.1.2 What is the patients’ attitude towards engagement in medicines research and development through this specific setting? ...... 65 5.1.3 What do the recruited patients think about the project’s partners’ involvement in this project? ...... 66 5.1.4 What are the project’s partners’ experiences with the execution of this kind of project where patients are recruited through community pharmacies? ...... 68

5.2 Methods discussion ...... 69

5.3 Act ...... 71

6. CONCLUSION ...... 73

7. REFERENCES ...... 74

6 List of abbreviations

EFPIA European Federation of Pharmaceutical Industries and Associations

EMA European Medicines Agency

ENLI Ethical Committee for the Pharmaceutical Industry in Denmark

EPF European Patients’ Forum

EUPATI European Patients’ Academy on Therapeutic Innovation

FAIM Patient Organisation of Autoimmune Diseases in Denmark, Danish: Foreningen for Autoimmune Sygdomme

FDA United States Food and Drug Administration

HTA Health Technology Assessment

IMI Innovative Medicines Initiative

IPA International Pharmaceutical Abstracts

Lif Danish Association of the Pharmaceutical Industry

N Sample size

PARADIGM Patients Active in Research and Dialogues for an Improved Generation of Medicines

PCWP Patients’ and Consumers’ Working Party

PDSA Plan-Do-Study-Act

PFDD Patient-Focused Drug Development

PoC Proof of Concept

R&D Research & Development

7 List of appendices

Appendix 1 Brochure used in the project

Appendix 2 Roll-up used in the project

Appendix 3 Print of the project’s website

Appendix 4 Press release about the project, Ritzau

Appendix 5 Article about the project, Ballerup Bladet

Appendix 6 Quotes used in Results (Danish and English)

Appendix 7 Interview transcript – patient 17 (Danish and English)

Appendix 8 Meaning condensation scheme – patient 17 (Danish and English)

Appendix 9 Interview guide – project’s partners (Danish and English)

Appendix 10 Interview guide – patient focus group before workshop (Danish and English)

Appendix 11 Interview guide – patient individual interview after workshop (Danish and English)

Appendix 12 Interview guide – patient individual interview without participation in workshop (Danish and English)

8 1. Introduction

The concept of patient engagement is described by the World Health Organization as “the facili- tation and strengthening of the role of those using services as coproducers of health, and health care policy and practice”.(1 p3) Through patient engagement in medicines Research & Devel- opment (R&D) the patients’ knowledge and experience are provided to scientists. Hence, it is possible to understand how it is to live with the condition, how care is administered, and how the medicines are used on a day-to-day basis, which can be included in the discovery, development, and evaluation of new medicines to make future medicines more useful for patients.(2) Addi- tionally, patient engagement is a concept imposed on the pharmaceutical industry by patients and patient organisations.(3-6)

The involvement of patients in medicines R&D has grown over the last decade, as seen in major coalition initiatives, such as the European Patients’ Academy on Therapeutic Innovation (EUPATI). The latter was established in 2012 as an Innovative Medicines Initiative (IMI) project by the European Patients’ Forum (EPF), an umbrella organisation working with patients’ groups across Europe.(3,7-9) EUPATI became a game-changer in patient involvement in medicines R&D by generating patient-driven educational resources to train patients, patient advocates, and the lay public in medicines R&D.(8) Today, EUPATI is still running under the leadership of EPF.(7) Another highly relevant initiative, also established in 2012, is the Patient-Focused Drug Development (PFDD) by the United States Food and Drug Administration (FDA) that later led to the development of guidance documents that will be specified in the background section.(10,11) Although the engagement of patients in medicines R&D is on the rise, it is not yet fully implemented as a standard practice neither in academic or commercial research.(12) This is partly due to the lack of well-tested, manageable recruitment techniques, reference to the objective of this master thesis. Furthermore, there is a lack of standardised approaches to assess the outcomes and impact of patient engagement. Also, the added value is not clear to all parties involved in patient engagement.(13)

The engagement of patients in medicines R&D requires platforms for the pharmaceutical industry to meet and recruit them for a role as e.g. advisors to R&D professionals. In Ballerup Municipality a group of partners was established with the aim of recruiting patients as advisors and insight providers in medicines R&D by using community pharmacies as a place of recruit- 9 ment. This way of recruiting patients will be described as a recruitment technique. The group of partners collaborating on patient engagement is Ballerup Pharmacy, Skovlunde Pharmacy, the Patient Organisation of Autoimmune Diseases in Denmark (FAIM) (Danish: Foreningen for Autoimmune Sygdomme), Ballerup Municipality, and LEO Pharma A/S. A partner-ship like this has not yet been established and is to be investigated and evaluated in this project. An ongoing IMI project called Patients Active in Research and Dialogues for an Improved Generation of Medicines (PARADIGM) describes three key decision-making points during R&D of medicines which are; research and priority setting, design of clinical trials, and early dialogues with regulators and Health Technology Assessment (HTA) bodies.(13) In this thesis, the focus will be on patient engagement in research and priority setting.

1.1 Thesis statement How can people with a rare disease diagnosis be identified in a community pharmacy to be engaged in the early research and development of medicines in a pharmaceutical company?

1.2 Research questions 1. Which recruitment technique can be constructed to recruit the patients in community pharmacies? 2. What is the patients’ attitude towards engagement in medicines research and development through this specific setting? 3. What do the recruited patients think about the project’s partners’ involvement in this project? 4. What are the project’s partners’ experiences with the execution of this kind of project where patients are recruited through community pharmacies?

1.3 Purpose The purpose of this master thesis is to concept-test and investigate how patients can be identified in a community pharmacy to be engaged in medicines R&D in a pharmaceutical company to enable a possibility to use the recruitment technique to recruit patients to medicines R&D in the future. The purpose is also to communicate experiences and potential suggestions for optimisa- tions of the concept, back to the partners of the project.

10 2. Background

In this section, the history of patient engagement will be outlined to obtain an understanding of how it has developed throughout the years. The legislation on patient engagement will be described to clarify what is allowed and prohibited to do in this area. To illustrate wherein the process of medicines R&D the patients can be engaged, a roadmap will be presented. Further, existing literature on the subject will be covered to be able to explain how patients have been engaged in studies previously, and to outline best practices that can be used to do the study design and analysis of the project. Finally, the idea for this project as well as the partners and their roles in the project will be presented.

2.1 History of patient engagement The engagement of patients in health care product development and delivery continuum has grown over the last three decades. In drug development, a paternalistic product-centric R&D paradigm has existed for over 50 years but is now changing into a patient-centric approach.(14) Information on the evolvement of patient engagement in the pharmaceutical industry is difficult to find and outline as they keep most of their initiatives confidential. However, regulatory agencies such as the FDA and the European Medicines Agency (EMA) are publishing patient engagement initiatives.(15,16) Regulatory agencies are progressively involving feedback from patients to inform decision making.(14)

The FDA has a long history of patient engagement dating back to 1988 where the HIV/AIDS patient group was founded.(17) The number of patient engagement activities in the FDA has been evolving over the years.(16,17) The FDA has multiple initiatives for patients, and two of those will be elaborated on.(18) One initiative is the FDA Patient Representative Program which is a programme that allows patients and caregivers to provide critical advice to the agency. In this role, they have the possibility to engage with scientific members and other experts by participating in the FDA Advisory Committees and panels.(19) As of today, the FDA has over 50 committees to obtain independent expert advice on scientific, technical, and policy matters.(20) The FDA included the patient perspective in the FDA Advisory Committee by having the first patient representative in 1991.(17) Another initiative is that one of the committees is dedicated to the patients, the Patient Engagement Advisory Committee, which is the first and only committee whose members are all patients, caregivers, and representatives of patient organisations. This 11 committee launched its first meeting in 2015 and allows a broader discussion of patient-related issues to advise the FDA. They may advise on topics as agency guidance and policies, design of clinical investigations, real-world data, the science of patient input, communication of device benefits and risks, and digital health technology.(21)

The EMA started their engagement with patients in 1995, the year the EMA was established.(15) They also have a long line of patient engagement activities evolving over the years.(22) Two initiatives have been selected to elaborate on. One initiative is the inclusion of patients as committee members which was started in 2000.(15) Today, the EMA has seven committees in which the patients are members and consultants on disease-specific requests.(23) Patients, consumers, and carers are involved as either one of the following, representatives of European Union patients’ and consumers’ organisations, representatives of their own organisation, or representing themselves as individual experts. They give their input to the different committees in the stages of medicines lifecycle present at the EMA, consisting of pre-submission, evaluation, and post- authorisation.(24) Another initiative is the Patients’ and Consumers’ Working Party (PCWP) which provides a platform for the EMA, and the patients and consumers to exchange information and discuss issues of common interest. The PCWP was established in 2006 and provides recommendations on matters of interest related to medicines to the EMA and its human scientific committees.(25)

In 2016, the FDA and the EMA started working together on the FDA/EMA Patient Engagement Cluster.(17) This is a workgroup on patient engagement that allows the FDA and the EMA to share best practices to involve patients in drug and biologic regulatory lifecycles. The discussions are covered by confidentiality agreements signed by both parties. The cluster workgroup discusses topics such as, how to find patients that appropriately speak for their community; how to ensure that patients involved in agency processes directly speak the concerns of their community; how to train selected patients and advocates to effectively participate in agency activities; and which strategies that can be used for reporting the significant impact of patient involvement. This workgroup meets up four times a year by telephone.(26)

Additional patient engagement history outside regulatory agencies could be a project called INVOLVE that was started in 1996 in the United Kingdom by the National Institute for Health Research. INVOLVE should achieve active public involvement in National Health Services, public health, and social care research.(27) In the United States, the Patient-Centered Outcomes 12 Research Institute was established in 2010 to include patients and other health care stakeholders throughout the research process.(28)

An example to show how patient engagement matters in the pharmaceutical industry could be Pfizer’s withdrawal of Exubera. In 2007, Pfizer had to stop the marketing of Exubera, which was a human insulin inhalation powder, after a year on the market. Exubera did not meet the patients’ wish for discretion as the device was too large and cumbersome to handle, and it was time- consuming when inhaling a higher dose of insulin.(29) The withdrawal of Exubera from the market ended up costing Pfizer 2.8 billion USD.(30) If patients had been involved in the development of Exubera, it might have saved time and expenses.

2.2 Legislation on patient engagement The legislation on patient engagement is rather limited and vague. However, the members of the Danish Association of the Pharmaceutical Industry (Lif) have committed themselves to follow a set of rules.(31) One of the rule sets is The Patient Organisations Code created by the Ethical Committee for the Pharmaceutical Industry in Denmark (ENLI) which covers the cooperation of the pharmaceutical industry and the patient organisations.(32) LEO Pharma is a member of Lif and thereby practises after both Lif’s and ENLI’s rules and codes.(33) The Patient Organisations Code was originally enacted by the pharmaceutical industry but was adapted to the code created by the European Federation of Pharmaceutical Industries and Associations (EFPIA) on 14 June 2011.(34) This code of practice exists for EFPIA’s members which should be followed, along with national codes, when the corporation with patient organisations takes place outside of Denmark but inside Europe.(35)

The purpose of ENLI’s Patient Organisation Code is to ensure, that the parties appear to be independent of each other, and any possibility of pressure or dependency should be prevented. The rule set states that agreements concerning funding must be clear and in writing and that the agreement should, as a minimum, be published on the website of the pharmaceutical company. Furthermore, the pharmaceutical companies must annually submit an overview of their collabo- rative projects to ENLI which is published on ENLI’s website. When an organisation provides any type of contracted service to a company, it is only permitted if the services are serving a purpose of supporting healthcare or research.(36) Organisations are allowed to be engaged as experts or advisors for services if a set of criteria is followed such as a written contract; a legitimate need for the services; records maintained by the 13 company; and the compensation must be reasonable. The rule set also states that the organisations should be independent and that no requirements on taking specific stands or favour specific products must be made. The main intention of the collaboration should be professional activities within the sphere of the organisation’s work. The activity must not contravene statutory regulations of information on medicinal products and advertising. Also, no exclusive agreements may be created as organisations are always free to collaborate with several pharmaceutical companies, and if issues of impartiality or independence of the parties are open to challenge, agreements must not be concluded.(36)

2.3 Roadmap for patient engagement in medicines R&D Guidance documents to include patients in medicines R&D were not available until 2018 where Warner et al.,(37) as a part of EUPATI, created a guidance document for four different engagement types: pharmaceutical industry-led medicines R&D, ethics committees, regulatory authorities, and HTA. However, no defined, recognised code of practice is available for patient involvement in R&D.(37) The guidance documents suggest at what points patients can be engaged,(37) and for the pharmaceutical industry-led medicines R&D, a practical roadmap was created by Geissler et al., 2017.(38) The roadmap is based on multiple stakeholder discussions that took place from 2013–2016 including EUPATI workshops, European Patient Advocacy Leadership Council Oncology workgroups, The Organisation for Professionals in Regulatory Affairs and EMA Annual Review conference, and lastly Amgen Europe workshops.(38) In this roadmap, the lifecycle of medicines R&D is divided into four stages which are, Research Priori- ties; Research Design and Planning; Research Conduct and Operations; and Dissemination, Communications, and Post-approval (Figure 1). Further, the expertise level of the patients is included in the roadmap, either medium or high expertise level in the disease area is required.(38)

Figure 1: Practical roadmap for patient involvement in medicines R&D.(38)

Stage 1 is the Research Priorities, an important place of involvement of patients to ensure that research priorities align with unmet medical needs for the patients to ensure the relevance of outcomes.(38) Other benefits are that research questions, hypotheses, interventions, and medical technologies turn out to be more relevant and useful to the patients. Additionally, more appropriate resource allocation and new funding opportunities emerge.(13) Stage 2 is the Research Design and Planning, which includes, protocol synopsis, protocol design, fundraising for research, practical considerations, patient information, informed consent, and ethical review.(38) Patient engagement is beneficial as it can result in more relevant endpoints or outcomes of the research as well as a higher rate of recruitment and retention in clinical trials. In addition, patient information and informed consent can become patient-friendlier regarding read- ability and content.(13,38) Stage 3 is the Research Conduct and Operations that covers trials steering committee; information to trial participants; investigator meetings; data, and safety monitoring committee; and study reporting. Engagement in this stage gives the possibility to anticipate challenges or opportunities of trial recruitment and gives insight into the issues that might be related to side effects, formulations, administration, or adherence to the medicine.(38) Stage 4 is the Dissemination, Communication, and Post-approval, which includes regulatory affairs, HTA, and post-study communication. Engagement at this point gives a possibility to 15 communicate results for nonexperts or lay audiences to explain the potential benefits and risks for patients. Also, the evaluation of the identified patient priorities and patient-relevant outcomes can be assisted with patient engagement.(13,38)

As mentioned above, no defined, recognised code of practice is yet available. Hence, an ongoing IMI project, Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle, aims to provide a better understanding of the recommended best-practice approach to patient- preference studies.(39) Furthermore, the IMI project PARADIGM, mentioned in the introduction of this thesis, has a mission “to provide a unique framework that enables structured, effective, meaningful, ethical, innovative and sustainable patient engagement (PE) and demonstrates the ‘return on the engagement’ for all players”.(40 p1) These two IMI projects show that a defined, recognised code of practice is needed to structure the process of patient engagement. Thus, guidance documents and frameworks are work in progress. Very recently, in June 2020, the FDA has published the first guidance in a series of four methodological PFDD guidance documents as a part of the PFDD initiative mentioned in the introduction.(10) The first guidance document outlines how to collect comprehensive and representative patient input. It covers an overview of how to conduct a study with patient input, including defining research objectives and questions; defining the target population; determining the study design and research setting; constructing a sampling frame; and how to standardise and operationalise the data collection and data manage- ment.(41)

2.4 Coverage of existing literature The purpose of the literature search was to broadly investigate the literature in the field of patient engagement in drug development to gain an understanding of the topic as well as include it in the analysis and processing of data achieved in the thesis. The literature search has been performed in October and November 2019 by searching the databases PubMed, Embase, and International Pharmaceutical Abstracts (IPA).(42-44) The search was divided into two blocks combined with ‘AND’. Block 1 covers the drug development part by combining the keywords with ‘OR’. Block 2 covers the patient participation part by combining the keywords with ‘OR’ (Table 1).

16 Table 1: Keywords used in the literature search. Block 1: Drug development (combined with OR) Block 2: Patient participation (combined with OR) 1. Drug development (PubMed term) 1. Patient participation (PubMed & Embase term) 2. Drug development (Embase term) 2. “Patient participation” 3. ”Drug development” 3. “Patient engagement” 4. Drug research (Embase term) 4. “Patient involvement” 5. ”Drug research” AND 5. “Patient and public involvement” 6. ”Medicines development” 6. “Patient centric” 7. ”Pharmaceutical industry-led medicines R&D” 7. “Patient centricity” 8. “Medicines research and development” 8. “Patient centered” 9. “Pharmaceutical development” 9. “Patient input” 10. “Medication development” 10. “Patient-reported outcomes” 11. “Patient focused medicines development”

The search resulted in 373 hits on PubMed, 591 hits on Embase, and 15 hits on IPA. The hits were screened by looking at the titles and abstracts of the articles and papers. The 13 most relevant articles were then selected to elaborate on how to involve patients in medicines R&D. The outcome of the literature search could be grouped into the topics: Stakeholder expectations, best practices of patient engagement, a master framework needed, and recruiting patients, which are presented below.

2.4.1 Stakeholder expectations In the literature, an agreement exists, that patients should be involved during the entire process of medicines R&D.(38,45,46) It is considered a major barrier that patient engagement is the industry’s attempt at ‘courting’ or ‘coercing’ patients instead of a first step in a mutually beneficial co-creation to find a solution to shared challenges.(47) To do this, a clarification of expectations and roles between stakeholders is necessary, and the added value should be clear.(13,46,48- 50) In this way, mutual trust between stakeholders, as well as transparency and dialogue can be achieved.(45,49,51) In a study by Boudes et al., 2018, the stakeholder expectations were studied, outlining what one stakeholder expected from the other stakeholders (Figure 2).(48)

Figure 2: Stakeholder expectations matrix.(48)

Interesting views to point out are, that patients expect from the industry and researchers to involve patients in all stages of drug development. Though, the industry expects from the researchers to seek active input in all stages of the drug development. However, the researchers expect from the industry to involve the patients in end-to-end development by setting a framework. Building a patient voice in an end-to-end development is also what the industry expects from themselves. Hence, the industry expects from the patients to participate in clinical trials. Another expectation from patients to the industry and researchers is that they ensure that research addresses the patients’ needs. According to the industry, this requires that the patients inform them about unmet medical needs through continuous dialogues. The patients are also expected, by the industry, to deliver individual and global experience. To ensure the patients’ needs, the researchers expect from the patients to advise the researchers in setting research priorities, reviewing research design, and supporting the implementation of research projects. Furthermore, the researchers expect that the patients can provide emotional, practical, and informational support to patients participating in research. The researchers also expect from themselves to understand patient priorities and secure the value of patient inputs. Last, the patients expect from the industry and researchers to take a holistic view of their requirements, which is what the industry expects from the researchers to ensure.(48)

18 2.4.2 Best practices of patient engagement Transparency of patient engagement is important to increase, which the company can do by disclosing their collaborations annually. A suggested working practice is to have internal cross- functional coordination of patient involvement in the company. Furthermore, a dialogue in the form of pre-engagement discussions is important to ensure mutually beneficial interaction and the right preparation. EUPATI recommends defining the interaction to have clearly defined processes and actions, as well as a timeline.(37) Different suggestions on how to do patient engagement best are described in the literature. It is stated that when engaging patients in the development of medicines in the pharmaceutical industry, it is best to involve a patient organisation in order to provide support and advice.(37,51) A suggested working practice is to foster and establish long-term partnerships between patients, patient organisations, and industry to deliver benefits for all parties.(37) It is also mentioned that patient education is important to give the patients a better understanding of the drug development process.(49) If the patient is engaged as a co-researcher, training may be provided to individuals with less knowledge in medicines R&D.(46) Furthermore, online health communities allow a cohort of patients to participate in drug development.(52) A specific way could be “The use of mobile devices to interact actively with participants in clinical trials may be a new way of engaging and empowering patients.”(53 p1013) The patient engagement, when engaging directly with the patients, should be achieved by actively listening to the patient.(45,46) The diversity of patients and their opinions should be recognised.(45) All ideas are valuable, and no ideas should be criticised.(46)

2.4.3 A master framework needed Another common agreement is that accepted, master platforms or frameworks for systematic patient involvement in medicines R&D should be established because a more structured systematic approach to patient engagement is needed to give the consistency that currently lacks.(13,37,38,45,47-49,54) However, the expectations on who should develop the master framework differ from each stakeholder group. The payers and the industry expect the policy- makers or regulators to create a framework whereas the researchers expect the industry to set the framework.(48)

2.4.4 Recruiting patients Places to identify patients to participate in medicines R&D can be patient advocacy organisations that recruit qualified individuals to be a part of research.(37,46) Also, training programs, such as EUPATI, can be used to recruit well-trained, highly qualified patients.(37,46) Using existing

19 advisory boards/groups or ‘patient panels’ can also be a way to identify patients.(37,46) Patient community platforms, e.g. online platforms, may be used to recruit patients.(46) Furthermore, existing relationships with healthcare providers, hospitals, and researchers are ways to recruit patients.(37) In conclusion, the literature shows a common agreement of including patients in end-to-end development. However, a master framework on how to do this in practice lacks but best practices of patient engagement are described. For instance, it is suggested for the industry to have a long- term partnership with a patient organisation and to educate the patients in drug development. Additionally, a broad consensus exists that the expectation between the stakeholders should be clarified and that transparency is of the essence. When recruiting patients, the following places are suggested, among others; patient organisations, training programmes, and advisory boards.

2.5 Roles and responsibilities in the project The idea for this project came from Ballerup Municipality during autumn 2018. The municipality had a meeting with LEO Pharma about engaging with the citizens in the municipality. Afterwards, the municipality had a meeting with Ballerup Pharmacy and Skovlunde Pharmacy, respectively, and they discussed the possibility of working with LEO Pharma, which both pharmacies were interested in. The collaboration was further developed, and FAIM joined the project along with the master thesis student, making this project the centre of a master thesis. The concept idea was to establish a pioneering local partnership with the purpose to complete a Proof of Concept (PoC) assessment of a new technique to recruit patients from community pharmacies to be involved in medicines R&D. By using the local pharmacies to recruit patients by informing about the project, the potential patients would be identified and offered participation. The project was also promoted by online campaigns by the partners of the project.

The partners had different roles in the project to ensure project execution in the best possible way. The roles and responsibilities are described as perceived by the master thesis student. The role of Ballerup Municipality was to coordinate preparational meetings and ensure that collaboration was initiated successfully since they were the project originator. They collaborated with LEO Pharma on a press release about the project. The municipality also drafted an article about the project, which was published in the local newspaper, Ballerup Bladet. Additionally, they posted a piece about the project on their own social media channels. The municipality also reviewed the written information materials on the project. Subsequently, they accepted to display printed project materials consisting of a roll-up and brochures at their City Hall. 20 The roles of both pharmacies, Ballerup Pharmacy and Skovlunde Pharmacy, were to make their facilities available to the project and potential patients by displaying printed project material and having the master thesis student present on different days. Also, some of the pharmacy staff should be able to inform about the project and know how to handle interested patients. The pharmacy owners were a part of the preparational meetings to make decisions on project execution, as well as reviewing written information material about the project.

FAIM had the role of ensuring that decisions made during the preparational meetings would benefit the patients and reach out to them. They contributed by arranging and donating a website for the project, as a part of their website, and they posted information on their own social media channels. Additionally, they reviewed the written project information material. Furthermore, FAIM was a part of the training of the pharmacy staff on the patient part of this project.

LEO Pharma’s role in the project was mainly to secure that the interested patients would be offered a workshop with scientists at LEO Pharma’s headquarters in Ballerup, including the execution of the workshops and related tasks. LEO Pharma was also a part of the preparational meetings to make decisions on project execution. They collaborated with Ballerup Municipality on a press release about the project. Furthermore, LEO Pharma was the co-supervisor to the master thesis student including advising on student tasks, reviewing written information materials, and assisting in the training of the pharmacy staff. Finally, LEO Pharma sponsored written information materials including roll-ups, brochures, and zebra rugs placed in the pharmacies and the City Hall.

The master thesis student’s role was the project coordinator including describing, driving, and reporting the project. This involved the following responsibilities; to create and set the frames and the plan of the project; following up on partners’ actions from meeting minutes; and keeping all partners in the loop. The student was responsible for securing and organising dates for pharmacy staff training, developing the content of the training, and participating in the training. Additionally, the student was responsible for creating the content for all written materials including roll-ups, brochures, and the website. Furthermore, the student had a dialogue with a design company about the layout of the roll-ups and brochures, as well as designing a logo for this specific project for all written materials. The student also handled the purchasing of the zebra rugs that were used in the pharmacies. Afterwards, the student delivered the roll-ups, brochures, and zebra rugs to the pharmacies and the City Hall. The role of the student was also to be the 21 project representative at the pharmacies eight hours a week during the three-month recruitment phase. All interested patients were directed to the student who referred the patients to LEO Pharma. When the patients contacted LEO Pharma, the student was responsible for ensuring an introduction call between patients and LEO Pharma. Subsequently, the student was responsible for collecting the patients’ information to create contracts between patients and LEO Pharma. The student also collected informed consent from all patients to include their data in the master thesis.

2.6 About the partners of the project Ballerup Municipality has more than 48,000 citizens and 35,000 citizens from other municipalities commuting to work in Ballerup every day. The municipality is one of the leading in the field of business and wants to contribute to the growth of the companies in the municipality. When networking opportunities arise, the municipality introduces companies to each other to establish a corporation to create growth and innovation.(55) This project may create growth for the companies due to more attention to the pharmacies and LEO Pharma, which hopefully imply an increase of customers. Furthermore, this type of project will draw attention to the municipality, which could result in start-up companies or companies moving to the municipality, which may create growth in the municipality. Also, the lay public might think of the municipality as a good place to live which may result in them moving there, and thus create additional growth in the municipality.

Ballerup Pharmacy has four branches whereof three, as well as Ballerup Pharmacy, are placed in Ballerup Municipality. They offer a long line of health services.(56) Skovlunde Pharmacy is also placed in Ballerup Municipality and offers a long line of health services. They have one branch, which is not placed in Ballerup Municipality.(57) The liberalisation of the Danish community pharmacies law in 2015, has entailed the opening of 141 new community pharmacies over 2.5 years. Included in the 141 new community pharmacies are two online pharmacies that are allowed to dispatch prescriptions equivalent to physical community pharmacies. This increases the competition between the pharmacies which has resulted in longer opening hours and less waiting time.(58) Hence, the availability of medicines is better for the consumers, but it requires much more of the pharmacy to stand out. This type of project, where patients are recruited to engage in medicines R&D through the pharmacies, gives this possibility to stand out.

22 FAIM is an organisation for patients who have autoimmune diseases and their relatives. Over one million Danish citizens have at least one autoimmune disease.(59) Several rare diseases with skin manifestations are autoimmune, thus FAIM embraces a large part of the participants in this project. They are the voice of all patients with autoimmune diseases, also when this project was developed to ensure the project was communicated and constructed in the best way for the patients to follow. With this project, they can contribute by giving their members or potential members the possibility to participate in the development of new medicines. FAIM’s participation in this project will potentially draw attention to their organisation, which may result in more members of FAIM, drawing even more attention to autoimmune diseases.

The pharmaceutical industry may have a hard time recruiting e.g. patient advisors, especially when it comes to rare diseases due to the small number of people with the disease. LEO Pharma is a global pharmaceutical company that develops drugs to dermatologic and thrombotic patients in more than 100 countries globally.(60) The company headquarters is in Ballerup Municipality, Denmark. The rationale for LEO Pharma to enter this partnership is to investigate if the project could be a new, pioneering concept where patients may know about the option to engage in medicines R&D. LEO Pharma considers the project a PoC assessment, which could inspire industry in and outside of Denmark to build similar partnerships; if PoC is obtained. In May 2018, the company expanded its mission into rare diseases, and in November 2018, started a collaboration with PellePharm, a biotech company, to address unmet medical needs in the field of a disease referred to as Gorlin Syndrome.(61) This and many other rare diseases are not investigated well enough to develop medicines, which means that research in rare diseases requires patient engagement (advice and insights) in addition to clinical trial participation by patients to succeed. Therefore, this project focuses on patient engagement within rare diseases with skin manifestations.

23 2.7 Conclusion The background in this thesis has covered information on patient engagement, including the history, legislation, existing literature, best practices, and stages of involvement. This gives the possibility to create more specific questions for the interview guides to the patients and the partners of the project. Views on patient engagement have been presented which can be investigated in the interviews and afterwards compared to the literature. The best practices of patient engagement can be used when analysing the results of the interviews giving a better insight and understanding. Finally, some of the opinions and expectations of the patients or stakeholders might better be recognised or understood after exploring the literature in this area.

24 3. Methodology

This project aims to investigate how to recruit patients with a rare disease diagnosis in local community pharmacies to be engaged in early medicines R&D in a pharmaceutical company. The Plan-Do-Study-Act (PDSA) cycle was chosen to develop the intervention presumably not previously seen, which is a new recruitment technique to identify patients in local community pharmacies to be involved in medicines R&D. The PDSA cycle is a process that provides a possibility to test a change on a smaller scale before implementing it on a broader scale. The cycle is divided into four parts, Plan, Do, Study, and Act. Plan describes the change to be tested or implemented by defining objectives, questions, and predictions. Do is the execution of the change. Study is the evaluation of the change by completing data analysis and summarising what was learned. Act is the recommendations of what to do next which could be to plan the next cycle or implement the change fully.(62) The use of a PDSA cycle in this study also enables a possibility to distinguish between the non-scientific, practical activities, and the scientific methods of the study. The non-scientific part is described under the Do-part of the PDSA which includes the practical activities related to the project. These activities are described in this thesis to understand the recruitment technique and why the scientific evaluation of the project was performed this way. The scientific evaluation of the intervention is described in the Study-part of the PDSA (Figure 3).

Figure 3: The PDSA cycle describing three of the four parts of the project. 25 Plan and Do will be reported in this section, methodology, whereas Study will be reported in the results section, and Act will be reported in the discussion section.

3.1 Plan A preparational meeting, leading up to the project, was held in February 2019 and followed up with a meeting in August 2019. These meetings were an essential part of planning the project where various decisions were made. First, it was decided that the project should consist of the following phases: • a preparational phase, • a recruiting phase, • a patient engagement phase, and • an evaluation phase.

The preparational phase was leading up to the recruiting phase and started gradually from the initial meeting in February 2019 and more extensively from 1 September 2019 until 31 October 2019. At these meetings, it was decided that the project should be the centre of a master thesis with the master thesis student as the project coordinator. The parties agreed that the recruiting phase for the project should run over three months from 31 October 2019 to 31 January 2020. In the preparational phase, the partners agreed that the master thesis student should be present at each pharmacy, Ballerup and Skovlunde, once a week for four hours. In this way, the student was available to the patients in a total of eight hours each week to recruit patients to medicines R&D and answer potential questions. Furthermore, the patients’ attention should be gained by placing brochures (Appendix 1), roll-ups (Appendix 2), and zebra rugs at the pharmacies and the City Hall in Ballerup Municipality. The zebra pattern for the rugs was chosen as the zebra is the official symbol of rare diseases,(63) and it was assessed that this would generate attention.

The target group was patients living with rare diseases with skin manifestations and it was set by LEO Pharma as they were the party inviting patients to participate in medicines R&D. Apart from this inclusion criteria, the following inclusion criteria were decided: Danish or English speaking, over 18 years old or with consent from parents, and willing to travel to LEO Pharma in Ballerup. To gain the patients’ attention, besides from the presence at the pharmacy, it was decided to initiate a marketing campaign. Additionally, the patients were urged to visit the pharmacies on the days the master thesis student was there. A website for the project was created, in collaboration with FAIM, to attract attention to the project and create a channel to register for the 26 project (Appendix 3).(64) Furthermore, it was decided that the partners should post the project on their social media channels and websites. The master thesis student should also post a campaign in different Facebook groups for rare diseases. To draw attention to the project, it was decided to write a press release, which was posted by LEO Pharma on Ritzau (Appendix 4).(65) Additionally, an article about the project written by Ballerup Municipality was sent to a local newspaper, Ballerup Bladet, that posted the article on 10 December 2019 (Appendix 5).(66) The pharmacy staff should also be prepared to answer questions about project content since they were advertising for it in the pharmacy. Therefore, the partners decided that the staff should be trained in the project content prior to the recruiting part of the project. After the recruiting phase, the patient engagement phase was started and ran from February 2020 until May 2020. In this phase, the patients were invited to a workshop with scientists at LEO Pharma, which will be described below in the Do-part. Last, it was decided that the master thesis student should evaluate the practical activities in the project as a centre of the master thesis. This was conducted in the evaluation phase running from February 2020 until August 2020, which will be described in the Study-part below.

3.2 Do The practical activities in the project were split into six different parts (Figure 4).

Figure 4: Flowchart describing the order of the practical activities in the project.

27 1. Training of pharmacy staff Training of the pharmacy staff included how to handle the project and how to inform the patients about it. Attending the training, were FAIM, the master thesis student, and LEO Pharma as a supervisor to the student. One training session was conducted at each pharmacy for 30–45 min. A sheet of Frequently Asked Questions was provided by the student to the staff, which included practical details, inclusion criteria, disease definitions, and how to handle certain situations.

2. Marketing of the project Marketing consisted of communication on the project via different channels. The project was posted on the following sites or media: • Roll-ups and brochures at the pharmacies and the City Hall in Ballerup Municipality • The project’s website (Appendix 3)(64) • As a press release (Appendix 4)(65) • The partners’ social media and websites • Facebook groups about rare diseases • Local newspaper, Ballerup Bladet (Appendix 5)(66)

3. Recruitment process The recruitment process covered the part where patients contacted the master thesis student to participate in the project. The patients could become aware of the project through the channels described above under 2. Marketing of the project. The channels where the patients could register to participate in the project were the following: • The pharmacies when the master thesis student was present • The project’s website through a contact formula • Sending an e-mail to the master thesis student • Calling or texting the master thesis student by phone • Facebook messaging to the master thesis student

Even though it was not a part of the plan, some patients ended up contacting LEO Pharma directly by phone or e-mail. LEO Pharma then forwarded the request to the master thesis student. At each pharmacy, Ballerup and Skovlunde, the master thesis student was present once a week for four hours during the three months the project was running. The master thesis student has spent 80 hours at the pharmacies in total. At the pharmacies, the master thesis student had dia-

28 logues with the potential patients which included information about the project, questions to the project, and how to participate in the project. When the patients contacted the master thesis student, LEO Pharma helped the student determining if the given disease was a rare disease with skin manifestation in the scope of this project. The additional inclusion criteria, as stated above, were exclusively assessed by the master thesis student: Danish or English speaking, over 18 years old or with consent from parents, and willing to travel to LEO Pharma in Ballerup. If the patient was in the scope of the project, the master thesis student encouraged the patient to contact LEO Pharma via e-mail as the student could not share the patient’s contact info with LEO Pharma without consent in accordance with data protection regulations.(67) So, it was decided that the patient should contact LEO Pharma directly to participate in the project.

4. Patients introduced to project at LEO Pharma As the patients contacted LEO Pharma, they were invited to a short introduction call with LEO Pharma. This call lasted for approximately half an hour via Skype or phone. The patients went through this to learn more about the project and to confirm they wanted to participate in a workshop with scientists at LEO Pharma.

5. Patients prepared for workshop at LEO Pharma The patients, who ended up being interested in, able to, and offered a workshop with LEO Pharma’s scientists, were invited for a preparation call approximately one week prior to the workshop. On this call, they were introduced to the agenda, presented to minor home assign- ments, and they could address potential questions prior to the workshop.

6. Workshops – patients as advisors to scientists at LEO Pharma Patients as advisors to scientists at LEO Pharma consisted of two separate 1,5-hour virtual sessions via Microsoft Teams which gave the patients the possibility to share their knowledge about their disease, e.g. their everyday life and unmet medical needs. The sessions were converted into virtual meetings through Microsoft Teams due to the COVID-19 pandemic in 2020. Two different workshops were held, divided into disease categories. The first one regarded the disease Scleroderma where the patients had the chance to tell about their disease journey. This included the onset of symptoms, time of diagnosis, treatment currently and previously used for their disease, and how they are managing their disease today, amongst other things. After the presentations, there was a Question & Answers session where the scientists at LEO Pharma could ask questions to the patients. The second one regarded the diseases Lichen Planus and 29 Lichen Sclerosus, where the patients introduced themselves one by one, describing their life with the disease both physically and psychologically, and how their disease has been treated medically and non-medically. Furthermore, a leader of a patient organisation presented results from a Lichen Sclerosus patient survey. In the end, there was a Question & Answers session where the scientists at LEO Pharma could ask questions to the patients.

3.3 Study The scientific part of the project, conducted by the master thesis student, is described under this part of the PDSA cycle. The following five methods were selected to answer the thesis statement and the research questions: A. Individual semi-structured interview with the project’s partners B. Quantitative overview of patient contact C. Focus group interview with patients before the workshop D. Individual semi-structured interview with patients after the workshop E. Individual semi-structured interview with patients who did not participate in a workshop

These methods have been used in different steps of the Do-part of the PDSA cycle (Figure 5).

Figure 5: Flowchart describing where the Study-part of the project is connected to the Do-part of the project. 30 The methods have been chosen due to the following reasons. A. Individual semi-structured interview with the project’s partners was selected to investigate what the partners think and feel about how the project was completed as outlined in research question four. The interview was chosen to be semi-structured as it is suitable when investigating thoughts and feelings as the participants are provided with an opportunity to express and develop their views during the interview. Moreover, the researcher could explore themes and areas that arise unexpectedly during the interview.(68)

B. Quantitative overview of patient contact was chosen to assess the recruitment technique in this project as mentioned in research question one. By providing an overview, information about the patients who made contact to participate in the project could be reported and investigated in an objective matter as a part of assessing the recruitment technique. The data could be quantified to make it easier to aggregate, compare, and summarise data, which provided an improved way to assess if the recruitment technique worked.(69) The use of a quantitative method in combina- tion with the qualitative interviews, and thereby using a multi-strategy design, gave the benefit of a broader and more comprehensive picture of the topic, in this case, the recruitment technique. Furthermore, the findings could be explained using different approaches and giving different answers to the research questions.(70)

C. Focus group interview with patients before the workshop was chosen mainly due to practical reasons around conducting this interview right before the workshop with scientists at LEO Pharma where several patients should participate simultaneously. Also, a focus group interview is a highly efficient method of qualitative data collection as numerous people participate at once.(71) The focusses were decided to be the recruitment process, expectations of the workshop, and opinion about the partners, to investigate research questions one, two, and three in this thesis. Furthermore, focus group interview was chosen as the method is good when seeking group dynamic, and to empower the interviewees to reflect on the topics as they are stimulated by the others in the group.(71) The focus group interview was also chosen to be semi-structured as thoughts and feelings were to be investigated, as described above.

D. Individual semi-structured interview with patients after the workshop was chosen to examine what the patients feel, think and believe about patient engagement in medicines R&D, especially in the activity they just participated in; the workshop at LEO Pharma. This method was chosen to

31 investigate research questions one, two, and three. This interview was also chosen to be semi- structured as thoughts and feelings were to be investigated, as described above.

E. Individual semi-structured interview with patients who did not participate in a workshop was chosen to explore the same as in the methods C and D except the experiences with the workshop at LEO Pharma. This interview was also chosen to be semi-structured as thoughts and feelings were to be investigated, as described above. The methods were grouped into two parts, partners and patients, as the study populations differ. However, all the interviews and focus groups conducted have a line of method conditions in common which is outlined in the below section. Afterwards, only the method conditions in each method that differ from the rest of the methods are described individually.

3.3.1 Similar method conditions for interviews and focus groups The concept, seven phases of an interview investigation, defined by Kvale and Brinkman, 2015, was used to plan, execute and process all interviews conducted, both individual and focus group interviews.(72) The concept was chosen to avoid a chaotic interview process and related troubles as well as to secure that the researcher’s plan, vision, and engagement are followed as intended through the interview investigation. Thereby, the quality of the knowledge produced through the interviews will be higher, and the analysis will be easier. The steps in the seven phases are 1) Thematising, 2) Design, 3) Interview, 4) Transcription, 5) Analysing, 6) Verification, and 7) Reporting. In step 1, the purpose of the investigation is established, as well as the understanding of the theme. In step 2, the investigation’s design is planned by considering all seven phases before the actual interviews are performed. In step 3, the interviews are completed by using an interview guide and a reflected approach to the knowledge. In step 4, the interviews are transcribed from spoken language into written text to prepare the material for analysis. In step 5, the interviews are analysed by using an appropriate method that fits the purpose and subject of the investigation. In step 6, the validity, reliability, and generalisability are determined to verify the interviews. In step 7, the methods and results of the investigation are reported in a format that meets the scientific criteria.(72)

1) Thematising • Why: the partners of the project were interviewed to evaluate the recruitment part of the project and examine their expectations and opinions on patient engagement in medicines R&D as described in research question four. The patients were interviewed to investigate

32 research questions one, two, and three in the thesis which is the patients’ attitude towards engagement in medicines R&D, and the patients’ thoughts about the partners involved in the project. This setting of recruiting patients to be engaged in medicines R&D is new and requires an evaluation of the patient’s side of the story. • What: the acquisition of knowledge was carried out in the background section. The use of this knowledge to develop the interview guide is described below. • How: incorporated in the design of the interview which is outlined below.

2) Design The interview type chosen for all interviews and focus groups is semi-structured interviews as described above. The interviews were planned to be conducted as face-to-face interviews prefer- ably, but if the interviewee wanted the interview to be virtual this would also be a possibility. However, all face-to-face interviews, except one, were converted into virtual or telephone interviews, due to the COVID-19 pandemic in 2020. The virtual platforms used to conduct the virtual interviews were Skype, Whereby, and Microsoft Teams. The interview language chosen was Danish as it is both interviewer’s and all interviewees’ native language. A detailed, yet flexible interview guide was made for all interviews. In semi-structured interviews, the interview guide can include anything from a series of subjects to a specific order of carefully phrased questions.(71,73) The latter was used in these interview guides but with the possibility to change the order of questions, if more applicable, and seek new directions. It was chosen to phrase the questions more carefully to ensure that all relevant aspects of the study were covered to achieve a higher quality of the analysis. When the interview situation is more structured, the units of meaning will be easier to identify in the analysis which is also an advantage.(73) As the interviewee’s thoughts and feelings were sought in all interviews, open-ended questions were chosen where possible. Open-ended questions were chosen due to the following reasons; they are flexible; they give a possibility of gaining a more truthful assessment of the interviewee’s believes; they allow a more in-depth interview; they clear up misunderstandings; they encourage cooperation and rapport; and they can produce unexpected or unanticipated answers.(71)

3) Interview All interviews were audio-recorded. All interviewees gave their written informed consent to have their data and information from the interviews included in this thesis. 33 4) Transcription All interviews were transcribed, and the transcriptions were completed as a word-for-word transcription, including ‘uh’, thinking pauses, and emotional expressions such as laughter. When quotes were made for the result section, they were transformed into written language and trans- lated from Danish to English (Appendix 6).

5) Analysing All interview and focus group data were analysed using meaning condensation which consists of five steps. Step 1 is to do a read-through of the interview to get an overall impression. Step 2 is to decide and mark the natural units of meaning which are quotes expressed by the interviewees. Step 3 is to thematise the quotes from the interviewees as understood by the researcher by rephrasing the theme to be as simple as possible. Step 4 is to raise questions to the units of meaning in relation to the purpose of the study. Step 5 is to connect the essential, non-redundant themes and turn them into a descriptive statement.(74) General for all interviews, unexpected and unanticipated data were searched for in the analysis and are presented in the result section. At last, the data were analysed in the original, Danish version. To follow the analysis trail, one patient interview transcript (Appendix 7), and the corresponding meaning condensation scheme (Appendix 8) have been attached.

6) Verification The interviews were verified by determining the reliability, validity, and generalisability of the interview results. The reliability is the credibility and consistency of the results and is often used by assessing whether the study can be reproduced by other researchers.(75,76) To secure reliability, even though it can be hard to achieve with qualitative research, an audit trail of the process was made. The validity is the truth or the strength of a statement.(75,76) To secure validity, the interviews were audio-recorded which contributes to the validity. When the data was interpreted, the route to reach this interpretation was traced by using meaning condensation schemes. Also, alternative explanations and understanding were searched for by identifying data not consonant with the literature.(76) Generalisability is accomplished by assessing if the results from the study can be transferred to other subjects, contexts, or situations. To contribute to generalisability, no selection between the recruited people was made. Analytic generalisability is a thorough assessment of whether the results can be transferred to other subjects or situations which includes the researcher’s arguments. The analytic generalisability was also assessed with regard to the results of this study.(75) 34 7) Reporting The interview results are reported in the results section of this thesis.

3.3.2 Ethical considerations To secure ethical principles, it is very important to have informed consent from the participants in a study.(77,78) Informed consent was obtained from every participant in the interviews and focus groups to ensure the anonymity and confidentiality of the participants. In the consent form, it was also stated that the participants can withdraw from the study at any point. The European Union General Data Protection Regulation was also followed by ensuring that the participants’ information was kept safe.(67) This was ensured by storing the data in the student’s OneDrive provided by the University of Copenhagen and deleting copies of recordings and transcriptions on local drives. The recordings and transcriptions from this study will also be deleted after this master thesis has been assessed. Furthermore, the participants were listened to during the interviews and given the option of not responding to a question if it made them uncomfortable. Last, for the partner interviews, it was approved, by each representative, that the results would be reported using the designation of the company, for instance ‘municipality’, where relevant. When the subject was considered a more sensitive matter, the results have been anonymised to the extent possible. Also, the source of each partner quote is not provided to ensure anonymity. Furthermore, an example of a partner interview transcript and corresponding meaning condensation scheme were not attached, as they could not be anonymised satisfactorily.

3.3.3 The project’s partners

3.3.3.1 Study population The inclusion criteria for the partners were; knowledge about the project process, and willing- ness to represent their organisation or company. The interviewee was recruited by asking the different partners to identify a representative to be interviewed. This can be argued to be purpos- ive sampling as the sample was built up to enable the researcher to satisfy the specific needs in a project.(79) In this case, the involved parties had to be interviewed to evaluate the project.

3.3.3.2 Method A. Individual semi-structured interview with the project’s partners Semi-structured interviews were completed individually with a representative from each partner, Ballerup Pharmacy, Skovlunde Pharmacy, FAIM, Ballerup Municipality, and LEO Pharma.

35 2) Design The interview guide was partly made by using the stakeholder expectation matrix (Figure 2) from the background section that provided different views and expectations on patient engagement. The stakeholder expectation matrix outlines what each stakeholder expects from themselves, what the stakeholders expect from each other, and whom they think should develop a master framework for systematically involving patients in medicines R&D.(48) These specific learnings were added to the interview guide to ask the partners of the project about their thoughts and opinions regarding this. An example of a question emerging from the background section: “What do you think your role as a pharmacy/municipality/patient organisation/industry should be in patient engagement in drug development?”. Further, the student’s experiences from the project execution were used to develop the interview guide by asking about specific circumstances. One circumstance was how the project has evolved from the preparational meetings to the execution of the project, where an example of a question is: “How do you experience the development of the specific project, from the first thoughts/meetings to the execution of the project?”. Other circumstances were the value of having a representative (the master thesis student) present at the pharmacies, and who should run a project like this if not the master thesis student. An example of a question is: “How do you see this project being carried out practically without a master thesis student?”. Finally, evaluation questions, as what was good, what was less good, and suggestions to improve the project, were asked. After the first interview, some questions in the interview guide were rephrased slightly and clarified to give a better understanding (Appendix 9).

3) Interview The flow was quite good in all interviews, and most questions were understood correctly. The first interview was face-to-face which gave a better flow and quality than the rest of the interviews that were conducted via video calls. However, the video calls went well and with only minor technical issues.

5) Analysing During the analysis of the interviews, learnings from the background section regarding stakeholder expectations (48) were kept in mind to better recognise or understand the opinions and expectations from the stakeholders. One learning, that was searched for in the analysis, was the partners’ expectation to whom should be responsible for developing a master framework to systematically involve patients in medicines R&D.(48) Other learnings that were searched for were 36 their expectations to themselves as well as who should be responsible for involving patients in medicines R&D.(48) Additionally, the partners’ views and suggestions for best practices of patient engagement in medicines R&D (37,45,46,49,51-53) were kept in mind to better understand the results of the analysis.

3.3.4 Patients in the project

3.3.4.1 Study population The inclusion criteria for the patients were the following; living with a rare disease with skin manifestations, Danish or English speaking, over 18 years old or consent from parents, and willing to travel to LEO Pharma in Ballerup. The interviewees were recruited during the three- month recruitment phase which is described above in the Do-part. It should be noted that some patients, who contacted the student during the project, did not participate in any activity due to different reasons, which is outlined in the results. As this is a new recruitment technique being tested, it cannot yet be categorised as a specific sampling method.

3.3.4.2 Methods B. Quantitative overview of patient contact A quantitative overview of the patients, who made contact during the recruitment phase running from 31 October 2019 to 31 January 2020, was made to present the results of the recruitment process. A quantitative overview was chosen to make the results easily readable and to quickly give a summary of the patient contact. The overview can be used to identify when in the process the patient made contact; where the patient noticed the project; where the patient registered to participate; the disease the patient is living with; the gender; and if the patient participated in a workshop inclusive interviews, or only an interview. This is useful to determine which channels the patients most frequently made contact through as well as if they noticed the project in one of the community pharmacies and registered there. After the overview, the reasons for not participating were presented to explain why some patients withdrew from the project after they signed up. All this knowledge will be used to explore and assess the used recruitment technique, hence, to answer research question one in this thesis.

C. Focus group interviews with patients before the workshop The interviews before the patient workshops with scientists at LEO Pharma were focus group interviews. One focus group interview was conducted two days prior to the workshop as it fitted better with the patients’ calendar, and the other focus group was conducted right before the workshop. 37 2) Design The interview guide was partly created by using the stakeholder expectation matrix (Figure 2) from the background section. The stakeholder expectation matrix outlines what the patients expect from themselves, and what the patients expect from the other stakeholders.(48) This was added to the interview guide to hear the patients’ thoughts and opinions about the partners of the project who are the stakeholders in this case. An example of a question is: “What do you think about the collaboration between the parties involved in this project?”. As also stated in the literature, transparency, and dialogue are important,(45,49,51) hence the patients were asked if they know what they were about to participate in. Furthermore, the motivation to participate in the project, their expectations for the workshop, and if they had any concerns regarding the project, were also added to the interview guide. An example of a question is: “Do you have any concerns about your participation in the project? If yes: Which ones? If no: Why are you not worried?”. After the first focus group interview, minor adjustments were made to the interview guide to make some questions more comprehensive (Appendix 10).

3) Interview Two focus group interviews were held, as the number of workshops was two. In the first focus group, two patients participated, and in the second focus group, three patients were scheduled to participate. However, one interviewee was late and joined in the middle of the focus group interview but was mainly listening and only added a few comments. So, it was decided to take all questions in the individual interview afterwards. The flow was natural in both focus group interviews, especially considering it was video calls, as it was difficult to see who was about to speak. The patients also supported each other well and even added in questions to each other which contributed positively to the flow.

5) Analysing During the analysis of the interviews, learnings from the background section were kept in mind to understand the results better. One learning kept in mind was the patients’ expectations from the stakeholder expectations matrix (Figure 2) to understand the participating patients’ expectations and opinions about the partners of the projects who are the stakeholders in this case. It could be that patients expect the industry and researchers to involve patients in all stages of drug development.(48) Another learning that was searched for was the involvement of a patient organisation,(37,51) and if that meant anything to the patients. Finally, it was kept in mind what

38 the patients felt about the transparency and dialogue as that is also reported in the literature to be important.(45,49,51)

D. Individual semi-structured interview with patients after the workshop The interviews with the patients after the patient workshops with scientists at LEO Pharma were individual interviews to cover more personal questions about, their disease history, the recruiting process in this project, their experiences with the workshop, and their knowledge about drug development. Each interview took place approximately 1–2 weeks after the workshop with the scientists at LEO Pharma to suit the patients’ availability best.

2) Design To create the interview guide, learnings from the background section were used. A good practice is to listen to the patients actively when engaging them in medicines R&D,(45,46) which the patients were asked specifically about. Further, a learning from the background section is that it is a major barrier that patient engagement is the industry’s attempt to “courting” or “coercing” patients.(47) Hence, the patients were asked, what they thought about their involvement, if their expectations were met, and whether they trust that their information will be kept safe. An example of a question is: “Did you feel that the scientists listened to you and understood your messages? Why/why not?”. The patients’ knowledge about their disease as well as medicines R&D, in general, was investigated as it was learned from the literature that patient education can be important to give the patients a better understanding of the process.(49) Furthermore, to give a better understanding of the patients’ motivation, and the need for patient engagement, the patients were asked about their disease, how it affects their everyday life, and the treatment they use. At last, the recruitment process was asked about, i.e. where they noticed the project, and if they needed to consider before registration, again to see what the possible barriers are for participation. An example of a question is: “When in the process did you decide that you would like to participate in the project?”. Minor adjustments were made to the interview guide after the first interview to make some questions more comprehensive (Appendix 11).

3) Interview In all interviews, the flow was quite good which might partially be because the interviewees had already spoken with the master thesis student several times; at the preparation calls, the focus group interview, and the workshop with the scientists at LEO Pharma. Some interviewees were more vocal than others, but all interviews went fluently, and the questions were well received. 39 5) Analysing While analysing the interviews, learnings from the background section were used. One learning, which was searched for, was whether the patients felt listened to by the scientists,(45,46) and if they felt that their information would be kept safe.(47) Further, it was searched for whether the patients felt they had the needed knowledge about their disease to participate in the workshop.(49)

E. Individual semi-structured interview with patients who did not participate in a workshop The master thesis student interviewed the patients who were unable to participate in a workshop with LEO Pharma’s scientists during the thesis period.

2) Design The same method as used in method D was used. The interview guide covered the same questions as in the methods C and D. The only difference was the exclusion of the questions related to their expectations to the workshop as well as the workshop evaluation questions as they did not participate in a workshop (Appendix 12).

3) Interview The flow in the interviews differed. One of the interviews had a very good flow where the interviewee was speaking freely and had very much to say about the topic. In that interview, the conversation had to be controlled to keep it on track. Conversely, in two of the interviews, the flow was less fluent as more elaborative questions had to be asked to start the conversation and reflections on the subject. However, the interviews went well, and all questions were answered, some more in-depth than others. Lastly, one of the interviews had a medium flow, and even though the interviewee was nervous, the person opened up and spoke freely.

5) Analysing The same learnings as for method D and E were used to analyse the interviews.

40 4. Results

4.1 Quantitative overview of patient contact In total 19 patients made contact to participate in the project where 17 out of the 19 patients were inside the scope of this project, meaning they were living with a rare disease with skin manifestations. Of the 17 patients inside the scope of the project, 15 patients contacted the master thesis student, and two contacted LEO Pharma directly (Table 2).

Table 2: Overview of patient contact in the project. The channel of registration describes contact with the student unless stated otherwise. Patient Date of Channel of Channel of Disease Gender Participated Participated no. contact noticing project registration in workshop in an inter- inclusive view only? interviews? 1 27-Oct-2019 Facebook Facebook Subacute Female No No private message Cutaneous Lupus Erythematosus 2 31-Oct-2019 Facebook Project website Lichen Sclerosus Female Yes No 3 31-Oct-2019 Facebook Project website, Scleroderma Male Yes No and afterwards Skovlunde Pharmacy 4 31-Oct-2019 Unknown Phone call Palmoplantar Female No No Pustulosis 5 01-Nov-2019 Unknown Project website Lichen Sclerosus Female No No 6 03-Nov-2019 Facebook Project website Morphea (Localised Female Yes No Scleroderma) 7 03-Nov-2019 Unknown Project website Lichen Planus and Female No No Lichen Sclerosus 8 05-Nov-2019 Through family Project website Erythromelalgia Female No Yes member 9 06-Nov-2019 Ballerup Ballerup Dermatomyositis Female No Yes Pharmacy Pharmacy 10 06-Nov-2019 Ballerup Ballerup Ehlers-Danlos Female No No Pharmacy Pharmacy Syndrome 11 08-Nov-2019 Press release E-mail to LEO Discoid Lupus Female No Yes Pharma Erythematosus 12 11-Nov-2019 Skovlunde Phone call Lichen Sclerosus Female No Yes Pharmacy 13 18-Nov-2019 Unknown Project website Ulerythema Female No No Ophryogenes 14 27-Nov-2019 Unknown Project website Lichen Sclerosus Female No No 15 04-Dec-2019 Google search Project website Lichen Planus Female Yes No 16 21-Jan-2020 Through family Phone call to Lichen Sclerosus Female No No member LEO Pharma 17 26-Jan-2020 Skovlunde E-mail Lichen Planus Male Yes No Pharmacy

41 4.1.1 The channels of noticing and registration for patients 80 hours of physical presence at the pharmacies, resulted in three patients being recruited directly by the master thesis student at the pharmacies. One patient noticed the project at the pharmacy and subsequently contacted the master thesis student via e-mail. One of the patients, who noticed the project on Facebook and registered through the contact formula on the website, ended up visiting the master thesis student at the pharmacy before agreeing to participate.

The rest of the patients in this project did not visit the pharmacies and were recruited in the following ways: Eight patients registered through the contact formula on the website, whereof two patients noticed the project on Facebook, one patient was made aware by a family member, one patient found the project through a Google search, and for the rest four patients the channel of noticing the project remains unknown, as they dropped out of the project before the information could be gathered. Additional noticing and registration channels were, that one patient registered to participate in a private Facebook message to the student and had noticed the project through Facebook. One patient registered by phone but dropped out of the project hereafter so the channel of noticing the project remains unknown. The two patients who registered to participate by contacting LEO Pharma directly, one by phone and the other by e-mail, noticed the project by seeing the press release about the project, and by hearing about the project from a family member, respectively.

The gender of the patients who made contact to participate was distributed as 88 % (15) females and 12 % (2) males. Within the first month, 82 % (14) of the patients reached out to participate, in the second month it was 6 % (1) of the patients, and in the third month, it was 12 % (2) of the patients.

4.1.2 Reasons for not participating The number of patients who participated in workshops at LEO Pharma during the master thesis period was five. Thus, 12 patients did not participate in a workshop at LEO Pharma during the master thesis period due to different reasons (Figure 6).

42 6

2 Number of patients

0 Difficulties with Medical Personal affairs Workshop is held Stopped virtual workshop conditions after thesis period responding to mails and calls

Figure 6: Reasons for not participating in the workshop applied to the number of patients.

However, four patients, who did not participate in a workshop, were interviewed by the master thesis student. Of the four patients, three patients were unable to participate in a workshop at LEO Pharma during the master thesis period and one patient had difficulties attending a virtual workshop. Nevertheless, one of the patients ended up participating in another activity at LEO Pharma.

4.2 Patient interviews In total nine patients participated in interviews related to the project by using methods C, D, and E as described in the methodology section. The time of each interview was as follows. The focus groups lasted 31 min, and 37 min, respectively. The individual interviews conducted after the workshops lasted 21 min, 24 min, 24 min, 30 min, and 36 min, respectively. The individual interviews with the patients who did not participate in a workshop lasted 24 min, 27 min, 28 min, and 56 min, respectively. An overview of the demographics related to the participating patients is provided (Table 3).

43 Table 3: Overview of the demographics related to the patients who participated in the project. Characteristics Demographics Characteristics Demographics Gender Female (n = 7) Member of a Member (n = 4) Male (n = 2) patient Not member (n = 5) organisation Age (years) 18 – 29 (n = 2) Diagnoses Scleroderma (n = 2) 30 – 39 (n = 0) Lichen Planus (n = 2) 40 – 49 (n = 2) Lichen Sclerosus (n = 2) 50 – 59 (n = 1) Discoid Lupus Erythematosus (n = 1) 60 – 69 (n = 2) Dermatomyositis (n = 1) 70 – 79 (n = 1) Erythromelalgia (n = 1) 80 – 89 (n = 1) Place of residence Ballerup (n = 2) Year of diagnosis Before 2000 (n = 2) (municipality) Egedal (n = 1) 2000 – 2004 (n = 1) Greve (n = 1) 2005 – 2009 (n = 2) Herlev (n = 2) 2010 – 2014 (n = 0) København (n = 1) 2015 – 2019 (n = 4) Odsherred (n = 1) Vordingborg (n = 1) Occupation Service industry (n = 2) IT industry (n = 1) Healthcare industry (n = 2) Financial industry (n = 1) Retiree (n = 2) Student (n = 1)

The key themes identified from the interviews and focus groups with the patients by using meaning condensation were the following; the patients’ lives with the disease and treatment used; reflections leading up to participation in the project; evaluation of participation in the project; and opinions about the partners and their cooperation in the project. An unpredicted theme that appeared from the analysis of the data was the patients’ trust in the public healthcare system.

4.2.1 The patients’ lives with the disease and treatment used Most of the patients are extremely affected in their everyday life because of the disease they are living with. This is both physical and psychological, and the disease is limiting their life. Several patients experience pain and itching daily, which was expressed as hard to live with.

“I do not know where to start. I cannot eat without having something to anesthetise my mouth with. I cannot brush my teeth without anesthetising my mouth. I cannot use the toilet without having anesthetised my intestines, i.e. the rectum. All the way through, I am affected by it.” (patient 15)

However, some of the patients are mainly affected cosmetically, and especially in the summer where more skin is exposed. This evokes feelings of shame and being at a display where other people stare at their skin.

44 All patients expressed a need for new medicines to treat their disease as the current treatments available are not good enough. The treatments available are also only symptomatic and must be used continuously or regularly to keep the symptoms under control. The most frequently used drug class is very potent topical corticosteroids, and the general experience is that the drug class controls the itching for a short period but does not really improve the rash. Also, the patients are afraid of the adverse effects that might appear, and therefore they do not like to use the treatment for a long period.

“I have used Dermovat […] and I am careful with that because when the skin is no longer young, it is also vulnerable to steroids that are potent.” (patient 11)

Other treatments used by the patients are local anaesthetics, biologicals, and different systemic treatments as immunosuppressants, proton pump inhibitors, and triptans. In common for all treatments, the patients experienced that only symptoms of the disease are treated and not sufficiently.

4.2.2 Reflections leading up to participation in the project The patients all expressed some reflections leading up to participation in the project, and the following subthemes were identified; motivation for participation in the project; considerations before participation in the project; expectations and concerns before participation in the workshop; and previous project participation and knowledge about medicines R&D.

4.2.2.1 Motivation for participation in the project In general, all patients felt that it was great, positive, and important to be given the possibility to participate in new medicines development. Several patients mentioned that medicines simply cannot be developed without the involvement of patients. One of the patients emphasised that it is a necessity to involve the patient from the very beginning of the development. Some patients found it motivational that a large pharmaceutical company was behind the project as they have the resources to set things in motion and start searching for a treatment. To see that specific research on the disease was finally conducted, was also a motivation factor for some patients. Also, it was a motivating fact that one could support research if possible and make oneself available. To one patient it was motivating to be a part of the front-line research due to curiosity and to be first-mover. It was widely mentioned that the possibility of helping others and contributing to making life easier for themselves and everyone else living with their disease was a great motivation factor.

45 Furthermore, some patients expressed that neither a cure nor a sufficient treatment is available yet, hence they would like to have new and better treatments developed due to the invalidating nature of the diseases. To one of the patients, the possibility of helping just one single person with their disease would mean a lot. To another patient, it gave a feeling that you have tried to help with the discovery or development of a treatment. To yet another, the feeling of doing something to lighten the burden of the disease was motivational, as the patient has been seeking possible alternatives for a long time.

“If you can help the next ones, who have to go through the same process that you have been through, with something, then it will also be worth it.” (patient 6)

The possibility of generating more knowledge about the disease was motivating to most patients, especially when the diseases are rare, and the knowledge is limited. To one patient, the motivation came from the possibility to gain knowledge about why no treatment had been developed yet, and why nothing can be done. To another patient, the public needed to gain more knowledge about the disease, as with other more common diseases where the public knows plenty and the disease is acknowledged. The patient hoped that participating in this project could contribute to that or at least set things in motion.

“There are some diseases which are very widespread in society and people actually have an understanding of and acknowledge it. Such as it is a critical illness, which this is not, so you cannot receive any insurance funds, any money for it when you have it” (patient 3)

Not only enlightening other people about the disease but also learning something oneself when listening to the stories from the other workshop participants was a motivation for a patient. Another motivation for some patients was the fact that the master thesis student was a part of the project as they appreciate that younger people are interested in their disease. In this way, new forces and strengths are applied to the research. One patient found it motivating to meet the student at the pharmacy to see an actual person behind the project, and to address possible questions there, as it was easily accessible.

4.2.2.2 Considerations before participation in the project Overall, the patients could agree that they did not have any or only very limited considerations before concluding that they wanted to participate in the project. Everyone was prepared to do whatever it took if it could contribute to the development of new medicines for their disease. Only one patient did not want to participate in the workshop as it was converted to a virtual

46 meeting due to the COVID-19 pandemic, and the person had no knowledge of using virtual tools.

“So, this is actually where I stand, I cannot figure it [virtual meetings] out. So, that is why I said no to join.” (patient 12)

Almost all patients decided immediately after discovering the project, that they would like to participate. One patient thought about it for a day because the patient is very shy, and also find it hard to talk openly about the disease as it is a personal matter but ended up coming to the conclusion to participate. Another patient came by the pharmacy to hear more about the project before registering but had actually already decided to participate before the visit. Yet another patient thought about it for some days just because it was a possibility but had actually already decided to register right after the project was discovered.

In general, the patients expressed a hard time imagining that anyone would deselect the project based on their experiences of the process. The only reflection, from some of the patients, was if anyone would find the participation intimidating or transgressive since some people find their disease tabooed and do not want to be recognised as a person with that disease.

“[…] you have read and spoken everywhere around, and it is so much your story, so I think it would be a shame if someone said no; let us put it that way.” (patient 15)

Some patients considered how much time would be spent by participating and if they were willing to prioritise the project in a busy everyday life. It was also considered by two patients if the travel time to and from LEO Pharma’s headquarters in Ballerup would take up too much time. However, the workshops were converted into virtual meetings, thus it was not a problem.

4.2.2.3 Expectations and concerns before participation in the workshop Overall, the patients expected to see some different specialists who would listen and analyse what was being said, and it was expected that these specialists would have the development of medicines as their area of expertise. Some of the patients also expected that the decision-makers would participate in the workshop to decide if they wanted to proceed with research in the disease. These patients also expected that LEO Pharma would decide to initiate research within their disease and start working on treatments. Further, they expected the scientists to be respon- sive, committed, and eager to start researching.

47 Another expectation was that the project would be further developed and not just be forgotten after the completion of the workshop. The patients also expected that they would receive feedback and status on the project after a while. Additionally, one patient expected that the participation of the municipality and the master thesis student in the project would provide certainty to the fact that the project would develop further.

“When Ballerup Municipality is participating and you [the master thesis student] also have a large share in this, in relation to what you have to deliver in your assignment, then it becomes such that it does not just level out at some point” (patient 3)

Some of the patients expected that they were to talk about their disease journey, and the related thoughts and feelings. One of the patients expected to learn more about the disease and why no treatment has been developed yet.

Overall, the patients did not have any major concerns about their participation in the project. However, there were some things they had given some thoughts before participating. One of the things was the change in format from a physical meeting to an online meeting, and how that would turn out. Also, some concerns about whether you would be heard in the project when living outside of Ballerup Municipality, but that concern was dispelled. One of the patients was concerned if their participation would lead to actual research in the disease, and also if LEO Pharma would take the patients and the master thesis student seriously in this project.

“Then, I was also worried whether it could be an approach or not to get these things brought up to a higher level in order to start some research” (patient 2)

4.2.2.4 Previous project participation and knowledge about medicines R&D Three patients had participated in a similar project before this, whereof two patients participated in a LEO Pharma project. The rest have not participated in a similar project, as they have not noticed a project concerning their disease. Regarding the patients’ knowledge about medicines R&D, some of them responded at first that they had no or very limited knowledge about it, but it turned out that they did know something.

“I do not know that much. I know it has to go through some phases. It must first be tested on some animals, and then it must be tested on a smaller group of people, then a slightly larger group of people and then a slightly larger group of people before it can be accepted to the general population.” (patient 8)

48 The patients knew most about clinical trials including that it consists of some phases and that tests are conducted on animals at first and later on humans. Furthermore, the patients knew that a lot of regulations exist and that it takes a long time to develop medicines. The patients’ primary sources for the knowledge were; working in the healthcare industry; relatives who participated in clinical trials; from a patient organisation; studied a post-secondary education within science; and self-study by searching PubMed.

4.2.3 Evaluation of participation in the project Regarding the evaluation of the patients’ participation in the project, they expressed various opinions which have been grouped into the following subthemes; feelings and reflections emerging during and after the workshop; the impression of the scientists who participated in the workshop; the perception of the next steps after the workshop; and suggestions for improvement of the project.

4.2.3.1 Feelings and reflections emerging during and after the workshop All the patients described the workshop the way it was conducted, and they have all experienced what exactly happened. The patients were asked to rank the project considering if it lived up to their expectations on a scale from 1–5 where 5 is the top ranking. Three patients ranked it 5 out of 5 as it lived fully up to the expectations. One patient ranked it 4 out of 5 as there is not yet closure regarding if research about the disease will be initiated. The last patient ranked the workshop alone 5 out of 5 as it lived fully up to the expectations, but the project as a whole was ranked 4 out of 5 as it lacked clarity about who the partners in the project were, especially the municipality.

The patients disagree on whether their level of knowledge about their disease was sufficient to participate in the workshop. Some patients believed they had a lot of knowledge as they lived with the disease for a long time and have done a great amount of research themselves to find out how to live with the disease. The rest of the patients thought they had a more limited amount of knowledge as one had a hard time finding relevant literature on the topic, and another was diagnosed with the disease as a child, hence the parents had the most knowledge about the disease journey.

“It was a little challenging in the sense that I got it at such a young age, so it was a little difficult to gather enough information for the whole emotional journey in it too […].” (patient 6)

49 To some of the patients who had a milder degree of the given disease, it was challenging to tell their story when the other stories were more severe. It evoked a feeling of being overlooked as the scientists had more questions to the patients with a more severe degree of the disease. One patient was also very affected by the other patients’ stories and how much the disease affects their lives with enormous consequences. One patient suggested to only include people with the same disease and severity degree in a workshop, for everyone to be at the same level. Some patients felt they gained more from the project personally than they expected and being part of the project started reflections at home. One of the patients helped out a friend who was recently diagnosed with a rare disease by passing on knowledge and experiences from participating in medicines development.

Several patients found it important to have a voice and be involved in medicines development, and they are willing to tell their story if it can help others and make a difference. One of the patients felt that it was a reward in itself to contribute because it would be positive if the person’s small amount of knowledge has contributed to the R&D of new medicines. Furthermore, the patient contributed to the project together with other patients with the same disease which the patient believed to have enlightened the topic from different angles. Even if no research is initiated after the workshop, the patient thought it was worth having contributed to a master thesis project.

“I am very positive in that sense if just my little knowledge and my small contribution can help push for research and development of some medicine or a treatment.” (patient 17)

To some of the patients, the feeling of being vulnerable when telling about their disease emerged as it is hard to articulate what has been hard. Moreover, the participation of a representative from a patient organisation in one of the workshops started some reflections about having them present. The patients felt that one’s story could be supported and substantiated with facts provided by the patient organisation which made a difference to the whole presentation of the disease.

The workshop being converted from a physical meeting into a virtual meeting, led to reflections upon that topic. The patients disagreed whether this was a good way of conducting a workshop. Around half of the patients thought it was easier with a virtual meeting due to the lesser amount of time spent on transportation, but also because the meeting was run more efficiently. At a physical meeting, more chitchat might occur, and too many open-ended questions. The other half of the patients thought that it would give a stronger impression if the meeting was physical as 50 you appear slightly amputated in the virtual format. However, all patients agreed that the workshop went well as a virtual meeting.

4.2.3.2 The impression of the scientists who participated in the workshop The patients’ overall impression of the scientists was good as they felt the scientists listened to them. Most patients pointed out that the questions, the scientists asked them, were highly relevant, and it reflected that the scientists had listened to them. The questions further reflected that the scientists were well prepared and that they were empathic and understanding.

“What also touched me a lot was the interest of these scientists, meaning I experienced it as a real interest that they would really like to go in and try to find out what is triggering these things in the disease so they can also do something about it.” (patient 2)

A patient emphasised that the scientists were very talented and professional and that they were empathetic without being pathetic, which was great. The patient also pointed out that the scientists were extremely welcoming which another patient agreed in and pointed out that it could certainly be felt even though the meeting was virtual. One of the patients pointed out that it is interesting that several nationalities participated in the workshop as it gives the impression that more people are interested in this, and not just local scientists. Another patient assumed that the scientists, during the workshop, realised how needed finding a treatment is.

The patients disagreed whether the scientists were a bit overwhelmed and frightened by the patients’ stories, as one of the patients felt they were, and another felt they were not. One of the patients was not satisfied with the fact that some scientists or employees were only listening to the meeting and not participating actively with any questions or comments. The patient wished that everyone in the meeting could have given just a small comment or response as they might have had something interesting to say.

4.2.3.3 The perception of the next steps after the workshop The patients could all agree that the scientists will look at the data and have post meetings about the workshops. One patient mentioned that the scientists would pinpoint what makes sense to continue working with from a commercial and business perspective. This patient also pointed out that the interesting part is that the scientists have a dialogue after the workshop because the potential development of new medicines will take a long time. The patient felt that if the scien-

51 tists do not have a dialogue afterwards, it would not have a strong appearance as initially expressed.

“What is interesting is that they actually have a dialogue after our workshop, so it was not just a workshop and then they go their separate ways. […] Thus, I do not think it gives the weight that you have aimed for in the whole process.” (patient 3)

Another patient highlighted that the scientists will discuss what can be done to develop a treatment and that they will reach out to the patients again, once they have decided if they select and work with the disease. The patient hoped that the scientists could be granted funding to research the disease. It was also mentioned, by another patient, that the scientists will have to decide which disease should be researched further in. Yet another patient believed that it will take a long time before any decisions are made and before the patients will receive feedback.

All patients had full trust and do no doubt that their information will be kept safe and treated properly after the workshop as a large pharmaceutical company is behind the project, and the patients are sure they will comply with rules and regulations. Furthermore, all the patients are ready to participate in medicines R&D again without a doubt if an opportunity arises. One of the reasons described is that this project went well and if their knowledge can be used, the person will participate again. Two patients also felt that if their knowledge can help others and make a difference, it would be great.

4.2.3.4 Suggestions for improvement of the project One suggestion for improving the project was that one could imagine that somebody wants more privacy in the pharmacy so maybe the patients could be given the possibility to ask supplemen- tary questions and hear more in-depth about the project in the back office of the pharmacy. Another suggestion was to only focus on one disease throughout the project, as the patient believed it would motivate more patients to participate if they knew their help would benefit others with the exact same disease. Further suggested by two different patients was to receive more information on who the partners behind the project are, for instance, receive a graphical overview of the organisation. Also, one of the patients wanted more in-depth information about the project in general and suggested ‘need to know’ and ‘nice to know’ information for the patients to decide how much information to seek.

52 “’Need to know’ is the absolute minimum, and ‘nice to know’ is for those who want to get more granular details, and then the others who only require ‘need to know’, they can skip it. So, it would be my advice to do something like this in different areas, ‘nice to know’ and ‘need to know’.” (patient 11)

Another suggestion was to have the health authorities involved in the project instead of a municipality as they might have the power to influence more. Further suggested was to meet the other patients several times before the actual workshop to get familiarised and know where you can support each other during the workshop. Last, it was suggested to have a clear timeframe set from the beginning of the project to know when you will be involved.

4.2.4 Opinions about the partners and their cooperation in the project Not one patient could mention all the partners in the project, and the majority thought it was only LEO Pharma doing the project in collaboration with the master thesis student. One patient did, however, mention all partners except FAIM. Then another patient could mention LEO Pharma and Ballerup Municipality, and another could mention LEO Pharma and the pharmacies. One patient thought it was only LEO Pharma and did not know it was a master thesis project. All patients could agree that even if they have not seen how the collaboration of the partners worked, they do think it is a good idea to collaborate in this way, as the project must be viewed from different angles. One patient assumed that all partners will benefit from the research.

In general, the patients could agree that it is very good that LEO Pharma involves patients in medicines R&D, and several patients described it as being the only way to develop medicines. Two patients mentioned that it is great to see a large pharmaceutical company involved in the project as they have the resources to actually do something. One patient believed it is great for the patients to be involved but also for the company to target their research, so they do not develop something terrible, according to the patient.

“It is good both for patients in general and then I also think it is good for the companies that they hear something from the users.” (patient 9)

Overall, the patients agreed that it is a very good idea to use the pharmacies to recruit for projects like this. Some of the patients mentioned that they do not know where else to notice it, and they probably would not have noticed this project anywhere else. The pharmacy was labelled as the ideal place to advertise for this type of project since almost all people with a diagnosis must visit the pharmacy to collect their medications. One patient spoke to the pharmacy staff at one of the

53 pharmacies and felt their elaboration on the project was decisive in reaching out to the master thesis student. The patient only asked the pharmacy staff about the project as the patient could relate to message in the advertisement.

“I would just say that they were quite informative at the pharmacy when I asked about it. So, it was their gain that I went on with it.” (patient 17)

In general, the patients did not notice that the municipality participated in the project as they thought it was not clearly stated. Some patients were wondering why they were interested in the project and started speculating if there was a financial gain in it for them or the other parties. However, the patients had no problem with their participation and thought it was good to view the project from different angles. For about half of the patients, it mattered that a patient organisation participated in the project whereas it did not matter to the other half. It mattered to some because it provides safety to know that someone knows the patients’ rights and can assist where needed. Overall the patients found it very good that the master thesis student was a part of the project, also to know that the knowledge generated from the project would be reported in a thesis. Several patients mentioned that it was great to have young, fresh eyes look at the research to give new angles and push for new research paths. One of the patients found it very motivating that a young individual was coordinating this project and wanted to support the various diseases. Another patient mentioned that it is great to contribute to education, similar to visiting the doctor where a student learns from the consultations with the doctor.

4.2.5 The patients’ trust in the public healthcare system Unpredictably, it turned out that nearly all patients touched upon their trust in the public healthcare system during the interviews. The time to be diagnosed was long for almost all patients as the doctors did not know the diagnosis. This resulted in many tests and being moved around the healthcare system for some of the patients. When the diagnosis was finally given, frustrations about being left alone were expressed. Several of the patients received a paper with the name of their newly diagnosed disease, and then a prescription for some medication. After- wards, they were left alone and had to do their own research on their disease.

“[…] at the dermatologist, I was seeing, I got his business card, and then he wrote by hand what the disease was called, and then he gave a prescription for Dermovat. That was it.” (patient 17)

54 One of the patients experienced that the doctor did not even bother to inform about another diagnosis, the patient had, so the patient found out randomly. Being left alone with a new diagnosis caused mistrust in the system, leading to one of the patients learning never to expect anything from the healthcare system. Also, another patient, who even works within the healthcare system, no longer works in the public sector but has moved to the private sector to provide better care. Another patient pointed out that worst case the patients will do their own medical evaluation by looking at social media or the internet. Several patients expressed that there must exist another way to help newly diagnosed patients, hence they suggested that the doctors could inform more about patient organisations for the specific disease. Some of the patients have never heard about the existence a patient organisation for their disease, and they have been through a lot in the healthcare system. The patients emphasise that if the doctors inform about patient organisations, the patients will have someone to talk to and receive help from, besides from the healthcare system.

4.3 Partner interviews The interviews with the project’s partners were conducted using method A from the methodology section. The project’s partners are Ballerup Pharmacy, Skovlunde Pharmacy, FAIM, Ballerup Municipality, and LEO Pharma. One representative from each partner was interviewed. The interviews lasted 27 min, 29 min, 32 min, 33 min, and 53 min, respectively. The key themes identified from the interviews with the partners by using meaning condensation were; the development of the project and the motivation behind; project evaluation; and the future of the project. Two unpredicted themes were identified, cross-sectoral collaboration; and the partners’ perception of patients.

4.3.1 The development of the project and the motivation behind The partners expressed their views, opinions, and thoughts about the development of the project and the motivation behind the project. The motivation to participate in the project differed from each partner. As the municipality initiated the project, their motivation was to support the new collaboration and ensure a satisfactory completion for the companies to collaborate again in the future. Furthermore, they also wanted to support their citizens. Both pharmacies agreed that they were motivated to show that community pharmacies have a greater role in society than what is currently perceived.

55 “[…] so, I think the pharmacy has a much bigger role in society, can have it in the future, and the only way to show it is to actually do it. It is to go through some projects, or something else, and thus also show the competence present here.”

Also, one of the pharmacies was motivated to help the master thesis student whereas the other pharmacy wanted to give a voice to the patients who normally do not have one. The patient organisation was motivated to help the patients and their potential members as they represent a part of those who were interviewed and engaged in the project. They also wanted to draw attention to the topic as many rare disease diagnoses cannot be treated currently. The pharmaceutical company was motivated by several factors, one of the most important being; removing the barriers between patients and scientists, and this is one way for the patients to join medicines R&D. The company expressed a wish to target the research by engaging the patients to develop medicines needed to solve the patients’ unmet medical needs.

“There are, of course, many reasons why we want to get involved. But first and foremost, it is about breaking down this barrier that exists between people who live with the disease and those who develop the medicines for them.”

The partners could overall agree that the development throughout the project has been good and enthusiastic with great interest from all. Everyone was very positive and also surprised that more people than expected were recruited through the project, as the majority of the partners were slightly sceptical and unsatisfied that the patient target group was quite narrow. However, they were all ready to give it a try and thought it went really well.

“We tried to outline what the probability of success was. It was not particularly large from the start. There was uncertainty about the recruitment at least, but my approach has always been that projects you participate in, they show a result no matter what. If the result then shows that it cannot be done, then that is the result.”

The partners’ roles and responsibilities in the developmental work of the project are reported in the background and methodology sections of this thesis. However, it must be outlined that it dif- fers how the partners describe their effort in developmental work. One partner perceived their effort as modest, whereas another partner described it as the project was prioritised to the extent possible. The rest of the partners described their effort as quite extensive as they have posted several resources in the project.

56 4.3.2 Project evaluation The partners have expressed their view when evaluating the project, which can be divided into the following subthemes, lessons learned during the project; practical execution of the project without a student; pharmacy recruitment site and related partner roles; the project’s contribution to growth in the company and society; and cross-sectoral collaboration.

4.3.2.1 Lessons learned during the project In general, the partners pointed out this new way of recruiting patients to medicines R&D as good, also the fact that it was possible to recruit the patients in scope. Some of the partners emphasised that the collaboration between the partners was very good, and an important result to show that they can actually collaborate.

“I think it has been great that we at least had some businesses working together, and you [the master thesis student] have written a thesis, so in that way, academia has joined the project, and I think it has been very good.”

One of the partners thought that the number of parties involved in the project, and the student as a project coordinator, made everyone want the project to succeed, which was good. Some of the partners also pointed out that it is good that the small voices are heard in a project like this, and that the patients acquire a feeling that something is progressing, and they can participate in a project. One of the partners thought that even if the patients were not in the target group, they reflected upon what disease they have, and if they might have undiagnosed diseases. Further- more, the partner can use the experiences the student obtains at a later stage.

The majority of partners expressed the narrow target group as less good as the project would have had larger potential if the target group had been broader. One partner expressed that if more common diseases were selected, the interest would probably be higher. A larger target group was also the most mentioned suggestion for improvements. Also, one of the partners mentioned that the pharmacies are placed close to each other, and it might have been better if they were more spread out. According to the same partner, the patients, roughly speaking, have not been recruited through the pharmacies but rather via online campaigns. Therefore, this partner suggested expanding the project by contacting hospitals and other pharmacies and ask them to have printed material available. Another thing, mentioned as less good by some of the partners, is the marketing campaign as it could have been more extensive, and the student was left a bit alone with that part. It was suggested that the communication part becomes clearer in terms of a specific system

57 and communication channels to make the project ownership more obvious. One partner also mentioned that the establishment of a solid project organisation did not quite succeed, thus it was suggested to create a more solid project organisation, which is clearly defined, and thereby the project can enter any municipality in Denmark.

4.3.2.2 Practical execution of the project without a student The partners all agreed that it is valuable to have a representative, in this case, the master thesis student, present at the pharmacies for the customers or patients to see an actual person in relation to the project. However, the majority felt that the student has spent too many hours at the pharmacies where nothing happened, and the pharmacy staff has not been an active part of the recruitment. Therefore, the partners believed that the role of a representative at the pharmacy should be changed. With the right training and a broader patient target group, all partners agreed that the pharmacies can take the student’s role of being a representative at the pharmacy, as well as take a larger part in the recruitment, as it would be easier for the staff to spot the patients.

“If it is a broad target group where customers come in every day, and they do, if it is, let us just say rosacea. Then you could train the staff to say, you know what, when you have a rosacea customer, then you have to hand out this questionnaire, or this email address, or what you think is needed.”

One partner also suggested that the recruitment of patients and data collection for a master thesis could be carried out by a pharmacy student during the sixth months of pharmacy internship which is mandatory in the master’s programme in pharmacy at the University of Copenhagen. All partners could agree that the project needs to have a coordinating party between the other partners as the master thesis student has acted as in the project. Thus, all partners agreed that it was good that the student joined the project because it was an advantage in the development and execution of the project. According to one partner, a coordinating party should be impartial from other stakeholders as otherwise, it can create doubt on who is represented and what the project comprises. One partner suggested receiving funding to create an actual coordinating organisation to run the coordination between the partners and ensuring the communication is completed.

4.3.2.3 Pharmacy recruitment site and related partner roles The partners could overall agree that using community pharmacies as the place of recruitment of patients to participate in medicines R&D was a very good idea and an ideal place to do so. It is suitable as many people visit the pharmacies because they need their medication, and it is also public and visible. To one partner it would be a pity not to do anything when the possibility is there. To another partner, the pharmacies are reasonably independent which makes them a good 58 meeting point. Another partner pointed out that it must be voluntary for the patients to enrol as in this project. This partner also believed that it will be easier for the patients to decide whether to participate, as they are in another stage of their disease or life when visiting the pharmacy compared to being recruited through the hospital system.

“Many times, they are recruited through the hospital system, but when you are out of it and living with the disease you have, you pick up your medicine at the pharmacy. Then you might be at another stage in your life, and therefore it is good to be able to recruit from there. So, thumbs up to that.”

Some partners also emphasised that the pharmacies are a good place of recruitment as the pharmacy staff has good skills in communicating with patients and want to help the customers, not just sell products. Some partners pointed out that pharmacies are not the only place of recruitment as patients have been recruited through patient organisations for a longer period, so the corporation between pharmacies and patient organisations is important.

Both pharmacies felt that they only contributed to the recruitment of the patients on a smaller scale, besides having the printed material available and the student present at their pharmacies. The patient organisation felt they were a crucial part of recruiting patients as they pushed the project out through all their own channels. The municipality thought they were a part of the recruitment by creating attention through their social media channels and drafting an article to the local newspaper, Ballerup Bladet. The pharmaceutical company felt they were a part of the recruitment by being a consultative party by talking the process through with the other partners and setting the inclusion criteria for the target group.

4.3.2.4 The project’s contribution to growth in the company and society To the majority of partners, the project could create some kind of growth within their company, except for one partner. Common for all the partners, the interest from customers or patients will potentially create growth, as attention will be drawn to their company or organisation. People will start seeing the company or organisation from another perspective, which could lead to them becoming customers or members. All partners could agree that the project can contribute to growth in the society as it helps patients or citizens in the long run such as having a product developed that will ease the patients’ everyday life. Some of the partners pointed out that well- treated patients can function more or less as healthy individuals and contribute to society in the same way.

59 “I believe for the society, it is important that everyone is well treated, everyone who can function as healthy people, they can contribute to society. So, in that way, there is also an economic growth or gain for society.”

One partner stated that the project might help break down the stigma related to being involved in medicines R&D and show that patients may have an influence that will contribute to society. Another partner expressed that the collaboration between different parties can contribute to less information being lost during transfer from one sector to another.

4.3.2.5 Cross-sectoral collaboration An unpredicted theme, which emerged during the interviews, was the cross-sectoral collaboration as several partners brought up the subject. One partner expressed that the more collaborating partners addressing the same issues, the better, and they would like to be the partner who brings people together. Another partner expressed that they generally give cross-sectoral collaborations a lot of thoughts and why it sometimes can go wrong. This partner also liked the wide band of partners in this project and considered if the municipality and the pharmacy may work together with the home care service in a project in the future. Yet another partner reflected upon that this project gave the possibility to connect companies as a cross-sectoral collaboration in Ballerup: the municipality as the unifying party, the pharmacy as the place where the people are found, the patient organisation who captures many of the patients through their communication, and a receiving party in the form of a pharmaceutical company. According to this partner, the fact that it is a cross-sectoral collaboration gives more security to the participating patients, and it shows that the project adds value to other parties than just the pharmaceutical company.

4.3.3 Future of the project The future of the project was also a theme expressed by the partners whereof the following two subthemes emerged, the future role of each company and the leading party; and expansion of the project to other disease and geographical areas.

4.3.3.1 The future role of each company and the leading party The partners have reflected upon their future role in this type of project where patients are invited to be part of medicines R&D. One of the pharmacies suggested that they could take a larger part in the recruitment of patients. The other pharmacy suggested that they could help by collecting information about the customer’s experiences with a specific medicine and report back to the company behind. The patient organisation expressed that their role is to act as an assessor

60 to the patients and can handle the contact between the patient and the pharmaceutical company. Furthermore, the patient organisation knows the patients and the process of recruiting patients to be engaged in medicines R&D.

“It is not that the industry is not on their side, but misunderstandings can arise quickly if you do not speak the same language.”

The municipality believed their role could be to bring the parties together to recruit patients to be engaged in medicines R&D. The pharmaceutical company thought their role is to make the engagement possible for the patients and be transparent about it. Besides the value to the patient, it creates a value to the company that they can develop better medicines and quicker without too many dead ends. In general, all partners could agree that they can imagine that their company would invest time in a project like this in the future, but they are more uncertain regarding financial investments. Some partners imagined that they could invest financially in the project depending on the goals of the project, and maybe if a societal health issue arises, a project like this could remedy the problem. One partner expressed that they could not imagine investing financially in a narrow project like this.

As to whom the partners thought should be leading a collaboration like this in the future, their opinions differ. However, all partners agreed that the responsibility to engage patients in medicines R&D lies on several parties. Some of the partners believed the pharmaceutical company should be the lead as they have the main interest in the engagement of patients, but they should be assisted by the other parties. Some of the partners thought it should be an independent, impartial party that coordinates and leads the project, which according to one partner could be the patient organisation who is commercially more independent. However, they can be accused of honouring their own cause if some diseases within the target group fall outside of their own expertise. One partner believed the pharmacies could be the lead, maybe in collaboration with other health stakeholders in the municipality. Finally, some partners suggested that the public system and the government could be co-responsible for ensuring patient engagement.

4.3.3.2 Expansion of the project to other disease and geographical areas All partners agreed that the project has the potential to expand to other disease areas as well as geographical areas. One partner thought it could become too broad and hard to control so it would require a lot of information to the parties, perhaps in the form of an informational video. Another partner outlined that it is important to secure the same seriousness in a new project like

61 this one, should it be expanded. Yet another partner believed that it is important to find parties who have an interest in development, as this project was run on a pioneering spirit and the next parties should be the ones to make an already tested project work.

“Their [other municipalities’] expectation may be different because it was kind of a pioneering spirit this [project] was running on. Next is that someone has tried it, now you have to make it work and it gives a different approach to it.”

Last, a partner considered the project to be a plug-and-play project where disease areas and companies can be replaced, and it will be functional, as it is a simple project with a large amount of value. All the partners could agree that a master framework would make it easier to do a systematic engagement of patients in medicines R&D in the future, as it would create a good overview. Hence, some partners mentioned that all involved parties would know which steps to follow and what to expect. As to whom should be responsible for developing a master framework, the partners disagreed. The suggested parties to be co-responsible were patient organisations, the pharmaceutical industry, academia, municipalities, and health authorities.

4.3.4 The partners’ perception of patients Another unexpected theme that emerged during the interviews was that several partners expressed care for the patients, as they pointed out that early patient engagement is an important part and the right way of developing medicines. One partner highlighted that it can take a long time for the patients to be diagnosed, and often the patients receive a temporary treatment from the doctor to calm the situation until the next time it happens. Some of the partners pointed out that what the scientists or doctors see is not necessarily the same as what the patients see, and what would work in their everyday life.

“This way of understanding how things are connected, that it is not only on the chemical or biological level that things work well, but they must also work in reality. It must work for the common person. That is what matters.”

One partner outlined that the patients should find the product great, otherwise, they will use the competitor’s product. Another partner pointed out that it is hard for the pharmaceutical industry to develop medicines without knowing who the consumers are. Yet another partner pointed out that it might create hope for the patients to participate in a project like this, and if the patients want treatment and create a better future, this is the way to be involved.

62 5. Discussion

5.1 Results discussion The most important results of the study are that patients could be and accepted to be recruited to medicines R&D through the newly developed recruitment technique where community pharmacies are used as a recruitment site. Community pharmacies are considered a good place of recruitment by both patients and the project’s partners, and the patients only had few considerations before deciding to participate. Furthermore, it is important to note that the patients thought the workshop lived either fully or almost fully up to their expectations, and only minimal challenges emerged such as feeling slightly overlooked in the workshop with a milder disease degree compared to a severe degree. However, all patients experienced the scientists as listening and understanding. The patients were unable to name all the project’s partners, but they emphasised the collaboration as a good idea. Furthermore, overall the patients believed that the involvement of each partner was great. All the project’s partners were positive and surprised how well the project went. Another important result is that the partners perceived the collaboration as cross- sectoral, and believed the project needs to have a coordinating party to run optimally. The partners believed that the project has a future and is applicable to other diseases and geographical areas.

5.1.1 Which recruitment technique can be constructed to recruit the patients in community pharmacies? During the three-month recruitment phase, 17 patients inside the scope of the project made contact to participate where 24 % of the patients noticed the project at one of the pharmacies. That could be considered a low number of patients but considering the target group was people with rare diseases with skin manifestations, and the physical recruitment sites were only two pharmacies, it could also be considered a quite high number. The recruitment of patients to medicines R&D through this specific setting and partnership has presumably not been seen previously. This could be because the patient-centric approach is an exceedingly evolving area from a more product-centric approach,(14) and hence, comparison to other studies is limited. However, in the literature, one recruitment site to engage patients in medicines R&D is for the pharmaceutical companies to have an existing relationship with health care providers or hospitals, but the recruitment places are only mentioned, not assessed.(37) A community pharmacy could be categorised as a health care provider, and an approach to use 63 them as a recruitment site in the future may build on an existing relationship with a pharmaceutical company. In this project, the pharmacies did not have the responsibility for the study, the master thesis student did. Thus, in case they may build an existing relationship with a pharmaceutical company in the future and thereby become study responsible, it should probably be reported to the Danish Medicines Agency as an affiliation to a company.(80) This should be done to support the pharmacies’ impartiality as the purpose is to secure the citizens’ trust in the healthcare system. The rules should support the fact that pharmacies are not affected by economic or other industrial interests when dispensing medicines and performing patient medication counselling.(80) Conclusively, this must be taken into account if using Danish pharmacies as a recruitment place in the future where the responsibility of the project is placed on the pharmacy, and not a student or coordinating party.

Within the first month of the three-month recruitment phase, 82 % of the patients reached out, which is a quite high percentage. This could be due to the online marketing campaign, especially by the patient organisation, as it could be argued that the patients were partly recruited through the organisation. The patient organisation pointed out they were a crucial part of recruiting which was also reflected in the number of patients who noticed the project online. In the literature, patient organisations are described as a place of recruitment, as they can recruit qualified individuals, however, the articles only mention this place, and do not assess it.(37,46,51) It is explained that individual patients and the industry cannot work alone, they both need patient organisations.(51) In this project, it did not matter to half of the patients that a patient organisation participated which may be because the patients themselves had sufficient knowledge about medicines R&D. Further stated in the literature, a suggested working practice is to establish long-term partnerships between patients, patient organisations, and industry to benefit all partners.(37) This is also the case with the two partners in this project as they have collaborated on other projects as well,(81) which may have contributed to the good execution of the project. Fur- thermore, some of the project’s partners pointed out that pharmacies and patient organisations could work together on the recruiting part. To achieve a good collaboration, some parameters should be considered; the patient organisation may not realise their value and importance to the industry; and it is important and essential with mutual respect, transparency, and commit- ment.(51) This may also be considered when the other parties are involved in the project. Hence, it is imaginable to also include the pharmacies and municipality in a long-term partnership with the industry and patient organisation.

64 5.1.2 What is the patients’ attitude towards engagement in medicines research and development through this specific setting? In general, the patients found the engagement in medicines R&D positive through this specific setting, and they found the workshop-scientists overall listening and understanding. In a study by Borup et al., 2016, the primary response, from a breakout session regarding the needed initiatives and prerequisites to involve patients in medicines R&D, was the following; it would require actively listening to the patient for the patients’ voice to be considered in the further R&D process.(45) Hence, this can be argued to be an accomplishment of this project. Furthermore, Borup et al. mention that the diversity of patients and their opinions, for example, the severity degree of the disease, should be kept in mind.(45) For the pharmaceutical company, it may be argued to be fulfilled as several severity degrees were included in the workshops. Opposite, the patients with a milder disease degree felt slightly overlooked during the workshop as fewer questions were addressed to them. To overcome this issue, the workshop might be divided into disease severity degrees for the pharmaceutical company to include all disease aspects in their research, and for every patient to feel appreciated. However, it could be a difficulty for the industry if they lack a sufficient number of participating patients. Last, it can be discussed if the parameters described by Borup et al. are well-tested as the response came from a breakout session (45) and thus do not test the impact of this. On the other hand, it indicates what is important to consider in patient engagement.

In this project, the patients expressed the benefits of the workshops as gaining knowledge about their disease and treatments option, as well as being very positive about the workshop. One patient even found the possibility to learn about one’s disease motivating. In a literature review by Vat et al., 2019, benefits, costs, and challenges have been identified for the patient partners in medicines R&D.(13) Some benefits mentioned are; learning about own condition and treatment options; and enjoyment and satisfaction,(13) which is a similarity to the results from this project. Some costs and challenges outlined by Vat et al. are the following; disappointment and frustration emerging from mismatched expectations; stress due to lack of knowledge and confidence; and investment of time and possibly own resources.(13) The patients, who participated in this thesis project, have not expressed disappointment or frustration emerging from mismatched expectations, as the patients expressed the project as living fully or almost fully up to their expectations. The patients might not have experienced this as they felt left alone with their disease by the healthcare system. Thus, it could be interpreted that they are positive that someone finally gives them some attention, almost regardless of the type of attention. Also, one patient 65 learned never to expect anything from the healthcare system which could be an expression of lower expectations. One could imagine this to be the case for other patients, even though they did not express it explicitly. As for stress due to lack of knowledge and confidence, the patients in the project did not express this, although some patients expressed a more limited knowledge level. However, they did not appear stressed about it. The patients might not be stressed about it as they are highly affected by their disease in their everyday life with no sufficient treatment, and therefore they are happy to contribute slightly or even just trying to contribute. Regarding the investment of time and possible own resources, some patients in this project did also consider this parameter as to how much time the project would take up, also regarding transportation time. On the contrary, some patients did not have these concerns, and it is therefore not necessarily a challenge. The difference in concerns about the investment of time could be due to the patient’s place of residency, or if they have plenty of other activities in their lives. Another reason for the difference could be the disease severity degree; how much a new treatment is needed compared to the time spend on a project. It can be hard to determine certain factors as all individuals are different, thus further research on the topic would be recommended to give a broader perspective.

5.1.3 What do the recruited patients think about the project’s partners’ involvement in this project? In general, the patients were unable to mention all the project’s partners, thus it can be argued that a need for a clearer project organisation exists. It was also suggested by a patient to provide more clarity by presenting an organisational chart for future projects. In the literature, the importance of clarified and defined stakeholder expectations and roles is emphasised, as well as the importance of transparency.(13,46,48-50) In this project, it can be argued that the stakeholder roles were not clearly stated to increase transparency of the project organisation, as the patients expressed that they were unaware of which partners were involved in the project. As a conse- quence, one of the patients rated, how much the project lived up to the expectations, lower. This is also described in the literature as a challenge, that patients get confused due to a lack of clarity about roles and procedures.(13) Overall, the patients found all the partners’ participation good. As can be seen in the stakeholder expectation matrix (Figure 2), developed by Boudes et al., 2018, the patients expected from the industry to involve patients in end-to-end development.(48)

Figure 2: Stakeholder expectations matrix.(48)

Similar, the patients in this project expressed that it is important to be engaged as medicines simply cannot be developed without patient participation. Further, the patients, in the study by Boudes et al. expected from the industry to ensure that the patients’ needs are addressed in the research and take a holistic view of the patients’ requirements.(48) In accordance, the patients in this project outlined that the scientists will do further research with the patients’ input in mind. Hence, it can be argued that they expected from the industry to take the mentioned perspectives into account. In the study by Boudes et al. the patients interviewed were mainly from the United Kingdom, but also France, the United States of America, and Australia.(48) However, the patients in that study, as well as this project, agree across borders which could be due to the many similarities in the healthcare systems and that the patients can set requirements for healthcare as the countries are a part of the Western World.

A barrier to patient engagement is that the industry is perceived to court or coerce patients,(47) however, this was not perceived by the patients in the project. They are not troubled with the pharmaceutical industry and have full trust in them. This could be because some of the patients have participated in similar projects before, and they know quite a lot about medicines development. Thus, it could be a barrier to some patients, but they might never try to register for this 67 particular reason, or they could be some of the patients who dropped out of the project by not responding. Consequences for medicines R&D if the participating patients are more uncritical towards medicines, could be that the needs of all patients living with that disease might not be covered. Hence, the developed products may not be accepted and taken by all patients. To clarify in a broader sense, if patients are troubled with the pharmaceutical industry and have a feeling of being coerced to participate, it would require further research.

5.1.4 What are the project’s partners’ experiences with the execution of this kind of project where patients are recruited through community pharmacies? In general, the partners agreed that community pharmacies are suitable places of recruitment, and they are surprised by the number of patients recruited. Furthermore, they found the partnership positive, and it was labelled as a cross-sectoral collaboration. In a study by Petersen et al., 2020, cross-sectoral care was studied within professionals working with patients living with lower back pain. In this study, some challenges with cross-sectoral collaborations were identified being; lack of collaboration, lack of knowledge sharing, and lack of acknowledgement.(82) These challenges can be assessed in regard to this study as the partners stressed that a coordinating party is required as the resources to collaborate directly are limited. One partner also expressed that a solid project organisation was not established which could also indicate a lack of collaboration. The lack of knowledge sharing was not expressed by the partners, which may not be a problem as the master thesis student was the project coordinator and did the relevant and required knowledge sharing between the partners. Furthermore, in the preparational phase of the project, meetings were arranged to make decisions and inform all partners. Regarding the lack of acknowledgement, it was not expressed by the partners, which can be perceived as they acknowledge each other. It may not be an issue as the partners initiated the partnership themselves and thus were committed to complete the project. Also, the project was run on a pioneering spirit, and it could be imagined that all partners thought of each other to be important to complete the project. The study by Petersen et al. is, however, another research area as stated above,(82) and the results cannot be directly transferred to this project, but certainly, it is possible to learn from the collaboration challenges as outlined.

Overall, the partners believed that patients should be engaged throughout the process of medicines R&D, which corresponds to the literature where this agreement also exists.(38,45,46) In the stakeholder expectation matrix (Figure 2), the industry is generally expected by other stakeholders, as well as themselves, to involve patients in end-to-end development.(48) This was also

68 expressed by the project’s partner; however, they thought that the industry is not exclusively responsible. The fact that the partners believed the responsibility to be on several parties could be due to their pioneering mindsets. It can be perceived, as they want to change the status quo by which all are ready to take co-responsibility. Thus, other stakeholders may disclaim responsibility leading to a similar project not being executed as well.

5.2 Methods discussion Throughout the methods discussion, the only methods discussed are the ones conducted by the master thesis student, as outlined in the Study-part of the PDSA cycle. The use of semi- structured interviews is assessed to be the right method as it gave the possibility to extract feelings and believes from the participants, as intended in a semi-structured interview.(68) However, it can be discussed if the two following quality criteria were fulfilled; to largely interpret the interview responses during the interview, and to try to verify the interpretations during the interview.(83) The two criteria require interview experience to be able to meet them sufficiently, and the researcher should know what, why, and how the interview is conducted.(83) In this thesis, a novice researcher did the interviews, and how to conduct the interview was a learning process by keeping track of all the important aspects such as ensuring all questions were covered and asking follow-up questions. Hence, the fact that a novice researcher conducted the interviews may lead to the question of reliability, whether the results could be reproduced at other times and by other researchers.(75) If a very experienced researcher was to conduct the study, the results might differ, or be more elaborative. Nevertheless, the two criteria of interpreting the interview responses and verifying them during the interview, as well as reliability were fulfilled to the best extent possible, and thus the overall results should be possible to reproduce.

The sample size (N) can be argued to not be sufficient with regards to information power as the number of participants in the semi-structured interviews were nine patients, and five partners, respectively. Typically, N is 15 ± 10 participants in semi-structured interviews.(72) This leads to the question of how large N should be in a qualitative study, and the answer is; the larger the information power, the lower N, and the opposite.(84) When a novice researcher who has limited theoretical knowledge conducts the study, it can be argued that a larger N is needed to conclude something new, also when the aim is well-focused, and the interview dialogues are good.(84) In this thesis, this is probably true, but new conclusions were made as the tested concept is new. Also, N has not been possible to control in this project since it corresponds to the number of patients recruited through the new recruitment technique, or the number of partners. 69 The recruitment technique through pharmacies can be argued to be a non-probability sampling strategy as it is a non-random process that may lead to the sample not being representative of the target population.(41) The recruitment through community pharmacies in this project is by the patients’ initiative, and hence, the researcher does not decide whom to include besides setting the inclusion criteria. It may therefore be difficult to determine if the sample is representative, and it could be discussed if the results from the patient interviews are reliable. In this case, the generalisability of the patient interview results to another population is limited as the results concern a particular patient group as they know plenty about medicines R&D, and some of them have already participated in similar projects related to medicines R&D. This is probably not a coinci- dence as a specific patient group register for this type of project. Further, the majority (78 %) of the participating patients were females, thus an additional argument that generalising to another population is limited.

The COVID-19 pandemic in 2020 has potentially affected the results as all interviews, except one, were conducted virtually instead of face-to-face which have both advantages and disad- vantages. Some advantages of a telephone interview are that it is quicker and could possibly reduce bias as it is not possible to see the interviewer characteristics on responses. On the other hand, a disadvantage is the lack of visual cues that can be disabling.(71) As some of the interviews were video calls, it was possible to see some facial expressions and body language; however, it was very limited, and the video picture froze occasionally. Additionally, some participants might be more comfortable and open in their answers when they can hide behind a screen, but it might have the opposite effect on others as the virtual media makes them more uncomfortable. So, it may have been more in-depth answers if the interviews were face-to-face, but it might not. Especially for the focus groups, the virtual meeting was difficult to facilitate, also by video, as it was important to verbally indicate who should speak next. The focus group interviews were planned to include around five participants an hour, depending on how many patients registered to participate. It turned out that two to three patients should participate in each workshop, however, the method was not changed to avoid confusing the patients with a change of process. With only two to three participants in each focus group, it can be discussed if it is even a focus group, as usually, six to 10 people participate.(85) However, it can be argued that a kind of group dynamic was created anyway, even with only two to three participants, which is an advantage of a focus group.(71)

70 5.3 Act This study is highly relevant as it shows how patients can be recruited in community pharmacies to be engaged in medicines R&D, hence, the technique has potentially started a rethinking of patient recruitment. The results are reported to the project’s partners for them to know how to develop the recruitment technique further. To run the project in the future, some recommendations are presented to improve the project further (Figure 7).

Figure 7: The fully completed PDSA cycle describing all four parts in the project.

The good parts of the project recommended to keep would be the coordinating party, as the student was, as the results show this to affect the good execution of the project. A project coordinator should be leading the project communication, being the one point of contact for the patients, and ensuring all practical aspects, for example, written materials. Additionally, it is recommended to keep the preparational meetings between the partners to ensure knowledge sharing and well preparation. Last, the structure of patient introduction call leading to patient preparation call, and finally, the patient workshop should be kept as the patients seemed overall satisfied and well prepared. It is recommended to study the new recruitment technique more in-depth to do a deeper assessment of the pharmacies as a place of recruitment. This could be done by the project’s partners by initiating another PDSA cycle. Since the pharmacy staff was not actively recruiting in the

71 project, this could be tested in a new PDSA cycle to investigate the impact of this, as well as circumvent the need of a representative at the pharmacy. Furthermore, a broader target group would be recommended to test the full capacity of the recruitment technique. If the project is viewed as a continuous concept, the recommendation would be to have the patient organisation take the project coordinator role due to the limited time a student can participate in a project. Further recommended is to present an organisational chart of the project’s partners’ involvement so the patients can discern who is involved in the project, as transparency is an important factor identified in the study and the literature.

72 6. Conclusion

In conclusion, this study served the purpose of concept-testing a novel way to recruit people with a rare disease diagnosis in a community pharmacy to be engaged in medicines R&D in a pharmaceutical company. The recruitment was mainly accomplished by advertising the project at the pharmacy, and by a representative being physically present. The recruitment technique is considered positive by both patients and the project’s partners. Overall, the patients found the engagement setting enriching as the project generally lived up to their expectations. Furthermore, the patients are interpreted to be pleased to receive attention as they mistrust the public healthcare system and hence, seek other ways to solve their issues. In general, the patients did not observe which partners were included in the project, however, they found their participation exciting due to the various angles. The project’s partners found the project successful, including their mutual collaboration which is interpreted to be successful as they acknowledge each other and are willing to take responsibility. Conclusively, the proven concept has the potential to develop further to enable a new, standard recruitment practice for engagement in medicines R&D.

73 7. References

1. World Health Organization. Patient Engagement: Technical Series on Safer Primary Care. Geneva (Switzerland): World Health Organization; 2016 [cited 2020 Jul 11]. Available from: https://apps.who.int/iris/bitstream/handle/10665/252269/9789241511629- eng.pdf?sequence=1. 2. European Patients’ Academy on Therapeutic Innovation. Guidance for patient involvement in industry-led medicines R&D. Brussels (Belgium): European Patients’ Academy on Therapeutic Innovation; 2020 [cited 2020 Aug 3]. Available from: https://toolbox.eupati.eu/resources/guidance-for-patient-involvement-in-industry-led- medicines-rd/#disclaimer. 3. European Patients' Forum. About Us. Brussels (Belgium): European Patients' Forum; 2020 [cited 2020 Jul 19]. Available from: https://www.eu-patient.eu/About-EPF/whoweare/. 4. International Alliance of Patients' Organizations. Patient involvement in research. London (United Kingdom): International Alliance of Patients' Organizations; 2020 [cited 2020 Aug 7]. Available from: https://www.iapo.org.uk/patient-involvement-research. 5. Patvocates. What we do. Riemerling (Germany): Patvocates; 2019 [cited 2020 Aug 7]. Available from: https://www.patvocates.net/what-we-do/. 6. PwC. More demanding and discerning consumers are opening doors for new entrants in healthcare provision. United States: PwC; 2020 [cited 2020 Aug 7]. Available from: https://www.pwc.com/gx/en/industries/healthcare/emerging-trends-pwc-healthcare/new- entrants-healthcare-provision.html. 7. European Patients' Forum. EUPATI. Brussels (Belgium): European Patients' Forum; 2020 [cited 2020 Jul 19]. Available from: https://www.eu-patient.eu/whatwedo/EUPATI/. 8. Innovative Medicines Initiative. IMI1 Final Project Report Public Summary. Brussels (Belgium): Innovative Medicines Initiative; 2017 [cited 2020 Aug 30]. Available from: https://www.imi.europa.eu/sites/default/files/uploads/documents/projects/EUPATI_summary _final_report.pdf. 9. Innovative Medicines Initiative. EUPATI. Brussels (Belgium): Innovative Medicines Initiative; 2017 [cited 2020 Jul 11]. Available from: https://www.imi.europa.eu/projects- results/project-factsheets/eupati. 10. Food and Drug Administration. FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and 74 Regulatory Decision Making. Silver Spring (MD): Food and Drug Administration; 2020 [updated 2020 Jun 18; cited 2020 Aug 7]. Available from: https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug- development-guidance-series-enhancing-incorporation-patients-voice-medical. 11. Food and Drug Administration. FDA-led Patient-Focused Drug Development (PFDD) Public Meetings. Silver Spring (MD): Food and Drug Administration; 2020 [updated 2020 Mar 30; cited 2020 Aug 11]. Available from: https://www.fda.gov/industry/prescription-drug-user- fee-amendments/fda-led-patient-focused-drug-development-pfdd-public-meetings. 12. European Patients' Forum. EPF 2019 Summary Report. Brussels (Belgium): European Patients' Forum; 2019 [cited 2020 Jul 11]. Available from: http://www.eu- patient.eu/globalassets/library/publications/epf---2019-summary-report---web.pdf. 13. Vat LE, Finlay T, Schuitmaker-Warnaar TJ, Fahy N, Robinson P, Boudes M, et al. Evaluating the "return on patient engagement initiatives" in medicines research and development: A literature review. Health Expect. 2019;00:1-14. 14. Stergiopoulos S, Michaels DL, Kunz BL, Getz KA. Measuring the Impact of Patient Engagement and Patient Centricity in Clinical Research and Development. Ther Innov Regul Sci. 2019:1-14. 15. European Medicines Agency. Patients and Consumers. Amsterdam (The Netherlands): European Medicines Agency; 2020 [cited 2020 Feb 24]. Available from: https://www.ema.europa.eu/en/partners-networks/patients-consumers. 16. Food and Drug Administration. Learn more about Patient Engagement. Silver Spring (MD): Food and Drug Administration; 2020 [updated 2020 Mar 16; cited 2020 Aug 2]. Available from: https://www.fda.gov/patients/learn-more-about-patient-engagement. 17. Food and Drug Administration. Learn about FDA Patient Engagement. Silver Spring (MD): Food and Drug Administration; 2019 [updated 2019 May 8; cited 2019 Nov 18]. Available from: https://www.fda.gov/patients/learn-about-fda-patient-engagement#evolution. 18. Food and Drug Administration. FDA Patient Engagement Overview. Silver Spring (MD): Food and Drug Administration; 2020 [updated 2020 Apr 7; cited 2020 Aug 2]. Available from: https://www.fda.gov/patients/learn-about-fda-patient-engagement/initiatives-patients. 19. Food and Drug Administration. About the FDA Patient Representative Program. Silver Spring (MD): Food and Drug Administration; 2018 [updated 2018 May 3; cited 2020 Aug 2]. Available from: https://www.fda.gov/patients/learn-about-fda-patient-engagement/about-fda- patient-representative-program.

75 20. Food and Drug Administration. Advisory Committee Research, Reports, and Announcements. Silver Spring (MD): Food and Drug Administration; 2018 [updated 2020 Mar 27; cited 2020 Feb 24]. Available from: https://www.fda.gov/advisory- committees/about-advisory-committees/advisory-committee-research-reports-and- announcements. 21. Food and Drug Administration. CDRH Patient Engagement Advisory Committee. Silver Spring (MD): Food and Drug Administration; 2019 [cited 2020 Feb 24]. Available from: https://www.fda.gov/about-fda/cdrh-patient-engagement/cdrh-patient-engagement-advisory- committee. 22. European Medicines Agency. Key milestones of EMA interaction with patients and consumers. Amsterdam (The Netherlands): European Medicines Agency; 2020 [cited 2020 Aug 2]. Available from: https://www.ema.europa.eu/en/documents/other/key-milestones- ema-interaction-patients-consumers_en.pdf. 23. European Medicines Agency. How the committees work. Amsterdam (The Netherlands): European Medicines Agency; 2020 [cited 2020 Jul 12]. Available from: https://www.ema.europa.eu/en/committees/how-committees-work. 24. European Medicines Agency. Getting involved. Amsterdam (The Netherlands): European Medicines Agency; 2020 [cited 2020 Feb 24]. Available from: https://www.ema.europa.eu/en/partners-networks/patients-consumers/getting-involved. 25. European Medicines Agency. Patients' and Consumers' Working Party. Amsterdam (The Netherlands): European Medicines Agency; 2020 [cited 2020 Aug 2]. Available from: https://www.ema.europa.eu/en/committees/working-parties-other-groups/chmp/patients- consumers-working-party. 26. Food and Drug Administration. FDA and European Medicines Agency Patient Engagement Cluster. Silver Spring (MD): Food and Drug Administration; 2018 [updated 2018 Jan 8; cited 2020 Feb 24]. Available from: https://www.fda.gov/patients/learn-about-fda-patient- engagement/fda-and-european-medicines-agency-patient-engagement-cluster. 27. INVOLVE. About INVOLVE. Southampton (United Kingdom): INVOLVE; 2020 [cited 2020 Feb 14]. Available from: https://www.invo.org.uk/about-involve/. 28. Patient-Centered Outcomes Research Institute. Our Story. Washington, D.C.: Patient- Centered Outcomes Research Institute; 2014 [updated 2017 Mar 27; cited 2020 Feb 14]. Available from: https://www.pcori.org/about-us/our-story. 29. Heinemann L. The Failure of Exubera: Are We Beating a Dead Horse? J Diabetes Sci Technol. 2008;2(3):518-29. 76 30. Herper M. Pfizer Kills Exubera. Jersey City (NJ): Forbes; 2007 Oct 18 [cited 2020 Jul 12]. Available from: https://www.forbes.com/2007/10/18/pharmacuticals-pfizer-exubera-biz-sci- cx-mh-1018pfizer.html#69a2c361040f. 31. Lægemiddelindustriforeningen. Regler og kodeks. Copenhagen (Denmark): Lægemiddelindustriforeningen; 2019 [cited 2019 Oct 18]. Available from: https://www.lif.dk/Etik/Sider/Regler-og-kodeks.aspx. 32. Etisk nævn for lægemiddelindustrien. Patientforeningskodeks. Copenhagen (Denmark): Etisk nævn for lægemiddelindustrien; 2019 [cited 2019 Oct 18]. Available from: https://www.enli.dk/regler/patientforeningskodeks/. 33. Etisk nævn for lægemiddelindustrien. Virksomheder, der har tilsluttet sig ENLI’s kompetence (pr. 1. august 2020). Copenhagen (Denmark): Etisk nævn for lægemiddelindustrien; 2020 [updated 2020 Aug 1; cited 2020 Aug 3]. Available from: https://www.enli.dk/media/50006/liste-over-tilsluttede-virksomheder-01-08-2020.pdf. 34. Etisk nævn for lægemiddelindustrien. VEJLEDNING til ”Etiske regler for lægemiddelindustriens samarbejde med patientforeninger mv.”. Copenhagen (Denmark): Etisk nævn for lægemiddelindustrien; 2020 [updated 2020 Feb; cited 2020 Jul 12]. Available from: https://www.enli.dk/media/49945/vejledning-til-patientforeningskodekset-version-20- februar-2020.pdf. 35. European Federation of Pharmaceutical Industries and Associations. EFPIA Code of Practice. Brussels (Belgium): European Federation of Pharmaceutical Industries and Associations; 2019 Jun 27 [cited 2020 Aug 30]. Available from: https://www.efpia.eu/media/554677/efpia-code-2020.pdf. 36. Etisk nævn for lægemiddelindustrien. Etiske regler for lægemiddelbranchens samarbejde med patientforeninger mv. (Patientforeningskodekset). Copenhagen (Denmark): Etisk nævn for lægemiddelindustrien; 2020 [updated 2020 Jan; cited 2020 Jul 12]. Available from: https://www.enli.dk/media/49895/etiske-regler-for-samarbejde-patientforeninger-version-20- januar-2020.pdf. 37. Warner K, See W, Haerry D, Klingmann I, Hunter A, May M. EUPATI Guidance for Patient Involvement in Medicines Research and Development (R&D); Guidance for Pharmaceutical Industry-Led Medicines R&D. Front Med (Lausanne). 2018;5:270. 38. Geissler J, Ryll B, di Priolo SL, Uhlenhopp M. Improving Patient Involvement in Medicines Research and Development: A Practical Roadmap. Ther Innov Regul Sci. 2017;51(5):612-9.

77 39. Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle. Including the patient perspective. 2019 [cited 2019 Oct 21]. Available from: https://www.imi- prefer.eu/about/. 40. Patients Active in Research and Dialogues for an Improved Generation of Medicines. PARADIGM. 2019 [cited 2019 Oct 21]. Available from: https://imi-paradigm.eu/. 41. Food and Drug Administration. Patient-Focused Drug Development: Collecting Comprehensive and Representative Input – Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders. Silver Spring (MD): Food and Drug Administration; 2020 June [updated Jun 16; cited 2020 Aug 7]. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient- focused-drug-development-collecting-comprehensive-and-representative-input. 42. International Pharmaceutical Abstracts [Internet]. Clarivate Analytics. 2019 [cited 2019 Nov 12]. Available from: https://www.ovid.com/product-details.109.html 43. Embase [Internet]. Elsevir Limited. 2019 [cited 2019 Nov 12]. Available from: https://www.embase.com/login. 44. PubMed [Internet]. National Library of Medicine. 2019 [cited 2019 Nov 12]. Available from: https://www.ncbi.nlm.nih.gov/pubmed/. 45. Borup G, Bach KF, Schmiegelow M, Wallach-Kildemoes H, Bjerrum OJ, Westergaard N. A Paradigm Shift Towards Patient Involvement in Medicines Development and Regulatory Science: Workshop Proceedings and Commentary. Ther Innov Regul Sci. 2016;50(3):304- 11. 46. Wilson H, Dashiell-Aje E, Anatchkova M, Coyne K, Hareendran A, Leidy NK, et al. Beyond study participants: a framework for engaging patients in the selection or development of clinical outcome assessments for evaluating the benefits of treatment in medical product development. Qual Life Res. 2018;27(1):5-16. 47. Boutin M, Dewulf L, Hoos A, Geissler J, Todaro V, Schneider RF, et al. Culture and Process Change as a Priority for Patient Engagement in Medicines Development. Ther Innov Regul Sci. 2017;51(1):29-38. 48. Boudes M, Robinson P, Bertelsen N, Brooke N, Hoos A, Boutin M, et al. What do stakeholders expect from patient engagement: Are these expectations being met? Health Expect. 2018;21:1035-45. 49. Hansen MB, Norgaard LS, Hallgreen CE. How and Why to Involve Patients in Drug Development: Perspectives From the Pharmaceutical Industry, Regulatory Authorities, and Patient Organizations. Ther Innov Regul Sci. 2019:1-9. 78 50. Young K, Kaminstein D, Olivos A, Burroughs C, Castillo-Lee C, Kullman J, et al. Patient involvement in medical research: what patients and physicians learn from each other. Orphanet J Rare Dis. 2019;14:21. 51. Stein S, Bogard E, Boice N, Fernandez V, Field T, Gilstrap A, et al. Principles for interactions with biopharmaceutical companies: the development of guidelines for patient advocacy organizations in the field of rare diseases. Orphanet J Rare Dis. 2018;13:18. 52. Anand A, Brandwood HJ, Jameson Evans M. Improving Patient Involvement in the Drug Development Process: Case Study of Potential Applications from an Online Peer Support Network. Clin Ther. 2017;39(11):2181-8. 53. Hughes A, Landers D, Arkenau HT, Shah S, Stephens R, Mahal A, et al. Development and Evaluation of a New Technological Way of Engaging Patients and Enhancing Understanding of Drug Tolerability in Early Clinical Development: PROACT. Adv Ther. 2016;33:1012-24. 54. Hoos A, Anderson J, Boutin M, Dewulf L, Geissler J, Johnston G, et al. Partnering With Patients in the Development and Lifecycle of Medicines: A Call for Action. Ther Innov Regul Sci. 2015;49(6):929-39. 55. Business Ballerup. Erhvervs- og vækstpolitik 2017-2021. Ballerup (Denmark): Ballerup Kommune; 2017 [cited 2019 Oct 14]. Available from: https://ballerup.dk/sites/default/files/bb_erhvervspolitik_2017_.pdf. 56. Danmarks Apotekerforening. Værd at vide om Ballerup Apotek. Copenhagen (Denmark): Danmarks Apotekerforening; 2019 [cited 2019 Oct 14]. Available from: https://www.apoteket.dk/systempages/pharmacyservicepage?pharmacyId=1127. 57. Danmarks Apotekerforening. Værd at vide om Skovlunde Apotek. Copenhagen (Denmark): Danmarks Apotekerforening; 2019 [cited 2019 Oct 14]. Available from: https://www.apoteket.dk/systempages/pharmacyservicepage?pharmacyId=1368. 58. Danmarks Apotekerforening. Markant bedre tilgængelighed til apoteker – for lavere medicinudgifter: 45 % flere apoteker, længere åbningstider og rekordlav ventetid giver bedre tilgængelighed. Copenhagen (Denmark): Danmarks Apotekerforening; 2018 Jan 22 [cited 2019 Oct 14]. Available from: https://www.apotekerforeningen.dk/- /media/apotekerforeningen/analysertilgaengelighed/22012018-flere-apoteker.pdf. 59. Foreningen for Autoimmune Sygdomme. Hvorfor FAIM? Farsø (Denmark): Foreningen for Autoimmune Sygdomme; 2020 [cited 2020 Jul 26]. Available from: https://www.faim.dk/om-faim/. 60. LEO Pharma. 111 years of building, sharing and reinvesting in knowledge. Ballerup (Denmark): LEO Pharma; 2020 [cited 2020 Jul 26]. Available from: https://leo-pharma.com/. 79 61. LEO Pharma. We address the huge unmet medical need for people living with a rare skin disease. Ballerup (Denmark): LEO Pharma; 2020 [cited 2020 Jul 26]. Available from: https://leo-pharma.com/your-health/rare-diseases. 62. ACT Academy for their Quality Service Improvement and Redesign suite of programmes. Plan, Do, Study, Act (PDSA) cycles and the model for improvement. United Kingdom: National Health Service; 2018 [cited 2020 Apr 15]. Available from: https://improvement.nhs.uk/documents/2142/plan-do-study-act.pdf. 63. National Organization for Rare Disorders. Get involved. United States of America: National Organization for Rare Disorders; 2020 [cited 2020 Apr 15]. Available from: https://rarediseases.org/rare-disease-day/get-involved/. 64. Foreningen for Autoimmune Sygdomme. Vi ses på apoteket. Farsø (Denmark): Foreningen for Autoimmune Sygdomme; 2019 [cited 2020 Mar 24]. Available from: http://visespåapoteket.dk/. 65. LEO Pharma. Ny offentligt/privat samarbejde sætter fokus på udnyttelse af patienters viden til medicinudvikling. Copenhagen (Denmark): Ritzau; 2019 Oct 29 [cited 2020 Apr 15]. Available from: https://via.ritzau.dk/pressemeddelelse/ny-offentligtprivat-samarbejde- saetter-fokus-pa-udnyttelse-af-patienters-viden-til- medicinudvikling?publisherId=12353927&releaseId=13582137. 66. Wolf U. Nyt samarbejde skal give bedre medicin. Ballerup (Denmark): Ballerup Bladet; 2019 Dec 10 [cited 2020 Jul 16]:[10 p.]. Available from: http://www.e- pages.dk/ballerupbladet/186/. 67. Datatilsynet. Lovgivning. Valby (Denmark): Datatilsynet; 2020 [cited 2020 Jul 16]. Available from: https://www.datatilsynet.dk/generelt-om-databeskyttelse/lovgivning. 68. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 2, Karakteristik af kvalitative forskningsinterview; p. 45-74. 69. Babbie E, Beiting-Lipps E, Kindstrom K. The Practice of Social Research. 14 ed. Mason (OH): Cengage Learning; 2015. Chapter 1, Human Inquiry and Science; p. 4-30. 70. Robson C, McCartan K. Real World Research. 4 ed. Chichester (United Kingdom): John Wiley & Sons Ltd; 2016. Chapter 8, Multi-strategy (mixed method) designs; p. 174-86. 71. Robson C, McCartan K. Real World Research. 4 ed. Chichester (United Kingdom): John Wiley & Sons Ltd; 2016. Chapter 12, Interviews and focus groups; p. 284-306.

80 72. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 6, Tematisering og design af en interviewundersøgelse; p. 151-76. 73. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 7, Udførelse af et interview; p. 177-96. 74. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 12, Interviewanalyser med fokus på mening; p. 267-84. 75. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 15, Den sociale konstruktion af validitet; p. 313-38. 76. Robson C, McCartan K. Real World Research. 4 ed. Chichester (United Kingdom): John Wiley & Sons Ltd; 2016. Chapter 7, Flexible desings; p. 145-73. 77. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 4, Etiske spørgsmål i forbindelse med interview; p. 105-26. 78. Robson C, McCartan K. Real World Research. 4 ed. Chichester (United Kingdom): John Wiley & Sons Ltd; 2016. Chapter 10, Ethical and political considerations; p. 205-40. 79. Robson C, McCartan K. Real World Research. 4 ed. Chichester (United Kingdom): John Wiley & Sons Ltd; 2016. Chapter 11, Surveys and questionnaires; p. 243-83. 80. Lægemiddelstyrelsen. Apotekeres anmeldelse og ansøgning om tilladelse til at være knyttet til virksomheder. Copenhagen (Denmark): Lægemiddelstyrelsen; 2019 [updated 2019 Jul 17; cited 2020 Aug 23]. Available from: https://laegemiddelstyrelsen.dk/da/godkendelse/sundhedspersoners-tilknytning-til- virksomheder/apotekere/. 81. Etisk nævn for lægemiddelindustrien. ENLI-tilsluttede virksomheders samarbejde med patientforeninger mv. 2019. Copenhagen (Denmark): Etisk nævn for lægemiddelindustrien; 2019 [cited 2020 Aug 23]. Available from: https://www.enli.dk/media/49950/samarbejder- med-patientforeninger-2019.pdf. 82. Petersen L, Birkelund R, Schiøttz-Christensen B. Challenges to cross-sectoral care experienced by professionals working with patients living with low back pain: a qualitative interview study. BMC Health Serv Res. 2020;20:164.

81 83. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 9, Interviewkvalitet; p. 219-34. 84. Malterud K, Siersma VD, Guassora AD. Sample Size in Qualitative Interview Studies: Guided by Information Power. Qual Health Res. 2016;26(13):1753-60. 85. Kvale S, Brinkmann S. Interview - Det kvalitative forskningsinterview som håndværk. 3 ed. Copenhagen (Denmark): Hans Reitzels Forlag; 2015. Chapter 8, Interviewvariationer; p. 197-218.