Clinical and Economical Assessment of Digital Health Innovations in Austria

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Entrepreneurship &Innovation Professional MBA Clinical andeconomicassessmentofdigitalhealth innovations inAustria-anecosystemview A Master'sThesissubmittedforthedegreeof “Master ofBusinessAdministration” Valéria Szijártó,MD,PhD Vienna, 20.06.2020 Deepa Mani,PhD supervised by 11831500 Powered by TCPDF (www.tcpdf.org)

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

I, Affidavit Vienna, 20.06.2020 VALÉRIA SZIJÁRTÓ,MD,PHD .thatIhavenotpriortothisdatesubmittedthetopicofMaster’sThesisorparts 2. thatIamthesoleauthorofpresentMaster’sThesis,"CLINICALAND 1. abroad. of itinanyformforassessmentasanexaminationpaper,eitherAustriaor or toolotherthanthosereferencedanyillicitaidtool,and AN ECOSYSTEMVIEW",104pages,bound,andthatIhavenotusedanysource ECONOMIC ASSESSMENTOFDIGITALHEALTHINNOVATIONSINAUSTRIA- , herebydeclare ______Signature Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. my gratitude towards all those helping me completing this work. The two e no have I times. extraordinary in apparent becomes support Extraordinary completing thesis. towards this for And my family, for their never-failing love and support. This thesis is d friends, My guidel motivated. formatting me keeping sending understanding, that social just distancing was not only on the physical, but or managers examples program thesis good the recommending and students Fellow work. attitud and patience remarkable showing “theoretical” for FWF, and FFG at Employees kno their share to energy and time the found - this like times, in il eemn te ok edn t ti mse tei. u CVD1 came includ billions, of life professional and personalthe in everything COVID-19 But thesis. master this to leading work the determine will were emphasized during the master program several times. I did not expect, that Adaptation and flexibility; two terms, that terms, two flexibility; and Adaptation P

re fa c e

questions . My My supervisor, Deepa Mani, for providing inspiring comments on the are 1 often associated with entrepreneurship and entrepreneurship with associated often ing wledge and experience. experience. and wledge my on every possible level. edicated toall of them. nicely planned route planned nicely xperts, xperts, who words to express to words n changed and these these terms towards e – even ines, Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. digital health solutions and position the countryand in the asleader position a healthdigital solutions digital of development the support could elements These reimbursement. for evidence ae o itrainl xmls te utin csse cud eei fo the from benefit could ecosystem Austrian the development of real-world testbeds and from a examples, international on Based createsa barrier health developers.for cli of resistance Furthermore, validation. health digital supporting private in and infrastructure thein gaps are there However, citizens. digi high the and health digital for program accelerator an opportunities, taey te rsne o presence the strategy, The Austrian digital health validation is ecosystem strengthened by the n reviewed. challengesThe barriers and in the ecosystemby were identified digital Austrian on publications the conductedand were interviews search literature Austria, in model this of elements the identify To First, First, a model of a digital health validation ecosystem was established based o these possibilities to international practices international topossibilities these In this thesis I explore the possibility to validate digital healt remain aboutnew health innovations. oftensceptical digital agencies clinical outcome and cost effectiveness. Therefore, clinicians, regulatory bod

there are just scarce examples scarce just are there efficiency efficiency and accessibility improvi by healthcare disrupt to expected are products health Digital A

bs tra c t

f . The The number of digital health products has grown rapidly, however an lcrnc elh eod ytm nmru pbi funding public numerous system, record health electronic of

pr oven benefits oven .

2 na tional tional of digital health solutions regarding their regarding solutions health digital fr amework describing the necessary h solutions in Austria and compare health clinical trials were clinicaltrials health health financing opportunities financing iin twrs digital towards nicians was performed, expert performed, was

ational digitalization g ult o care, of quality ng expert interviews. expert tal literacy of the of literacy tal . ies ies and insurance

n the literature validated .

List of tablesof List List of abbreviations ofabbreviations List 1. on Model of a digital health validation ecosystemhealth ofa validation digital Model Validation of digital health in Austria digital of healthinAustria Validation Methods Methods Literature overview Literature Introduction Introduction 5.4. Limitations Limitations 5.4. 5.3. Austrian digital health validation ecosystem health Austrian digital 5.3. 5.2. Results of expert interviews ofexpert interviews Results 5.2. 5.1. Institutes involved in published clinical trials for clinical digital trials published in involved Institutes 5.1. 3.3. Qualitative expert interviews Qualitative 3.3. 3.2. Literature review Literature 3.2. 3.1. Establishing a model for digital health validationa modelecosystem for Establishing health digital 3.1. 2.4. Digital health and eHealth in Austria eHealthhealth and in Austria Digital 2.4. 2.3 Validation of digital health innovation digital of healthinnovation Validation 2.3 2.2 Industry overview and prospect and overview Industry prospect 2.2 2.1 Digital health: definitions and categories definitionshealth: Digital categories and 2.1 1.3 Course of investigation ofinvestigation Course 1.3 . Objective 1.2 formulation Problem 1.1 5.2.2. Analysis of interviews Analysis 5.2.2. 5.2.1. Summary of interviews of Summary 5.2.1. 3.3.2 Content analysis Content 3.3.2 3.3.1. Interview checklist Interview 3.3.1. 2.4.1. Digital health companies in Austriahealth companies Digital in 2.4.1. 2.3.3. Real-world testbed as a tool for digital health validation aforas validation tool health digital testbed Real-world 2.3.3. 2.3.2. Economic validation and validation reimbursement Economic 2.3.2. 2.3.1. Clinical validation validation Clinical 2.3.1. te nts ......

...... 3 ...... health in Austria healthinAustria ......

72 65 60 58 48 58 49 46 56 56 45 43 43 43 41 37 31 27 22 18 15 12 10 12 10 1 2 5 6 7 9 9

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Appendix Appendix Bibliography Bibliography 6. Discussion Discussion ...... 4

85 76 73

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Own illustration. Figure14 Model aof supportive ecosystem for the clinical and economic validation o interventions. Source: (World Organization, Health 2019a, p. xiii) Figure13 Components anof enabling ecosystem for theimplementation of digital heal identifiedliterature by review, Figure15 Members theof digital health validation ecosystem Austria in (own illustration). Figure12 Digital Sub-indicis Health the on levels. 3 Source: (Rainer etThiel al., p. 2018, 22 16) environmentand the technology readiness testedof innovations. Source: (Arntzen et al., 2 Figure The9 relationship the different of testing methods based on the level of control the in Thiel etThiel al., 2018, p. 30) Figure11 Levels of the Digital Health Indexdeveloped the Bertelsmannby Stiftung. Source: ( Source: (LeFevre al.,et 2017, p.13) Figure Economic8 evaluation methods recommended for the different stages of development. 2019, p. 7) Figure Evidence6 tiers definedNICE. by Source: (National Institutefor Health andExcellence, Care p. 11)) Figure10 eHealth index of 37 European countriesin 2016.(Piotr Source: Arak & Anna Wójci from https://www.flaticon.com/. Figure Elements5 validationof process for digital health products. Own illustration fre with on figureon in (IMDRF Softwareas a Medical Device WorkingGroup, 2017, p. 4) Figure Comprehensive7 clinical evaluation SaMDof proposed by the IMDRF. Own illustration (Sean, 2020) Figure VCfunding4 digitalof health companies globally according to productcategories. So Figure 3 healthDigital tools. Source: (IQVIA Institute for Human Science,Data 2017, p.3) Source:(Istepaniana AlAnzi, & 2020, p. 721) Figure Relationship2 between the different domains and definitions related to digitalhealth. FigureCourse 1 investigationof (own illustration) Ta

bl ...... e ...... of fi ...... gu ...... re s

...... ** ...... Elementsidentified from expertinterviews ...... 5 ...... f digital health. th ...... 2) 2) *Elements 019, p. e iconse urce: k, 2017, ...... Rainer based

71 50 39 51 39 30 38 24 18 32 17 19 13 12 11

Table 3Table Interviewchecklist topicswith and questions Table 2Table Summary of inclusionexclusion and criteria in the literaturereview Table 1Table Overview of methods and full partial for economicevaluation Li

s t of ta bl e s

6 ......

46 56 44 28

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. FDA FDA EMA EMA Gesundheitsakte Record)(Electronic - elektronische ELGA Health EIT eHealth Group Stakeholder- eHSG Health (Digital -Act)Digitale-Versorgung-Gesetz Service DVG DHI analysisCostutility - CUA CRO Centres - for & Services CMS Medicare Medicaid Costminimization - analysis CMA Costeffectiveness - analysis CEA consequence Cost CCA - analysis analysis Cost benefit CBA- (Federal Selbsthilfe Association Österreich Bundesverband Self-Help of - BVSHOE AIT AI -AustralianDigital Agency ADHA Health ABM Li ICT HTA HSC GP (Austrian GmbH GesundheitInstitute) Österreich Health Public - GÖGNational GmbH GDPR derFondsFörderung zur ForschungSciencewissenschaftlichen- F (Austrian FWF FederalCommission Trade- FTC Agency) FFG

s – t – – – – Artificial Intelligence – – – general practitioner – – European Institute Innovation Technology & for – Information and Communication Technology Communication Information and AustrianInstitute ofTechnology GmbH of abbrev – – digital healthintervention Österreichische Forschungsförderungsgesellschaft (Austrian Research Promotion(Austrian Österreichische Forschungsförderungsgesellschaft Health SystemHealth Challenge Food and Food Administration Drug Health Technology Assessment Contact Research Contact Organization – European Medicines Agency Agency European Medicines Agent-basedmodelling – General Data Regulation Protection Gesellschaftmit Haftung (Limited beschränkten Company) Liability iatio n s

7 und) und) Austria) Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. SME SaM -Patient Remote Monitoring RPM Randomized Controlled - RCTTrial QUALY applicationapproval - premarket PMA Officeof Coordinator National- theHealth forONC Information Techn Health Care and for Excellence - Institute National NICE NHS ML LBI UnionInternational Telecommunication - ITU IMDRF ICT WHO VC USP Unfair Commercial- Practices UCPD Directive – – – – D venturecapital – Ludwig Boltzmann InstituteBoltzmann Ludwig – information communication technology informationcommunication technology Machine Learning Machine – unique selling point selling unique – Service HealthNational Small EnterprisesMedium-sizedSmall and - Softwarea as - Medical Device World Health Organization Organization Health World – – InternationalDevice Regulators Medical Forum Quality-adjusted yearslife 8 ology ology Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. into digital health grew 1000-fold over the last 10 years (Mercom Capital lastGroup(Mercom theover grew10 digital years into health 1000-fold invest VC sector: the into investments large to led have promises These options. care healthcare system and the unequal accessibility of people to latest,the o particularly are chro of burden these increased the globally, population of aging the considering All costs). lower potentially (and efficiency gai could system the while care, of quality higher and to access increased coul Patients follow-up. and management till diagnosis through prevention, of domains all improve to potential the have innovations health Digital managing it well will become attractive for innovators(Arntzen 2019) et al., attractivebecomewillit wellinnovators(Arntzen managing for 2019) al., et (Ferretti val complex a creating costs, the on companies insurance and efficacy, Wh 2020). al., et (Goldsack aut health protection, data and privacy on focus authorities protection evidence of body this develop to how on guidance as knowledge comprehensive of lack a is there However, 2019). al., and et (Mathews usability clinical, privacy, (technical, perspectives several from validated Since digital healthproductsare different atthet interception of Velthoven few(van etal., a name to 2018). h of worseningand care proper accessing in delay dissatisfaction, stress, techno harmful even or unsafe quality, low of use The apps). health (e.g. without the oversight by medical andprofessionals are out of the scope of re health digital many since problematic, particularly is This 2019). al., et we products few a only validation, economic of lack the to Similar unproven remained estimations & Jones, 2013). (Kellermann these 2012), Commission, (European services eHealth similarly, large savings were projected by the Commission European from the in (Hillesta USA the in annually billion $81 save could system record medical Although the RAND Health Information Technology Project team estimated t yet. demonstrated been not have promises clinical and economic these of most However,

1.1 1.

I P ntro r oble ducti m

formu on . Countries try to lower this complexity in different ways, and those, and ways, different in complexity this lower to try Countries

lat ion

9 echnologies, they should beechnologies, they should re clinically tested (Safavi tested clinically re products can be used be can products horities on safety andsafety on horities ften ften most expensive ealth condition, justcondition, ealth nic diseases on the on diseases nic n on the increased the on n dto ecosystem idation d et al., 2005) and 2005) al., et d logies can lead to lead can logies . motn when important hat an electronic elhae from healthcare

gulatory gulatory control d benefit from benefit d troduction troduction of ,2019) economic) economic) l data ile el as well . ment

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. To answer these questions, I conduct I questions, these answer To answer toI try Therefore, researchwill following the questions: Austriantheecosyst digital health guide furthercan the developmentof Additionally, I will try to identify international lackingood practices an clinical conducting of barriers potential and factors supportive eco Austrian the explore I will validation. health digital of aspect economic foc will I thesis, this o In health. digital evidence to specific frameworks methods, growing a also is There products. health digital non-validated of There are international examples on how countries or even regions try to addre of tothe trough disillusionment of productivity. plateau and 2012) Commission, (European health digital of potential full productsis validateddigitalquality and health increaseinhigh The model th placed and literature the on based ecosystem validation health digital identi I step, next a As validation. economic and clinical for practices iia hat ciia til i Asra n fnly y targeted t overview by literature the from selected finally were good-practices international and Austria in trials clinical health digital review systematic conducting by experts, local with interviews from model

- 1.2 1.3 - - -

Are there any examples any there Are Which support do digital health companies get for the validation of validation the for get companies health digital do support Which health companies in Austria? companies inAustria? health from playersin different from innovationAustrianhealthcare ecosyste the and supplement Austriansupplement the scenery? Are there any examples for supportive internationally elements ecosystem that coul r tee n gp i te csse o ay hlegs ae b dgtl health digital by faced challenges any clinicalwhenconducting or companies economic or validation? ecosystem the in gaps any there Are For the analysis of the Austrian ecosystem, I identified the Austrian el Austrian the identified I ecosystem, Austrian the of analysis the For O C ours bje ctiv e of e

in v est igatio on n

the clinical and/or economic assessments done by digital digital by done assessments economic and/or clinical the ed literature search to explore requirements and good and requirements explore to search literature 10

guide the industry from the the from industry the guide ieaue erh Finally, search. literature g in Austria, which finding seen as key to exploit asexploit seen the key to eooi validation. economic d fied the elements of a of elements the fied us on the clinical and clinical the on us em. em in an ecosysteman in em o fill the gaps and gaps the fill o system to identifyto system their product their ss the problem ements of this this of ements f published of validation n m? m? d s

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Figure 1 Course of investigation 1 of Course illustration) (own Figure ecosyste validation 1 courseinvestigationinsummarizing the Figure is of health digital Austrian the in challenges the solve • • 2) (Chapter overview Literature health validation methods tailored to digital health, practices good health, digital to tailored methods validation of digital validation economic and on clinical requirements • 4) (Chapter ecosystem validation health the digital of Model ecosystem (literature analysis) (literature ecosystem identification of the elements of a digital health validation validation health digital of a elementsthe of identification • • 5) (Chapter ecosystem validation health digital Austrian the of Elements interviews) (expert interviews, literature review, targeted literature search) literature targeted review, literature interviews,(expert Gaps and challenges in the Austrian ecosystem (expert ecosystem Austrian the in challengesand Gaps Elements of digital health validation ecosystem in Austria Austria in ecosystem health validation of digital Elements (expert interviews, literature analysis) literature interviews,(expert 6) (Chapter ecosystem the Austrian supplement to Good-practices 11

. h flowchart The m.

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. health” o use “the is eHealth) of part is (which Health, mobile health- and health of support in technology communications of use “the is eHealth (WHO), Organization Health World the to According eHealth, mHealth, digital health interventions to name a few name to mHealth, healthinterventions eHealth, digital d eHealth Stakeholder Group (eHSG) for digital health products and solutions: solutions: and products health digital for (eHSG) Group Stakeholder eHealth Su Reimbursement the by used was definition informative more but similar, A 721)2020, p. the2Relationship related between definitions and Figure domains different i shown as depicted be may terms different the between relationship repre (ICT) Technology” eHealth, mHealth, telemedicine and telehealth. As summarized by Istepanian Communication and “Information term the Additionally, intelligence” artificial and genomics data’, areas, emerging as well as mHealth), heal digital to related definitions and nomenclatures several are There

& AlAnzi, 2020). & AlAnzi, (ICTs) across the whole range of functions that affect the health sector, th sector, health the affect that functions of range whole the across (ICTs) related and technologies services utilise which information communi and efinition of health,as digital a “broadumbrella encompassingterm eHealt 2.1 2.

Lite Wrd elhOgnzto, 09,p x. h sm gieiepoie te l the provides guideline same The ix). p. 2019b, Organization, Health (World D igi t ra al ture he alt ove h : def rv ini iew t i ons

and such as the use of advanced computing sciences in ‘big in sciences computing advanced of use the as such

categories 12 Wrd elh raiain 21b p ix) p. 2019b, Organization, Health (World

to digital health. Source: &AlAnzi, (Istepaniana digital to health. mbl wrls tcnlge for technologies wireless mobile f . related fields” and mHealth or mHealth and fields” related

n Figure n cation technologies cation technologies hincludes(which h technologies th information and information at can improve can at and AlAnzi, the 2 bgroup of the of bgroup (Istepaniana “Medi sents atest

cal : . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. devices” orofoutsideproducts categorydevices” medicaldevices. the of In EU, the prod health digital the perspective, regulatory a From Appprovided completeThe categorizationof digital is health. benefits demon to (HSC) Challenge System Health a with interventions the linking categories The are further and divided into guideline subcategories the interventions for health system and resource managers and iv) intervent categories: i) targeting clients (and caregivers of clients), ii) targeting h interventi health digital the groups it and users, targeted the on based Th 2018). Organization, Health (World language shared of use the ensure heal ofdigital classification comprehensive a promoted WHO the 2018, In tools.3 health Digital Source: Figure was solutions health digital of breadth The 3) inby IQVIAInstitutedemonstrated the(Fig. services. 2017 and products diverse covers range broad a uses health digital definition, actual the of Independent comprehensivehealth,the digital of definition is eHSG thefollowe and mHealth eHealth areaccording used to the latest ofdefinition the WHO health” man and prognosis prediction, monitoring, treatment, diagnosis, prevention, reagent, material or other article intended by the manufacturer to be used, be to manufacturer the by intended article other or material reagent, device’ “‘medical eS Sbru o Ribreet 21, . ) I ti tei, h expressions the thesis, this In 6). p. 2019, Reimbursement, on SubGroup (eHSG en ay ntuet aprts apine sfwr, implant, software, appliance, apparatus, instrument, any means (IQVIA InstituteScience, (IQVIA Humanp. 2017, 3) Data for 13

cs a b casfe a “medical as classified be can ucts d. d.

ealthcare providers, iii) endix 1. ions for data services. provides examples provides ofexamples of technologies and technologies of strate the potential the strate ons into four main four into ons th interventions tointerventions th

is classification is classification is above, while above, for gmn of agement alone or in or alone

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

European regulations: regulations: European device medical defines Act Cosmetic and Drug Food, Federal the USA, the In The category Software as a Medical Device (SaMD) was introduced by the International International the Medica by introduced was (SaMD) Device Medical a as Software category The n wih os o ahee t picpl nedd cin y pharmacolo by action its in function suchby mean intended principal its b achieve may which but body, human onthe or in metabolicmeans, or immunological not does which and - - - - medi specific following the of more purposes: or one for beings human for combination, - - - part, relatedother similar or or or article,includinga component i contrivance, machine, implement, apparatus, instrument, “an 2017, 15). Union, p. does does not include upon software functions excluded pursuant to dependent section not term The is purposes. intended primary which its of and achievement the animals for metabolized other or man of body the on or and which does not achieve its primary intended purposes through chemi DEFINITIONS Sec. 321 - Definitions; Generally, -Definitions; p. 2006,32) Sec. 321DEFINITIONS - II SUBCHAPTER ACT COSMETIC AND DRUG, FOOD, FEDERAL - 9 CHAPTER DRUGS AND

human body, including organ, blood and tissue donations, includingbody, and organ, human donations, tissue blood providing information by means of in vitro examination of specimens derived from pathological process or state, state, process or pathological physiolo a of or anatomy the of modification or replacement investigation, disability, disability, an for, compensation or of, alleviation treatment, monitoring, diagnosis, disease, disease, alleviat or treatment prognosis, prediction, monitoring, prevention, diagnosis, intended to affect structureoraffect oftheany function body intended to the of nedd o ue n h danss f ies o ohr odtos o i th in or conditions, other animals, in man or of disease, other or treatment,mitigation, or prevention or disease of diagnosis the in use for intended any supplement to them, to supplement any them, Formulary, United or Statesthe inPharmacopoe official recognized National the l Device Regulators Forum as “software intended to be used for one for used be to intended “software as RegulatorsForum Device l s”

(TheEuropeanandParlament the ofCouncil European the 14

mplant, in vitro reagent, vitro in mplant, 520(o).” accessory is: which man or other animals, man or otheranimals, or more medicalmore or cal action within (Title 21 - FOOD similar to the the to similar nuy or injury e assistede "device" ia or gical cure, e ion of ion being gical, gical, ia, ia, or the cal Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

(Gordon et al., 2020). Often, these non-regulated products are direct- are products non-regulated these Often, 2020). al., et (Gordon health for taskssimple automate or treatment), suggest not does (but disease patients help either that apps medical and software are exceptions such 201 Health, Radiological and Devices for Center Administration, Drug and Food Services, Human and Health of Department (U.S. developers aid to examples several where the FDA does not enforce requirements (exercises enforcement discreti are exempt from regulations. A recent policy by the FDA defines software an di of types few a only device, medical of definitions broad the on Based further growth was projected with 3.8 billion users by 2021 2021 by users billion 3.8 with projected growthwas further billion 3.2 reach to predicted were worldwide users smartphone of number Boar Executive Organization Health (World countries middle-income and u broad the their to due potential, large represent smartphones phones, in Mobile potential huge still (bankin is there digitalization. and finance lagging still including is it industries, changes, many resistance the to Due industries. movie transforming and music communication, and role increasing technologies wireless mobile the of that particularly technologies, Digital effect into on May 26, 2021 appearance of new technologies, e.g. the Medical Regulation Device (MDR) of t changin constantly are device medical a of class risk the determining rules demonstrate to needed evidence of level the and regulations with comply manufacturer how determine categories risk These subclasses. risk) (high III and IIb Medical devices are further classified based on their risk profile into c Working Group, SaMD (IMDRF 2013, 6). p. without purposes these perform that purposes related services (Cohen services et al., related d with patients providing while companies pharmaceutical traditional compa insurance the surpassing patients) i.e. consumer, to directly (sold to Class IIa or higher (Gathani &or Class(Gathani IIa higher Cowlishaw, 2019) to 2.2

I ndu st r y

ov erv ie w an results inresults the “up 2020) d pr . ospec t

15 - classification” Classof most

being part of a hardware medical device” medical hardware a of part being .

(O’Dea, 2020) (O’Dea, lass lass I (very low risk), IIa, to irect access to health- to access irect f elh ae o fas to care health of gital health products health gital -consumer products -consumer rnfrain by transformation I softwaremedical nies, clinicians and clinicians nies, d 142., 2017). The 2017). 142., d , with the majority the with , 9) self-manage their their self-manage g to adapt to the to adapt to g d d app categories, . As an example, example, an As . their value. The value. their se even in low- low- in even se r giig an gaining are by 2019, and 2019, by care providers providers care on) on) and brings he EU coming s must s g), t

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. possibilities for digital healthcompanies digital for possibilities system healthcare digitalized more even an for way the pave perception 2018 2018 in so did them of half nearly 2014, in use their reported consumers while apps: mobile/tablet health of use the in healt growth In continuous 2012). Bank, (World countries developing in living users the of artificial artificial intelligence in healthcare (Accenture, 2018). These changes in and telemedicine) e.g. (through care virtual use to woul willing increasingly consumers healthcare report, same the to According 2018). (Accenture, 33%) tri than more wearables using patients of number the 2014-2018, of Besides mobile phones, wearable devices represent large potential as well, durin Organization, 2020). Digital health was used as a response to the outbreak the to response a as used was health Digital 2020). Organization, implemented in an unprecedented technol pace and scale globally (Keesara health et al., 2020; digital where scene, a created pandemic COVID-19 the Finally, healthinsurance related bystatutory of healthcaretheprovision digital health solutions for the diagnosis, monitoring or treatment of diseases (DVG Gesetz (Digi Act Service Health Digital the is example recent most the Europe, In certifyindividually all. them but firm product. This way if the FDA finds a firm safe, the agency will not re certificatio This different from 2019). the regular as one, it is based on and the firm the devel (FDA, device medical a as software for program PreCert its in c a established(FDA) Administration Drug and Food U.S. the Additionally, delivery.initiatives care step and value-based toward important It services. health digital using for reimbursed get to professionals medical an physicians for conditions the establish codes These 2019). (Validic, services management treatment remote for and RPM) or monitoring patient (remote parameters phys of monitoring remote for codes CPT new three introduced (CMS.gov) Services Medi for Centres the USA the in example, For healthcare. of digitalization are important advances in the regulations and reimbursement polici and products health digital for consumers of appetite increased the Besides aid aid risk assessment, help public health with surveillance methods. It was ) .

)) accepted in 2019 in Germany, which allows the reimbursement of low-risk low-risk of reimbursement the allows which Germany, in 2019 in accepted .

(Schickert, 2019).(Schickert, gulate each of product the es pled (2014: 9% vs 2018: vs 9% (2014: pled only 16% of healthcare of 16% only

opers rather than their as well, promoting healthcare consumers’ also also used to mitigate e te eei of benefit the see ertification process ertification (48%) (Accenture, (48%) is also seen as an as seen also is cr, hr i a is there hcare, : tale-Versorgung- care & Medicaid & care or improvement to track people, track to services, there services, n oe new open and World Health g the periodg ge were ogies licensed d iologic be d is n the

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. have further great potential (MobiHealthNews, furthergreat 2020). have potential otherto emergingSimilar industries, Asian the digital healthrisingmarketand is investment in 2019 vs 2018 shows moderations, rather than contraction (Sean, contraction 2020) ratherthan vs in2019 shows investment 2018 moderations, oie elh ehoois icuig ps n waal tcnlge) n tel and technologies) Group,Capital (Mercom 2020) wearable and apps (including technologies health supp hea mobile decision clinical the and analytics within data products for (mainly into category goes management investment VC most the Recently accordin funding companies globally 4VC digital health Figure of pandemics the of effects the largest largest numbers The 2020). (Sean, USA the in invested was billion $7.4 which from (Mer in2019 billion $8.9 with funding the of majority the covered raised $10.6 billion in 2019 globally including VC funding, debt and hea digital and years 10 last the for steadily grown has investment This sector. Funding of digital health companies can be used as an indicator fo bodr taei ipeetto o dgtl elh n t te digi the to and health nowfor(Keesara2020)it remains et al., question a healthcare, digital of implementation strategic broader a c the to responses these Whether 2020). al., et(Keesara implementation the health digital certain of authorisation emergency with responded Regulators Health (World ePrescription and monitoring remote telemedicine, of – after 2018 - in the sector since 2010 (Fig. 4). According to RockHealth, t RockHealth, to According 4). (Fig. 2010 since sector the in - 2018 after by .

decreasing the need of in-person doctor visits with the help help the with visits doctor in-person of need the decreasing 17

g productcategories g to .

r the general interest thein . public public market funding: VCs com Capital Group, 2019) Group, Capital com Source: Source: Organization, 2020). 2020). Organization, (Sean, 2020) a rvlto of revolution tal t information lth risis will lead to lead will risis services to aid to services are the second the are r systems), ort expectedto lth start-ups start-ups lth

he lower he ehealth ehealth .

, Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. ensure, thatperformance ensure, meetspatients’, user’s requ(clinicians’, etc.) the in to bench from results verification of translation the is validation even more stakeholders (e.g. insurance, public health, healthcare organization healthcare health, public insurance, (e.g. stakeholders more even evcs r cmlx rcse t cvr l apcs Adtoal, hy r otn facing often are professionals they care clinicians/health and patients Additionally, aspects. all cover to processes complex are services th of validation the therefore sciences), engineering and behavioural science, disciplin distinct several of intersection the at is health Digital iue Eeet o vldto poes o dgtl elh pro health digital for process validation of https://www.flaticon.com/. Elements 5 Figure sili in or bench the (at developers by internally done typically is it and documents guidance en test/model/simulation a in development during performance of evaluation the and standards verificat general, several In 2020)). al., et (Goldsack of in 1 Table in is summary (comprehensive found be can p systems as clinical) management and (analytical validation and verification of Definitions (Sean, ofinvestors as 60% investors field, the the in 2019 repeat were becomin are investors health digital industry, the of maturity the Indicating

ahw e a. rpsd ht iia hat invtos hud be (Mathews5)(Fig. 2019) et al., aspects should innovations health digital that proposed t al. capture et To Mathews stakeholders. different the for validate to needed be might theref distinct, be may stakeholder each to health digital by delivered 2.3

Vali Technical dat Clinical ion of Privacy

dig

ita l healt h inno .

v at 18 ion

si multaneously, while claiming benefits for benefits claiming while multaneously,

ut. w ilsrto wt fe ios from icons free with illustration Own ducts. s lf sine computer science, (life es vivo and is intended to intended is and vivo

Economic 2020) ore dedicated studies dedicated ore

Usability evaluated from four four from evaluated r o te quality the of art irements. o. n contrast In co). . g specialized to specialized g i complexity, his

e products and products e s) . vironment The value The

ion is ion ,

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Health and Care Excellence, 2019) Excellence, Care and Health solutions(Nation new of value economic and clinical the ensure to standards Figure 6 Evidence tiers defined by NICE. Source: 6Evidence defined tiers Figure by Source: NICE. evidence an Care developed and Health for Institute National the UK, the In p.7.). 2019 al., et pat real for thatwork products impactful produ different between discrimination versus(private Recentlypublic).system the Digital Scorecard wasHealth p insuran health and classes) risk (see product the of profile risk EU), in GDPR (e.g. geo on depend requirements regulated and aspects different the regulate differen validation, comprehensive a such for requirement formal no is There ratio. and operators (clinicians, doctors, nurses), while economic validation explo of perspectives the from experience; user the on focuses usability of Testing com potentially and workflow clinical existing the into fits it answersClinicalvalidation assessment. whetherthe improvesproduct clin to do. An important part of the technical validation is data security, privac it what does product the whether answer would validation Technical elh rdcs a dvlpd n cnetd hs o h lvl f evidence of level InstituteCareand Healthfor Exc value 6)(National their demonstrate (Fig. the to this connected and developed was products health

tnad faeok o dgtl elh ehoois ht provide that technologies health digital for framework standards . (National Institute for Health(National Institutefor andp. 2019, 7) Care Excellence, For this, a pragmatic, functional classification of digital of classification functional pragmatic, a this, For ients, providers, and healthcare systems” healthcare and providers, ients, t ad uh dgtl elh opne t build to companies health “digital push and cts 19

ae i t eitn solutions. existing to it pares is expected/promised is y and confidentiality ellence, 2019) ellence, ical outcome,ical how roposed thetoaid res the cost-value res the xelne (NICE) Excellence graphic location graphic al Institute for Institute al both patients patients both institutes t

ce systems ce edd to needed (Mathews .

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. DHI DHI contributes to the enhancement of an individual’s health behaviour/ only criteria evaluation that found authors the Interestingly, 2019). al., et (Kowatsch intervention publish criteria evaluation the consolidate to publication 36 reviewed Kowat identified. was frameworks evaluation of aspect interesting an Recently (none of them comprehensive enough for health technology assessment) (Moshi to frameworks 45 identified review recent a products: health digital of Additionally, there is a plethora of frameworks developed with the aim ( onartificialITU/WHO Focus Group intthe Evaluation AI/MLClinical by on factors: factors: produc health digital of validation (systematic) in interest growing The aspect. includethis to agencies, therefore, other frameworks developed by scientificthe community does b evaluated strictly is it that argue, can one however criterium, neglected most aesthe adherence, finding, their is interesting accountability, more Even 2). (Appendix security, benefit perceived and privacy quality, content use, a developed produc guidelines mHealth (Working Group on mHealth assessment guidelines, of 2017) assessment assessment the for mHealth guidelines existing on of overview Group comprehensive Working the mHealth, For et al., 2019, p. 256)) and it was only the 8 the only was it and 256)) p. 2019, al., et es a oeve o vldtd rdcs Chn t l, 00 VnWnl e Winkle Van 2020; al., et (Cohen products validated of overview an least digital on information systematic provide to try organizations profit) products.V health compare different digital assess and the could etc.) (patient customers healthcare the where and collected are assessments broad where source public no isthere standards, accepted of lack the to Similar (Rowlandguidelines of et developed arevalidation inexamples mHealth for ou Gop on Group Focus a)

to the contribution of the technology to the health and well-b and health the to technology the of contribution the to accord appropriate be must resources human) and (financial of Allocation people. care resources and how the useinappropriate of these resources can lead to na limited the on light clear a shed pandemic Coronavirus current The on the awareness resources limited inIncreasing healthcare Atfca Itliec fr Health,” for Intelligence “Artificial 17 was related to effectiveness (defined as “the degree to which the which to degree “the as (defined effectiveness to related was 17 th in the ranking of evaluation criteria after ease of ease after criteria evaluation of ranking the in 20

.. Wead t l, 09. Further 2019). al., et Wiegand n.d.; . There There is plan for a Working Group arious (mainly not-for- (mainly arious eing of the humanity. the of eing health products or at or products health to aid the assessment condition” ed for digital health digital for ed that safety was the was safety that ts is due to several to due is ts elligence for healthfor elligence assess health apps apps health assess the results of such of results the al., al., 2020). among the 331 the among ture of health health of ture a. 2019) al., t et al., 2018). y regulatory y s, clinicians, s, (Kowatsch sch et al. al. et sch not not need is and tics death of ing ts . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Patient-Assessment-Instruments/Value-Based-Programs/Value-Based-Programs 1

The actual value-based programs are described at at described are programs value-based Theactual c) b)

s o dgtl elh rdcs a ps t ptet’ elh Teeoe t Therefore, health. patients’ to pose may products health digital of use t about likely is concern main their harm; do to not vowed Clinicians expectation Consumer important An innovators. of products. these can strategy be differentiation validation (broad) the health digital for difficult increasingly different product that suggest, and options the with overwhelmed o and patients that demonstrate numbers These 2017). Science, Data H for report Institute (IQVIA samedevices wearable consumer The different 340 over 2017). counted Science, Data Human conditi health for on Institute focused (IQVIA apps management those of 40% with 2017 in mark 300,000 the health and fitness apps available in the Google Play and the Apple A rapidly. growing are solutions health digital available of number The amount newOverwhelming digital of healthproducts solutions. wit metrics quality specific of improvement the demonstrate and on focus to also puts innovations, pressu it bigcatalyst bea for health can digital p. i.) p. and integration appropriate th promote and evidence generate to necessary is eHealth of evaluation rigorous Eval eHealth Bellagio WHO the of statement consensus the to According demonstrated. be must resources), up free consequently (and efficiency care health enhancing hype and hype shou investment public approaches and digital to approaches non-digital from diverted inappropriately private of allocation the Additionally, introduced introduced 7 value-based programs in the last decade C metrics. quality on based providers care health reimburses and patients p service of quantity the over quality of increase the promotes model this healt in allocation resource the change to expected is model care Value-based Wh l tee s cness ta eelh n dgtl elh a lre potent large has health digital and eHealth that consensus, a is there ile .

a large,promise. unproven but ain ru, T ipoe elh n reduc and health improve “To Group, uation

use of technologies.” of use https://www.cms.gov/Medicare/Quality-Initiatives- 21

(World Health Organization, 2019a Organization, Health (World 1 . While value-based care model (last accessed 02.05.2020) 02.05.2020) accessed (last hat inequalities, health e ppStore surpassed iation is becoming is iation re oninnovators re he risks that the that risks he n valuable and The number of number The eaos are perators this promise this d o be not ld rovided to rovided MS have MS due to a to due h their h hcare; uman uman a in ial on he e Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. methods, validation specialtestbeds. and the on a ofproducts form economic and clinical the on details further provide will I Below, phases is in Appendix 3 is inAppendix phases dig and classical of comparison and overview An solutions. existing with digital product would be further refined to increase adoption, integratio implementa actual the be would study 4 phase the of analogue The usage. on assessingofefficacy the in thesolution target users and potentialfinancialb equivalent3 phase The ofefficacy. indication usabilityassessand to users feasi scale large involve could study 2 phase a of analogue The features. i to tested being is which prototype, ainvolve wouldstudy 1 phase su In 2018). al., et (Sheon health digital of validation clinical the NODE.Hea by proposed were testing clinical staged classical the of analogues The scale. effect and costs safety, monitor to studies observational post-marketing are compared between the two groups to find statistically significant d o measured the study, the of end the At control). as solutions alternative (or are randomly assigned to a group using the new drug/device or a group getting stan a (RC trialscontrolled randomized usually are studies 3 Phase distribution). (e.g. monitored is body the in drug/device the of behaviour the (typicall population target the in tested is safety the studies, 2 phase In In 1phases. phase studies, is and safety testedtoxicity small,ina usually h produc development, clinical device medical and drug classical the In 2.

3.1.

expected by users where health related data are involved by expected users where healtharerelated datainvolved protection data stringent more even an hand other the on i regions, geographic requirement legal a is rules protection data with compliance the hand the protection of sensitive data from both the operators and the p awareness increasing an is there general In methods. standard gold to compared cost-bene favourable a and workflow the into solution of the integration easy u to them for solutions. On essential the other hand, hospitals, as operators expect the demon is efficacy and safety clinical of demonstration C lini c al validation .

(V validation of digital health digital of validation ifferences. Phase studies4 mmary, the equivalent of equivalent the mmary, dentify use cases, missingcases, use dentify ayena 2018) ayena et al., ts go through 4 clinical clinical 4 through go ts mnsrto, dosage, dministration, n and interoperability y sick patients), and patients), sick y bility testing in end end in testing bility atients side; on one Ts), where patients where Ts), ital clinical testing clinical ital ealthy population. ealthy study wouldfocus study iveness on large on iveness tion where the the where tion stration of the enefitsof the utcomes are utcomes dard care lth to aid to lth certain n e these se fit ratio fit .

on is is Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. validation may vary (Rowland et al., etal., may2020)(Rowland vary validation quali the however needed, is validation clinical that means this cases 2020 FDA, 2016; Commission, (European certification device medical the achieve da performance of validation and verification product territories, both In manufacturer.withthe lies body”, i body”, organization appointed by the competent authorities to conduct such as such conduct to authorities competent the by appointed organization or Conformity) of Declaration as devices medical I Class of case (in itself manufacturer and a premarket approval application (PMA) for Class III device (FDA, (FDA, 2020a) (PMA) a for premarket Class IIIdevice and approval application In the US, the FDA requires a 510(k) application for medical device i 2.3.1.1. On different through happen can health digital of validation Clinical the advantages and challenges of each design are summarized in (Coly 2017) & Parry, of challenges and advantages each are the indesign summarized o scope not is methods study the of comparison The 2.3.3. Chapter in discussed which testbed, the is experiments natural of form special A 2020). Jandoo, & (Coly experiment) natural called (also design study observational or designs qu like used, be can designs study innovative more interventions, health

th th that is territories other and regulation EU the between adifference without marketed be cannot device medical a (MDR), regulation device rec more the and (MDD) directive device medical the to according EU, the In belonging regulatory theto belonging categoryof “medical h digital to apply regulations specific regulations, advertisement the Besides Compar ofa digital thatshou any claims healthproduct health-related imply and Misleading ru general These the 2017). Wójcik, Anna & Arak and (Piotr (2006/114/EC) Directive Advertising (UCPD) Directive Practices Commercial Unfair (Federal Trade Commission, n.d., 2013) n.d., strongly Commission, Trade (Federal (FTC) Commission Trade Federal health-rel the of case in advertisement in US claims proven for need the emphasizes the In services. or product a about Regulations by trade commissions prohibit deceptive acts including e product by its manufacturer based on a conformity assessment either performed by t by performed either assessment conformity a on based manufacturer its by product e one hand, a classical randomized controlled trial (RCT) can b

n case of Class II-III devices). All liability regarding the product conformity ulti conformity product the regarding liability All devices). II-III Class of case n Digital health products classified as medical devices classified products asmedical Digital health devices . The EU advertising standards are described in the in described are standards advertising EU The .

device”. device”.

e used. However, for complex ways (Coly & Parry, 2017) Parry, & (Coly ways n risk category Class I or II false or misleading claims ld be proven. ld be proven. ty requirements of that of requirements ty e CE mark is affixed to affixed is mark CE e sess ta are necessary to necessary are ta asi-experimental asi-experimental . ment (“notified ment

will be further further be will ealth products ealth E ak The mark. CE ated products ated Parry, 2017; Parry, f this thesis, this f n medical ent b). In most In b). . mately by an by

ative les he . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Figure 7 Comprehensive clinical evaluation of SaMD proposed by proposed SaMD of evaluation clinical Comprehensive 7 Figure and (premarket) product the du undertaken is evaluation clinical the devices, medical of case In “e of evaluation clinical the for principles developed IMDRF the example, an are thetoaiddigital guidelines of clinical of developed evaluation design hea regulations. realistic and clear for stakeholders all from builds p and arise technologies new as evolving continuously is landscape regulatory The Commission,et (Europeanal., 2016). 2016) (Murray alternative product can be used for the new product as well and n dat validation clinical the demonstrating equivalence, biological) maybe after (and technical case, this In equivalence). substantial (or equivalence developer the purpose, same the for validated already been has product similar 2016). Commission, (European reporting) and surveillance (post-market product evaluation proce evaluation c systematic a proposes document The 7). p. 2017, Group, Working Device Medical SaMD” of performance and effectiveness safety, the demonstrating that the high level of connectivity in SaMD provides a unique opportuni unique a provides SaMD in connectivity of level high the that Software asaMedical Workingp. Device 2017, 4) Group, o otnosy oio is aey n efciees n mdf t achi to modify and effectiveness and inreal-world. performance safety its monitor continuously to stablish a common and converged understanding of clinical evaluation clinical of understanding converged and common a stablish analysis, clinical trials) guidelines,data professional condition (literature, the targeted clinical the SaMD andoutput associationbetween Show valid clinical Clinical Clinical Association ss, where clinical validation is the last step (Fig. 7). It strongly It 7). (Fig. step last the is validation clinical where

– as an ongoing process process ongoing an as Clinical Evaluati Clinical precision) (accuracy, reliability, generation process Prove correct data Analytical/Technical Validation

the IMDRF. Own illustration based on figure in in figure on based illustration Own IMDRF. the on

– hogot h lf cce f the of cycle life the throughout specificity, odds ratio) population (sensitivity, purposein the target achieve the intended thatVerify dataoutput o repeated testing is needed o seii css specific cases, specific For Clinical Validation Clinical ring the development of development the ring IDF otae s a as Software (IMDRF ty to the developer the to ty a generated for the for generated a and principles for principles and SaMD in 2017 to 2017 in SaMD lth products.Aslth v highest eve h clinical, the emphasizes s can claim can s If a highly a If

ressure ressure linical linical (IMDR

s F Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. 2018). et al., Velthoven devices”) medical to management risk of “Application 14971:2007 ISO and purposes” software 13485:2016 “Medical devices— “Medical 13485:2016 tools, as cited in (Cohen et al., 2020 p. e164) p. 2020 al., et (Cohen in cited as tools, men novel and devices, neurostimulation monitors, rhythm heart (e.g. products non-regulated about concerns ethical and quality safety, growing are there demonstrate value on outcome, cost or access to care (Safavi et al., 2019). On the other comp that found company health digital US-based top-funded 20 the 2020) al., et (Gordon see overview an (for quality low cases, those in even and setting clinical in tested were which apps), health digital non-regulated of percentage low a is there Currently 2.3.1.2. device in products health digital for applicable standards several has (ISO) Besides the territorial regulatory agencies, the International Organization for Sta (Cerretaet stageal atan early agencieswith even of development developers the interact successful allow to openness an promotes FDA (EMA) Agency the Medicines both European technologies, emerging for unclear validation remain clinical may and for periodically requirements and landscape regulatory the While published, high quality clinical data is perhaps the biggest co biggest the perhaps is data clinical quality high published, hl ciia vldto o nnmdcl eie i aprnl sti advantages apparently is devices non-medical of validation clinical While companies”health related digital to - -

: Differentiation: Creation Creation of trus products with this unique selling point (USP). (USP). sellingpointuniquewith this products and using for marketing purpose to differentiate among the overwhelming scale data scale

the —Software life cycle processes”) cycle life —Software :

tnad n otae o mdcl eie (E 63420 “eia device “Medical 62304:2006 (IEC devices medical for software on standard Digital health products outside of the category Digital productsofhealth outside devices of medical ba ses and customer consent in data sharing (Vayena etal., sharingin data 2018) ses consent customer and t:

separation of legit offerings from “snake oil” “snake from offerings legit of separation public trust is particularly important, if there is a need to bui Quality management systems management Quality (Cohen et(Cohenal., p. e174). 2020 and the main standards for medical devices (ISO devices medical for standards main the and 25 ).

Accord ). Similarly, a recent study focusing on focusing study recent a Similarly, ing to Cohen et al., “ al., et Cohen to ing — ntributor to th to ntributor Requirements for regulatory for Requirements the category of medical of category the l ae i hs potential has it rare, ll products (mainly health health (mainly products evidence tends to be of be to tends evidence

anies have yet failed to failed yet have anies (M athews et al., 2019) al., et athews e r changing are The paucity of paucity The iia health digital potential risk potential ndardization ., 2020). ., 2020). number number of a health tal ld ld large- o of ion the d hand, (van Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. 3 2 d validation: clinical of management and planning the with challenges major are there above), (s advantages several has process validation meaningful a and bringin market the for to step products necessary a is validation clinical cases most in Although 2.3.1.3.

https://www.nhs.uk/apps-library/ http://www.rankedhealth.com/ ------

2016) conditions health chronic typically complex, on focusing Products systematic and complex testing of healthcare innovation and of systematic complex testing healthcare innovation culture: of Clashes Health collection Devices Curated Health Ranked e.g. products, “validated” of collection in get to possibility the offers validation clinical 2019)), al., et (Mathews in found be can initiatives such for examples of collection representative Accreditation: opportunities, user innovation) opportunities, user (potential patients clinicians, with dialog building efficacy, higher possibilities: improvement Iterative Lack of specialists with knowledge bridgingwithknowledge specialists both Lackof worlds on hypertension) on Prove the “obvious”: Financial challenges challenges Financial health products. health among product developers, and at regulatory agencies and investors evaluatin mi It needed. is differentdisciplines from specialists of collaboration development risk,ifisclinical testing needed development risk: development Increased (long) clinical testing clinical (long) team multidisciplinary for Need

: long term follow-up and methods complex read-out longterm follow-up Challenges of clinicalofvalidation digitalhealthof Challenges products eea iiitvs fe aceiain ae o seii ciei (a criteria specific on based accreditation offer initiatives several 2 or or the Library Apps NHS atpcd xeietto o dgtl opne v. careful, vs. companies digital of experimentation fast-paced last accessed 03.05.2020. 03.05.2020. accessed last test and prove well-accepted dogmas (e.g. effect of regular sport (Rowland et (Rowland last accessed on 03.05.2020. 03.05.2020. on accessed last

compared to typical IT products, there is a much higher much a is there products, IT typical to compared al : ., 2020) ., o mliicpiay rdcs a dgtl health, digital as products, multidisciplinary for better integration into the healthcare system, healthcare the into integration better 26

3 : typical IT start- IT typical

(Goldsack et al., 2020) mi up ghtneededbe /SME budget vs. costly vs. budget /SME ght be necessary beght o dniy new identify to dgtl health digital g (Murray et al., al., et (Murray to a curated a to gtl health igital Table 1 of of 1 Table ps and Apps g digital

ee Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. assessment, defined as defined assessment, throug happen can services and products health of validation Economic reimbursement). fra clear (creating regulators and companies insurance makers, policy by as well deployment complete evi economic of the (generation developers the by action urgent for calls This eHealth. for barriers key the among are schemes) reimbursement (particularly framework legal inadequate/fragmented the and solutions eHealth 2012 Plan Action evidence eHealth large-scale limited that 2012) Commission, (European its in highlighted Commission European the However, ultimately leading to decreased costs or at least higher value for same cost same for costsleading decreased ortovalue atleast ultimately higher (O’Rourke et al., 2020, et (O’Rourkep. 2) decision-making in order to promote an equitable, efficient, and high and efficient, equitable, an promote to order in decision-making lifecyc its in points different at technology health a of value the

health can manage the whole patient pathway through health care wi care health through pathway patient whole the manage can health or Addition monitoring. distant eliminatinge.g. through care of delivery of costs in the decreasing potential large has SubGr health (eHSG Digital care of 2019). delivery Reimbursement, the on is spending care health of ~77% EU, the In 2.

3.2. - - - -

Learningalgorithms: with smartphoneoperation systems) with may go through several updates to remain usable (see health apps staying compatib clini lengthy a of completion the of time the by hand, other the upgrades re-testing of extent the and need the define must companies different rules could apply to different parts of a product ofa parts different apply could rules different to products: Complex Continuously evolving standards and fragmented landscape of regu Fast changes Fast Ec mak agencies regulatory different as well (Ferretti2019). et al., difficult compliance as requirements, global used and easily guidelines be can apps) health (particularly products health onom ic

valida of tion and current and emergence of new technologies new of emergence and current a “ multidisciplinary process that uses explicit methods to determine to methods explicit uses that process multidisciplinary rdcs a ft eea ctgre o rgltos therefore regulations, of categories several fit may products . how to test continuously changing algorithm changing tocontinuously how test r e imburs e m e 27 nt

le. The purpose is to inform to is purpose The le. on the cost-effectiveness of cost-effectiveness the - (Murray et al., 2016) al., et (Murray quality health system” health quality th higher efficiency higher th h health technology health h cal trial, technology technology trial, cal . lations:

/updates. On /updates. significantly significantly y multiple ly, mework for mework the lack of lack the ally, digital ally, dence) as dence) as digital u on oup -2020 of le e : ,

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. reimbursement (Bergmo, 2015). reimbursement whe standard gold the considered are they and solutions different of gains hea comparing in effective are QALYs (QALYs). years life quality-adjusted as valued and The most commonly used method is CUA, where the outcom 1Overview for methods of economicTable partial evaluation and full altern care health in evaluation economic defines al et Drummond Th Drummond, 2005) Drummond, provided in Table 1 Table in provided methods evaluation economic traditional of overview An 2015). (Bergmo, demonstrated c in - value their and well, as health digital for used already been int care health for developed methods evaluation economic traditional The collection dataclinical information from (i.e. direct trial) or mod from can value/benefit and costs on data evaluation, economic For 2019). (Rahimi, a in ultimately and solution the of use frequent more in results broaderpopulat for access the increase solution) diagnostic (e.g. solution cheaper poi Rahimi of comment recent a sources; unexpected highly from appear technolo health of case In opportunities. missed of costs potential as well as comparison. comparison. alternative no when evaluation, partial a or compared are solutions consequences of all resources needed for the forimplementation, the use an th evaluate to needs one intervention the of price actual the beyond costs, Among canclinical2.3.1.Chapter (see the trials be used efficacy/benefit For testing, and (i.e. consequencesof the technology the costs efficacy, benefits)of it. Cost minimization analysis (CMA) minimization analysis (CMA) Cost analysisconsequence Cost (CCA) (CBA)benefit analysis Cost (CUA) utility analysis Cost analysiseffectiveness Cost (CEA) economicevaluation Full ese definitions imply, that for a proper economic evaluation, on evaluation, economic proper a for that imply, definitions ese tv cuss f cin n em o bt ter ot ad consequences” and costs their both of terms in action of courses ative . .

The evaluation can be a full evaluation, where two or more alternat more or two where evaluation, full a be can evaluation The 28 analysis outcome description Cost analysis description Cost analysis Costing evaluationeconomic Partial

e is measured as "healthy years" “as a comparative analysis of analysis comparative a “as net rise in total expenditure total in rise net e has to analyse both the the both analyse to has e ertain cases - has been been has - cases ertain solution is available for available is solution elling. elling. td u, ht a that out, nted ). erventions have erventions d the follow-

n deciding on on deciding n gies cost may cost gies derive from derive ion, which ion, M E. (M. e cost e up ive lth is , Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. design is natural experiment (Craig et al., 2017), or a special form of that, ofspecialform a or 2017), al., (Craig et naturalexperiment is design Chapter 2.3.3.). In a natural experiment effectiveness and cost can be a be can cost and effectiveness experiment natural a In 2.3.3.). Chapter complex evaluation meth non-linear value, on setting between intervention and input output). For such complex digital hea of influence (strong pathway causal complex (influencing contagious” “socially th well), as members family of health the influencing (e.g. complex is outcome thei improvements), (updates, time over change to likely are They 2016). al., et (McNamee c considered be can solutions health digital most hand, other the On and limit of virus doctorsthe and patientsmay spread the reduce 8). 8). differe the at critical are steps which on guidance provide and economi the for process stage-based a proposed they Furthermore, 2017). al., as solutions health digital other for used be can which mHealth, for method t right the of selection guide to flow-chart a developed al et LeFevre 2018). (ABM)(J Chen, agent-based through achieved be can which needed, be may modelling complex comp the to Similar present. are interactions complex these where world, telemedicine was clearly demonstrated: the reduction in face- in reduction the demonstrated: clearly was telemedicine On one hand, digital health often generate value, which cannot be expresse di wh limitations severe have and procedures medical or devices medical products, pharma for developed were methods evaluation economic traditional the However, the current COVID-19 pandemic, a previously unrecogni previously a pandemic, COVID-19 current the et (McNamee professionals care health by saved time in result or time, waiting redu information, and care health to access improve often they example, For QALY. or gital health. gitalhealth. od s was proposed (McNamee et al., 2016). Such an alternative study h ue f h itreto b ohr) n te have they and others) by intervention the of use the 29

ze or at least undervalued benefit of benefit undervalued least at or nt stages of development (Fig. development of stages nt to the epidemic. -face interaction between interaction -face lth products, the use of omplex interventions omplex aiinl evaluation raditional ssessed in the re the in ssessed called testbed (secalledtestbed d lex study design, study lex as health benefit well(LeFevre et well(LeFevre relationship al., 2016). In 2016). al., c evaluation c en used for used en y a be can ey modelling ceutical al ce e r - Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. builds on 4 pillars. builds 2019). Reimbursement, on SubGroup (eHSG EU the in reimbursement fair through facilitat healt to digital useful plan economically and action clinically of an implementation developed eHSG the of Subgroup Reimbursement The mining, data and improvement and of technologies. innovation advertisement c reimbursement through of lack the Third, sources performance. and privacy revenue compromising alternative on rely may a the provide even (or low price keep to Second, cannot. who those, need to charge patients, it can result in divide between patients w developers if First, 2020). al., et (Gordon phenomena harmful several to lead to shown was digitalfor health toreach its full potential.Inof case mobil ele key a as seen is Reimbursement reimbursement. towards path clear a importantly standards/guidel clear developersneed validation, economic meaningful a For differen the for recommended methods evaluation Economic 8 Figure

p. 13) 3) 2) 1)

products and solutions. and solutions. products evidence digital for European onguidelinesDevelop generate to how relevant national level. national on innovation investment Setting up digitalfunds for healthproducts health products and solutions. health products and makespecificcriteria appropriatereimbursementdecisions to Develop digitalfor 30

t stages of development. Source: development. of stages t e apps, the lackof reimbursement ho can afford apps and the pout ad solutions and products h pp for free) developers free) for pp anEU on and (LeFevre et al., 2017, al., et(LeFevre n discourage an ines and most and ines The plan plan The health health e the the e ment

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Appendix 4 Appendix c EU in solutions and products health digital of reimbursement of Examples In a broad sense, testbed sense,testbed broada In 2.

anr i w in manner) t in conditions real-world, to close or real-world, in innovation real- for definition ( development in technology new or simultaneously (NHS England, 2015) 2015) England, (NHS simultaneously hospitals) or cities whole real where study, observational an for used is expression the care, health In testi it allow to (closereal-world retain environment the but to) the the reduce not do testbeds real-world environment, simulated or laboratories existing working practices and systems servicesofour practicesworking and existing health partners) partners) to work together at all stages from ideation until scale- 2017) (Rob Living Independent and Healthcare for Lab Living as exists concept similar A lab”. “living invtr, elh ae rfsinl, ains n patients, professionals, multipl for care platform a health provide (innovators, Testbeds 5). p. 2018, Health, Global for The relation of testbeds to other testing methods is shown in Figure 9 relationother testbeds of methodsThe to testing shown inis Figure Living Labsalongwith of beds. test for used the evaluation ofcomplex inthis digital health products, both As scaled. and improved tested, be can solutions market-ready where more are beds test while development, during early feedback user incorporate 3.3. 4)

. Based. Living Labs areon thedefinitions spacesrather co-design wher stakeholders. stakeholders. solutionshealthand valuep rewarding theproducts thatdigital fitting instrumentsspecific digital assessmento Develop thehealth for and R e al . -

wor

ih hy il e sd r operated.” or used be will they hich ld testbe ol tsbd a “otold r one evrnet fr testing for environments bounded or “Controlled as testbeds world d (o

as r test bed) is any device, facility or platform to test a theory, tooltheory,a ortest toplatform facility any device, is bed)r test r ue fr vlaig utpe cmiaoil innovations ‘combinatorial’ multiple evaluating for used are

a t oo l

for “ ete - Wikipedia - Testbed

digital hea o vlae oh nerto ad mat ihn the within impact and integration both evaluate to 31

lth

valida Anzn t l, 09 p 6. Unlike 6). p. 2019, al., et (Arntzen ot frpoi ognzto, industry organization, -for-profit tion n.d.) , ” (CHF and The George Institute George (CHFThe and

up he manner (or close to the to close (or manner he thesis I thesis analysethe use will . et poie a specific a provides Nesta in a continuously evolving ng and adaptation. ng adaptation. and .

rovide for all -world sites (e.g. sites -world e innovatorse can concepts can be can concepts stakeholders e ountries are in are ountries complexity of complexity t and and t ert Picard, ert upstream,

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. readiness of tested innovations. Source: innovations. Source: tested of readiness Figure 9 The relationship of the different testing methods based on the

cities, regions).summarized As cities, by th where location, the for valuable highly are testbeds Additionally, 15) The use of testbeds for innovation testing provide numerous (Arntzbenefits : - - - - -

Focusing and attracting investment and resources in innovation in specific specific in strengthen competitiveadvantage. and innovation in resources develop to seeking is area local the where and areas research and sectors, technologies, investment attracting and Focusing which is necessary for developing an innovation-driven growth necessarydeveloping forecosystem. is which aninnovation-driven structured and clear a within organisations research-intensive other innovati real-w the allowing by innovation, into investment encourage they ethical context ethical comp in technologies for also experimentation risk, of management the allows user groups, physical environment and specialregulations and groups, physicalenvironment user infrastructu special providing by testing, to innovators of access the increase “ teghnn claoain ewe te ulc etr bsns, universities business, sector, public the between collaboration Strengthening on

(Arntzen etp. 2019, al., 16) Ar ntzen et al., they help these places with: ethelp al.,ntzentheyplaces these 32

level level of control in the environment and the technology ey are established (e.g. established are ey en et al., 2019, p. rd etn of testing orld framework, re with re

and lex Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. et al., 2019, 8) et p. Institute for Global Health, 2018) (NHS England, 2015) (EIT Health,(NHS (EIT n.d.) England,2015) Health, Global for Institute 2018) health digital of impact real-world the evaluating on focused largely three Below the large testbed are analysed. programs further (Schickert 2019). et al., period with evidence such develop to need and provision care health on impact environments aretheyenvironments intended for wh differently behave or fail could they but conditions, laboratory in innovation, innovation, like most digital health products. usef particularly be can testbeds consequently and testing, Real-world be be reimbursed by statutory health insurance d of health can be seen as a national scale test bed: registered low scheme risk digital reimbursement German the in developments recent the Interestingly, Programme ( programs testbed 7 in patients 4000 over wave the announced NHS The 2.3.3.1.

------

Ma to app systemic more a enables that policy innovation for framework a Providing assets such as sectordata. assets public of and locally, done research of value and potential economic the Maximising mrvn te eiey f o rdcn te ead o) ulc evcs thro services public for) regulatory right the and policy, systems. creating governance demand the reducing (or of delivery the Improving a supportive and enlightened local innovation ecosystem. local and supportive enlightened a knowledge-inten develop and invest confidence area the in firms existing and to investors potential giving functions, place good a as area an Promoting and supporting them to develop to an investment-ready stage.supporting antoinvestment-ready them and develop to ch policy and regulatory influence fail, iterate, to them for space safe a De

evaluation of effectiveness. evaluation effectiveness. of -risking the process of developing new products and processes for firms, p firms,for processes and products new developingof process the -risking king better use of publicly available infrastructure. available betterof publicly king use , n.d.). The wave 2 was started in 2018 with 7 sites selected: 4 funded by th by funded 4 selected: sites 7 with 2018 in started was 2 wave The n.d.). , National Health Service (NHS) England testEngland Health(NHS) bed programsService National . Not surprising, that the three large scale testbed projects in in healthcareprojects scalelarge the three testbed surprising,thatNot 1 of its testbed projects in 2016 with 51 digital technologies digital 51 with 2016 in projects testbed its of – often complex, adaptive, real- , NHS England Launch Second Phase of Test Bed Test of Phase Second Launch England NHS As phrased by Nesta, “Innovati even without pre-existing evidence on a positive 33

” (Arntzen etal.,(Arntzen 2019, 21) p. world systems” world en introduced into the into introduced en (CHF and The George The and (CHF ul in case of complex of case in ul . health solutions can

in 12 month of test of month 12 in on may work wel ange needed, needed, ange (Arntzen roviding roviding there is there roach other other igital sive ugh are e l l

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. tool, tool, NHS testbeds bring together NHS, academia, industry, patient groups as with organizationalchanges maximizetoAs benefits and minimizecosts. bu care), existing of top on (and isolation in not innovations digital evaluate sr a a iia o lwr cost lower or similar a at users across the health and care system is most likely to improve outcomes for patients at innovations digital of uptake the how and where about increased decisions inform diagnosis, early empowerment, patient accessibility and more effective and efficient care. promoting The services and tools test STRATEGY 15 sites throughout Australia ( testbed $8.5million a started (ADHA) AgencyHealth AustralianDigital The 2.3.3.2. 2018b)( England, (NHS Beds Test Collaborative » England NHS the by 3 and government (NHS England, 2018a):(NHS England, t summarize I Here (NHS 2018a) England, alsofirst provided the wave key aboutleanings method the itself theWhile aimmain of NHSthe testbed program is toevaluate and validate ucms ( outcomes 2017 in STRATEGY - - -

- -

Set realistic budgetrealisticSet (e.g. and speed realistic requirement oftime pati a consensus and manage expectations and consensus expectations managea stakeh individual the of goals and purpose the identifying of Importance specialist) governanc information and data and th representatives patient in services, of involved delivery people project, the of designer/implementer (sponsors, process of theearlyImportance whoof all on-boardingareparties, needed for Define cleargovernance (including roleof advisory Define board) arrangement and keep communication channels keep open communication throughout the and projects (inearly collection data settin and the Set up sharing platforms/processes

ht ecie “iia hat a ky o mrvn srie eiey n health and delivery service improving to key as health “digital describes that , n.d.). The n.d.). , , n.d.) , utai’ Ntoa Dgtl elh taey NTOA DIGI NATIONAL | Strategy Health Digital National Australia’s Test bed systems in Australia inbedsystems TestAustralia ” . The testbeds focus on chronic care, emergency care, end of life care and care life of end care, emergency care, chronic on focus testbeds The . he se key learnings of the wave 1 related to the whole testbed ecosystem ecosystem testbed whole the to related 1 wave the of learnings key testbeds are part of the National Digital Health Strategy announced Strategy Health Digital National the of part are testbeds , n.d.) , Digital Digital Health Test Beds Program | NATIONAL DIGITAL HEALTH . These testbeds focus on long-term or chronic conditions and orThese chronicfocus testbeds and on long-term conditions ” NS nln, 08 p 1) Ipraty tee testbeds these Importantly, 1.). p. 2018a England, (NHS 34

overall aim is to generate “evidence to H Aclrtd Access Accelerated NHS amulti-stakeholder project in 2018 atin2018 project new innovations, ent recruitment) t in combination in t well well as charities scale and pace pace and scale the evaluation A HEALTH TAL olders, build olders, g up phase)up g and service . e e .

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. M until open were requests funding projectslaterbeavailabletheinformation on might year. this for calls The program. the join to of digital health, the following challenges identified:were the challenges following health, digital of too as testbeds on experience published the and literature the on Based 2.3.3.4. from EIT f financial own their with join can Organizations scale-up. until biotech and supports all stage of development from co-creation with users MedTech of field the on product/service ready market a with innovators) Labs ( netwo existing an leverage to seems program The n.d.). Health, (EIT Health EIT of Beds Test and Labs Living the on available information limited is There 2.3.3.3. 2018) Institut Health, George The and (CHF Australia of Forum Health Consumers the and residential aging as proposed by a recommendation of The George Institute for Glo eod ytm ptet eod ytm ad mhszd h ne fr sophisticat for need the emphasized and al., (Biggs 2019). measureset and methodology scaling for up system) record (patient system Record testbeds, the ADHA developed a framework for the complex evaluation of e.g. the organisations” other and government industry, p viable and sustainable through cooperatively delivered and instigated multi- a building elhae hlegs. h program The challenges”. healthcare on cooperate patients, and professionals healthcare including stakeholders, -

Forum LLSA > Accueil Timeframe: program digital health solutions typically change fast, therefore during a 2-year-p a during therefore fast, change typically solutions health digital th On used. be to proposed were projection and results interim effects During this time, short- and medium-term effects could be assessed and f

. Additionally, EIT Health invites testbeds and living labs outside of it . Challenges of real-world Challenges of testbeds Institute ofInnovation European Technologyand (EIT)Accelerator Health iia t te H Egad ete poet te utain ytm s also is system Australian the project, testbed England NHS the to Similar stakeholder platform to assess “new digital “new assess to platform stakeholder the published digital health testbed projects run for a min , n.d.) in Europe to provide an “open innovation ecosystems where invit es 35 n Erpa ognzto (start- organization European any

(Biggs et al., 2019 p. s11.). Along with the with Along s11.). p. 2019 al., et (Biggs -enabled models of care that are that care of models -enabled und or can request funding request can or und Accelerator program Accelerator artnerships betweenartnerships ls for the evaluation the for ls through through validation s current network , digital health or health digital , imum of 2 years. y 00 more 2020, ay e other hand, other e ouin to solutions eriod severaleriod fr Global for e or long term rk of Living of rk My Health bal Health bal up SME, ed

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

------

2016) al., et (McNamee USA) the in smoking/obesity towards attitude in change drastic in change may status disease, technology, towards attitude or acceptability Historicity Historicity or path dependence: to the Germanmodel). the to need and consequently price) or should they be reimbursed during the tes significan compani which investments, (through testing clinical finance pharma how similar financially: period this survive should companies developing the how question, a opens it Third, 2019). Vijayan, technologies)(Shana matured less of case in (especially used be might service or product the of changes a particular cluster (social group, friends), that can lead to correlat to lead can that friends), group, (social cluster particular a Use of statistical methods: statistical of Use 2018a) (NH technologies tested of changes iterative the to due especially needed recruitment, new strategies had to be tested and iii) implemented, h successfulimplementatio a for keyis buy-in cliniciansi) that wave 1 of the NHS testbed systems: among the important learnings, it was ment the in observed as difficulties, further face may management project stakeholders, difficulties: management Project stakeholders (NHS England, 2018a). stakeholders t of expectations and goals often-diverting the managing of challenge the along set of stakeholders can participate at all stages of the project. Thi team: Multi-stakeholder and complex measurement is needed (McNameemeasurement2016)complex et al., neededand is experiment. the during veryTherefore, thorough is planning needed to (observed)define all measured variables followed not were which variables, of effect variables: unobserved of Effect Cost very low (Craig et al., 2017)(Craig et al., very low Reproducibility: (NHS England, 2018a)

from a scientific point of view, reproducibility of real-world re real-world view,reproducibility scientific of a frompoint of the advantage of the real-world experiment is, that a broad broad a is,thatreal-world experiment of the advantagethe in test beds there is high interaction between patients in patients between interaction high is there beds test in it seems critical, when the study is performed ( in such a complex system, one cannot rule out the out rule cannot one system, complex a such in eod h calne f aaig h various the managing of challenge the beyond 36

n, ii) too enhancepatient too ii) n, tly increase investment increase tly s advantage can bring igh flexibility was ion between the the between ion ting (similar S England, S time e.g. time upfront sults issults ioned, pu blic he es

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. the same time, health spending in Austria was 10.4% of the GDP in 2017 2017 in GDP the of 10.4% was Austria in spending health time, same the heal on diseases chronic of effect and prevalence increase an suggest numbers These todiabetesstrongly due andhas increased heart stilldiseases were versus versus 64 years in the EU). While cancer de atsignificantlybirth years is lower, than EU the average(57 years offree healt the however 2017), in years 80.9 vs. years (81.7 average EU the above slightly e life report, this to According 2019). Policies, and Systems Health on (OECD/Europea Austria about published was profile health country a 2019 In 2.4.

on health care expenditure and particularly onlong-term care. care expenditure and health on particularly EU). The outlooks for health care in Austria emphasize the population ageing pu

------

D longer (NHS England,2018a) longer consent the arranging and broader much is individuals affected of range issues: Ethical needed (Sanson-Fisher etal., 2014) needed t insensitive methods statistical Therefore, patients. those of results Need for pre-existing high-qualitydatapre-existing Needfor from the state/sponsor is important (NHS 2018a)isEngland, important state/sponsor the from commitment: level Top England, 2018a)(CHF and2018a)(CHF George InstituteThe Health, England, for 2018) Global security analysis and Data collection, validation a-priory to be included (McNamee et al., a-priorybe to 2016) included validation validation: prior for Need of the testbed (Arntzen etal., testbed the 2019). of the on based community testbed the market and projects further for used test the in achieved is literacy digital of level high a once hand, other costs) increased timelines, of (prolongation project the into incorporated training technical available, not is technology the of exploitation maximum literacy digital for Need igi t al he alt h a it is essential to clarify, if consent is needed from participant from needed is consent if clarify, to essential is it nd eH ealth need for national leadership, strong top-level commitment top-level strong leadership, national for need (Topol Review, 2019): if the level of literacy needed for the the for needed literacy of level the if 2019): Review, (Topol products and services evaluated in testbeds need some need testbeds in evaluated services and products . in Aust

at h was reported to decrease since 2000, mortality r : these were mentioned as key in all cases (NHS all cases key as in mentioned were these : 37 ia (surveys, databases) (NHS (surveys, England,2018a) databases)

the causemain of death. xp disability or disease or disability ectancy at birth is birth at ectancy (3 o correlation are correlation o rd highest in the in highest n Observatory n bed, it can be can it bed, tting tting pressure existence a take can . must be must , as the as , On the the On hy life hy th. At th. Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. of actual use of digital entailingthe of health technologies, exchange of data (Fig. implementat technical of state the and maturity of level the ii) activity, i) t levels: on34 indicatorsIndexHealth based on three Digital a established analys and included healthcare digitalization strategy was that study benchmarking international latest The F m telehealth, & HealthWorld Organization, This2016). book a provides snapshot from 2015 about eHealth for Observatory Global (WHO globally coverage health universal of support in The latest survey by the WHO on eHealth was performed in 2015 to assess the u much much has changed in eHealth since 2015, importantly the electronic h to increase health care efficiency (including social media and big data) in nie ponmns n uae f Pecitos y P rne Asra 29 Austria ranked GPs by ePrescriptions of usage and appointments online on of availability in measure appointment booking, e-Prescription status, 24/7 summary healthcare info service a a 2017): Wójcik, Anna & Arak (Piotr pic country different a showed 2016 in scenery eHealth European the about report A Index. Development already better developed compared to other countries and the country ranked 21th European countries(Fig. European ig ure ure 10 eHealth index of 37 European countries in 2016. Source: (Piotr Arak & Anna Arak Wójcik, Source: 2016. 2017, index eHealth in (Piotr 37European countries 11) of p. He alth, eLearning, electronic health record system and the strategic use of use strategic the and system record health electronic eLearning, alth, 10 ).

ca rried out in 2018 (Rainer Thiel et al., 2018). The study 38

ealth ealth recor 125 countries. While ion, and iii) the level level the iii) and ion, vailability, usage of d h Austrian the ed he levelofpolicy he ) 11 d se of eHealth th ue f the of ture mn 37 among system was ). the use of in the ICT

line line ICT

12 beingactivities thestrongest, followedby technical readiness and the ). 12 11 Digital Health Sub-indicis on the 3 levels. Source: 3 the HealthDigital Sub-indicis on Source: the by HealthLevels the Index of Digital developed Stiftung. Bertelsmann al ysis, Austria ranked 10 ranked Austria ysis, th among the 17 countries examined: the level of policy policy of level the examined: countries 17 the among (Rainer Thiel etp. 2018, al., 222) 39

(Rainer Thielp. et 2018, al., 30)

actual use of data(Fig.

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. gradually, as more and more institutes join the s the join institutes more and more as gradually, re patient The Austria. throughout states federal and facilities healthcare implementation started in December 2015, and it is currently being bcbn” o “backbone” ( Gesundheitsakte) elektronische (or ELGA called place, 2016) Organization, Health World & eHealth 2015 in profile eHealth country the in already mentioned As 5 4 the mentioned in different emphasis re to the electronic may health record system scenery when establish health digital Austrian the of ranking the in differences The Austria framework the under development under currently is strategy ambitious more even Economic Affairs coordin Affairs Economic With the aim to fully exploit the potential of digitalization, the as bewillresponsible well thefuturefor development modules. of the eVaccination (vaccination documentation). C documentation). (vaccination eVaccination COVID the to due states all in implementation (interim eMedikation to ytm toe h d nt at o ae ces o t a ot u ayie Te syst The anytime. out w that GmbH,ELGA by operated opt can it to access have to want not do who those system, an opt out method out opt an long term”long be a knowledge and project platform to implement digitalization proje digitalization implement to platform project and knowledge a be and know-how,communicate opportunities and and networkcoordina and affairs (Federal Ministry for Digital andMinistry(FederalDigital for Economic affairs Affai In the Digital Roadmap Austria, the following aims were defined for the h German at in Available https://www.ffg.at/en/node/70661 guarantee and expand prosperity, and job opportunities, as well as the qu nationwide nationwide •

4 “

Implement and continue to develop the electronic healthrecordschem continue and Implementtodevelop the electronic with the aim “to establish and further ensure Austria’s role as a lea (European Commission, 2019a). A Digitalization Agency was developed in 2018 te utin Hat sse, t il noprt frhr oue, li modules, further incorporate will it system, eHealth Austrian the f st : udies. https://www.digitalaustria.gv.at/ it is available for everybody who has access to the Austrian health care health Austrian the to access has who everybody for available is it at ed the Digital Roadmap Austria initiative established in 2017. An 2017. in established initiative Austria Roadmap Digital the ed

last accessed on 22.03.2020. 22.03.2020. on accessed last as responsible for the development of the current record asrecord current ofthe development the for responsible , Austria has an Electronic Health Record system in system Record Health Electronic an has Austria , 40 ur rently, ELGA is voluntary for citizens through citizens for voluntary is ELGA rently,

(last accessed on 07.06.2020) 07.06.2020) on accessed (last ys tem. The system is planned to serve as a as serve to planned is system The tem. rs , n d) : Federal Ministry for Digital and WO lbl bevtr for Observatory Global (WHO ELGA: ELGA: rolled out to the different testakeholders ct 1 pneis and pandemics) -19 ing the ranking index ealth ealth care and social Ab s, provide expertise provide s, cords are growing are cords ding ding digital nation out ality ality of life in the nd) The n.d.). , ut rm the from sult e (ELGA) e Digital Digital 5 m is em . to ke

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. regulatory experts and selected investors. Besides connecting the start- the connecting Besides investors. selected and experts regulatory pharmaceutic medical device manufacturers, private and public insurance companies, of healthc network a to access offers Vienna Hub Health prototype. start- tech med health, medical digital on device, focuses program accelerator The GmbH. Wien Gründerservice Universitäres Digital health start- health Digital country.the in software for medicine, telemedicine and eHealth proving dynamica life sc Vienna 9 8 7 6 Europe in leader science life be to goal a has Austria/Vienna 2.

rvdr ad eerh n euain nttts n uti, includin Austria, in Currently institutes developers. education and research and providers cec Directory Science start- biotech for location attractive an offer to out built ecosystem 11.06.2020. on Stand https://healthhubvienna.at/ http://www.lifesciencesdirectory.at/ https://www.lisavienna.at/life-sciences- 4.1. 2016 p.30) andScience,Federal of Research Economy, Ministry Chancellery and group, allergies and drug intolerances), and can and be allergies and group, intolerances), drug • and supported contact advisory(TEWEB) electronically initial service • consenpatient’s the • electronic (ePrescription) and prescription • specialneeds people with •

9 Develop a patient summary that contains key medical data on the patient a summary patient containsDevelop keythat datamedical patient theon Encourage the widespread usetheof assistancesystems widespread toEncourage help people and elderly Set up contact and advice centres for contact e.g. inthe population, theand for whole Set up advice centres Prepare a framework for electronic health services e.g. telemedicine e.g. a Prepare framework services electronic for health Design an electronic vaccination record, vaccination an electronic Design electronic mother-and-chil D , and and , igital heal it’s accelerator program organized by the Viennese high-tech incubator high-tech Viennese the by organized program accelerator 7 th it al ie cec/itc/eTc cmais sples service suppliers, companies, science/biotech/MedTech life all lists up c s can get special support at the open innovation platform, Health Hub Health platform, innovation open the at support special get can s ompa 8 tee r 6 dvlpr opne i te ietr wrig on working directory the in companies developer 63 are there , t last accessed on 11.06.2020. 11.06.2020. on accessed last ni

e s last accessed on 11.06.202 on accessed last in Aus in -vienna/for-start-ups/ tr up ia

wt a em ie f -0 ad ih working a with and 8-10, of size team a with s 41

vi last accessed on 11.06.2020 11.06.2020 on accessed last 0. ewed in countries ewed other subject to

6 . There is a broad, supportive broad, a is There . up s. The Austrian Life Austrian The s. up ience environment mdcl device medical g dmedical card s with important with s ” (Federal form of anform l companies, al are are suppliers, (e.g. blood (e.g.blood INiTS

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. fundraising.certification processes and affairs, reg on support specific gives also programaccelerator the mentors, and stakeholders 42

ulatory ulatory Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Artifacts are Artifacts d of validation economic and clinical the for important are that ba system inputs and and incl inputs outputs, system an resources, non-technological and technological resources, intangible ecosystem were added to this model this to added were ecosystem Wrd elh raiain 21a p xiii) p. 2019a, Organization, Health (World he digital of implementation the enabling components the on WHO the used I model, the of structure basic the get To model. a into put were valid health digital the of elements the landscape, Austrian the analyse To paper, paper, 27.05.2020. 27.05.2020. 10 PubMed® publicat scientific peer-reviewed Relevant health. digital of testing in actively Austria in institutes identify to used was review Literature 3.2. 2020) Holgersson, & (Granstrand used was Holgersson and Granstrand To define the 3.1. 3.

ntttos n rltos icuig opeetr ad substit performance innovative the for important and complementary including relations, and institutions publications indexed on PubMed until 27. May 2020 were screened wi screened were 2020 May 27. until PubMed on indexed publications (terms: terms innovation health digital variat and Austria) (term: area geographic the trial), (term: evaluation (clinical) used as the as the used vali health digital a thesis this in this, on Based p.3.). 2020 Holgersson, & Biomedical library of the National Institute of Health of Health Institute National the of Biomedical library

rriers of successful digital health evaluation (Kowatsch2019). et al., rriers digital of successful health evaluation

M “ e Lit Es innovation ecosystem is the evolving set evolving the is ecosystem innovation tho 10 t era ablishi n a 2020 May on evolving set of actors, activities and aftifacts andinstitutionsevolving the and activitiesand actors, set aftifacts of according to Granstrand and Holgersson “products and “products Holgersson and Granstrand to according ds “digital “digital health t ure

ng a r ev ie m w od .

validation ecosystem erh em wr cmiain o sadr epeso for expression standard of combinations were terms Search el for

uding innovation” uding

dig ba ita sed on the publication of Kowatsch et al. describing al. et Kowatsch of publication the on sed l hea of dgtl elh o eelh r mHealth). or eHealth or health” “digital .

43 an h eeet o te iia hat validation health digital the of elements The https://pubmed.ncbi.nlm.nih.gov/

lth actor (Granstrand & Holgersson, 2020(Granstrand 3.). p. ”, the definition of innovation ecosystem by of v actors, activities, and artifacts, and the and artifacts, and activities, actors, alid or a population population a at ion ec osyst igital health innovations health igital ions were retrieved from retrieved were ions t rltos ta are that relations, ute services, tangible and tangible services, of volved in the clinical the in volved the one proposed by proposed one the em actors . dation ecosystem is is ecosystem dation th this strategy. All strategy. this th As defined in their in defined As last accessed on on accessed last alth interventions interventions alth ohr ye of types other d

ation ecosystem ation ” ions of broad of ions (Granstrand relations, relations, All . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. is not scope of this thesis. scopeof not is thesis. this The full review of the digital health solutions, the methods and th and methods the solutions, health digital the of review full The highestonly institutionalthedifferent, once.affiliation is shown level institutional highest same the from were affiliations the if an author had multiple affiliations from different institutes, all group). research or department the not and university the only professor, university i (e.g. considered was level organizational highest the only however retrieved, To identify the Austrian institutes involved in the testing, Austrian criteria the in 2Summary literature inclusion and rev of Table exclusion exclusion areinclusioncriteriaTheand described screened. wastext full titles, author lists and abstracts were read to identify relevant p Exclusion criteria Exclusion criteria Inclusion • • • • • • •

clinical clinical decision management software as sta or systems management information of description or Testing algorithms or diagnostic tools) or diagnostic tools) algorithms predictive establish to data collect (e.g. services or product performed trials Clinical oioig follow- monitoring, of a digital health produ health digital a of testing field or clinical performed or planned a of Description human human disease or disease predisposing condition (e.g. obesity) people orchildcare people testing of a digital health productor service supporting elderly field or institutional performed of planned a of Description site institutional or clinical Austrian one least at in done is Testing No restrictionthe publicationtrial or ofthe on time No testing the of execution successful or results the on restriction No 44

up mngmn o te ramn o a of treatment the or management , iew iew ct n re t develop to order in or service supporting the diagnosis the supportingservice or institutes were considered, but ublications. Where necessary,

and only the divisions were divisions the only and affiliations affiliations of authors were e results e in Table 2. 2. inTable nd of -alone -alone product iia health digital these projects these cs o a of case n Where , Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Qualitative content analysis (Mayring, 2014) of expert interviews w interviews expert of 2014) (Mayring, analysis content Qualitative 3.3.

eg itoutr prs ecre o tpc ta ntrly rs de o the to due arose naturally that topics on excurses parts, introductory (e.g. (Mayring, 2014). For selection, all parts of the interview, which were unr For transcription of the interviews, a mixtureof clean read and selec interruption. without minutes) smartphone a of recorder voice elements elements of the Austrian ecosystem As health. conductin and supportive explore andfactors while barriersplanning to pandemics) were removed. For the rest of the discussion the following rulesrest following removed. For thewere pandemics) of the discussion the w working two heal digital developing companies workingat two chosen,experts were Four selected. were validation conducting of planning innovators with worked clinical or field validation of digital health or a key me least least (see questions checklist interview the and #2) (Expert call phone a or f made were appointments interview, the to agreeing After described. qu research the and topic the where emails, via made was contact first The Appendix 5 Appendix provi is experts interviewed two the of (anonymized) experience work of summary four the with made be could contact No briefly. email via questions answ expert third a while interview, an in participating on agreed experts Appendix 6) Appendix of Declaration the of (template Consent of Declaration a signed Interviewees on 04.06.2020.#2 Expert expert, a person was as defined someone with • •

3 those of Expert #2 with E#2 of Expert those #2 with Quali The interview was transcribed literally inEnglish interview wasliterally transcribed The Passages by the interviewer are marked with VSZ, those of the Expert #1 with with #1 Expert the of those VSZ, with marked are interviewer the by Passages days in advance. The interview with Expert #1 was conducted in 07.04.2020, wit 07.04.2020, in conducted was #1 Expert with interview The advance. in days . .

Interviews were recorded either with the built-in function of Sk of function built-in the with either recorded were Interviews t at at e isiue spotn vldto o dgtl elh Hwvr o However, health. digital of validation supporting institutes key iv e ex pe r t

i nter . Both interviews lasted for ~1 hour (49 minutes and 48.5 and minutes (49 hour ~1 for lasted interviews Both v ie . Structured interviews were conducted with local experts w s

experiencein planning conductingand/or mber of the ecosystem, who works or . Only experts active in Austria in active experts Only . tive protocols were used th expert selected. Brief selected. expert th as or a Skype (Expert #1) (Expert Skype a or elated to the research used to identify the identify to used g validation of validation digital g below) were sent at sent were below) we ered two direct two ered th products and products th ype or with the with or ype Consent is in is Consent sin were estions reused: COVID- l two nly ded in ded E#1 E#1 19 h h , Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Table 3Interview checklist questions and Table topics with it and frame a as used solely was checklist The questions. with intervi developed an interviews, the during discussed are topics all that ensure To 3.

The topics and the questions are summarized in Table 3 are and in topicssummarizedTable questions The the an (indicatedas werevariantsE questions two#1 developed of interview (E#2), slightly different angles of the topic compared tothe one work th to interview. As the experience of expert according working at an institution in t questions, certain omit or additional review and approv review one least at and experts the to sent were interview the of Scripts

3.1. • • • •

3. 2. 1.

Language and punctuation were slightly modified to retaintoclarity punctuationmodifiedslightly and Language were Ex I added in parenthesis and marked with the author initials (VSZ) markedauthor withand the(VSZ) in parenthesisinitials added fitt more or right a incorrectly, word English an used expert the Where removed or modified to retain the main message but maintain anonymitymaintain but or toremoved modifiedthe message mainretain in the of identity the reveal could which expert, the by Comments nt xets on expertise Internal and external validation health towards companies pushing digital Factors Introduction cessive expletiv cessive e r vie w c he al . Interview transcripts are provided in Appendix 7 (E #1) and 8#2).(E 7(E Interview and . #1) are in Appendixprovided transcripts c kli st es

, removedweredecorating words and CROs with open mind for digital innovation testing? Do testing? innovation digital for mind open with CROs and clinicians have you do Austria in opinion, your is What #1: E (ofwith clinical experience testing health)? digital already had who team, the on anyone have you Do #2: E companies? between these makes differencewhat this investors, the among founders, the among factors critical testing and one clinical does not consider running it. Do you a see any running considers one and products similar on working roughly are which companies, consider you If #1: E health? of digital validation economic and clinical with experience your is What #2: E testing? clinical/economic to E #1: In your experience what is the attitude of thestart-ups 46

e nomto gie drn the during gained information ing ing at a digital health company -week time was offered foroffered was time -week he he ecosystem (E #1) covers ew checklist was was checklist ew terviewees were terviewees allowed to as to allowed ing word was was word ing d E #2).d k Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. 11

asking about it. it. about asking This question was not asked, as the expert discussed the topic during the topic during the discussed expert the as not asked, was question This 7. 6. 5. 4.

reimbursement reimbursement validati conducting validation validation during Challenges f application supporting elements Ecosystem and Austriain validation Financing validation health digital in insurance) (regulatory, Policies on inAustria or

with internal with study the manage and design/organize to plan you Do #2: E healthvalidation? organizationdigital supporting your is How few? a name you could yes, If Austria? in clinicians you see the emergence of specialists in digital health among experience goingvalidation?for experience your in companies for challenges major the are What #1: E parties process? external insupport this you which yes, If reimbursement? for go to plan you Do #2: E insurancecompanies? reimbursement by for go to plan companies think you do much, How #1: E support would that funding validation? public people about investors know your you among Do innovation? your have of validation for you supporting/pushing Do #2: E local place? invalidations such about funding or from VCs) that they get easier funding if there are know E #1: Do you see any financial benefits for companies (public you informationabout get them? Do insurance companies on digital health? If yes, where did oryou regulators of #2: policies/frameworks/recommendations E digital in among regulatorshealth Austria? in specialists of emergence the see you Do #1: E the study? Doyoureach study? the toexternal advisors? out of function) (which coordinator internal the be will Who it? e h ms calnig uig h suy designing, study: the executing/managing organizing, or financing? during challenging most the be E #2: Based on your experience, which part do you expect to pe 47

ople, or do you get a CRO/consultantsdotoget a you ordo ople, interview without specifically specifically without interview 11

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

umrzd o aayi. nhr ape (asgs utd rm h interviews) the int from (Mayring, toillustrate2014) selected the categories quoted both (passages from samples collected Anchor analysis. were for category summarized each within Parts marked. were ad interview the of parts all transcripts, interview the of through established based on the research questions and the theoretical D background. nlss f h itriw srcuig a ue (arn, 2014) (Mayring, used was structuring interviews the of analysis analys further before collected were points key and summarized were Interviews 3.

3.2 8.

C comparison comparison International onte nt analy sis

#: o o se n international any see you in have to Do useful be would that method practice/institute/evaluation #2: E health? of digital validation think you that internationally, practices good any there Are #1: E are motn rgrig lncl n economic and clinical regarding important Au striatosupport digital health testing? 48

dressing a certain category certain a dressing . rel, aeois were categories Briefly, uring the run- riw and erviews is. For the For is. were Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. i te bec o a out nbig niomn, hr i te ik o risk the is there environment, enabling robust a of absence the “in health digital of integration consequently and validation successful economic and clinical methods evaluation are needed. Therefore, a complex necessary ecosystem might thfor be their to and team multidisciplinary devel involves their products, complex are innovations health digital above, described As c the supporting ecosystem an validation healthinnovation. of digital economic of elements the collect to aims section This 4.

health intervention” health sustainabi and effectiveness the on impact severe a and systems unconnected implementation of digital health products of implementation healthproducts digital contrib are ecosystem an of components following the WHO, the to According (Kowatsch et al., 2019)(WorldHealth et al., 2019a) (Kowatsch Organization, digital on publications recent two used I model, ecosystem the build To

- - - - M

o Foundation layer: enabling environment through leadership and governance layer: enablingenvironment through governance Foundation and leadership the intervention and health content contentand intervention the health facilitating channels communication and systems ICT applications: Digital Digital health interventions and technologies and interventions health technologies Digital elh otn: nomto aind ih eomne hat patcs an practices health recommended content health validated with aligned information content: Health de l of ad (World Health Organization, 2019a, p. xiv) p. (WorldOrganization,Health 2019a, igita l hea lth va l id (F at ig. ig. 49 i on 13

) (World Health Organization,Health (World ) 2019a): ec o s yste .

m .

. As stated by the WHO, the by stated As . aiain complex validation a pro a f health intervention intervention health ieain of liferation uting to the the to uting delivery of delivery iia and linical iy f the of lity opment d e Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. managed by of by parties differentecosystem: managed the codn t Kwtc e a. mi brir o te vlain hs ae fundi 2019) are al., et phase (Kowatsch methodology evaluation and guidelines (maintenance), the of barriers main al., et Kowatsch to According Figure

Organization, p. 2019a, xiii) individual elements of the model are analysed elements below. of theindividual model patient patient that below the leadership and governance layer, These stakeholders were placed into the basic model structure proposed by t - - - -

13 Components of an enabling ecosystem for the implementation of dig Funding: public and private funding opportunities tailored t to tailored and public privateFunding: opportunities funding health innovati health testing for channels delivery and infrastructure existing (maintenance): Costs ehdlg: upr o aaei isiue, ttsiin ad aa analys data and statisticians institutes, expertCROsadvisors and health digital to specialized evaluation; academic of support Methodology: udlns spot hog sadrs eeoe b ntoa ad internation and regulatory policiesinstitutes, legislations national and by developed standards through support Guidelines: re presentatives

on ” w , high quality pre-existing health data sets sets , healthdata quality high pre-existing ere ere added to represent the patients in the ecosystem (Fig. 50

an an additional foundation layer “ ital health interventions. Source: . heevaluation phase These barriers can be can barriers These he he WHO, except, patients and (World Health g costs ng,

digital ). The

ts ts al Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Figure Figure

data sets data h in (e.g. testbed of project health testing digital real-life forabove, mentioned sets data quality of existence the is element this of element special A systemare requirementsbasic available. the which are deployed. In a supportive ecosystem the health care system is wel health innovation, since only those benefits can be measured in practice evalu the influence factors same The internetconnection). instable and speed mi infrastructure IT necessary literacy, digital of level low (e.g. products/services wh t exploit system, fully to able care be not may users health hand, one On the implemented. on depends innovation health digital of value The Content: Health em parti is content Health validation. of analysis the during or up set making decis making analys in possibilities new open technologies these review, Topol the in 2 advances” these of adoption the to barriers major remain expertise of lack be recognized be these of potential full the as well, as validation economic and clinical Digital Health Innovation:Digital 019 erging digital health technologies, like AI or data analysis (big data) (big analysis data or AI liketechnologies, health erging digital , p. 11) p. , 14 Model of a supportive ecosystem for the clinical and economic the of di clinical and for Model of validation supportive a ecosystem are . ions, however “(u)neven NHS data quality, gaps in infor in gaps quality, data NHS “(u)neven however ions, needed (NHS England, 2018a) England, (NHS needed Based on this, the lack of even, high quality data would pose a barrier duri barrier a pose would data quality high even, of lack the this, on Based .

. 51 These data might be used during the planning, the during used be might data These

gital health. illustration. Own gital health. cularly important for new, new, for important cularly mation governance and governance mation he benefits of the new the of benefits he technologies could not could technologies . As described recently described As ing, interpreting and interpreting ing, (also during testing), s) such (historical) such s) l developed, l and (Topol Review, (Topol ation of digital of ation at cr. As care. ealth ssing, low ssing, r i is it ere

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. (start- innovators health Digital Similarly, research institutes, universities developing and testing digital heal digital testing and developing institutes,universities research Similarly, other the to company one from workforce the of move the through - or way ecosystem. They may share experience with each other on digital health validat complexityduring the need testing (users and tolearn theuse add but testing the of costs the increase only not does it testing, need channels new If health. digital of validation the during necessary otae n cmuiain hnes o eie dgtl elh intervent health digital deliver to channels communication and Software DeliveryChannels: solidtheshare and in studiesresults publications scientifically condu to used are they because particularly ecosystem, the of members other educate the results and experience during studduring and the results the experience interim communication channels are used during validation, it reduces t st trio a by characterised are countries “(s)uccessful leadership: report, #SmartHealthSystems political needs transformation digital that concluded, care health of digitalization compared report recent a Additionally, initiatives” isolated as operating lever to can opposed interventions digital different necessary which is in implementation, approach strategic a general, In level or statenational commitment. need necessitates resource largearedeveloped, and (e.g. thewhere testbeds) methods financial evaluation particul is This needed. be might validation health digital of support level state or national a Therefore, maintained. be to need stakeholders and interest the where task, complex a is health digital of Validation StrategyInvestment: and nesadn te ie n fnnil eurmns f uh testin such of requirements financial and supporting time VCs the or understanding validation health digital in supp be experienced can it investors sector private the In essential. is purpose this for funding availabi the financed, be to needs validation health digital Naturally, 4). p. rategy, rategy, political leadership and coordinating national institution up s, SMEs, large corporates) are important elements of the of elements important are corporates) large SMEs, s, y for usea for real-life later, 52 (World Health Organization, 2019a, p. xiii). xiii). p. 2019a, Organization, Health (World

t spot chsv apoc to approach cohesive a support “to . new channels) s” s an additional level to the to level additional an s (Rainer Thiel et al., 2018, lity of public and private private and of public lity aim of several different several of aim arly true, if large-scale if true, arly g oe nte, as another, one age s ecie b the by described as he ability to translate ) Lre cl and scale Large g). n 7 onre and countries 17 in to be built for the for built be to h tsig (and testing the taey o the for strategy ions ions th solutions solutions th . re b angel by orted Additionally, ifAdditionally, ion in a direct institutional – ar of effective of indirectly. already e may ct ct

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. advance and approval certification, requirementsof regulatoryand legal the about sh developers testing, clinical conducting on background legal clear a Besides clinicalvalida and Policies legislation on the of requirements contractresearchcan the managers, plan process. organizations understan gaining help can platforms information centralized guidance, the in involved Institutes protection. clinical data on legislation applicable national with compliance tes at approval committee ethical getting of requirements the understanding include These testing. clinical on background regulatory and legal the on understanding an need Developers clinical and conducting Policies onlegislation testing: (testbeds, sites experiment livinglabs). natu servers), dedicated (e.g. hardware necessary with testing the for specialized sites special require may health digital of validation economic and Clinical Infrastructure: for UK 2019)).(NICE,in the by developed NICE standards evidence the (e.g. institutes national from or 2017)) Group, Worki Device Medical a as Software (IMDRF SaMD on IMDRF the of guidance the (e.g. meaningful evaluation of digital health. Such guidance can come from i plan on guidance clear with developers provide can frameworks and Standards and Frameworks: Standards canbeincludedinto section this of digital health evaluation field conduc universitiesand institutes research sense, broada In companies. f specialized accelerators and training providing institutes validation, such (CROs) specialized for this field, advisors or consultants supporting th Resea Contact be can health digital of validation the supporting Services Services: government establishingorprogramsenvironment. of grants form the testing simultaneous testing of digital health inventions is likely to req how how they ca eilto ad oiy ih b ta i nes o e lxbe to flexible be to needs it technologies, specific to that diverse digital technologies and stable, so comp be might policy and legislation . This knowledge will guide the design of the clinical test and inf and test clinical the of design the guide will knowledge This n generate evidence for economic value and get reimbursed. The challenge of t 53

uire uire public financing, e.g. in the tion: tion: . . e planning/execution of comdt future accommodate infrastructure: clinical infrastructure: nternational institutes prvl a provide can approval anies can plan ahead. ding and clinical trial clinical and ding ting research on the on researchting orm developers on developers orm reimbursement in reimbursement ra regulations, trial rch Organizations Organizations rch or digital health digital or ould also know also ould ting sites, and sites, ting iia health digital conducting ning ral he he ng a Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. in the Digital Single Market; empowering citizens and building a healthier s healthier a building and citizens empowering Market; Single Digital the in using processes grounded inreal- processes using that “(t)he healthcare workforce needs exper A recent review analysed the impact of new technologies on the NHS workfor date in this dynamically developing fielddynamicallyin this date developing p all for necessary is field the on education continuous Additionally, considerations. ethical 2018) Chen, (J modelling based agent can support the planning, modelling and analysis of new, compl clinical managers trial get experience digital on digital healths testing, and necessary gain to trained are professionals care health other curriculum, health digital include universities medical exist: facilities teaching retention or even low willingness to participate (recruitment rate) (recruitment participate to willingness low even or agai retention which technology, the of mistrust in Walt result (Alexander also can dropout understanding patient higher in digital result Low which health. inconvenience, digital of validation the support can patients workforce, healthcare native digitally of importance the to Similar and Patient Patients Representatives: https://www.europarl.europa.eu/doceo/document/B-9-2019-0239_EN.html 12 In a supportive ecosystem, workforce needed Workforce: ininsurance can companies, and advise who at regulatory bodies needed Additionally,departments, validation. digital dedicated or experts groups on me evaluation long-term when important particularly is stability Such they are important they ecosystem. partnersinthe hea digital support may execution and planning the in involvement their Thi 2019). al., et Velthoven (van innovation health digital facing patient health was identified as an important facilitator of the successful development the throughout representatives) patient (or patients of Involvement longerdata, t Alltheseregistered it. from lowto lead clinical quality itcan also causeFinally, useofincorrect the technologyand conseq See point 38 in the European Parliament resolution on enabling the digital the enabling on resolution Parliament European the 38 in See point

world world evidence” ). 12 motn apc o te riig s understanding is training the of aspect Important .

tise and guidance to evaluate new technologies, 54 to

validation the is available. requiresThis that (Topol Review, 2019, p. 10). Review, 2019, (Topol p. 10). ociety (2019/2804(RSP)) available at at available (2019/2804(RSP)) ociety ex experimental schemes (e.g. transformation of health and care and health of transformation (last accessed on 07.06.2020) 07.06.2020) on accessed (last and sustainable adoption of (A uentlylow or no benefits tatisticians, data analysts rial and higher costs. rialand costs. higher and its validation in the in validation its and ih iia ltrc of literacy digital high lexander Walter, 2013) Walter, lexander arties to remain up to up remain to arties thods are used for the for used are thods ieay a la to lead may literacy s finding implies that implies finding s novators. novators. lth validation and validation lth las o lower to leads n ce and identified r 21) Low 2013). er, health might behealth of digital of literacy, . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. block. governance and leadership validation, they were placed in the ecosystem model model ecosystem the in placed were they validation, ae o ti cmlx role complex this on Based user experienceincrease therefore,and successofthe testing. Re the “end the well as health digital of support highly could activity This costs. development of funding through or patients from collection data through e.g. pharmaceuticals of testing) (i development and research the pa in involved 2017), increasingly are Lingen, organizations van & (Sienkiewicz Forum Patients European the by described As engagement recruitment, patient retention ofvalidationduring the health. digital enhance may activities these All view. end-user workf care health developers, health digital (including stakeholders various the an material information product/service-related of dissemination and health evaluation through arranging patient education on technologies, aiding organizations patient field, education and capacity-building the On &(Sienkiewicz Lingen, supportand 2017)van research & development peer capacity-b policy, on focus can organizations patient of activities The Lingen, p. van 2017,3). patients when latter m a represent themselves) represent (the carers and/or patients whereby and focused, de are organisations “(p)atients’ EMA, organizati patients’ are representatives Patient the lated to policy development, patient organizations can help policy makers to lea to makers policy help can organizations patient development, policy to lated patients) - user view” user .

on digital health validation (e.g. which factors are important t important are factors which (e.g. validation health digital on , through realistic planning, incorporating end-user view early on to on early view end-user incorporating planning, realistic through of ains n ptet raiain drn dgtl health digital during organizations patient and patients ajority of members in governing bodies” governing in members of ajority fined as not-for-profit organisations which are patient patient are which organisations not-for-profit as fined 55

n ad efhl gop. codn t the to According groups. self-help and ons as a foundation level similar to the the to similar level foundation a uilding and education, education, and uilding ih spot digital support might hog educating through d real-life evaluation real-life . oeie even sometimes ncluding clinical clinical ncluding

(Sienkiewicz &(Sienkiewicz r ual to unable are the the generation orce) on the on orce) o test for test o rn about rn and tient

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. 2020 wer which identified were publications 64 Austria AND trial AND trial AND Austria 6, with the terms eHealth AND trial AND Austria PubMed PubMed digitaldescribing trials health in Austria testing of digital health. Therefore, I analyse peer-reviewed scientifi Table 4 Published clinical trials for digital health in Austria in health 4 fordigital clinical trials Published Table whic Austria in institutes identify to used was review Literature 5.1. 5.

disease program for chronic heart MyCor telemonitoring patientsacne Mobile teledermatologyin cancerof patien nutritionalmonitor status a with mobileapp to System complemented Clinical Decision Support nutritionenteral weaning children off Net coachingto support treatmentin women intervention for obesity W servicefor elderly App basedassistance amyotrophic heartinfarct behaviourchange after feedback system for Telehealth delivered solution for diabetes Telehealth nutrition patients (eSMARTStudy) monitoring cancerof Real- apneasleep (HOPES study) diagnosis and treatment of Telemonitoring for the patients with telemedicine Palliativecare of oncology Projectdescription

Va . -based follow- ti After screening of all publications, 16 16 identpublished screeningpublications, trialswere After of all me remote me symptom I l nst id at itut i on es ts

of up inv di ol v gi ed ta

Medical Medical University of Graz Vienna, Medical UniversityVien of Süd,FEM KaiserFranz JosefHospital- m.b.H., MOWI, Salzburger Hilfswerk Salzburg Forschungsgesellschaft Research Me - UMIT University for Health Sciences, GmbH, Medical University Innsbruck,of AITAustrian Institute Technologyof Bergbau, Medical University of Graz GmbH, Versicherung fürEisenbahnen und AITAustrian Institute Technologyof Medical University Vienna of University Viennaof NRZ Rosenhügel, Vienna, Medical Sankt Josef Medical University Vienna, of Hospital Affiliations of authors inAustria Austrian Institute of Technology, Institute Medical University Innsbruck, of AIT InformaticsMedical and Technology - UMIT University for Health Sciences, Dermatology Practice Medical University of Graz, Private St. John of God (Salzburg) Rudolfstiftung, Hospital of the Brothers of General Hospital, Krankenhaus Medical University Vienna, of Vienna in l hea dical Informatics and Technology pub lt lis h inAu he d

cl 56 i nic s

tria . With With searchthe terms “digital ANDhealth” al t

r ial s

for na

dig 64, with the terms mHealth ita h e published until 27. May May 27. until published e c publications indexed on ciey novd n the in involved actively l (Maguire al.,et 2017) (Kotzian et al., 2018) (Nemeceket al., 2019) Reference 2015) (Ammenwerth al., et (Frühaufal., et 2015) (de Bruin et al., 2018) 2014) (Marinscheket al., (Rader etal., 2017) (Willneret al., 2015) (Kreiner et 2015)al., 2018) (Schusterbauer al.,et he ified (Table 4). ified(Table alt h in Aust r ia

When When analysing the authors, I identified, tha medicationstroke,monitoring, and obesity elderlyassistance. disease, tri one and diseases, circulatory on focused 4 diabetes, with patients trials Ei institute in 1 study 1 in institute stu 6 in hospitals non-university studies, 2 in universities non-medical University Medical the 5, in Graz of University Medical the 7, in Monitoring management of patients (Kollmann et al., 2007) al., used et (Kollmann were patients of applications management mHealth where trial one only was There remotely. All, but one trial tested telemedicine solutions: either to monitor, f notthesis. is scopeof this projects The full review of the digital health solutions, the methods and t companies(start- developing an authorshad ofthe nonereview, literatureby identified studies Graz. Technology, of University the at Devices Medical of Body Notified European a) diabet insulin treatment in application for assisting Diab-Memory mHealt diabetesinadolescents Telemedical supportin failureheart (Mobitel) Telemonitoring after acute pacemaker Telemedical follow-up of medicationadherence solution to monitor blisterSmart and mHealth mHealth was used in other trials as a tool for telemedicine and not as the mai as the not and telemedicine tool for a as trials other in used was mHealth senbahnen senbahnen und Bergbau), one private doctor practice and one author with

affiliation is only indicated at the highest institutional level (e.g. dep highest(e.g. level affiliation is institutional atthe only indicated es

) mn te uhr, n hat isrne nttt (esceug für (Versicherung institute insurance health one authors, the among . There was only one private consulting firm (Science Consulting & Clini Consulting& (Science consultingfirm private one only wasThere h Health CentreSouth, Vienna ConsultingScience & Clinical Monitoring, AITAustrian Institute Technology,of University Technologyof Notified Body of Medical Devices,Graz Healthof Care Engineering Europeanwith Vienna Technology), Medical University of name of the Austrian Institute of Austrian Research Centers (former GmbH Medical University Vienna of hospitals through Austria Institute of Technology GmbH, various Medical University of Graz, AITAustrian GmbH, Medical University Grazof AITAustrian Institute Technologyof up

s, SMEs or large corporates). or s, SMEslarge t the Austrian Institute of Technology GmbH (AIT) 57

14 13 studies, (the Medical University of Vienna . Three trials involved cancer patients, 3 patients, cancer involved trials Three artment not shown, only university) artmentnot shown, only university) he results published for these ollow-up ollow-up or coach patients of Innsbruck in 2 studies) 2 in Innsbruck of n tested product product tested n affiliation to digital healthdigital to affiliation (Kollmann etal., 2007) (Rami et al., 2006) (Scherr etal., 2009) (Hayn et al., 2013) Moraket al., 2012) (Brath et al., 2013; dies, other research other dies, affiliation to the al for each skin each for al o i self- aid to In the 15 the In cal

Digital health ecosystem elements identified during the interview: healthecosystem identified Digital duringelements the and second, when asking about international practices, that could be use testbeds twice during the interview: oncea as possibility to make regula the to everyth not in force driving economic large particularly a is that reimbursement but reimbursed, highlights she reimbursement, discussing When solutions. of case in important so be not may testing clinical stated, she as However, selling point (brought the example of mySugr, the first FDA approved app, acquire needed to the success of the start- the of success the to needed validat for support specific provide may program the ecosystem: validation health accelerator Vienna Hub Health the of role the discussed She paths. people make and innovation hinder may experience much too however, helps, l in experience her, to According varies. testing clinical towards attitude complying with the rul the with complying up that clinical validation can bring financial benefit for the for benefit financial bringcan validation clinical that and itsoisinevitable is developmentin particularly technological to toward attitude in solutions. Regarding regulations, she concluded, that the change lagging of positive highly and fundamental a predicted introduc immediate the of example the brings She health. digital pandemicschanged COVID-19 theshe highlighted, other that hand, the experts among clinicians for digital health, but mentioned, that these of localSheecosystem.the open opportunity networking mindedmembers with 2.1. is different; depending on the technology, business model, experience of t of experience model, business technology, the on depending different; is s - coming from non-EU countries understand the certification process

Health Hub (accelerator Health Vienna program) Sum Result patients’ m ar s

y of of expectation of free healthcare. She brought up the topic of regulatory regulatory of topic the up brought She healthcare. free of expectation exp

inte es ert r vie of healthcare is needed. Clinical testing can also become a unique a become also can testing Clinical needed. is healthcare of

inte ws

r v up iew up s in Austria. As E#1 highlighted, each digital health start- health digital each highlighted, E#1 As Austria. in s . She highlighted, that the program can support start- support can program the that highlighted, She . s

up , h itriw agl fcsd n the on focused largely interview the s, start- up s, investors understand thatunderstand investors s, tion of ePrescription. She ePrescription. of tion the regulations compared experts are still rare. On Europe. She Europe.mentioned, She program in the digital the in program . attitudetowardsthe Additionally, Additionally, tions tions more effective, ful in Austria as well. ife science industry industry science ife over-the-counter over-the-counter dgtl health digital s he team their team he Austria, due Austria, blind to new to blind d d by Roche). n cn be can ing alsonamed ion, if it is is it if ion, it offers

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. a CRO, that they would search internationally (with a preference for an for preference a (with internationally search would they that CRO, a not need external advisors. Still, they would outsource the actual clinical opin his in enough, be can expertise internal This soon. assistant research they have internal expertise in clinical testing, and plan to clini hire so far. Regarding internal and external functions supporting their their supporting functions external and internal Regarding far. so cl reimbu for going is which company, health digital a at works #2 E As #2: Expert Potential gaps or or gapsbarriersidentified: Potential

f e oiin edr. ae o hs xeine pltcl eae i st are bureaucracyAustria. barriers in in debates political experience, his on Based leaders. opinion key of towards overcome clinicians this barrier he confirmed, that clinical validation results coul of resistance the mentioned he testing, clinical for a (FFG) Agency partner be financingPromotion options. could public for discussed with investors, neither had public support, but he thi valid of topic the him, to According relationship. existing to due (EMA) Agency Euro the with questions regulatory discuss they regulators, local contacting intern large finds also i promote and product He their about word the them. spread to helpful companies help could process this through went already that thinks they he but this, now, in support external Until no have they Austria), them. lat about the said, know he (as not bureaucratic and did complicated process reimbursement just he exist, CROs such that highlighted, he CROs, local potential any name not could he Although inical inical validation of certain products, the interview focused on ------

Resistance of doctors (fear of becoming obsolete) of (fearof becoming Resistance doctors obsolete) Lagging and particularly strict regulations regulations and Lagging particularly strict Lack of regulatory Lack testbeds of Lack of specialists among clinicians and regulators amongclinicians specialists Lackof Telemed Austria Telemed Ex xitätsforschung, Medical University of Vienna) Vienna) of Medical University xitätsforschung, pert clinicians clinicians pert (e .g. Programmdirektor für E-Health, Telemedizin und Komple- Komple- und Telemedizin E-Health, für Programmdirektor .g. 59

their experience in these areas nks that the Austrian Research cal trial manager or clinical pl ans, he mentioned, that that mentioned, he ans, another company, who company, another rsement and may plan may and rsement d help and the support that it is still possible still is it that trial management to As biggest challengeAsbiggest iia hat. To health. digital Austrian partner) Austrian ion, so they would they so ion, t isiue and institutes ate ts use. Instead of Instead use. ts e i tpcl to typical is ter pean Medicines Medicines pean ation was not not was ation id the find ational . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Potential gaps or barriers identified: gaps or Potential barriersidentified: - product/service - mentioned factors: #1 following the oth why and validation for decide start-ups certain why discussing, When her In naïve. more are some do), to clinic not about decision conscious a making do to deciding strategically (either it about strategic very are some valid towards start-ups of attitude the experience, her In program. accelerator Expert #1 is working with numerous digital health start-ups in A validationin ecosystem. Austrianelements the health digital relevant identify to experts two with conducted were interviews Structured 5. healthecosystem identified Digital interview: duringelements the Expert #2 is working at a digital health company in Austria, developin decision. this of those, who make the decision to get clinical validation, may under products and may run a clinical validation as well with a or partner as with andwell run may validation a products clinical th of one with reimbursement for go to planning is company the Currently haslargeinHeexperienceclinicalstudies theirdigital where health 2.2. - - - - - 1. - - 2.

“ preventive care. We all know, that doing sports helps to stay healthier stay to helps sports doing that know, all We care. preventive Resistance of Resistance doctors Lack of external support in getting reimbursement external gettingsupport Lack reimbursement of in Research (Austrian Forschungsförderungsgesellschaft Agency)) Promotion (Österreichische FFG Bureaucracy, complicatedprocesses Bureaucracy, pharma international companies Large Political debatesinstate Political institutes healthcompanies digital Other A it may not make sense to go for a clinical validation if I have a solution t solution a have I if validation clinical a for go to sense make not may it n aly Experience with digital Experience healthvalidation Factors driving companies driving towardsvalidation Factors sis

of i nterviews

ustria and globally through an estimate the resource need ganization. g Class 1 product is as used a tool. ers do not, Expert not, do ers experience, eve experience, eir digital health digital eir me l aiain or validation al , but you will you but , dical devices. lyr and players ation varies; ation hat is in the the in is hat n Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. As Expert #1 mentioned, clinicians and CROs open open CROs and clinicians mentioned, #1 Expert As coordinatewould internally. study the or clinical manager trial assistant validatio clinical the on CRO a with work would they that confident, i but study, the for advisors external need not will they that believes, validatio clinical considering just still are they although company, runnin in expertiseinternal already have they confirmed,that #2 Expert

- team experience,awarenessteam - however, she sees that due to COVID-19, this may shethis sees COVID-19, change: that duehowever, to contribute to the fact, that E #2 could on thi localEnot name CROs #2 active fact, the contribute to that visibili of lack the least at or experts specialized of rarity This accelerator programaccelerator Health - Hub networkingsupportVienna: providing and tailor (E Telemed#1)Austria - (E #1) at the UniversityKomplexitätsforschung Vienna Medical of E-healt for director Program e.g. clinics: university at clinicians expert - validationinterviews: during identified the health digital Austrian the of components following The 3.

sense incase. such a sense feel it only, when you are 50. It does not make sense to go for clinical an easiertime decision right an taking the the health domain and has come maybe from a pharmaceutical background, they have th if level, founders’ the on level; every on be can “It “ change a in are we change; radically will It go. one in doors closed many open “ answer with certainty answer but it does not mean, that with a little bit of search we will So So all aof sudden digital solutions have become very, very interesting. And i We do not know about such partner in Austria, who could support us us support could who Austria, in partner such about know not do We Internal and external expertise on conducting validation in Austria and expertise invalidation Austria external on conducting Internal .” (E .” #1) .

” (E #1) .” (E.” #2)

61 .”

(E #1) to digital health are still rare in Austria, in rare still are health digital ere is one really experienced in experienced really one is ere ed tothe start-upsneedsin the ty of specialized experts might experts specialized of ty not find I someone. cannot n and a clinical research clinical a and n o ter rdc. He product. their of n

s s field csse cud be could ecosystem g clinical trials at the at trials clinical g nterestingly, he was was he nterestingly, h, Telemedicine and Telemedicine h, validation in that .

in this study, this in very radical very t t pushed

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. solution and the business model. As E #1 summarized: E and#1 the solution businessAs model. of benefit financial ofthe is (clinical) theor not validation Whether se Based on the interviews, regulatory policies are lagging behind technology, in

However, However, thi (EMA) for regulatory advice, because of an existing connection with the Agencwith the ofanregulatoryadvice,becauseexisting for (EMA) connection Europe the to went rather they #2, Expert of company the of case In - large pharma companies using digital health (E #2)companieslarge pharma health - digital using validation(E companies #2)digital health experiencedin - em 4. 5. s to be overregulated: beoverregulated: tos

“ “ “ “ certification process. And they goto hand need process. incertification hand. custo the or patient the to solution the bring I do (how model business process etc process approach the right people, to get into the clinics, to get into the no rules how to go through the process. rulesgothe to through how no before,seenso there not coming have they up, technology alwaysnew is a There they know already aboutus they t with relationship working a established already We ask. to directly EMA “ it is is thefor of bad speed it innovation. There is a very close link between the technological solution (what itdoesd the technologicalsolution between very close link a is There And And this is one of the expectations in the program, to help them to do th Well, Well, what is always the case is that the regulatory bodies lag behind No, when this topic came up or when we decided to take this path, this take to decided we when or up came topic this when No, And in Europe, we tend to overregulate everything, which is good for Policies (regulatory, insurance) in digital health validation (regulatory,validation indigital insurance) Policies health Financing and validation in Austriavalidation and in Financing s lag mightinevitable,Expert lag phrased it, #1 as be .” (E #1) (E .”

.” (E #2) (E .”

”#1) (E 62 ” (E ” #1)

”#1) (E start-ups depends on the on depends start-ups European certification

an Medicine Agency Medicine an Europe this area we went to the to went we the technology. the patient, but y: y: mer) and the the and mer) e right thing, hem, there there hem, o) the to are Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. E #1 mentioned during the interview, that ofclinical that valid resourceinterview, #1 during mentionedthe E needs are the Exceptions govern the by reimbursed is everything that expects doctors in and system, patients insurance Austrian health social the to Due Austria. in so particularly which force, driving economic big a is reimbursement concluded, #1 E As marketing on case in model, resourcesshouldAccording toE ofover the #1, b counter the Agency could be a potential source ofsuch source funding.bea could potential Agency Au the that speculated, but topic, this for opportunities funding know not did #2 Expert Additionally, yet. t discussed not was works, validation #2 Expert where company health digital the of investors the Among Roche). wasby acquired MySugr, which brought of example(USP) (she the any external help during the process. external process. during help any the the process is unclear and bureaucratic in Austria. He also confirmed, th Applying for reimbursement is also a topic at the company of Expert #2, and Another challenge might be lackmightofexperienthe be challenge Another 6. 7.

maybe within KLIPHA. maybe within Promotion Agency)) and I think they could be a partner in this, to support suc Research (Austrian Forschungsförderungsgesellschaft Österreichische recent experience. It is typical to Austria, it is rather unclear and di Itexperience. recent is it istoAustria,typical rather and unclear “ “ also torecordeverythingright etcthe also way, “ “has to take care of my health”.” (E # (E care tohealth”.” of my take “has “ Because Because it costs a lot of time and it costs a lot of resources, and it costs a So, in Austria in particular, it is a normal attitude of the patattitudeofthe a normal it is inparticular,inAustria So, I do not know about specific grants for it, but we had good experienc It seems to be a complicated, bureaucratic process at the moment, according to ou accordingto moment, the at process complicated,bureaucratic a be to seems It Ecosystem elements supporting application for reimbursement elementssupportingapplication reimbursement Ecosystem for Challenges during validation during Challenges , whereas in other cases validation can be an asset and a unique selling poi selling unique a and asset an be can validation cases other in whereas – rare – private patients. ” (E #2) (E ”

1) 63

ce instart-ups: .” (E #1)(E .”

strian Research Promotion Research strian ient that the government government that the ient at they did not receive about specific public specific about h h clinical validation, ation is very high: is ation fficult in his experience, es with FFG (VSZ: in her opinion is opinion her in eratherfocused e experience, her .”#2) (E lot of energy e oi of topic he ment.

nt r Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. the Austrian ecosystem. Austrianecosystem. the be could beds test regulatory that interview, the during mentioned #1 E

Resistance of health carof health Resistance As she concluded, these test beds could support the system at several l supportcouldsystem beds concluded,the at she these As test the ecosystem creates a challenge while navigating it. createsecosystema it. while the challengenavigating Interference of based politics: on the experience of Expert #2, the

“ “ “ “ Affairs if in not do fail, buy they expertise. system and and obsolete. become system squeez be can they that be might it because solution, all of afraid were do it differently” attitude in the healthcare system everywhere on every “ I would say, that the most challenging is to make the doctors accept these these accept doctors the make to is challenging applications most the that say, would I ee rpae hi kolde n eprec’ S, hy o o lo a such and replacecould threat them at look not do some they rather but easier, work their make can that tool, a So, as application experience’. and knowledge their replace never “ because there are not a lot of nations did nations who that oflot there a because are not where digital innovations can thrive and also create the halo and mak ecosystem, an creating system, the in money saving everything: up speed really and tik n o te oan te (S: eea Mnsr fr iia ad Eco and Digital for Ministry Federal (VSZ: they domains the of one think I Those people, who approach this problem in a very naive way, they most likel most they way, naive very a in problem this approach who people, Those Because they (VSZ: digital health companies) companies) health digital (VSZ: they Because If you were asking about the most challenging part, but without listing Some (VSZ: doctors) are really negative and say and negative really are doctors) (VSZ: Some I think that (VSZ: test-beds) test-beds) (VSZ: that think I International comparison comparison International ) should really focus on is eHealthshouldreally on is regulatory beds. focus test .” (E #2) e users was mentioned users was by both interviewees: sh ”#1) (E .”#2) (E ould be a focus area, to help and create anecosystem create and help to area, focus a be ould ”#1) (E

were stumbling over the “we used to used “we the over stumbling were ‘a digital health applicati health digital ‘a .” (E #1)(E .” interference of politic with

” (E ” #1) evel: evel: an improvement of improvement an

e Austria visible, any examples, thing that is a a is that thing level: doctors ed out of the of out ed on should on nomic nomic y will y Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. ecosystem elements ecosystem on information public available on based added were members Additional Health content and Delivery channel: Health • • modelinthe and were through review put literature interviews countr the in gaps potential and strength identify in ecosystem validation health digital the of elements the collects section This 5.3.

Digital health innovation: healthinnovation: Digital • Digital healthstrateg Digital

In families” caregivers patients, for relevance high with knowledge generate and society ambi the teams with interdisciplinarytrans-sectoral and for grounds fertile areas, under-researched currently these in results excellent deliver to was established in 2019 together with the LBI C [S: elh ae rfsinl] (I) sitn decision- assisting (III) professionals]; care health [VSZ: HCP digital tools that will enhance patient safety by: (I) empowering pa Safety: Patient institute located at the and Medical University of Vienna aims Health Digital for (LBI) Institute Boltzmann Ludwig EL as of 05.06.2020). as ways. They also delegate mentors to the accelerator program of Health Hub Vienna (sta Expert #2, these companies can support digital health start-ups and SM As Austria. of scene science life the of part are companies pharma large Large pharma companies active in Austria (Boehringer Ingelheim, Sanofi, Pfizer,

stitut stitut for Digital Health and Patient Safety | Ludwig Boltzmann Gesellschaft GA: the Austrian Patient Record System, ELGA is described in details inChapt the GA: Austrian RecordPatient System,ELGA inis described details Aust (MedizinischeInnsbruck, Universität 2018, p. r ian digital . y: These members are:These members

he alt h vali dat ion e 65 cosy

for Digital Health and Prevention, “not only y. st Ecosystem members identified by the by identified members Ecosystem e m

1) “to develop, and later introduce, .

established in Chapter 4 establishedinChapter making” tients; (II) empoweringtients; (II) , n.d.) Es through various ( uwg Boltzmann Ludwig u t provide to but the respective respective the h research the mentioned by mentioned tion to impact to tion . The institute Austria, to Austria, Novartis): er 2.4. 2.4. er and their and .

te Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Public funding: Public https://www.foerderpilot.at/foerderstellen.html 18 17 16 05.06.2020) 15 06. 2020) 05. on 14 • • • •

The actual list of all partners behind Förderpilot is available at: at: available is Förderpilot behind partners of all list The actual German in is platform at German in available is initiative the on information Limited https://www.ffg.at/quickcheck https://www.ffg.at/foerderservice website after opinion her or his submit can interested Anyone city”. health Vie how on view public and ideas submit to platform methods provided by the Agency. One is the Förderservice the is One Agency. the by provided methods validation. To identify funding opportunities at FFG fitting a c of funding public of source potential as mentioned was Agency Promotion Research Austrian or FFG the (FFG): Forschungsförderungsgesellschaft Österreichische Digital Digital Health in Wien: as part of the HEALTH.DigitalCity.Wi discuss funding possibilities. The second is the QuickCheck the is second The possibilities. funding discuss options. options. the on feedback expert receives and sketch, project brief a and costs) and start planned title, project (e.g. information basic submit can one ödrio: cnrlzd nie ltom o id rns las bn gu an bank loans, grants, find to investment, incubator/accelerator and coaching opportunities pro platform online centralized a Förderpilot: the project the about information basic provide can one recommendation; personalized QuickCheck option at FFG at option QuickCheck where background, the in experts by search personalized a is option other The background. background. partner all from come will information the here however time, of period od zr ödrn dr isncatihn oshn (W) te W (r Austri (or FWF Science Fund) funds non-commercial clinical research in the Programme Cl the (FWF): Forschung wissenschaftlichen der Förderung zur Fonds d regional institutes in Austria in institutes regional d form 15 . The inputs will be used to develop a digital healthstrategyinputs the for citwill . The todevelop be used a digital of a drop-down menu) and get immediately an automatic selection of options of selection automatic an immediately get and menu) drop-down a at https://www.partizipation.wien.at/de/consultation/8900 last accessed on 05.06.2020. 05.06.2020. on accessed last

last accessed on 05.06.2020. 05.06.2020. on accessed last – one receives feedback and recommendation after a short short a after recommendation and feedback receives one 18 . There are two possibilities on the website to receive to website the on possibilities two are There . (last accessed on 05.06.2020) 05.06.2020) on accessed (last 66

https://health.digitalcity.wien/ nna can become a leading “digital leading a become can nna en 17 16 ertain topic, there are 2 easy 14 :

– it is an online form, where form, online an is it initiative there is an online to contact the agency and agency the contact to vided by many national (last accessed on on accessed (last registration on the on registration end, approximate end, institutes in thein institutes inical inical Research itn funding fitting (last accessed accessed (last digital health digital y. y. – arantee, like the like (in an . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Services supporting Servicesvalidation: on 05. 06. 2020) 06. 2020) 05. on 23 05.06.2020) 22 21 =13457 finder/?search%5Bwhat%5D=&search%5Bscience_discipline_id%5D=&search%5Bpro 20 14.06.2020. on accessed 19 • •

• Infrastructure: Infrastructure:

The website of the institute is available in German language at at German language in available is institute of the The website at can found be partners actual of the List fromGerman translated Information https://pf.fwf.ac.at/en/research- https://www.fwf.ac.at/en/research-funding/fwf-programmes/programme-clinical- Module for 3D Reconstruction, 3D forproject Module number: KLI 678). identified (as of to that in the database the in that It proposal. project the on reviewers external of evaluation positive the no (only services or products health digital of validation clinical the clinical practice and patient treatment” patient and practice clinical “ge to need Projects (KLIF). LBI for Digital Health and Prevention and Health Digital for LBI of Vienna is described in Chapte Vienna Hub Health of role the Vienna: Hub Health at Mentors ecosystem members were identified. membersidentified. ecosystem were DEXHELPP prs eiie Peeto ad eaiiain n azug is to aims Salzburg in consequences risksand the reduce to changes lifestyle of sustainability Rehabilitation and Prevention Medicine, Sports rdcs n bcm a iprat lmn o te utin iia hat vali health ecosystem. digital Austrian the of element important d an of become benefit and products economic calculate to help could DEXHELPP health, digital of development the Through interventions. different on based scenarios and new data to analyse the current status quo, forecast future d usi by achieved is This systems. care health of control and planning analysis, support the development of a digital health solution (Virtual and Au and (Virtual solution health digital a of development the support last accessed on 02.06.2020. 02.06.2020. on accessed last 21 wrig ih eea prnr institutes partner several with working : 02 .0 6.2020) of the funded clinical trials no digital health validation project w project validation health digital no trials clinical funded the in

-practice/project- 20 nerate new scientific knowledge and insights that improve improve that insights and knowledge scientific new nerate at . Only Only one project was found, where clinical trial was funded

www.dexhelpp.at r 2.4. 2.4. Among the mentors in the accelerator program, key http://www.dexhelpp.at/en/partner 23 : the institute hosted by the University Institute of Institute University the by hosted institute the : 19 67 to be supported. The fund would also finance also would fund The supported. be to

(last accessed on 05.06.2020) 05.06.2020) on accessed (last https://www.lbg.ac.at/lbi- 22 DXEP am t support to aims DEXHELPP , in the digital health scene health digital the in s/ evelopment evelopment and develop (last accessed on on accessed (last research-klif/ motion_category_id%5D n-commercial), after n-commercial), of cardiovascularof gmented Reality gmented dhp new models for models new must be noted, be must (last accessed accessed (last mrv the improve igital health health igital ng existing existing ng last last dation as Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. zentrum/zentrum-fuer-praezisionsmedizin/ 24 •

• • Policies, legislation: Policies,

Information translated from German from the project website: website: project from the fromGerman translated Information diseases Legal and regulatory framework for conducting clinical testing: Austria, clinicaltesting: conductingforregulatory and framework Legal to be similar to a cardiovascular disease prevention and management bed. management test adiseasesimilarcardiovascularand prevention to be to Ins the of descriptions available publicly in mentioned explicitly individ healthy for even potentially and patients other for out an units rehabilitation cardiological outpatient in patients involve will Medicine and Therapy and Therapy and Medicine Centa (togetherwith in2022 Vienna in Medicine Precision for Centre a is there Medicine): Precision for (Centre Präzisionsmedizin für Zentrum and and existing digital techniques and tools in the model region of Salzburg i concurrently and develop to partners consortium several with together studies, studies, genome technology, bioinformatic and IT research, biomedical on is the focus where Medicine, Century 21. the for location data of clinicians, nurses involved). nurses of clinicians, data involved). parties all for applicable is GDPR trials, clinical during protection data Regardi 2020). Care, Health in Safety for Office Federal (Austrian collected are devices centralized information page where all applicable and laws regulations h rel (BASG)) Gesundheitswesen im Sicherheit für (Bundesamt Care Health in Safety f Office Federal high Austrian The at 2014). Commission, but an (European transparency trial, increased create clinical conduct to to favourable aims is regulation that environment, new This 2014). Commission, Regulation 536/2014)(European Trial Clinical EU new the apply will and adopted has the planning andplanning the researching publicthe healthcareIts inAustria. Österreich GmbH (GÖG)): the Austrian National HealthPublic Insuran Insura Health Public National Austrian at Innovation and Health Digital health validation ecosystem.health tri clinical run (to element infrastructural important an become could (Niebauer J & Stütz T, 2019)

a Centre for Technology Transfer). The aim is to establish a establish to is aim The Transfer). Technology for Centre (last accessed on 05.06.2020) 05.06.2020) on accessed (last . During the 7 years of the project, the institute will work 68

24 . https://www.zpm.at/de/d Based Based on this information, this Centre titute, the project seems project the titute, three business unitsthree are business as Atog i i not is it Although uals. Rglto (U No (EU) (Regulation ce is responsible for re for Translationalfor re involved (including involved d if successful, roll successful, if d as part of the EU of the partas . als) of the digital the of als) nce (Gesundheit nce First, the project as ated ated to medical mplement new new mplement s sft and safety est plan to build a build to plan - clinical s a as ng or , Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. • selbsthilfe.at/leuchtturmprojekte/ 28 05.06.2020) on accessed (last 27 26 25 •

Patients Representatives: Patient and Patients

Translated from German from the Association website at at website Association the from fromGerman Translated at German in available is project the of Description from Information at available chart Organization Hub Vienna Hub Commission, 2019b) Commission, Austria literacy, digital high relatively continues to invest into this the development of digital skills Despite of broad citi average). EU the as 58% (versus skil digital the mentioned, 66% of citizens between 16 essentially and 74 years of age have covers at least b which part, capital human the on aim aim is to bring Patient Organizations and ELGA GmbH. As stated on the website of the Associ channel communication direct a form to and system ELGA the about opinion righBVSHOEhas member the organizations: representativeitsa as of project Health, Care and Consumer Protection. The Association is already involved Sozialversicherung (Austrian Social Insurance) and by the Federal Ministry b funded organization not-for-profit a is It Austria. gr in Organizations Self-Help specific disease for organization umbrella an is Austria Self of Association Federal the (BVSHOE): Österreich Selbsthilfe Bundesverband and Innovation directly reports to the Executive Board knowle increase o Head The health. digital in to expertise special a have they website, their projects mention for As Fund). Promotion Health support Austrian (The networks financial and competences, providing for and Care) regist f Institute andNational Austrian (The care standards health quality high developing maintain for Research), Services Health Aus for (The Institute analysis data through facilitation innovation for responsible report in 2020 (European Commission, 2020), Austria is above the EU average (rank 9 (rank average EU the above is Austria 2020), Commission, (European 2020 in report member states with the Digital Economy Society and Index (DESI). According to th digit the monitors Commission European the citizens: of literacy Digital 26 https://healthhubvienna.at/ . GÖG has performed an economic evaluation of Austria telemedicineof evaluation anin economic performed . GÖGhas the the patient’s view .

https://goeg.at/Organization_Chart (last accessed on 05. 06. 2020) 2020) 06. 05. on accessed (last

in how ELGA works

(l ast accessed on 05.06.2020) 05.06.2020) on ast accessed https://goeg.at/Telegesundheitsdienste_oekonom_Eval 69

https://www.bundesverband- . This This mission is very much in line with 25 (last accessed on 05.06.2020) 05.06.2020) on accessed (last and is listed as a mentor of Heal zen zen groups (European y the Österreichische the y or Quality in Health in Quality or us n Patient and oups s f iies As citizens. of ls of Social Affairs, asic digital skills f Digital Health Digital f in digital health lzto o EU of alization trian National National trian ation t to express express t to between 28 e latest is to ries d on ed , their -Help dge, 27 th th .

)

The summary of the Austrian digital health validation ecosystem is inecosystem Figurehealthsummaryis validation AustriantheThe of digital

Information available in German at at German in available Information incorporate incorporate ae rvdr, eerh n ntoa ad nentoa pam companies pharma international and national and research providers, care between intermediary an as acts Groups Help Organi Patient Disease Rare for organization umbrella the Additionally, providing patient focusedproviding criteria. di of validation and development support can organizations patient how ProRare: ProRare (Alliance for Rare Diseases) Diseases) Rare for (Alliance ProRare ProRare: oiin f h Alac mks t srn prnr n h eoytm of ecosystem the validation. in partner strong a it makes Alliance the of position the patient’s view into the developme the into view patient’s the

https://www.prorare-austria.org/projekte/strategiepapiere/ 70

pa su tients and politics, authorities, health health authorities, politics, and tients prs h iiitv o BSO, to BVSHOE, of initiative the pports nt of ELGA (Pro Rare Austria, 2019) Austria, Rare (Pro ELGA of nt zations and Self- and zations 15 gital health by health gital iia health digital .

last last 29 This . . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Figure Figure

15 d) c) b) a) Members ecosystemMembers validation the Austria in digital of illustra health (own Collection nationalof and EUand laws regulations for medical deviceavailable https:/ at Complete list Austrianof hospitals involved in published digital health is trials in Chapter 5.1. https://www.bundesverband-selbsthilfe.at/and https://oekuss.at/selbsthilfelandschaft_oesterreich ÖsterreichiseSozialversicherung tion) . *Elements by literature review, identified 71

/www.basg.gv.at/en/about-us/legal-basis#c12785 ** Elements identified from expert from expert identified interviews Elements

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. jo how in experienced ii) institutes), research academic (mainly journals res the publishing in interested i) are who identified, be liter can members scientific of review the with that is limitation third The last accessed on 14.06.2020. 14.06.2020. on accessed last 30 Austria (MySugr in suc one only (probably validation proper did who companies,many not are in both clinical and economic validation of digital health is prob reimb experience in clinical use, but not in for validation. However, accessibility applying of process the in is #2 actuall Expert who validation; economic interviewed, was expert no that is, limitation second The inthe network Austrian healthindustry. digital probably has she and reimbursement) or testing clinical through went However, one of the experts has worked with several digital health compan th of elements identify to interiewed experts 2 only were there hand, one On haslimita study Austrian digital of the healthecosystem validationThis 5.4.

term covers highly different technologies, products and services, which health digital approached it that is study the of limitation a Finally, institutes).published, particularlyby rarely not non-academic out out health. categories disjunct of perspectives the from questions research same the answer consequently may become indifferent to economic Furthervalidation). studi Diabetes management app developed in Austria, acquired by Roche in 2017 in Roche by acquired Austria, in developed app management Diabetes urnal and finally iii) satisfied with the results of the clinic the of results the with satisfied iii) finally and urnal –

ma L im y not require clinical testing and may not have a chance for reimbursement for chance a have not may and testing clinical require not y ita t ions 30

)).

al testing (as negative results are results negative (as testing al lematic even globally, as there tr ol toe ecosystem those only ature https://www.mysugr.com/en/ in general, however the the however general, in to to experts experienced

– tions. tions. to publish a scientific a publish to ults in peer-reviewed peer-reviewed in ults as Expert #1 pointed reet n has and ursement n o te largest the of one dd lncl and clinical did y ies ies (some of those h company exists company h es are needed to e ecosystem. e of digital digital of (and

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Gaps: network strong organizations. of patient representative Austri of literacy digital high by strengthened is ecosystem the Finally, EU. inthe requirements the on the of outside from coming start-ups for attractive particularly be can the of stakeholders relevant the of most with network and contact (par companies health digital for one-stop-shop a is it Vienna; Hub A remarkable element of the ecosystem is the platform and accelerator program of th a gie iia hat cmais einn ter aiain strate validation ofgood-practic ininternational involvement helps import projects the their designing companies health digital guide can for validated innovations; these activities help building experts on well as their active involvement in the clinical testing of digital in research and universities the of activity development and research strong The developers to find financial support for the validation easily support validation thefor find developers to financial The presence of centralized online platforms on the various public from further from development. This study identified two major gaps and a few segments, where c the ecosystem comp external intointegrated become ELGA) by developed modules (if innovations new of validation the for highconsistent, serve could and asquality deliv asourcefor data a good become Health Electronic The ecosystem.the of development furtherfor support o city digital leading a become to Vienna capital, the of initiative country. the in health digital deploying on strategy comprehensive and The Austrian digital health validation benefits ecosystem from a strong nati Strengths: practicesgood strengthenfurther to international and ecosystem propose it i identified gaps and strengths the conclude will chapter This 6.

Di s c u s s i on

health can form a fertile ground te utin iia health digital Austrian the n the the field, their publications financing opportunities aid f the world could warrant could world the f validation ecosystem. It It ecosystem. validation es an citizens and by the the by and citizens an EU to get information get to EU iual satus to start-ups) ticularly y n fnly their finally and gy to Record system cansystem Record onal onal commitment Austria. diinly the Additionally, . . ould ould benefit ery channel channel ery stitutes, as stitutes, ne can anies e Health Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. 31 • inAustria. ecosystem Based on the literature overview in Chapter 2, three elements were identifi One clear gap is the lack of pha instit the a of establishment the identified trials, clinical was such on specialized Medicine Precision for Centre the Although health. secondinA gapis infrastructure necessary the thefor clinical testingan c health digital couldhoweverstudy this not as validation evaluate support their well, will in investors current that imagine to easy is it However, Th internallyto test aims solutions.developed clear, not ifit is stillopen is however platform the external for Potential good-practices to support the Austrian digital healthvalidati supportAustriangood-practices the to digital Potential ecosystemvaluablecompanies. could the on digital for be health Furthermore, the lowering of bureaucracy and the reduction of the im as health digital towards doctors of resistance i the mentioned interviewees It validation. health digital towards workers healthcare of training reimbu for application the aiding (particularly Validation for •

Similar to the NHS app library ( library app NHS the to Similar e ecosystem could be strengthen be could ecosystem e Testbeds: make Austria even more attractive for digital health innovators (Arntzen et(Arntzen evenmoreal Austria makehealthinnovatorsattractive digital for Addition implementation. successful of likelihood higher with mean, that the scene supports the development of new innovations wit efficien more and easier health digital of validation make to elements a collection of “ of collection a th through journey their establis with supplemented be could in framework evidence This 2019). process(NICE, developers help could reimbursement for necessary health: adescribingeviden framework the pragmatic UK, by indeveloped the NICE framework digital for standard evidence National utin institutes Austrian se. Additionally, the LBI Digital Health and Patient Safety seems to be a s xet 1 etoe, elwrd etes ol b ueu infrastruc useful be could testbeds real-world mentioned, #1 Expert as approved 31 Ti cleto cud ud ptet ad elhae providers healthcare and patients guide could collection This . – https://www.nhs.uk/apps-library/ visible

” digital health s health digital ” – ed private investment options financing validation process.

by developing additional Standards and Frameworks and Standards additional developing 74 olutions

iia t te vdne standard evidence the to similar (particularly wearables and apps) by apps) and wearables (particularly ) last accessed on 14.06. on accessed ) last smn) n yte targeted the by and rsement) digital health productsor only digital ute is still in the planning the in still is ute ly sc tsbd could testbeds such ally, pact of political debates on ecosystem: d validation d of digital srkn, ht both that striking, s real-world testbed, a oeta site potential a s ingness. ingness. h h higher value and ed, ed, to support the t. This would also would This t. mais would ompanies 2020. a key challenge. challenge. key a

., 2019) ., hing . ture

ce e Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

the necessary validation to getintocollection. validation thenecessary to the require necessary the fulfil to companies health digital for incentive Additional solutions. health digital validated and valuable selecting 75

ments and perform and ments y i cud e an be could it ly, Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Austrian Federal Office for inSafety Care. Health (2020). Australia’sNational Digital Health Strategy | NATIONALDIGITAL HEALTH Arntzen, S.,Wilcox, Z.,Lee, N., Hadfield, C.,Rae,J., Hadfield, C.,& Rae, J. (2019). Brath, H., Morak,J., Kästenbauer,T., Modre-Osprian, R., Strohner-Kästenbauer, H., Schwarz, M., Biggs, J. S.,Willcocks, A.,Burger, M., & Makeham, M. A. health B.(2019). Digital benefits evaluation Bergmo, T. S.(2015). How to measurecosts and benefitsof interventions: ehealth An overviewof Alexander Walter. (2013). Uphill PathThe to Successful Clinical Trials. Accenture. (2018). B

Ammenwerth, E., Woess, S.,Baumgartner, C., Fetz, B., der van A., Heidt, Kastner, P., Modre-Osprian, Cohen, A. B.,Cohen, Mathews, S.C., Ray Dorsey, E., Bates, D. Safavi,W., & K. (2020). Direct- CHF andGeorge The Institute for Global Health. (2018). Cerreta, F.,Ritzhaupt,A., Metcalfe, T., Askin, S.,Duarte, J., Berntgen, M., Vamvakas,& S. (2020). Craig, P., Katikireddi, S. V., Leyland, A., & Popham,Natural F.(2017). Experiments: An Overviewof Coly, A.,& Parry, G. (2017). ib liog strategy May 19, 2020, from https://conversation.digitalhealth.gov.au/australias-national-digi real-world/ in the Real - World Real-world testbeds me Kort, W., Schreier,& G. (2013). Mobilehealth (mHealth)based medication adherence 397. https://doi.org/10.3414/ME15- Medical Journal ofAustralia frameworks: buildingevidence the to support Australia’s National Digital Heal InternetMedical Research. https://doi.org/10.2196/jmir.4521 methods and frameworks. In https://www.basg.gv.at/en/about-us/legal-basis#c12785 Clinical Pharmacology Team-Patients-Doctors-Machines.pdf#zoom=50 https://www.accenture.com/_acnmedia/PDF-71/Accenture-Health-Meet-Todays-Healthcare- system patients in with oronaryheart disease. R., Welte, S., & Poelzl, G. Evaluation(2015). of anintegrated telemonitoring surveillance 38 - a- health policy report Discovery technologiesDigital for medicines: shapingframework a for success. In digital health. Methods, Approaches, and Contributions to PublicHealth Intervention Research. Designs 7500(20)30057-1 (4), 225 asurement- a pilot trial using electronic blistersin diabetespatients. ra phy . https://www.academyhealth.org/files/AH_Evaluation_Guide_FINAL.pdf . Nature Research. https://doi.org/10.1038/d41573-020-00080-6 – 227. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3684189/?report=classic

Accenture 2018 Consumer Survey on Digital Health The Lancet Digital Health , Evaluating Complex Health Interventions: A Guide to Rigorous Research . https://chf.org.au/sites/default/files/gdigital_report_j060618.pdf 76 (S1), 47 , 210 Journal ofMedical InternetResearch (S6),S9 – 55. https://doi.org/10.1111/bcp.12184 02 -0002 . https://www.nesta.org.uk/report/testing-innovation- , – 2 S11. https://doi.org/10.5694/mja2.50034 (4), e163 76

Methodsof Information in Medicine

Going Digital to deliver a healthierAustralia – Legal Basis e165. https://doi.org/10.1016/S2589- . . Pharmacy and Therapeuti (Vol. Journal17, Issue 11). of STRATEGY Nature Reviews Drug British Journal of Testing InnovationTesting th Strategy. . (n.d.). Retrieved to -consumer Annual , 54 tal-health- (5), 388 cs ,

– Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

de Bruin,de J. S.,Schuh, C.,Seeling, W., Luger,E., Gall, M., Hütterer,E., Kornek, G., Ludvik, B., European Commission. (2012). ELGA: About Health.EIT (n.d.). eHSG SubGroup Reimbursement.on (2019). Digital Health Testprogram Beds | NATIONAL DIGITAL HEALTH STRATEGY European Commission. (2014). Regulation (EU)No 536/2014 of the European Parliament and o European Commission. (2019a). European Commission. (2016). European Commission. (2020). European Commission. (2019b). FDA. (2020b). FDA. (2020a). FD Federal Chancellery and Federal Ministry Scienceof Research and Economy, Digital Roadmap Austria A. (2019). Review ofPublic Health Syntax. clinical decision support system for nutritional triagein oncology outpatients using Arden Hoppichler,F., Schindler,& Assessing (2018). K. the feasibility a of mobilehealth- 044327 ce https://eithealth.eu/project/living-labs-and-test-beds/ content/uploads/2019/04/30042019_eHSGSubGroupReimbursement.pdf#page=14 Digital Health Productsand Solutions 2020, from https://conversation.digitalhealth.gov.au/digital-health-test-beds-prog https://doi.org/10.1016/j.artmed.2015.10.001 2001/20/EC. Council clinicalon trials on productsmedicinal for human use, and repealing Directi manufacturersand notified bodiesunder directives 93/42/EECand 91/385/EEC 1/reg_2014_536/reg_2014_536_en.pdf https://ec.europa.eu/health//sites/health/files/files/eudralex/vol- https://ec.europa.eu/digital-single- (Country reportAustria) files/Digital_Government_Factsheets_Austria_2019_3.pdf https://joinup.ec.europa.eu/sites/default/files/inline- regulation/classify- https://www.fda.gov/media/129047/download (Vienna, 2016) 510k/content-510k#link_12 ntury . (n.d.). Retrieved June11, http://www.elga.gv.at/en/about-elga/from2020, . https://eur-lex.europa.eu/legal-content/HR/ALL/?uri=CELEX%3A52012DC0736 Software Precertification Program 2019 UpdateMid-Year Artificial Intelligence Medicine in Classify Your Medical Device Content ofa 510(k) Living Labsand TestBedsEIT- Health Official Journal oftheEuropean Union yo ur , -medical-device . 38 eHealth Action Plan 2012-2020- Innovative healthcare for the 21st Guidelineson Medical DevicesClinical- Evaluation: Aguide for Digital Economyand Society Index(DESI) 2020 Austria (1), (1), 39 Di Monitoring progress in national initiativeson digitising industry . https://www.fda.gov/medical-devices/premarket-notification- gitalGovernment Factsheet 2019 Austria – 56. https://doi.org/10.1146/annurev-publhealth-0318 ma . https://www.fda.gov/medical-devices/overview-device- . https://www.medtecheurope.org/wp- rket/en/scoreboard/austria Proposed Guidingfor Principles Reimbursement of , 92 77 , 34

– 42. . Retrieved May 19, 2020, from , L L 158/1 . . . (n.d.). Retrieved May 19, . . . supported ram ve 16 f the - Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Federal TradeCommission. (n.d.). Federal Ministry for Digital and Econom

Gathani, R., Cowlishaw,& S. (2019). Federal TradeCommission. (2013). Frühauf, J., Kröck,S., Quehenberger, F., Kopera, Fink-Puches,D., Komericki, R., P., Pucher, S., Forum LLSAForum Accueil > FocusGroup on “Artificial Intelligence Health.” for Ferretti, A.,Ronchi, E., Vayena, & E. (2019). From principles to practice: benchmarking government Goldsack, J. C.,Coravos, A., Bakker,J. Bent,P., B., Dowling, Fitzer-Attas, A.v., C., Godfrey, A.,Godino, Gordon, J., W. Landman, A.,Zhang, H., & Bates, D. W. (2020). Beyond validation: getting health Hayn, D., Kollmann, A., Perl, S.,Kos, C., Rotman, B., Lercher, Tscheliessnigg,P., K. H., Schreier, & G. Granstrand, O., Holgersson,& (2020). M. Innovation ecosystems: Aconceptual review andne a and-social-affairs/>[accessed 20 June2020]. Austria” .

on – -the-qualification-and-classification- 6. https://doi.org/10.1038/s41746-019- (Vol. (Vol. https://doi.org/10.1016/S2589-1). - Journal ofthe European Academyof of -action-and-measures/health-care- Journal of Telemedicine and 020 -0260-4 logies (BioMeTs). alth-claims le-app-get- . the the of - w apps n it - Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Hillestad, R., Bigelow,J., Bower, A., Girosi, F., R.,Meili, Scoville,R., & Taylor, R. (2005). Can electr Kowatsch, T., Otto, L., Harperink,S., Cotti, A.,& Schlieter,H. (2019). Adesign and evaluation S. Kotzian, T., Schwarzinger,A., Haider,S., Saletu, B., Spatt, J., & Saletu, M. T. (2018). Home Istepaniana, R.S.H., AlAnzi,& T.(2020). Mobilehealth (m-health): evidence-based progress or IQVIA Institute for Human Science.Data (2017). SoftwareIMDRF as a Medical DeviceWorking Group. (2017). SaMDIMDRF Working Group. (2013). Kollmann, A.,Riedl, M., Kastner, Schreier,P., G., & B.Ludvik, (2007). Feasibility of phonea mobile Keesara, S.,Jonas, A., Schulman,& (2020). K. Covid- Jandoo, T. (2020). DIGITAL HEALTH WHOguidance for digital health: What it means researchers. for J Chen. (2018). Kellermann, A. &L., Jones, S. S.(2013). What It Will Take To AchieveThe As-Yet-Unfulfilled Promis Kreiner, Kreiner, K., Welte,S., Modre-Osprian,Fetz, R., B., Heidt,A., Kropf, Ammenwerth,M., Pölzl, E., G., & Jörg Schickertand others, Health “Digital Solutions to BecomeReimbursable Germany– in Health Affairs medical record systems transform health care? Potential benefits, health savings, and costs. frameworkfor digital health interventions. BMJ Open inapnoea stroke (HOPES Study): study protocol forsingle-blind, a randomised controlled tri polygraphic recording with telemedicine monitoringdiagnosis and for treatment sl of scientific retrogression.scientific In Feng D. D. (Ed.), https://www.iqvia.com/insights/the-iqvia- Clinical EvaluationGuidance- Industry for and Food and Administration Drug Staff 140901.pdf http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd of Medical Internet Research based dataservice for functional insulin treatment of type diabetes mellitus1 patients. https://doi.org/10.1377/hlthaff.2012.0693 England Journal ofMedicine in Digital HealthDigital 2018%2BDissertation%2529.pdf&type_of_work=Thesis n%2BHealthcare%2BOperations%2B%25281010130%253B%2B2017- 0b7e17e0e62f/download_file?file_format=pdf&safe_filename=SysReview%2Bof%2BABM% Quality and Implementation Academic Press. Of Health InformationOf Health Technology Care Transformation. Kastner, P. (2015).A Personalized FeedbackSystem for Supporting Behavior Change for https://doi.org/10.1515/itit-2019-0019 [accessed 1 June2020]. Agent-based Modelling Healthcare in Operations: ASystematic Reviewof its Scope, , 8 , 18847. https://doi.org/10.1136/bmjopen-2017-018847 , , 24 6 , 1 (5), 1103 – 4. https://doi.org/10.1177/2055207619898984 – 1117. https://doi.org/10.1377/hlthaff.24.5.1103 . https://ora.ox.ac.uk/objects/uuid:9b71bc9f-61c1-4485-93b2- , , 382 9 (5). https://doi.org/10.2196/jmir.9.5.e36 Softwarea as Medical Device (SaMD): Key Definitions (23),e82(1)-e82(3). https://doi.org/10.1056/nejmp2005835 egulation,” 2019 in The Growing Value of HealthDigital Biomedical Information Technology ITInformation- Technolo 79 stitute/reports/the-growing-value- 19 and Health Care’s Di

Softwarea as Medical (SAMD): Device Health Affairs gy gital Revolution. to , , 32 -become-reimbursable- 61 (1), (5 -key-definitions- – 6), 253 63 (pp. 717 . of – -digital-health 68. . – Milestone eep 263. . New – Journal 733). onic onic 2Bi al. - es Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Ludwig Boltzmann Institut LeFevre, A.E., Shillcutt, S. Broomhead,D., S.,Labrique,A. B., Jones,& T. (2017). Definingstaged- a Mayring, (2014).P. Maguire, R., Fox, A.,P. Mccann, Miaskowski, L., C., Kotronoulas, G., Miller, Furlong,M., Ream, E., E., Marinschek, S., Dunitz-Scheer, M., K.,Pahsini, Geher, Scheer,& B., (2014).P. Weaning child Mathews,S. C., Mcshea, M. J., Hanley, C. Ravitz,L., A.,Labrique,A. B., Cohen, & A. B.(2019). Digital MedizinischeUniversität Innsbruck. (2018). Mercom Capital(2020). Group. Mercom Capital Group. (2019). Morak,J., Schwarz, Hayn,M., Schreier, D., & G. (2012).Feasibility of mHealth and Near Field MobiHealthNews, “MobiHealthNews’ Predictions for 2020: Big Tech’s Here to Stay, HealthDigital McNamee, P., Murray, E., Kelly, M. Bojke,P., L., Chilcott,J., Fischer, A.,West, R., & Yardley, L. (2016). 212 Patients afteran Acute Myocardial Infarction. and Allocation Resource based processfor economicand financial evaluations of programs.mHealth 0111-3 Open symptom managementsystem (ASyMS) remotetechnology for patientswith cancer. randomised controlled trial to evaluate electronicsymptom managementusing the K., Harris, J., Katsaragakis, S.,A. Buick, R., & Kearney, N. (2017). The eSMARTstudy protocol: a Armes, J., Patiraki, Gaiger, E., A., Berg, G. Flowerday,v, A., Donnan, P., Mccrone, Apostolidis, P., (n.d.). Retrieved May 26, 2020, from https://www.lbg.ac.at/lbi- Paediatrics and Child Health nutritionenteral by netcoaching versus onsitetreatment: A comparative study. health: ahealth: path to validation. Software Soluti Centered Health Systems https://mercomcapital.com/product/q1- content/uploads/2020/01/MercomDigitalHealthFundingMAQ42019ExecSumm.pdf Annual Report Health-Institutes.pdf 2012 Annual International Conference of the IEEEEngineering Medicine in and Biology Society, EMBS Communication technology based medication adherence monitoring. stay-digital-health-moves-through-gartner-hype> [accessed June7 2020].

, 2 524 (38). https://doi.org/10.1038/s41746-019- -1- 50

dhps

Proceedingsof the it -funding- Cost Effectiveness Journal of ma

advanced BMJ Americ -report/ ren off off ren nd here- . , an . , Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Moshi, M. R., Tooher,R., & Merlin, T. (2018). Suitability ofevaluation current framewor Murray, Hekler, E., E. B., Professor, A., Andersson, G., Collins, M.,L. Doherty, A., C., Hollis, Rivera, D. National Institute for Health andExcellence. Care (2019). NHS England. (2015). Nemecek,R., Huber, Schur, P., S., Masel, E. K., Baumann, Hoeller, L., C., Watzke, H., Binder, M. & NiebauerJ, & Stütz T. (2019). Neues Ludwig-Boltzmann-Institutfür digitale Gesundheitam NICE. (2019). NHS England launch second phaseof Test Bed programme NHS England. (2018b). NHS England. (2018a). NHS Accelerated Access Coll O’Dea, S.(2020). OECD/European Observatoryon Systems Health and Policies. (2019). International Journal ofTechnology Assessmentin Health Care th Prev MedPrev Evaluating digital health interventions: key questions andHHS approaches Public Access. E., West,R., of Health Psychology, Wyatt,P., Director, J. C., F., & FRCGPMurray (2016).FRCP, E. https://doi.org/10.1017/S026646231800051X Health TechnologiesContents programmes/evidence-standards-framework-for-digital-health-technologies cancerand their family caregivers. (2019).Telemedically augmented palliativecare: Empowerment patients for advancedwith Universitätsinstitutfür Sportmedizin Salzburg. in test-bed/ https://www.digitalhealth.net/2018/02/nhs-england-office-life-sciences-lau wave-2-competition.pdf https://www.england.nhs.uk/wp-content/uploads/2018/04/test-beds-supporting learning-from-wave-1.pdf https://www.england.nhs.uk/wp-content/uploads/2018/11/test-beds-programme https://www.england.nhs.uk/aac/what- https://doi.org/10.1007/s00508-019-01562-3 prospectus.pdf innovators 144. Austria.pdf?ua=1 http://www.euro.who.int/__data/assets/pdf_file/0005/419450/Country-Health-P 2019, Stateof Health in the EU https://www.statista.com/statistics/330695/number-of-smartphone-users-world e health e technology assessment of mobile medical applications: Asystematicreview. Evidence StandardsFramework for Digital Health Technologies , . https://www.england.nhs.uk/wp-content/uploads/2015/03/test-bed- 51 Smartphone usersworldwide 2016-2021 (5), 843 Real world testi NHS England Test Beds Programme: Evaluation learning from Wave 1 Test Beds ProgrammeInformation supportthe launch to of wave 2 – 851. https://doi.org/10.1016/j.amepre.2016.06.008 aborative » Test beds . https://www.nice.org.uk/about/what- . ng of “combinatorial innovation” Aglobal invitation to Wiener Klinische Wochenschrift we -do/how-can-the- 81 . (n.d.). Retrieved May 19, 2020, from

German Journal ofSports Medicine Evidence StandardsFramework for Digital . . (n.d.). Retrieved May 19, 2020, from aac , 34 Austria: Country Health Profile -help-me/test-beds/ (5), 464 , 131 we . (23 -do/our- – – 475. 24 nch-second-phase- ), 620 wide/ -information- rofile-2019- ks forks use in -evaluation- – , . 626. 70 . , 143 AmJ – Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Rowland, S. Rowland, Fitzgerald,P., J. E.,Holme, T., Powell, J., & Mcgregor, A. (2020). Whatthe is clinical value Safavi, K., Mathews,S. C.,Bates, W.,D. Ray Dorsey, E., Cohen,& A. B. (2019). Top-fundeddigital Robert Picard. (2017). Rami, B., Popow, C., Horn, W., Waldhoer, T., & Schober,E. (2006). Telemedical supportto improve RainerThiel, Lucas Deimel, Schmidtmann,Daniel Klaus Piesche, Tobias Hüsing, Jonas Rennoch, Veli Pro Rare Austria, PatientenstrategieEine ELGA- Initiative von BVSHÖUnd Pro RareAustria, 2019 Arak,Piotr Anna & Wójcik. (2017). O’Rourke, B., Oortwijn, W., Schuller, & (2020). T. Rahimi, Rahimi, K. (2019). healthDigital and the elusivequest for costsavings. In Rader,S., Thomas, ·, Dorner, E.,Schoberberger, R., H.& Wolf, Effects (2017). of afollow- web-based Sanson- Schickert, J., Goehl, B., Thiermann, A., Wel& Scherr, D., Kastner, P., Kollmann, A.,Hallas, A.,Auer, J., Krappinger,H., Schuchlenz, H., Stark, G., Methods andMethods Too of of formHealth patients? health companies and their impact high-burden, on high-costconditions. x 472 165 glycemic control in adolescentswith type diabetes1 mellitus. stiftung.de/fileadmin/files/Projekte/Der_digitale_Patient/VV_SHS-Studie_EN.pdf digital strategies Stroetmann, & Stroetmann.Karl (2018). . 115 Sy Public Health Research an episode of acute decompensation: randomized controlled trial. telemonitoring using phone mobile technology theon outcome heartof fail Grander,W., Jakl, G.,Schreier, G., Fruhwald,& F. (2009).M. Effect home-basedof in https://www.hlregulation.com/2019/11/15/digital-health-solutions- reimbursable in Germany -germany-milestone-for-patients-and-providers/ stems-Oriented PublicHealth Interventions: Alternative Research Designs. Fisher, R. W., D’E – (10), 701 – 481. https://doi.org/10.1007/s00508- 123. https://doi.org/10.1377/hlthaff.2018.05081 , 11 – (3). https://doi.org/10.2196/jmir.1252 , 705. https://doi.org/10.1007/s00431-006-0156-6 35 . https://www.bertelsmann- (1), 9 Co ls (Robert Picard, Ed.). Wiley ISTE. ste,C. A., Carey, M. Noble, L., N., & C. Paul, (2014).L. Evaluation of -design Living in forLabs Healthcare and Independent Living: Concepts, re – - 27. 27. https://doi.org/10.1146/annurev-publhealth-032013-182445 – Npj Digital Medicine e109).Elsevier Ltd. https://doi.org/10.1016/S2589-7500(19)30056-1 – , 1 milestone for patients and providers |Focuson Regulation – Transforming eHealtha political into and economic advantage 4. https://doi.org/10.1017/S02664623200002154. ter #SmartHealthSystems International comparison of -Birk, T. (2019). 017 The new definition healthof technology 82 ,

-1198-7 3 (4). https://doi.org/10.1038/s41746-019-0206- International Journal of Technology Digital health solutions becometo European Journal of Pediatrics Wien Klin Wochenschr Journal of Medical Internet The Lancet Digital Health to -become-reimbursable- Health Affairs urepatients after Annualof Review cal-and- . , , 38 129 (1), . , ,

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. van Velthoven, H.,M. Carlos Cordon, ;, Helena, M., & Velthoven, V. Sustainable (2019). Adoption of Validic. (2019). U.S. Department Healthof and Human Services, Food and Drug Administration, Centerfor Device Topol Review. (2019). Title Title 21FOOD- AND DRUGS CHAPTERFEDERAL- 9 FOOD, DRUG, AND COSMETICSUBCHAPTER ACT II EuropeanThe Parlament andCouncil of the the European Union. (2017). -Testbed Wikipedia Schusterbauer,V., Feitek, Kastner, D., P., Toplak,& Two-Stage H. (2018). evaluation oftelehealth a van Velthoven, H.,M. Smith, J., Wells, G.,& Brindley, (2018).D. healthDigital app development Sienkiewicz, D., & van Lingen,C. (2017). Sean, D. Sean, D. (2020). VanWinkle, B., Solad, Vaswani, Y., N., Rosner,& B. I. (2019). Navigating theDigital Health Ecosystem Sheon, A.R., VanWinkle, B., Solad, Y., Atreja, & A.(2018). An Algorithm Digital for Medicine Testing: Shana Vijayan. (2019). Internet Research HealthDigital Innovations: PerspectivesFrom a Stakeholder Workshop. ruling-means-for-rpm/ Preface Public Comment Mobile Medical ApplicationsGuidance Industry for and Food and Administration Drug Staff and Radiological Health,B. E. C. for and(2019). R. subchapII-sec321.pdf https://www.govinfo.gov/content/pkg/USCODE-2010-title21/pdf/USCODE-2 repealing Council Directives 90/385/EEC and 93/4 - DEFINITIONS- Sec. 321- https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-201 Directive 2001/83/EC,Regulation (EC) of the European ParliamentandCouncil ofthe of April5 2017 on medical devices, amending patient.eu/globalassets/library/publications/epf_added_value_report_final.pdf and Informatics nutrition management servicein support Diabesityof therapy. standards: Asystematic reviewprotocol. 2 highs Evidence. Bridgeto the Gap from Innovation to Transformation: ANODE.Health Perspective on Digital 2018-022969 Systems Navigate the Trial and Testing Period prior to Adoption. Full-Scale A NODE.Health Perspective Intended to Help Emerging Technology Companies and Healthcare regulation, experimentation, implementation and evaluation. investment-edged-off-record-highs/ (3), 139 . https://rockhealth.com/reports/in-2019-digital-health-celebrated-six-ipos- – What CMS’sLatest Rulin 154. https://doi.org/10.1159/000494365 Di In 2019, digital health celebrated six IPOs as venture investment off recordedged gital Biomarkers . (n.d.). Retrieved May 18, 2020, from https://en.wikipedia.org/wiki/Testbed , 248 Preparing the healthcare workforce deliver to the digital future Test Beds: Digital Health Technologiesin the NHS: The case agile for , 21 , 314 (3), e11922. https://doi.org/10.2196/11922 . https://www.fda.gov/media/80958/download De – 321. https://doi.org/10.3233/978-1-61499-858-7-314 finitions; generally, Pub.21 L. U.S.C. No. 321 (2006). , 3 (2), 83 g Means for RPM The addedThe value ofpatient organizations – 91.https://doi.org/10.1159/000500194 No BMJ Open 178/2002 and RegulationNo (EC) 1223/2009 and 83

. Policy for Device Software Functionsand , . https://validic.com/what-cmss-latest- 8 (8). https://doi.org/10.1136/bmjopen- www.england.nhs.uk Studiesin Health Technology Regulation (EU) 2017/745 9-printable.pdf Journal ofMedical Digital Biomarkers . 010-title21-chap9- https://www.eu- . as -venture- s

, Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Wi Wiegand, T., Krishnamurthy,R., Kuglitsch, M., Lee, N., S.,Salathé, Pujari, M., Wenzel, M., Xu,& S. WHO Global Observatory for eHealth, World& Health Organization. (2016). Wo Bank.World (2012). Working Group on mHealth assessment guidelines. (2017).

Vayena, E., Haeusermann, T., Adjekum, A., Blasimme,& A. (2018).Digital health: meeting the ethical World HealthWorld Organization. (2020). HealthWorld Organization. (2019b). World HealthWorld Organization. (2019a). HealthWorld Organization Executive Board (2017). 142. llner,V., Schneider, C, ; Feichtenschlager, ; M, Willner,V., Research, S.,Jakob, B.H., F.M. & rld Health Organization. (2018). life oflife elderly users. Straße, H. (2015). eHealth Special2015 Issue: Effects anassistanceof serviceon the quality of https://doi.org/10.1016/S0140-6736(19)30762-7 Lancet (2019). WHO and ITU establish benchmarking process artificialfor intelligencein health. In the third global survey on eHealth, 2015 profiles : theof use eHealth support in of universal health coverage : based on the findingsof https://doi.org/10.4414/smw.2018.14571 https://doi.org/10.1596/978-0-8213-8991-1 market/en/news/report-working-group-mhealth-assessment-guidelines 0033 assessme and policy challenges. In health-what-c COVID-19? health system strengthening. Executive summary. https://apps.who.int/iris/bitstream/handle/10665/311977/WHO-RHR-19.8-eng. strengthening (WHO/RHR/19.8). Licence: CC BY- language describe to the usesof digital technology for health https://apps.who.int/iris/handle/10665/274134 technologiespublicfor health:report bythe Director-General. (Vol. 394, Issue pp.10192, 9 nt guidelines| Shaping Europe’ https://www.who.int/china/news/feature-stories/detail/covid- an Information and Communications Developmentfor 2012: Maximizing Mobile -digital-health-offer-for-cov Appl Clin Inform Swiss medical weekly

COVID-19 and digitalhealth: What can digital healthfor offer Classification ofdigital health interventions v1.0: ashared Recommendationson digital interventions for health system WHO guideline: recommendationson digital interventionsfor – , 11). Lancet Publishing Group. 6 , 429 . WHO Document Production Services. s digitals future – 84 id 442. https://doi.org/10.4338/ACI-2015- - (Vol. 148, Issue 0304, p. w14571). EMH Media.

19 NC

-SAIGO. 3.0 mHealth - ofUse appropriatedigital Reportof the Working Group on mHealth . https://ec.europa.eu/digital-single- .

Atlas ofeHealth country 19 -and-digital- pdf?ua=1 03 - RA - The . Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Organization, 2018, pp. 2018, Organization, pp. 6 Source: Classification ofDigital1 Appendix Health Interventionsv.1.0. A

ppen di x

– 9)

(World Health (World

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. inthat category allcriteria % of (in criteria i.e. Source:331). n the frequency in publishedevaluation #ECframeworks.shows Ranking 2. Appendix of digitalhealthinterventionevaluation cr 86

(Kowatsch et al., etal., (Kowatsch iteriabased on their umber of umber evaluation 2019)

(Sheon et al., etal., (Sheon 2018, 6) p.

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. EU Examples reimbursementof 4 for Appendix digitalhealthprodu . Source: Source: . (eHSG SubGroup on Reimbursement,(eHSGSubGroup pp. 14 2019, 88

– 15) ctsand solutions inthe

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. and products validation gotheir may productclinical foranother of wi reimbursement for going currently is company The already. clinicians by used wi developed were which applications, health digital develops company and medical affairs. He is currently chief medical officer at a digital h clinical till management grant and project through development, and research experienc His companies. health digital and biotech pharma, at and research aca in experience of years 20 over has He training. by doctor medical a is #2 Expert digit with experience broad has She institutes. both in position leading on Vienn focus Hub Health in with and INiTS incubator, economy business the through in years as well as biology s science life with worked has molecular She innovation. and entrepreneurship in degree a has #1 Expert 5 Appendix Expert #1: Expert summary) Expert #2:Expert of t through ecosystem health digital Austrian the in network a as well Health HubVienna. . Professional for ofExperts experience interviewed 90

th is ealth SME in Austria. The (anonymizedthesis he accelerator program accelerator he tart- th a partner. ath partner. al health start- health al a. She is currently in currently is She a. th the help of and of help the th up s for the last 16 last the for s e ranges from ranges e development with one of one with up demic s, as s, Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek.

be retained by authorbe years the 3 forafter thesis submitting the any i (without of theinterview and the transcript written consent my identity will remain anonymous. I understand, that the original re any in that understand I thesis. master described above the in results an interview the of audio-recording the to consent my declare I Hereby ------Szijarto Valeria ______study,inthis youyour collaboration Thank for and published) beusing not inwill my interviewresults the t master Wi validation. health Consent, I would like to request your consent digital in recording int the of field the on experts with interviews Austria in and „Clinical thesis, master my of part As Mr/MsDear Declaration 6. ConsentAppendix of template Signature of Expert of Signature ______, on ______—

n csse ve” t h W Eeuie cdm, wud ie o conduct to like would I Academy, Executive WU the at view” ecosystem an Declaration of Consent ------place dat economic assessment of digital health innovations health digital of assessment economic 91

. erview (recording and script

dentifying information) willdentifyinginformation) audio recording, the signed hesis. hesis. h hs elrto of Declaration this th d to the use of the of use the to d port on the results results the on port e e ------Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. into the Health Hub Vienna, as that can help them becoming aware. aware. becoming them help can that as Vienna, Hub Health the into reach they why reason the is this Yet, system. the into get to certified, pr depends it course of and validate, clinically should they not why or know they whether team, the of awareness the on depends always it and cases sense to go for clinical validation in that sense in such a case. I whe only, it feel will you but healthier, stay to helps sports preventive the in is that solution a have I if validation clinical E #1: Again, it depends on whether there is there whether on depends it Again, #1: E suppor also program accelerator The VSZ: wer they when stage, veryearly a on decision that taken had have they thought unique their of part one be will certification device medical the that on, early very decision strategic the took really they mySugr, example, successful very a with start I If factors. many on depends it lot; a varies really It #1: E VSZ: transcript 7. Interview Appendix Expert with #1 do if you do so? do you if do everything of aware you Are that? do you if to up are you what know, you certification process. How is it going? How much emphasis is given? it Howgoing?is How process. muchgiven? certification is emphasis plan, did they run, what did they think about it?about run, they did think they whatdid plan, case, I think. As we started to discuss about the it timeframe: in sense make not does It sense. makes it not or whether varies, really It that. beyond vali clinical and device, medical a as certification about talking thereAnd are others from be right seriously very it took mySugr etc. way, right the everything record to also itandco resources, of a lot itcosts and oftime a itcosts lot Because again. the Europ within not e.g. are which countries other from start-ups the of Some whether they had already done it in their respective countries or not, w oject, the innovation, innovation, arethey oject,tryinggenerate.theto is allfor There noone answer ginning, because they were aware of it early on what it on what they takes. ginning, because ofitawareearly were y are not that aware yet, what needs to be done in Europe in order to to order in Europe in done be to needs what yet, aware that not are y In your experience what is the attitude of the start-ups to the clin the to start-ups the of attitude the is what experience your ” Making them aware of what it really means, may shake such a decision a such shake may means, really it what of aware them Making who very strategically take the decision not nowto dothat.weare And ts a h start- the 92 need for this emphasis or not. It depends on depends It not. or emphasis this for need

ups lnig hs id f validation, of kind this planning n you are 50. It does not make not does It 50. are you n t t does not make sense in all the may care. We all know, that doing doing that know, all We care. ain s e aohr level another yet is dation not make sense goto for a eln pooiin They proposition. selling on the technology, the technology, the on hether hether they are aware. And this is one of the of one is this And out, they want to get to want they out, very good example, a example, good very ical testing: do they do testing: ical sts a lot ofenergylot a sts that you have to have you that e, in most cases most in e, get everything get e asked, asked, e exactly why exactly to that. tothat. every “ the do do Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. reall peo the are these And decision. right the take to helps definitely is different and needs different things. In general, having experience i fro maybe b might it Yet answer. same the be should it solution, same the come is has and domain health deci right the taking time easier the an have they background, pharmaceutical in experienced really one is there what are the benefits and disadvantages. It can be on every level; on the the on level; every on be can It disadvantages. and benefits the are what cli have to brings it what team: the of awareness the to back comes It #1: E start- these critical factors among the founders, among the investors, what makesi this differen running consider not does one and testing similar clinical a on running considers working roughly are which start-ups, consider you If VSZ: so itdepends. other, solut one Not it. done not have they why answer, great a give them of some and blue-eyed simply are some and this, do to need fact the of aware very are and enough wise are them of Some years. 2-3 next the next the during face will you think you that challenges, main the are answered be to questions the of one also is this Thus, di accomplished they as it, needs everybody not But program. acceleration certification European the into get to clinics, the into get to thing, right the do to them help to program, the in expectations eti ivsos h ae yial psig o i? oe xeine in experienced More it? investors?) experienced for pushing typically are who investors certain see certain see patterns havethey seen over and overagain, makesthis you blind thatradical is innovationis in so usually calle by not brought the domain is certainly helpful. Yet, what can also be seen and also in expertise. buy ifnot dothey fail, mostlikelywill way, they naive within EU there is lots of differen of lots is there EU within (We now see it in Corona, the health systems work completely differ jus Switzerland,and Germany and Austria betweenis it different what it really needs to be successful and how it really works, h y think about the decision. The newbies, that do not know anything about t about anything know not do that newbies, The decision. the about think ups ? (E.g the founders have a background, that they already did it? Are there Are it? did already they that background, a have founders the (E.g ce s.) Those people, who approach this problem in a very a in problem this approach who people, Those s.) 93

do wh not see that they have to do it. And it. do to have they that see not en applying to the program: what program: the to applying en d industry d Becauseexperts. they ow sale cycles really work, how process etc. This is part of the the of part is This etc. process t to name the DACH region. DACH the name to t approach the right people, right the approach e, that the business model business the that e, is is true for other industries, ple, who are aware, who aware, are who ple, To access the expertise in To accessthe ent in the USA vs EU and n n the sciences life sector couple of months and months of couple rdcs n one and products t. Do you see any see you Do t. sion. Because if it if Because sion. founders’ nical testing and testing nical etr v less vs vestors forcompletely frn things. fferent ion is like the like is ion ce between he domain, he , that they that , level, if level, a m Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. to these real radical solutions. real radicalsolutions. these to but you can fall off the horse on the other side, to have too much cli are questions validation clinical and questions certification device Does paths. different completely take that solutions to blind are an stones and trees the all and path know the and again, over and different paths.completely reason And that is the why those, who come that different those, angle, they come usually up with radical solutions, is that really mak It industry. every in true is it care, health n vr rdcl hne Js t gv yu n sml eape i Asra h Austria, in example: simple one you give to Just change. radical very a in will It go. one in doors closed many open pushed it And interesting. b have solutions digital sudden a of all So possible. as good as contacts shu to needs everybody contact, much too of scared is everybody because solutions, for open is everybody now crisis, Coronavirus) (VSZ: Corona the before rare very be di it put me Let rare. very still are they but some, are there Yes, #1: E and just testing not clinicalinnovation follow the testi traditional What tests. in opinion, Austria do you have such clinicians and these CROs with open mi arranging are who CROs, the for also and tests, these running f true also probably is it things, new the for eyes open with least p and in partially are who people need you mentioned, you as And testing. medi even and drug medical to compared approach and mindset different VSZ: What you just mentioned, that probably these digital health tes the as described isthat and solutions new Dorado of all the digital h digital the all of Dorado now. right phase interesting very a in are we why reason the is This fact. it that solution, the to blind be might they way, that works it that i everybody because Just meant. I what also is This works. it sudden and easy so not was it said always a we says: faster everybody now done And this interaction. get to need sheer the for just changed medicat radically your get to order in paper of sheet physical that get to doctor a real signature before and there was no aroundway that. So, you had to go p medication at a pharmacy to a patient, it had to be a reci physical ealth solutions. Because they were stumbling over the “we used to used “we the overstumbling were they Becausesolutions. ealth “ innovators dilemma innovators 94

” ng path of health care? ng path ofhealthcare? . T . p could be so much easier ineasier much so be could ee es the entire industry walk d gates on that path, they path, that on gates d walked old the paths over nical nical expertise to be blind ha o ma, ht medical that mean, not or the clinicians who are who clinicians the or tings tings requires completely radically change; we are we change; radically not relevant any longer, any relevant not (VSZ: prescription) with n the pharmacy knows, pharmacy the n nds eyes and nds for digital t is true not only in the only not true is t fferently: they used to used they fferently: d o rvn social prevent to nd with a completely completely a with It is literally the El the literally is It cm vr, very very, ecome o. n ti has this And ion. cal device clinical device cal artially out, or at or out, artially hysically hysically to the now all of a of all now nig ou anding t down t s your is digital t Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. Corona in a different way and all of the sudden we realize that sudden that ofthe werealize all differentwayin a and Corona to to do the different way. been easier, we just never tried it. We would not have thought that it already loosening up a bit on this kindon of this things? loosening bit a already up There There is always a new technology coming up, they have not seen before, so there behin lag bodies regulatory the that is case the always is what Well, #1: E insurances the safe side. Just coming back to the clinical testing, you worked insurances) private holding for back all these because health innovations, they companies insurance even or makers policy regulators, blame people Often VSZ: cells brain The needed. still are we yet, and, easy so r everybody and Corona of afraid more even are they now But obsolete. become th of out squeezed be can they that be might it because solution, all of afraid health the in attitude differently” it do in the next couple of months, I am sure. I am thesure. couple next months, in of through regulatory process onl use to supposed are clinicians the because of, aware be to have how to go through the process. And those regulatory bodies are the bodies, that safe. And there there And safe. say people least at or easy, so not is protection data because Skype, e.g. using and responsible responsible for damages, the process. The documentation the doctors have to do in terms of interms doto have doctors documentation The the process. the comply with, and they all try to protect the patient, which technology is is yet there not technology regulat the through technology new a take to how on rules the set cannot t course, of And bodies. regulatory the on dependent are clinicians lagging always is process entire the So bodies. regulatory the of because is that I done,did I thetell patient effects whatcan side happen, f ugn poies f rat nagmn srey sd o mdcl eie silicone device medical not used surgery enlargement breast of providers surgent of more strict and it is the effect, that many years ago there was this case, th regulatory the that EU, in difficulty a was there this, all of top On – howyou do see itinAustria. Are stillpushingthey clinical for testi are many other rules, regulations, that doctors have to comply with. Being with. comply to have doctors that regulations, rules, other many ’

An doctors’ es . d this is something that will radically change the healthcare system

. That is the reason why doctors were afraid of breaking the rules law, hospital law and lots of legal frameworks they have to care system everywhere on every level: doctors were doctors level: every on everywhere system care 95

ne is is important, but they all slow down ed to be connected by the fear of fear the by connected be to ed ’ .They haveto it document all, and ‘ oh yeah, in fact italways had yeah, infact oh withadvisors, regulatory or process have been made been have process ‘ what do I do, what have whatI do, whatdo works, because we used y solutions, that went that solutions, y he regulatory bodies bodies regulatory he at silicone at silicone implants d the technology. technology. the d ory process if the if process ory ng are ng or they want want to be on e system and system e the clinicians behind. The behind. are ealizes, it is it ealizes, (public or (public , it is not not is it , no rules but

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. VSZ: In the accelerator,In the you do network VSZ: with thesepeople? Vien of Komplexitätsforsch University und Telemedizin E-Health, für Programmdirektor (VSZ: Al e.g. is there clinics, university the at some are There definitely. Yes, #1: E Austria? in VSZ: Do you see the emergence of specialists on digital health among clinic And they redesign affected. So, proble health horrific to led really which silicone, industry that association. association. that of couple a is there and Austria) Telemed (VSZ: Austria in Telemedicine the eHealth solutions toeHealth the come. that from the clinical side. And if it is really set up well, it mi that is a good idea. There will always be need for clinicians who way of testing innovation on patients, in a real case scenario and make this c safe a for beds test like setups clinical define we can How go. to way and care health in also domains, different many in beds test creating th what is this And bed. test a of kind is It that. do to how of ways grou and domains describing of way systematic very a but else, nothing is clinical study. In the different phases, one has to test differ you ahave is new technology,thatis not yet is This what proven tobesafe. manner controlled very a in out set are rules the where setup, a but nothing, Basi now. right domain the in moves more are there that think I Yet, good isis thewhich for but bad it patient, thefor of inn speed tend we Europe, in And that. around way no is there behind, situation for everyone in that system. It shows, that on the regulatory sid an rules new the for check to certified were who bodies regulatory Europe. within bottleneck certification a to led really that and had body regulatory the So, body. regulatory own their in rules new the ha they that of top on behind,lagging were bodiesregulatory that th is is led to the question, within how can Europe this be prevented in t ed na of all the medical device certification process ), Freddie Meryn. There is a couple more. There is the Association of of Association the is There more. couple a is There Meryn. Freddie ), 96

ent things. Basically, a clinical study ght reallyght speed up the process of s f hs lde, ht were that ladies, those of ms ovation. ovation. are open to that, and support There were not even enough even not were There to overregulate everything, everything, overregulate to es d to re-train and re-justify and re-train to d . e EU is now working on: working now is EU e d that was very difficult difficult very was that d That That led to that not only I think that is the right the is that think I cally, a clinical study is is study clinical a cally, linical trial, for a safe a for trial, linical clinician members of members clinician e, we e, will always lag to be certified first certified be to ians and regulators faster. And I think, you test inafirst ps of people and people of ps exander Gaiger exander , because what because , n, Medical ung, he future? Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. it is a no way, nobody will get reimbursed? away, is no nobodyget willreimbursed? it plan to go for reimbursement by insurance companies. Is it a dream or they t thi you do much, How models. business different the mentioned You VSZ: B2Croute, you takethe verywhy a explanation good tohave either en the at that knows start-ups into invests who everybody because Sure, #1: E mentione You VSZ: the clinics, you need the clinicians. likeJust there are who clinics si applier the on openness this need still you Because Definitely! #1: E

benefits for the the for benefits yo Do (integration). usability clinical the also but workers, care health the testing just not sense: broad the in mean I testing clinical And they see that there are such validations inseevalidationsplace? they arethat such there a very close link between the technological solution (what does it do) to t to do) it does (what solution technological the between link close very a Definitely this is a very important decision, which way sol do we take, why do into invest to sense make not does it So, fail. will simply you there, in into it healthcare the ifAnd system. n everybody do you knows, that who do not do clinical studies and the same isthe and clinical truestudies digital for donot who do Venture (it was in the start- in (it was Venture the fund trace to able were they and initially invested investors the why reason together with the regulatory body, both learned and lot a this is the as for the FDA it was also the first time, they certified an app. in the US definitely drew up the price. They definitely worked it out w bought by Roche and the fact that they were the first medical device cert take you if defi mySugr that USP this have might higher you end, the at but certification, much is spend you money the of load the hand, you you have to invest differently into communication, rather invest into C the Over the take you that be might It hand. in hand go to need they (how do I bring the solution to the patient or the customer) and the cer “ normal ” biotech, e.g. vaccine development, and there are other hospitals and doctors, and hospitals other are there and development, vaccine e.g. biotech, st art d, -ups? Public funding or from VCs, that they are more willing willing more are they that VCs, from or funding Public -ups? that in certain cases it brings the startups benefit to run suc run to benefit startups the brings it cases certain in that up as investor before Roche bought it). Definitely, it is as an asset. it investor beforeit). bought Definitely, Roche 97

So they walked this new terrain solutions. solutions. are used to do clinical studies in benefits for patients and patients for benefits way to go and that is the marketing or ot comply ruleswith the ith the regulatory body, you need to integrateyou need to tification process. tification And de. If it is really in the in really is it If de. nitely had. They were They had. nitely ounter route, where route, ounter ified app ified by app the FDA u see any financial financial any see u lncl etn and testing clinical we we take it. There is he busi he ake it granted or nk the start-ups the nk utions like that. that. like utions . s from Roche from s On the other d you need you d ne to invest if invest to h testing. h ss model ss

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. health pe u everything up speed c and help to area, focus a be should that think I rules. all fulfil form of such a test bed that might be considered. It does not rule o start- eHealth itenables because changed, in system remuneration the Recently beds. test regulatory eHealth is on focus question on how createto regulatory test beds. I think one of dom the E #1: The Austrian Federal Ministry of Digitalization and Economic of health? digital validation VSZ: Are there any other points, that you think is important regarding c #1: E

innovations can thrive and also create the halo and make Austria visib Austria make and halo the create also and thrive can innovations system is really cracking, because of exactly that t in problems the all see we now because attitude, social very a is it hand, ains n te otr ae sd o etn eeyhn ribre. o in So, reimbursed. everything governm the that patient the of attitude normal a getting is it particular, to used are doctors the and patients Austria in Particularly system. the by accepted solut of force driving economic big the are schemes reimbursement course of Yet months ago people were discussing were people ago months beds, care intensive many so have we expensive, so is system our that happy moment the At out. left are others the and treatment, the afford can insurance these a these afford can nobody and government, a as system, a as everything afford cannot simply it if questionable, is it still and fact, medic in there is the It important. within classes 2 about discussion this have we Austria in that is a relatively small percentage of the population, that is t used are which patients, private are they Unless that. pati of because remunerated, the forces which solution, a prescribe not will doctors the paid by the insurance, so remuneration schemes are really big economical drivin iswho kindof theattitude of the user inthe patient, the force. force. It ” depends on the solution, because not everything has the chance to get rei get to chance the has everything not because solution, the on depends . And it is re really difficult questions at the moment. B moment. the at questions difficult really a very bad attitude in fact, not to take your own Onresponsibility. the : aig oe i te ytm cetn a eoytm wee digital where ecosystem, an creating system, the in money saving ‘D o we really need that man that need really we o ups 98 to clinically test and be remunerated. It is one one is It beremunerated. and test toclinically –

“ also true for most of Europe of most for true also unsocial ut again: it is the big economic driving driving economic big the is it again: business model, business is gettingtoused it ” attitude. So only those, who have is good. On the other hand, we hand, other the On good. is willing willing to pay themselves. Then, reate an ecosystem and really and ecosystem an reate s has started to work on the ent ent y ut all rules, but enables to ent to pay, which is not not is which pay, to ent beds “ has to take care of my of care take to has ains shouldthey really linical and economical le, because there are there because le, l ytm, hc is which systems, al ?’ . In general, I think I general, In . he USA, that the that USA, he – etc. Couple of Couple etc. g forces. Also, Germany has Germany it is that the the that is it , we are very are we , o pay, but pay, o uti in Austria ions to be to ions mbursed. it. So, it. other Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. possible, why did possible, we haveto wait crisis thefor

says, everybody now And COVID19. by enforced bed test regulatory e the of introduction The that. did who nations of lot a not 99 ?’

Prescription is also a kind of a of kind a also is Prescription ‘I wud ae been have would t Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. E #2: Yes. Yes. #2: E So, youwent there VSZ: European theto directly, centre? already. us about with relationship working a established already We ask. to directly E #2: No, when this topic came up or when we decided to take p this invalidationAustria. health regulator of policies/frameworks/recommendations local insurance companies on about digital health? So, such things that really know you Do VSZ: advisor. such dedicated won’t need al knowledgeenoughexperienceand have thatwe possible,is It#2: E CRO. procedurethe managing selecting whole and the c who Someone, advisors? external any with work to plan you Do VSZ: are open So, we logistics. every for possibility. rea geographic the for also that, choose will we well, as Austria in we If internationally. search likely most would we But certainty. with not will we search of bit little a with that mean, not does it support could who Austria, in partner such about know not do We #2: E could you whom and health digital with experience has who Austria, abou know you do study, the on CRO a with work to plan you If VSZ: future.very near a created was position the exist, but not does such function Exact #2: E person a there Is company? the could company, who fulfilfunction? this within exist already function this Does VSZ: research(CRA) clinical Clinical (CRM). manager#2: trial assistant or E (which offunction) Who coordinator internalth be thewill VSZ: course.It isCRO,#2: company of rare E that outsourc of any not does size CRO/consultantstogeta plan it? doto internal with study the manage and design/organize to plan you Do VSZ: Yes,#2:we havesuch E person. experiencealready testing with (of clinical digital health)? anyone haveyou Do validation. clinical during the experience this use alread clinic inthe yourproduct onhow use to experience also have large You getti is product your that possibility, a is mentioned, there you As VSZ: also also transcript 8. Interview Appendix Expert with #2 – looksuch partners for internationally?

100 find someone. I cannot answer cannot I someone. find e study? estudy? focus on how to run a digital sons, i.e. convenience in convenience i.e. sons, ath, we went to the EMA them, there they know they there them, t a potential partner in partner potential a t rely on or would you you would or on rely ready internally, so weinternally,soready find a compatible CRO compatible a find on the team, who had who team, the on ng tested in the clinic. the in tested ng nd will be filled in the in be filled will nd us in this study, butstudy, this in us ould help planning, help ould e such project. suchproject. e people or do you do or people y and you mayy and ihn the within or s –

Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. easier? easier? c would opinion, your In ways’? this and this validated was ‘it that said you IVSZ: see. And when looking at methods how to get over this barrier, could s application health kno digital ‘a say and negative really are Some even but countries, on depend only not does it And not. are others and really, we see that it depends on the countries, cultures, some are open for such these accept doctors the make to is challenging most the that say, would without but part, challenging most the about asking were you If #2: E side,arranging andfinancial the togetor designing, managing validation: t trials clinical running and and ahead look you when all, in All VSZ: No,it.I do#2: know not E you heard I recentlya heardcalledFörderpilot.at.website, Have about VSZ: KLIPHA. such support to this, in partner a be could they think I and Agency) Promotion Research (Austrian Forschungsförderungsgesellschaft Österreichische experi good had we but it, for grants specific about know not do I #2: E cover t to youknow about Do funding VSZ: public opportunities validation for cometopic This#2: so not up did E far. No. push particularly who investors money.effort, itrequires time, that have you support least at but it, for pushes particularly not do if or reimbursement, general, In cert to VSZ: out reach could we Maybe point. next my been bea who incouldThisthis.possibility. have experience companies have would this Yes, #2: E compan health process? this through digital companies, other with touch in maybe you Are VSZ: difficult. bureaucratun rather is complicated,it Austria, to a typical is It experience. be recent our to to according moment, seems It no. is answer short The #2: E institutes? you know about any external partners, who helps you in this process? one with reimbursement for go to plan you that mentioned, You VSZ:

effort. effort. clinician by accepted application digital such making and challenge rep could rather and threat a something that is easier, but work their wledge wledge and experience’. So, they do not look at such applications – what do you think, what will be the most challenging part of a of part challenging most the be will what think, you do what – based on your experience working with clinicians clinicians with working experience your on based 101 clinical validation, maybe within maybe validation, clinical his his process? ol nvr elc their replace never hould s always takes time and time takes always s as a tool, that can make lace them. So, it is a biga it is So, them. lace Any Any state or any private listing any examples, I examples, any listing ences with FFG (VSZ FFG with ences of your products. Do products. your of s it and understands understands and it s on the specialties. the on applications. And applications. he money for it. it.hemoneyfor ic process at the the at process ic linicians accept it accept linicians that bethat helpful, if about it? it? about e, h went who ies, applications clear and clear and ain ) ) : Die approbierte gedruckte Originalversion dieser Masterarbeit ist an der TU Wien Bibliothek verfügbar. The approved original version of this thesis is available in print at TU Wien Bibliothek. that is a is huge that help. using start they and running study marketing post 4 phase e.g. an have they application such about hear companies large If country. another to but specialty same p patient same the to spread easily can it here, it using starts someone borders. the across communicate also companies pharma multinational these because governmentthe differentand (led by parties). or not even the AGES. Only those that can be a scene for political de specifically talking about state institutes, bureaucracy and not abo b Austria in) challenges the (VSZ: about talking was I When are. They Yes. #2: E Arein Austria large alsoVSZ: pharmathe partnersinthis? companies it using population start and application such of benefit the see they be will and helpful very are companies pharma large the opinion my In #2: E su health companies running clinical validation, and in that would well very works that companies, or institutes practices, international very bureaucratic and maybe you would look for the CRO internationall th mentioned, you practices: international at looking When VSZ: get experiences it,toaboutthethen it helps talks recommends and introduce the application to a key opinion leader and then he have we if concrete: a too being to Without thing. huge close a is it get then specialty, we if that is, experience our And help. could it helps doctors either in diagnosis or severity during (as monitoring we ea discussed ared there and itworks If help. definitely would Validation #2: E

– in my view it is a very good path. They can later spread the word, es word, the spread later can They path. good very a is it view my in 102 be very useful in Austria as well? or or she goes to a conference and – usually on a narrow patient patient narrow a on usually ata that demonstrate that itthat thatdemonstrate ata ut international companies, key opinion leader of that that of leader opinion key at in Austria the system is system the Austria in at bate, bate, like between Vienna moreon people board. y. Are there any other opulation or for the for or opulation an opportunity to opportunity an prig digital pporting very helpful. If helpful. very the app there, app the efore, I was I efore, rlier), rlier), then pecially So, if So, and