Enterprise Architecture for Information System Analysis Modeling and assessing data accuracy, availability, performance and application usage PER NÄRMAN Doctoral Thesis Stockholm, Sweden 2012 TRITA-EE 2012:035 Industrial Information and Control Systems ISSN 1653-5146 KTH, Royal Institute of Technology ISRN KTH/ICS/R--12/01--SE Stockholm, Sweden Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy © Per Närman, September 2012, Copyrighted articles are reprinted with kind permission from El- sevier and Taylor & Francis. Universitetsservice US AB Abstract Decisions concerning IT systems are often made without adequate decision-support. This has led to unnecessary IT costs and failures to realize business benefits. The present thesis presents a framework for analysis of four information systems properties relevant to IT decision-making. The work is founded on enterprise architecture, a model-based IT and business management discipline. Based on the existing ArchiMate framework, a new enterprise architecture framework has been developed and implemented in a software tool. The framework supports modeling and analysis of data accuracy, service performance, service availability and application usage. To analyze data accuracy, data flows are modeled, the service availability analysis uses fault tree analysis, the performance analysis employs queuing networks and the application usage analysis combines the Technology Acceptance Model and Task-Technology Fit model. The accuracy of the framework’s estimates was empirically tested. Data accuracy and service performance were evaluated in studies at the same power utility. Service availability was tested in multiple studies at banks and power utilities. Data was collected through interviews with system development or maintenance staff. The application usage model was tested in the maintenance management domain. Here, data was collected by means of a survey answered by 55 respondents from three power utilities, one manufacturing company and one nuclear power plant. The service availability studies provided estimates that were accurate within a few hours of logged yearly downtime. The data accuracy estimate was correct within a percentage point when compared to a sample of data objects. Deviations for four out of five service performance estimates were within 15 % from measured values. The application usage analysis explained a high degree of variation in application usage when applied to the maintenance management domain. During the studies of data accuracy, service performance and service availability, records were kept concerning the required modeling and analysis effort. The estimates were obtained with a total effort of about 20 man-hours per estimate. In summary the framework should be useful for IT decision-makers requiring fairly accurate, but not too expensive, estimates of the four properties. Keywords: Enterprise Architecture, Metamodeling, Decision-making, Data Accuracy, Service Availability, Service Performance, Technology Acceptance Model, Task Technology Fit iii Sammanfattning Beslut rörande IT fattas ofta med otillräckligt beslutsunderlag. Detta leder till onödigt höga IT- kostnader och svårigheter att realisera IT-nyttor. Denna avhandling presenterar ett ramverk för ana- lys av IT-systemegenskaper av intresse för IT-beslutsfattare. Enterprise Architecture är en modellbaserad disciplin för IT- och verksamhetsutveckling och utgör grunden för denna avhandling. Baserat på ramverket ArchiMate har ett nytt arkitekturramverk utvecklats och implementerats i ett verktyg. Ramverket kan användas för att modellera och analysera datakorrekthet, tjänstetillgänglighet, tjänsteprestanda och IT-systemanvändande. För att utvärdera datakorrekthet modelleras dataflöden, tillgänglighetsanalysen använder sig av felträdsanalys, prestandaanalysen baseras på köteori och systemanvändande på modellerna Techno- logy Acceptance Model och Task-Technoloy Fit. Tillförlitligheten i analysresultaten har testats empiriskt. Resultaten avseende datakorrekthet och tjänsteprestanda testades i studier på samma elbolag, tjänstetillgänglighet undersöktes i flera studier på elbolag och banker. I dessa fall samlades data in genom intervjuer med utvecklings- eller under- hållspersonal. Systemanvändande testades inom domänen underhållsstyrning genom en enkät med 55 respondenter från tre elbolag, ett tillverkningsföretag och ett kärnkraftverk. Uppskattningarna av tjänstetillgänglighet var korrekta sånär som ett fåtal timmars nertid på års- basis, datakorrekthetsuppskattningen var mindre än en procent från det mätta värdet. Fyra av fem tjänsteprestandaupps5kattningar var inom 15% från uppmätta värden. Analysen av systemanvän- dande kunde användas för att förklara en stor del av variationen av systemanvändande i underhålls- styrningsområdet. I studierna avseende datakorrekthet, tjänstetillgänglighet och tjänsteprestanda undersöktes också kostnaderna för att genomföra analyserna. Totalt krävdes det ungefär 20 timmar per uppskattning. Sammanfattningsvis bör ramverket vara användbart för praktiker som behöver relativt korrekta, men inte alltför dyra, uppskattningar av de fyra egenskaperna. Nyckelord: arkitektur, metamodellerande, beslutsfattande, datakvalitet, tillgänglighet, prestanda, systemanvändande v Acknowledgments I relied quite heavily on the help and support of a number of people whose contributions deserve acknowledgment. And since this is the Acknowledgments section, lets name them here: Torsten Cegrell was not only kind enough to hire me, but together with Judy Westerlund has created what I believe to be a uniquely rewarding working environment. Thank you both. Judy, together with Annica Johanneson also proved to be particular helpful for me when I decided to stop being a full-time employee and decided to venture out into the Real World. Pontus Johnson and Mathias Ekstedt have both provided invaluable assistance in their role of supervisors in which they have been teaching me the immense value of having a really, really good metamodel. Some of my fellow PhD students – the PERDAF posse – also put in a substantial amount of work as paper co-authors and deserve recognition for this: Hannes Holm, Ulrik Franke, Markus Buschle, David Höök, Moustafa Chenine, Johan König and Nicholas Honneth all contributed in shaping this thesis to turn out as good as it eventually did. Khurram Shahzad also made significant contributions by bringing the EAAT tool to a new level which greatly aided this work. Thank you all, I hope you enjoyed the projects as much as I did. My other dear co-workers – Mårten Simonsson, Magnus Gammelgård, Robert Lagerström, Jo- han Ullberg, Teodor Sommestad, Joakim Lilliesköld, Erik Johansson, Lars Nordström, Liv Gignell, Evelina Ericsson and everybody else at the department all deserve a big thank you for stimulating projects, courses and coffee breaks. A lot of kind people in Swedish organizations and companies volunteered their time to help me collect empirical data. I owe you all a great deal and in particular I would like to thank Ulf Larsson, Conny Peterson Edlund, Niclas Almlöv, Jalal Matini, Miguel Aguilar, Stefan Huss, Hans- Erik Carlsson, Hans Kumlin, Mauno Kamarainen, Richard Hejdenberg, Göran Ericsson and Sven- Olof Eriksson. Both my bosses at Capgemini Consulting Per-Ola Niblaeus and Ulf Larsson have been very kind and supportive about me finishing this thesis. Thank you for this, now I can focus solely on becoming a better management consultant. Mum, Dad, Kirsti and Donald have all helped by providing me with a lot of much-needed writing time during the past two years. Without you, this thesis would not be finished this soon, thank you! Finally, Pia – for all the love and patience, Wilhelm – for all the love and impatience. Thank you! Stockholm, September 2012 Per Närman vii Papers List of included papers Paper A: Per Närman, Hannes Holm, Pontus Johnson, Johan König, Moustafa Chenine, Mathias Ekstedt, "Data Accuracy Assessment Using Enterprise Architecture," Enterprise Information Sys- tems, vol. 5, issue 1, pp. 37-58, (jan) 2011, DOI:10.1080/17517575.2010.507878 Paper B: Per Närman, Ulrik Franke, Johan König, Markus Buschle, Mathias Ekstedt, "Enterprise Architecture Availability Analysis Using Fault Trees and Stakeholder Interviews", Enterprise Infor- mation Systems, DOI:10.1080/17517575.2011.647092 Paper C: Per Närman, Hannes Holm, Nicholas Honeth, Mathias Ekstedt, "Using Enterprise Ar- chitecture Analysis and Interview Data to Estimate Service Response Time", accepted with minor revisions to the special issue on Service Management & Engineering in the Journal of Strategic Information Systems, (aug) 2012 Paper D: Per Närman, Hannes Holm, David Höök, Nicholas Honeth, Pontus Johnson, "Using Enterprise Architecture and Technology Adoption Models to Predict Application Usage", Journal of Systems and Software, vol. 85, issue 8, pp 1953-1967, (aug) 2012, DOI: 10.1016/j.jss.2012.02.035 Paper E: Per Närman, Markus Buschle, Mathias Ekstedt, "An Enterprise Architecture Framework for Multi-Attribute Information Systems Analysis", accepted with minor revisions to the special issue on Enterprise Modeling in the Journal of Software and Systems Modeling, (aug) 2012 Author contributions In Paper A, the general research concept is due to Närman, Johnson and Holm, the metamodel is designed by Närman, Johnson, König, the empirical data was collected by Holm and Närman, and authoring was done by Närman, Holm, Chenine, König and Ekstedt. In Paper B, the general research concept