Lincoln University Digital Thesis Copyright Statement The digital copy of this thesis is protected by the Copyright Act 1994 (New Zealand). This thesis may be consulted by you, provided you comply with the provisions of the Act and the following conditions of use: you will use the copy only for the purposes of research or private study you will recognise the author's right to be identified as the author of the thesis and due acknowledgement will be made to the author where appropriate you will obtain the author's permission before publishing any material from the thesis. Designing a Framework for End User Applications A thesis submitted in partial fulfilment of the requirements for Degree of Doctor of Philosophy in Software and Information Technology at Lincoln University By Yanbo Deng Lincoln University 2013 Abstract of a thesis submitted in partial fulfilment of the requirements for the Degree of Doctor of Philosophy in Software and Information Technology Designing a Framework for End User Applications by Yanbo Deng End user developers (i.e. non-professional developers) often create database applications to meet their immediate needs. However, these applications can often be difficult to generalise or adapt when requirements inevitably change. As part of this thesis, we visited several research institutions to investigate the issues of end user developed databases. We found that different user groups in the same organisation might require similar, but different, data management applications. However, the very specific designs used in most of these systems meant it was difficult to adapt them for other similar uses. In this thesis we propose a set of guidelines for supporting end user developers to create more flexible and adaptable data management applications. Our approach involves professional and end user developers working together to find a “middle way” between very specific and very generic designs. We propose a framework solution that allows the data model to have several co-existing variations which can satisfy the requirements of different user groups in a common domain. A “framework provider” (IT professional) will create the initial framework and data model. Configuration tools are then provided for a “framework manager” to easily customise the model to the specific needs of various user groups. The system also provides client toolkits and application generators to help end user developers (EUDs) to quickly create and customise applications based on the framework. The framework approach was applied to a case study involving a Laboratory Information Management System (LIMS) for data on research experiments. We demonstrated that the framework developed could be successfully applied to several groups working in the same domain and could be extended to include new or changed requirements. We also evaluated the framework through software trials at several research organisations. All participants successfully used the configuration tools to extend the LIMS framework within an average of 40 minutes. EUDs were also able to easily create basic applications within an average of 25 minutes. The overall feedback was that the framework approach was a useful and efficient way to create adaptable data management applications. More importantly, i participants were able to immediately see how the framework could be applied to their own laboratory data. Keywords: End user development, software flexibility, data management, framework approaches, database evolution ii Acknowledgements First, I want to acknowledge my supervisors, Clare Churcher and Walt Abell, for their support and guidance. They made significant efforts to provide detailed comments and suggestions for this research. Without them, I could not have completed this thesis. Secondly, I would like to express my thanks to my external advisor, John McCallum (from New Zealand Institute of Plant & Food Research Ltd), for his contribution to the research. I would also like to acknowledge the support from a FRST sub-contract. Thirdly, I would like to acknowledge all those who helped me with this research. I wish to thank all participants from: the Department of Applied Computing, Lincoln University; the New Zealand Institute of Plant & Food Research Ltd; the Faculty of Agriculture and Life Sciences, Lincoln University; the IT Department, AgResearch Ltd; Diversity Arrays Technology Pty Ltd, Australia; the Research School of Chemistry, Australian National University; the Department of Environmental Engineering, China University of Mining and Technology; the Department of Psychology, Peking University ; the College of Life Science, Peking University; and the Beijing Institute of Genomics (BIG), Chinese Academy of Sciences. Special thanks go to Caitriona Cameron for improving my thesis writing skills, and Douglas Broughton for his excellent help in ensuring all services, resources and facilities were available for my study. Finally, I would like to thank Aimei Xiong and Rong Shang for their encouragement and support. iii Table of contents Abstract ...................................................................................................................................... i Acknowledgements .................................................................................................................. iii Table of contents ...................................................................................................................... iv List of figures ........................................................................................................................... ix List of tables ............................................................................................................................. xi Chapter 1 Introduction ............................................................................................................ 1 1.1 End user development issues ............................................................................................ 1 1.2 Aims and scope of the research ......................................................................................... 1 1.3 Structure of the thesis ........................................................................................................ 2 Chapter 2 Flexible software development .............................................................................. 3 2.1 Flexible software development ......................................................................................... 3 2.1.1 Object Oriented (OO) techniques .......................................................................... 3 2.1.2 Software components ............................................................................................ 5 2.1.3 Model View Controller (MVC) and Layered architecture .................................... 6 2.1.4 Software application generator .............................................................................. 8 2.1.5 Object Oriented frameworks ................................................................................. 9 2.1.6 Software Patterns ................................................................................................. 10 2.2 Flexible data storage ....................................................................................................... 11 2.2.1 Schema evolution ................................................................................................ 11 2.2.2 Metadata model ................................................................................................... 12 2.2.3 Ontology-based system ....................................................................................... 13 2.2.4 Application and database co-evolution ............................................................... 13 2.3 Web services ................................................................................................................... 14 2.4 Summary ......................................................................................................................... 15 Chapter 3 End User Development ........................................................................................ 16 3.1 End user development ..................................................................................................... 16 3.2 Potential end user development tools .............................................................................. 17 3.2.1 Spreadsheet .......................................................................................................... 17 3.2.2 Reporting tools .................................................................................................... 17 3.2.3 End user databases .............................................................................................. 18 3.2.4 Web development ................................................................................................ 18 3.3 Application customisation and implementation .............................................................. 19 3.3.1 Approaches .......................................................................................................... 19 3.3.2 Challenges ........................................................................................................... 21 iv 3.4 Evolution of end user applications and databases ........................................................... 22 3.4.1 Collaborative design to evolve applications .......................................................