PhD Dissertation International Doctorate School in Information and Communication Technologies DISI - University of Trento An effective end-user development approach through domain-specific mashups for Research Impact Evaluation Muhammad Imran Advisor: Prof. Maurizio Marchese Universit`adegli Studi di Trento arXiv:1312.7520v3 [cs.DL] 6 Jan 2014 Co-Advisor: Prof. Fabio Casati Universit`adegli Studi di Trento March 2013 Abstract Over the last decade, there has been growing interest in the assessment of the performance of researchers, research groups, universities and even countries. The assessment of productivity is an instrument to select and promote person- nel, assign research grants and measure the results of research projects. One particular assessment approach is bibliometrics i.e., the quantitative analysis of scientific publications through citation and content analysis. However, there is little consensus today on how research evaluation should be performed, and it is commonly acknowledged that the quantitative metrics available today are largely unsatisfactory. The process is very often highly subjective, and there are no universally accepted criteria. A number of different scientific data sources available on the Web (e.g., DBLP, Microsoft Academic Search, Google Scholar) that are used for such anal- ysis purposes. Taking data from these diverse sources, performing the analysis and visualizing results in different ways is not a trivial and straight forward task. Moreover, the data taken from these sources cannot be used as it is due to the problem of name disambiguation, where many researchers share identical names or an author different name variations appear in the data. We believe that the personalization of the evaluation processes is a key element for the ap- propriate use and practical success of these research impact evaluation tasks. Moreover, people involved in such evaluation processes are not always IT ex- perts and hence not capable to crawl data sources, merge them and compute the needed evaluation procedures. The recent emergence of mashup tools has refueled research on end-user de- velopment, i.e., on enabling end-users without programming skills to produce their own applications. Yet, similar to what happened with analogous promises in web service composition and business process management, research has mostly focused on technology and, as a consequence, has failed its objective. Plain technology (e.g., SOAP/WSDL web services) or simple modeling lan- guages (e.g., Yahoo! Pipes) do not convey enough meaning to non-programmers. We believe that the heart of the problem is that it is impractical to design tools that are generic enough to cover a wide range of application domains, power- ful enough to enable the specification of non-trivial logic, and simple enough to be actually accessible to non-programmers. At some point, we need to give up something. In our view, this something is generality since reducing expressive power would mean supporting only the development of toy applications, which is useless, while simplicity is our major aim. This thesis presents a novel approach for an effective end-user development, specifically for non-programmers. That is, we introduce a domain-specific ap- proach to mashups that \speaks the language of users", i.e., that is aware of the terminology, concepts, rules, and conventions (the domain) the user is comfort- able with. We show what developing a domain-specific mashup platform means, which role the mashup meta-model and the domain model play and how these can be merged into a domain-specific mashup meta-model. We illustrate the ap- proach by implementing a generic mashup platform, whose capabilities are based on our proposed mashup meta-model. Moreover, the thesis proposed an architec- tural design for mashup platforms, specifically it presents a novel approach for data-intensive mashup-based web applications, which proved to be a substantial contribution. The proposed approach is suitable for those applications, which deal with large amounts of data that travel between client and server. Keywords[End-user development, Domain-specific mashups, Research evalua- tion] 4 Acknowledgements This thesis would not have been possible without the support of many people, whom I want to acknowledge in this section. First of all, thanking God for giving me the amazing opportunity of coming to Trento to pursue my PhD degree. I would like to express my sincere gratitude to my supervisors Prof. Maurizio Marchese and Prof. Fabio Casati for their valuable guidance, support and constructive comments throughout the journey toward my PhD. I would also like to express my sincere gratitude to Dr. Florian Daniel for his immeasurable attentive guidance, valuable insights and technical advice throughout my PhD. Thank you Florian, this thesis would not have been pos- sible without your support. I thank my fellows (Soudip roy chowdhury, Stefano Soi) and friends (Zeeshan Munir, Musawar Saeed, Talha Rehman), who have been with their kind behavior contributed to this work directly or indirectly. This PhD is also the result of much love, encouragement and prayers from my parents and family. Especially my dearest dad, who has been a great source of support and encouragement for me throughout my life. He is truly a great father and a kind person. Dad & mom, I owe you everything I have. Finally, I want to thank my partner in life, my dear wife. Her constant support and love got me through this process. Thank you all!! Muhammad Imran i ii Contents 1 Introduction 1 1.1 Research Evaluation: A Multi-dimensional Field . .2 1.1.1 Diverse Information Sources . .3 1.1.2 Diverse Evaluation Indicators . .4 1.1.3 Diverse Evaluation Procedures . .5 1.2 Problems, Challenges and Objectives . .6 1.3 Solution Overview and Contributions . .8 1.3.1 Overview . .8 1.3.2 Contributions . 10 1.4 Structure of the thesis . 12 2 Research Impact Evaluation: State of the Art 15 2.1 Overview . 15 2.2 Multiple Faces of Research Impact Evaluation . 16 2.2.1 Quantitative and Qualitative Research Evaluation . 19 2.2.2 Bibliometrics, Scientometrics and Informetrics . 20 2.3 Research Evaluation Through Bibliometrics Approaches . 21 2.4 Bibliographic Databases . 26 2.4.1 Web of Science . 26 2.4.2 SciVerse Scopus . 27 2.4.3 Microsoft Academic Search . 28 2.4.4 Google Scholar . 29 2.4.5 DBLP . 29 2.5 Research Impact Evaluation Tools . 30 iii 2.5.1 Publish or Perish . 30 2.5.2 Scholarometer . 31 2.5.3 ResEval . 32 2.5.4 Research Gate . 33 2.6 Analysis and Discussion . 33 3 End-user Development & Mashups: State of the Art 39 3.1 Overview . 39 3.2 End-user Development . 40 3.3 Enabling Practices and Techniques . 43 3.3.1 Simple Programming Models . 43 3.3.2 Domain-specific Modeling. 44 3.3.3 Domain-specific Languages (DSLs) . 44 3.3.4 Web Service Composition. 45 3.3.5 Intuitive Interaction Paradigms . 45 3.3.6 Reuse of Development Knowledge . 46 3.4 Domain-Specific Languages: Discussion . 47 3.5 Mashups from an End-User Development Prospective . 48 3.5.1 Web 2.0 & Enabling Technologies . 48 3.5.2 Tool-Assisted Mashup Development . 50 3.6 Analysis and Discussion . 54 4 Research Evaluation Example Scenarios and Requirements Un- derstanding 57 4.1 Overview . 57 4.2 University of Trento Department Evaluation Procedure . 58 4.3 Italian Professorship Selection Scenario . 60 4.4 Analysis and Domain-Specific Requirements . 62 4.4.1 End-user centric requirements . 63 5 End-User Oriented Mashup Platform Development Methodol- ogy 65 iv 5.1 Overview . 65 5.2 Concepts & Definitions . 66 5.3 Challenges and problems . 68 5.4 Methodology . 69 5.5 The Domain Concept Model . 72 5.6 The Generic Mashup Meta-Model . 74 5.6.1 The mashup meta-model . 76 5.6.2 Operational semantics . 79 5.6.3 Generic mashup syntax . 80 5.7 The Domain-Specific Mashup Meta-Model . 80 5.7.1 Domain process model . 82 5.7.2 Domain rules . 83 5.7.3 Domain syntax . 84 5.7.4 Domain instances . 85 6 Domain-Specific Mashup Platform Development 87 6.1 Overview . 87 6.2 Components & Compositions Execution Insights . 88 6.2.1 Orchestration style . 89 6.2.2 Data-passing style . 89 6.2.3 Compositions execution . 90 6.3 Components Definitions . 91 6.3.1 Component Definition Language (CDL) . 91 6.3.2 Component Definition Language in Action . 95 6.4 Mashup Compositions Definitions . 97 6.4.1 Mashup Definition Language (MDL) . 97 6.4.2 Mashup Definition Language in Action . 98 6.5 The Mashup Engine . 100 6.5.1 Mashup Engine Architecture . 101 6.5.2 The Repository Module . 102 6.5.3 Component- and Composition Mapper . 102 6.5.4 Component Descriptor and Component . 103 v 6.5.5 Composition . 104 6.5.6 Data Mapper . 104 6.5.7 Data Processor . 105 6.5.8 Configuration Interface . 106 7 ResEval Mash: A Domain-Specific Mashup Tool 109 7.1 Overview . 109 7.2 Design Principles . 110 7.2.1 Intuitive graphical user interface . 110 7.2.2 Hidden data mappings . 110 7.2.3 Data-intensive processes . 111 7.2.4 Platform-specific services . 111 7.2.5 Runtime transparency . 111 7.3 ResEval Mash Architecture . 112 7.3.1 Overview . 112 7.3.2 Mashup Engine . 113 7.3.3 Composition editor . 114 7.3.4 Component Registration Interface . 116 7.3.5 Server-Side Services . 118 7.3.6 CDM Memory Manager, CDM Module & Shared Memory 118 7.3.7 Local Database and the Web . 119 7.4 Intelligent Switching between Data-flow and Control-flow . 120 7.5 Components Models and Data Passing Logic . 123 7.6 The Domain-Specific Service Ecosystem . 128 7.7 ResEval Mash in Action: Various Mashup Compositions .