Monitoring Virtual Team Collaboration: Methods, Applications, and Experiences in Engineering Design
Total Page:16
File Type:pdf, Size:1020Kb
Monitoring Virtual Team Collaboration: Methods, Applications, and Experiences in Engineering Design Dissertation zur Erlangung des akademischen Grades eines Doktors der Ingenieurwissenschaften (Dr.-Ing.) eingereicht an der Mathematisch-Naturwissenschaftlichen Fakult¨at der Universit¨atPotsdam von Matthias Uflacker, M.Sc. Potsdam, November 2010 Abstract What distinguishes high-performance engineering teams from lower-per- forming ones in the design of complex software, products, and services? Answering this question traditionally involves extensive protocol studies and retrospective assessments of team effectiveness, e.g., in terms of adher- ence to budget and timelines, customer satisfaction, or innovation. Little attention has been paid to developing applicable techniques for observ- ing performance-relevant differentiators directly in the behavioral aspects of digitally-mediated creative teamwork. The expanding role of `virtual collaboration' in engineering projects requires new computational instru- ments to efficiently study if and how designing is reflected in the implicit processes, tactics, and strategies carried out over today's dense network of groupware, e-mail, and Web 2.0 services. This dissertation handles two important aspects in the realization of such an instrument. First, it de- velops an adaptable monitoring service platform called, d.store, to capture and analyze virtual collaboration activities live, i.e., while a project is still ongoing. Secondly, it applies the services in global, small-group engineer- ing teams to identify structural differences in collaboration behavior that correlate with independent team performance measures. With the services provided by the d.store platform it is possible to tap into heterogeneous online communication channels and to automatically generate a descriptive model of how teams virtually communicate, interact, and share information over the course of a project. The semantics and temporal attributes of the identified actors, resources, and relationships are represented in so-called Team Collaboration Networks. The platform is evaluated in the conceptual design phases of eleven distributed, multi- disciplinary engineering projects over a period of eight months each. The activities monitored in the e-mail archives, Wikis, and file sharing systems provide the basis for a detailed visual and quantitative examination of differences and similarities in the collaboration behavior of the observed teams. The analysis conducted on the generated Team Collaboration Networks indicates that high-performance design teams produce different collabora- tion patterns than lower-performing ones. Furthermore, the patterns that ii Abstract correlate with team performance suggest that an adherence to basic design principles has positive effects: teams who applied an `outside-in' perspec- tive by emphasizing interactions with team-external stakeholders, contacts to domain experts, or group-internal knowledge sharing were generally more satisfied with their work, explored more design alternatives, or re- ceived higher ratings from independent judges. This is relevant, because it demonstrates that automatically collected objective real-time collaboration metrics can provide valuable insights into performance-relevant aspects of teamwork. The contribution of this work is a tested, non-interfering monitoring instrument, which establishes a technological foundation for the scientific observation, comparison, and analysis of virtual collaboration activities as a service. A pilot application in engineering design gives first evidence that meaningful team performance indicators can be drawn from this approach. The results encourage a continued and intensified utilization of the instru- ment to assist in the evaluation of IT-mediated collaboration processes, ultimately promoting a new paradigm in the conduction of real-time team diagnostics and support in engineering design. Acknowledgements I am grateful for the many people who have accompanied me along this journey. First, I would like to thank Prof. Hasso Plattner for his inspiring, generous, and liberal guidance over the course of this dissertation. His passion and striking commitment for the topic encouraged me to pursue this line of research. The experiences that I was able to gain during this time were exceptional, invaluable, and are never forgotten. I also would like to thank the professors of the HPI research school, es- pecially Prof. Christoph Meinel and Prof. Andreas Polze, for their valuable feedback and guidance when it was needed. Many thanks also to Alexander Zeier, who has provided me with the environment and freedom to finish this dissertation and to pursue my research interests in various projects. I am deeply grateful to have met Prof. Larry Leifer and the people at the Center for Design Research, who were an invaluable source of inspiration and the second lighthouse as I progressed through this endeavor. Many of you have become good friends. Philipp Skogstad and Martin Steinert deserve special recognition. Thank you for the fruitful discussions, support, and motivation that I have received from you. I am thankful to have worked with my colleagues from the EPIC re- search group and the HPI research school, of which I can mention only few here: J¨urgenM¨ullerand Thomas Kowark for providing unbiased feedback on the drafts of this dissertation; Martin Faust and David Schwalb for their support during the implementation and data analysis. My special thanks go to Vishal Sikka and Sam Yen from SAP for their interest in my research and for providing professional feedback and industry perspectives. Finally, and most importantly, I would like to thank my parents, the rest of my family, and especially Meike. Thank you for your patience, un- derstanding, and support while I took my time to complete this chapter of my life. Potsdam, Matthias Uflacker November 2010 Table of Contents List of Tables ............................................... ix List of Figures .............................................. xi Listings ..................................................... xiv 1 Introduction ............................................ 1 1.1 Motivation . .1 1.2 Why Monitoring of Design Collaboration is Relevant . .2 1.3 Problem Definition . .4 1.4 Research Objective . .6 1.4.1 A Service Platform for Virtual Collaboration Monitoring . .7 1.4.2 Scope of the Dissertation . .8 1.4.3 Underlying Principles and Assumptions . .9 1.5 Research Approach & Guiding Questions . .9 1.5.1 Step 1: Development of a Descriptive Model . 10 1.5.2 Step 2: System Implementation & Customization . 10 1.5.3 Step 3: Application in Conceptual Engineering Design 11 1.6 Results and Contribution. 11 1.7 Outline of the Thesis . 12 Part I Background & Preliminaries 2 A Review of Engineering Design Literature . 17 2.1 Conceptual Engineering Design . 18 2.1.1 The Fuzzy Front End of Innovation . 20 2.1.2 User-Centered Design . 22 2.1.3 Design Thinking . 24 2.1.4 Conclusions Drawn From Review . 26 2.2 Teamwork, Information & Virtual Collaboration . 26 2.2.1 Design Teams: A Working Definition . 27 2.2.2 Models of Design Work . 28 vi Table of Contents 2.2.3 Virtual Collaboration in Design . 31 2.2.4 Conclusions Drawn From Review . 34 2.3 CSCW and Groupware in Conceptual Design . 34 2.3.1 Basics of CSCW in Design . 34 2.3.2 Synchronous & Asynchronous Groupware . 36 2.3.3 Hypermedia & Web-based Collaboration Platforms . 38 2.3.4 Application Lifecycle Management Platforms . 41 2.3.5 Conclusions Drawn From Review . 42 2.4 Instruments for Virtual Collaboration Monitoring . 42 2.4.1 Monitoring of Information Artifacts . 43 2.4.2 Monitoring of Process Participants . 44 2.4.3 Combined Monitoring of Information and Participants 45 2.4.4 Derivation of System Requirements . 46 2.4.5 Moving Beyond the Existing Literature . 48 2.5 Chapter Summary . 49 3 Technological Foundations .............................. 51 3.1 Definitions . 51 3.2 Representational State Transfer . 53 3.3 Of Resources and Semantics . 56 3.4 Semantic Web . 58 3.4.1 Ontologies . 59 3.4.2 The Resource Description Framework . 59 3.4.3 The OWL Web Ontology Language . 61 3.4.4 A Graphical Notation for RDF/OWL Ontologies . 62 3.5 Chapter Summary . 64 Part II Models for Team Collaboration Capture 4 Team Collaboration Networks ........................... 67 4.1 Foundations . 67 4.2 Temporal Network Properties . 69 4.3 Representing Team Collaboration Networks in OWL . 72 4.3.1 Motivation . 72 4.3.2 Terminological Components . 73 4.3.3 Assertion Components . 76 4.4 Chapter Summary . 78 5 An Ontology System for Team Collaboration Networks . 79 5.1 Foundations . 79 5.2 Named Graph Partitioning . 80 Table of Contents vii 5.2.1 Domain Ontologies & Rule Graphs. 82 5.2.2 The TCN-S Concept Graph . 84 5.2.3 The TCN-S Instance Graph . 84 5.2.4 TCN Concept Graphs . 85 5.2.5 TCN Instance Graphs . 85 5.3 Chapter Summary . 86 Part III System Implementation 6 d.store: A Resource-oriented Team Collaboration Network System ........................................ 89 6.1 Platform Architecture Overview . 89 6.1.1 Client Applications . 89 6.1.2 RDF/OWL Graph Component . 90 6.1.3 Service Interface . 91 6.2 The d.store Concept Graph . 92 6.3 Processing Temporal Network Properties . 94 6.3.1 Storing Temporal RDF Statements . 94 6.3.2 Modifying the RDF/OWL Subsystem . 96 6.3.3 Modifying the Relational Storage Interface . 97 6.3.4 Advantages and Disadvantages of the Approach . 100 6.4 Implementing the Service Interface . 101 6.4.1 Platform Resources . 101 6.4.2 Exploring Team Collaboration Network Resources . 102 6.4.3 Manipulating Team Collaboration Network Resources 106 6.5 Chapter Summary . 108 7 System Configuration ................................... 109 7.1 Domain Ontologies for Online Collaboration . 109 7.1.1 web: An Ontology for Hyperlinked Collaboration Resources . 110 7.1.2 wiki: An Ontology for Wiki-based Collaboration . 111 7.1.3 email: An Ontology for Email-based Messaging . 111 7.1.4 file: An Ontology for Shared Document Storages . 113 7.2 Inference Rules . 115 7.3 Preparing the Data Collection Process . 116 7.3.1 Initializing the Networks . 116 7.3.2 Setting up the Sensor Clients. 117 7.3.3 Specifying Participant Roles and Alias Names .