DESIGNING THE SAKAI OPEN ACADEMIC ENVIRONMENT: A DISTRIBUTED COGNITION ACCOUNT OF THE DESIGN OF A LARGE SCALE SOFTWARE SYSTEM VOLUME 1 A Dissertation Presented to The Academic Faculty By Klara Benda In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in Human-Centered Computing in the College of Computing, School of Interactive Computing Georgia Institute of Technology August 2014 COPYRIGHT © 2014 BY KLARA BENDA DESIGNING THE SAKAI OPEN ACADEMIC ENVIRONMENT: A DISTRIBUTED COGNITION ACCOUNT OF THE DESIGN OF A LARGE SCALE SOFTWARE SYSTEM Approved by: Dr. Nancy J. Nersessian, Advisor Dr. Elizabeth Mynatt School of Interactive Computing School of Interactive Computing Georgia Institute of Technology Georgia Institute of Technology Dr. Colin Potts Dr. Hanne Andersen School of Interactive Computing Department of Physics and Astronomy Georgia Institute of Technology - Centre for Science Studies Aarhus University Dr. Geoffrey C. Bowker Department of Informatics University of Irvine Date Approved: [May 12, 2014] ACKNOWLEDGEMENTS I would like to thank my advisor, Professor Nancy Nersessian, for taking me on as a last student, and for the intellectual support and encouragement that she expended for my dissertation work. I also want to thank the School of Interactive Computing at Georgia Tech for the financial support I received throughout my doctoral studies. Finally, I will also thank my husband, Bence, for embarking on an international adventure with me, and having the patience and good humor to support me in all adversities. iii TABLE OF CONTENTS ACKNOWLEDGEMENTS iii LIST OF TABLES x LIST OF FIGURES xi SUMMARY xv CHAPTER 1. Introduction 1 Outline of chapters 3 Chapter 1. Introduction 3 Chapter 2. Problem formulation 3 Chapter 3. Method 3 Chapter 4. Creating the space for innovation in domain-driven open source 4 Chapter 5. Constructing a new conceptual model of user experience 4 Chapter 6. UX-driven design 4 Chapter 7. The social-technical gap 4 Chapter 8. Knowing the contexts of use 4 Chapter 9. Infrastructural implosion 5 Chapter 10. Discussion 5 CHAPTER 2. Problem statement 6 2.1. Social approaches to technological change and software design 7 2.1.1. Responses to technological determinism 7 2.1.2. Two threads in social accounts of technological change 11 2.1.3. Human-centered approaches to the design of software 17 2.2. Cognitive processes in design 20 iv 2.2.1. Research on design and creativity 20 2.2.2. Distributed cognition in design 23 2.3. Participation in cognitive process 26 2.4. Revisiting context in light of distributed cognition 29 2.5. The practice framework 33 2.6. The research questions addressed by the research 38 CHAPTER 3. Methodology 40 3.1. General considerations behind research design 40 3.1.1. Validity 40 3.1.2. Reliability 42 3.2. Outline of the ethnographically-informed cognitive-historical method 43 3.2.1. Characteristics of practice studies 43 3.2.2. Applying practice-based methods to online practices 46 3.3. Data collection 50 3.4. Data analysis 55 3.4.1. Iteration between data collection and theorizing 56 3.4.2. The comparative approach 63 3.4.3. Theoretical sampling 65 CHAPTER 4. Creating the space for innovation in domain-driven open source 68 4.1. Introduction 68 4.2. Domain-driven open source and innovation 69 4.2.1. Building software for higher education from within 69 4.2.2. Licensing 75 v 4.2.3. Innovation in open source development 75 4.3. Making space for design 79 4.3.1. Open source governance in Sakai: creating the conditions of epistemic collaboration 80 4.3.2. Reflecting on architecture for organizing epistemic work with software 87 4.3.3. Creation of a design space for a new Sakai 94 4.3.4. Guiding design through the web of discourse 99 4.4. Discussion 105 CHAPTER 5. Constructing a new conceptual model of user experience 116 5.1. Designing a new interface: conceptual construction from content widget to group dashboard 119 5.1.1. The starting point: the content model 119 5.1.2. Groups enter the scene 123 5.1.3. Exploring new conceptual models of groups 133 5.1.4. Going beyond initial group conceptualizations 143 5.2. Discussion 157 CHAPTER 6. Design framed by professional methods 163 6.1. The framing of new technologies by the professions 163 6.2. A threefold strategy for the construction of meaning 165 6.2.1. Prototyping 167 6.2.2. UX-led design: tools from professional practice 174 6.2.3. User experience as a source of coherence in conceptual modeling 185 6.3. Unsuccessful professional framing efforts 189 vi 6.3.1. Investigation project: Glimpses of a socio-technical system 190 6.3.2. The Instructional Visioning initiative 196 6.4. Discussion 206 6.4.1. The role of prototypes and prototyping in framing the direction of design 206 6.4.2. Missing perspectives 209 6.4.3. The framing role of professions 211 VOLUME 2. 212 CHAPTER 7. THE SOCIAL-TECHNICAL GAP 213 7.1. Introduction 213 7.2. Case study: the lack of social perspectives in design 214 7.2.1. The failures of Sakai 214 7.2.2. The origins of the kernel and the kernel team 219 7.2.3. The introduction and failure of Jackrabbit 221 7.2.4. Participants explain the failure to themselves 226 7.2.5. Making components work together: the kernel team’s approach to development 229 7.2.6. The failure of Sparse Map and Solr 241 7.2.7. The disconnect of the back end 247 7.3. Discussion 257 CHAPTER 8. Knowing the CONTEXTS OF USE – a distributed cognitive-epistemic strategy 275 8.1. The empirical nexus: mediating contexts of use to software design 275 8.2. Introduction to case studies 288 vii 8.3. First case study: Conceptually grounded collections 290 8.3.1. A glossary 292 8.3.2. A spreadsheet outlining group types in higher education. 296 8.3.3. Mental models collection 300 8.3.4. Contextual collections 315 8.3.5. Discussion of the first case study 330 8.4. Second case study: Memory practices and the contexts of use 332 8.5. Discussion: reconfiguring the empirical and analytic stance in the social approach to software design 341 CHAPTER 9. Infrastructural implosion 346 9.1. Designing within an evolving software landscape 346 9.2. Case study 349 9.2.1. Practical reasons for the use of open-source components 349 9.2.2. Epistemic engagement with open source 352 9.2.3. Contribution to the design space through promise and validation 365 9.2.4. Infrastructural implosion 374 9.3. Discussion 378 9.3.1. What is Sakai 3? 378 9.3.2. Infrastructural implosion as an alternative evolutionary account of technology 381 CHAPTER 10. DISCUSSION AND CONCLUSION 386 10.1. Discussion of the analysis of the case studies 386 viii 10.1.1. A distributed cognitive account of the domain-driven design of a software system in higher education 389 10.1.2. Replacing social distribution of interpretations with the social distribution of cognitive process 398 10.1.3. Social-technical imagination within a distributed cognitive process of design 401 10.1.4. Reinserting conceptual construction within social theory 404 10.2. Limitations 408 10.3. Contributions 412 10.4. Implications for further work and practice 413 References 417 ix LIST OF TABLES Table 3.1: Overview of content in the core corpus ........................................................... 53 Table 3.2: Example of data display created for a Confluence page .................................. 59 Table 3.3: Example of data display created for the same Confluence page ..................... 60 Table 3.4: Excerpt of the timeline used for keeping track of the evolution of S/OAE ..... 62 Table 4.1: Overview of the social-organizational ecosystem of the Sakai community .... 74 Table 4.2: Overview of projects related to Sakai 3 ........................................................... 84 Table 6.1: Professional methods in Sakai 3 projects ...................................................... 175 Table 7.1: Overview of the server team's account of epistemic reasons behind Sakai OAE's challenges ............................................................................................................ 229 Table 8.1: Examples of perspectives given as columns for describing group types in the spreadsheet ...................................................................................................................... 297 Table 8.2: Selection of entries from the Group types matrix spreadsheet ...................... 299 Table 8.3: Overview of domains covered by the contextual collections ........................ 316 Table 8.4: Epistemic grounding of contextual scenarios in terms of educational context and software .................................................................................................................... 330 x LIST OF FIGURES Figure 4.1: Tools within Sakai 2’s portal architecture ...................................................... 90 Figure 4.2: The Announcements Widget, relying on Sakai 2 data, is set in parallel with other web-based widgets on a user’s desktop ................................................................... 91 Figure 4.3: The division of labor between Java and UX programmer in a presentation by Ian Boston about the widget approach. ............................................................................. 92 Figure 5.1: My diagram of the widget model as a context of functionality and content 120 Figure 5.2: A comparison of Sakai 2’s course site with a dashboard-type page, based on my diagrams .................................................................................................................... 120 Figure 5.3: A chart with a conceptual