Towards Value-Sensitive Learning Analytics Design
Total Page:16
File Type:pdf, Size:1020Kb
Towards Value-Sensitive Learning Analytics Design Bodong Chen Haiyi Zhu University of Minnesota University of Minnesota Minneapolis, MN, USA Minneapolis, MN, USA [email protected] [email protected] ABSTRACT The increasing emphasis on ethical considerations in learning ana- To support ethical considerations and system integrity in learning lytics is situated within growing public concerns with algorithmic analytics, this paper introduces two cases of applying the Value opacity and bias, as well as awakening advocacy of algorithmic Sensitive Design methodology to learning analytics design. The transparency and accountability [1]. In the field of learning ana- first study applied two methods of Value Sensitive Design, namely lytics, a rich body of theoretical, practical, and policy work has stakeholder analysis and value analysis, to a conceptual investi- emerged to promote the ethical collection and use of learning data. gation of an existing learning analytics tool. This investigation In 2016, the Journal of Learning Analytics published a special section uncovered a number of values and value tensions, leading to design featuring work on ethics and privacy issues [13]. Ethical frame- trade-offs to be considered in future tool refinements. The second works, codes of ethical practice, and ethical consideration checklists study holistically applied Value Sensitive Design to the design of a have been developed [11, 25, 32]. New policy documents are also put recommendation system for the Wikipedia WikiProjects. To proac- forward by higher education institutions to enhance transparency 1 tively consider values among stakeholders, we derived a multi-stage with regard to data collection and data usage. design process that included literature analysis, empirical investi- In contrast with the private sector, education as a social func- gations, prototype development, community engagement, iterative tion also has a moral responsibility to promote student success. testing and refinement, and continuous evaluation. By reporting on Therefore, ethical considerations in learning analytics need to go these two cases, this paper responds to a need of practical means beyond issues that are foregrounding the current public discourse to support ethical considerations and human values in learning an- surrounding surveillance and privacy [13]. Educational institutions alytics systems. These two cases demonstrate that Value Sensitive have “an obligation to act”—to effectively allocate resources to fa- Design could be a viable approach for balancing a wide range of cilitate effective teaching and learning through ethical and timely human values, which tend to encompass and surpass ethical issues, interventions [26]. Ethical practice in learning analytics entails in learning analytics design. considerations of not only typical ethical issues but also factors and values pertinent to the larger education system. Recently, Buck- CCS CONCEPTS ingham Shum [2] proposes a Learning Analytics System Integrity framework that goes beyond algorithmic transparency and account- • Information systems → Data analytics; Social networking ability to consider multiple pillars of a learning analytics system sites; • Human-centered computing → Visual analytics; • Ap- including learning theory, algorithm, human–computer interaction, plied computing → Collaborative learning; pedagogy, and human-centered design. While learning analytics KEYWORDS as a field is rightfully giving more attention to ethical challenges with data analytics, to foster ethical practice we also need effective learning analytics, values, value sensitive design, data ethics, social methods to consider ethical challenges in a manner that would media engage perspectives from different stakeholders while allowing ACM Reference Format: education—as a complex system—to fulfill its societal function. Bodong Chen and Haiyi Zhu. 2019. Towards Value-Sensitive Learning An- To this end, this paper proposes to apply Value Sensitive De- alytics Design. In The 9th International Learning Analytics & Knowledge sign—a theory and methodology initially developed in the field Conference (LAK19), March 4–8, 2019, Tempe, AZ, USA. ACM, New York, NY, of Human–Computer Interaction (HCI)—to support the design of arXiv:1812.08335v2 [cs.HC] 28 Jan 2019 USA, 10 pages. https://doi.org/10.1145/3303772.3303798 value-sensitive learning analytics systems. We posit that Value Sen- sitive Design can (1) guide researchers and practitioners in the 1 INTRODUCTION process of evaluating and refining existing learning analytics ap- Over the past few years, considerations of the ethical implications plications, and (2) provide systematic guidance on developing new of learning analytics have become a central topic in the field [27]. learning analytics applications to more holistically address values pertinent to stakeholders, educators, and the society. Permission to make digital or hard copies of all or part of this work for personal or The remainder of this paper is structured as follows. We first classroom use is granted without fee provided that copies are not made or distributed briefly review work on ethics frameworks, algorithmic transparency, for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the and accountability in the field of learning analytics. Then we turn author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or to the Value Sensitive Design literature, with a focus on its appli- republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. cation to the development of information systems. After outlining LAK19, March 4–8, 2019, Tempe, AZ, USA our research goals, we introduce two studies in Sections 3 and 4. © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-6256-6/19/03...$15.00 https://doi.org/10.1145/3303772.3303798 1See the SHEILA project for examples: http://sheilaproject.eu/la-policies/ LAK19, March 4–8, 2019, Tempe, AZ, USA Chen & Zhu We conclude this paper by discussing the importance of consid- learning analytics, we need means to interrogate the values that are ering values and value tensions in learning analytics design and often in tension with each other. The learning analytics accountabil- development. ity analysis put forward by Buckingham Shum [2] considers system integrity from multiple angles and holds promise for eliciting values 2 BACKGROUND beyond current thinking on ethical issues. Take automated feedback 2.1 Learning Analytics Systems: Ethics and on academic writing for example. Values supported by an academic writing analytics tool include the accurate detection of rhetorical Values moves based on linguistic features and usable feedback interface Learning analytics is defined as “the measurement, collection, anal- that can facilitate student sense-making. However, the accurate ysis and reporting of data about learners and their contexts, for detection of rhetorical moves may demand more learner data and is purposes of understanding and optimising learning and the envi- hereby in tension with the value of privacy foregrounded in ethical ronments in which it occurs” [24, p. 34]. By harnessing increasingly considerations. To better support learning analytics design, the abundant educational data and data mining algorithms, learning field needs strategies for weighing competing values and deriving analytics aspires to build applications to personalize learning based design trade-offs to mitigate value tensions. on individual progress, predict student success for timely inter- vention, provide real-time feedback on academic writing, and so forth [33]. As learning analytics applications are increasingly in- 2.2 Value Sensitive Design tegrated in the existing academic technology infrastructure (e.g. Stemming from an orientation towards human values, Value Sensi- student information systems, learning management systems, stu- tive Design “represents a pioneering endeavor to proactively con- dent support systems), more educational institutions are becoming sider human values throughout the process of technology design” equipped with strong, connected analytics capacity to better under- [8, p. 1]. In Value Sensitive Design, value is operationally defined stand students and hopefully to also better support student success. as “what is important to people in their lives, with a focus on ethics Empirical evidence accumulated so far has indeed demonstrated and morality” [15, p. 68]. Value Sensitive Design offers a systemic promise of learning analytics in supporting student learning [21]. approach with specific strategies and methods to help researchers While learning analytics are garnering international interests and designers explicitly incorporate the consideration of human across different levels of education, ethical and privacy concerns values into design [8, 15]. It recognizes the complexity of social life are increasingly mentioned in both public and scholarly discourse. and attempts to illuminate value tensions among stakeholders. Slade and Prinsloo [34] listed a number of ethical challenges in Value Sensitive Design is a theory, a methodology, and an estab- learning analytics, including the location and interpretation of data; lished set of methods [8]. As a methodology, Value Sensitive Design informed consent, privacy, and