Jean-Claude Falmagne · Dietrich Albert Christopher Doble · David Eppstein Xiangen Hu Editors
Knowledge Spaces
Applications in Education
Knowledge Spaces
Jean-Claude Falmagne • Dietrich Albert Christopher Doble • David Eppstein • Xiangen Hu
Editors
Knowledge Spaces
Applications in Education
Editors
Jean-Claude Falmagne School of Social Sciences, Dept. Cognitive Sciences University of California, Irvine Irvine, CA, USA
Dietrich Albert Department of Psychology University of Graz Graz, Austria
Christopher Doble ALEKS Corporation Irvine, CA, USA
David Eppstein Donald Bren School of Information
& Computer Sciences University of California, Irvine Irvine, CA, USA
Xiangen Hu Department of Psychology University of Memphis Memphis, TN, USA
- ISBN 978-3-642-35328-4
- ISBN 978-3-642-35329-1 (eBook)
DOI 10.1007/978-3-642-35329-1 Springer Heidelberg New York Dordrecht London
Library of Congress Control Number: 2013942001
© Springer-Verlag Berlin Heidelberg 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Preface
As can be gathered from the database maintained by Cord Hockemeyer at the University of Graz1, the literature on Knowledge Spaces is expanding rapidly. The recent book by Falmagne and Doignon (2011) gives a comprehensive account of the theory up to its date of publication. However there have been important developments after that. We thought that a volume gathering some of these developments would be timely.
This volume has two parts. Part I describes a number of chosen empirical works. Part II deals with recent theoretical results. Two chapters play a special role. The first chapter in Part I, called ‘Overview’, gives an informal, intuitive presentation of all the important concepts of learning space theory, which is an important special case of knowledge space theory. This chapter is intended for readers primarily interested in the applications of learning space theory. Chapter 8, the first chapter in Part II, gives a technical presentation of the theory, including all the main theorems (without the proofs) and a description of some important algorithms. This chapter may serve as the basic reference for the works described in Part II.
In Part I, Chapters 2 to 6 describe some applications of the ALEKS system in education. The ALEKS system is the most elaborate application of learning space theory to date. It is equipped with an assessment module and a learning module. This system is bilingual (English and Spanish) and currently covers all of K–12 mathematics2 (excluding calculus), and beginning chemistry. Using extensive assessment and learning data, Chapter 2 investigates various statistical measures of the validity of the assessment and learning modules of the ALEKS system. Chapter 3 reports an application of the ALEKS system at the University of Illinois. The chapter demonstrates that the use of ALEKS as a placement test replacing the SAT and the ACT resulted in a substantial decrease of F grades and withdrawals. Chapter 4 describes two studies evaluating whether online individualized instruction by the ALEKS system will increase student scores on a standardized high-stakes test. (The answer is ‘Yes.’) The students were elementary and middle school students of the Memphis area schools. The ALEKS system was used at the University of Memphis to teach a statistics course to students majoring in psychology or social sciences. Chapter 5 compares the results for black and white students in this course with those obtained in a traditional lecture type course. While the white students, on the
1See: http://liinwww.ira.uka.de/bibliography/Ai/knowledge.spaces.html.
2Some of these subjects are also used in colleges and universities.
V
- VI
- Preface
average, do much better than the black students in the lecture type course— which is consistent with traditional results—the discrepancy disappears in the online course. Chapter 6 describes an application of the ALEKS system to the teaching of General Chemistry to college students. The chapter presents a selection of learning and assessment data for the course, and gives interpretations of those data in the framework of knowledge space theory.
Chapter 7 analyzes the ability of students to make logical connections between fundamental chemical principles and the various representations of chemical phenomena. The ALEKS system is not used in this study. The authors build their own knowledge structures.
As mentioned earlier, the introductory chapter of Part II, Chapter 8, gives a condensed description of the most important mathematical results. Distinguishing between behavioral performance and its underlying skills and competencies, Chapters 9 and 10 deal with performance while Chapters 11 and 12 focus on competencies; all of them go well beyond Albert and Lukas (1999). Chapter 9 describes recent extensions in knowledge space theory (multiple answer alternatives, relations between sets/tests), relationships between knowledge space theory and other theoretical approaches (formal concept analysis, FCA; latent class analysis, LCA; item response theory, IRT) as well as methods for data driven generation of knowledge structures, their empirical validation (item tree analysis, ITA; inductive item tree analysis, IITA; measures and indices of t) and respective software resources. Methodological considerations and applications in Chapter 10 exemplify empirical research dealing with generating and validating knowledge structures for sets of items or tests. The different skill- and competence-oriented approaches have been developed independently. Thus, Chapter 11 for the first time relates systematically these approaches to each other by presenting a united framework which allows for identifying their commonalities and differences. These approaches are further developed in Chapter 12, which asks how to deal with distributed information, how to formulate a probabilistic approach, how to link observable navigation and problem solving behavior to cognitive and competence states, how to support self-regulated learning behavior, and how to assess competencies in educational games noninvasively. Furthermore, respective applications in technology enhanced learning and competence management are described.
Chapters 13 and 14 describe a data structure, the learning sequence, that can be used to efficiently implement learning-space based systems. In Chapter 13, learning sequences are applied to the tasks of generating the states of a learning space and using learning spaces to assess the knowledge of a learner. Chapter 14 discusses the use of learning sequences to project learning spaces onto smaller sets of concepts (important for the efficiency of assessment in large learning spaces) and to modify learning spaces by adding or removing states.
- Preface
- VII
We are most grateful to all the referees whose reports led to improvements of the presentation of the works described in this volume. We thank in particular Eric Cosyn, Cornelia Dowling, Yung-Fong Hsu, Mathieu Koppen, Jeff Matayoshi, Alexander Nussbaumer, Martin Schrepp, Luca Stefanutti, Reinhard Suck, and Nicolas Thi´ery. We also thank Brian Junker and Don Laming for their useful reactions to a presentation of the material in Chapter 2.
The Editors March 11, 2013.
Contents
Part I LEARNING IN A KNOWLEDGE SPACE 1
23
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Jean-Claude Falmagne and Christopher Doble
3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Basic Concepts: Knowledge States and Spaces . . . . . . . . . . . . . . .
35
1.3 Uncovering a Knowledge State . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.4 The Case of Very Large Knowledge Spaces. . . . . . . . . . . . . . . . . . 18 1.5 Historical Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.6 A graph of a Larger Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.7 Nontechnical Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Assessing Mathematical Knowledge in a Learning Space . . . 27
Eric Cosyn, Christopher Doble, Jean-Claude Falmagne,
- Nicolas Thiery and Hasan Uzun
- Arnaud Lenoble,
´
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2 Measuring the Validity/Reliability of an Assessment . . . . . . . . . 28 2.3 Statistical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.4 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.5 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.6 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
ALEKS-based Placement at the University of Illinois. . . . . . . . . 51
- Marc Harper
- Alison Ahlgren Reddy and
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.2 Placement at the University of Illinois . . . . . . . . . . . . . . . . . . . . . . 52 3.3 The University of Illinois Math Placement Program . . . . . . . . . . 52 3.4 Description of the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
IX
- X
- Contents
3.5 Analysis of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
- 4
- The Impact of a Mathematical Intelligent Tutoring
System on Students’ Performance on Standardized High-Stake Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
- Jeremiah Sullins, Rob Meister,
- William M. Wilson,
- Scotty D. Craig,
Xiangen Hu
Anna Bargagliotti and
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.2 ALEKS in Memphis Area Schools . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3 Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.4 Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
- 5
- A Potential Technological Solution for Reducing the
Achievement Gap Between White And Black Students . . . . 79
- Xiangen Hu, Yonghong Jade Xu,
- Charles Hall,
Kristin Walker and Theresa Okwumabua
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.2 Narrowing the Achievement Gap through Technology in the
Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
67
A Commercial Implementation of Knowledge Space Theory In College General Chemistry . . . . . . . . . . . . . . . . . . . . . 93
Christopher J. Grayce
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 ALEKS Gen Chem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Using Knowledge Space Theory to Assess Student Understanding of Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Mare Taagepera and
Ramesh D. Arasasingham
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 7.2 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 7.3 Determination of Knowledge Structure . . . . . . . . . . . . . . . . . . . . . 120 7.4 Analysis and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
- Contents
- XI
Part II RECENT THEORETICAL PROGRESS
- 8
- Learning Spaces: A Mathematical Compendium . . . . . . . . . . . 131
Jean-Paul Doignon, Jean-Claude Falmagne and Eric Cosyn
8.1 Axioms for Learning Spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 8.2 The Base and the Atoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 8.3 The Fringe Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 8.4 Projections of a Knowledge Structure . . . . . . . . . . . . . . . . . . . . . . 137 8.5 Building a Learning Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 8.6 Probabilistic Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 8.7 The Assessment Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 8.8 About Practical Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . 143
- 9
- Recent Developments in Performance-based Knowledge
Space Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
¨
- Ali Unlu¨
- , Martin Schrepp, Ju¨rgen Heller, Cord Hockemeyer,
Gudrun Wesiak and Dietrich Albert
9.1 Extensions of Knowledge Space Theory (KST) . . . . . . . . . . . . . . 147 9.2 Relationships between KST and other Theoretical Approaches 157 9.3 Methods for Construction and Validation of Knowledge
Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
9.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 9.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
10 Generating and Validating Relations between Sets/Tests . . 193
¨
- Dietrich Albert, Gudrun Wesiak and
- Ali Unlu¨
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 10.2 Basic Concepts of SRbT and Corresponding Test Knowledge
Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
10.3 Generation of Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 10.4 Validation of Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 10.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 10.6 List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
11 Skills, Competencies and Knowledge Structures . . . . . . . . . . . 229
¨
Ju¨rgen Heller, Ali Unlu¨ and Dietrich Albert
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 11.2 Relating Skills to Knowledge Structures . . . . . . . . . . . . . . . . . . . . 230 11.3 Relating Knowledge Structures to Skills . . . . . . . . . . . . . . . . . . . . 235 11.4 Korossy’s Competence-Performance Approach . . . . . . . . . . . . . . . 236 11.5 Extending the Competence-Performance Approach . . . . . . . . . . 240 11.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
- XII
- Contents
12 Recent Developments in Competence-based Knowledge
Space Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Ju¨rgen Heller, Luca Stefanutti and
- Cord Hockemeyer,
- Thomas Augustin,
Dietrich Albert
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 12.2 Distributed Skill Functions and the Meshing of Knowledge
Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
12.3 Probabilistic Competence Approach . . . . . . . . . . . . . . . . . . . . . . . 253 12.4 Linking Navigation Behavior to the Cognitive State in eLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
12.5 Micro-Adaptivity: Non-invasive Skill Assessment in
Educational Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
12.6 Applications in Learning and Competence Management . . . . . . 278 12.7 General Conclusions and Discussion. . . . . . . . . . . . . . . . . . . . . . . . 285
13 Learning Sequences: An Efficient Data Structure for
Learning Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
David Eppstein
13.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 13.2 Learning Spaces and Quasi-Ordinal Spaces . . . . . . . . . . . . . . . . . 289 13.3 From State Generation to Assessment . . . . . . . . . . . . . . . . . . . . . . 290 13.4 Learning Spaces from Partial Orders . . . . . . . . . . . . . . . . . . . . . . . 292 13.5 Learning Spaces from Learning Sequences . . . . . . . . . . . . . . . . . . 296 13.6 Representing States by Numerical Vectors . . . . . . . . . . . . . . . . . . 299 13.7 Generating the States of a Learning Space . . . . . . . . . . . . . . . . . . 301 13.8 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 13.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
14 Projection, Decomposition, and Adaption of Learning
Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
David Eppstein