CORE Metadata, citation and similar papers at core.ac.uk Provided by Open Access LMU ASSESSMENT OF (COMPUTER-SUPPORTED) COLLABORATIVE LEARNING This is a post-print of an article submitted for consideration in the IEEE Transactions on Learning Technologies © 2011 Institute of Electrical and Electronics Engineers. Personal use of this manuscript is permitted. Permission from Institute of Electrical and Electronics Engineers must be obtained for any other commercial purpose. This article may not exactly replicate the published version, due to editorial changes and/or formatting and corrections during the final stage of publication. Interested readers are advised to consult the published version which can be found at: http://www.computer.org/csdl/trans/lt/2011/01/tlt2011010059-abs.html doi:{10.1109/TLT.2010.37} Please refer this manuscript as: Strijbos, J. W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies , 4, 59-73. 2 IEEE TRANSACTIONS ON XXXXXXXXXXXXXXXXXXXX, VOL. #, NO. #, MMMMMMMM 1996 Assessment of (Computer-Supported) Collaborative Learning J.W. Strijbos Abstract —Within the Computer-Supported Collaborative Learning (CS)CL research community there has been an extensive dialogue on theories and perspectives on learning from collaboration, approaches to scaffold (script) the collaborative process, and most recently research methodology. In contrast, the issue of assessment of collaborative learning has received much less attention. This article discusses how assessment of collaborative learning has been addressed, provides a perspective on what could be assessed, and highlights limitations of current approaches. Since assessment of collaborative learning is a demanding experience for teachers and students alike, they require adequate computer-supported and intelligent tools for monitoring and assessment. A roadmap for the role and application of intelligent tools for assessment of (CS)CL is presented. Index Terms —Assessment, Collaborative learning, Computers and Education, Psychology —————————— —————————— 1 INTRODUCTION ollaborative learning (CL) is nowadays a common Assessment criteria, in turn, are shaped by the purpose C practice at all levels of education. Since the 1970s of assessment. Broadly two purposes are distinguished: small group dynamics have been intensively studied summative and formative [7], [8]. Summative assessment in education (for a detailed historial overview see [1]). (also referred to as ‘assessment of learning’) is Research in the 1980s focused initially on face-to-face decontextualised and individualistic, it is isolated from student-student interaction in primary education, but it the learning process, and it takes place only at the end of was soon extended to secondary and higher education. a course to judge how well a student performed. Rapid developments in computer-mediated Summative assessment focuses strongly on the cognitive communication in the late 1980s led to a new discipline in aspects of learning, often applies a single performance the 1990s now referred to as Computer-Supported score, and it is designed and conducted by the teacher. Collaborative Learning (CSCL). Formative assessment (also referred to as ‘assessment for The introduction of computers in collaborative learning’) is contextualised and aims to build a learning induced a shift in the behaviors being studied. comprehensive picture of learners’ characteristics. It is an Initially research focused on individual cognitive learning integral part of a learning process and takes place several gain (1970–1990), whereas the rise of computer times during a course rather than only at the end. technology stimulated a shift in studying how group Formative assessment focuses on cognitive, social, and process affects individual and group cognition (1990– motivational aspects of learning, often applies a multi- present). At present there are roughly two dominant foci method approach and it leads to a profile instead of a in (CS)CL research: (a) understanding (successful) CL single score. Although distinguishing both purposes can practices, and (b) determining effective conditions for be useful, it should be kept in mind that any assessment – (succesful) CL [2], [3]. when communicated to a student – involves the use of Although it is common knowledge that assessment can feedback information and whether the use of this strongly influence learning [4], interestingly assessment information is more summative or formative, is an issue of CL has thus far remained an implicit issue – despite of interpretation rather than one of absolutes [7], [9], [10]. several publications highlighting the diverse theoretical Alike any other individual learning context, key to the and methodological positions in (CS)CL [2], [5], [6]. In assessment of CL is the operationalisation of relevant CL essence, what are considered relevant CL outcomes outcomes, i.e ‘why and what to assess’. The next section governs their (a) operationalisation and (b) subsequent first reviews assessment in CL approaches developed in measurement. The assessment of CL is directly shaped by the 1970s and 1980s, followed by common approaches to what is measured; however, it is more than merely assessment of CL. Subsequently four metaphors on measurement because it also contains a statement on the learning in CL are discussed, and it concludes with three quality of the CL process or product in relation to major challenges for assessment of CL. The third section prespecified criteria. presents an alternative framework of (CS)CL that explicitly addresses these challenges. The final section ———————————————— will discuss the need to align CL and its assessment, the • J.W. Strijbos is with the Institute of Education and Child Studies, Leiden teacher’s role in assessment of CL, the students’ role, and University, the Netherlands, E-mail: [email protected]. how technology can offer them (intelligent) tools for monitoring and assessment of CL. Manuscript received (insert date of submission if desired). Please not e that all acknowledgments should be placed at the end of the paper, before the bibliography. xxxx-xxxx/0x/$xx.00 © 200x IEEE AUTHOR ET AL.: TITLE 3 2. WHY ASSESS AND WHAT TO ASSESS ? Cohen’s [19] seminal review reoriented CL research stressing the role of tasks and interaction over the ‘reward Presently CL researchers widely agree that it is crucial for versus goal interdependence’ debate. Nowadays, it is collaborative learning that all group members perform a agreed that individual accountability and positive fair share of the task and that problems arise when one or interdependence are both crucial preconditions for several group members do not [2], [3]. This lack of potential productive interaction in any CL approach [21]. individual effort is referred to as social loafing and free- Moreover, most current CL methods achieve individual riding. Social loafing is a tendency to reduce the accountability by other means than reward individual effort when working in a group, as compared interdependence, for example through the use of roles to the individual effort expended when working alone [22], [23]. [24], [25], rather than through assessment [11]. Free-riding exists when an individual does not bear (reward structures). a proportional amount of the CL process and yet s/he shares the benefits of the group [12]. Social loafing and 2.2 Common Approaches to Assessment of CL free-riding are two often voiced complaints regarding Although assessment of CL gained attention in the past unsatisfactory assessment of CL. decade in face-to-face [26], [27], [28] and online [29], [30], 2.1 Assessment and Accountability [31], [32], [33], [34] CL contexts, it is currently still (a) mostly summative, (b) designed and conducted by the Understanding the current CL assessment practices teacher, (c) consists of individual tasks (i.e., tests, quizzes, requires revisiting the theoretical origins of the essays, etc.), a group task with each student receiving the approaches developed in the 1970s and 1980s – at the same grade or a combination of group and individual time referred to as ‘cooperative learning’. Two basic CL tasks, and (d) nearly exclusively focused on cognitive principles were introduced early-on to ensure that all outcomes (achievement). Especially group grades and/or members contribute to the group: individual grading based on a mix of group and individual tasks can accountability [13] and positive (goal) interdependence be problematic in a practical as well as theoretical sense. [14]. With respect to assessment of CL Slavin’s [15], [16] Kagan [35] argues that group grades should never be original operationalisation of individual accountability is used, despite the appeal for grading efficiency, because: of particular interest. Slavin’s view emphasizes reward (a) they violate individual accountability and invite free- and goal structures under which students operate. More riding or that the most able group member conducts most importantly, it emphasizes that group rewards enhance (or all) of the work (‘sucker effect’; see [36], [37]), (b) an student learning only if group rewards are based on the individual student typically has little influence on group individual learning of all group members. In most of formation and due to the coincidental presence of high or Slavin’s CL methods (e.g., Student Teams Achievement low achieving students or a free-rider, a group grade Divisions (STAD), students take individual quizzes