Higher Education Institutions and Learning Management Systems: Adoption and Standardization

Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal Senior Editorial Director: Kristin Klinger Director of Book Publications: Julia Mosemann Editorial Director: Lindsay Johnston Acquisitions Editor: Erika Carter Development Editor: Myla Harty Production Editor: Sean Woznicki Typesetter: Jennifer Romanchak Print Coordinator: Jamie Snavely Cover Design: Nick Newcomer

Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com

Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data Higher education institutions and learning management systems: adoption and standardization / Rosalina Babo and Ana Azevedo, editors. p. cm. Includes bibliographical references and index. Summary: “This book provides insights concerning the use of learning management systems in higher education institutions, to increase understanding of LMS adoption and usage”--Provided by publisher. ISBN 978-1-60960-884-2 (hardcover) -- ISBN 978-1-60960-885-9 (ebook) -- ISBN 978-1-60960-886-6 (print & perpetual access) 1. Internet in higher education--Case studies. 2. Education, Higher--Computer-assisted instruction- -Case studies. 3. Web-based instruction--Design. I. Babo, Rosalina, 1967- II. Azevedo, Ana, 1977- LB2395.7.H55 2012 378.1’7344678--dc23 2011022151

British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library.

All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. Editorial Advisory Board

António Andrade, Universidade Católica Portuguesa, Portugal Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Schiopoiu Burlea, University of Craiova, Romania Luís Borges Gouveia, University Fernando Pessoa, Portugal Paulo Coelho Oliveira, Instituto Superior de Engenharia do Porto, Portugal Demetrios G Sampson, University of Piraeus, Greece Steve Wheeler, University of Plymouth, UK

List of Reviewers

Kamla Ali Al-Busaidi, Sultan Qaboos University, Sultanate of Oman António Andrade, Universidade Católica Portuguesa, Portugal Judy van Biljon, University of South Africa, South Africa Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Burlea, University of Craiova, Romania Nicholas Caporusso, Institute for Advanced Studies, Italy Lai Yiu Chi, The Hong Kong Institute of Education, Hong Kong Thomas Connolly, University of the West of Scotland, UK Dorota Dżega, West Pomeranian Business School, Poland José Manuel Mesa Fernández, University of Oviedo, Spain Robert W. Folden, Texas A&M University-Commerce, USA Jose Albors Garrigos, Universidad Polytecnica de Valencia, Spain Luís Borges Gouveia, University Fernando Pessoa, Portugal Malinka Ivanova, College of Energetics and Electronics, Bulgaria Arpan Jani, University of Wisconsi, USA Alexandros Karakos, Democritus University of Thrace, Greece Ray Kekwaletswe, Tshwane University of Technology, South Africa Arne Wolf Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Atik Kulakli, Beykent University, Turkey Jack Lee, The Chinese University of Hong Kong, Hong Kong Carla Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Lourdes Moreno, Universidad Carlos III de Madrid, Spain Muhammad Abdul Mugeet, Aga Khan University, Pakistan Lino Oliveira, Escola Superior de Estudos Industriais e de Gestão, Portugal Paulo Oliveira, Instituto Superior de Engenharia do Porto, Portugal Wieslaw Pietruszkiewicz, SDART Ltd, UK Mário Pinto, Escola Superior de Estudos Industriais e de Gestão, Portugal Ricardo Queirós, Escola Superior de Estudos Industriais e de Gestão, Portugal Marina Ribaudo, University of Genova, Italy Sandra Ribeiro, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, Escola Superior de Estudos Industriais e de Gestão, Portugal Lorenzo Salas-Morera, University of Córdoba, Spain Demetrios G. Sampson, University of Piraeus, Greece Anthony Scime, State University of New York, USA Errikos Ventouras, Technological Educational Institution of Athens, Greece Steve Wheeler, University of Plymouth, UK Table of Contents

Foreword...... xv

Preface...... xix

Acknowledgment...... xxiii

Section 1 Generalities and Perspectives

Chapter 1 General Perspective in Learning Management Systems...... 1 Robert W. Folden, Texas A&M University-Commerce, USA

Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness...... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa

Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course...... 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK

Section 2 Implementing and Evaluating

Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs...... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong Chapter 5 ANGEL Mining...... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA

Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems...... 116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman

Section 3 Trends and Challenges

Chapter 7 A Comparative Study on LMS Interoperability...... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal

Chapter 8 Mobile Learning Management Systems in Higher Education...... 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece

Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires...... 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece

Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility...... 199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain

Chapter 11 The Technological Advancement of LMS Systems and E-Content Software...... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK Section 4 Case Studies

Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education...... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal

Chapter 13 LMS Adoption at the University of Genova: Ten Years After...... 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy

Chapter 14 Effective Use of E-Learning for Improving Students’ Skills...... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain

Chapter 15 Strategies of LMS Implementation at German Universities...... 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany

Compilation of References...... 335

About the Contributors...... 360

Index...... 370 Detailed Table of Contents

Foreword...... xv

Preface...... xix

Acknowledgment...... xxiii

Section 1 Generalities and Perspectives

Chapter 1 General Perspective in Learning Management Systems...... 1 Robert W. Folden, Texas A&M University-Commerce, USA

In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective.

Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness...... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa

The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness.

Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web2.0 technologies provided by the chosen LMS.

Section 2 Implementing and Evaluating

Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University. Chapter 5 ANGEL Mining...... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA

This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation.

Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’ supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS. Section 3 Trends and Challenges

A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study.

Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle.

The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems (LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented.

Currently, the great majority of institutions of higher education use Learning Content Management Sys- tems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects. The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle . In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents.

Chapter 11 The Technological Advancement of LMS Systems and E-Content Software...... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK

This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomera- nian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines. Section 4 Case Studies

The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in Education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies.

The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy. Chapter 14 Effective Use of E-Learning for Improving Students’ Skills...... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain

The educational system promoted by the European Higher Education Area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academic performance.

In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms.

Compilation of References...... 335

About the Contributors...... 360

Index...... 370 xv

Foreword

Learning Management Systems (LMS) are now ubiquitous in institutions of higher education. This has occurred very rapidly with adoption being widespread but with little standardization. LMS’s were first used to support delivery with some communication between teachers and learners, but use has now been extended to support learning activities in innovative and diverse ways. They are also being used to increase student engagement and to track student progress – a vastly different approach to the early years of pushing resources to students. Adoption of LMS’s started with experimentation by a few with small systems. Familiarization facilitated wider adoption until eventually the “institution-wide” adoption of large commercial systems became common. Recent developments have extended the range of options from a few large commercial systems to a wider selection of open source, adaptable and specialized systems. In the intervening years from early adoption to now, higher education institutions have also gained valuable expertise in selecting, implementing, using and evaluating technologies to support learning and teaching some of which has been gathered in the chapters of this book. The authors have focused on a number of areas including: implementation strategies; use of learning management systems and other eLearning technologies; technical developments; evaluation; adoption and acceptance; and supporting skills. Chapter 1 takes a broad historical focus, reflecting on the background of eLearning from the very early days of teaching machines and computer assisted instruction through to correspondence courses and video conferencing. The role of information and communication technologies in supporting higher education processes, including teaching and learning, is explored leading to the overarching concept of learning management systems and how these support different modes of learning. The major conclusion reached is that pedagogy should drive the development and use of an LMS. Chapter 2 argues that learning and knowledge is facilitated by social interactions, implying that communication should be a key component of any eLearning system. The author argues that an LMS should facilitate social interactions at a number of levels independent of temporal or geographic constraints or the context that the knowledge activity takes place in. The communication should be facilitated by whatever means are available independent of specific LMS characteristics. The use of Web 2.0 technologies to support learning in a vocational setting is explored in chapter 3. The authors posit that despite the uptake of eLearning technologies in higher education generally, there is less evidence of uptake in the vocational sector. They aim to answer questions about whether technology can supplement the hands-on approach of vocational training, in particular the use of web 2.0 technologies such as wikis and forums. They present a case study the outcomes of which suggest that although there is potential for educational technologies to offer great benefits for vocational training, there is still much work to be completed in this area. xvi

Chapter 4 investigates the use of LMS usage logs to facilitate teaching and learning. This is not an uncommon activity in commercial settings and, in an eLearning situation, can reveal valuable information about how the LMS is being used. The authors explore how the data can be analyzed to better inform their understanding of the online activities and, as a result, inform and improve the eLearning strategies that support institutional, faculty and departmental use of the LMS. In a similar vein, chapter 5 explores usage patterns in a specific LMS, ANGEL. Here data mining is used to explore LMS feature use. Using machine learning techniques it is possible to predict: the courses will use the ANGEL system most effectively based on course specific attributes; the interaction between features or sets of features that impact on usage; and those features that are most commonly used. This leads to a set of results that can be used by the institution to inform future decision making regarding feature selection within an LMS, the design, selection and implementation of an LMS, as well as identifying areas requiring additional training. Chapter 6 further explores usage, investigating critical factors that influence the acceptance and use of an LMS by instructors. The author suggests that although LMS survival is determined by instructor and student use, adoption is initiated by the instructor’s acceptance and subsequent use. Through a comparison of instructors’ acceptance of technology, LMS characteristics and organizational characteristics for the acceptance of eLearning, a model of overall acceptance and use is developed in a distance education setting. The study also provides insights into what additional support is needed in situations where computer use and Internet literacy is not high. In chapter 7 the issue of LMS interoperability with other systems in educational institutions is explored. The authors point out that although the LMS is directed at supporting learning it cannot be isolated from other systems in the institution. Two systems have been selected for the comparison representing a significant market share. A number of facets were selected for the comparison, using currently accepted standards, including: system communication with operational environment; learning content management; and academic management. The overall conclusion is that LMS interoperability leaves a lot to be desired. Standards relating to communication and content are relatively well developed but significant work still needs to be done in the area of interoperability of academic management. The advent of mobile technologies and the impact on course management systems in higher education is explored in chapter 8. The authors address the issue of designing such a system in the context of the Moodle LMS, specifically they investigate the application of W3C Mobile Web Best Practices 1.0. A framework is designed for a server-based mobile version of Moodle that follows the W3C guidelines. The enhancing of electronic examinations is explored in chapter 9. The authors explore an extension to multiple-choice questionnaires which allow for novel grading methodologies to be employed. They suggest that simple positive scoring rules and mixed-scoring rules introduce bias for a number of reasons. However the paired scoring method introduced here overcomes some of the shortcomings of these other methods. The authors do emphasis that the initial workload associated with constructing the question bank for multiple-choice questions is high, but overall it is concluded that the enhancements suggested here add to the value of examination tools within an LMS. With the widespread adoption and use of LMS’s in higher education, the authors of chapter 10 highlight the need for these systems to be accessible to all students. An evaluation framework is developed to evaluate the accessibility of LMS’s based on particular accessibility standards as well as other technological and human criteria. The framework is tested using the widely used LMS, Moodle, using the perspective of visual-impairment. A number of findings are reported, not least of which is that Moodle does not meet accessibly standards fully. xvii

In chapter 11, the authors describe the legal requirements of using eLearning so support distance learning in higher education in Poland. They explore the implications of the regulations and how this has shaped LMS use. They compare the educational processes required of blended- and e-learning pathways and how different technologies can support aspects of those pathways. Extensions to the LMS that are required to support learning needs are discussed. The authors also point out that there are implications for the development of materials (e-content) which increase the cost of production on a number of levels. Finally the authors present an eLearning presentation engine that overcomes some of the difficulties associated with producing e-content. A comparison of Internet and LMS use is undertaken in chapter 12. The authors propose that more attention should be given to users’ Internet skills when developing LMS strategies since most LMS’s are web-based and most students are already adept at navigating the Internet before starting to use the LMS. The results support this proposition but also indicate that consideration should be given to using collaborative platforms such as web 2.0 technologies in place of email since many students are more familiar with using wikis and blogs for example. These outcomes can inform the selection and implementation of suitable digital pedagogical processes and tools that meet both teacher and student needs as well as informing the development of eLearning policies. Chapter 13 presents the process used by one higher education institution to determine an appropriate institution-wide LMS. The institution had a history of pockets of use and innovation to base the selection on. A subsequent evaluation of the adopted system has shown that use of the system is limited to resource sharing and organizational support, the most commonly used activities in the previous collection of systems. The authors suggest that the introduction of an eLearning policy could be instrumental in enhancing and extending the use of the LMS more fully. However they have identified that any extensions of use are beyond the current capacity of the system and support services, a catch-22 situation given the imperative of supporting the modern student digitally. A case study of using eLearning strategies to improve students’ generic skills is presented in chapter 14. Problem-solving, information management, group work and communication skills are the focus of this study. A combined strategy incorporating discussion boards, group tutoring, collaborative learning and peer assessment were implemented together with a number of assessment regimes including surveys, online activity analysis and group evaluations. These have resulted in improved student performance as well as improved perceptions of the accessibility of teachers, even though this is online. The study found that teacher participation is a key factor in motivating students to engage with learning activities as well as to lead discussions. The concluding chapter, chapter 15, explores different strategies that can, and have, been used when implementing an LMS. Two main influences are whether a top-down or a bottom-up approach is adopted. These determine the different levels at which key decisions are made. The authors present three case studies which demonstrate the different approaches. All three use a blend of the two approaches but in different mixes. All institutions eventually implemented the same system despite the different foci. Regardless of strategy it seems the key to success includes good networking and communication throughout the implementation process.

Jo Coldwell Deakin University, Australia xviii

Jo Coldwell joined Deakin in 1997 where she is currently Associate Head (Teaching and Learning) in the School of Informa- tion Technology. Before joining Deakin she gained a wealth of experience in both academia (in Australia) and industry (in both Australia and the UK). Jo was eLearning Manager for the Faculty of Science and Technology for a number of years during which Deakin University undertook the first institution-wide implementation of a major learning management system (WebCT Vista). During this time she was intimately involved in the deployment at University, Faculty and teaching levels. Since 2000 she has taught extensively online and her research interests lie in a number of areas associated with engaging tertiary teachers and learners in and with technology. xix

Preface

E-Learning plays a significant role in education, and its importance increases day by day. Learning environments can take a myriad of distinct forms. Learning Management Systems (LMS) emerge as an important platform to support effective learning environments. According to Wang and Chen (2009), “an LMS employs a range of information and communication technologies to offer an online platform over the Internet, where a whole course can be planned, facilitated and managed by both the teacher and the learner”. In their work it is presented the main functions of some of the LMS nowadays available for educational purposes such as: learning material management, discussion forums, group emailing, audio conferencing, video conferencing, text chat, and whiteboard and synchronous document sharing. For Watson and Watson (2007), the term LMS is used to describe different educational computer applications. LMS is the framework that holds all sides of the learning process, including skills gap analysis. It is responsible to deliver and manage the infrastructural content, to identify and assess individual and organizational learning or training goals, to follow the process in order to reach those goals, and to collect and present data for supervising the learning process of an organization as a whole. With the rising of Web 2.0 and Web 3.0, learning environments are also overflowing Learning Management Systems and Institutions’ boundaries. Learning Management Systems are used all over Higher Education Institutions (HEI) and the need to know and understand its adoption and usage arises. On the one hand, there are different institutional cultures and characteristics and, on the other hand, there are several distinct LMS tools. Considering this it is expected to find out distinct experiences in the adoption and usage of LMS. The richness of each of the experiences can help the worldwide community to better understand how LMS are being used. The most used LMS according to a survey (Babo & Azevedo, 2009) answered by 51 universities from 19 different countries in 5 continents, were Moodle (Moodle, 2009), Blackboard / WebCT (Blackboard, 2009), and Sakay (Sakay Project, 2009). In that study several other LMS were referred such as ItsLearning, Desire2Learn, Claroline, METU Online, Chisimba, High Learn, Formare, Learning Space, First Class, Dokeos, eCollege, Class Fronter, KEWL. The results can be seen as an evolution. In the past years, the proprietary platforms were the most used but currently there is an increase of open source free platforms usage (Bradley et al., 2007). Consequently, there are not many studies regarding the usage level of such tools, concerning students, teachers, tools functionalities, usability, and the entire technological environment. Generally, both proprietary and open source free LMS provide several functionalities, such as, electronic distribution of course syllabi, grades and teachers feedback to students, ability to post hyperlinks to websites, forum for the exchange of ideas, wikis which allows students to swap ideas and information on projects, chat rooms for real time discussion, facilitating emailing and xx

messaging among the participants (teacher/students, students/students), facilities for students to submit work assignments electronically, the means to administer quizzes and texts online (Janossy, 2008). It is frequent to observe that despite LMS on HEI is offered and usage stimulated, only a few of those functionalities are adopted, either by teachers, or by students. LMS are a powerful technology that has not achieved its full potential yet. As far as we know, understanding the actual aspects of LMS usage in HEI is an issue that is not sufficiently explored on research. Consequently, this is an interesting aspect to be explored and studied. The primary objective of this book is to provide insights concerning Learning Management Systems on Higher Education Institutions. The book aims to increase understanding of LMS adoption and usage providing relevant academic work, empirical research findings and an overview of LMS usage on Higher Education Institutions all over the world. The target audience of this book is composed of Education government members, Higher Education Managers, researchers, academicians, practitioners and graduate students in every field of study. LMS are not limited to a specific academic area being a trend and a new learning approach in any scientific field.

BOOK STRUCTURE

This book includes fifteen chapters divided into four sections, namely, LMS Generalities and Perspec- tives, Implementing and Evaluating, Trends and Challenges, and Case Studies. It counted with the collaboration of researchers from 40 different universities and companies from 35 countries. Despite of the overall quality of the received proposals it was not possible to include all of them. As editors and after serious consideration during the review process, supported by our reviewers’ team and by the Editorial Advisory Board, we chose the best chapters in order to achieve the proposed goals of this book and those chapters which better fits the main focus of the book. In the first section Generalities and Perspectives Robert Folden very well understood the need of a general view of all the aspects related with the Learning Management System issue. One of the problems with much of the Scientific Literature is the assumption that all readers have the same background of the writers. This author very well assumes that is rarely true presenting the readers with the necessary foundations to go further in the field to be studied in depth. This is the chapter that any editor would appreciate to open with a book. Chapter 2 lies upon the premise of the asynchrony of the learning and knowledge sharing process which is a “human-to-human process that happen independent of time and space”. A reflection is presented towards “social awareness to determine the appropriateness of a LMS tool” considering asynchrony and ubiquity through all the process.Uncommonly a social concerning contrib- utes from a technological university author. Both chapters are an excellent treatise of LMS in education. In the line of the two previous chapters, Chapter 3 provide us with an insight of the Distance Edu- cation evolution and the evaluation of the use of web 2.0 technologies offered by a chosen LMS in an educational context. Furthermore an interesting case study was developed by University of the west of Scotland and hereby presented. The use of e-learning in vocational courses is well explored in this chapter. Adoption and evaluation are two important and related issues regarding LMS that are presented in section 2. LMS store users’ logs in specific internal databases. These logs contain an immeasurable xxi

wealth that can and should be used to evaluate LMS usage, in order to help Higher Education decision makers to take better decisions regarding LMS policies. Paul LAM, Judy LO, Jack LEE, and Carmel NAUGHT present, in chapter 4, an interesting study developed in the Chinese University of Hong Kong. The study took place during three years, between 2007 and 2009. The authors define three levels of analysis for the e-Learning activities, namely Popularity, Nature, and Engagement. Using these levels, it was possible to become aware of the different types of LMS usage and to define strategies in order to align its usage with the policies of the institution for e-Learning. Also using users’ logs analysis, but with a different even still very interesting perspective, Tyler Swanger, Kaitlyn Whitlock, Anthony Scime, and Brendan Post study, developed in The College at Brockport, State University of New York, is presented in chapter 5. The study was developed during the academic years of 2007 and 2008. Three data mining tasks, namely classification, clustering, and association, are implemented in order to extract useful knowledge and to obtain meaningful insights on LMS evaluation. This chapter refers to a new and interesting data mining application. In chapter 6, Kamla Ali Al-Busaidi, and Hafedh Al-Shihi present a model that intend to explain instructors’ acceptance of LMS. Despite that LMS usage depends on both students and instructs, it is up to the instructor to start the process and thus it is fundamental to understand which factors affect instructors’ acceptance and consequent use of LMS. The presented model is a valuable contribution in this direction. LMS usage in higher education is gaining momentum each day. As a consequence, new trends and challenges arise. In section 3, these issues are explored. José Paulo Leal & Ricardo Queirós explore, in chapter 7, some issues concerning LMS interoperability. In order to analyze and compare some of LMS interoperability features, a framework was developed and tested using Moodle and Blackboard. The framework defines two facets for LMS interoperability, exploring the main related issues in a stimulating, methodical, and efficient manner. Chapter 8 focuses on the use of mobile devices to access LMS supported courses. Demetrios Sampson & Panagios Zervas present a device developed in order to allow the deployment of LMS courses through online devices. This is advantageously achieved by means of the implementation and validation of a mobile version of Moodle that conforms to guidelines proposed by the World Wide Web Consortium. In chapter 9, Dimos Triantis & Errikos Ventouras contribute with the presentation of an interesting grading scheme applied in multiple-choice questionnaires. The presented grading scheme intends to prevent students from guessing the correct answers and thus develop a fair grading system. The proposed methodology is compared with more traditional ones in a real situation, bringing good insights to this issue. In chapter 10, the important theme of LMS accessibility is introduced. Lourdes Moreno, Ana Iglesias, Rocio Calvo, Nuno Delgado, & Luis Zaragoza, evaluated Moodle LMS, studying in detail accessibility concerns regarding visually impaired users. A new method was developed that leads to a better perceptiveness, which can conduct to the definition of better policies and practices. Dorota Dzega & Wieslaw Pietruszkiewicz presents, in chapter 11, a new technological solution designed to address some specific necessities of a higher education institution. Those necessities are also general necessities of a vast majority of higher education institutions. The referred solution is presented as an innovative extension of Moodle LMS, and consists of a new layer in the eLearning platform, offering additional advantages. The case Study section presents four case studies among Portuguese, Spanish, Italian and German universities. While the first three chapters present deep case studies whose main concern is the better understanding how LMS are being used, the German case study focuses in adoption strategies for LMS. xxii

Chapter 12 characterizes the students Internet Usage and their LMS usage patterns in a Portuguese University. Chapters 13 and 14 case studies present long-lasting experiments and observation made in Spanish and in an Italian Universities. Chapter 13 presents the effort made by Genova University over the last 10 years in order to adopt “ICT educational support”. Next chapter relates the adoption of an e- learning system which involved new teaching strategies and take into consideration student’s workload. The last chapter discusses the taken approaches – Top-Down or Bottom-up- of three German universities regarding the LMS implementation process. Not all the universities have had the same approach. Nevertheless the overall goals were achieved.

REFERENCES

Babo, R., & Azevedo, Ana (2009). Learning Management Systems usage on Higher Education Institu- tions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883-889). Blackboard (2009). Blackboard. Retrieved July 30, 2009 from http://www.blackboard.com/ Bradley, M., Carter, J., Fitzsimons, D., Graham, J., Hurlbut, N., Marshall, D., et al. (2007). Learning Management System Evaluation Report. Executive Summary, Humboldt University. Janossy, J. (2008). Proposed Model Evaluating C/LMS Faculty Usage in Higher Education Institutions. Paper presented at MBAA Conference, Chicago, IL. Moodle (2009). moodle.org - Open-source Community-based Tools for Learning. Retrieved July 30, 2009 from http://moodle.org/ Sakay Project. (2009). Sakai Project Home. Retrieved July 30, 2009 from http://sakaiproject.org/portal Wang, Y., & Chen, N. S. (2009). Criteria for Evaluating Synchronous Learning Management Systema: Arguments from the Distance Language Classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007- 0023-y

Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal xxiii

Acknowledgment

We have learned a great deal from those who lead us along our academic career and gratefully acknowledge our debt to them, especially Professor João Álvaro Carvalho. Our students have contributed in a fundamental way to our work with their editorial assistance, namely: Daniela Gonçalves, Joana Oliveira and Maria Duarte. To all the members of the Editorial Advisory Board, to the reviewers and to the authors, whose names are published in this book and who have assisted us one way or another, we feel very much grateful. We would like to express our thanks to the Publishing team at IGI Global for their expert support and guidance. This book is dedicated to our respective families.

Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal Section 1 Generalities and Perspectives 1

Chapter 1 General Perspective in Learning Management Systems

Robert W. Folden Texas A&M University-Commerce, USA

ABSTRACT In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective.

INTRODUCTION with a birds-eye view, you will not initially see the fine detail of any individual topic or aspect Before one begins an extensive study of any of the material of interest, but you will gain an topic, it is to their advantage to view the topic in understanding of the directions that impacts have a general fashion. It is important to look at the come from and being able to better understand historical development of the content, as well as, where the subject is likely to go. some the theoretical underpinnings of the subject. This helps to develop a healthy perspective for viewing the information that will be studied in eLEARNING ROOTS depth. It can also lead one to consider those areas of greatest interest for future research. Beginning In understanding any system, it is important to understand its roots (see Figure 1). As we look at DOI: 10.4018/978-1-60960-884-2.ch001 the past we can understand the present and pos-

sibly project the future (Rose, 2004). Without this same material would have been repeated or new knowledge, we may never understand the present material presented. This form of instruction was and not be able to speculate effectively on where very linear in nature and mastery was the end goal we are going (Rose, 2004). To actually understand (Baggaley, 2008). Others provided some form of the foundation of learning management systems, grading or required mastery before the student was you must begin with a totally different domain allowed to progress. See http://www.greenchame- of knowledge; that of psychology; most notably, leon.com/gc/blog_detail/weve_been_imagined/ educational psychology (Holmberg, 2005). One for pictures showing students using a selection must also look at the developments that have oc- of these machines. curred technologically (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Wagner, N., Hassanein, Computer Assisted Instruction K., Head, M., 2008). Where we are today is an outgrowth of where we have been and it is neces- When computers came on the scene in the 1950’s sary to understand that path if we are to formulate (Watson, W. & Watson, S., 2007), they became the a good sense of where we are going. There are teaching machines and the process was referred multiple generations that we have come through to as computer assisted instruction (CAI) (Rose, (Taylor, 2001). 2004). The foundation of this learning was individualized instruction (Rose, 2004). The theory Programmed Learning/Teaching was that individual students needed to learn at his Machines pace and in his way. This system was also referred to as an Integrated Learning System (ILS) (Rogers, At the beginning of the twentieth century, a group L. & Newton, L., 2001) (Underwood, 1997).The of psychologists were concerned with conditioning instructor ensured the proper design of materials so as an explanation of behavioral adaptation. They logical organization allowed the student to move were generally referred to as ‘behaviorists’. They through the material in an appropriate manner. This believed that all behavior (learning) could be ex- process required that each student had access to a plained by the concept of conditioning. ‘Learning’, computer for the appropriate amount of time for as they saw it, could be accomplished by control- learning to occur. Originally students used ‘dumb ling the use of stimuli and rewards, both positive terminals’ attached to mainframe computers. Each and negative. An outcome of this process was the student could access his or her own files with the development of programmed learning tools (Rose, results stored in a centralized database. In the 2004). Initially these tools were in the form of 1980’s and 1990’s intelligent computers in the booklets that allowed the controller to manage the form of PC’s assumed this role (Keegan, 2002) stimulus by applying the appropriate reward and (Eteokleous-Grigoriou, 2009). These computers thus produced the desired behavior/learning. These were eventually connected through local area booklets were later moved to a mechanical device networks and had the software stored on a cen- or teaching machine (Rose, 2004) (Keegan, 2002). tralized server. Students worked in a networked The material was presented in very small steps environment with the instructor moving about the that were referred to as a frame. The student was classroom to help the students over the difficult then presented with a blank to fill in after which portions and to keep the students on task. they were provided with the correct answer. In their original conception they were not officially graded or ranked, but the student worked through the material. Based upon student performance, the

2 General Perspective in Learning Management Systems

Figure 1. eLearning roots

Computer Managed Instruction more than one course and allowed the students to move forward in the curriculum according to her One of the early goals of computer assisted instruc- individual time frame. Many of the CMI systems tion was to free the instructor from the instructional provided remediation modules that allowed the process and enable her to become a coach for the students to repeat instruction in concepts that they students (Rose, 2004). Therefore, course systems had not sufficiently mastered on previous attempts were designed to allow the student to progress through the material (Underwood, 1997). Student through a set of materials according to a plan es- scores, number of attempts, and other pertinent tablished by the instructor (Brush, T., Armstrong, information were stored in a system database for J., Barbrow, D., & Ulintz, L., 1999) (Underwood, the instructor or course administrator to view 1997). To track the student’s progress and provide (Gilman, D., Emhuff, J., Bender, P., Gower, A., & for a record keeping process, computer managed Miller, K., 1991) (Brush, T., Armstrong, J., Bar- instruction (CMI) became available (Rose, 2004) brow, D., & Ulintz, L., 1999) (Underwood, 1997). (Underwood, 1997). This generally included

3 General Perspective in Learning Management Systems

Correspondence Course contained classroom environment. This approach Learning (CCL) used broadcast television to send the video and the instructor audio (Gaspay, A., Dardan, S., & Beginning in the mid-nineteenth century, students Legorreta, L., 2008). Because of its use of the were able to study in remote locations using television broadcast medium, it tended to be quite correspondence courses (Gaspay, A., Dardan, expensive. In the 1980’s and 1990’s this was S., & Legorreta, L., 2008) (Homberg, 2005). enhanced and somewhat replaced by the sending These text-based programs allowed the students of videotapes to the students to complete their to receive written information and a series of coursework (Taylor, 2001). It was a blend between questions or projects that tested or reinforced the correspondence courses and ITFS. While this desired learning. In most cases the students worked helped to bring the cost down, it did not provide independently (Taylor, 2001). Instructor involve- what either the instructor or the student preferred ment came in the form of written communication and so did not do well in the long run. to students after they had submitted their work for grading (Holmberg, 2005). In the 1970’s and Video Conferencing (VC) 1980’s, these courses were enhanced with audio or video recordings to provide the student with This approach gave way in the 1980’s and 1990’s some content that was not easily communicated to video conferencing systems (Taylor, 2001). by words alone. In most cases the students and These conferencing systems provided two-way instructors were not in the same location, making video and audio links between multiple locations. this a form of distance learning (Keegan, 2002). In most cases this was compressed video and an This was an attempt to enable students who could audio bridge (Baggaley, 2008). While the students not afford, for one reason or another, to travel could see and hear, the quality often left much to to the instructor to receive an education. Again, be desired. Over time, the systems improved and this was basically individualized instruction, but use increased. They were hampered by the inability it was not as individualized as the Programmed of systems to communicate with other manufac- Learning, CAI, or CMI. turers’ systems. Theoretically, they all complied with a uniform set of standards that should have Instructional Television made communication possible, but each manufac- Fixed Service (ITFS) turer included proprietary algorithms that made it virtually impossible. During the latter part of Instructional Television Fixed Service was devel- the decade, there was more cross-collaboration, oped beginning in the 1950s and greatly expanded but it was never an effective method of allowing in the 1970’s and 1980’s as a course delivery remote students to participate in a classroom. platform (Saba, 2000). The students participated in classes that were often remote from the instructor. There was usually an instructor present CIRCLE OF FOCUS with one group of students while the others were at one or more remote sites. In many cases this Classroom Based Learning was two-way video and audio, but was mostly one-way video with two-way audio. The audio Programmed learning, CAI, and CMI focused in those cases was done over an audio bridge. on improving classroom teaching. They were This allowed the students to see the instructor and basically built upon the model of behaviorism interact in much the same was as they would in a which emphasized individualized instruction

4 General Perspective in Learning Management Systems

with instructor interaction. The focus was upon INFORMATION COMMUNICATION improving the learning of students in a contained TECHNOLOGY (ICT) environment with synchronous learning. This can be seen as allowing technology to find a place in While all of this technology development within the classroom. While there was talk about replacing education was going on, information communi- the instructor, it was mostly a changing of the role cation technology was developing rapidly (Gar- of the instructor. Instead of the instructor being rison, 2000). The development of the mainframe the direct dispenser of knowledge, the instruc- computer allowed companies and schools to tor became the facilitator/coach in the learning automate business knowledge acquisition, storage, environment. and processing. It led to many improvements in the functioning of the business side of the edu- Distance Based Learning cational enterprise. Computers began to invade every aspect of business and then began to spread Correspondence course learning, video confer- throughout the lives of individuals. As a result, it is encing, and ITFS focused on distance education nearly impossible to find a major device that does (Guri-Rosenblit, 2005). They were attempting to not possess at least a rudimentary computer. Also, solve the problem of meeting the needs of students individuals often depend on computers to allow who could not or would not participate in a self- them to access information and share information contained classroom (Wagner, N., Hassanein, in an open and free fashion (Hamuy, E. & Galaz, K., Head, M., 2008) (Wagner, N., Hassanein, K., M., 2010). Most individuals find it so difficult to Head, M., 2008). With students distributed over carry on a normal life without computer interac- great distances (Guri-Rosenblit, 2005), there was tion that we are in the process of making wireless little or no interaction with one another. Their internet a ubiquitous factor in our lives (Gaspay, only source of interaction was with the instruc- A., Dardan, S., & Legorreta, L., 2008). tor or the instructor-assigned mentor/proctor. While ITFS and video conferencing theoretically Networking allowed for some interaction, it rarely occurred because everything generally came through the During the 1980’s and 1990’s personal computers instructor and that tended to stifle any student to became networked with one another so that they student interaction. In some cases, distance-based could share information across platforms. They learning was mated with CAI to form a sort of a were able to present information in a graphical hybrid approach. The students would complete manner. This provided for a more enriched en- computer simulations or drill and practice but vironment for the delivery of instruction (Rose, also participate in video conferences to receive 2004) (Gilman, D., Emhuff, J., Bender, P., Gower, input from their instructor. Methodically, distance A., & Miller, K., 1991). The systems could blend education has adapted by using the ever increas- audio and video in such a manner as to appear ing improvements in the information technology seamless. They also allowed for greater interac- to meet the needs of the students and improve tivity with the students (Gilman, D., Emhuff, J., the interaction between and among students and Bender, P., Gower, A., & Miller, K., 1991). Because faculty (Baggaley, 2008). they were networked, students could now work on any available computer to continue the work that they had begun previously. Instructors could view individual student work unobtrusively, providing feedback as necessary. Students could participate

5 General Perspective in Learning Management Systems

in elaborate collaborative environments to de- (Rose, 2004). The technology could fade into velop their individual learning. As the computer the background and allow the course material to continued to improve in capability, the range of move to the forefront, regardless of the overall activities that the students could participate in purpose of the information delivery. The nature continued to expand. of the information delivered was more a matter of perspective than one of substance. Internetworking E-Learning/Virtual Learning In the mid-1990’s we saw the development of internetworking with the introduction of the World This ability to move education materials elec- Wide Web (WWW) (Keegan, 2002). This allowed tronically around the world in both synchronous networks to communicate with other networks and asynchronous manners created the ability to far removed from the original source in a reliable develop a classroom irrespective of time, space, fashion (Rose, 2004). This whole development and proximity (Miller, T. & hutchens, S., 2009). coincided with the development of the graphical Even with VC and ITFS, you needed to be in user interface (GUI) and the use of media rich fairly specific locations to participate in class environments for the delivery of information. activities (Guri-Rosenblit, 2005). While Pro- Technology was developing at a very rapid pace, grammed Learning, CAI, and CMI could be done with processors able to handle larger workloads at remotely, they usually were done with the students much faster rates than ever before. Storage space in one location connected to the same network was cheaper and cheaper to provide (Gaspay, A., and using the same hardware system (Baggaley, Dardan, S., & Legorreta, L., 2008). Sound and 2008) (Guri-Rosenblit, 2005). Today, all of those video delivery systems could provide greater limitations seem irrelevant (Dillenbourg, 2008). enrichment for less expense. These developments Operating on the platform of ICT, educational allowed the designers to enrich their information institutions are delivering all levels of education delivery, ignoring most of the limitations of the and training over the Internet (Hamuy, E. & Galaz, past (Taylor, 2001). M., 2010) (Ozkan, S., Koseler, R., & Baykal, N., Course material could be asynchronously 2009). Corporations seek to reduce their training delivered to many individuals in an individual- expenses by providing training over the Internet ized manner or synchronously delivered to large rather than sending their employees to far off places masses at one time or in some other combination with the related expenses. Even conferences have (Rose, 2004) (Wang, Y. & Chen, N., 2009). As moved to the Internet to provide similar types the Internet became ubiquitous, the barriers to of experiences at the attendees’ desktops rather delivery of information fell away, allowing for than requiring them to attend the conference at ubiquitous access to the information (education) some exotic location. These are also recorded and provided (Taylor, 2001). Students began using made available in an asynchronous manner for computers more and more in their everyday those who could not or would not attend when lives. They became ever more comfortable with the event was going on. computer delivery of information in text, video, ICT is more than just a delivery mechanism; it and audio. This familiarity made it easy to move has become a shaper of what we do (Rose, 2004). educational delivery to this platform (Keegan, It reaches into every area of our lives and has 2002). Students were able to use the Internet to touched nearly every individual on the face of the seek out and obtain information on their own with earth (Guri-Rosenblit, 2005). As the devices that or without the guidance of an instructor/mentor access and share the information have become

6 General Perspective in Learning Management Systems

ever smaller, they have transformed more and other businesses, this focus will enable a clearer more of our lives (Taylor, 2001). They have be- correlation to learning management systems in come the medium for developing communities far the context that we are viewing them. and wide. Individuals can find out even the most We need to remember that nearly all of these mundane or inconsequential pieces of informa- ICT systems existed in paper form before they tion in a split second (Garrison, 2000). They can appeared in electronic form. Most of what was also share the most intimate details of their lives done in the early systems mirrored what had been with the world just as quickly. These tools also done in paper forms. The basic purpose of moving allow instructors and students to share with one these systems to digital format was to automate another in near-real-time, no matter where they the redundant activities in order for the users to are located. Information is not limited to text or focus on those aspects that could not be auto- even text and still images. It is possible to share mated, thereby creating performance improvement full motion video and audio in very high quality (Rentroia-Bonito, A., Martins, A., Guerreiro, T., over very great distances. These videos can be of & Jorge, J., 2008) (Jones, 2004). As the systems real events or simulated reality. This has opened have developed, they have increasingly moved the way to virtual reality to function in an educa- into areas that were not possible to do or were tional environment. easily done in print form (Eteokleous-Grigoriou, This pervasiveness of ICT is forcing academic 2009) (Cradler, 2008) (see Figure 2). institutions at all levels to move their education offerings to an electronic platform to some extent Finance Information Systems (FIS) (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Eteokleous-Grigoriou, 2009). It does not require, The first ICT system to be developed was the at this point, that everything be done in an elec- finance information system. This system allowed tronic medium, but it does mean that some of it enterprises to track income and expenses (Paulsen, must (Taylor, 2001). This push is coming from 2002) and use the information to make informed multiple sources; from almost every stakeholder decisions. It is sometimes referred to as an ac- group that is involved in the educational process. counting information system (AIS). It has been That pervasiveness also means that instructors defined as: A system “ . . . that processes financial have a deeper well of knowledge about their transactions to provide (1) internal reporting to students than ever before (Eteokleous-Grigoriou, managers for use in planning and controlling 2009). While electronic media allow a degree of current and future operations and for non-routine anonymity, it also opens up a greater breadth of decision making; (2) external reporting to outside information about each participant to all of the parties such as to stockholders, creditors, and other participants in the educational process (Dil- government agencies.” (Answers.com, accessed lenbourg, 2008). on 08-18-2010) Information was generally input using batch files and outputs were done in much the same way. As technology improved, it was pos- INFORMATION COMMUNICATION sible to do input during real time and to schedule TECHNOLOGY AND LEARNING major outputs during the off hours so as not to MANAGEMENT SYSTEMS degrade the performance of the system. These systems generally are comprised of three For this discussion, we will focus on academic major components; the database and its front end, institutions and the systems that are involved the control system, and the reporting system. The there. While they are much the same as those of database stores all of the essential information and

7 General Perspective in Learning Management Systems

Figure 2. Information Communication Technology (ICT)

provides the input screens (front end) that allow Faculty Information System (FIS) the users to input the information. That information may be added to the database in real time or stored Faculty information systems are basically a human in a temporary ledger to be added at a more effi- resource information system (HRIS) that allows cient time. Generally, if it is stored in a temporary an institution to maintain information on nearly ledger, it is added to the database at the close of everything that needs to be tracked or analyzed business and the information will be available for about faculty (current and former) and those who reports after that time. The databases also allow are applying for positions. It is able to be main- for locking of the records so that they cannot be tained by professionals in human resources and changed in any fashion without creating a record by the faculty themselves in order to maintain the of the change (a part of the control system). The most accurate and up to date information. Not control system allows for the ability to develop only does it maintain information on the classes appropriate controls on the system and users to taught by each faculty member, but their academic ensure the integrity of the information contained credentials and relevant experience as well. It in the system. The reporting system allows for provides a place to maintain a record of their the processed information to be displayed in an scholarly activity and their service involvement. appropriate manner for the decision makers and These systems also allow for the publication of other interested parties. documents necessary for the faculty to perform their duties in an efficient and effective manner.

8 General Perspective in Learning Management Systems

Initially these systems did not communicate tion to the decision makers on par with enterprise with any other systems, but more and more these resource planning systems or the HRIS systems. systems are integrated with all of the information It is rare for an institution of any size to not have systems of an institution so that information only some form of an SIS system. While many of these needs to be input one time. were developed in-house initially, most are commercially available today. In the K-12 market, Library Information System (LIS) these systems are being outsourced to a hosted environment allowing for many other enhance- The initial library information system focused ments to be included in the SIS system. Most of on the card catalogue and the record of where an the commercially available systems today provide individual item resided. It was basically a large the ability to integrate with all of the other ICT inventory tracking database for library hold- systems functioning in an academic institution. ings. It served to automate what a librarian was responsible to do in order to maintain control of Learning Management System (LMS) the assets of the library. As more and more assets became electronic in nature, it also began to be In a simple way learning management systems a repository of those items and became then a involve the application of ICT to the process of digital library system. This system now makes it education, but it is a bit more complicated than possible for patrons to access the resources of the that (Dillenbourg, 2008). An LMS provides the library without having to be physically present. It structure for the delivery and management of also enables libraries to share their resources with instructional content, while providing assessment one another in a seamless fashion; enabling their of individual and organizational learning goals, patrons to have access to a much broader set of tracking of progress toward those goals, and pro- materials than would be possible otherwise. While viding the data necessary for the management of LISs served to just automate library processes in the learning process of the institution as a whole the beginning, they have now entered into the (Watson, W. & Watson, S., 2007) (Kim, S. & Leet, process of transforming the way that the library M., 2008) (Rogers, 2001) (Perry, 2009). In today’s delivers information and the way that patrons ac- educational environment ICT is often involved cess information. With LISs, libraries have moved in all types of learning systems (see Figure 3). from being islands of information to becoming The term was first used to refer to the manage- portals to the information stored in the network ment system of the PLATO K-12 learning system of libraries worldwide. (Watson, W. & Watson, S., 2007). Today, there are basically four broad system designs used, as Student Information System (SIS) seen in the accompanying diagram. These designs are determined by the pedagogical focus of the Student information systems provide the ability to education enterprise. enter student information into a database that will provide an electronic grade book, student course Traditional Learning schedules, and other student-related data needs for a school, college, or university (Paulsen, 2002). Historically, this form of learning has been the These systems grew out of the need to automate staple for hundreds of years, from the very found- the various record keeping responsibilities of an ing of higher education institutions. Students meet academic institution. They have grown to the together in real time and in a specified location point of being able to provide advanced informa- with the instructor present. This instructor cen-

9 General Perspective in Learning Management Systems

Figure 3. Learning system models

tered approach is the predominant modality of Distance Learning instruction in higher education. ICT can be used to enhance the educational experience with the Distance education dates from the mid nineteenth use of technology delivering the instruction and century. Students and instructors are separated by maintaining student records. The instructor is the time, location, or both in this model of instruction primary determinant of what use will be made (Gaspay, A., Dardan, S., & Legorreta, L., 2008). of ICT in the delivery of instruction and to some The materials may be delivered to remote locations extent to its use in the completion of assignments via print or ICT, but this form of instruction does required for the instruction. Most information not preclude the use of remote classrooms. This will be supplied by an instructor directly or in form of instruction may be done synchronously preprinted materials, but may be supplemented or asynchronously. The emphasis of this mode of with various forms of ICT in the classroom. The instruction is the separation of the student from students may utilize ICT outside of the classroom the instructor and not the delivery mechanism to produce class assignments, submit class as- (Gaspay, A., Dardan, S., & Legorreta, L., 2008). signments, or locate information for class, but It does not matter that the separation is physical the primary delivery of instruction is done in a or temporal, just that there is some separation face to face mode. This instruction is delivered between the two participant groups. This mode synchronously. of instruction does not focus on the delivery mechanism or the technological involvement. The

10 General Perspective in Learning Management Systems

focus may be the individual or a group (Keegan, tance; they are most generally used as an adjunct 2002). Early forms of this modality focused on to traditional learning. The primary function of individual instruction (correspondence courses, these systems is to remediate performance deficits etc.), with technology enabling group instruction in basic skills. This form of blended learning is beginning in the 1980s. This form of instruction much more like the programmed learning roots was not highly valued until the advent of open from which all of this grew. These systems contain universities in the 1970s. a management system that controls the flow of data Most distance education in the United States between the other components, curriculum content is group focused, while most distance education that provides the tutorial, practice and assess- in Europe is individual focused (Keegan, 2002). ment modules, and the student record system that Regardless of the focus, most materials are of maintain registration and performance information the pre-prepared variety. Instructors may utilize on every student enrolled in the system (Rogers, materials that were prepared by another or even 2001). These are not usually integrated with the by a group of others. The teaching materials may ICT systems of the institution. Their structure mir- be produced up to ten years before the student rors the structure of learning management systems interacts with them in a learning environment. Fur- without the broad connectivity found in those thermore, the institution that created the materials systems. Some examples of ILS are CentraOne, may not be the institution that is awarding credit IntraLearn, Lyceum, and Silicon Chalk, Odyssey, for the course completion. There exist clearing and Plato (the same company that originated the houses for courses that can be combined to create name learning management system). distance learning programs of study. E-Learning Blended Learning E-Learning had its beginnings in late 1994 or In the blended learning mode, parts of the instruc- early 1995 and really soared after 1996 when the tion are delivered in a traditional format while first modern content management system was other parts of it are delivered using ICT (Gaeta, developed. In E-Learning all of the instruction M., Orciuoli, F., & Titrovato, P., 2009). It is this is online using ICT to deliver the content. By blending of the delivery modality that attempts to 1998, it was viewed a mature field of distance use the strengths of both formats to enhance the education. Michigan Department of Education educational experience (Dillenbourg, 2008). ICT defines this as “A combination of structured, is more than just an adjunct to the process, but sustained, integrated, online experiences accessed the students and instructor are required to utilize via a telecommunications network.” This may ICT in order for learning to occur properly. This involve synchronous and asynchronous delivery form requires that parts of it are synchronous and of instruction. It blends aspects from all of the parts are asynchronous. other models into a model that is not just ICT One form of blended learning is called an inte- enhanced, but one in which it would not occur grated learning system (ILS) in which networked without the use of ICT. In today’s implementation computers or terminals with a management system of E-Learning, ICT is transformative and not just monitors and records student performance results supportive of the process of education (Gaspay, and distributes learning modules based upon those A., Dardan, S., & Legorreta, L., 2008). In some results (Dillenbourg, 2008) (Brush, T., Armstrong, circles, E-Learning is used to refer to all learning J., Barbrow, D., & Ulintz, L., 1999). These are that occurs through the mediation of ICT, but I truly digital systems and can be used over a dis- use the term more formally to refer to structured/

11 General Perspective in Learning Management Systems

purposeful learning that utilized ICT for its de- system (Jones, 2004). However, one must con- velopment, delivery, and management in such sider theoretical orientation when considering a way that the instruction is transformed by the the appropriateness of any learning management medium in which it occurs. It is much similar to the system (Wang, Y. & Chen, N., 2009). distinction between learning that occurs through While this seems pretty straight forward, independent effort of the student and that which there is much confusion as to what is meant by is accomplished through the collaborative effort the various systems and components of systems. of instructor and student. Learning happens in Some authors state that CMSs are LMS that are both situations, but in the latter, it is substantially used by academic institutions while LMS are different than it would be otherwise because of used in industry (Hamuy, E. & Galaz, M., 2010) the nature of the delivery medium. (Daniels, 2009). Course management systems are While a learning management system could be also confused with content management systems, utilized in some fashion in each of these models, which both use the same acronym, CMS (Daniels, it is in E-Learning that it is an absolute necessity. 2009). Course management systems and virtual ICT in all of these models could be referred to learning environments (VLE) are basically the as learning support systems. The problem is that same thing with terms differing by the region there are no universally accepted definitions for in which they occur (Daniels, 2009). Course these learning systems that will serve to distin- management system is the term used in North guish them from other ICT systems used in the America and VLE is the term used in Europe educational process. For this reason, the terms (Daniels, 2009). Content management systems are used haphazardly to refer to various systems are generally systems used in industry to publish or parts of the system as if they were the same. information to various audiences to support their work processes. A related term is learning content management system. It refers to the system that COMPONENTS OF A LEARNING delivers the specific portions of learning content MANAGEMENT SYSTEM to the learner during the use of the LMS. We will not be referring to content management systems In a nutshell, LMS is the software that auto- apart from learning and so CMS will always refer mates the administration and delivery of learn- to course management systems in this chapter. ing (Watson, W. & Watson, S., 2007) (Baturay, 2008) (Rogers, L. & Newton, L., 2001). It is the Course Management System (CMS) overarching ICT that provides all of the functions necessary to provide learning in a digital format The CMS provides the linkage with all of the other at a distance (Wang, Y. & Chen, N., 2009). This systems involved in the learning endeavor. It is does not preclude the ability for it to also serve truly the brains of the whole system. This system in a blended learning environment, but it must be allows the appropriate individuals to add or remove able to provide that learning where there is spatial, courses, to sequence courses within a curriculum, temporal separation or both. The system may have add students to the course, assign instructors to any number of components, but we will look at individual courses or sections of courses, to moni- the major components that one should expect to tor other processes of the system, etc. (Watson, find in a modern learning management system. W. & Watson, S., 2007) (Daniels, 2009). This It is important to remember that the technology is probably the most frequent system confused is generally learning theory independent and can with LMS. Many individuals believe that is all an be utilized to function in most any theoretical LMS does and so what is the difference. To do its

12 General Perspective in Learning Management Systems

job, this system must interface with the FIS and system are accomplished by third party apps that SIS systems. It is also often linked with the LIS are added to the system piece meal. This process system. These linkages are often rudimentary and can create some very unusual difficulties if the not well established. This lack of strong linkage system is not managed effectively. This is also can cause considerable problems to the end users the area where mobile connections are usually and the institution alike. When it is functioning made and managed. well, it is like a good traffic officer seeing that It is within the CLS that many instructors will everyone and everything gets to where it needs develop the social learning that should be part to be in the most efficient manner. of any learning program that seeks to meet the needs of a diverse student population. However, Learning Content Management just using collaborative tools without the requisite System (LCMS) understanding and application of social learning theory will not produce the desired results. The The primary purpose of the LCMS is to develop, importance of this area is supported by belief by store, organize, and distribute multimedia content some educational practitioners that only 20% of to support the delivery of E-Learning (Watson, W. learning occurs based upon formal instruction & Watson, S., 2007). This over arching system and that the other 80% is produced by informal has many subsystems that allow it to perform its instruction that occurs when students interact in duties. Most of these occur in the background and social environments involving each other and the users do not see them or even need to. Course the instructor. content authoring is one of these background activities. While the instructor uses this system Assessment Management System to design the course and upload the appropriate content, they really don’t need to understand what All forms of assessment are managed from within is being done to make it happen. Once the material assessment management system. This is also is stored, usually in the form of learning objects where the grade book is located. Assessments (Watson, W. & Watson, S., 2007), it needs to be could be exams, homework, projects, etc. They accessed at the appropriate time to support student could involve student submissions, as well as learning (Watson, W. & Watson, S., 2007). These student participatory activities. The purpose of processes need to function effectively as purposed this subsystem is to ensure that proper delivery or they will negatively impact the quality of the and recording of results for all assessment items learning delivered. is maintained. The system should provide the instructor with the tools necessary to assess stu- Collaborative Learning System (CLS) dent performance on the appropriate measures. It should allow the instructor the ability to design The CLS is the system that allows the use of the these measures with some degree of flexibility. newer aspect of Web delivered content (Web 2.0). The assessment management system should enable This system provides the tools for email commu- the instructor to provide the student with adequate nication, discussion groups, newsgroups, instant and timely feedback on their performance so that messaging, blogs, bookmarking, notice board, the student will benefit from the learning and be search tools, etc. Not all LMS have these capabili- able to improve the desired outcomes of the learn- ties, but these and many more are managed by the ing experience (Watson, W. & Watson, S., 2007). CLS. This is the part of the LMS that changes the most rapidly. Many times the capabilities in this

13 General Perspective in Learning Management Systems

LEARNING MANAGEMENT are able to utilize the system to improve their SYSTEM STAKEHOLDERS educational outcomes and to meet the goals that they set for themselves in the process, as well There are basically four stakeholder groups that as the goals that may be set by the faculty and are directly involved in LMS (see Figure 4), administrators. but there is a fifth that is peripherally/externally involved. The four primary/internal stakeholders Faculty are students, faculty, administrators, and IT staff (Wagner, N., Hassanein, K., Head, M., 2008). Instructors, more than anyone else, determine the The peripherally involved stakeholder group is nature of the E-Learning experience (Wang, W. & that group who will hire the students once they Wang, C., 2009). Depending on their theoretical complete the education using the system or are foundation, they play a larger or smaller role deter- involved in accrediting the education institution mining the educational experience of the students. to be able to offer a degree or credential. Each of Generally, they are the arbitrators of whether or these groups holds slightly differing sets of wants not the students will participate in group activi- and needs, but they also share some commonalities. ties, independent study, or synchronous events, to just name a few ways in which they control the Students E-Learning experience. If their theoretical persuasion is behaviorist, they may design instruction in Students are the ultimate consumers of the LMS such a way as to move the student in a structured output. Generally speaking they are either under- manner to the desired outcome, but if they are graduate or graduate students of the educational constructivists, they will allow the students to institution. They may be enrolled in one or more determine how they will achieve the end result. courses delivered by the LMS. While there are a These are two of the many facets of their role in number of motivations for their using an LMS, the instructional process. some of the primary ones are access to an educa- Ultimately, E-Learning creates many changes tion, convenience, and, for a few, because they for the instructor (Wagner, N., Hassanein, K., prefer this mode of learning (Wagner, N., Has- Head, M., 2008). Their role is transformed re- sanein, K., Head, M., 2008). Some will be using gardless of their theoretical persuasion, because an LMS because there is no other way to complete the technology and distance of the students will the program of study that they desire. In a sense, force that change somewhat. Delivery of infor- then, this relates to access. Modern students have mation, motivation of students, interaction with grown up with an increasing involvement with students, assessment, etc., will all differ because technology in their day to day lives. Few, if any, of the medium of delivery. Instructors will require know of a time when they did not have access to a differing level of technological sophistication the internet to further their contact with one another depending upon the level of technical support and with the information that is available from available, but all will need some level of techno- a broader perspective, especially in the western logical knowledge to be able to function in this world. Most students today have an extensive environment (Wang, W. & Wang, C., 2009). background with ICT and use it on a daily basis. Instructors in the past spent a great amount This technological background of these students of their time in knowledge creation, either in the will impact what they expect from any delivery form of original research or by consuming that system (Wagner, N., Hassanein, K., Head, M., research (Jones, 2004). With E-Learning, they 2008). It will also play a major role in how they will be spending more time in the development

14 General Perspective in Learning Management Systems

Figure 4. LMS stakeholders

and delivery of learning materials than they have operational 24/7 (Jones, 2004). These systems historically done (Wang, W. & Wang, C., 2009). are mission critical, but are used by personnel Some studies have indicated that they may spend who often times do not understand the intricacies twice as much time in this process, even with the of their function. While the end users will have aid of support staff. This shift in time commitment some technical expertise, they will not generally will result in a shift in the role and expectation of be technologists, but will have some strong expec- instructors in the future. tations from the systems that they use. This will increase the pressure on the IT staff to provide IT Staff the services desired. IT staff will have to better understand the IT staff are the individuals who have traditionally educational theories within which the system only been involved in supporting the business will function in order for the system to perform side of the institution. If they were involved in according to the needs and expectations of the educational delivery, it was to support systems users (students and faculty). It will no longer be that were used in traditional classrooms. They sufficient for IT staff to just understand technol- now are involved in systems that must be kept ogy, but they will have to also be cognizant of

15 General Perspective in Learning Management Systems

the environment in which the technology will be learning/E-Learning, all accrediting bodies must implemented (Jones, 2004). It will also force the certify some E-Learning offerings through insti- staff to include the end user in the design process tutional members. from the very beginning and throughout the life- Another group that plays a major role in this cycle of the product. These will be different roles process is the potential employer of the students for the typical IT staff. in E-Learning programs. They want to ensure that students have received an adequate education Administrators irrespective of manner in which it was delivered. They want the education to provide content that Administrators in traditional institutions are is relevant to the work environment and that primarily seeking to create easier access to their meets minimum quality standards. They also institution and remove geographic barriers to may become involved by supporting their current student participation (Wang, W. & Wang, C., employees as they pursue further education to the 2009). In this manner, they are increasing their benefit of the employer. In this manner they are student population and expanding their academic also concerned with student access. offerings to a wider market. They are also trying The funders of education are an often over- to remain responsive to the market trends and looked stakeholder group. These are the entities the desire of their targeted market (Jones, 2004). that provide various aspects of the funding for They are attempting to expand the institutional educational institutions. They may be public system while maintain fiscal responsibility. In funders in the form of government agencies or the process of expanding their delivery modali- they could be private funders who provide grant ties, they want to maintain the quality of their funding or give to foundations that support the course offerings so as to not destroy their brand educational programs of an institution. They image. This often requires that they need to pay may be the parents who are funding their child’s attention to the broader issues of the Environment education. Members of this group are primarily (the outer ring of our diagram). While the other concerned that the funds will be utilized to provide stakeholders focus on a few of the other stakehold- a quality education to the students and ensure a ers, this group must pay attention to and balance respected outcome from their investment. the needs and concerns of all of the stakeholders. All of these various stakeholders interact with They also must focus on the future needs of the one another seeking to have each other’s needs institution by establishing the strategic plans for faced and resolved in the E-Learning effort. None technological innovation. of them stand isolated from the others and individually do not have a preeminent role to play in Environment the eLearning offering. If one is ignored or their needs and desires are not met, it creates the po- The environment is the hodge-podge of others tential for the eLearning effort to fail. that have an interest in the outcomes of the eL- earning endeavor (Jones, 2004). They encompass the various accreditation bodies that must certify MODELS OF LEARNING the quality of the instruction offered. They are MANAGEMENT SYSTEMS primarily concerned that minimum standards are met that all courses are comparable regardless of The various LMS models differ according to the the modality of offering. While there are a num- selection of LMS elements (see Figure 5) and ber of accrediting bodies that focus on distance how they are applied to the system and therefore

16 General Perspective in Learning Management Systems

to the users of that system. While there are many being absorbed by one of the competitors. Some elements that could be utilized to discriminate examples of these would be: Blackboard (which one LMS from another, we will focus on three. recently acquired WebCT Manager and Angel), These are the primary elements that influence Desire2Learn, ECollege, & .LRN. There are sites most decisions concerning the system of choice. where one can compare these systems in order to The accompanying diagram identifies the three decide which one is right for you. broad elements, but one must bear in mind that With these systems, you can either have a most systems are not just one or the other, but are hosted environment, where some provider supplies in fact somewhere on a continuum between the the hardware and software to make the system extremes within each element. work or you, or you can host it yourself on your own servers. There are many pros and cons to Source In-House each approach and should be considered carefully.

Initially all LMS were developed in-house and Free/Open Source were dependent upon the support of the institutional programmers. Most of these systems pos- These systems are supported by a network of pro- sessed limited functionality and they were difficult grammers around the globe who provide updates to modify or improve. A few of those systems, like to the systems on an almost constant basis (Pan, the PLATO LMS, were later transferred to busi- G. & Bonk, C., 2007). Many of these systems are nesses that have either used them in proprietary free to own, but are also often available through a products or marketed them as proprietary systems. supplier who will provide for the installation and Most in-house developed systems, therefore, are support of the system. They can also be available no longer in existence, although there are some as a hosted option. They provide a great deal of who are advocating a return to the in-house de- flexibility to the decision making process. They veloped systems as a way to provide quality at a have traditionally been a niche market, but with more affordable cost. Moodle gaining in popularity, they are gaining a larger market share. The basic premise of these Proprietary systems is what is commonly known as Linus’s Law: Given enough eyeballs, all bugs are shallow Proprietary systems comprise the largest block of (Pan, G. & Bonk, C., 2007). They assume a self- systems in operation today. While they all claim to correcting mechanism in the peer review process provide the functionality needed by the institution, that open-software engenders. This further pushes they are by no means identical. They differ on some the costs of these products lower and encourages very important factors and it would behoove the users to expand their functionality. The develop- using institution to choose carefully. While they do ment of these products promotes common stan- differ on the basis of functionality, most provide dards that establish those that are most beneficial a basic set of functions along the lines discussed to the market. This process also encourages the in course management systems. They may or development of expertise in the broader market may-not interface with academic administrative arena by encouraging many individuals to be systems or an online library system, for instant. involved in the development of the product. They may include some Web 2.0 functionality, but it may be limited in utility. There is constant change within the industry, with companies coming and going by either leaving the industry or

17 General Perspective in Learning Management Systems

Figure 5. Elements of a learning management system

Time easily justified. However, there are systems that focus on a synchronous delivery mechanism, such Time refers to whether the student and instructor as LaunchForce and LeanLine. These systems put are involved concurrently or separated by time, the instructor and students online at the same time. in other words, synchronous or asynchronous These systems may involve, audio only, audio (Wagner, N., Hassanein, K., Head, M., 2008). and video, whiteboard, program share, and other Most LMS are designed to be operated in an such technologies. In this mode, there is much asynchronous mode. In this mode, an instructor more the virtual classroom experience with the prepares the learning content and interacts with ability to engender the collaborative involvement the students mostly on a one to one basis. The in- of all of the students and instructor in the learning teraction does not occur in real (concurrent) time, exercises. Collaboration is possible in both syn- but with time lapses. This creates the possibility chronous and asynchronous modes, but is more of instructor-student disconnect as one has to wait natural in the synchronous environment. for a period of time for the other to respond to the Some systems allow for both modes of delivery inputs. As the system supports a more dispersed to be blended together. Such systems as ECol- student population, possibly spanning the globe, lege, Embanet, Jones e-education, and LUVIT this becomes more of a necessity or least more eLearning are thought of as total solutions. They

18 General Perspective in Learning Management Systems

allow delivery in the most effective time frame the foundation out of which the others flow (Dab- for the individual course. Many of these systems bagh, 2005). From the constructs will flow the allow for the content that is delivered synchro- pedagogical strategies that determine the teaching nously to be recorded and then delivered in an methods that are used in the program. From the asynchronous fashion. There are tools that often strategies one will determine the pedagogical tools can be utilized with other asynchronous systems that will be need in instructional delivery. It is in to allow for the delivery of portions of the class this manner that pedagogy will determine the in a synchronous mode. nature of the LMS used in a specific setting.

Pedagogy PEDAGOGICAL CONSTRUCTS Pedagogy is concerned with how a course will be delivered in instructional terms. It will affect These are the basic formulation of the teaching/ the type of interaction that may be involved in learning process (Dabbagh, 2005). They are the the learning process, such as, student to content, models or the components of the models that will student to instructor, and student to student guide the development and delivery of instruction (Hamuy, E. & Galaz, M., 2010) (Wang, Y. & Chen, (Sabry, K. & Barker, J., 2009). They arise out of N., 2009). In the student to content, the student one’s understanding of cognition and knowledge. interacts with instructional information, either They provide the mechanism for putting theory provided through the course system or accessed by into practice (Dabbagh, 2005). Some major ex- other means, textbook, online, etc. In the student amples include: behaviorist, cognitivist, and to instructor mode, the student interacts with the constructivist. While these are not the only models content expert or experts to retrieve the needed used, most others grew out of them. learning content. In the student to student mode, the students will interact with one another to retrieve Behaviorist information or perform learning activities in a collaborative manner. Pedagogy will also determine Behaviorist pedagogical systems will provide a the nature of the course structure(see Figure 6); will structured/programmed approach to the teaching/ the course allow the student to move around and learning process. This pedagogical system sees choose the order in which material is accessed and the mind as a ‘black box’ that is not knowable work is completed or must they move through in a and so is not a concern. No attention is paid to predetermined orderly sequence (Dabbagh, 2005). the internal processes of learning, in fact, many Will the student be required to achieve mastery of would say internal processes don’t exist, but this the material to some predetermined level or will pedagogical system focuses on the measurable the student construct their own learning with the outcomes of learning. These outcomes therefore instructor serving as a mentor/guide? Pedagogy equal learning and rewards are the motivators that plays a fundamental role in the determination of support the production of desirable outcomes. the nature of the learning experience, whether the Learning is linear and structured. Theorists in learning constructor recognizes the role or not. this paradigm are Watson, Thorndike, Skinner, Pedagogy involves three broad aspects in its and Pavlov. structure (Dabbagh, 2005). Each of these aspects plays some role in the impact of pedagogy on the learning process. Pedagogical constructs provides

19 General Perspective in Learning Management Systems

Figure 6. Pedagogical structure

Cognitivist Constructivist

The cognitivists focus on how the mind processes Constructivists built off the cognitivistic view that and uses information to produce learning. They are the mind is more than a ‘black box’ responding interested in the mental structures and processes to stimuli. It instead focused on the processes that are necessary to explain human behavior involved in learning, seeing those processes (Dabbagh, 2005). They pay greater attention on as internal and, therefore, not visible directly the learner’s thoughts, beliefs, attitudes, and values (Homberg, 2005). Constructivists see learning as in explaining the learning outcomes (Ardichvili, an active process that works within a context to A. & Yoon, S., 2009). They are more focused on produce (construct) knowledge (Dabbagh, 2005) the learners’ differences and seek to accommo- rather than just acquiring it. Learners must be ac- date them by varying the instruction material and tively involved in both acquiring and processing processes. To them learning is an individualized information subjectively (Baggaley, 2008). As process. Group activities are relatively unimport- learners actively process the information, they ant and rarely addressed. They seek to provide will gain a deeper understanding of the content organizational structure that allows the learner the (Ardichvili, A. & Yoon, S., 2009). The learner ability to move through the material in a highly needs to be encouraged and enabled to search individualized fashion that ties the new learning for the knowledge or solve problems on their to existing informational structures. Theorists own rather than provided the content or solutions in this paradigm are Gagne, Briggs, and Bruner. (Ardichvili, A. & Yoon, S., 2009). All knowledge

20 General Perspective in Learning Management Systems

is seen as social in origin, developed while en- outcomes. Practitioners may or may not pay any gaged in activities (projects), mediated by tools attention to individual differences or care much (Lytras, M. & Pouloudi, A., 2006). Instructors in about the intellectual processes that are involved this tradition will provide the learners with real- in developing the learning outcomes. world simulations, collaborative experiences with others, or by providing them with the knowledge Constructivist Strategy and ability to access knowledge at the time that it is needed (Ardichvili, A. & Yoon, S., 2009). The constructivist framework/strategy focuses on Theorists in this paradigm are Dewey, Piaget, individualized learning. These practitioners see the Vygotsky, and Bruner. process of learning as very interactive with the These models/constructs are almost never used student interacting with the teacher and with the solely. They each will often find a place in some learning materials. The student plays the major part of the E-Learning program. Most practitio- role in the learning outcome. Students determine ners will have a preferred approach to the design what is and is not important to learn. The teacher and delivery of instruction, but will generally use moves from the ‘sage on the stage to the guide some of the less preferred methods within their on the side’ in this process. Learning activities program to meet the needs of the widest number and materials must be somewhat fluid in this of students or to overcome areas of difficulty in process, because it is not possible to know what the instructional program. the student will need before the student needs it. The primary educational methods/tools used will be case studies, self-instructional materials, and PEDAGOGICAL STRATEGIES questions and answers. The pace of learning will be dictated by the student and their desires. The application of pedagogical constructs to the teaching process leads to three broad pedagogical Social Constructivist Strategy strategic systems (Dabbagh, 2005). Each system seems to lean toward a specific set of educational The social constructivist framework/strategy fo- methodologies/tools. There is an overarching cuses on collaborative learning (Huang, S. & Yang, perspective that guides the choices that are made C., 2009). Learning is seen as a social endeavor by these educators in the development of the and requires the collaboration between and among learning program. This produces the framework all participants. The teacher will very often be just for E-Learning (see Figure 7). one of the learners in this process. The group will determine the nature of the learning. Learning Behavioral/Cognitive Strategy activities will vary with the class involved; the students will bring many of the materials from their The Behavioral/Cognitive framework/strategy own search for answers. The primary educational promotes facilitated learning. The teacher pro- methods/tools used will be case studies, problem vides the guidance and direction that is needed based learning, discussion forums, teamwork, and for the student to learn. The primary educational seminar sessions. The emphasis will be upon the methods/tools will be lectures, presentation, group participation in the learning process with the textbooks, and teacher directed discussions. The group playing a major role in the decisions about teacher is the source of information that leads to what will transpire in the process. This is a very knowledge acquisition in this process. The pro- interactive strategy requiring consistent feedback cess is very linear and focuses on measureable from the teacher and fellow students. Learning is

21 General Perspective in Learning Management Systems

Figure 7. Pedagogical frameworks

seen as a process rather than an end. Control rests system to allow the students to ask questions of in both the teacher and students within a context the teacher and allow the teacher to respond to of continuous communication. those questions. You would also benefit from having a document delivery mechanism and a means for the students to submit their work and PEDAGOGICAL TOOLS take exams. Most systems available today provide these features and more. Your pedagogical strategy will determine the fea- If your pedagogical strategy is in a construc- tures, pedagogical tools, that your LMS will need tivist framework, you will need the entire afore- to provide (Dabbagh, 2005) (Sabry, K. & Barker, mentioned, but also some means for the student J., 2009). If your strategy is a behavioral/cognitive to navigate through the material individually and framework, you can probably get by with a simple to set personal learning objectives. They will course management system that uses prepackaged also need to be able to acquire their own learn- course material to deliver instruction in a linear ing materials to provide for their individualized fashion with teacher graded outcomes measures. learning plan. There will need to be flexibility in You will need a simple email communication the assessment of learning outcomes to provide

22 General Perspective in Learning Management Systems

for the individualized nature of those outcomes. upon LMS and their use within the educational You will want to provide for simulations, blogs, environment. The nature of that impact is influ- message boards, etc. to allow the students to be enced by pedagogy. Pedagogy will drive what able to construct their own learning activities and the LMS will have and how it will be used in the provide mechanisms for easy feedback from the delivery of E-Learning. In most cases, teachers teacher/coach. will not stick with one pedagogical framework, If your pedagogical strategy is a social con- but will use aspects of all of them at some point structivist framework, you will need everything in the delivery of learning. Students will benefit already mentioned, but also include mechanisms from this eclectic use of pedagogy. We have only for multi-way communication so that all partici- scratched the surface of what is involved in the pants can communicate with each other in a free function of an LMS based on pedagogical decision and open manner. To be most effective, you will making. The better the teacher understands the need to provide for both synchronous and asyn- pedagogy underlying the decisions being made, chronous communication modalities. Students will the more effective the LMS will be in meeting need to work individually and in groups; as much those needs. Many LMS implementations fail as possible, the groups should be self selected. because there was not a clear understanding of the pedagogical needs of the teachers using the system (Dillenbourg, 2008). FUTURE RESEACH DIRCTIONS This chapter has only lightly touched on the topics presented. It is not intended to be an Too much of research focuses on finding some- exhaustive treatment of any of the subjects. It is thing new or defending a current stance on a sub- hoped the readers will continue to pursue those ject. We need to have researchers who are willing topics of interest and to be aware of those of lesser to integrate the findings of others into some sort of interest. In this manner, we should all be able to a coherent system. This should involve the gather- understand the changes that have occurred and ing of information from related disciplines so that be prepared for those that will come. we can improve learning management systems by using the knowledge from all appropriate sources. If we become too enmeshed in our limited view of REFERENCES the world, either by just looking at what is being produced in distance learning publications or by Ardichvili, A., & Yoon, S. (2009). Designing looking at only what is being done in our limited integrative knowledge management systems: geographical sphere, we will not be able to meet Theoretical considerations and practical applica- the transformations that will occur in the future. tions. Advances in Developing Human Resources, We also need to focus on those aspects that will 11(3), 307–319. doi:10.1177/1523422309337593 improve the quality of what we do. Baggaley, J. (2008). Where did didtance education go wrong? Distance Education, 29(1), 39–51. doi:10.1080/01587910802004837 CONCLUSION Baturay, M. (2008). Analysis of a learning man- Learning management systems, while a modern agement system model. E-Journal of New World phenomenon, have very deep roots in a broad Sciences Academy, 3(3), 464–477. array of disciplines. Each of these disciplines is still functioning today and produces impacts

23 General Perspective in Learning Management Systems

Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. Gilman, D., Emhuff, J., Bender, P., Gower, A., & (1999). Design and delivery of integrated learning Miller, K. (1991). A comprehensive study of the systems: Their impact on student achievement effects of an integrated learning system. Indiana and attitudes. Journal of Educational Computing State University. Research, 21(4), 475–486. Guri-Rosenblit. (2005). ‘Distance education’ Cradler, J. (2008). Research on e-learning. Learn- and ‘e-learning’: Not the same thing. Higher ing and Leading with Technology, 3(5), 54–57. Education, 49, 467–493. doi:10.1007/s10734- 004-0040-0 Dabbagh, N. (2005). Pedagogical models for e-learning: A theory-based design framework. Hamuy, E., & Galaz, M. (2010). Information International Journal of Technology in Teaching versus communication in course management and Learning, 1(1), 25–44. system participation. Computers & Education, 54, 169–177. doi:10.1016/j.compedu.2009.08.001 Daniels, P. (2009). Course management systems and implications for practice. International Holmberg, B. (1995). Theory and practice of Journal of Emerging Technologies & Society, distance education, 2nd edition. London, UK, and 7(2), 97–108. New York, NY: Routledge. Dillenbourg, P. (2008). Integrating technologies Holmberg, B., & Ortner, G. E. (1991). Research into educational ecosystems. Distance Education, into distance education. Frankfurt am Main, 29(2), 127–140. doi:10.1080/01587910802154939 Germany: Peter Lang. Eteokleous-Grigoriou, N. (2009). Instilling a new Homberg, B. (2005). The Evolution, Principles learning, work and communication culture through and Practices of Distance Education (Vol. 11). systemically integrated technology in education. Hagen, Germany: FernUniversität, ZIFF. Systems Research and Behavioral Science, 26, Huang, S., & Yang, C. (2009). Designing a se- 707–716. doi:10.1002/sres.983 mantic bliki system to support different types Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). of knowledge and adaptive learning. Comput- Advanced ontology management sytem for per- ers & Education, 53, 701–712. doi:10.1016/j. sonalised e-learning. Knowledge-Based Systems, compedu.2009.04.011 22, 292–301. doi:10.1016/j.knosys.2009.01.006 Jones, D. (2004). The conceptualisation of e- Garrison, R. (2000). Theoretical challenges for learning: Lessons an implications. Studies in distance education in the 21st century: A shift Learning, Evaluation, Innovation and Develop- from structural to transactional issues. Interna- ment, 1(1), 47–55. tional Review of Research in Open and Distance Keegan, D. (2002, November). The future of learn- Learning, 1(1), 1–17. ing: From eLearning to mLearning. ZIFF Papiere Gaspay, A., Dardan, S., & Legorreta, L. (2008). 119. Hagen, Germany: FernUniversitat, Ziff. Distance learning through the lens of learning Kim, S., & Leet, M. (2008). Validation of an models: New outlets for innovation. Review of evaluation model for learning management sys- Business Research, 8(4), 53–62. tems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x

24 General Perspective in Learning Management Systems

Lytras, M., & Pouloudi, A. (2006). Towards the Sabry, K., & Barker, J. (2009). Dynamic Interac- development of a novel taxonomy of knowledge tive Learning Systems. Innovations in Education management systems from a learning perspective: and Teaching International, 46(2), 185–197. An integrated approach to learning and knowledge doi:10.1080/14703290902843836 infrastructures. Journal of Knowledge Manage- Taylor, J. (2001). 5th generation distance education. ment, 10 6), 64-80. 20th ICDE World Conference on Open Learning Miller, T., & Hutchens, S. (2009). 21st century and Distance Education (pp. 1-11). Dusseldorf, teaching technology: Best practicess and effec- Germany. tiveness in teaching psychology. International Underwood, J. (1997). Integrated learning sys- Journal of Instructional Media, 36 3), 255-262. tems: Where does the management take place? Ozkan, S., Koseler, R., & Baykal, N. (2009). Education and Information Technologies, 2, Evaluating learning management systems: Adop- 275–286. doi:10.1023/A:1018677616969 tion of hexagonal e-learning assessment model Wagner, N., Hassanein, K., & Head, M. (2008). in higher education. Transforming Government: Who is responsible for e-learning success in higher People, Process and Policy, 3(2), 11–130. education? A stakeholder’s analysis. Journal of Pan, G., & Bonk, C. (2007). The emergence of Educational Technology & Society, 11(3), 26–36. open-source software in North America. Interna- Wang, W., & Wang, C. (2009). An empirical study tional Review of Research in Open and Distance of instructor adoption of Web-based learning Learning, 8(3), 1–17. systems. Computers & Education, 53, 761–774. Paulsen, M. (2002). Online education sytems in doi:10.1016/j.compedu.2009.02.021 Scandinavian and Australian universities_A com- Wang, Y., & Chen, N. (2009). Criteria for evalu- parative study. International Review of Research ation synchonous learning management systems: in Open and Distance Learning, 3(2), 1–14. Arguments from the distance language classroom. Perry, B. (2009, June). Customized content at Computer Assisted Language Learning, 22(1), your fingertips. Training & Development, 29–31. 1–18. doi:10.1080/09588220802613773 Rentroia-Bonito, A., Martins, A., Guerreiro, T., Watson, W., & Watson, S. (2007). What are learn- & Jorge, J. (2008). Evaluating learning support ing management systems, what are they not, and systems usability an empiracle approach. Com- what should they become? TechTrends, 51(2), munication & Cognition, 41(1 & 2), 143–158. 28–34. doi:10.1007/s11528-007-0023-y Rogers, L., & Newton, L. (2001). Integrated Learn- ing Systems -- An ‘open’ approach. International Journal of Science Education, 23(4), 405–422. ADDITIONAL READING

Rose, E. (2004). “Is there a class with this content?” Abel, R. humes, L., Mattson, L., McKell, M., WebCT and the limits of individualization. The Riley, K., & Smythe, C. (2007). Achieving Learn- Journal of Educational Thought, 38(1), 43–65. ing Impact 2007. August 2007. Retrieved from Saba, F. (2000). Research in distance education: Http://www.imsglobal.org/learningimpact2007/ A status report. International Review of Research li2007report.cfm. in Open and Distance Learning, 1(1), 1–9. Bates, A. (1995). Technology, Open Learning and Distance Education. London, UK: Routledge.

25 General Perspective in Learning Management Systems

Castells, M. (2000). The Information Age: Hulsmann, T. (2004). Guest editorial – Low cost Economy, Society and Culture: Vol. I. The rise distance education strategies: The use of appropri- of the Network Society. Oxford, UK: Blackwell. ate information and communication technologies. International Review of Research in Open and Cooper, P. (1993). Paradigm shifts in designed Distance Learning, 5(1), 1–14. instruction: From behaviorism to cognitivism to constructivism. Educational Technology, (May): Keegan, D. (Ed.). (1993). Theoretical principlesof 12–19. distance education. London, UK, and New York. NY: Routledge. Cristea, C. (2010). Education and media in the postmodern pedagogy. Petroleum – Gas University Ley, T., Kump, B., & Albert, D. (2010). A meth- of Ploiesti Bulletin. Education Sciences Series, odology for eliciting, modellling, and evaluating 62(1A), 102–106. expert knowledge for an adaptive work-integrated learning system. International Journal of Human- Elias, T. (2010). Universal instructional design Computer Studies, 68, 185–208. doi:10.1016/j. principles for Moodle. International Review of ijhcs.2009.12.001 Research in Open and Distance Learning, 11(2), 110–124. Lockwood, F. (1995). Open and distance learning today. London, UK, and New York. NY: Routledge. Freishtat, R., & Sandlin, J. (2010).. . Educational Studies, 46, 503–523. Mackenzie, O., & Christensen, E. L. (Eds.). (1971). The changing world of correspondence study. Uni- Garrison, D. R. (1990). An analysis and evaluation versity Park, PA: Pennsylvania University Press. of audio teleconferencing to facilitate education at a distance. American Journal of Distance Education, Moore, M. G. (2003). Network systems: The 4(3), 13–24. doi:10.1080/08923649009526713 emerging organizational paradigm. [Editorial]. American Journal of Distance Education, 17(1), Green, N., Edwards, H., Wolodko, B., Stewart, 1–5. doi:10.1207/S15389286AJDE1701_1 C., Brooks, M., & Littledyde, R. (2010). Recon- ceptualising higher education pedagogy in online Moore, M. G., & Clarke, G. C. (Eds.). (1989). learning. Distance Education, 31(3), 257–273. do Readings in principles of distance education (pp. i:10.1080/01587919.2010.513951 29–37). University Park, PA: The American Center for the Study of Distance Education. Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical founda- Moore, M. G., & Kearsley, G. (2005). Distance tions and implications for practice. Management education: A systems view. Belmont, CA: Wad- Information Systems Quarterly, 23(4), 497–530. sworth Publications. doi:10.2307/249487 Ozkan, S., & Koseler, R. (2009). Multi-dimension- Hinostroza, J., & Mellar, H. (2001). Pedagogy al students’ evaluation of e-learning systems in the embedded in educational software design: Report higher education context: An empirical investiga- of a case study. Computers & Education, 37(1), tion. Computers & Education, 53, 1285–1296. 27–40. doi:10.1016/S0360-1315(01)00032-X doi:10.1016/j.compedu.2009.06.011

26 General Perspective in Learning Management Systems

Paulsen, M. (Ed.). (2003). Online education and Cognitivism: A theoretical approach in under- learning management systems: Global e-learning standing the mind using quantitative, positivist and in a Scandinavian perspective. Oslo, Norway: scientific methods that describes mental functions NKI Forlaget. as information processing models. Constructivism: A theory of knowledge Sellar, S. (2009). The responsible uncertainty (epistemology) that argues that humans gener- of pedagogy. Discourse: Studies in the Cul- ate knowledge and meaning from an interaction tural Politics of Education, 30(3), 347–460. between their experiences and their ideas. doi:10.1080/01596300903037077 E-Learning: Learning that is done at a distance Sewart, D., Keegan, D., & Holmberg, B. (Eds.). using information communication technologies (1983). Distance education: International perfor delivery. spectives. London, UK: Croom Helm (Routledge). Information Communication Technology: Consists of all technical means used to handle Skinner, B. F. (1968). The technology of teaching. information and aid communication, including New York, NY: Appleton-Century-Crofts. computer and network hardware as well as nec- Watson, D. M. (2001). Pedagogy before technol- essary software. ogy: Re-thinking the relationship between ICT and Instructional Television Fixed Service: A teaching. Education and Information Technologies, band of twenty (20) microwave channels available 6(4), 251–266. doi:10.1023/A:1012976702296 to be licensed by the U.S. Federal Communica- tions Commission (FCC) to local credit granting Zawacki-Richter, O., Backer, E., & Vogt, S. (2009). educational institutions. Review of distance education research (2000 Pedagogy: Refers to strategies of instruction, to 2008): Analysis of research areas, methods, or a style of instruction. and authorship patterns. International Review Programmed Learning: A learning methodol- of Research in Open and Distance Learning, 10 ogy or technique first proposed by the behavior- 6), 21-50. ist B. F. Skinner in 1958. It has three elements: (1) it delivers information in small bites, (2) it is self-paced by the learner, and (3) it provides KEY TERMS AND DEFINITIONS immediate feedback, both positive and negative, to the learner. Behaviorism: The theory or doctrine that hu- Video Conferencing: A set of interactive tele- man or animal psychology can be accurately stud- communication technologies which allow two or ied only through the examination and analysis of more locations to interact via two-way video and objectively observable and quantifiable behavioral audio transmissions simultaneously. events, in contrast with subjective mental states. Blended Learning: The use of both classroom teaching and on-line learning in education.

27 28

Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Ray M. Kekwaletswe Tshwane University of Technology, South Africa

ABSTRACT The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness.

INTRODUCTION learners share knowledge. The chapter argues that to be able to design LMS that ought to enable This chapter does not directly address a specific social interaction among learners not confined to learning management system (LMS) platform particular places, we must first understand how but addresses forms of communication that can learners interact and the tools they use. In this be encountered as tools of LMS platforms, as chapter, learners use social awareness to determine

DOI: 10.4018/978-1-60960-884-2.ch002

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Knowledge Sharing in a Learning Management System Environment Using Social Awareness

the appropriateness of an LMS tool to engage in with awareness of context and social presence. a knowledge activity, as they traverse the varied Context is understood as the situation in which contexts. Knowledge is not a fixed commodity, a learner or a group of learners find themselves. but a function of our interactions with external Accordingly, context is defined as any information resources including tools, media, and other hu- that can be used to characterize the situation of an mans (Ryder & Wilson, 1997). This suggests that entity. An entity is a person, place, or object that human knowledge transforms as people socially is considered relevant to the interaction between interact with others and the surrounding environ- a user and an application (Dey and Abowd, 1999). ment. Consequently, the chapter is premised on Social presence is re-defined and understood to the notion that knowledge is created and trans- be the mediated presence of another learner who ferred through the dynamic interactions among could provide personalized on-demand knowledge individuals and between individuals and their support for a learning problem as the learner environments (Nonaka, 1994). Thus, knowledge traverses varied learning contexts. sharing is social and sensitive to context. Learners in contact universities come from It is inferred from the notion of knowledge varied social backgrounds, with diverse languages creation, sharing or transfer that knowledge can and cultures. To this point, one of the prevailing be perceived to transform when context and so- educational challenges is that of providing per- cial presence awareness interact. In this regard, sonalized academic support to under-prepared context and presence awareness influences the learners (Jaffer et al, 2006). Awareness of the social interaction and the problems that could be solved environment and social resources is, therefore, and how they are solved. In this chapter, social fundamental to the provision of personalized aca- awareness is synonymous with context and social demic support to a learner. Learning is made ease presence awareness. Although a great number of when a learner has consistent awareness of context studies (e.g., Shariq, 1999; Polanyi, 1966; 1958) and presence of social resources (Kekwaletswe, have shown that knowledge creation and transfer 2009; 2007). Ubiquitous personalized knowledge is essentially a human-to-human process or an support refers to the provision of context sensitive outcome of social interaction (Nonaka, 1994), and anywhere, anytime help as learners traverse the relationships or roles of context and social varied locations. presence awareness as catalysts for knowledge Learners use awareness of context and social sharing and transformation in a learning environ- presence as a means to access ubiquitous learn- ment has not been explored. This chapter aims to ing support, interpret and adjust their knowledge contribute to that effect. – sharing what they know with others through This chapter is about exploring and understand- social interaction. The chapter, thus, focuses on ing how, through a learning management system the peer-to-peer interaction and the learning envi- environment, a learner uses social awareness ronment. The social interaction whose outcome is to leverage personalized knowledge sharing. It transformed knowledge and provision of support reveals the actual nature of ubiquitous learning is location and time independent. Since sharing through social interaction where awareness of learning experiences is a ubiquitous phenom- context and social presence is argued to be the enon, learners continuously use awareness of the underlying process of the activity. The chapter is environment and awareness of available social on how varied forms of communication for knowl- resources they can draw upon to facilitate knowledge sharing in an LMS learning environment are edge consultation (Kekwaletswe, 2009: 2007). an outcome of social interaction coordinated by This chapter is about the advancement of the social awareness. Social awareness is synonymous human-centric approach to knowledge creation

29 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

and sharing through enhanced person-to-person locations, interpret and use social awareness for interaction – where context and social presence interaction whose outcome is knowledge shar- awareness is of vital significance to how learners ing. I argue that the effectiveness of a knowledge create, use, and share knowledge. The chapter sharing within a learning management system is uses the contextual inquiry research method to fundamentally affected by the social interaction understand how a learner goes about sharing and the context in which a problem-driven learn- knowledge, and the tools they use. The practical ing activity takes place. contribution of the chapter is the understanding In the chapter, learning through a learning man- of learning management systems environments agement system environment is defined as any sort and how learners use social awareness to model of learning and knowledge sharing that happens their actions for the provision of personalized due to social awareness when the learner is not at a academic support. The rest of the chapter is as fixed, predetermined location – in varied learning follows: firstly, the background to the research contexts. An LMS learning environment conveys problem is given; secondly, the theoretical foun- ubiquitous learning that is not confined to specific dations for the chapter are articulated followed locations, and is time independent. Context aware by the research methodology; lastly, the chapter ubiquitous learning is meant to support learning discusses, with examples of empirical evidence, by identifying a learner’s surrounding contextual how social awareness is used to leverage knowl- environment and social presence to provide a edge sharing in the learning environment. The rounded and seamless learning experiences. In a chapter is then concluded. contact university, there tend to be disproportionate access to available social resources between when learners are attending scheduled or formal classes BACKGROUND TO THE and when they are away from scheduled classes RESEARCH PROBLEM (Kekwaletswe and Ng’ambi, 2006) especially as they move away from campus locations. The chal- The practical relevance of the chapter is towards lenge for universities is that instructors and tutors enriching personalized on-demand academic sup- are not always available to provide learners with port through social awareness. To this point, this ubiquitous support as learners move away from chapter focuses on LMS communication tools ap- formal locations or contexts. The alternative for propriated by awareness of learners’ context. The these learners is to consult with a knowledgeable ideal social awareness for knowledge sharing is peer who, for the most part, shares a background. one that is sensitive to the background of a learner There are three types of contexts within which (social context includes culture and language), a learner is mobile and for which a learner needs a arranging these aspects to provide immediacy on learning management system and social awareness the available social network, independent of the as a medium to transform knowledge. The follow- learner’s location and task at hand (Kekwaletswe ing three learning contexts have been discussed and Ng’ambi, 2006). The focal point of the chapter by Kekwaletswe and Ng’ambi (2006). is therefore how social awareness is used to support personalized social interaction for a South Formal Learning Contexts African university learner as s/he traverses varied learning contexts. The objective of this chapter is These contexts represent formal structures – such not to confirm previously established premises and as scheduled lectures and laboratory sessions – in theories but to find out, through engagement with which a learner’s behaviour and action is shaped learners, how they, as they traverse varied learning according to the university class timetable. Inter-

30 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

action in these spaces is usually one-way from Informal Learning Contexts instructor to learner using English as the official language of instruction, although learners often Although it is difficult to be explicit on the char- interact with each other using their own diverse acteristics of an informal learning context, these languages. In the formal learning context, the contexts include working during after-hours or instructor delivers a lecture and a learner either weekends at university residences or private takes notes or is given a handout. Even though homes. In these environments, a learner usually learners are invited to ask questions, there is little uses his or her mother tongue to consult with time to assimilate the material and meaningfully peers (Kekwaletswe, 2006) or may write down engage with the learning materials (Ng’ambi, questions to ask the instructor when they do get 2004). Learning and knowledge seeking action is, into the formal context (Ng’ambi, 2004). There therefore, mostly passive. In this context, social are three things that must be known to provide presence is usually availed through wired PCs in ubiquitous personalized learning and knowledge the computer labs and the face-to-face presence support to a mobile learner in an informal learning of tutors, instructors and peers. environment: a) Knowledge about the location of a learner so as to help identify the potential Semi-Formal Learning Contexts knowledgeable peer; b) the preferred language of a learner in which he or she is likely to be These contexts represent informal spaces on cam- conversant, and c) the awareness of a peer’s social pus used by learners, usually while waiting for presence and contexts – including location and the next lecture to start or after it finishes. They situation (Kekwaletswe, 2006). include the library, cafeteria, mingling areas and walk-in laboratories. As learners begin to reflect The Research Problem on the previous lecture and skim through the learning materials, questions begin to arise for which One of the prevailing educational challenges in the clarifications are required (Ng’ambi and Hardman, new South Africa is that of providing personalized 2004). The challenge a learner is faced with is academic support to under-prepared learners (Jaf- how to find an instructor or tutor who is available fer et al., 2006). In a contact university (as opposed for immediate or on-demand consultation. Most to distance learning university), where students instructors schedule consultations and are often attend formal lectures and scheduled laboratory unavailable for ad hoc interactions. The dilemma sessions, there tends to be inconsistency in social for most learners is that the consultation periods presence or access to available social networks are limited and not always suitable (Ng’ambi, for academic support between when learners are 2004). Consequently, the learner’s alternative is on campus and off campus (Kekwaletswe, 2006). to find the nearest socially present knowledgeable Based on empirical evidence, I argue that in a peer or class-mate who can provide social sup- learning environment that has a learner population port (Kekwaletswe and Ng’ambi, 2006). In the with diverse social backgrounds and languages, semi-formal learning context social presence is social awareness of peers with a shared background still availed through wired PCs in the computer is fundamental in the provision of personalized labs and face-to-face presence of peers. Since the academic and social support. I distinguish between environment is on campus, where most learners a learner being on campus in a formal context meet and come for formal classes, a learner is still and being on campus or off campus in informal very much aware of the available social network. contexts.

31 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Since knowledge sharing and learning tasks THEORETICAL FOUNDATIONS are not confined to particular locations but are carried across different learning contexts, I ar- This section lays out the epistemological and on- gue that learning and social resources available tological foundations of the inquiry and reviews to learners ought to move with a learner. I use the existing literature that informs the chapter. the term social resources pragmatically to mean The inquiry is about learning management system knowledgeable peers, tutors, instructors and the environments that support learning and sharing awareness of context and presence (Kekwaletswe, of knowledge, with context and social presence 2006; Kekwaletswe and Ng’ambi, 2006). In this awareness as catalysts for learning actions. The regard, the problem is that of ensuring that the theory of knowledge cannot be divorced from the quality of resources available to learners remains situations under which knowledge and learning consistent for supporting a knowledge transform- experiences take place and this is reviewed next, ing task or action regardless of time and location followed by a specific focus on learning environ- of a learner (Kekwaletswe, 2006). In other words, ments. Social presence and context for interaction learners should have a consistent social aware- in such environments also receive special attention ness of presence and context. The argument is in this section. that when learners solve and engage in a learning task – problem based learning – they bring prior Knowledge knowledge and experience to the social interaction situation, where the knowledge transformation To understand knowledge and how it is shared in outcome is influenced by social awareness. Thus, a learning environment as an outcome of social learning management systems ought to leverage interaction, I unpack the logical development of this. But first, we need to understand how LMS how knowledge is created, retained, and used. communication tools are and could be used. The construction of knowledge requires process- In conventional interaction with the learning ing of data into information, new information activity, learners have an impoverished mechanism is then created which is then communicated or for providing social awareness of the available transferred outside of the human brain. With that social network. Consequently, personalized social premise, I briefly define data as raw facts that can support is denied when learners do not have the be shaped and formed to create information. Thus opportunity to access a consistent social network information is data that is given a meaning within (ibid.). In order to understand social presence a context. Therefore, only data and information can and context awareness and their role in learning be captured, transferred or stored outside the brain. system environments, we must understand both Knowledge is created from the processing what the context is and how social awareness can of information, and during this processing, new be used to facilitate the ubiquitous learning. An knowledge can be acquired or created for future understanding of the context and a sense of social use, when more or new information is acquired presence enables the learner to model behavior and processed (Van Beveren, 2002). Knowledge along the expectations or the shared understanding is transformed into information within the brain of a social learning community. to be communicated externally through language or demonstration (ibid.. 2002). Language may include different forms of communication such as text, verbal and body gestures. What then constitutes knowledge?

32 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

There are probably as many definitions and social psychology and cognitive science; from explanations of knowledge as there are theories economics to management and organization of knowledge from diverse disciplinary perspec- analysis. The breadth and depth of the subject tives. Without joining the philosophical debate would not allow me to trace a lineage of existing of what exactly knowledge is, it is encouraging knowledge theories. to notice that literature has adopted a pluralistic Nevertheless, cognitive theories have looked epistemology, acknowledging that there are many at knowledge as a representational phenomenon. types or forms of human knowledge. The following Winograd and Flores (1986) point out that “a are only some of the definitions of knowledge. cognitive being ‘gathers information’ about things and builds up a ‘mental model’ which will be in • Knowledge is an individual’s stock of in- some respects correct (a faithful representation of formation, skills, experience, beliefs and reality) and in other respects incorrect. Knowledge memories (Alexander et al., 1991). is a storehouse of representations, which can be • Knowledge is the stock of conceptual tools called upon for use in reasoning and which can and categories used by humans to create, be translated into language while thinking is a collect and share information (Laudon and process of manipulating representations” (op. cit., Laudon, 1995) p73). In cognition-action theory, the foundational • Goffman (1978) explains knowledge by hypothesis is that action always possesses a cogni- asserting that the “making of knowledge is tive basis which is reflected in the representational a way of enacting reality, giving existence activities of the mind. The duality of cognition and to things and events, and organizing the action underlies the conceptualization of knowing world.” as a computational activity (Patriotta, 2003). Since • Nonaka (1994) deems knowledge to be human behaviour is always oriented towards a “justified true beliefs”. The theory of goal, action is a form of problem-solving, where knowledge creation sees knowledge as a the actor’s problem is to find a path from some dynamic human process of justifying per- initial state to a desired goal state, given certain sonal beliefs as part of an aspiration for the situations along the way. “truth”. Accordingly, there is a need for researchers to understand how learners acquire new problem Generally, scholars of knowledge observe representations for dealing with new problems. A knowledge in two ways: “know how” and “know- general theme uniting many situated approaches that”. The former is created ‘here and now’ in a to cognition is a change in the way the person- specific, practical context and conveyed through environment relationship is envisaged. Rather than analogies and metaphors; the latter is contained a person ‘being’ in an environment, the activities in manuals and procedures and oriented towards of person and environment are parts of a mutually a context-free theory (Patriotta, 2003). constructed whole. The inside-outside relationship between person and environment is replaced by Knowledge as a Multifaceted a part-whole relationship (Bredo, 1994). Learn- Phenomenon ing is centered around problem-solving and is intricately related to the context; ‘context’ here The evolution of institutional knowledge has been means understanding (a) the problem’s conceptual informed by a wide spectrum of theoretical tradi- structure as well as (b) the purpose of the activity tions. Knowledge is a multifaceted phenomenon and (c) the social milieu in which it is embedded which has been debated in a variety of disciplin- (Scribner, 1987). ary contexts – from philosophy and sociology, to

33 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

The above briefly highlighted the multifaceted This notion of Short et al. (1976) was, how- views on learning and knowledge and how it is ever, questioned by communication researchers created. The highlight is what informs my own (Gunawardena & Zittle, 1997; Byam,1995; definition of knowledge sharing. Having given an Walther, 1994) who showed that perceived social overview of what knowledge is and how it may presence in mediated interactions varies among be created, the following section will highlight participants in the same mediated conversations. the role of social awareness in transformational That is, many of their research participants per- learning. In order to understand social presence ceived mediated discourse as more personal than and context awareness and the degree of their role traditional classroom discussion. The claim by in a learning management system environment, Short et al. (1976) that the quality of the com- we must understand both what the context is and munication media determines its social presence how the social awareness can be used to facilitate or richness was also disputed by Ngwenyama and the ubiquitous learning. Lee (1997) who showed that the communication richness of a media is dependent on who uses Social Presence Theory the media and how they use it. Gunawardena and Zittle (1997), for example, defined social presence In this section, the concept and Theory of Social as “the degree to which a person is perceived as Presence is discussed. As I examined the social ‘real’ in mediated communication” (op. cit., p8). presence studies and literature, it became apparent They, like Ngwenyama and Lee, also argued that that most, if not all, of the studies on the concept social presence was as much a matter of individual employ a positivist approach. Whilst this chapter perceptions as an objective quality of the medium. employed an interpretive approach, it is essential In a survey to measure students’ perceptions that I highlight the extent to which the theory and of the social presence of others in a computer concept has been explored in previous research conference, Gunawardena and Zittle found that – which in turn informed the redefinition of the perceived social presence predicted more than concept as used in this chapter. half of the variance in students’ satisfaction with Short, Williams and Christie (1976) asserted the conference. Their results also indicated that that different communication media express vary- students who felt a higher sense of social presence ing degrees of social presence based on their abil- enhanced their computer communication using ity to transmit nonverbal and vocal information. emoticons to express missing nonverbal cues in Thus, they initially introduced the Social Presence textual form. Theory as “technical social presence,” defining Rourke, Anderson, Garrison and Archer (2001) it as the capacity of the medium itself to present regard social presence as one of the three fun- the “salience of the other person in interpersonal damental “presences” that support learning, the interaction” (p65). Two concepts associated with other two being cognitive presence and teaching social presence are “intimacy” (Argyle and Dean, presence. Thus, they define social presence as “the 1965) and the concept of “immediacy” (Wiener ability of learners to project themselves socially and Mehrabian, 1968). Intimacy depends on fac- and affectively into a community of inquiry” tors such as physical distance, eye contact, facial (p50). Rourke et al. identified three categories of expression and personal topics of conversation. social presence indicators – affective experience, Immediacy is a measure of the psychological cohesive experiences, and interactive experiences distance which a communicator puts between – and explored their use in online discussion. Af- himself and the object of his communication. fective experience contain personal expressions of emotion, feelings, beliefs, and values; Cohesive

34 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

experience are communication behaviours that Context Awareness build and sustain a sense of group commitment, such as greetings and salutations and group or Context and context-awareness are fundamental personal reference; Interactive experience are concepts in a learning environment where a learner behaviours that provide evidence that others are is not fixed to particular locations. The follow- attending, such as agreement/disagreement, ap- ing discussion of context and context awareness proval and referencing previous messages. studies and literature is intended to show how the There is, evidently, a lack of social presence concepts are relevant to learning and ubiquitous research in learning management systems envi- social interaction. ronment. Most studies of social presence have Lonsdale et al. (2003) describe context as a set focused on the nature of online discussion and of changing relationships that may be shaped by accordingly conceptualized social presence as a the history of those relationships. The figure below single construct with an emphasis on perceptions gives their hierarchical description of context as of the presence of peers (e.g., Swan, 2003 & 2002). a dynamic process with historic dependencies. As noted by Richardson and Swan (2003), there In Figure 1, a snapshot of a particular point is some indication that instructor social presence in the ongoing context process can be captured may be equally important. Social presence of in a context state. A context state contains all of instructors has been considered in explorations of the elements currently present within the ongoing “teaching presence” (Shea et al., 2003; Anderson context process that are relevant to a particular et al., 2001). Researchers have demonstrated learning focus, such as the learner’s current proj- both that students perceive the presence of others ect, or a learning activity. A context substate is (Picciano, 2002; Gunawardena & Zittle, 1997; the set of those elements from the context state Gunawardena, 1995) and that they socially pres- that are directly relevant to the current learning ent themselves (Swan, 2003 & 2002; Rourke et and application focus, that is to say, those things al., 2001) in online course discussions. that are useful and usable for the current learn- Nevertheless, there is lack of studies – positivist ing system. Context features are the individual or interpretivist – looking at learners perceiving elements found within a context sub-state. Each the presence of others and socially presenting feature is atomic and refers to one specific item themselves in a learning environment or context. of information about the learner or his/her set- This chapter addresses this shortcoming and it does ting. In implementing context awareness within so by following an interpretive tradition, diverting their architecture, Lonsdale et al. (2003) derive from the positivist tradition. In the chapter learners a context sub-state and use the context features use awareness of a social presence for purposes contained within it to determine what content of social interaction whose outcome is knowledge might be appropriate for a learner. sharing. In this chapter, social presence is defined Context related to the human environment is and understood to be the mediated presence of structured into three categories (Schmidt et al., another learner who could provide personalized 1999): on-demand social support for a learning problem as the learner traverses varied learning contexts. • Information on the user; knowledge of Context and context awareness are also funda- habits, emotional state, bio-physiological mental concepts in a learning environment where situations. a learner is not fixed to particular locations. The • The user’s social environment; co-loca- concepts of context and context awareness are dis- tion of others, social interaction, group cussed in the next section, in view of the fact that dynamics. they form the second phenomenon of the chapter.

35 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Figure 1. Context hierarchy (Lonsdale et al., 2003)

• The user’s tasks; spontaneous activity, en- means being aware of what you can know about gaged tasks, general goals. your context while concrete context awareness means being aware of what you do know about Context related to the physical environment your context (op. cit.). is also structured into three categories:

• Location; absolute position, relative posi- RESEARCH METHODOLOGY tion, co-location. • Infrastructure; surrounding resources Empirical Inquiry in Formal for computation, communication, task Learning Contexts performance. • Physical situations; noise, light, pressure Interactions of learners in formal learning contexts (Schmidt et al., 1999). at the University of Cape Town – as a contact university – are usually passive (Ng’ambi, 2004). An entity needs contextual information to That is, learners who are mobile hardly need choose between alternative strategies in order to to actively interact with each other in a formal reach its objectives, or to do useful work. Context lecture. The assumption is that it is mostly out- gives hints about what is or what is not achievable side the formal contexts that university learners, (Rakotonirainy et al., 2000). If the learner’s initial notably at UCT, begin to interpret and engage knowledge sharing or transfer objectives are not effectively with learning materials and therefore reachable then they are changed to suit the cur- need social presence and context awareness about rent context, i.e., the objectives change or actions available peers for social support. Since aware- are taken according to the context to maintain ness of context and social presence is not much an objective. Abstract context awareness then of a need for learners in formal environments, the

36 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

empirical inquiry does not focus on formal learn- to day (Beyer and Holtzblatt, 1998). Essentially, ing contexts. The empirical evidence suggests that contextual inquiry consisted of observing learners’ social interaction of learners, where they begin to actions and talking with learners in their learning simplify meanings and begin to apply what they environment while they were engaged in authentic have learned in class, happens mostly outside learning tasks. The focus of the contextual inter- of the formal learning context. The argument is views and textual interactions was on the learners’ that South African learners, who traverse varied mundane activities, notably problem-driven social locations, need more personalized academic sup- interactions related to learning. port through social awareness as they move away In Figure 2, learners use social awareness to from formal learning contexts. In view of this, the interact with others who are not in the same loca- empirical study and evidence does not include tion using Web-based mediated communication learning management system environments in to send and read emails or chat through instant formal learning contexts but focuses on semi- messaging. Learners interacted with learning formal and informal learning contexts. resources, posing or responding to questions via The empirical evidence was gathered at the a Web-based learning management system. In University of Cape Town campus and residences the figure, learners also use mobile devices such using contextual inquiry methodology. Contextual as Portable Digital Assistants (PDAs) to interact inquiry is a field research framework that depends via mobile instant messaging, and mobile phones on conversations with users in the context of their that could be carried around to interact verbally work (Holtzblatt and Jones 1994). It is based on or send short text messages (SMSs). ethnography, where the researcher goes into the The study began with sixty learners complet- research participant’s own environment. It is ing a qualitative questionnaire meant to understand an explicit step for understanding who the user communication tools, forms of interaction, social really is and how the work progresses from day presence and context awareness. The initial sixty

Figure 2. Representation of the research framework

37 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

learners were randomly identified as they traversed KNOWLEDGE SHARING IN the university campus and in libraries and labo- A LEARNING MANAGEMENT ratories. There were no specific criteria for selec- SYSTEM ENVIRONMENT USING tion other than that the mobile learners had to SOCIAL AWARENESS represent or give a true reflection of typical learners at the University of Cape Town. Socio-cultural theories of mediated learning Twenty of the sixty were asked to take part suggest that what is learned will emerge from the in the Web-based LMS and PDA pilot based on relationship between human action, and the social, their willingness and commitment to be available cultural, institutional and historical contexts in for the duration of the study. Since the study was which action occurs. This makes it essential for on understanding how university learners com- us to understand these contexts and activities municate, the tools they use and how they exhibit before we begin to investigate issues of learning awareness, it was not necessary to determine (Sutherland et al., 2000). who should participate or not participate in the study, other than that they should be registered In view of what Sutherland et al. (2000) suggests, university learners. The WiFi-enabled PDAs it is first important for us to understand how social were to facilitate the “anywhere anytime” on- awareness manifest in a learning environment if line learning interactions and the opportunistic we were to design appropriate learning manage- mobile instant messaging. Eight of the twenty ment systems that could leverage knowledge shar- learners were interviewed and observed as they ing and learning actions. This section discusses interacted with others in authentic LMS learning how social awareness manifest amongst learners environments. Although this study was not on a sharing learning experiences. In earlier sections, particular LMS platform, participants also used I noted that one of the prevailing challenges in Vula, a university SAKAI-based online learning the new South African higher education is that of environment. The textual interactions were in the providing personalized support to under-prepared form of chats and instant messages. The contextual learners – who by the very nature of the South Af- verbal interviews were recorded using a digital rican population come from diverse backgrounds recorder and transcribed. The storylines were in and cultures. the form of thick descriptions including all the In most African cultures, it is uncommon for environmental details that could be observed and younger people to interact with or question their documented. Pictures capturing the actions and elders. This phenomenon tends to apply in an environment situations were constantly taken. The educational environment where learners are not focus of the inquiry was on how learners interact, comfortable questioning the instructors. The al- the tools they use and how awareness is used to ternative for these learners is then to consult and support personalized learning. interact with their close friends and knowledge- In the next section, the empirical evidence is able peers to provide academic support. For this discussed in terms of how learners interact, the reason the inquiry focused on social presence of appropriateness of the tools they use and how peers and not “teaching presence”. Consequently, social awareness manifest in a learning environ- this chapter is about interactions amongst learners ment. It is worth noting that the aim of the chapter (peers) for purposes of knowledge sharing, as they was not to study a specific LMS platform but to traverse varied learning locations. However, it is study how knowledge is shared through aware- worth noting that in South Africa, availability and ness of context and social presence – awareness access to wireless and wired computers, remains determines learning actions and the appropriate- a challenge. These resources become even more ness of LMS tools. inadequate as you move away from campus. The

38 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

study was therefore guided by the three localized remote locations was mostly mediated by mobile learning contexts discussed in section two, which technologies, e.g., mobile phones and PDAs. apply to a typical South African contact university. Knowledge sharing involves two actions: The framework depicts social awareness and transmission (sending or presenting knowledge presence in three different learning contexts of the to a potential learner) and absorption by the audi- mobile learning environment. Social awareness is a ence (Davenport and Prusak, 1998). If knowledge mental concept where a peer and a learner become is not absorbed, it has not been shared. In other aware of the social network that follows them as words, merely making knowledge available is not they move across the varied learning contexts In sharing. In this regard, social interaction achieved Figure 3, a learner is consciously aware of avail- and enhanced by awareness of context and pres- able knowledgeable peers should s/he encounter a ence is necessary. The research investigated how learning problem for which s/he needs to consult. learners use social awareness – social presence By the same token, a peer is consciously aware of and context awareness – to enable a knowledge the presence of other learners should s/he not be sharing interaction. It sought to understand ubiq- able to address a problem encountered by a learner. uitous learning where learners support others as That is, a learner and a knowledgeable peer have they traverse the three learning contexts. a consistent social awareness of a social network Communication or interaction that is medi- (social resources) regardless of their location ated by technology is generally grouped into and context. Even though learners interact via two categories: asynchronous and synchronous. Web-based environments, in this study, the social Asynchronous communication occurs between awareness and availability of social resources in learners independent of time and location. This kind of communication does not need the send-

Figure 3. A framework for mobile learner-to-learner environment (Kekwaletswe, 2006)

39 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

ing and the receiving learners to be “available” why the situation is occurring. In this chapter, the concurrently. Examples include leaving a phone context-aware learning interaction environment voicemail, posting to or reading a discussion board, supports (who) a learner as s/he engages with the and sending and receiving email (although this (what) learning materials, (where) whether in could also be considered synchronous). On the semi formal or informal learning contexts (when) other hand, synchronous mediated interaction is anytime through the use of context awareness considered a “real-time” experience between two and social presence mechanisms. In this chapter, or more learners. Examples of tools that facilitate social awareness, and not a learning management synchronous communication include telephones, system, does actually determine why a learning audio-video conferencing software, instant mes- situation is occurring. saging, virtual chat, virtual classrooms, and white- boards. Asynchronous and synchronous mediated- Social Awareness in a communication can be used in individual or group Learning Environment learning situations, as well as traditional or online learning environments. Although a significant In semi and informal contexts, learners engaged body of research validates the notion that learning in a learning activity are able to use implicit and is a social act, learners may still acquire knowledge explicit context awareness to increase or decrease in a mediated interaction. There are several forms the interaction whose outcome is learners “help- of technology mediated communication, such as ing” or “supporting” each other. The notion of electronic mail, web-based consultation environ- social interaction presupposes an existence of two ments, instant messaging on wired networks and or more learners speaking or acting. And also, short message service (SMS) on mobile phones. embedded in this notion is the assumption that a Although they support mobility of a learner and learning community is socially present without may afford presence awareness, they do not do which interaction – “help” or “support” – is im- so in real-time. possible. The following tables give examples of In the study, there was a need to select a mediat- how learners use implicit and explicit context and ing technology that supports mobility of a learner social presence awareness to increase or decrease as well as presence awareness of available peers social interaction whose outcome is personalized in real-time. Thus, mobile instant messenger (IM) learning. To help the reader understand how Jabber client for Pocket PCs (PDAs) called iMov awareness manifests, the examples are structured messenger (www.jabber.org) was selected. The as follows: what the researcher wanted to know, mobile instant messaging on PDAs supports an the learner’s experience and a brief discussion. immediate on demand formal or informal expres- The experiences are grouped in a table form per sive interaction. Such real-time interaction could research theme and query. Showing the empirical be one-to-one or several concurrent dyadic con- evidence this way was to allow the reader to see versations. Mobile IM provides mobile learners where the low level research questions (elements with a real-time interactive space to share learning of awareness) that support the main question are experiences – exchanging textual messages that addressed. do not require detailed email-like messages or Table 1 shows how learners are conscious or face-to-face interactions. aware of the situation and the context in which Dey and Abowd (1999) suggest that context- they are interacting and sharing experiences with aware applications look at the who’s, where’s, others, e.g., how fast they would need the response, when’s and what’s (that is, what the user is doing) the content of interaction, etc. It also highlights of entities and use this information to determine how presence and availability help the decision

40 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

regarding how to communicate. The different acquire knowledge and experience through social mediating tools offer different opportunities and interaction, drawing upon available social re- accessibility. sources for the solution of practical learning The preceding tables showed just some of the problems. The empirical evidence showed that examples of how awareness of context and social social awareness of the social resources, the learn- presence manifest in a learning environment. In ing task, and the environment does influence the a learning environment, the central activity is action a learner undertakes – hence the appropri- knowledge where peers provide personalized ateness of an LMS communication tool. social support through social interaction. Learners

Table 1. How does a learner share knowledge and learning experiences with others?

Researcher’s Intention Different Learners’ Researcher’s Response / experiences Comments How does a learner usually communicate …through SMS, Here the learner’s intention is to seek with peers, besides …I call or email, help or share knowledge about a learning face-to-face meetings, when faced with a …I use my mobile phone, problem. learning problem and/or …we do online chat, He or she picks up an appropriate tool that needing help? …I facebook sometimes best mediates the thought. …mobile phone chat (viamiXit). NB: MiXit is a mobile phone service that allows friends to chat (send/receive text) instantly and in real-time. What is a learner’s reason for …Depends on how desperate I am, if des- The experiences show evidence of the communicating using a particular tool? perate I phone, if not I SMS. learners’ situation as awareness of context. ...Depends on what I want to say Location, cost, need and reason to interact …It depends on the situation at the time are taken as context awareness which influ- …Depends on the seriousness of the situ- ences how learners consult each other, e.g., ation calling will give immediate response. …Availability of funds …Economic limitations and time available …It depends on where I am at the time that I need to contact them. How does a learner decide which method of …With email, it.s efficient because I can Learners use content of the interaction interaction to use? phrase the issues better and they can as context for which a mediating tool is review and reply with specific and thorough chosen. answers. Learners decide on the method of interac- …It depends on the issues I need to discuss tion with peers based on how quickly and or message I need to get across. fast they need the response. ...How they will respond. Where a short message is enough for the …With SMS and calls, the response is faster interaction, they text each other. and I can get help I need in time. Issues of time are noted as an influencing …It is the fastest way to get feedback from factor in picking a mediating tool, noting them. that most interactions are done on-demand …Whichever will reach the fastest in terms to solve a learning problem instantly. of being seen by the person, and the im- Learners use social presence awareness mediate need. as an influencing factor for choosing a …It.s quick and direct (Mobile phone specific mediating tool. calls). …They are always with them (peers always carry their mobile phones) ...I know they will always have their cell ...Depends on if they are available, and what will be quicker at that moment. ..If they are available (via mobile phones)

41 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 2. How does location as context awareness influence the knowledge sharing social interaction?

Researcher’s Intention Different Learners’ Researcher’s Comments Response / Experiences From which locations is a learner willing …Home, library, hospital Learning tasks and problems are not to contact his/her peers for a learning task …University, basically anywhere confined to fixed locations, thus learners or problem? on campus could share knowledge about a problem ...Only within university premises from any location. …Everywhere …Malls and shops …In the lab If a learner wanted to interact with a peer, .If they are in a lecture or the library and I Learners use awareness of location to how would s/he consider a peer’s present have a small issue then I won’t bother them. determine if the peer’s location favors a location? In essence, I would just take into consid- knowledge sharing social interaction. eration my need to contact them and their availability in terms of what they’re doing and where they are. .If they were on campus or at the library or maybe at home, I’d feel free to contact them since they will probably be more flexible and relaxed to respond. How does a learner’s knowledge of where a ..Makes it easier because you would know The experiences show that awareness of peer is (their location) affect the knowledge who is closest to you for help a peer’s location would determine if it is sharing social interaction? ..It would be great to know if they are plausible or would make sense to consult nearby, but if they’re not anywhere where a peer. I can reach them, then I’ll have to call regardless of where they are. …The closer he or she is, the better, because it will be easier for me to go to her when push comes to shove. …If I knew they were in a club or something, I wouldn’t bother them with school work.

Awareness of context and social presence is fundamental with the role it plays in the sharing used as a tool that enables personalized sharing decisions. That is, awareness of a social presence of experiences. An aspect of social awareness enables an opportunistic learning and knowledge includes the location factor and context, which interaction and can also motivate a learner to manifests as valuable to how the interaction is in- engage with a learning task. fluenced. Culture and social background manifest as playing a fundamental role in learning contexts where learners often would rather seek help or CONCLUSION support from peers who share a background. A common background allows better understand- This chapter did not directly address a specific ing and better communication and thus eases up learning management system (LMS) platform but learning and knowledge interactions. addressed awareness and forms of communication Awareness of context also manifests in how an that can be encountered as tools of LMS platforms. interaction is influenced by a learner’s emotional The chapter argued that to be able to design LMS and physical state, including behaviors and actions that would efficiently enable social interaction of others. Environmental situations also manifest among learners not confined to particular places, as playing a role in a learner’s knowledge sharing then there is a need to first understand how learn- decisions. Social presence of peers manifest as ers interact and the tools they use. In this chapter,

42 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 3. How does a peer’s activity and emotional state as context awareness influence knowledge sharing social interaction?

Researcher’s Intention Different Learners’ Researcher’s Comments Response / Experiences How would knowledge of what a peer is …Issues like a person sleeping or eating A peer’s activity influences the decision doing (current activities) affect a learner’s may make me wait a while. about whether to consult with that learner interaction decision? ..I need to know if they got my full attention or consult with someone else. ..If sleeping, I will leave them be, if studying will interact with them …Would respect that they are really busy and seek help elsewhere How do actions and behaviors of others ..I am motivated by seeing others work and Presence of others, that is, seeing what they affect a learner’s decision to learn or share tend to slack when others aren’t working are engaged with influences the decision of experiences? …When others are discussing work, it a learner to get involved and do the same. increases chances of grasping concepts. For some, what others are doing does not ...It doesn’t affect me much, because we all influence their decision to work have different learning capabilities. . ..I take seriously any feedback or advice from friends who have done courses I’m doing. ..If I am studying and people are noisy, I will not be able to concentrate. If they are serious, I too will be. To what extent do emotional states of a peer …Very little, unless they’re my friends and Learners could be sensitive to the mental affect a learner’s actions? I am concerned, or if it’s group work and state of a peer, thus altering the way they they’re slacking due to moods. consult for help or sharing experiences. …You know what you can ask and can’t, their moods may limit your questions. ...Lets you know how to approach them How does the awareness of a peer’s current …Very helpful, you don’t want to interact The experiences of learners suggest that emotional situation (e.g., s/he is stressed, and learn with a sad guy, so they ought to emotional states and activities of peers do happy, sad, cheerful, etc.) help in interact- be happy, cheerful. If anything they can be influence a social interaction. ing with them? stressed. The social awareness determines how …It’s useful in the sense that I can know learners approach their peers for a learning whether to bother them or not. It makes purpose. little difference in group work unless their The emotional states could be regarded as situation is very serious. signs and rules, thereby altering the context .It makes me understand their behavior and in which a knowledge sharing activity the way in which they react to my questions happens. or experience. Although these are unwritten signs and …It makes me understand them better and rules, learners know not to disturb or pester assist in determining what things I can or a peer who is not in the best of emotion and cannot say. spirit to help with a learning task. …Will try to approach them correctly, depending on what kind of mood they are in? learners used social awareness to determine the and the degree of their role in learning, we must appropriateness of an LMS tool to engage in a understand both what the communication tools knowledge activity, as they traverse the varied are and how the social awareness can be used to contexts. Mobile phones, SMS, PDAs, email, facilitate the ubiquitous learning. An understand- instant messaging were in this chapter seen as ing of the context and a sense of social presence tools and forms of LMS communication, without enables the learner to model behavior along the having to study a specific LMS platform. In or- expectations or the shared understanding of a der to understand learning management systems social learning community.

43 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 4. How do environmental situations as awareness of context influence knowledge sharing actions?

Researcher’s Intention Different Learners’ Response / Researcher’s Comments Experiences To what extent is a learner aware of ..Temperature, I don’t like studying alone The environmental context and situations his/her surroundings, during a learning when it’s too cold so I normally chat a lot do influence how learners act and how a activity, and how does that influence his/her when it’s cold. learning action happens. decision to interact with peers? ...The time of the day determines when I The temperature during the day or need to chat or where I need to work. ambience or noise levels may help a ...Noise, too much movements, then I can’t learner resort to a different knowledge concentrate on my work. action. For example, a learner saying that …When I talk I can tell who is an under- she prefers to do more social interaction standing person from a lost one. than studying in isolation when the tem- …When I am with people I am not so aware peratures drop. of my surroundings in the sense that I am A learner consults with peers, taking when I am alone, however, it is when its si- advantage of their social presence and sur- lent and cold that I tend to SMS/call people roundings. with my problems. ...If I see a lot of other students studying and discussing with their friends, they keep me motivated and I ask them questions.

Table 5. What is the role of Social Presence Awareness in the knowledge sharing environment?

Researcher’s Intention Different Learners’ Researcher’s Comments Response / Experiences How is it important for a learner to know a Very important Awareness of social presence or a peer’s peer’s availability before he/she decides to For me, it is quite important availability is essential for knowledge interact with them? interaction decisions How does awareness of presence influence ..It helps me determine who I can get as- Social presence of knowledgeable peers is a learner’s knowledge action? sistance from important for interaction whose purpose is ..Puts me at ease since I know my friends sharing knowledge experiences. could help when I get stuck Learners are consciously at ease when they …I know I’m going to get help know their peers are available to help with ..If they are available and willing to help, I a learning task, at anytime. will contact them. Knowing a peer is socially present (aware- …I have a choice to pick any depending on ness of social presence) gives a learner a who is more knowledgeable on that particu- sense of having a personalized academic lar topic. support that “follows” them regardless of a …I know that when my academic questions learning problem and location. arise I’ll be answered with certainty ...Then I could reach them whenever I need them ..It makes learning very easy, we could exchange ideas and information anytime and I will know that I am always able to ask for help How is a learner able to keep the sense of ..Make the messages as short as possible Short, quick and instant responses presence during a technology-mediated ..Instant or quick responses grab a learner’s attention in a text-based interaction, particularly when using text- …I don’t want to wait for a response to my social interaction. based instant messaging. question, which is why I use IM. How is a learner able to read emotions of . ..Words, descriptive uninhibited words Even though they are not in the same peers during IM interaction and how can s/ ... I know when they have something else in location, learners are able to exhibit social he sense if the peer is interested in talking their minds awareness levels during a mediated social to them or not? . ..One word answers would indicate that he interaction isn’t keen to talk. . ..The level of response and language used (enthusiasm)

44 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

The role of context awareness, and how it REFERENCES is experienced, is significant to ubiquitous social interaction whose outcome is personalized Alexander, P. A., Schallert, D. L., & Hare, V. C. knowledge sharing. The context awareness in (1991). Coming to terms: how researchers in learn- the learning management system environment ing and literacy talk about knowledge. Review of manifests in three categories: 1) Mediating tools Educational Research, 61(3), 315–343. or applications as context of the sharing action - Anderson, T., Rourke, L., Garrison, D. R., & learners use context awareness to decide on the ap- Archer, W. (2001). Assessing teaching presence propriateness of the mediating tool. For example, in a computer conferencing context. Journal of if a learner is faced with a time-driven learning Asynchronous Learning Networks, 5(2). task she cannot resolve in isolation then she uses a PDA or a mobile phone to call or text message Argyle, M., & Dean, J. (1965). Eye-contact, dis- a peer as opposed to writing and sending an email tance and affiliation. Sociometry, 28, 289–304. to a peer. In this situation the learner is aware of doi:10.2307/2786027 the need to get instantaneous response, aware of Beyer, H., & Holtzblatt, K. (1998). Contextual how best to get the instant support, aware of how design, defining vustomer-centred systems. MA, best to get social presence, as well as aware of USA: Morgan Kaufmann Publishers, Inc. the best mediating tool to exploit. 2) Location of a learner as context of the activity - learners need Bredo, E. (1994). Cognitivism, situated cogni- awareness of where they are themselves and where tion and Deweyian pragmatism. Philosophy of their peers are located, which is significant in de- Education. Retrieved June, 2010, Online at http:// termining the plausibility of a social interaction. www.ed.uiuc.edu/EPS/PES-Yearbook/94_docs/ 3) Learner’s situation –refers to context awareness BREDO.HTM] dealing with culture and language, the learner’s Byam, N. (1995). The emergence of community in current activity as well as environmental states. computer-mediated communication. In Jones, S. This chapter contributed towards an informed G. (Ed.), Cybersociety. Newbury Park, CA: Sage. insight on how learners share their knowledge in a learning environment – focusing on LMS com- Davenport, T. H., & Prusak, L. (1998). Working munication tools and their appropriateness. The knowledge: How organizations manage what they chapter has shown that awareness of context and know. Boston, MA: Harvard Business Review social presence (synonymous with social aware- Press. ness) is a useful and significant characteristic for Dey, A. K., & Abowd, G. D. (1999). Towards a bet- sharing learning experiences and the consequent ter understanding of context and context-aware- learning support that occur independent of location ness. GVU Technical Report GIT-GVU-99-22. and time. When learners socially interact through College of Computing, Georgia Institute of LMS mediation, they use context awareness to Technology, Atlanta, Georgia. identify an appropriate tool and peer, in order to increase or decrease the knowledge sharing Goffman, E. (1978). Behavior in public places. experience. Thus, in a learning environment, the Notes on the social organization of gatherings social awareness is important since it influences (4th ed.). New York, NY: Free Press. the learning and knowledge decisions and actions, as the learning contexts change.

45 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Gunawardena, C. (1995). Social presence theory Kekwaletswe, R. M., & Ng’ambi, D. (2006). and implications for interaction and collaborative Ubiquitous social presence: Context-awareness learning in computer conferences. International in a mobile learning environment. In Proceedings Journal of Educational Telecommunications, of the IEEE International Conference on Sensor 1(2/3), 147–166. Networks, Ubiquitous, and Trustworthy Comput- ing, Taichung, Taiwan (90-95). Taichung, Taiwan. Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a Laudon, K. C., & Laudon, J. P. (1995). Informa- computer mediated conferencing environment. tion systems: A problem-solving approach (3rd American Journal of Distance Education, 11(3), ed.). Orlando, FL: Dryden Press. 8–26. doi:10.1080/08923649709526970 Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, Holtzblatt, K., & Jones, S. (1994). Contextual T. N. (2003). A context awareness architecture inquiry: A participatory design technique for sys- for facilitating mobile learning. In Proceedings tem design. In Schuler, D., & Namioka, A. (Eds.), of MLEARN 2003 (LSDA), London. Participatory design: Principles and practice. Ng’ambi, D. (2004). Towards a knowledge shar- Englewood Cliffs, NJ: Prentice-Hall. ing framework based on student questions: The Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). case for a dynamic FAQ environment. PhD thesis. The role of ICTs in higher education in South South Africa: University of Cape Town. Africa: One strategy for addressing teaching Ng’ambi, D., & Hardman, J. C. (2004). Towards and learning challenges. Proceedings of Emerge a knowledge-sharing scaffolding environment Online Conference: Learning Landscapes in based on learners’ questions. British Journal Southern Africa, 10-21 July. Retrieved from http:// of Educational Technology, 35(2), 187–196. emerge2006.net. doi:10.1111/j.0007-1013.2004.00380.x Kekwaletswe, R. M. (2006). Social presence and Ngwenyama, O. K., & Lee, A. (1997). Com- context awareness for knowledge transformation munication richness in electronic mail: Critical in an m-learning environment. In Proceedings of social theory and the contextuality of meaning. Emerge Online Conference: Learning Landscapes Management Information Systems Quarterly, in Southern Africa (10-21 July). Retrieved from 21(2), 145–167. doi:10.2307/249417 http://emerge2006.net. Nonaka, I. (1994). A dynamic theory of organiza- Kekwaletswe, R. M. (2007). Knowledge trans- tional knowledge creation. Organization Science, formation in a mobile learning environment: An 5(1), 14–37. doi:10.1287/orsc.5.1.14 inquiry of context and social presence awareness. PhD thesis. South Africa: University of Cape Patriotta, G. (2003). Organizational knowledge Town. in the making: How firms create, use, and institutionalize knowledge. Oxford, UK: Oxford Kekwaletswe, R. M. (2009). Conceptualizing University Press. ubiquitous learning through context-aware wired and wireless Web services. In Proceedings of the Picciano, A. G. (2002). Beyond student percep- IADIS Mobile Learning conference, Barcelona, tions: Issues of interaction, presence and perfor- Spain (25 – 28 February). mance in an online course. Journal of Asynchro- nous Learning Networks, 6(1).

46 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Polanyi, M. (1958). Personal Knowledge. Chi- Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). cago, IL: The University of Chicago press. A follow-up investigation of “teaching presence” in the SUNY Learning Network. Journal of Asyn- Polanyi, M. (1966). The tacit dimension. London, chronous Learning Networks, 7(2), 61–80. UK: Routledge & Kegan Paul. Short, J., Williams, E., & Christie, B. (1976). The Rakotonirainy, A., Loke, S. W., & Fitzpatrick, social psychology of telecommunications. Toronto, G. (2000). Context awareness for the mobile Canada: Wiley. environment. Retrieved Oct 11, 2009 from ftp:// ftp.cc.gatech.edu/pub/gvu/tr/2000/00-18r.ps.Z. Sutherland, R., Facer, K., Furlong, R., & Fur- long, J. (2000). A new environment for educa- Richardson, J. C., & Swan, K. (2003). Examin- tion? The computer in the home. Computers ing social presence in online courses in relation & Education, 34(3-4), 195–212. doi:10.1016/ to students’ perceived learning and satisfaction. S0360-1315(99)00045-7 Journal of Asynchronous Learning Networks, 7(1), 68–88. Swan, K. (2002). Building communities in online courses: The importance of interaction. Education Rourke, L., Anderson, T., Garrison, D. R., & Communication and Information, 2(1), 23–49. Archer, W. (2001). Assessing social presence in doi:10.1080/1463631022000005016 asynchronous text-based computer conferencing. Journal of Distance Education, 14(2). Swan, K. (2003). Developing social presence in online discussions. In Naidu, S. (Ed.), Learning Ryder, M., & Wilson, B. (1997). From center to and teaching with technology: Principles and periphery: Shifting agency in a complex techni- practices (pp. 147–164). London, UK: Kogan cal learning environments. Paper presented at the Page. Meeting of the American Educational Research Association, Chicago, IL. Van Beveren, J. (2002). A model of knowledge acquisition that refocuses knowledge manage- Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). ment. Journal of Knowledge Management, 6(1), There is more to context than location. Computers 18–22. doi:10.1108/13673270210417655 & Graphics, 23, 893–901. doi:10.1016/S0097- 8493(99)00120-X Walther, J. (1994). Interpersonal effects in computer mediated interaction. Com- Scribner, S. (1987). Thinking in action: Some munication Research, 21(4), 460–487. characteristics of practical thought. In Sternberg, doi:10.1177/009365094021004002 R., & Wagner, R. (Eds.), Practical intelligence: Nature and origins of competence in everyday Wiener, M., & Mehrabian, A. (1968). Language world. Cambridge, UK: Cambridge University within language: Immediacy, a channel in verbal Press. communication. New York, NY: Appletoncentury- Croft. Shariq, Z. S. (1999). How does knowledge transform as it is transferred? Speculations on Winograd, T., & Flores, F. (1986). Understanding the possibility of a cognitive theory of knowl- computers and cognition: A new foundation for edgescapes. Journal of Knowledge Management, design. Norwood, NJ: Ablex. 3(4), 243–251. doi:10.1108/13673279910303998

47 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

ADDITIONAL READING Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness in real-time distributed Bush, M. D., & Mott, J. D. (2009). The transfor- groupware: Framework, widgets and evaluation. mation of learning with technology: Learner-cen- In Proceedings of HCI on People and Computers tricity, content and tool malleability, and network XI (pp. 281-298). effects. Educational Technology, 49(2), 3–20. Jonassen, D. H., & Rohrer-Murphy, L. (1999). Delta Initiative (2009). The state of learning man- Activity theory as a framework for designing agement in higher education systems, report for constructivist learning environments. Educational the California State University System. Technology Research and Development, 47(1), Dourish, P., & Bellotti, V. (1992). Awareness and 62–79. doi:10.1007/BF02299477 coordination in shared workspaces. In Proceed- Lane, L. M. (2009). Insidious Pedagogy: How ings of ACM CSCW Conference (pp. 107-114). course management systems impact pedagogy. Dourish, P., & Bly, S. (1992). Portholes: Sup- First Monday, 14(10). porting awareness in a distributed work group. Mlitwa, N., & Van Belle, J. P. (2010). A proposed Proceedings of the CHI ‘92 Conference on Human interpretivist framework to research the adoption Factors in Computing Systems (pp. 541-547). of learning management systems in universities. Engeström, Y. (1987). Learning by expanding: Communications of the IBIMA (Vol. 2010). Re- An activity theoretical approach to developmental trieved from http://www.ibimapublishing.com/ research. Helsinki, Sweden: Orienta-Konsultit. journals/CIBIMA/cibima.html. Engeström, Y. (1999). Activity theory and indi- Morgan, G. (2003). Faculty use of course manage- vidual and social transformation. In Engeström, Y., ment systems (Vol. 2). ECAR Research Bulletin. Miettinen, R., & Punamaki, R. (Eds.), Perspectives Sclater, N. (2008). Web 2.0, personal learning on activity theory. Cambridge, UK: Cambridge environments, and the future of learning manage- University Press. ment systems,” ECAR Research Bulletin (vol. Gardner, C. (2009). A personal cyberinfrastructure. 2008, no. 13). EDUCAUSE Review, 44(5), 58–59. Suchman, L. (1987). Plans and situated actions: Gobbin, R. (1998). Adoption or rejection: Infor- The problem of human–machine communication. mation systems and their cultural fitness. In H. Cambridge, UK: Cambridge University Press. Hasan, E. Gould, and P. Hyland (Eds.), Informa- Taylor, J. (2006). Evaluating mobile learning: tion systems and activity theory: Tools in context What are appropriate methods for evaluating (pp. 109-124). University of Wollongong Press, learning in mobile environments? In M. Sharp- Wollongong, N.S.W. les (ed.), Big issues in mobile learning: Report Gould, S. J. (1987). Time’s arrow: Time’s cycle: of a workshop by the Kaleidoscope Network of Myth and metaphor in the discovery of geographi- Excellence Mobile Learning Initiative: University cal time. Cambridge, MA: Harvard University of Nottingham, UK. Retrieved November 29, Press. 2009, from http://telearn.noe-kaleidoscope.org/ warehouse/Sharples-2006.pdf. Tollmar, K., Sandor, O., & Shömer, A. (1996). Supporting social awareness at work, design and experience. In [New York, NY: ACM Press]. Proceedings of CSCW, 96, 298–307.

48 Knowledge Sharing in a Learning Management System Environment Using Social Awareness

KEY TERMS AND DEFINITIONS dience. If knowledge is not absorbed, it has not been shared. Awareness: An understanding of the activi- Social Awareness: Is synonymous with con- ties of others, which provides a context for your text and social presence awareness. own activity. Social Presence: Defined and understood to Context: Understood as the situation in which be the mediated presence of another learner who a learner or a group of learners find themselves. could provide personalized on-demand social sup- Contextual Inquiry: A field research frame- port for a learning problem as the learner traverses work that depends on conversations with users in varied learning contexts. the context of their work. Ubiquitous Personalized Support: Refers to Knowledge Sharing: Involves two actions; the provision of context sensitive and anywhere, transmission (sending or presenting knowledge anytime support as a learner traverses varied to a potential learner) and absorption by the au- locations.

49 50

Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course

Thomas Connolly University of the West of Scotland, UK

Carole Gould University of the West of Scotland, UK

Gavin Baxter University of the West of Scotland, UK

Tom Hainey University of the West of Scotland, UK

ABSTRACT Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web 2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web 2.0 technologies provided by the chosen LMS.

DOI: 10.4018/978-1-60960-884-2.ch003

INTRODUCTION with a discussion of the findings and provides some recommendations for the implementation There has been considerable research into the of eLearning within vocational training. perceived benefits of eLearning and Learning Management Systems (LMS) within education and it is clear that LMS now play a pivotal PREVIOUS RESEARCH role in the delivery of eLearning within many educational institutions. The research literature eLearning can be defined as “… any use of Web cites many advantages of eLearning, particularly and Internet technologies to create learning ex- the convenience and flexibility offered by the periences” (Horton, 2003, pp. 13). eLearning is (asynchronous) ‘anytime, anywhere, anyplace’ essentially an evolved form of distance education, education. However, much less research has been which Connolly and Stansfield (2007a) describe carried out into the use of educational technolo- through a six-generation model, as depicted in gies and tools within vocational training environ- Figure 1. The first generation (the ‘correspondence ments. The purpose of this chapter is to discuss model’) was provided mostly through paper-based the impact on learning with the introduction of a instruction, characterized by the mass production LMS into a vocational engineering course. This of educational materials. The difficulty with cor- chapter discusses the pilot implementation of Web respondence education has been the infrequent and 2.0 tools within an LMS and aims to answer a inefficient form of communication between the number of questions: instructor and the learners. Further, it was difficult to arrange for peer interaction in correspondence (i) Can technology supplement the hands-on based distance education. The second generation nature of vocational training? (the ‘multimedia model’) was provided through (ii) Can the use of wikis and forums aid voca- integrated multimedia such as delivering courses tional training? via television or introducing material like audio (iii) Can the pilot be considered a success? and video tapes, computer-based learning (CBL) in addition to printed material. The third generation Much of the research in this area has been was provided through two-way communications mainly anecdotal and has not considered the media such as audio/video-conferencing and different nature of vocational training with most broadcast technology. The fourth generation of of the research focusing on the traditional edu- distance education (the first generation of eL- cational environment. This chapter utilises both earning) is defined as mainly passive use of the qualitative and quantitative surveys on the views Internet, consisting primarily of conversion of of trainees and instructors and aims to identify the course material to an online format, low-fidelity areas within the training programme where the streamed audio/video, and basic mentoring using LMS could be utilised further to aid learning. It email. However, the educational philosophy still also considers the areas where the use of the LMS belongs to the pre-Internet era. The fifth genera- has not been as successful as anticipated and the tion of distance education (the second generation reasons for this. The next section of this chapter of eLearning) uses more advanced technologies discusses the literature on LMS, the use of Web consisting of high-bandwidth access, rich stream- 2.0 technologies within education, and ePortfolios. ing media, online assessment (eAssessment) and The subsequent sections introduce the research LMS that provide access to course material, com- rationale, the case study and an empirical analysis munication facilities, and learner services. The of the pilot implementation. The chapter concludes sixth generation of distance education (the third

51 Learning 2.0

generation of eLearning) is a more collaborative We are also now starting to see the development learning environment based much more on the of mLearning (mobile learning) through devices constructivist epistemology, promoting reflective like PDAs (Personal Digital Assistants), mobile practice through tools like ePortfolios, Web 2.0 phones and smartphones, and tablet devices. technologies such as blogs and wikis, online com- For second and third generation eLearning, munities, and using interactive technologies such LMS support new approaches for people to learn as online visualizations, games, and simulations. and assist with the delivery but also with the way

Figure 1. Models of distance education (adapted from Connolly & Stansfield, 2007a)

52 Learning 2.0

in which information is presented leading to ac- used to support an institution’s entire set of teach- quisition of new knowledge (Holmes & Gardner, ing and learning courses. Although LMS are used 2006). eLearning and the use of LMS are now an extensively within educational institutions glob- integral part of most educational institutions with ally, their use is relatively low within vocational educational technologies witnessing exceptional training companies. One reason for this may be levels of growth in recent years. To support the that vocational trainers do not appreciate that growth of LMS, many schools, colleges and some aspects of vocational training may lend itself universities have invested heavily in up-to-date well to online delivery. There are many different technology. Also referred to as Virtual Learning LMS products available, some at considerable Environments (VLEs), ‘learning platforms’, cost and others available as open source. Regard- ‘distributed learning systems’, ‘course manage- less of which product is chosen, most LMS con- ment systems’ and ‘instructional management tain similar functionalities as shown in Table 1. systems’, LMS combine a range of course/subject As with most things, eLearning has advan- management and pedagogical tools to provide a tages and disadvantages. The research literature means of designing, building and delivering online cites many advantages of eLearning, particularly learning. LMS are scalable systems that can be the convenience and flexibility offered by the

Table 1. Comparison of LMS functionality

Features/Tools LMS Blackboard ProSites Moodle Learnwise Frog E-portfolio N Y Y Y File up-load Y Y Y Y Notice/bulletin board Y Y Y Y Course outlines Y Y Y Y Assignments Y Y Y Y Assessments Y Y Y Y Multi-media resources Y Y Optional extra Y Evidence gathering Y Y Y Y Calendar Y Y Y Y Administration tools Y Y Y Synchronous collaboration tools Y Y Optional extra N (video conferencing) Forum/discussion board Y Y Y Y Email Y (Internal) Y (Internal) Y (Internal) Y (Internal) External links Y Y Y Y Student home page Y Y Y Y Real-time chat N Y Y Y Quiz design Y Y Y Y Costs £6,655 per annum Open source Ranges from £3,300 per Bespoke system - £26,000 (200 users/licences) Additional costs if annum for 1,000 users to Yearly support - £4,500 £3,152 per annum hosting required £27,500 per annum for Standard package - £22,500 (100 additional users/ 50,000 users Yearly support - £4,500 licences

53 Learning 2.0

(asynchronous) ‘anytime, anywhere, anyplace’ Perhaps one of the most damaging criticisms is education (McDonald, 2002), which gives learn- that some eLearning simply replicates the social ers time for research, internal reflection, and organization of traditional education and training ‘collective thinking’ (Garrison, 1997). Moreover, and that the potential benefits of eLearning - of the text-based nature of eLearning normally re- personalized and accessible learning experiences quires written communication from the learner, - are missed. Taking this on board, this is one of which along with reflection, encourage higher the reasons this research is of particular impor- level learning such as analysis, synthesis, and tance. There is a high chance that the traditional evaluation, and encourage clearer and more pre- nature of the organisation under investigation cise thinking (Jonassen, 1996). In addition, eL- may simply deploy an LMS but not utilize it to earning courses also have the capability to present its full potential. multiple representations of a concept, which allows learners to store and retrieve information Learner Expectations and Web 2.0 more effectively (Kozma, 1987). It is also argued that increased social distance provides a number Web 2.0 is a term often associated with “… the of distinct advantages to online conferences (syn- social use of the Web which allow[s] people to chronous or asynchronous). In written communi- collaborate, to get actively involved in creating cations anonymity of characteristics such as content, to generate knowledge and to share gender, race, age, or social status can be preserved, information online” (Grosseck, 2009, pp. 478). which can reduce the feeling of discrimination Web 2.0 applications, such as blogs and wikis, are and provide equality of social interaction among being introduced into LMS such as Blackboard participants. In turn, this can permit the expression and Moodle, providing students with increased of emotion and promote discussion that normally flexibility in terms of how they communicate with would be inhibited (Gunawardena, 1993). fellow students and gain feedback from peers. In eLearning is not without its disadvantages; for industry the use of Web 2.0 applications are being example (Connolly & Stansfield, 2007b): gradually introduced as organisations have begun to realise their potential for the purposes of learn- • Costs may initially exceed more traditional ing and information sharing. The way in which methods; education and training institutions and industrial • More responsibility is placed on the organisations facilitate learning and information learner who has to be self-disciplined and sharing is being determined by the expectations motivated; and prior learning experiences of the individuals • Some learners lack access to a PC/Internet within them. Within most learning or working or have difficulty with the technology; environments today, there is a sense of expectancy • Increased workload for both students and that innovative learning and communication chan- faculty; nels should already be in place to accommodate • Non-involvement in the virtual commu- the diverse ways in which individuals learn. nity may lead to feelings of loneliness, low The aspects of institutional and organisational self-esteem, isolation, and low motiva- competitiveness that are closely related to the tion to learn, which in turn can lead to low concept of the ‘knowledge economy’ means that achievement and dropout; institutions promoting learning via technological • Dropout rates tend to be higher in eLearn- means now have an interest in developing and ing courses than in traditional face-to-face running learning initiatives with the minimum courses, often 10 to 20 percentage points of effort (DeRouin, Fritzsche, & Salas, 2005). higher. These particular factors are having an impact

54 Learning 2.0

upon the format and delivery of eLearning initia- individual learning. The use of social software and tives within educational and industrial contexts. Web 2.0 technologies have given rise to the term Web 2.0 technologies are attractive within these ‘Learning 2.0’, sometimes referred to as eLearning contexts as they allow greater student indepen- 2.0, which broadly summarizes all opportunities dence and autonomy, increased collaboration and arising from the use of social media for learning, an increase in pedagogic efficiency (Franklin & education or training. Harmelem, 2007, pp. 1). For this reason, the use of Web 2.0 technologies could be an invaluable The Pedagogy of Learning 2.0 asset to those institutions delivering ‘on-the-job’ training, especially where an element of group The interactive and collaborative nature of social work is undertaken. It also facilitates knowledge software makes it highly suited towards sustaining sharing and promotes learning in an industrial and facilitating what are known as communities context, as well as taking account of pedagogical of practice (CoPs) or “groups of people informally issues. This was important to our study given that bound together by shared expertise and passion for one part of the vocational training to be delivered a joint enterprise” (Wenger & Snyder, 2000, pp. was a group-based project. 139). In conjunction with the concept of CoPs, the learning theory of social constructivism appears Social Software and Learning 2.0 to complement and accommodate the principles surrounding the use and learning benefits associ- It could be argued that the concept of eLearning ated with Learning 2.0. The constructivist view is being enhanced by the rapid development of of learning adopts the stance that learners do not ‘social software’. McKelvie, Dotsika, and Patrick learn individually from one another and stresses (2007) state that “social software is a community the relevance regarding the socio-cultural context driven technology which facilitates interaction of learning. Predominately, social constructivism and collaboration and depends largely on social contends that knowledge is formulated through the convention.” Though social software can be used processes of social interaction and collaborative on an individual basis it is predominately con- learning. Though the concept of social software cerned with the notions of open and collective can, in theory, support a wide range of learning communication, dialogue and the ability to liaise approaches, it is inherently applicable towards with individuals collectively. Social software is the learning theories of social constructivism and having an effect upon eLearning delivery within CoPs. It has been generally regarded that one of the education and industry as it has altered the way salient aspects of any CoP is its ability to construct in which learning is taught and learnt. The use and store collective knowledge in what has been of social software allows the learner to generate referred to as a ‘shared repertoire of communal knowledge and share their learning experiences resources’ (Wenger, 2000). Additionally, CoPs are on a collective level as well as allowing users to most usually distinctly defined by the concepts openly reflect upon what they have learnt. eL- of collective understanding, mutual engagement earning distinguishes itself from social software and shared repertoire (Wenger, 2000). It could be as it is predominately associated with electronic argued that for this reason, the use of Learning 2.0 instruction and is better suited for education could be an invaluable asset to organisations. The and training purposes. Web 2.0 is transforming examples given below demonstrate the growth of the way in which people learn as the learning is those technologies and tools that facilitate knowl- predominately social and self-directed in nature edge sharing in the wider community. whereas eLearning is normally associated with

55 Learning 2.0

ePortfolios and companies (Curyer et al., 2007). This could also help to fulfil the concept of an ePortfolio being As well as the Learning 2.0 technologies, many utilised throughout lifelong learning. Richardson educational institutions and organisations are and Ward (2005) carried out an in-depth study utilising ePortfolios, the electronic version of the of 12 different ePortfolios and found, amongst paper portfolio. As with LMS, the use of ePort- other things, that no two systems were identical folios is also growing in popularity, particularly or offered the same range of functions. However, within vocational training. A portfolio can be it should be considered that an ePortfolio tends to defined as “a collection of documents relating to be chosen on a ‘fit-for-purpose’ basis and often a learner’s progress, development, and achieve- vendors may customise their ePortfolios to suit ments”; an ePortfolio simply indicates that some a particular customer, as is the experience of the or all of the evidence is collected in digital form authors. (Beetham, 2009). EDUCAUSE (2005) defines an Most ePortfolios are driven by the learner; that ePortfolio as “a valuable online tool that learners, is, the learner is responsible for the maintenance faculty, and institutions can use to collect, store, of the ePortfolio and decides who has access to update, and share information. E-Portfolios allow its contents but in some environments, as in the students to reflect on their learning, communica- study under investigation, this may not be desir- tion with instructors, document credentials, and able, as some aspects of vocational training need provide potential employers with examples of their to be driven by the instructor, not the learner. work”. ePortfolios are well suited to education, Internal examiners and external verifiers often vocational training and working environments need access to trainee assessment material and as they capture the concept of lifelong learning that assessment material cannot be amended once and support individuals as they progress through verified. In our study, the instructors required school, college, Higher Education, training and a measure of customisation of the ePortfolio, employment (Richardson & Ward, 2005). Learn- which included allowing multiple assessors and ers gather learning evidence and define these the instructors themselves having overall control evidences through a self-reflection process. They of the ePortfolio. attribute their competences to learning products or outcomes and reflect on how they acquired those ePortfolio Use in Higher/ competences. From a pedagogical perspective this Further Education process helps learners to better understand how they learn and helps them to become self-directed According to JISC (2008) ePortfolios represent learners (Berlanga et al., 2008). the latest in a line of technology-based innovations ePortfolios can be classified into various types that are becoming an integral part of the learning – assessment, presentation, learning, personal landscape in Higher and Further Education. Stu- development and shared ownership (Curyer et al., dent ePortfolios developed out of faculty-assigned, 2007) but in reality most are a combination. If a print-based student portfolios as far back as the standard approach was adopted for ePortfolios, mid-80s. They were typically found in art-related institutions and organisations could share and disciplines or those that consisted of substantial exchange ePortfolio data, which could lead to the written components, such as English studies, and streamlining of the processes connected to prior gained greater importance in education during the learning, with student transitions through courses, mid-90s. Student ePortfolios are commonly found and with training that involves either sequential or in college programmes where teachers use them parallel movement through multiple institutions to provide evidence of competence. This includes

56 Learning 2.0

communications, maths, business, nursing and and some resistance to change was also noted. engineering to record students’ learning experi- The literature demonstrates that ePortfolios offer ences and skill set (Lorenzo & Itellson, 2005). a valid way to assess and ensure completion of The Open University in the UK has been using an individual’s training, and while usage is not ePortfolios as an assessment tool in online courses pervasive, uptake is growing. for many years (Mason, Pegler, & Weller, 2004) and the University of Washington developed an ePortfolio in 2001 to allow students to record METHODOLOGY their entire educational learning experiences in an organised and integrated manner. A survey carried The aim of this study was to determine whether out by Strivens (2007) found that of those who eLearning could be used successfully within a participated in the survey, 20 institutions (54% vocational training environment for engineering of those with an ePortfolio) commented that the students. More specifically, the objectives of the ePortfolio was available to all students across the project were (i) to select suitable LMS and ePort- institution. The increase in the use of ePortfo- folio systems, (ii) develop some of the provision lios in Higher Education is further supported by in an online format, and (iii) evaluate the success Beetham (2009) who states that work is currently of this provision. This study started in late 2008 being carried out to integrate LMS, student record and the initial part of the study was completed in systems and ePortfolio tools to provide formative May 2010. Empirical data was obtained from a feedback. It appears that educational institutions cohort of students going through their Induction have embraced the concept of ePortfolios and programme, which was the first course developed appreciate their value. in an online format, and from their final group project before completing their training. In ad- ePortfolio Use in Vocational Training dition, the instructors were surveyed in order to ascertain their views of the LMS and the use of Portfolios (paper-based) have been utilised for the wiki and forum for the group project. The in- many years in vocational training; for example, structors’ survey also included questions relating Austria has been using portfolios in teacher to the use of the LMS as a learning tool. training for the past 12 years and covers topics For this study, the research philosophy of such as supervision and professional upgrade in interpretivism was adopted. Interpretivism is an vocational education and is regarded as a working epistemology that advocates that the researcher portfolio, as examination for teachers is impracti- must understand the differences between humans cal (Dorninger & Schrack, 2007). ePortfolios have in their social roles as actors. This philosophy been endorsed by some of the major vocational places emphasis on conducting research among examiners such as City & Guilds in the UK. people as opposed to objects, such as cars and City & Guilds (2009) undertook a survey of 95 houses. Interpretivism stems from the intellectual colleges and training providers and found that, tradition of phenomenology, which is concerned although cost savings were significant, the main with the ways in which humans make sense of the advantage was the reduction in time taken for world around them (Saunders, Lewis & Thornhill, candidates to complete their qualification when 2007). According to Whisker (2001), phenomenol- using an ePortfolio. However, although feedback ogy encourages both quantitative and qualitative was positive, only 16% of those surveyed used methods. However, it should be considered that ePortfolios. Some centres expressed fear over quantitative research is more often associated with technical glitches, which could result in lost work, positivism. McBride and Schostak (1994) state:

57 Learning 2.0

“Where quantitative forms of research, employing Development but apprentice engineering training questionnaires and sampling procedures attempt is EKGTA’s core business and for this reason the to eradicate the individual, the particular and main focus is directed towards that training. the subjective, qualitative research gives special A Modern Apprenticeship Programme may attention to the subjective side of life...qualitative involve: researchers are more likely to ask how it feels... they focus upon the social construction of such • A period of training in an approved train- things...”. As such, this study is therefore adopting ing centre (off the job). a mixed methodology with an Action Research • Completion of a Level 2 Vocational and within case study approach as the authors are Qualification. actively involved in the project. Gerring (2007, • Attainment of core skills to intermediate pp. 20) defines case study as “the intensive study one level (minimum). of a single case where the purpose of that study- • Completion of a National Certificate (day at least in part- to shed light on a larger class of release at FE College). cases”. It is hoped that at the termination of the • Completion of a Level 3 Vocational project it will be possible to develop a framework Qualification in company. that will aid practitioners in the successful implementation of an LMS in vocational training, as Level 2 competence involves the application well as promote a rich area for future research. of knowledge and skills in a significant range of The next section will outline the company under varied work activities, performed in a variety of investigation, the type of training delivered and contexts. Some of the activities are complex or how the chosen technologies were selected. non-routine, and there is some individual responsibility and autonomy. Collaboration with others Case Study: Vocational Training may often be a requirement On the other hand, Level 3 competence involves the application of This project came about when University of the knowledge and skills in a broad range of varied West of Scotland entered into a two-year partner- work activities performed in a wide variety of ship programme with a local training organisation, contexts, most of which are complex and non- East Kilbride & District Engineering Group Train- routine. There is considerable responsibility and ing Association (EKGTA), to explore how much autonomy, and control or guidance of others is of a Modern Apprenticeship programme could often required. There are six instructors who be delivered online. EKGTA is an employer-led deliver the practical element of the qualification training provider for the engineering industry with the academic element delivered onsite by a with charitable status. Established in 1966 and local Further Education college. Once the trainees recognised as one of the premier training groups return to their company, advisors visit trainees on throughout Scotland, it aims to serve the needs of average every 12 weeks and oversee the continu- the employer, whilst ensuring candidates have the ation of the training. opportunity to develop the knowledge and skills Management appreciate the advancements in necessary in employment. The Association spe- technology that could potentially support and en- cialises in training Modern Apprentices at craft and hance the training programmes offered by the As- technician levels, and in basic engineering skills sociation, however, as the project has progressed, training to national standards. EKGTA provides it has become obvious that this sentiment is not training in other disciplines, such as, Health & echoed by all staff. Before the project, technology Safety, Computer Based Training, Professional within EKGTA was used primarily for the record-

58 Learning 2.0

ing of completed assignments (ePortfolio) and shortlisted LMS systems and it would be necessary the storing of lecture material, and was therefore to select a separate ePortfolio system to the LMS, used far below its potential. In the early stages of although it was hoped the two could be integrated the project the pilot of the LMS was to be rolled together in some way. After extensive analysis out within the electrical area of the workshop, and consultation with management and staff, but on further discussion it was decided that this Moodle was chosen as the LMS as it was highly would not be an inclusive approach. The decision modular and provided the same features as the was taken to implement the LMS throughout the commercial systems but at no cost to the company. entire organisation in two stages: the Induction The other LMS investigated were more suited to programme and the group project, thus including educational institutions with large numbers of all instructors. students. For this project, the maximum student uptake in one year would be limited to 120 and Selecting the LMS so the need for a large commercial product could not be justified. In addition, Moodle has a large In discussion with EKGTA, a number of fea- and growing community with almost 40 million tures were identified that the chosen LMS had to users and over 50,000 sites worldwide at the time support. Existing students were also consulted. of writing (Moodle, 2010). The next stage of the Two informal group forums were set up with ten project was to identify a suitable ePortfolio system. participants in each. This helped to identify those features that students believed would enhance Selecting the ePortfolio learning. Almost all students agreed that 24x7 access to learning materials would be advanta- Again, a short list of potential ePortfolio systems geous, more use of multimedia, such as video was drawn up based on instructors’ requirements, demonstrations and interactive quizzes, with im- which included Learning Assistant, One File, mediate feedback to reinforce learning. To identify Pebble Pad and Mahara, which is open source a suitable LMS, it was first necessary to carry and an add-on module for Moodle. EKGTA pre- out desk research on a number of different LMS viously used two ePortfolios, one at Level 2 and to identify the functionalities that each one sup- one at Level 3, which were not integrated in any ported and to match these against the company’s way. The ePortfolio at Level 2 gives on-campus and the students’ criteria. A short list of potential students access to all lecture material, standard systems was drawn up that included Blackboard assessment documentation, and in some instances ProSites, Learnwise, Frog and Moodle, Moodle multimedia. At Level 3 advisors use the Modern being the only open source option. The outcome Apprenticeship online ePortfolio system. At this of the research demonstrated that there was very level, students are off-campus, therefore regular little difference in the functionalities of each, as remote communication between advisors and shown in Table 1. students needs to be undertaken. The Modern The ability to automatically map student Apprenticeship online ePortfolio did not fulfil evidence of their work to individual components some of the criteria required by the instructors of assessment was regarded by the instructors and was no longer being fully supported and so as the most important function that needed to be was excluded from further consideration. supported by the ePortfolio component as this After in-house evaluation, it was concluded was their most time intensive activity, however, that Learning Assistant best suited the require- after much research it became apparent that this ments. Learning Assistant is marketed as an function was not readily available with any of the ePortfolio and eAssessment solution for training

59 Learning 2.0

centres that deliver vocational qualifications. It it was decided that this would not be an inclusive has been designed specifically to meet the needs approach. The decision was taken to implement of the vocational training environment. Learning Moodle throughout the entire workshop, thus Assistant, as well as being fully integrated from adopting an inclusive approach. All trainees go Level 1 through to Level 4, is a fully supported through a week long induction programme - a system with regular updates to meet the chang- general induction and a workshop induction. A ing needs of the students and any changes to the course was developed and populated with two engineering qualification. In addition, the Learn- units: general induction and workshop inducing Assistant vendor could customise the system tion. All lecture material and presentations were to support multiple assessors and facilitate the uploaded to Moodle and self-assessment quizzes automatic mapping of student evidence of their designed to reinforce learning. Learning Assistant work to individual components of assessment. was customised by the vendor and installed for students to use and a single sign-on was devel- Identifying Priority Features oped between Moodle and Learning Assistant so students only had to log in once to access both After researching a variety of LMS and ePortfolio systems. Figure 2 shows an example page in the systems to identify their generic features and the Moodle system and Figure 3shows the reporting system(s) that is more suited to the vocational system within the ePortfolio. training environment, it was then necessary to survey the six workshop instructors and one training advisor to identify those features that would FINDINGS AND DISCUSSION be regarded as a priority for implementation. This gave an indication of where the end-users see In this section, we discuss the findings from the technology enhancing learning and to help in the first year of using the LMS for training the engi- delivery of training. Table 2 provides details of this neering students. We start with an evaluation of survey. The lower the number assigned, the higher the results from the Induction programme followed importance assigned. The data demonstrates that by the group project. the top five priority features are: Induction Programme Results • ability to automatically map student evidence of their work to individual compo- A five-day Induction programme was run for a nents of assessment; cohort of students in Autumn 2009. Demographic • a suitable ePortfolio system; information was collected at the start of Induction • evidence gathering; and an evaluation questionnaire was distributed • uploading of assignments; at the end. • a file upload capability. Demographic Questionnaire These features were subsequently prioritised for implementation. 26 participants completed the demographic questionnaire. 25 (96%) of the respondents were male Implementation and 1 (4%) of respondents were female. The mean age of participants was 17 (SD = 0.748) with a In the early stages of the project the pilot of range from 16 to 19. 17 (65%) of participants Moodle was to be rolled out within the electrical started an apprenticeship at EKGTA when leav- area of the workshop, but on further discussion ing school, 9 (35%) of participants did not. The

60 Learning 2.0

Table 2. Survey results for priority features

ePortfolio Student home page Course outlines Assignments Map evidence to Multimedia assessment 1 6 4 7 2 2 4 13 16 3 2 14 1 13 8 6 2 10 6 16 4 5 3 1 5 12 13 8 1 7 1 12 3 5 2 4 4 10 5 1 2 8 22 82 53 35 14 46

File up-load Evidence gathering Notice board External links Quiz design Forum 3 3 5 8 14 15 7 1 12 11 15 8 9 3 7 12 11 14 2 7 12 8 13 14 4 2 14 11 16 9 6 11 14 7 8 13 7 3 9 12 13 11 38 30 73 69 90 84

Real time chat Calendar Email Admin tools 16 2 6 3 9 10 5 6 15 16 4 5 15 11 10 9 15 10 3 6 15 9 16 10 16 14 15 6 101 72 59 45 majority of the participants (18, 69%) left school in that were undertaken were primarily in the areas 2009, 6 (23%) left school in 2008 and 2 (8%) left of construction (joinery, roofing, brick-laying, in 2007. The majority of the participants (23, 88%) etc) and creative industries (art, fashion design, indicated that they had no other full-time employ- media, etc). These courses were undertaken at ment in the past while 3 (12%) did. 1 participant intermediate, NC and HNC level. Table 3 shows indicated that they worked in the hospitality sector. the standard grade qualifications and Table 4 the 19 (73%) of participants indicated that they had intermediate/higher qualifications achieved by not attended college prior to attending the training the participants. course, whereas 6 (23%) of participants indicated that they had attended college first. The courses

61 Learning 2.0

Figure 2. Screenshot of Moodle page

Table 3. Standard grade qualifications achieved Table 4. Intermediate or higher qualifications achieved Standard Grade Qualification Number with qualification Mathematics 21 (81%) Higher Grade Qualification Number with qualification English 24 (92%) Mathematics 20 (77%) Physics 17 (65%) English 15 (58%) Craft and Design 13 (50%) Physics 11 (42%) French 16 (61.5%) Craft and Design 5 (19%) Biology 6 (23%) French 3 (12%) Geography 13 (50%) Biology 1 (4%) Chemistry 10 (38%) Geography 5 (19%) Modern Studies 13 (50%) History 5 (19%) German 2 (8%) None 3 (12%) Spanish 1 (4%) Modern Studies 2 (8%) Other 20 (77%) Others 14 (54%)

Evaluation Questionnaire level and 4 being a high level) for each section of the Induction course, namely Day 1 General Students were asked to complete a questionnaire Introduction, Workplace Environment, Tools & at the end of the Induction programme. 41 par- Maintenance, Business Improvement Techniques, ticipants completed the evaluation questionnaire. ICT, COSHH & Hand Care, Fire Prevention, Risk Participants were asked to rate their level of interest Assessment, Electrical Safety, Measurements and in the training on a scale of 1 to 4 (1 being a low Materials. The overall results were generally posi-

62 Learning 2.0

Figure 3. Screenshot of ePortfolio page

tive indicating that the training was sufficiently of the areas. The area rated as having the highest interesting for the participants. The area of the level of knowledge improvement was COSHH training that had the highest level of interest was (Control of Substances Hazardous to Health) & Electrical Safety (Mean = 3.41, SD = 0.59). The Hand Care (Mean = 3.51, SD = 0.60). The area areas of training that received the lowest levels of rated as having the least level of knowledge im- interest were Business Improvement Techniques provement was the General Introduction (Mean = (Mean = 2.93, SD = 0.65) and the General Intro- 3.17, SD = 0.68). Participants were also asked to duction on the first day (Mean = 2.93, SD = 0.80). rate how appropriate they considered the duration Participants were also asked to rate whether they of the activities to be. Overall the results were not found the training sufficiently challenging (1 = particularly positive. The most appropriate rated lowest, 4 = highest). Electrical Safety was rated duration was for Electrical Safety (Mean = 3.15, as the most challenging (Mean = 3.29, SD = 0.60) SD = 0.76) and the least appropriate rated duration while the General Introduction on the first day was for the General Introduction (Mean 2.75, SD was rated as the least challenging (Mean = 2.75, = 0.74). Participant ratings for the helpfulness of SD = 0.71). The overall approachability of the teaching aids were generally positive. The highest instructors was rated highly by the participants in rating for teaching aids was in the area of Electrical all areas. The average rating for approachability Safety (Mean = 3.49, SD = 0.51) and the lowest across all areas was Mean = 3.46, SD = 0.86 with rating was in the Helpful Work Place Environment a range of 3.32 to 3.59 out of 4. teaching area (Mean = 3.22, SD = 0.52). Partici- Participants were asked to rate how much they pants were also asked to rate if Moodle helped believed that their knowledge improved in each in each of the teaching areas. Table 5 shows the

63 Learning 2.0

rankings of perceived help Moodle provided in Participants also provide qualitative points each of the teaching categories. for action and improvement. These primarily Overall, the results of the evaluation question- included: more interaction, more participation naire were generally positive. Electrical Safety and the reduction of the duration of the General consistently achieved the highest ratings for Introduction. level of interest, challenge, appropriate duration and perceived help provided by Moodle. The General Introduction on the first day received the GROUP PROJECT highest amount of criticism primarily because of its duration as it was considered by participants The group project was designed to develop and to be too long and lacked participation and inter- promote team-working skills that are applicable action. Participants provided some of the follow- to real-life working in the engineering industry ing qualitative comments regarding the duration and is the last assessment the students take before of the General Introduction: going out into industry. The overall objective of the project is for team members to build a truck Day 1 was not that interesting because there was within a nominal budget. Each member of the not a lot of participation - it was all listening. I team is assigned a role, for example, project think the 3 day introduction as a whole could be manager, chairman, secretary or quality control. delivered as a participation lesson. To trial Moodle in the group project, it was decided to introduce a wiki for formal recording of The first day was not interesting enough. We were the project, a forum to facilitate communication just thrown information and didn’t have enough when not all members of the team were present time to absorb the information. and an assignment folder to allow candidates to upload all documentation for assessment. Two tests Duration - induction was too long and it could were designed: a pre-test to help identify current involve the trainees more to make it more exciting. knowledge and a post-test to help identify new knowledge and students’ views of the wiki and forum. A total of 43 candidates participated in the

Table 5. Ratings of help provided by Moodle in each teaching category

Teaching Area Rank Mean Standard Deviation Risk Assessment 1st 3.56 0.50 Electrical Safety 1st 3.56 0.55 Materials 1st 3.56 0.55 Measurements 2nd 3.51 0.60 Work Place Environment 3rd 3.49 0.51 Tools and Maintenance 3rd 3.49 0.55 Fire Prevention 3rd 3.49 0.55 COSHH and Hand Care 4th 3.46 0.50 Business Improvement Techniques 5th 3.44 0.59 ICT 6th 3.41 0.59 Day 1General Introduction 7th 3.33 0.58

64 Learning 2.0

group project. The candidates were divided into If someone comes up with an idea then they can 6 teams with 7-8 students per team. The findings post it at anytime even when team members are are discussed below. not there.

Pre-Project Test You can have access when you are off and record progress. 33 participants completed the pre-test questionnaire. All of the respondents were male. The mean Those who indicated that a wiki would not age of participants was 17.18 (SD = 0.73) with a encourage teamwork gave some of the follow- range from 16 to 19.8. The majority of participants ing reasons: (30, 91%) indicated that they had been involved in group work before, while 3 (9%) had not. The Better to discuss the project in person so that you majority (27, 82%) of the participants who had are able to give them a clearer picture of what participated in group work before indicated that is done. they took part in the group work at school, 10 (30%) at college and 3 (9%) at work. The whole team would have to use it to benefit None of the participants had ever used a wiki and they were not. before. The majority of the participants (24, 73%) described a wiki as an ‘an online collaboration 22 (67%) of participants believed that the use tool with built in tracking’, 7 (21%) described of a forum would help open up communication a wiki as ‘a social networking tool’ and 2 (6%) between members while 11 (33%) believed this described it as ‘an online instant messaging tool’. not be the case. Those indicating that the forum The majority of participants (21, 64%) indicated would open up communications gave the some that they had used an online forum before while of the following reasons: 12 (35%) had not. 18 (55%) participants had used an online forum for social use, 5 (15%) had used If one team member is not in one day he/she can an online forum for a school project, 2 (6%) had then see what his/her team has done that day. used one for a college project and as part of a work project. 12 (36%) of participants believed It can get everyone involved. that an online forum was ‘a social networking tool’, 14 (42%) believed that it was ‘an online 17 (52%) of the participants were not aware chat room’ and 4 (12%) believed that it was ‘an of any project management tools, while others online collaboration tool with built in tracking’. listed Excel, Word, PowerPoint, efacts, wiki, 23 (70%) of participants believed that the use of graphs, flip charts, CAD, Gantt charts and Moodle a wiki 24x7 would encourage teamwork, while as project management tools. The majority (22, 10 (30%) believed this not to be the case. Those 67%) indicated that they had not used any project who indicated that a wiki would encourage team management tools in the past while 11 (33%) had. work gave some of the following reasons: The majority of participants (21, 64%) indicated that they thought a Project Route Map to be ‘a Allows you to keep every member of the team up tool that helps to identify all areas of the project to date. that need to be considered’, 11 (33%) believed it to be ‘a tool that is used to help plan the project and then consulted on a regular basis’ and 1 (3%) believed it to be ‘a tool that is only used to plan

65 Learning 2.0

the project but then filed away’. Participants were tool’, 2 (8%) described it as ‘an online chat room’ asked to list the project resources that they were and ‘an online instant messaging tool’. 10 (38%) aware of, 12 (36%) indicated that they were not identified a forum as ‘an online collaboration aware of any, 3 (9%) indicated that they were tool with built in tracking’, 7 (27%) identified it aware of Moodle as a project resource; 4 (12%) of as ‘a social networking tool’, and ‘an online chat participants also mentioned the Internet and books room’. 2 (8%) described it as ‘an instant messag- as a project management resource. 18 (55%) of ing tool’. The majority of participants 15 (58%) participants believed a budget to be ‘a set amount said that they did not think that a wiki promoted of money that can be spent on the project from teamwork, while 10 (38%) believed that it would start to finish’, 14 (42%) of participants believed promote teamwork. Those who indicated that the that a budget was ‘an amount of money that should wiki would promote teamwork gave some of the cover all cost of the project from start to finish’ following reasons: and 1 (3%) believed that is was ‘the amount of money that is set for the first stage of the project People could catch up on work they may have but is ‘topped up’ as the project progresses’. 14 missed. (42%) of participants identified the information pyramid as something that ‘allows the presenter to Helped each other with the difficulties. identify between the different types of information ensuring that the most important information is Those who indicated that the wiki would not not omitted’, 10 (30%) as something that ‘helps promote teamwork gave some of the following the presenter to divide the information into equal reasons: parts’ and 9 (27%) as something that ‘helps the presenter to ensure that the correct information Took too long to complete and was repetitive. is conveyed during a presentation’. We didn’t use it much. Post-Project Test Participants were asked how much they agreed 26 participants completed the post-test. Partici- with the following statement ‘if the wiki was a pants were asked how much they agreed with the compulsory element of the group project all team following statement: ‘everyone participated in members would have used it’. To produce a mean the group project’. To get an overall rating of the score of the expected participation level, strongly perception of overall participation, strongly agree agree was recoded as 5, agree as 4, neither agree nor was recorded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating was 2.76 (SD = 1.03) with as 1. The mean rating for perceived participation a range of 1 to 4. This means that approximately was 2.69 (SD = 1.19) with a range from 1 to 5. 50% of participants would use the wiki if it were This indicates that the level of perceived participa- a compulsory part of the group project. 13 (50%) tion was not particularly high with the majority of participants believed that the forum helped with of participants either disagreeing 11 (42%) or communication and 13 (50%) did not. Those who strongly disagreeing 4 (15%) that everyone was believed that the forum helped with communica- participating in the group activity. tion gave some of the following reasons: 13 (50%) of participants described a wiki as ‘an online collaboration tool with built in track- All people can communicate if not present. ing’, 8 (31%) described it as ‘a social networking

66 Learning 2.0

People could see things they missed. identified it as ‘a set amount of money that can be spent on the project from start to finish’, and Those who believed that the forum did not 1 participant (4%) identified it as ‘the amount of help with communication gave some of the fol- money that is allocated to material for the project’. lowing reasons: 6 (23%) of the participants identified the information pyramid as ‘a tool that helps the presenter to We didn’t use it much. divide the information into equal parts’, 9 (35%) as ‘a tool that helps the presenter to ensure that Easier to talk. the correct information is presented during the presentation’ and 11 (42%) as ‘a tool that helps Participants were asked to rate their level of the presenter identify between the different types agreement of this statement: ‘If the forum was a of information ensuring that the most important compulsory element of the project, all team mem- information is not omitted’. bers would have used it more for communicating with those team members who are not present on Comparison of Pre and Post-Tests a full-time basis’. To produce a mean score of the expected participation level, strongly agree was 21 participants completed both the pre- and the recoded as 5, agree as 4, neither agree nor disagree post-tests. A Wilcoxon Signs ranked test indicated as 3, disagree as 2 and strongly disagree as 1. The that the increase in knowledge was not significant mean score rating of expected participation was with regards to wikis (Z = -.890, p = 0.03) and 3.23 (SD = 0.95) with a range from 1 to 5. This forums (Z = -.816, p = 0.414). However, when indicates that the forum was more popular than combined together, a Wilcoxon Signs ranked test the wiki in terms of expected participation. indicated that there was a significant decrease in Participants were asked to list all of the project forum and wiki knowledge (Z = -1.807, p = 0.04). management tools that they were now aware of, 6 The mean in the pre-test was 1.14 out of 2 (SD (23%) of participants mentioned a wiki, 8 (31%) = 0.65) and the mean in the post-test was 0.81 answered ‘none’, 2 (8%) mentioned Gantt charts out of 2 (SD = 0.68). This was primarily due to a and 1 (4%) mentioned Moodle. 13 (50%) of par- significant reduction in wiki knowledge after the ticipants now identified a Project Route Map as: training (Z = -.890, p = 0.03). This is possibly due ‘a tool to help identify areas of the project that to the fact that the instructors did not promote the need to be considered’, 2 (8%) identified it as wiki as it was not compulsory and instead sug- ‘a tool used to plan the project and is then filed gested that the students used the forum to maintain away’ and 11 (42%) identified it as ‘a tool that is the project documentation. used to identify areas of the project that need to A Wilcoxon Signs ranked test indicated that be considered and consulted on a regular basis’. the increase in knowledge was not significant Participants were asked to list all of the project with regards to project route maps (Z = -1.000, resources that they were familiar with now that p = 0.317), the information pyramid (Z = -.378, the project was complete: 6 (23%) of participants p = 0.705) and budgets (Z = -1.265, p = 0.206). mentioned a ‘wiki’, 11 (42%) said ‘none’ and a However, when combined together there was a small number mentioned talks, PowerPoint, Gantt significant increase in the project management charts and Moodle. The majority of participants knowledge between the pre-test and post-test (Z (14, 54%) identified a budget as ‘an amount of = -1.615, p = 0.05). The mean in the pre-test was money that can be spent on the project to cover 1.10 out of 3 (SD = 0.94) and the mean in the post- all project costs from start to finish’, 11 (42%) test was 1.42 out of 3 (SD = 0.87). A significant

67 Learning 2.0

correlation was detected between undertaking I am keen to start uploading my lectures and re- group work previously and the knowledge score structure my lessons to incorporate Moodle for budget in the pre-test (r = -.414; n = 21; p = 0.03). There was also a significant correlation All courses could be delivered online but invest- between undertaking group work previously and ment in teaching material, software and manpower the knowledge score for forums in the post-test (r = would have to be made to progress the project -.459, p = 0036). A significant correlation between using project management tools previously and Reinforcement of the tutor lead sessions and the knowledge scores about wikis in the post-test (r ability to access the information from anywhere = -.539, p = 0.01) was also detected. In addition, in the centre there was a significant correlation between using project management tools previously and knowl- It gives those trainees who are self motivated the edge about Project Route Maps in the post-test (r = opportunity to keep pace or catch -.408, p = 0.03). While some of these correlations were expected, some of the others were not and Groups were able to share information with team these will be tested again with subsequent cohorts members and use it as a note pad for ideas going through the project module. There was a definite divide between the instructors. Some could identify the support that INSTRUCTORS RESULTS Moodle and the Web 2.0 tools could bring to the training programme, whist others did not. Part of Instructors were surveyed to obtain feedback on this could be attributed to the traditional nature of their opinion of the Web 2.0 tools used during engineering; however, the instructors who were the group project, to gauge their acceptance of new to the company were more open to new ideas. Moodle, and to identify areas where they believed The students who were working under the the LMS could be utilised further. supervision of those instructors who promoted the use of the technologies were far more recep- The course is a practical course and would not tive and willing to use Moodle and the Web 2.0 lend itself well to online delivery tools. However, the students who were supervised by those instructors who did not encourage the It was long winded and the trainees had to build use of the technologies were far more resistant it themselves, the wiki would be ideal for large to their use. This demonstrates that there must be academic projects that spans a university course enthusiasm and guidance in the use of technology or is used in a notational sense to ensure its successful uptake. It should be noted that these issues were brought to the attention of It is limited to providing a storage facility for management by the authors, but management course material with the odd quiz to reinforce felt they could not force instructors to use the training technologies.

The wiki only helped to confuse matters in this short space of time CONCLUSION

Some of the positive comments made by in- This chapter set out to discuss the impact on structors were as follows: learning with the introduction of a LMS into a

68 Learning 2.0

vocational training course and to answer (i) can the successful implementation of a LMS into a technology supplement the hands-on nature of private vocational training organisation but as the vocational training, (ii) can the use of LMS and empirical data grows this should become possible. Web 2.0 tools (specifically wikis and forums) At this stage the authors suggest, tentatively, that aid training, and (iii) can the pilot be considered eLearning can support and in some instances re- a success? Much of the work carried out in this place some elements of a vocational programme. sector is hands-on and on-the-job training but it The authors acknowledge that the practical also consists of a substantial knowledge element element would be very difficult to replace with that would lend itself well to the use of eLearning. technology but it could be supplemented through From the literature review provided within this other tools such as multimedia and even games chapter, it may be argued that the use of educa- technology. Educational technologies offer the tional technologies for practically-based courses opportunity to open up learning in the vocational (e.g. vocational training) should be a relatively training sector, rather than restricting it to the straight forward process. Early eLearning was traditional 9-5 scenario. The authors would recom- used in a practical capacity; however, the case mend that those educators who deliver traditional study demonstrates that the introduction of learn- vocational training courses be fully consulted during technologies into a vocational training course ing the implementation process as well as receive was received with mixed reviews. This could the necessary training needed to highlight the largely be accredited to the traditional nature of potential benefits that technology could bring to engineering training but what was obvious from the overall course being delivered. If buy-in from the data collected and by the authors’ experience students can be obtained and instructors can learn in working with the organisation, much of the to appreciate the value that could be gained from opposition to the technology use was due to the educational technology then its use could diffuse lack of flexibility and understanding from the in- throughout the vocational training sector. structors. From the analysis it can be seen that the technology was received with a mixed response. The online induction course was well received ACKNOWLEDGMENT by the students (and was also accepted by the instructors). Unfortunately, the use of wikis and This project received financial support from the forums for the project module, while well received Knowledge Transfer Partnerships programme by the students was not fully adopted by the in- (KTP). KTP aims to help businesses to improve structors, possibly as they were not confident of their competitiveness and productivity through their knowledge and use of the technologies. As the better use of knowledge, technology and skills a result, they made the use of the wiki optional that reside within the UK Knowledge Base. KTP and instead asked the students to use the Moodle is funded by the Technology Strategy Board. forum to maintain their project documentation. This caused the students to become confused as to the distinction between forums and wikis, REFERENCES resulting in a decrease in learning. The authors conclude that the research dem- Beetham, H. (2009). E-portfolios in post-16 onstrates that educational technologies could learning in the UK: Developments, issues and potentially aid the vocational training of appren- opportunities. Retrieved October 2, 2009, from tices and that this area could offer a rich source http://www.jisc.ac.uk/uploaded_documents/ep- of future research. However, it is early days to ortfolio_ped.doc design a framework that will aid practitioners in

69 Learning 2.0

Berlanga, A. J., Sloep, P. B., Brouns, F., Bitter- Franklin, T., & Harmelen, M. (2007). Web 2.0 Rijpkema, M. E., & Koper, R. (2008). Towards for content for learning and teaching in higher a TENCompetence e-portfolio. [iJET]. Inter- education. Joint Information Systems Commit- national Journal of Emerging Technologies in tee (JISC) report. Retrieved August 8, 2010, Learning, 3, 24–28. from http://ie-repository.jisc.ac.uk/148/1/web2- content-learning-and-teaching.pdf City & Guilds. (2009). How are centres using ePortfolios. Retrieved November 12, 2009, from Garrison, R. (1997). Computer conferenc- http://www.cityand guilds.com/44912.html ing: The post-industrial age of distance education. Open Learning, 12(2), 3–11. Connolly, T. M. & Stansfield, M. H. (2007a). doi:10.1080/0268051970120202 From e-learning to games-based e-learning. In- ternational Journal of Information Technology Gerring, J. (2007). Case study research: Prin- and Management, 26(2/3/4), 188-208. ciples and practices. Cambridge, UK: Cambridge University Press. Connolly, T. M., & Stansfield, M. H. (2007b). From e-learning to online games-based e-learning: Grosseck, G. (2009). To use or not to use Web Implication and challenges for higher education 2.0 in higher education? Procedia Social and and training. In Li, F. (Ed.), Social implications Behavioral Sciences, 1, 478–482. doi:10.1016/j. and challenges of e-business. Hershey, PA: Idea- sbspro.2009.01.087 Group Publishing. Gunawardena, C. N. (1993). The Social context of Curyer, S., Leeson, J., Mason, J., & Williams, A. online education. In Proceedings of the Distance (2007). Developing e-portfolios for VET: Policy Education Conference, Portland, Oregon. issues and interoperability. Australian flexible Holmes, B., & Gardner, J. (2006). E-learning learning framework. Retrieved October 16, 2009, concepts and practice. London, UK: SAGE from http://e-standards.flexiblelearning.net.au/ Publications. docs/vet-eportfolio-report-v1-0.pdf Horton, W. (2003). E-learning tools and technolo- DeRouin, R. E., Fritzsche, B. A., & Sa- gies: A consumer’s guide for trainers, teachers, las, E. (2005). E-learning in Organizations. educators, and instructional designers. India- Journal of Management, 31(6), 920–940. napolis, IN, USA: Wiley. doi:10.1177/0149206305279815 JISC. (2008). Effective practice with e-portfolios: Dorninger, C., & Schrack, C. (2007). E-portfolio’s Supporting 21st century learning. Joint information in education-learning tools or means of assess- systems committee. London, UK: JISC. ment? In Proceedings of the International Com- puter-Aided Learning Conference (ICL2007), Jonassen, D. H. (1996). Computer-mediated Villach, Austria. communication: Connecting communities of learners. Computers in the Classroom (158–182). EDUCAUSE. (2005).Retrieved on February Edgewood Cliffs, NJ: Prentice-Hall, Inc. 11th, 2011, from E-Portfolios Web page, URL: http://www.educause.edu/ELI/Archives/EPort- Kozma, R. (1987). The implications of cognitive folios/5524 psychology for computer-based learning tools. Educational Technology, 27(11), 20–25.

70 Learning 2.0

Lorenzo, G., & Ittelson, J. (2005). An overview of Whisker, G. (2001). The Postgraduate Research ePortfolios, Educause learning initiative. Handbook. Basingstoke, UK: Palgrave MacMil- lan. Mason, R., Pegler, C., & Weller, M. (2004). E- portfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727. doi:10.1111/j.1467-8535.2004.00429.x ADDITIONAL READING

McBride, R., & Schostak, J. (1994). What is Agostini, A., De Michelis, G., & Loregian, M. qualitative research. Retrieved May 7, 2009, from (2009). Using blogs to support participative http://www.enquireylearning.net/ELU/Issues/ learning in university courses. Int. J. Web Based Research/Res1Ch1.html Communities, 5(4), 515–527. doi:10.1504/ McDonald, J. (2002). Is “as good as face-to-face” IJWBC.2009.028087 as good as it gets? Journal of Asynchronous Learn- Baxter, G. J., Connolly, T. M., & Stansfield, M. ing Networks, 6(2), 10–23. (2009). The use of blogs as organisational learn- McKelvie, G., Dotsika, F., & Patrick, K. (2007). ing tools within project-based environments. Interactive business development, capturing Int. J. Collaborative Enterprise, 1(2), 131–146. business knowledge and practice: A case study. doi:10.1504/IJCENT.2009.029285 The Learning Organization, 14(5), 407–422. Berge, Z. L., & Muilenburg, L. Y. (2005). Stu- doi:10.1108/09696470710762637 dent Barriers to Online Learning: A factor ana- Moodle. (2010). Moodle Statistics. Retrieved lytic study. Distance Education, 26(1), 29–48. August 25, 2010, from http://moodle.org/stats/ doi:10.1080/01587910500081269 Richardson, H. C., & Ward, R. (2005). Developing Cartelli, A., Maillet, K., Stansfield, M. H., Con- and implementing a methodology for reviewing nolly, T. M., Jimoyiannis, A., Magalhaes, H., e-portfolio products, The centre for recording & Toland, J. (2008). Identifying and promoting achievement (CRA). Retrieved October 10, 2009, best practice in virtual campuses and e-searning, from http://www.jisc.ac.uk/uploaded_documents/ ED-MEDIA 2008 Conference - World Confer- epfr.doc ence on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria (30 June-4 Saunders, M., Lewis, P., & Thornhill, A. (2007). July 2008) (pp. 897-902). Research methods for business students (4th ed.). Pearson Education Ltd. Chapman, C., Ramondt, L., & Smiley, G. (2005). Strong community, deep learning: Strivens, J. (2007). A survey of e-pdp and ePortfo- Exploring the link. Innovations in Education lio practice in UK higher education. UK: Higher and Teaching International, 42(3), 217–230. Education Academy. doi:10.1080/01587910500167910 Wenger, E. C. (2000). Communities of practice Cole, M. (2009). Using Wiki technology to support and social learning systems. Organization, 7(2), student engagement: Lessons from the trenches. 225–246. doi:10.1177/135050840072002 Computers & Education, 52(1), 141–146. Wenger, E. C., & Snyder, W. M. (2000). Com- doi:10.1016/j.compedu.2008.07.003 munities of practice: The organizational frontier. Harvard Business Review, 78(1), 139–145.

71 Learning 2.0

Connolly, T. M., & Stansfield, M. H. (2007). Lorenzo, G., & Ittelson, J. (2005). An overview of Developing constructivist learning environ- e-portfolios. D.Oblinger (Ed.), Educause learn- ments to enhance learning. In N. Buzzetto-More ing initiative paper (pp. 1-28). Retrieved October (Ed.), Principles of effective online teaching: A 18, 2010, from: http://www.sorteoudla.org.mx/ handbook for experienced teachers developing promueve/ciedd/CR/tecnologia/AnOverviewof- e-learning, 19-38. EPortfolios.pdf Gerrard, C., Connolly, T. M., & Stansfield, M. Paulsen, M. F. (2003). Experiences with learning (2006). The role of staff development to enhance management systems in 113 European institutions. the integration of e-learning within the HE cur- Journal of Educational Technology & Society, riculum, European Conference on e-Learning 6(4), 134–148. (ECEL) 2006, 11-13 September 2006, University Schön, D. A. (1983). The reflective practitioner: of Winchester, UK. How professionals think in action. New York, Grippa, F., & Secundo, G. (2009). Web 2.0 project- NY: Basic Books. based learning in higher education: some prelimi- Schön, D. A. (1987). Educating the reflective prac- nary evidence. Int. J. Web Based Communities, titioner: Towards a new design for teaching in the 5(4), 543–561. doi:10.1504/IJWBC.2009.028089 professions. San Fransisco, CA: Jossey-Bass Inc. Hall, H., & Davison, B. (2007). Social software Shin, J., & Bickel, B. (2008). Communities of as support in hybrid learning environments: the practice: Creating learning environments for value of the blog as a tool for reflective learning educators (Kimble, C., & Hildreth, P., Eds.). and peer support’. Library & Information Sci- Information Age Publishing. ence Research, 29(2), 163–187. doi:10.1016/j. lisr.2007.04.007 Stansfield, M. H., & Connolly, T. M. (Eds.). (2009). Institutional transformation through best Illeris, K. (2002). The Three Dimensions of Learn- practices in virtual campus development: Advanc- ing: Contemporary learning theory in the tension ing e-learning policies organizations. Hershey, field between the cognitive, the emotional and the PA: IGI Global Publishing. doi:10.4018/978-1- social. Copenhagen: Roskilde University Press. 60566-358-6 Kim, S. W., & Lee, M. G. (2007). Validation of Stansfield, M. H., & Connolly, T. M. (2009). An an evaluation model for learning management exploration into key issues relating to the adop- systems, Journal of Computer Assisted Learning, tion of good practices in e-learning and virtual Blackwell Publishing Ltd. campuses. In Mayes, T., Morrison, D., Mellar, Larusson, J. A., & Alterman, R. (2009). Wikis to H., Bullen, P., & Oliver, M. (Eds.), Transforming support the “collaborative” part of collaborative higher education through technology enhanced learning. Computer-Supported Collaborative learning. Higher Education Academy. Learning., 4(4), 371–402. doi:10.1007/s11412- Wenger, E., McDermott, R., & Snyder, W. M. 009-9076-6 (2002). Cultivating communities of practice. Lawless, N., & Allan, J. (2004). Understanding Boston, MA: Harvard Business School Press. and reducing stress in collaborative e-learning. Electronic Journal of e-Learning, 2(2).

72 Learning 2.0

Williams, J. B., & Jacobs, J. (2004). Exploring LMS: The components in which learners and the use of blogs as learning spaces in the higher tutors participate in online interactions of various education sector. Australasian Journal of Educa- kinds, including online learning (Becta, 2003) tional Technology, 20(2), 232–247. Portfolio: A collection of documents relating to a learner’s progress, development, and achievements (Beetham, 2009). Social Software: Social software is a commu- KEY TERMS AND DEFINITIONS nity driven technology which facilitates interaction Communities of Practice (CoPs): Groups of and collaboration and depends largely on social people informally bound together by shared ex- convention (McKelvie et al., 2007). pertise and passion for a joint enterprise (Wenger Web 2.0: The social use of the Web which & Snyder, 2000, pp. 139). allow[s] people to collaborate, to get actively E-Portfolios: A valuable online tool that learn- involved in creating content, to generate knowl- ers, faculty, and institutions can use to collect, edge and to share information online (Grosseck, store, update, and share information. E-Portfolios 2009, pp. 478). allow students to reflect on their learning, com- Wiki: A website (or other hypertext document munication with instructors, document credentials, collection) that allows users to add content, as on and provide potential employers with examples an Internet forum, but also allows anyone to edit of their work (EDUCAUSE, 2005). the content (G. Avram, 2006) eLearning: Any use of Web and Internet technologies to create learning experiences (Horton, 2003, pp. 13).

73 Section 2 Implementing and Evaluating 75

Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs

Paul Lam The Chinese University of Hong Kong, Hong Kong

Judy Lo The Chinese University of Hong Kong, Hong Kong

Jack Lee The Chinese University of Hong Kong, Hong Kong

Carmel McNaught The Chinese University of Hong Kong Hong Kong

ABSTRACT Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized Web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.

INTRODUCTION provision of learning content, communication tools, and organizations of user groups (Paulsen, Learning management system (LMS) is a broad 2002). Jovanovic et al. (2007) defined an LMS as term that is used for a wide range of systems that “a software environment that enables interactive organize and provide access to online learning web-based teaching and supports administration of services for students, teachers and administrators. distance courses, allowing instructors to distribute These services usually include access control, information to students, producing course mate-

DOI: 10.4018/978-1-60960-884-2.ch004

rial, preparing assignments and tests, engaging in data in WebCT (version 3.8) is even more chal- discussions, and managing courses and distance lenging. As it does not employ a database structure, classes” (p. 46). these records or logs of activities can be extracted In 2005, 95% of all higher-education institu- using the provided display logs functions which tions in the UK were using an LMS (Browne, are very limited in functionality. The locations Jenkins, & Walker, 2006). At The Chinese Univer- where the record information is stored are not sity of Hong Kong (CUHK), two LMS (WebCT clear because of the lack of a database structure, and Moodle) are centrally supported. Indeed, the and so time and effort are needed for: 1) testing majority of the University’s eLearning activities through trial and error for the allocation of the are supported by these central services; apart from desired information; 2) checking whether the data one faculty (Engineering), there are relatively few are accurate; and 3) developing software to extract non-centrally-hosted course websites. the information for all courses in the University. Effective record-keeping, and extraction and After retrieval of information, additional work is interpretation of eLearning logs, can reveal valu- needed to standardize and automate the integra- able information on how these LMS are used tion, interpretation and reporting processes of to facilitate teaching and learning. As noted by the log data so that we have a common ground to Sen, Dacin and Pattichis (2006), the use of logs compare and contrast eLearning uses over time. for tracking user activities is quite common in Romero, Ventura and Garcia (2008) discussed commercial settings where customer habits and using data-mining techniques to explore the raw trends are traced and monitored. Reading user log data of servers in order to understand vari- logs also applies in educational settings – for ous aspects of learner activities. However, such example, the study by Black, Dawson and Priem strategies, though allowing great flexibility in (2008) on how to obtain measures of ‘community’ the topics of study, are technically complex. As in online courses. noted by Black, Dawson and Priem (2008) “server In universities with centralized web-based logs are plagued with a low signal-to-noise ratio: teaching and learning systems, the logs can be simply preparing the data for modeling can con- monitored through inbuilt mechanisms to track sume 80% to 95% of a project’s resources” (p. and record a certain amount of information about 67). A system that is more powerful than the LMS online activities. Colace and De Santo (2003) inbuilt activity log systems, and can regularly commented that monitoring an LMS can enable retrieve and interpret the logs into a number of detailed and useful information on the LMS’s uti- fixed representations for year-by-year comparison lization and efficacy. This information can include and contrast, seems to be what we need. Zhang trend data if the logs have been collected regularly et al. (2007) reported a similar system called over time. Such information can provide the basis Moodog which monitor students’ activities live for various decisions related to the implementation on the Moodle LMS. Our system looks at the is- and promotion of eLearning. sue more from an institutional point of view. We However, these inbuilt web-log tracking sys- may not need to monitor student activity every tems do not normally provide institution-level moment but retrieve and analyze the logs once data. The weblogs are reported in an interface every semester. Also, because of our particular designed for individual teachers to get a sum- context, our system reads and integrates as much mary of activities recorded in individual courses as possible of the logs from both of the LMS we (Mazza & Milani, 2005) rather than for analyses host – WebCT and Moodle. of online learning activities at higher levels (e.g. Earlier development and the framework of department, faculty or institution). Retrieval of the system were reported in Lam et al. (2006).

76 Evaluations of Online Learning Activities Based on LMS Logs

The work at that time was more focused on the present paper serves only to illustrate the advan- data-retrieval stage. The present paper extends tages as well as limitations to using weblogs as our discussion to the automation of the data in- a data source. Apart from relatively easy access terpretation and reporting processes. to the data, the extraction of system logs is also As noted in Lam, Keing, McNaught, and Cheng completely non-intrusive to both teachers and (2006), system logs data can provide information students. Furthermore, repeated measures can be on: 1) popularity; 2) the nature of the functions/ taken for a long period of time so that comparison strategies in use; and 3) engagement of teachers of usage over time is possible. The largely auto- and students. matic extraction of the logs (once the software for logs extraction is devised), and the standardized 1. The notion of popularity is a very simple methods in the follow-up analysis, reporting, and yes/no specification for each course in the interpretation of the data, enable an institution University whether any eLearning activities to build a mechanism which is suitable for ad- are recorded in our logs or not. ministration on a regular basis (e.g. annually). A 2. The nature of the eLearning activities re- comparable system of logs extraction and analysis corded for each web-enabled course refers can then be used in subsequent monitoring. to uses of forums, assignment-submission One limitation of this approach, however, is service, course-content delivery, online that it monitors only uses of the web that utilize quizzes or surveys, grade-book facility, etc. the central system. Also, it has a bias on quantity 3. Engagement reflects how involved teachers rather than quality as logs are numbers rather than and/or students are in these activities. This a full picture of the educational quality of the is the level among the three that convey the content on course websites. Also, not all online finest amount of detail about a site. activities and teachers’/ students’ engagement in these activities can be effectively recorded by After the recognition that there is a course the logs. For example, the availability of course website (popularity), more information can reveal outlines on course websites is an online activity the actual features and activities having occurred that is of interest to universities. However, ‘on- on the site (nature). After learning about the line course outline’ is not an activity separately nature of the website, further information can recorded by the logs of WebCT or Moodle. It is gauge how active teachers and students are on site impossible to identify course outlines unless the (engagement). The data report the actual activi- researchers go into individual websites and read ties recorded and, to a certain extent, they fit the all the documents. requirements of the naturalistic research paradigm The picture portrayed by the logs is only a (Alexander & Hedberg, 1994; Grasha, 1990) that partial representation of the total online learning collection of evaluation data is usefully done in activities and the engagement in these activities. non-intrusive ways (Lam, McNaught & Cheng, Even if the logs reveal certain information on 2008) and should, where possible, be situated in popularity of, nature of and engagement with authentic educational settings (Froehlieh, 1994). activities in websites, care has still to be taken to We do not claim that weblog data represent a understand the exact meanings of these logs based comprehensive solution to all evaluation needs. on the characteristics of the platforms and how logs More comprehensive evaluation studies would are kept in them. Very often, minor adjustments consider evidence, both quantitative and quali- have to be made or there are decisions to make tative in nature, from a variety of sources. The

77 Evaluations of Online Learning Activities Based on LMS Logs

concerning the cut-off points beyond which the in our study. A website that had none of these records are deemed to fall into another category. active features may mean a site that was used by In this paper, we: the teacher for only announcing course information such as examination dates or news of events. 1. Explain how the system works. 2. Use empirical evidence recorded from • Whether or not a particular course used ‘fo- 2007 to 2010 to show how the data could rum’ for ‘active discussion’, for example, be employed to achieve various levels of was judged not only from the existence of analysis; and a forum on the site, but rather on whether 3. Discuss how the better understanding of the there was at least one student access to the online activities have enabled decisions in a forum as well. number of eLearning support initiatives in • ‘Active online quizzes’ meant quizzes that our University. We consider that the data had at least one student attempt. have assisted in better understanding and • ‘Active assignment submission’ meant that refining our eLearning strategies and at least one student submitted work to the supports at both an institutional level and platform. faculty/ department level. • ‘Active content’ meant the website contained at least one document (could be PowerPoint, Word or PDF documents, or HOW THE SYSTEM WORKS any other multimedia files) for download, and there was at least one student down- The following measures have been taken to refine load recorded in the logs. and standardize the data so that we identified the right types of logs for the types of activities we The use of eLearning strategies is more mean- targeted. Clear definitions are also necessary, ingful in courses that have considerable class particularly in our case where we have merged sizes. We have introduced a mechanism to filter weblog readings from two different LMS. The cases based on class size. In our web logs study, standardization makes sure the readings from we only studied classes with a class size equal to the two platforms record the same underlying or larger than 10. activities and record them in a compatible manner. Lastly, it is worth noting that the unit of analy- Not all websites created on the servers should sis in our study was normally a course. In most be considered ‘active’ websites; this means that cases, it was relatively easy to decide whether a there are some sites that are developed but not made certain course had used eLearning strategies or accessible to students. For example, some of the not. However, some decisions had to be made in websites might have been created by the teachers many other cases. Sometimes one course was run or the teaching assistants for testing purposes (e.g. in a number of sessions (at different times and perhaps to be used in the next term) and there were even by different teachers). Our standpoint was actually no real student activities on the site. In our to consider a course eLearning-enabled IF at least weblog study, therefore, we took care to isolate one of the sessions had an active website on any only the so-called ‘active’ websites – websites that one of the LMS. had at least one student access during the course Figure 1 shows the report interface of the period would be included in the data. weblog system. The interface is comprised of The similar ‘active’ concept was also applied several parts such as searching filters (Boxes 1 and in the study of the other four eLearning strategies 2), result window (Boxes 3a and 3b) and graph

78 Evaluations of Online Learning Activities Based on LMS Logs

Figure 1. Inquiry layout of weblog system

window (Box 4). Once a set of filters is submitted In Box 2, administrators can specify the exact into the search engine, results and a graph will be year or term to report on, as well as the exact shown in the result and graph windows. Also, in faculty, course type and even the course code the order to provide the flexibility to enable multiple report should be limited to if desired. 2009 on levels of analysis, different parameters can be set the system means the 2009–2010 academic year for the filters. which is from September 2009 to May 2010 in For example, we can ask for a report on a our context. In addition, there is a choice about certain year/ term or a report that contrasts ac- class size, i.e. the minimum student number in a tivities in multiple years or terms. Reports can be class before it can be included (the default is 10). on institution, faculty, department, or course Boxes 3a and 3b in Figure 1 show the weblog levels. We can call for records of undergraduate results of the whole University in the year 2009. courses, postgraduate courses, or both. The unit Both the ‘popularity’ (Box 3a) and the ‘nature of analysis is normally a course but there is an of activities’ (Box 3b) types of information are option to drill down into the use of LMS by the shown. Table 1 further explains the sub-categories individual sessions of a course (taught by the same in these two areas. or different teachers). Also, the system can report Box 4 contains graphical representations of actual numbers (e.g. how many active course the various numerical data reported in Box 3. websites and how many courses we have in the They are usually sufficient for normal reporting University) or report percentages (e.g. the percent- purposes. However, the data and graphs can be age of courses that have an active website). These exported as an Excel file for further processing parameters are available for the administrators in and analysis if needed. the Box 1 area in Figure 1.

79 Evaluations of Online Learning Activities Based on LMS Logs

Table 1. Definitions of the sub-categories in the report

Notion Sub-categories Definition/ Description Course Total Total number of courses None Total Courses without using any LMS Popularity WebCT Total Used WebCT Moodle Total Used Moodle Multi Total Used both WebCT and Moodle Active website (AW) At least one student access to LMS and view website page during the course period At least one document on the active website for access and at least one student down- Active content (AC) load recorded in the logs Nature of activities Active discussion (AD) At least one student posts a topic or a thread Active assignment (AA) At least one student submitted work to assignment drop-box Active quiz (AQ) At least one student attempted quiz on active website

Figure 2 illustrates how the weblog system Box 6 shows the tabs that lead to analyses of represents teachers’ and/or students’ ‘engagement’ various types of activities. In the result-grid region in various eLearning activities (called level 3 now showing (Box 7), we can tell that among all analysis in the system: Box 5). In this example, the courses in the year, 128 of them had a website the data show the amount of activity recorded in in which student access over the year was between using the various functions over all our active 1 to 100, 731 of them had 101 to 1000 visits, 858 WebCT and Moodle websites in the year 2009. had 1001 to 10,000 visits and 167 had over 10,000 visits in the year. 1586 courses did not have a

Figure 2. Demonstration of result window reporting engagement data

80 Evaluations of Online Learning Activities Based on LMS Logs

website in any of our two platforms or their web- has the capability to differentiate faculty- or de- sites did not record any student visits at all. partment-level eLearning practices (Types D, E Similarly, data and graphs concerning engagement and F analysis in Figure 3). Type A refers to how in the various activities can be exported as an popular LMS are across the whole University. Excel file for further processing and analysis. Type B refers to the nature of the features used in the LMS. Type C refers to how students and/ or teachers in general are engaged in the various VARIOUS LEVELS OF ANALYSIS activities. Type D refers to how popular LMS are used in a certain faculty/ department or even a The flexibility of the system allows us to conduct certain course. Type E refers to the analysis of various levels of analysis easily. Six main types how various eLearning functions are used in the are distinguished in our framework as shown in different faculties/ departments or courses. Figure 3. The framework is an extended version Lastly, Type F analysis distinguishes the engage- of the one reported in Lam et al. (2006) as a re- ment in activities by students and/or teachers in sult of enhanced analytical power through recent various faculties/ departments. developments. Below, we use empirical evidence recorded In brief, the system provides a general insti- from the years 2007 to 2009 to illustrate how the tution-wide overview of eLearning activities data could be employed to achieve these various (Types A, B and C analysis in Figure 3); it also levels of analyses. We looked at all undergraduate

Figure 3. Model of the monitoring mechanism through system logs

81 Evaluations of Online Learning Activities Based on LMS Logs

and postgraduate courses that had class sizes of The overall impression is that an increasing 10 or above at our University. These illustrations number of courses began to have a web presence are not meant to be a comprehensive discussion in our LMS. The courses that had a website in of the eLearning activities in our University. Only one or both of our LMS grew steadily from around a few key features are discussed. We also only 42% (1432 active websites) in 2007, to 46% (1555 report up to the faculty level, rather than drilling active websites) in 2008 and then to over 54% into what happened in individual departments (1891 active websites) in 2009. More websites or even courses. The actual names of the facul- were built on the WebCT platform because ties are also hidden as the interest of the present Moodle was not introduced at CUHK until 2007. paper is not on detailed disciplinary differences. Nevertheless, the increase in Moodle use over the The data represent a few snapshots of some of years is impressive. the analyses the system allows us to do that, as will be discussed later, reveal useful information Comparison of Two Functions about the actual use and usage of the LMS in our University. What are the common functions used on these sites? Data show that the main use of these web- Popularity of LMS Across sites was for content delivery. The more interac- the Whole University tive functions such as quizzes and discussion were used less. We illustrate this large contrast Figure 4 shows the overall websites built on the by highlighting the use of ‘active content’ and WebCT and the Moodle platforms across three ‘active quiz’ in three years in Figure 5. years. Over 90% of the active websites contained some sorts of content (various types of files such

Figure 4. Usage of an LMS in CUHK courses in the academic years 2007–2009

82 Evaluations of Online Learning Activities Based on LMS Logs

Figure 5. LMS functionality used in the academic years 2007–2009

as doc, PowerPoint or PDF) which have been during the year). A few even visited the websites accessed by students in the course of study. In over 1000 times. contrast, only a small percentage of courses had However, the data also showed that students employed online quizzes as an active strategy. did not visit the websites for the interactive func- The percentage of quiz-containing websites actu- tions. Only 1995 students (recalling that over ally decreased quite significantly over the years. 15,000 students ever visited the LMS that year) In fact, low use was also observed for other had written anything on any of the online forums interactive functions such as ‘active discussion’ (many of these 1995 students wrote only one or and ‘active assignment’. On average, active-online two postings). Similarly, relatively few students discussions were found only in about 21.5% used the assignment submission function and the (2007), 20.9% (2008) and 14.8% (2009) of the quizzes. Many of the students who used the quiz- total websites respectively. Also, 27.6% of the zes, however, had completed multiple online websites used the active-assignment feature in quizzes. 4742 students used quizzes and about 2007. It gradually decreased to 24.0% in 2008 one-third of them attempted 11–20 quizzes on the and then to 14.6% in 2009. LMS. One-tenth even made 20 times or more attempts. Engagement in Four Areas Use of LMS in Four Faculties Figure 6 shows students’ visits to the websites and their engagement in some of the website functions Different faculties used LMS quite differently. in 2009. The data first of all confirmed that LMS ELearning experience was not the same across are actively used by students. The ‘active website’ disciplines. analysis showed that more than 15,000 students Figure 7 shows the use of the two LMS in four (over 90% of the total student population) visited of our eight faculties. First of all, we observed one or more of the websites. Many of the students that an increasing number of courses had a course paid quite frequent visits (more than 30% of the website and this was true in all of the faculties. This students accessed the LMS more than 20 times finding is in line with the general picture portrayed on Figure 4. However, the interest in using LMS

83 Evaluations of Online Learning Activities Based on LMS Logs

Figure 6. Frequency of usage of function within the LMS (WebCT) in 2009

varied quite a lot from one faculty to another (e.g. four faculties inspected (Figure 8). Most of the over 90% of courses in Faculty B versus around websites (around 90% or more) in these faculties 50% to 60% in the other three faculties). It is had active content that had been accessed by stu- also interesting to note that students were likely dents in the respective years. This content-delivery to be told to use two LMS because some of the focus was similar among the faculties. teachers used WebCT while some used Moodle What makes faculties different from each even if they were in the same faculty. This was other is their use of the other functions of LMS. particularly true in Faculty A where the prefer- In Faculty B, we found the use of online discus- ences of WebCT and Moodle by teachers were sion a popular strategy (nearly half of the cours- roughly half and half. Using two different LMS es used it) in 2007 but the enthusiasm of using and getting used to the navigation and controls this function then dropped significantly (dropped of two systems might have imposed considerable to only about 5% in 2009). The interest of teach- confusion and unnecessary workload on students. ers in this faculty in the use of online quizzes also dropped to zero in the period. On the contrary, Use of Three Functions in Four the use of online discussion in Faculty A and Faculties Faculty C courses has been relatively stable. Comparatively, teachers in Faculty D never Similar to the University-wide phenomenon as seemed to have any great interest in online com- portrayed on Figure 5, using LMS for content munication. delivery was also a common practice across the

84 Evaluations of Online Learning Activities Based on LMS Logs

Figure 7. Use of LMS in different faculties (A–D)

Faculty C impressed us by their interest in using Engagement in Two Functions the quiz function. Percentage of courses having in Two Faculties quizzes in this faculty has been well above the University mean all the years. Figure 9 selectively looks at two online activities Based on personal communications the authors in two of the faculties. It shows that, as expected, have had with teachers in these faculties, there students’ engagement in content-oriented uses are many factors involved in explaining these was similar but their involvement in other online observed differences. Faculty B, for example, was activities could differ from one faculty to another, newly established in 2007 and then had significant basically reflecting the uses of these functions as expansion in teacher and student numbers in the indicated in Figure 9. three-year period. The faculty boosted the use of More than half of the websites in both Facul- eLearning strategies in the early years but the focus ties A and B had 16 or more content files that were was quickly shifted to other matters as more and accessed by students in 2009. Websites were more new teaching staff members were recruited content-heavy and accessing of the content by and teaching loads became heavy. Faculty C is students was popular in both faculties. Their use a professional discipline which requires students of the quiz function was, however, quite different. to remember and understand significant numbers Websites in Faculty C obviously had a higher of terms and information. Online quizzes seem concentration of quizzes in them: over 10% of to be a useful tool to facilitate factual learning so the websites contained 6 or more quizzes that that classroom time can be used for activities that have been actively used by students in the year. require higher-order thinking. Faculty A focuses In general, thus, there is a ‘cultural’ difference on social and humanities subjects where discus- in eLearning use in different faculties; perhaps not sion may be a more suitable online activity than so much in using the web for content but much quizzes. Topics in Faculty D centre on established more when using more interactive functions. laws and formulae and there appears to a perception that online discussions are not needed. The use of the online discussion function in the websites of this faculty has been decreasing over the years.

85 Evaluations of Online Learning Activities Based on LMS Logs

Figure 8. Use of LMS functions in four faculties

REFINING OUR eLEARNING valuable guidelines in terms of eLearning design STRATEGIES can be developed. Effective record-keeping, and extraction and interpretation of eLearning logs Monitoring the LMS, or weblogs in our case, can reveal valuable information on standards of can achieve the objective of evaluating students’ design and development, and program delivery. online learning activities. By analyzing these data,

Figure 9. Websites and students’ engagement in activities in two faculties

86 Evaluations of Online Learning Activities Based on LMS Logs

Using LMS for More than findings thus suggest that institutional eLearn- Content Delivery ing support should not merely focus on having a web presence in courses or using the Web for There are both positive and negative trends in our courseware delivery. Attention also needs to be data concerning adoption of eLearning at CUHK. on the diffusion and sustained use of interactive The overall impression seems to be that more online learning activities. courses have begun to have a web presence in our Sustainability of these strategies seems to LMS. However, when we looked at these course impose particular challenges in our context as websites closely, we did not find many functions the data show not only underuse of the related were being used actively. Most of the websites functions but also a general decrease of use for had content for students to download. There some of them. might also be course information and announce- The question still remains about whether ments, but the other discussion, assignment and teachers are reluctant to interact with students quiz functions were not popular (35% or below). on the Web in general, or whether this is only a Moreover, many of these functions did not seem limitation imposed by LMS. to be sustainable in the sense that we observed a It can be an LMS problem; at CUHK, we are decrease in use of the more interactive functions beginning to see a trend that teachers are interested such as online discussion and online quizzes over in Web 2.0 strategies and social-media software the years. and services. Web 2.0 or social software tools such Many scholars have also commented on this as blogs, wikis, podcasts and media-file-sharing phenomenon that teachers used LMS mainly for systems such as YouTube are supplementing or delivery of content; our findings thus are more gen- even “supplanting” (Gray, Chang & Kennedy, erally applicable outside our own context. Folajimi 2010, p. 33) the basic Web 1.0 strategies such as (2009) remarked that “computers aren’t fulfilling emails and forums in education. Users world-wide their potential to effect significant changes in produce text and other media files for sharing and education, are under-utilized, and are not being these materials can be easily adopted as teaching implemented in very effective or creative ways” and learning resources elsewhere. The communi- (p. 1617). Ginsberg and McCormick (1998) also cations that are facilitated in social software also remarked that computers have only been used have the potential to extend communications to mostly to complement traditional teaching but a scope far beyond the boundary of any single are not designed as tools for effective learning university. For example, Dillon, Wong and Tearle activities on their own. (2007) described internationalization of teaching One major drawback of existing LMS is that and learning defying physical boundaries in terms they are content-centric. Many teachers simply of teachers and students in various places of the move all their teaching materials to the system. world teaching and learning together. The materials are presented uniformly to all learn- We also know of a number of instances at our ers regardless of their background, learning styles University where teachers have begun communi- and preferences. Modern trends in education are cating with students through blogs and Facebook. for learner-centric design where learners are fa- Pituch and Lee (2006) observed that, although cilitated to actively engage in the learning process factors such as perceived usefulness influence to construct their personal knowledge. Teachers LMS use, the strongest influence on student use play the role of ‘facilitator’ who guides the learn- might be system characteristics. Do these social- ing process instead of being the sole information media services possess certain characteristics that provider (Ong & Hawryszkiewycz, 2003). The make them better choices than the LMS, espe-

87 Evaluations of Online Learning Activities Based on LMS Logs

cially when implementing interactive activities? The fact that students at CUHK use a num- Is the LMS losing ground to these popular Web ber of LMS throughout their studies has led the 2.0 technologies? Integration between the ‘Web University to plan to adopt a single LMS in the 1.0’ LMS and Web 2.0 technologies seems to be future. At the time of writing the paper, an evalu- needed (Boulakfouf & Zampunieris, 2008). ation of different LMS is underway; the imminent However, teachers may simply refuse online demise of WebCT is a facilitating factor. LMS logs interactions for teaching and learning in general. have been important information in this evalua- Levin and Arafeh (2002) suggested that a gap exists tion process. For example, we understand that between students and teachers in the manner they the future LMS should be very strong in course use technology for teaching and learning purposes. management and content delivery as these features Students use the internet for daily academic tasks, are used by most teachers. However, we will also e.g. the internet as a virtual textbook and refer- examine LMS for user-friendly interactive and ence library, or virtual tutor, etc. However, most communicative functions, and for convenient teachers are slower to adopt the more complex and integration with existing Web 2.0 tools so that interactive features of the Web (Morgan, 2003). student-oriented eLearning strategies might be If this is the case, our support should focus on better used in the future. motivating teachers to rethink teaching rather than just to promote functionalities of the LMS only. Disciplinary Differences: In a traditional, largely face-to-face university Differentiating Support Strategies such as CUHK, blended learning (e.g. Garrison & Vaughan, 2007) offers possibilities that allow The LMS logs showed differences in the use of teachers to retain their traditional roles while eLearning strategies between the faculties. This gradually increasing the suite of online tools and piece of information is crucial to refinement of our strategies they use. In our context eLearning and approaches to supporting eLearning in individual LMS are at present supplementary tools to assist faculties. For example, we will focus on exploring in the process of teaching and learning. What we the decline of interest of using certain eLearning want to achieve is that eLearning becomes more strategies in places where the strategies do not integrated into course design and thus comple- seem to be sustained (e.g. Faculty B in our study). mentary, rather than merely supplementary, to The support in that particular context will focus on classroom teaching. known challenges rather than simply explaining Our support should not be limited to teach- and promoting eLearning in a traditional fashion. ers only. Students may use technology in their Another key factor is the perceived intrinsic everyday lives, but they may not be able to use return on effort in terms of better teaching practice the technology wisely for learning purposes. and students’ learning, as well as the expected ex- Kirkwood and Price (2005) pointed out that trinsic benefit during performance appraisal. Our students may have ability with different com- support to teachers in this circumstance includes puter applications, but only a few can transform both assisting them in lowering the effort in using and apply their digital competence to learning technology, as well as showing them the usefulness processes. For instance, students being “familiar of the strategies in fulfilling one or more of their with the use of email does not imply expertise in needs. However, Thomas et al. (2009) interviewed rigorous online debate and discussion” (p. 271). eight teachers at CUHK and noted how these teach- Support for meaningful eLearning thus requires ers considered innovative e-learning projects to be professional development and support for both lonely and time-consuming. Being self-motivated teachers and students. and intellectually committed was a key factor that

88 Evaluations of Online Learning Activities Based on LMS Logs

sustained these teachers and allowed them to stay • informing us students’ online learning ex- engaged. Our University is currently reviewing periences which started a process to revisit its strategies for assessing teaching quality and support and services needed, including the innovation in teaching will gain more prominence. discussion of a single platform; and This extrinsic reward should support a move to • assisting us in formulating faculty/depart- more active uses of eLearning. ment- specific support; eLLPs provide an- In disciplines where teachers seem to have other source of information on discipline- a growing interest in using certain eLearning specific eLearning needs and habits. strategies, we can strengthen our support for these strategies and then also disseminate these teaching FUTURE RESEARCH DIRECTIONS ideas to other teachers in related fields who are newer to eLearning strategies. As mentioned above, LMS logs provide con- Another structural condition that supports venient data about web uses but are limited to diffusion is the support provided by the depart- showing the quantity rather than quality of these ment or by the University – in terms of monetary uses. Evaluating the LMS weblogs cannot provide development grants, technical services, and con- information on the mode of online participation, sultative teaching and learning support in terms interaction patterns and group dynamics. The of eLearning pedagogical design. This support, interaction between users is of most importance though somewhat limited, still has significant when evaluating the function and effectiveness impact on workload. The contents of this chapter of an LMS (Gunawardena, Carabajal, & Lowe, will be presented at the University committee that 2001). The data highlight areas where further considers policy for eLearning at CUHK. exploration seem to be warranted and we need We know that differences in practice are also other evaluation strategies (e.g. surveys and focus at the department level. We have devised an groups) to confirm these trends and patterns and ‘eLearning Liaison Person’ (eLLP) strategy to look for the reasons behind them. One important better inform us the needs of individual depart- area that is certainly worth exploring further is ments. ‘eLLPs’ are representatives of individual the observed decline in the use of communicative departments who act as liaisons to inform central functions in the LMS. Is there a genuine decline services of departmental needs and also act as of e-communication for learning or there are other conduits for conveying information about central software and services (such as Web2.0 technolo- services and events. gies) that are taking teachers’ and students’ at- To conclude, the LMS logs are useful in assist- tention away from similar functions in the LMS? ing us to continually adjust our eLearning support strategies. Some of the influences include: CONCLUSION • informing us the need to further investigate teachers’ attitudes and habits of using in- Effective record-keeping, and extraction and teractive e-functions; interpretation of eLearning logs can reveal valu- • leading us to focus on meaningful uses able information on eLearning use. In universities of the functions rather than merely on ex- with centralized web-based teaching and learning plaining the functions of LMS in our pro- systems, monitoring the logs can be accomplished motion and support; because most eLearning platforms have inbuilt • informing us the important considerations mechanisms to track and record a certain amount when planning our future LMS; of information about online activities. We reported

89 Evaluations of Online Learning Activities Based on LMS Logs

our strategy in building a system to automatically Boulakfouf, S., & Zampunieris, D. (2008). Inte- retrieve and then interpret the logs of eLearning grating Web 2.0 features into a learning manage- activities in our two centralized LMS. The method ment system. Conference on e-Learning 2008. has become a convenient and repeatable method Retrieved December 29, 2010, from http://pub- for the evaluation of the richness of eLearning lications.uni.lu/record/1695/files/DZ_Pages%20 resources and interactions over time. In so do- from%20ecel08-cd1.pdf ing, however, we have to acknowledge that the Browne, T., Jenkins, M., & Walker, R. (2006). A weblog analyses have limitations and should be longitudinal perspective regarding the use of VLEs interpreted with evidence about eLearning uses by higher education institutions in the United King- collected by other means and from other sources. dom. Interactive Learning Environments, 14(2), We explained how the system works, and we 177–192. doi:10.1080/10494820600852795 used empirical evidence recorded from the academic years 2007 to 2009 to show how the data Colace, F., & De Santo, M. (2003). Evaluating on- could be employed to achieve various levels of line learning platforms: A case study. Proceedings analyses. Weblogs represent a comparatively easy, of the 36th Hawaii International Conference on automatic, and non-intrusive method to provide System Sciences. Retrieved December 29, 2010, relatively quick and accurate data. We found that from http://www.hicss.hawaii.edu/HICSS36/ the logs have informed us the pattern of use of HICSSpapers/ETWBE03.pdf various online activities as well as the change in Dillon, P., Wang, R., & Tearle, P. (2007). these activities over time. The new understanding Cultural disconnection in virtual education. has facilitated decisions in a number of eLearning Pedagogy, Culture & Society, 15, 153–174. support initiatives in our University, including the doi:10.1080/14681360701403565 evaluation of new LMS systems for the future, and an increase in sensitivity towards faculty and Folajimi, Y. (2009). Transforming distant peda- departmental needs. gogical learning to Web based collaborative system: An Intelligent Tutoring Systems Archi- tecture. In T. Bastiaens et al. (Eds.), Proceedings REFERENCES of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Alexander, S., & Hedberg, J. (1994). Evaluating 2009 (pp. 1618–1622). Chesapeake, VA: AACE. technology-based learning: Which model? In Beattie, K., McNaught, C., & Wills, S. (Eds.), Froehlich, T. J. (1994). Relevance reconsidered - Multimedia in higher education: Designing for Towards an agenda for the 21st century: Introduc- change in teaching and learning (pp. 233–244). tion to special topic issue on relevance research. Amsterdam, Netherlands: Elsevier. Journal of the American Society for Information Science American Society for Information Sci- Black, E. W., Dawson, K., & Priem, J. (2008). ence, 45(3), 124–134. doi:10.1002/(SICI)1097- Data for free: Using LMS activity logs to mea- 4571(199404)45:3<124::AID-ASI2>3.0.CO;2-8 sure community in online courses. The Internet and Higher Education, 11, 65–70. doi:10.1016/j. Garrison, D. R., & Vaughan, N. D. (2007). iheduc.2008.03.002 Blended learning in higher education: Frame- work, principles, and guidelines. San Francisco, CA: Jossey-Bass.

90 Evaluations of Online Learning Activities Based on LMS Logs

Ginsberg, R., & McCormick, V. (1998). Computer Lam, P., McNaught, C., & Cheng, K. F. use in effective schools. Journal of Staff Develop- (2008). Pragmatic meta-analytic studies: ment, 19(1), 22–25. learning the lessons from naturalistic evaluations of multiple cases. ALT-J, 16(2), 61–79. Grasha, T. (1990). The naturalistic approach to doi:10.1080/09687760802315879 learning styles. College Teaching, 38(3), 106–113. Levin, D., & Arafeh, S. (2002). The digital Gray, K., Chang, S., & Kennedy, G. (2010). Use disconnect: The widening gap between Internet of social Web technologies by international and savvy students and their schools. Pew Internet domestic undergraduate students: Implications for & American Life Project. Retrieved December learning and teaching in Australian universities. 29, 2010, from http://www.pewinternet.org/~/ Technology, Pedagogy and Education, 19(1), media//Files/Reports/2002/PIP_Schools_Inter- 31–46. doi:10.1080/14759390903579208 net_Report.pdf.pdf Gunawardena, C., Carabajal, K., & Lowe, C. A. Mazza, R., & Milani, C. (2005). Exploring usage (2001). Critical analysis of models and meth- analysis in learning systems: Gaining insights ods used to evaluate online learning networks. from visualizations. 12th International Conference Education Resources Information Center (ERIC). on Artificial Intelligence in Education (AIED Retrieved December 29, 2010, from http://eric. 2005). The Netherlands: Amsterdam. Retrieved ed.gov/PDFS/ED456159.pdf December 29, 2010, from http://linux3.dti.supsi. Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, ch/~mazza/Web_area/Pubblicazioni/AIED05/ M., Eap, T., Richards, G., & Brooks, C. (2007). aied-ws2005.pdf Using Semantic Web technologies to analyze Morgan, G. (2003). Faculty use of course learning learning content. Internet computing, 11(5), 45–53. systems. Educause center for applied research Kirkwood, A., & Price, L. (2005). Learners reports. Retrieved December 29, 2010, from and learning in the 21st century: What do we http://net.educause.edu/ir/library/pdf/ecar_so/ers/ know about students’ attitudes and experi- ers0302/ekf0302.pdf ences of ICT that will help us design courses? Ong, S. S., & Hawryszkiewycz, I. (2003). To- Studies in Higher Education, 30(3), 257–274. wards personalised and collaborative learning doi:10.1080/03075070500095689 management systems. Third IEEE International Lam, P., Keing, C., McNaught, C., & Cheng, K. Conference on Advanced Learning Technolo- F. (2006). Monitoring e-learning environments gies (ICALT’03). Retrieved December 29, 2010, through analyzing Web logs of institution-wide from http://www.computer.org/portal/web/csdl/ e-learning platforms. In L. Markauskaite, P. Good- doi/10.1109/ICALT.2003.1215113 year, & P. Reimann (Eds.), Who’s learning? Whose Paulsen, M. F. (2002). Online education systems: technology? (pp. 429–440). Proceedings of the Discussion and definition of terms. Retrieved 23rd Annual Australian Society for Computers in December 29, 2010, from http://www.porto.ucp. Learning in Tertiary Education 2006 conference, pt/open/curso/modulos/doc/Definition%20of%20 Sydney, Australia. Retrieved December 29, 2010, Terms.pdf from http://www.ascilite.org.au/conferences/syd- ney06/proceeding/pdf_papers/p62.pdf

91 Evaluations of Online Learning Activities Based on LMS Logs

Pituch, K. A., & Lee, Y. K. (2006). The influence of Becker, R., & Jokivirta, L. (2007). Online learning system characteristics on eLearning use. Comput- in universities: Selected data from the 2006 Ob- ers & Education, 47(2), 222–244. doi:10.1016/j. servatory survey–November 2007. Observatory compedu.2004.10.007 on borderless higher education (OBHE) online. Retrieved December 29, 2010, from http://www. Romero, C., Ventura, S., & Garcia, E. (2008). Data obhe.ac.uk/documents/view_details?id=15 mining in course management systems: Moodle case study and tutorial. Computers & Education, Coates, H., James, R., & Baldwin, G. (2005). 51, 368–384. doi:10.1016/j.compedu.2007.05.016 A critical examination of the effects of learning management systems on university teaching and Sen, A., Dacin, P. A., & Pattichis, C. (2006). learning. Tertiary Education and Management, Current trends in Web data analysis. Com- 11(1), 19–36. doi:10.1080/13583883.2005.996 munications of the ACM, 49(11), 85–91. 7137 doi:10.1145/1167838.1167842 Collis, B., & van der Wende, M. (2002). Models Thomas, K., Lam, P., & Ho, A. (2009). Knowledge of technology and change in higher education: An diffusion in eLearning: Learner attributes and ca- international comparative survey on the current pabilities in an organization. In G. Siemens & C. and future use of ICT in higher education. Center Fulford, (Eds.), ED-MEDIA 2009 (pp.493–497). for Higher Education Policy Studies (CHEPS). Proceedings of the 21st Annual World Confer- Retrieved December 29, 2010, from http://doc. ence on Educational Multimedia, Hypermedia utwente.nl/44610/1/ictrapport.pdf and Telecommunications, Honolulu, Hawaii. (pp. 22–26). Chesapeake, VA: Association for the O’ Reilly, T. (2005). What is web 2.0? Retrieved Advancement of Computers in Education. December 29, 2010, from http://www.ttivanguard. com/ttivanguard_cfmfiles/pdf/dc05/dc05ses- Zhang, H., Almeroth, K., Knight, A., Bulger, M., sion4003.pdf & Mayer, R. (2007). Moodog: Tracking students’ online learning activities. In C. Montgomerie & Piccoli, G., Ahmad, R., & Ives, B. (2001). Web- J. Seale (Eds.), Proceedings of World Confer- based virtual learning environments: A research ence on Educational Multimedia, Hypermedia framework and a preliminary assessment of ef- and Telecommunications 2007 (pp. 4415–4422). fectiveness in basic IT skills training. Management Chesapeake, VA: AACE. Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989 Selim, H. M. (2007). Critical success factors ADDITIONAL READING for eLearning acceptance: Confirmatory factor models. Computers & Education, 49(2), 396–413. Alavi, M., & Leidner, D. E. (2001). Review: doi:10.1016/j.compedu.2005.09.004 Knowledge management and knowledge management systems: conceptual foundations and Wang, W. (2007). Features of future learning research issues. Management Information Systems management system. In T. Bastiaens & S. Carliner Quarterly, 25(1), 107–136. doi:10.2307/3250961 (Eds.), Proceedings of World Conference on E- Learning in Corporate, Government, Healthcare, and Higher Education (pp. 1332–1335). Chesa- peake, VA: AACE.

92 Evaluations of Online Learning Activities Based on LMS Logs

Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, Active Websites: Websites that have recorded J. F. (2004). Can e-learning replace classroom student access (meant viewing the front page of learning? Communications of the ACM, 47(5), website in our study) in the specified period of time. 75–79. doi:10.1145/986213.986216 Engagement: LMS logs revealing the amount of activities (e.g. frequency of use) recorded for the various functions. LMS Logs Retrieval and Reading System: KEY TERMS AND DEFINITIONS An automatic system to retrieve suitable logs Active Content: Content on the website that from an LMS server, interpret the logs and then has been accessed (meant downloading in our represent them in ways that facilitate our under- study) by students in the specified period of time. standing of the types and level of activities hap- Active Discussion: Online forums that have pened on the LMS. been accessed (meant posting in our study) by Nature of Activities: LMS logs revealing students in the specified period of time. the common functions being used on the course Active Quizzes: Online quizzes that have been websites. accessed (meant attempting the quizzes in our Popularity: Using LMS logs to find out how study) by students in the specified period of time. many courses opened an active website on one or more of the LMS in our study.

93 94

Chapter 5 ANGEL Mining

Tyler Swanger Yahoo! & The College at Brockport, State University of New York, USA

Kaitlyn Whitlock Yahoo!, USA

Anthony Scime The College at Brockport, State University of New York, USA

Brendan P. Post The College at Brockport, State University of New York, USA

ABSTRACT This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation.

INTRODUCTION registration and administration, as well as skills gap analysis, tracking, and reporting (Sclater, A Learning Management System (LMS) is a course 2008; Watson & Watson, 2007; Paulsen, 2003). independent framework that provides, delivers, A Learning Management System generally has and manages instructional content, identifies and the same interface and features for all courses at assesses learning, and tracks and records progress a given school. Typically they have discussion towards those goals. It may also provide course forums, calendars, quiz capability, group work and chat spaces, and gradebooks, and perhaps

DOI: 10.4018/978-1-60960-884-2.ch005

some customization capability for the instructor internal relationships and patterns that characterize or individual student (Feldstein & Masson, 2006). the data set as a whole. In the case of this study, Learning Management Systems undergo the usage of ANGEL by students and instructors is evaluation for a number of reasons. Evaluations of described by the ANGEL data. This study clearly LMS data may be conducted to assist instructors demonstrated that data mining techniques can in understanding how to enhance student learn- be applied to find unknown patterns, interesting ing (García, Romero, Ventura, & Calders, 2007, patterns, confirm assumptions, and consider sta- Romero, Ventura, & García, 2008). Schools may tistical results for making decisions on the future evaluate existing LMS options to select a particular of ANGEL or another LMS at a college. The data initial choice (Sclater 2008), or when a decision mining methods of classification, association, is being considered on whether to replace or not and clustering were applied to analyze the data. the current LMS; replacement may be due to the Classification produces a decision tree which current system no longer being supported (Sturgess predicts which courses will use the LMS system & Nouwens, 2004). in the future, based on course specific attributes An investigative analysis of the usage of A such as course type, discipline, and the number of New Global Environment for Learning (ANGEL), students enrolled. Association mining discovers learning management system was conducted at a the usage of one feature’s effect on the usage of comprehensive 4-year college. This implementa- all other sets of features. Grouping features with tion of ANGEL has been used by the college’s similar values is the process of clustering. From faculty, staff, and students for eight years. The the results of these analyses, metrics are formed ANGEL Learning Management System “enables to indicate usage of features in the LMS. efficient and effective development, delivery and management of courses, course content and learning outcomes. Engaging communication and BACKGROUND collaboration capabilities, enhance instruction to deliver leading edge teaching and learning” There is a similarity between LMS and Knowledge (ANGEL Learning, 2008). Management Systems (KMS); both provide a re- This investigation was prompted by the merger pository for knowledge which is valuable for the of the ANGEL company with a competitor, Black- user. In a KMS the knowledge is kept and used by board, Inc; and Blackboard’s decision to stop sup- an organization’s employees. In a LMS the purpose port of the college’s version of ANGEL in 2012 is to disseminate knowledge from instructors to (Blackboard, 2010). These changes present the students and to share knowledge in a way to en- college with the opportunity and need to assess hance student learning (Haldane, 1998). There are the usage of the system. These events have forced also similarities between Learning Management the college to examine the need for a LMS, and if Systems and distance learning. Distance learning so what features are necessary in the event a new uses LMS like software to provide students with product needs to be purchased. learning materials and activities while tracking Given these circumstances, data mining student activity (Falvo & Johnson, 2007). techniques were applied to evaluate which LMS Whether the system is an LMS, a KMS, or a features are used most commonly and most ef- distance learning system the organization needs fectively by instructors and students. Data mining to select and implement the system best suited to techniques are applicable to situations where large their needs. Studies have been done to compare amounts of data exist, and the data may contain different Learning Management Systems (LMS)

95 ANGEL Mining

prior to installation or to evaluate effectiveness are administration, security, access, integration (Beatty & Ulasewicz, 2006). There have been with other systems, course design, development many approaches to system evaluation and se- and integration, course monitoring, assessment lection. design, online collaboration and communications, Traditional systems requirements analysis and productivity tools. These features each have a has been used to evaluate and select LMS. Func- number of sub-features that are given weights to tionality desired by faculty has been achieved measure their relative importance. The user rates through requirements analysis followed by system each candidate LMS on each criterion and feature. selection and modification (Sclater, 2008) and The evaluation tool computes an overall score for functionality has been achieved through the use each LMS in each criterion (Commonwealth of of design patterns (Avgeriou et al., 2003). Learning, 2004). Studies have been done to determine frame- Additionally, the independent research com- works for evaluating LMS. Kim and Lee (2008) pany Brandon Hall Research, has completed a validated a model for evaluating Learning study of over 90 LMS with custom comparisons Management Systems (LMS) used in e-learning across more than 200 features. The report of this fields using exploratory factor analysis. Factor study is available on the World Wide Web in an I was instruction management, screen design, interactive format allowing users to specify their and technology and factor II covered interaction requirements and complete side-by-side compari- and evaluation. Roqueta (2008) used Moore’s sons of selected LMS (Chapman, 2010). (1993) transactional distance theory, the diffu- Studies have shown that most evaluations of sion of innovations theory (Rogers, 2003), and LMS look toward the technology of the system. Malikowski’s (2007) model for evaluation of Specifically, evaluations are done on communica- learning systems to conclude that there is a dif- tion, education management, and file management ference between learning management systems features and how effectively these features are and course management systems, and that LMS implemented in the technology. Hall (2003) pro- are preferable in most cases. vides a list of technically oriented considerations in A commercial tool for evaluation has been evaluating learning management systems. Lewis, developed by 3Waynet Inc. The LMS Evaluation et al. (2005) evaluated nine LMS by considering Tool is spreadsheet like and designed to assist in and comparing the features and capabilities of the selecting a LMS. With this LMS Evaluation Tool systems in terms of content development, group users specify criteria for evaluation, which are work and participation, the calendar, communica- considerations that are important to the institution tion capability, study tools available to students, adopting the LMS, these criteria are cost of own- handling of audio and video, monitoring of student ership, maintainability and ease of maintenance, participation and progress, and usability in terms usability, ease of use, user documentation, user of navigation, interface design and administration. adoption/ vendor profile, openness, standards Evaluations often done on the technology itself compliancy, integration capacity, learning object are conducted by experts and not participants. metadata integration, reliability and effectiveness, Feldstein and Masson (2006) outline features scalability, security, hardware and software consid- that teaching professionals, as well as technical erations, and multilingual support. Each criterion is staff, should consider when selecting an LMS. expanded with relevant questions to be answered. Specifically they suggest looking for the ability The candidate LMS undergoing evaluation are also to customize the LMS to meet the pedagogical rated on their features. The features considered methods of individual instructors, which may vary

96 ANGEL Mining

by discipline, instructor, and learning styles of branches that can be converted to rules unique to individual students. Sturgess and Nouwens (2004) the data set. Machine learning research produced report on a different approach where participa- a number of classification models, the best known tion in the evaluation includes an understanding of which is the C4.5 algorithm (Quinlan, 1993), of the organizational culture and sub-cultures. which uses information gain to define the model’s Specifically they identified academics, informa- structure. tion technology staff, multimedia development A product of machine learning research, as- specialists and managers/administrators, as sociation mining is used to find patterns of data participant groups in selecting a new learning where sets of attribute-value pairs occur frequently management system. in the data set. With association mining what class Evaluation of student use of LMS found that attribute should be in the results is not predeter- the LMS needs to be integrated with other educa- mined. Apriori (Agrawal, Imieliński, & Swami, tional activities and other aspects of everyday life. 1993) is the predominate association mining This is more important than the interface design. algorithm. It is an algorithm that produces many Students’ idea of what an LMS should provide rules, and special techniques are needed to reduce differs from the impression of the technical staff the rule set to those that are interesting. and other stakeholders who select and implement The final technique used in this chapter is the system. While students effectively use an LMS clustering. Clustering represents each attribute remotely this is only undertaken when there is a as a dimension and shows where data records practical advantage to the student, not just for some occur in this multidimensional problem space. educational benefit (Alsop & Tompsett, 2002). The most popular clustering algorithm is k-means Student learning style and its impact on learn- (MacQueen, 1967). Again, analysis of the clusters ing management systems is investigated by Graf needs special techniques. and Kinshuk (2006). Bayesian Networks have In an attempt to increase the robustness and been used to discover user preferences to allow reliability of rules, often a combination of data adaptation by the LMS to those preferences (Kri- mining methodologies are applied. Deshpande and tikou, Demestichas, Adamopoulou, Demestichas, Karypis (2002) and Padmanabhan and Tuzhilin Theologou, & Paradia, 2008). (2000) improved classification rules by first us- Instructors can use data mining techniques ing association mining. Li, Han, and Pei (2001) such as classification, association and cluster- classified records using CMAR (Classification ing on student LMS data to improve instruction. Based on Multiple Class-Association Rules), Clustering techniques can find groups of similar which identifies frequent patterns and associa- students so that the instructor can direct instruction tions between records and attributes. Jaroszewicz to the group’s particular needs classification will and Simovici (2004) employed user background identify the characteristics of the students in each knowledge and a Bayesian Network to determine group. Association mining may discover relation- the interestingness of sets of attributes prior to asso- ships between these characteristics and student ciation mining. Fu and Wang (2005) reduced data attributes (Romero, Ventura, & García, 2008). dimensionality using a separability-correlation There are a number of data mining techniques measure that ranks the importance of attributes to and associated algorithms. A common technique improve classification and the usefulness of rules. shared with statistics is classification analysis. A Scime and Murray (2007) and Murray, Riley, and decision tree model is constructed which finds a Scime (2007) used expert knowledge to reduce path to a predetermined class attribute for each data dimensionality while iteratively creating data record. A classification decision tree contains classification models. Rajasethupathy, Scime,

97 ANGEL Mining

Rajasethupathy, and Murray (2009) improved models determine the usage of each feature based the usefulness of rules by identifying “persistent on student count, department, level, and semester. rules,” which are those rules identified by both Association mining finds patterns in the data that classification and association mining. reoccur frequently. This is an effective method for finding features that co-occur. Association rule mining was preformed to find the relation- DATA MINING ANGEL DATA ships between the features of ANGEL. This could lead to acceptance or rejection of sets of features. Data mining is a process of inductively analyzing Clustering finds natural clusters occurring in data. data to find interesting patterns and previously These multi-dimensional clusters divide the data unknown relationships in the data. Data mining into groups; sufficiently large groups indicate the is used not only to predict the outcome of a future most important features in the data. In the ANGEL event but also to provide knowledge about the data a large cluster will indicate the most com- structure and interrelationships among the data. monly used features. The WEKA data mining tool The data mining process identifies relationships is used to apply the data mining techniques, and that are expressed as classification rules, associa- Microsoft’s Excel was used to prepare the data. tion rules, and clusters of records. Data mining algorithms are used to create models that describe Preprocessing existing data and relationships within the data. These methodologies create rules used to analyze Data often contains attributes and records that are new data and predict future outcomes. not pertinent to the analysis being conducted. In The data set used in this analysis contains in- the case of the ANGEL data set, it consisted of formation across two years. Data is recorded in the records from all classes offered by the college for ANGEL data base for every class offered during the Fall and Spring semesters from Fall of 2007 Fall, Spring, and Summer for every feature. This to Spring of 2009. For each of these classes the data consisted of class specific data, e.g. semester, feature usage was collected. This original data department, course number, and student count, and set contained 9430 records, one for each class feature data, e.g. feature (files, messages, folders section, and 20 attributes (features). etc.) and frequency of feature usage. The data set From this data set records for Thesis Continua- was processed using classification, association, tion Credit (TCC) classes were removed, because and clustering algorithms as implemented by the TCC classes are an extension of a previous thesis Waikato Environment for Knowledge Analysis course but were not identified by department. The (WEKA, Witten and Frank, 2005). information gained from these sections would The intent of this study is not to compare have been inconclusive since the original class ANGEL to other LMS but rather to assess the is not identified. The General Education Program usage of the ANGEL features by students and (GEP) class records with sections of 0 or 99 were instructors. Basing this study on recent historical also removed. These sections are used to place all data provides a good indication of the expected incoming transfer students in the computer skills ANGEL use in the near term future. The college’s exam, passing the exam is a graduation require- course offering and student and faculty profiles ment, but an actual class does not exist. are not expected to change in the next few years. Five attributes were removed from the data Classification mining determines the influence of set. The attribute identifying the section number combinations of attributes on a specific goal. In the was not relevant to the scope of our analysis. case of this ANGEL data, multiple classification Determining the difference between section 1

98 ANGEL Mining

and section 2 is trivial. The attributes indicating discreteized to nominal data using six bins with the use of wikis, games, blogs, and assessments equal depth binning. This method creates six bins features were removed due to their extremely low for each attribute where the number of records in usage. For example, assessments, which posted each bin is kept close to equal. Nominal data for the highest usage of the four, was only used by 64 student count used the college’s Common Data Set classes or .68% of total classes. After removal of from 2008-2009. This document binned student the records and attributes the data set was reduced count by the ranges 2-9 (very low), 10-19 (low), to 9312 department-course level- semester records 20-29(medium low), 30-39 (medium), 40-49 with 15 attributes (Table 1). (medium high), 50-99 (high), 100+ (very high). Some attributes had their values adjusted. The Classes of a single student are not specified in course number was changed to a nominal Level the Common Data Set, therefore they were as- number, for example 436 became L400, Semester signed a value of ‘single’. Preprocessing resulted was changed to a nominal value, for example in a data set characterized by department-course 200709 became S200709 for the Fall semester level-semester records with binned counts of 2007. Other attributes had values for the number feature usage. Beyond this data preparation, pre- of times the feature was used in the course. For processing done on the data set is specific to the all empty attribute values a zero (0) was inserted, data mining method. because the absence of a value meant the feature was used zero times for that class. These adjust- Classification Using Decision Trees ments created the base data set. The data mining algorithms required slight The goal of classification of the ANGEL data is variations from this base data set. For classifica- to examine the influence of semester, department, tion and association mining a nominal data set level, and student count on each feature. This was needed. The numerical feature data was requires classifying the data with respect to each

Table 1. Attributes

Name Description Department The academic department of the course Drop Box Number of drop boxes used to electronically collect assignments Files Count of files available for download Folders Number of folders used to organize content Form Number of surveys to be completed by students Forum Number of online discussions Grade Book Number of grade books (usually one) Links Number of URLs to resources outside the LMS Level The level of the course (e.g. 100, 200, 300, etc.) Messages Number of emails sent from within the course Pages Number of content pages available for students to view Quizzes Number of LMS administered tests Semester The semester in which the section was taught Student Count Number of students in the section TurnItIn Number of assignments collected and evaluated for originality

99 ANGEL Mining

feature individually. From the data set a feature recursively on all child nodes (Bagui, 2006). For specific data set was created consisting of semester, example, if given a data set consisting of both boys department, level, student count and one feature. and girls and their favorite sport, using entropy These data sets were processed individually to the algorithm selects gender as the first decision create feature specific decision trees. attribute producing two child nodes. The data Decision tree analysis creates a tree structure, set records are divided at this node. Each subset where each internal node denotes a test on an undergoes the same algorithmic process at the attribute, each branch represents an outcome of individual gender nodes to further divide the the test, and every leaf represents a class or class records. This process continues until only pure distribution (Bagui, 2006). Decision tree analysis nodes occur, creating leaves. allows for easy conversion to classification rules. An example of a completed classification de- A decision tree starts as a single node. If all cision tree is presented in Figure 1. This generic records in that node are the same it becomes a leaf. decision tree has four nodes or points at which If they are not all the same, a selection algorithm decisions are made. A record would be classified uses entropy, a measure of the inconsistency of depending on the values of its attributes into one the data, to decide how to divide the records. An of the leaf nodes, where the class attribute has attribute is chosen that best divides the records a specific value. In this example the possible into further purer nodes. The attribute chosen is class attribute values are w, x, y, and z. A record labeled as the decision attribute. Branches are with the attribute values (Attr1 = a then Attr2 = then created for the known values of the decision 4) would be classified as the class attribute with attribute, and the records are divided accordingly the value x. This record is classified by follow- into the child nodes. This process is performed ing the Attr1- Attr2 edge of the tree and then the

Figure 1. Generic decision tree

100 ANGEL Mining

Attr2-ClassAttr=x edge, reaching a ClassAttr=x Information gain is the change in information leaf. Other records may also reach a leaf with entropy from the current state of the set of records ClassAttr=x (e.g., Attr1 = c then Attr3 = g then to the proposed state of the set of records. Entropy Attr4 = s), but via a different path. is a measure of the randomness of the distribution After the decision tree is constructed, each of records in a subset of records with respect to branch of the decision tree is converted into a the class attribute. rule. The tree presented in Figure 1 can be con- The attribute with the greatest information gain verted into the following Rules: is selected as the node for dividing the data in the decision tree at each node. The information gain • Rule 1: IF Attr1 = a AND Attr2 ≤ 5 THEN with the overall highest gain with respect to the ClassAttr = x dependent attribute is the root of the tree. • Rule 2: IF Attr1 = a AND Attr2 > 5 THEN The confidence that a record is correctly ClassAttr = y classified provides the accuracy of splitting the • Rule 3: IF Attr1 = b THEN ClassAttr = z record set. Confidence is calculated by dividing • Rule 4: IF Attr1 = c AND Attr3 = g AND the number of records correctly classified by the Attr4 = s THEN ClassAttr = x total number of records that are classified at the • Rule 5: IF Attr1 = c AND Attr3 = g AND node. Confidence is used to prune the decision Attr4 = t THEN ClassAttr = w tree, ensuring a tree with a reasonable number of • Rule 6: IF Attr3 = h THEN ClassAttr = y leaf nodes and acceptable accuracy. The level of pruning is controlled by setting a minimum con- The rules provide insight into how the class fidence level. If a node falls below the minimum attribute’s value is, in fact, dependent on the other confidence that branch is pruned; that is, the attributes. A complete decision tree provides for records are rolled up to the parent node until the all possible combinations of the attributes and minimum confidence is met This allows for only their allowable values reaching a single, allowable rules that meet the required accuracy to remain class attribute value. on the final tree. The decision tree algorithm C4.5 builds the The data set was divided into training and test decision tree from a training data set using infor- sets, where the training set contained two-thirds mation entropy (Quinlan, 1993). The goal of the of the records, randomly selected. The remaining algorithm is to classify all the training set records records are in the test set. The decision tree was according to the goal or class attribute. The algo- constructed using the training set and validated rithm first determines the attributes available in the with the test set. To determine the appropriate first node. The normalized information gain with confidence level multiple trees were constructed respect to the class attribute is calculated for each using individual features as the class attribute. attribute to determine how the records should be Evaluation of multiple decision trees is common split into subsets. The attribute with the highest in classification problems (Osei-Bryson, 2004). normalized information gain is selected as the After several trials of varying confidence, 55% decision attribute. The records are then broken into confidence was selected. This choice was deter- branches on the values of the decision attribute, mined after trials focusing on minimizing the creating branches of the tree ending in child nodes. percent incorrectly classified, while maintaining This process is performed recursively (Quinlan, quality leaf nodes. However, the direct relationship 1993). The class attribute and its values constitute of these features needed to be taken into account. the leaf nodes of the decision tree. As the number of leaf nodes decreases, the num-

101 ANGEL Mining

ber of incorrectly classified records decreases, istering a test or having students turn in a paper, as well as the quality of rules formed from the have usage that is directed more by the department nodes decreasing. and level than the class size. This suggests users The goal of classification of ANGEL data is who are more willing to experiment with the new to examine the influence of semester, department, technology create the demand. Those users then level, and student count on a feature. To accomplish have an effect on their colleague’s use of that this task, our fully nominal data set was broken feature. Whereas, with the more commonly used into eleven data sets, which in turn were divided online features, such as e-mail messages or file into training and test sets. Each data set consisted sharing, usage is directed by need. This suggests of semester, department, level, student count and that the more students in a class, the more likely one feature, each having a different feature. For a professor is to post files rather than print the example, semester, department, level, student files to hand out in class. count, and file. Dividing the data set into these These findings support Nichols’ (2003) hypoth- smaller data sets allowed each feature to be a esis that the movement to eLearning, including the class attribute. Records with zero values for the use of LMS, is an evolutionary, not revolutionary class (feature) attribute were removed from the process. As success is seen in one course where respective feature data set. The C4.5 algorithm an LMS is used the LMS will be adopted in other, was executed on each data set with a confidence similar courses. level of 55%. The decision tree is converted into IF-THEN Association Rule Mining rules. Each rule represents one branch of the decision tree from the root node to a leaf node. The The goal of association mining the ANGEL data intermediate nodes provide additional information was to evaluate each feature’s affect on the usage in the form of branching on the branches. There of the other features. This required modification to is one rule for each of the leaf nodes of the tree. the original data set. Non-feature attributes were The results of the classifications followed a removed so that patterns of feature usage would pattern. For the top five features (messages, files, show the use of one feature with another in the folders, grade book, and links), the attribute with association rules. the highest information gain was student count Association rule mining, unlike classification followed by department. From this several rules rule mining, does not need a class attribute in order were formed that concluded that as class size in- to find meaningful results. Association mining al- creased the usage of the feature increased. Figure lows for rules to be generated for any combination 2 provides a sample of the rules found. of attributes within the data set. Association Rules For the little used features (drop box, form, are not intended to be used together (Witten and forum, page, quiz and turn it in) the attribute with Frank, 2005). Each rule generated in association the highest information gain was first department mining implies a different regularity within the followed by level. From this classification a data set, and each could result in a very different definite conclusion pertaining to all classes cannot conclusion. With the ANGEL data, relationships be made. Several departments have lower level are found between the different features used for classes with heavier users, while other departments a class on campus. have higher level classes with the heavier users To construct association rules itemsets found (See Figure 3 for rule examples). in the data set are used. An itemset is a collection From classification it is found that features of attribute-value pairs (items) that occurs in the with a less traditional online use such as, admin- data set. Each itemset can be converted into a

102 ANGEL Mining

Figure 2. Sample classification rules of top features

number of rules, where each item in an itemset Rules that come from the same itemset all have implies and is implied by every other item and the same support value. A rule contains a premise combination of items in the itemset. This results (IF-part) and consequent (THEN-part) and states in a very large number of rules. that when the premise is true the consequent will Since association rule mining results in a very be true. The confidence that this rule is correct is high number of rules, restrictions need to be ap- also calculated. The confidence is a conditional plied to specify whether a rule is significant and probability that a record containing the premise valid. The metrics of support and confidence are will also contain the consequent. It is calculated used to determine the significance of a rule. The by dividing the support for the rule by the support support of an itemset is the count or percent of just for the itemset that is the same as the prem- records that contain all the items in the itemset. ise. Only those rules meeting the user-specified A support threshold is set which must be met or confidence threshold are kept. Each rule is an exceeded for a itemset to be considered a frequent observation of the data’s behavior. itemset. In the example above the support count The algorithm chosen for this investigation threshold was set to two. is the Apriori algorithm. The Apriori algorithm uses itemsets to generate rules, and support based

103 ANGEL Mining

Figure 3. Sample classification rules for forums

pruning to help control the growth of rules. It also level, it gets discarded as well as any rule that is uses the Apriori Principle. This principle states a subset of that rule. that if an itemset is frequent, then all of its subsets The goal of association mining the ANGEL data must also be frequent (Agrawal, Imieliński, & was to evaluate a feature’s affect on the usage of Swami, 1993). one or more other features. To create the data set The Apriori algorithm first considers the single necessary to achieve this goal several attributes itemsets, those with one only item and counts (semester, department, level and student count) how frequently the item appears in the data. The were removed from the original data set. These itemsets that do not meet the minimum support attributes were removed to generate only rules threshold are discarded from the possible one pertaining to the use of one feature’s influence on itemsets. The Apriori Principle ensures that all another. For example, the use of folders indicates supersets of the one itemsets that are infrequent, are the use of files. also infrequent. A list of two itemsets is generated The minimum value for support was set to from the list of one itemsets. The frequency of the 0.7, the value for confidence was set to 0.9, and two itemsets is determined and those not meet- the number of rules was set at 3000. Support and ing the minimum support threshold are removed. confidence were set at such high values to ensure Three itemsets are created from the two itemsets presentation of rules with certainty. The number and the process continues until no more frequent of rules was set so as to be exhaustive and find all itemsets can be found. Once all of the frequent possible rules within the bounds of support and itemsets are generated, the rules are generated. confidence. The algorithm generated 2,638 rules. If a rule does not meet the minimum confidence

104 ANGEL Mining

The useful rules can be defined by applying the number of messages and emails do not influ- a template (Klemettinen et al. 1994). Only those ence the feature usage. rules where the predicate consisted of a single In association mining student Moodle activity feature were kept. The rules then had the appear- in a course, Romero, Ventura, and García (2008) ance of, if attribute X is used, then these other found that students that do not send messages attributes are used. These rules show how using (or email) do not read them either, which then one feature of the ANGEL system influences us- leads to course failure. While there is no rela- age of other features. tionship between using the message components However, only rules where the feature count of ANGEL clearly use of a LMS features, while was zero were found. Nevertheless, this still is an not ensuring a passing grade, does mitigate the interesting, and meaningful result. The rules now likelihood of failure. have the form, if you do not use this feature, then you will not use these other features. In order to Cluster Rule Mining reduce the number of rules further, if there existed a rule X where the consequence was a subset of The goal in clustering the ANGEL data set was to the consequence of rule Y, then X was removed provide an indication of the most used features in (Rajasethupathy, Scime, Rajasethupathy, & Mur- larger classes and therefore by the most students. ray, 2009). After the filtering of rules, 37 rules Additionally, indications of infrequent use of some remained. (Appendix A). features, while identifying where these features Inspecting the rules from association rule are used, is found in smaller clusters. mining leads to a few general conclusions. Given In conventional terms, clustering is when you the level of confidence and support in this study, group similar objects together. In data mining, this the use, or non-use, of email messages or files definition still holds. Clustering in data mining is uploaded to ANGEL have no affect on the other taking similar records and grouping them together features. There is no correlation between those into clusters based on a measurable distance two features and any of the others. One might as- between them. Once these clusters are created, sume that if a particular class on ANGEL uploads similarities can be determined within the data a large number of files that this high frequency of set that may not have been previously apparent. usage could be seen with other features, however In clustering each attribute is a dimension in this investigation did not find that to be the case. the problem space. Each record is placed as a point Another conclusion is that if a class does not use in the problem space based on the values of the the drop box feature, then they will not use any record’s attributes. The process of clustering is feature, or any combination of features. When based on comparing the records, calculating the rules were being purged, there existed only one distance between them, and then grouping like rule with drop box in the predicate. This rule can records together. The collection of clusters (known be used as a metric for ANGEL usage. (Note: it is as a clustering) has clusters consisting of records not logically correct to say that if they do use the that are very similar to records inside the cluster, drop box feature then they will use other features.) but dissimilar to those within other clusters. At the theoretical level, García, Romero, The goal of clustering is to indentify common Ventura, and Calders (2007) determined that in feature groupings found in ANGEL, as well as, association mining critical factors in determin- the count of students associated with them. The ing usage are the number of messages, emails, algorithm used for clustering for this investiga- documents, and Web pages on the course site. tion is k-means. Since clustering is graphical However, the findings of this study indicate that the data must be fully numeric. Therefore, the

105 ANGEL Mining

pre-discretized data set was used. This data set The goal in clustering the data set was to find contained numerical data for feature usages and natural feature groupings in the data set and the student count. The nominal attributes (semester, class size associated with that grouping. In the department, and level) were removed. 23 cluster clustering, five clusters were formed The k-means algorithm requires an input for the that contained 72% of the records, and revealed number of clusters (k). The algorithm randomly files and messages as the only features used by selects k points to be the initial cluster centroids. those clusters. Furthermore, clustering revealed The records are formed into these k clusters based seven clusters totaling 3.7% of records, that used on to which centroid a given record is closest, nine to ten of eleven features. Lastly, the largest closeness being determined by the distance mea- cluster holding 24% of records contained an av- sure. The centroid of each cluster is recalculated erage student count of 1.78 and average of 0 to using the records in the cluster. 0.45 for feature use. That cluster formed the rule The distance from each record to each centroid that if a class has two or less students they will is recalculated and records may change clusters. not use ANGEL. This process continues until the difference between The results received by clustering show that the current centroid and the previous centroid is the most commonly used features in ANGEL are zero or near zero for each cluster. files and messages. Most classes do not use the The value of k determines the number of more advanced features in ANGEL, yet there is clusters, varying k results in different cluster- a small subset of instructors that heavily use ad- ings or grouping of the records. To determine vanced features. Finally, classes with two or less the optimal clustering the sum of squared errors students do not use any feature in ANGEL; this (SSE) are compared to the number of clusters. As may imply that it is not worth the instructor’s time the number of clusters increases the SSE should to create content for just a few students. decrease, when the SSE begins to remain nearly Romero, Ventura, and García’s (2008) study constant the smallest number of clusters in typi- of types of Moodle students found three clusters cally chosen. defining types of students: very active, active, and This process was performed on the ANGEL non-active. Very active students are characterized data set. Fifteen clusterings were computed, as having sent more than one message, read about increasing k by two from two to thirty. For each three messages, passed a large number of quizzes, analysis of k the SSE was recorded and graphed and spent time on Moodle doing assigned work. (Figure 4). Non-active students did no assignments, read few From the graph, 23 clusters were chosen for messages, completed very few quizzes, and did not cluster rule mining. This cluster size was chosen spend time on Moodle. Active student’s behavior as it is the point where the graph levels out. The was between the other two groups. While messages area of the graph where leveling occurs is con- are one of the most commonly used features in sidered to be optimal for cluster size because once ANGEL, it is up to the individual student to define the graph begins to level off the measure of SSE themselves as very active, active, or non-active. from one k to the following k+2 is less meaningful. Therefore, the analysis is producing sub optimal clusters, or with less meaningful information FUTURE RESEARCH DIRECTIONS than their predecessor. For example, in our data set using twenty-six clusters would not produce Today, the use of the computer has become com- better results. It would be more likely to divide mon for statistical analysis of data. Software pack- good clusters. ages are easy to use, inexpensive, and fast. But

106 ANGEL Mining

Figure 4. Number of clusters vs. SSE

today’s vast stores of data with immense data sets of data types. Data mining research is being con- make comprehensive analysis all but impossible ducted to find interesting patterns in data sets of using conventional techniques. A solution is data all these data types. Beyond the LMS data, other mining. Classification mining finds the rules that data sources may supplement the LMS data. Data can predict future behavior. Association mining from corporate and government data warehouses, discovers patterns in the data. Clustering provides transactional databases, and the World Wide Web insight into the data. (Scime, 2008) may be added to enhance the LMS. Currently data mining requires some special- The data mining of an LMS of the future will ized skill – the ability to understand the processes, be multidimensional, accessing all these data the data, and expertise in the domain. In the case sources and data types to find the optimal use of of Learning Management Systems, how the system a school’s LMS. operates and instruction techniques may be neces- In addition to the new forms of data that will sary to understand the results. This combination need to be mined in future iterations of the LMS, of skills is difficult to find in a single person. In the very nature of the LMS is likely to see signifi- the future data mining tools will become easier to cant changes in approaching years. As colleges use, as statistical and spreadsheet tools are today. and universities become more attune to the needs Then domain experts will be able to apply data of both regional (Middle States) and program ac- mining to their data without the need for a data creditation (NCATE, AACSB, ABET) the usage of mining expert. The rules found and conclusions the LMS for purely learning or course management drawn can then be used with more confidence activities is likely to transform to put greater em- when making decisions within the domain. phasis on assessment and accreditation activities. Data mining is commonly conducted against Where the use of specific system features has not transactional data, but data have gone beyond been traditionally mandated, it is likely that there simple numeric and character flat file data. Fu- could be a shift to the mandated use of features ture LMS data will come in many forms: image, such as rubrics and assessment tools. video, audio, streaming video, and combinations

107 ANGEL Mining

With this potential shift comes the opportunity CONCLUSION for data mining to expand beyond evaluating just the usage of the LMS but to also provide deep In this investigation, statistical methods were used insights into the performance of students against to determine whether or not features were being institutional and program objectives. Correlating used in the ANGEL system. Data mining was the use of LMS features with student performance able to generate results, and from these results could provide the opportunity to enhance the conclusions were made from the three different use of the LMS to provide stronger educational data mining techniques; classification, association, outcomes. This in turn, could provide guidance and clustering. in improving college and university programs to A few general conclusions can be made, the produce stronger and more competitive classes majority of classes at this college do not use the of graduates. more advanced features of ANGEL. The only Finding more interesting and supportive rules parts of the system being used by a large number for a domain using multiple methods places con- of classes are the messaging and file sharing straints on the mining process. This is a form of features. Another conclusion is that there exists constraint-based data mining. Constraint-based a relationship between need and use. If there are data mining uses constraints to guide the process. a large number of students, then using ANGEL to Constraints that can be used can specify the data send messages or share files alleviates the instruc- mining algorithm. Constraints can be placed on tor from a certain amount of additional effort. For the type of knowledge that is to be found or the example, instead of printing off enough copies for data to be mined. Dimension-level constraints the class, the file is shared on ANGEL. Whereas, research is needed to determine what level of a classes with a low student count do not use this summary, or the reverse, detail is needed in the feature as frequently. Association suggests that data before the algorithms are applied (Hsu, 2002). there is not a relationship between the use of one Research in data mining will continue to find feature to the use of another feature. However, new methods to determine interestingness. Re- association does conclude the non-use of features search is needed to determine what values of a implies the non-use of other features. Furthermore, particular attribute are considered to be especially during pre-processing, it was found that there interesting in the data and in the resulting rule are features that are not being used at all (wikis, set (Hsu, 2002). Currently there are 21 different blogs, assessments, and games). Nevertheless, it statistically based objective measures for deter- has been found that there are a small number of mining interestingness (Tan, Steinbach & Kumar, instructors who are heavy ANGEL users. These 2006). A leading area of research is to find new, users often utilize many of the features available. increasingly effective measures. With regard to These results can be used by this institution, as subjective measures of interestingness, research in well as similar institutions, for decision making domains that is both quantitative and qualitative concerning feature selection and overall usefulness can lead to new methods for determining inter- of LMS design, selection and implementation or estingness. Further data mining research will find to identify feature areas needing additional train- new methods to support existing knowledge and ing in their use. perhaps find new knowledge in domains where it has not yet been applied (Scime, Murray & Hunter, 2010).

108 ANGEL Mining

REFERENCES Deshpande, M., & Karypis, G. (2002). Using conjunction of attribute values for classification. Agrawal, R., Imieliński, T., & Swami, A. (1993). Proceedings of the 11th International Conference Mining association rules between sets of items in on Information and Knowledge Management, large databases, Proceedings of the 1993 ACM McLean, VA (pp. 356-364). SIGMOD International Conference on Manage- ment of Data, Washington, DC. (pp.207-216). Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United Alsop, G., & Tompsett, C. (2002). Grounded States. TechTrends, 15(2), 40–45. theory as an approach to studying students’ uses of learning management systems. ALT-J Feldstein, M., & Masson, P. (2006). Unbolting Research in Learning Technology, 10(2), 63–76. the chairs: Making learning management systems doi:10.1080/0968776020100207 more flexible. E-learn magazine. Retrieved from http://www.elearnmag.org/subpage.cfm?section= Avgeriou, P., Papasalouros, A., Retalis, S., & tutorials&article=22-1 on April 19, 2010. Skordalakis, M. (2003). Towards a pattern language for learning management systems. Journal Fu, X., & Wang, L. (2005). Data dimensionality of Educational Technology & Society, 6(2), 11–24. reduction with application to improving classification performance and explaining concepts of data Bagui, S. (2006). An approach to mining crime sets. International Journal of Business Intelligence patterns. International Journal of Data Ware- and Data mining, 1(1), 65-87. housing and Mining, 2(1), 50–80. doi:10.4018/ jdwm.2006010103 García, E., Romero, C., Ventura, S., & Calders, T. (2007). Drawbacks and solutions of applying Beatty, B., & Ulasewicz, C. (2006). Faculty Association Rule mining in learning manage- perspectives on moving from blackboard to the ment systems. Proceedings of the International moodle learning management system. TechTrends, Workshop on Applying Data mining in e-Learning, 50(4), 36–45. doi:10.1007/s11528-006-0036-y Crete, Greece (pp. 13-22). Blackboard (2010). Blackboardlearn+ ANGEL Graf, S., & Kinshuk (2006). Considering learn- edition. Retrieved on April 19, 2010, from http:// ing styles in learning management systems: In- www.blackboard.com/Teaching-Learning/Learn- vestigating the behavior of students in an online Resources/ANGEL-Edition.aspx course. Proceedings of the 1st IEEE International Chapman, B. (2010). LMS KnowledgeBase 2010: Workshop on Semantic Media Adaptation and In-depth profiles of 90+ learning management Personalization, Athens, Greece (pp. 25-30). systems, with custom comparison across 200+ Haldane, A. (1998). The convergence of e-learning features. Brandon Hall Research, Retrieved on and knowledge management: Fusion or confusion? November 9, 2010, from http://www.brandon-hall. People development in a knowledge economy, 2. com/publications/lmskb/ lmskb.shtml Common- Learning Futures Ltd., Retrieved on April 26, 2010 wealth of Learning (2004) LMS Evaluation Tool from http://www.learningfutures.co.uk/LFdocs/ User Guide, 1.2, 3waynet Inc. and the Common- MKelleher_06-01-03_15-29-27.pdf. wealth of Learning. Hall, J. (2003). Assessing learning management systems. Chief learning officer. Retrieved on April 19, 2010, from http://www.clomedia.com/ features/2003/January/91/index.php.

109 ANGEL Mining

Hsu, J. (2002). Data mining trends and develop- Li, W., Han, J., & Pei, J. (2001). CMAR: Accu- ments: The key data mining technologies and rate and efficient classification based on multiple applications for the 21st century, Proceedings class-Association rules. Proceedings of the 2001 of the19th Annual Conference for Information IEEE International Conference on Data mining Systems Education (ISECON 2002), San Antonio, (pp. 369-376). TX (Art 224b). MacQueen, J. (1967). Some methods for classifi- Jaroszewicz, S., & Simovici, D. A. (2004). Inter- cation and analysis of multivariate observations. estingness of frequent itemsets using Bayesian Proceedings of the 5th Berkeley Symposium on networks as background knowledge. Proceedings Mathematical Statistics and Probability (pp. of the 10th ACM SIGKDD International Confer- 281-297). ence on Knowledge Discovery and Data mining Malikowski, S. R., Thompson, M. E., & Theis, (pp. 178-186). Seattle, WA. J. G. (2007). A model for research into course Kim, S. W., & Lee, M. G. (2008). Validation management systems: Bridging technology and of an evaluation model for learning manage- learning theory. Journal of Educational Comput- ment systems. Journal of Computer Assisted ing Research, 36(2), 149–173. doi:10.2190/1002- Learning, 24(4), 284–294. doi:10.1111/j.1365- 1T50-27G2-H3V7 2729.2007.00260.x Moore, M. G. (1993). Theory of transactional Klemettinen, M., Mannila, H., Ronkainen, P., distance. In Keegan, D. (Ed.), Theoretical prin- Toivonen, H., & Verkamo, A. I. (1994). Finding ciples of distance education (pp. 33–38). New interesting rules from large sets of discovered York, NY: Routledge. association rules. Proceedings of the 3rd Interna- Murray, G. R., Riley, C., & Scime, A. (2007, May). tional Conference on Information and Knowledge A new age solution for an age-old problem: Mining Management (CIKM’94) Gaithersburg, Maryland, data for likely voters. Paper presented at the 62nd USA (pp. 401-408). Annual Conference of the American Association Kritikou, Y., Demestichas, P., Adamopoulou, of Public Opinion Research, Anaheim, CA. E., Demestichas, K., Theologou, M., & Paradia, Nichols, M. (2003). A theory for elearning. Journal M. (2008). User profile modeling in the context of Educational Technology & Society, 6(2), 1–10. of Web-based learning management systems. Journal of Network and Computer Applications, Osei-Bryson, K.-M. (2004). Evaluation of deci- 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 sion trees: A multicriteria approach. Computers & Operations Research, 31(11), 1933–1945. ANGEL Learning (2008). The ANGEL learning doi:10.1016/S0305-0548(03)00156-4 management suite. Padmanabhan, B., & Tuzhilin, A. (2000). Small Lewis, B. A., MacEntee, V. M., DeLaCruz, S., is beautiful: Discovering the minimal set of un- Englander, C., Jeffrey, T., & Takach, E. …Ja- expected patterns. Proceedings of the 6th ACM son W. (2005). Learning management systems SIGKDD International Conference on Knowledge comparison. Proceedings of the 2005 Informing Discovery and Data mining, Boston, MA (pp. Science and IT Education Joint Conference, 54-63). Flagstaff, Arizona.

110 ANGEL Mining

Paulsen, M. F. (2003). Online education and Sturgess, P., & Nouwens, F. (2004). “Evaluation learning management systems. Bekkestua, Nor- of online learning management systems. Turkish way: NKI. Online Journal of Distance Education, 5(3). Quinlan, J. R. (1993). C4.5: Programs for machine Tan, P.-N., Steinbach, M., & Kumar, V. (2006). learning. San Francisco, CA: Morgan Kaufmann. Introduction to data mining. Boston, MA: Ad- dison Wesley. Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Murray, G. R. (2009). Finding “persistent Watson, W. R., & Watson, S. L. (2007). An argu- Rules”: Combining association and classifica- ment for clarity: What are learning management tion results’, Expert Systems With Applications, systems, what are they not, and what should they 36(3P2), 6019-6024. become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques Romero, C., Ventura, S., & García, E. (2008). (2nd ed.). San Francisco, CA: Morgan Kaufman. Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016 ADDITIONAL READING

Roqueta, M. (2008). Learning management sys- Anand, S. S., Bell, D. A., & Hughes, J. G. (1995). tems: A focus on the learner. Distance Learning, The role of domain knowledge in data mining. 5(4), 59–66. Proceedings of the 4th International Conference Scime, A. (2008). Web page extension of data on Information and Knowledge Management, warehouses. In Wang, J. (Ed.), Encyclopedia Baltimore, MD (pp. 37-43). of Data Warehousing and Mining (2nd ed., pp. Andoh-Baidoo, F. K., & Kweku-Muata, O.-B. 1211–1215). Hershey, PA: Idea Group Reference. (2007). Exploring the characteristics of internet Scime, A. & Murray, G. R. (2007). Vote predic- security breaches that impact the market value of tion by iterative domain knowledge and attribute breached firms. Expert Systems with Applications, elimination. International Journal of Business 32(3), 703–725. doi:10.1016/j.eswa.2006.01.020 Intelligence and Data mining, 2(2), 160-176. Ankerst, M., Martin, E., & Kriegel, H.-P. (2000). Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Towards an effective cooperation of the user and Testing terrorism theory with data mining. Inter- the computer for classification. Proceedings of national Journal of Data Analysis Techniques the 6th ACM SIGKDD International Conference and Strategies, 2(2), 122–139. doi:10.1504/ on Knowledge Discovery and Data mining (pp. IJDATS.2010.032453 179-188). Boston, MA. Sclater, N. (2008). Large-scale open source Bremer, D., & Bryant, R. (2005). A comparison e-learning systems at open university UK. Re- of two learning management systems: Moodle search Bulletin EDUCAUSE Center for Applied vs blackboard. Proceedings of the 18th Annual Research, 2008(12), 1-13. Conference of the National Advisory Commit- tee on Computing Qualifications (pp. 135-139). Tauranga, NZ.

111 ANGEL Mining

Brusilovsky, P. (2003). A distributed architecture Li, J., Tang, J., Li, Y., & Luo, Q. (2009). RiMOM: A for adaptive and intelligent learning managament dynamic multistrategy ontology alignment frame- systems. Proceedings of the AIED 2003 Workshop work. IEEE Transactions on Knowledge and Data Towards Intelligent Learning Management Sys- Engineering, 21(8), 1218–1232. doi:10.1109/ tems (pp. 5-13). Sydney, Australia. TKDE.2008.202 Carliner, S. (2005). Course management systems Machado, M., & Tao, E. (2007). Blackboard vs. versus learning management systems. Learning Moodle: Comparing user experience of learning Circuits, Retrieved from http://www.astd.org/ management systems. Proceeding of the 37th LC/2005/1105 _carliner.htm on April 19, 2010. ASEE/IEEE Frontiers in Education Conference, (pp. S4J-7 - S4J-12). Milwaukee, WI. Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning Magidson, J. (1988). Improved statistical management systems on university teaching and techniques for response modeling. Journal of learning. Tertiary Education and Management, Direct Marketing, 2(4), 6–18. doi:10.1002/ 11(1), 19–36. doi:10.1080/13583883.2005.996 dir.4000020404 7137 Magidson, J. (1994). The CHAID approach to Giarratano, J. C., & Riley, G. D. (2004). Expert segmentation modeling: Chi-squared automatic systems: Principles and programming (4th ed.). interaction detection. In Bagozzi, R. P. (Ed.), New York: Course Technology. Advanced Methods of Marketing Research. Cam- bridge, MA: Basil Blackwell. Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. Boston, MA: Morgan Memon, N., & Qureshi, A. R. (2005). Investigative Kaufmann. data mining and its application in counterterror- ism. Proceedings of the 5th WSEAS International Hofmann, M., & Tierney, B. (2003). The involve- Conference on Applied Informatics and Commu- ment of human resources in large scale data min- nications (pp. 397-403). Malta. ing projects. Proceedings of the 1st International Symposium on Information and Communication Paulsen, M. F. (2003). Experiences with learning Technologies (pp.103-109). Dublin, Ireland. management systems in 113 european institutions. Journal of Educational Technology & Society, Kass, G. (1980). An exploratory technique for in- 6(4), 134–148. vestigating large quantities of categorical data. Ap- plied Statistics, 29, 119–127. doi:10.2307/2986296 Phelps, C., & Michea, Y. (2003). Learning management systems’ evaluation focuses on technology Kumar, D. A., & Ravi, V. (2008). Predicting credit not learning. AMIA Annual Symposium Proceed- card customer churn in banks using data mining. ings, (pp. 969). International Journal of Data Analysis Tech- niques and Strategies, 1(1), 4–28. doi:10.1504/ Ramu, K., & Ravi, V. (2008). Privacy preservation IJDATS.2008.020020 in data mining using hybrid perturbation methods: An application to bankruptcy prediction in banks. Lane, T., & Brodley, C. E. (2003). An empirical International Journal of Data Analysis Techniques study of two approaches to sequence learning and Strategies, 1(4), 313–331. doi:10.1504/IJ- for anomaly detection. Machine Learning, 51(1), DATS.2009.027509 73–107. doi:10.1023/A:1021830128811

112 ANGEL Mining

Robinson, S. M. (2008). Moodle vs. WebCT: Wang, G., Zhang, C., & Huang, L. (2008). ‘A How the design of features affect the efficiency of study of classification algorithm for data mining e-learning by influencing interaction and collabo- based on hybrid intelligent systems. Proceed- ration. Retrieved from http://web.njit.edu/~sr89/ ings of the 9th ACIS International Conference MoodlevsWebct.doc on April 19, 2010. on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing Roiger, R., & Geatz, M. W. (2003). Data mining: (pp. 371-375). Phuket Thailand. A tutorial-based primer. New York, NY: Addison Wesley. Wang, Y., Zhang, Y., Xia, J., & Wang, Z. (2008). Segmenting the mature travel market by motiva- Sclater, N. (2008). Web 2.0, personal learning tion. International Journal of Data Analysis Tech- environments, and the future of LMS. Research niques and Strategies, 1(2), 193–209. doi:10.1504/ Bulletin EDUCAUSE Center for Applied Re- IJDATS.2008.021118 search, 2008(13), 1-13. Webb, G. I., & Zheng, Z. (2004). Multistrategy Su, X., Khoshgoftaar, T. M., & Greiner, R. (2009). ensemble learning: Reducing error by combining Making an accurate classifier ensemble by vot- ensemble learning techniques. IEEE Transactions ing on classifications from imputed learning on Knowledge and Data Engineering, 16(8), sets. International Journal of Information and 980–991. doi:10.1109/TKDE.2004.29 Decision Sciences, 1(3), 301–322. doi:10.1504/ IJIDS.2009.027657 Tozicka, J., Rovatsos, M., & Pechoucek, M. KEY TERMS AND DEFINITIONS (2007). A framework for agent-based distributed machine learning and data mining. Proceedings Association Mining: A data mining method of the 6th International Joint Conference on used to find patterns of data that show conditions Autonomous Agents and Multiagent Systems (art where sets of attribute-value pairs occur frequently 96). Honolulu, Hawaii. in the data set. Turban, E., McLean, E., & Wetherbe, J. (2004). Association Rule: A rule found by associa- Information Technology for Management (3rd tion mining. ed.). New York, NY: Wiley. Classification Mining: A data mining method used to find models of data for categorizing in- Vatsavai, R. R., & Bhaduri, B. (2007). A hybrid stances; typically used for predicting future events classification scheme for mining multisource from historical data. th geospatial data, Proceedings of the 7 IEEE Inter- Classification Rule: A rule found by clas- national Conference on Data mining Workshops sification mining. (ICDMW 2007) (pp. 673-678). Omaha, NE. Clustering: A data mining method used to Wagstaff, K., Cardie, C., Rogers, S., & Schrödl, group similar records together based on a measur- S. (2001). Constrained k-means clustering with able distance between the records. background knowledge. Proceedings of the 18th Clustering Rule: A rule found by the analysis International Conference on Machine Learning of clusters. (pp.577-584). Williamstown, MA. Data Mining: A collection of processes that inductively analyze data to assess known relationships as well as to find interesting patterns and unknown relationships.

113 ANGEL Mining

APPENDIX A: ASSOCIATION MINING RULES

countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6545 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6502 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 6496 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 6493 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 6463 conf:(0.97) countFolder=(-inf-0.5] 6649 ==> countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6463 conf:(0.97) countDropBoxes=(-inf-0.5] 8366 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7573 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countTurnItIn=(-inf-0.5] 8306 conf:(0.92) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countPage=(-inf-0.5] 8184 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countTurnItIn=(- inf-0.5] 8161 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] 8137 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 8130 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 8032 conf:(0.91) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countDropBoxes=(-inf-0.5] countQuiz=(- inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6713 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countLink=(- inf-0.5] countTurnItIn=(-inf-0.5] 6704 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countForm=(- inf-0.5] countLink=(-inf-0.5] 6694 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countQuiz=(-inf-0.5] countForm=(- inf-0.5] countLink=(-inf-0.5] countTurnItIn=(-inf-0.5] 6684 conf:(0.9)

114 ANGEL Mining

countLink=(-inf-0.5] 7955 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7312 conf:(0.92) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] 7206 conf:(0.91) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7198 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7197 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countPage=(-inf-0.5] 7170 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7164 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.93) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7744 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7714 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] 7682 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countTurnItIn=(-inf-0.5] 7668 conf:(0.9) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7963 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 8306 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 8278 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 8202 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countDropBoxes=(-inf-0.5] countForm=(- inf-0.5] 8161 conf:(0.9)

115 116

Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Kamla Ali Al-Busaidi Sultan Qaboos University, Oman

Hafedh Al-Shihi Sultan Qaboos University, Oman

ABSTRACT Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’ supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS.

DOI: 10.4018/978-1-60960-884-2.ch006

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

INTRODUCTION students or promoting community (Woods, Baker & Hopper., 2004). In addition, research indicates Information and communication technologies that fear of technology and lack of time may limit (ICT) and the Internet have become major enablers instructors’ adoption of LMS (Yueh & Hsu, 2008). of growth in business. The geographical outreach Instructors should embrace and prepare for LMS of the Internet and the wide global adoption of Web use before preparing students for online learning 2.0 technologies provide educational institutions (Chan, 2008). Even when designing LMS appli- with unprecedented opportunities to enhance their cations, teachers’ needs and capabilities should offerings. These technologies have transformed thoroughly be investigated (Yueh & Hsu, 2008). students’ perception of information and what they Therefore, teachers’ perspective within the context think about Web content and how to use it (Burnett of LMS is crucial and should be studied carefully & Marshall, 2003). Schools and universities are to ensure comprehensive uptake of LMS. forced to investigate new means to revamp the Thus, the objective of this study was to inves- educational process utilizing these technologies. tigate the critical factors influencing instructors’ Learning Management Systems (LMS) and acceptance and use of LMS, which may be influ- e-learning have become the hype lately among enced by technical and non-technical issues such stakeholders in education and training. The e- as the instructors’ personal characteristics and the learning market was worth more than US $18 organization’s characteristics. It is important to billion worldwide in 2004 (Saady, 2005). Several analyze non-technical factors that promote the leading universities around the world have ad- adoption and diffusion of LMS initiatives (Al- opted LMS for teachers and students to enhance birini, 2006; ElTartoussi, 2009). Consequently, the educational process (Hawkins & Rudy 2007; this study specifically aimed to examine the Browne, Jenkins & Walker, 2006; National Center impact of instructor’s individual characteristics for Educational Statistics, 2003). About 95 percent (computer anxiety, technology experience and of participating institutions in the UK have adopted self efficacy), LMS characteristics (system qual- LMS (Browne et al., 2006); likewise, more than ity, information quality and service quality), and 90 percent of all participating universities and organizational characteristics (management sup- colleges in the US are adopting LMS (Hawkins port, training and incentives) on the instructors’ & Rudy, 2007). acceptance (perceived ease of use and perceived Users’ acceptance and actual use of informa- usefulness) and use of LMS as a supplementary tion systems is critical to its success. Likewise for tool. The study also assessed the impact of the a learning management system, its success to a instructors’ acceptance and use of LMS on their great extent depends on users’ acceptance and use. intention for continuous supplementary use of Evaluating individual users’ acceptance and use of LMS and intention for pure use of LMS for dis- the e-learning systems is a “basic marketing ele- tance education. Many organizations start their ment” (Kelly & Bauer, 2004). Although a learning LMS adoption as a supplementary tool to tradi- management system survives through instructor tional teaching, hoping that this supplementary and student use, the adoption of LMS is initiated by adoption eventually will promote the pure use of instructors’ acceptance and use. Even when LMS LMS for distance education. The following sec- are well in place, instructors may not fully utilize tions discuss the background literature, research all the features. For example, a survey of more framework and methodology, analysis and results, than 800 instructors at over 35 institutions using and the conclusion. Blackboard learning management system found that very few teachers use LMS tools for assessing

117 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

BACKGROUND 2008). Moodle learning management system, on the other hand, is sometimes preferred over Learning Management Systems the previous popular LMS packages (see, for example, Beatty and Ulasewicz, 2006). It is scal- A learning management system is “software that able open-source software used mainly by North automates the administration of training” (World American and European universities that supports Bank, 2010). It is the use of a Web-based com- group forum participation and features interactive munication, collaboration, learning, knowledge tools between students and instructors (Beatty & transfer, and training to add value to learners and Ulasewicz, 2006). businesses (Kelly & Bauer, 2004). Specifically, LMS applications provide several features to a learning management system is an Internet instructors, especially to those who see the real application that aims to support education and benefits of the Internet in the teaching process training activities (Cavus & Momani, 2009) and (Burniske & Monke, 2001). Yueh and Hsu (2008) provides a platform to support e-learning activi- described course management tools, group chat ties (Yueh & Hsu, 2008). Course Management and discussion, assignment submission, and course Systems (CMS) and Learning Content Manage- assessment as the primary tools in LMS. In ad- ment Systems (LCMS) are sometimes used to dition, LMS help teachers provide students with indicate LMS (Yueh & Hsu, 2008); other related educational materials and track their participa- terms are Computer-assisted Learning (CAL), tion and assessments (Falvo & Johnson, 2007). Computer-based Learning (CBL), and Online Yildirim, Temur, Kocaman and Goktas (2004) Learning (Chan, 2008). It should be noted, how- described more technically sophisticated LMS ever, that LMS applications are not unique to features such as maintaining office hours online, educational institutions; even public and private creating student groups, and assigning online organizations use such systems for training pur- projects to groups. Ceraulo (2005) mentioned poses. Reiser and Dempsey (2002) stated that in ePortfolios as a key feature in some LMS ap- large US corporations, 20 percent of all training plications, which enable instructors to maintain is delivered via LMS. student submissions throughout the course (i.e., tests, assignments, projects). LMS solutions tend LMS Applications and also to increase interest in learning and teaching Benefits to Instructors among students and teachers, respectively (Mah- dizadeh, Biemans & Mulder, 2008). Aczel, Peake Many LMS applications are available. The most and Hardy (2008), and Naidu (2006) stated that popular LMS used at colleges and universities in LMS enhance teaching process efficiency and the US is Blackboard followed by WebCT, which result in cost-savings. was acquired by Blackboard, Inc. in 2006 (Falvo & Johnson, 2007). WebCT was used by millions Learning Management Systems of students from more than 2,500 universities in Acceptance and Use more than 80 countries (Chan, 2008). Other LMS solutions are Moodle, ATutor, Learn.com, Joomla, User Technology Acceptance and Use and Krawler. Blackboard has course management features that support integration with student da- LMS have been adopted by academic institutions tabases (Kneght & Reid 2009; Blackboard, 2009). to support their distance education and/or supple- WebCT supports electronic communications such ment their traditional way of teaching (Rainer, as email, bulletin boards, and chat rooms (Chan, Turban & Potter 2007). User acceptance and use

118 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

of information systems is critical to their suc- to stay with actual use behavior (DeLone & cess. The same is true for a learning management McLean, 2003). Thus, it is important to evaluate system: its success, to a great extent, depends on the direct impact of these critical determinants on users’ acceptance and use. The assessment of tech- actual system use. System use is and will continue nology success has been conducted by utilizing to be an important indication of IS success (De- various acceptance and use dimensions. Several Lone & McLean, 2003). In addition, the actual studies assessed the determinants of technology benefits of any technology may be realized only acceptance (Bailey & Pearson, 1983; Davis, 1989; from actual technology use. Thus, technology DeLone & McLean, 2003; Doll & Torkzadeh, use is the main aim of organizations to promote 1988; Venkatesh & Davis, 2000). For instance, LMS and realize some of its expected benefits. Davis (1989) assessed technology acceptance Perceived usefulness and ease of use may be by perceived usefulness and intention to use; important factors for continuous intention to use. Venkatesh and Davis (2000) assessed technology Consequently, As indicated earlier this study acceptance by perceived usefulness, intention to aimed to examine the direct impact of instruc- use, and usage behavior. Alternatively, DeLone tors’ characteristics, LMS’ characteristics, and and McLean (2003) assessed technology success organization’s characteristics on instructors’ actual by user satisfaction and usage. usage behavior, instructors’ perceived ease of use In the LMS context, researchers also assessed of LMS, and instructors’ perceived usefulness of instructors’ acceptance and use in various ways. LMS. Furthermore, the study examined the im- Liaw, Huang and Chen (2007) assessed LMS ac- pacts of these acceptance (perceived ease of use, ceptance by learners’ and instructors’ behavioral perceived usefulness), and actual use measures on intention to use e-learning, which is influenced continuous intention for supplementary use and by perceived usefulness, perceived self-efficacy, intention for pure use of the LMS for distance and perceived enjoyment. Ball and Levy (2008) education. Perceived usefulness and perceived assessed LMS acceptance by instructors’ inten- ease of use may be important to ensure continuous tion to use. Teo (2009) assessed LMS acceptance use of the system and future intention to adopt it by teachers’ perceived usefulness and perceived for distance education. ease of use. None of these studies, however, investigated Instructor Characteristics the direct impact of instructors’ characteristics, LMS’ characteristics, and/or an organization’s The acceptance and use of LMS may, to a great characteristics on actual system use. Perceived extent, be determined by the characteristics of usefulness, perceived ease of use, users’ satisfac- its users. Several dimensions of users’ charac- tion, and intention to use are important measures teristics have been proposed and investigated as for technology acceptance and may eventually cor- determinants of technology acceptance. Some of relate with actual use behavior. Some researchers these are users’ experience of technology, users’ have found direct effects between such external self-efficacy, and users’ computer anxiety. Us- factors and technology use (Igbaria, Guimaraes ers’ experience of technology has been highly & Davis, 1995). Nevertheless, these acceptance utilized to determine users’ technology acceptance measures do not explain all the variance of actual (Venkatesh & Davis, 2000; Thompson, Compeau, usage behavior. In addition, measuring attitudes Deborah & Higgins, 2006). Individual technol- and their link to actual usage behavior is extremely ogy experience is the individual’s exposure to the difficult; therefore, many researchers may choose technology as well as the skills and abilities that

119 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

s/he gains through using a technology (Thompson teachers’ perceived usefulness, perceived ease of et al., 2006); users’ self-efficacy pays a major use, and behavioral intention in Singapore. Liaw role in users’ acceptance and use of technology. et al. (2007) found that perceived self-efficacy Self-efficacy is defined as “people’s judgments of determines instructors’ behavioral intention to use their capabilities to organize and execute courses e-learning in Taiwan. Albirini (2006) investigated of action required to attain designated types of the perception of school teachers of the use of ICT performances” (Bandura, 1977, p.391). Thus, in education in Syria, and the results highlighted computer self-efficacy means individual’s self- the importance of teachers’ vision of technol- assessment of their ability to apply computer skills ogy, their experiences with it, and the cultural to accomplish their tasks (Compeau & Higgins, conditions on their attitudes toward technology. 1995). Furthermore, a user’s computer anxiety is Mahdizadeh and his colleagues (2008) found that considered an important factor for technology ac- teachers’ previous experience with e-learning ceptance and use. Computer anxiety is defined as environments and ease of use explain teachers’ “the fear or apprehension felt by individuals when perception of the usefulness of e-learning environ- they used computers, or when they considered the ments and their actual use of these environments. possibility of computer utilization” (Simonson, Maurer, Montag-Torardi & Whitaker, 1987, p. LMS Characteristics 238). Fear of computers may negatively impact technology acceptance and use. An individual’s The characteristics of LMS may have a great computer anxiety, self efficacy and experience impact on the instructor’s acceptance and use of with technology may share some correlations, but LMS. Characteristics of any information system, they are not exactly similar individual traits (Ball including LMS, may be related to system, informa- & Levy, 2008). Computer anxiety is not simply tion, and service support quality as classified by a negative, short-term attitude toward computers DeLone and McLean (2003). E-learning systems’ that can be reduced by increasing technology ex- quality was found to be significant on the instruc- perience; it is the users’ individual fear associated tors’ perceived usefulness, perceived enjoyment, with computer use with or without experience and perceived self-efficacy, which consequently (Ball & Levy, 2008). Likewise, individuals’ self affect their intention to use the system in the efficacy is an individual trait that some individuals classroom (Liaw et al., 2007). might have with or without technology experience. System quality, which refers to the characteris- Some individuals are capable to accept and use tics of a system, is a key in users’ acceptance and LMS without a prior experience with technology use of any technology, including LMS. Research- because they have high level of self efficacy. ers, such as Bailey and Pearson (1983), DeLone In the context of e-learning, few studies have and McLean (2003), and Seddon (1997) have investigated the impact of instructors’ dimen- highlighted the impact of system quality on tech- sions on LMS acceptance. Ball and Levy (2008) nology acceptance and introduced several ways investigated the impact of self-efficacy, computer to measure system quality. Information quality, anxiety, and technology experience on instructors’ which refers to the perceived output produced by intention to use emerging learning experience in a the system, is also an important factor in instruc- small private university in the US and found that tors’ acceptance and use of LMS. Information self-efficacy was the only major determinant of quality refers to the accuracy, relevance, timeli- instructors’ intention. Teo (2009) found that com- ness, sufficiency, completeness, understandability, puter self-efficacy directly impacts pre-service format, and accessibility of the information (Bailey

120 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

& Pearson, 1983; Seddon, 1997). In addition, Organization Characteristics service quality may be a factor on the instructors’ acceptance and use of LMS; it refers to the quality An organization’s characteristics play a major of support services provided to the system’s end- role in the behaviors of its employees, including users. DeLone and McLean (2003) indicated that the acceptance and use of any technology such ignoring service quality endangers IS effectiveness as LMS. Corporate culture plays a key role in the measurements. Common measurements of service success of any project. Schein defines culture as quality are tangibles, reliability, responsiveness, “the way we do things around here” (1985, p. 12). assurance, and empathy (Parasuraman, Zeithaml, Cultural values shape an organization’s norms & Berry, 1988; Kettinger & Lee, 1994). and practices, which consequently influence In the e-learning context, few studies have employees’ behaviors such as LMS utilization. examined the general quality of technology or Some of an organization’s characteristics that specific dimension. For instance, from instruc- might be relevant to the utilization of LMS are tors’ and learners’ perspective, Liaw et al. (2007) management support, incentives, and training. investigated the impact of e-learning systems’ Organizational support, represented by senior general quality on perceived usefulness, perceived managers’ support, is also important for instructors enjoyment, and perceived self-efficacy, which to accept and use LMS in their teaching. Manage- consequently affect their intention to use the ment’s support of end-users significantly improves system in the classroom, and found it significant. computer usage (Igbaria, 1990). In the e-learning Albirini (2006) indicates that instructors’ vision context, senior management support and the align- of technology impacts their attitudes toward the ment of e-learning with the department and uni- use of ICT in education. Two significant studies versity curriculum are important for its adoption on the impact of technology on users’ acceptance (Sumner & Hostetler, 1999). Motivators are also of LMS are Pituch and Lee’s (2006) and Roca, an important factor for instructors’ acceptance to Chiu and Martinez’s (2006), but they are from the integrate the technology in teaching. Motivators learners’ perspective. Roca et al. (2006) investi- or incentives for instructors can be enforced by gated learners’ perceived system quality from three having the use of the technology as a factor in a dimensions (system quality, information quality, nomination for teaching award, promotion, and and service quality). They found that learners’ tenure (Sumner & Hostetler, 1999). Finally, train- perceived system quality factors (system quality, ing end-users is important, and can be in form of information quality, and service quality) directly workshops, online tutorials, courses, and seminars. affect their e-learning satisfaction and intention There is a lack of empirical studies that capture to use and indirectly their perceived usefulness. the influence of organization factors on the ac- Pituch and Lee (2006) examined the impact of ceptance and use of LMS generally. One of these system quality from three dimensions: the system’s is a qualitative study by Sumner and Hostetler functionality, interactivity, and response. (1999), who categorize the organizational fac- As indicated, limited studies provide a detailed tors that may influence the use of technology in examination of the influence of the three dimen- teaching in terms of motivators/demotivators, sions (system quality, information quality, service training, technology alignment, and organizational quality) of LMS on instructors’ acceptance. This and technical support. In addition, Teo (2009) study integrates these three dimensions of LMS found that facilitating conditions, measured by on the instructors’ acceptance. technical support, training, and administrative support, indirectly affect teachers’ acceptance of technology in education.

121 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

INSTRUCTORS’ LMS ACCEPTANCE and usage of LMS and, consequently, continuous AND USAGE FRAMEWORK supplementary use and future pure use intention. Figure 1 illustrates this study model. Framework Development Instructor Individual Characteristics This study aimed to examine the impact of in- Hypotheses structor’s individual characteristics, LMS’ characteristics, and an organization’s characteristics Computer Anxiety Hypotheses on instructors’ acceptance ad usage of LMS as a supplementary tool and, consequently, on con- Computer anxiety is “the fear or apprehension felt tinuous use and pure use intention for distance by individuals when they used computers, or when learning. As indicated, few studies have examined they considered the possibility of computer utiliza- this integrated investigation of instructors’ LMS tion” (Simonson, et al., 1987, p. 238). Computer acceptance and usage. This study assessed the anxiety is an important factor for the acceptance individual characteristics based on instructors’ of the technology (Ball & Levy, 2008; Piccoli, computer anxiety and technology experience, Ahmad & Ives, 2001; Raaij & Schepers, 2008; LMS characteristics based on system, information, Sun, Tsai, Finger, Chen & Yeh, 2008). Fear of and service quality; and organizational charac- computers may negatively affect the acceptance of teristics based on management support, incen- LMS and the user’s perceived satisfaction (Piccoli tives and training. The study assessed the impact et al., 2001). Empirical evidence of the impact of these factors on the instructors’ acceptance of computer anxiety was mixed. Ball and Levy (perceived ease of use and perceived usefulness) (2008) did not detect a significant link between

Figure 1. Instructors LMS acceptance and use model

122 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

computer anxiety and instructors’ intention to use Hypothesis 2a: The instructor’s experience with the e-learning; however, Sun et al.(2008) found the use of technology is positively associated that computer anxiety significantly impacts the with perceived ease of use LMS. learners’ perceived satisfaction of e-learning, and Hypothesis 2b: The instructor’s experience with Raaij and Schepers (2008) found the computer the use of technology is positively associated anxiety impacts the learner’s perceived ease of with perceived usefulness of LMS. use of e-learning. Therefore we hypothesized that: Hypothesis 2c: The instructor’s experience with the use of technology is positively associated Hypothesis 1a: Instructors’ computer anxiety is with use of LMS. negatively associated with perceived ease of use of LMS. Self-Efficacy Hypotheses Hypothesis 1b: Instructors’ computer anxiety is negatively associated with perceived useful- Self-efficacy is “people’s judgments of their ness of LMS. capabilities to organize and execute courses Hypothesis 1c: Instructors’ computer anxiety of action required to attain designated types of is negatively associated with use of LMS. performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individuals’ self- Technology Experience Hypotheses assessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, Users’ experience with the technology (EUT) also 1995). Several empirical studies found significant plays a major role in the acceptance of technol- effects of computer self-efficacy on the perceived ogy (Venkatesh & Davis, 2000; Thompson et al., usefulness on information systems (Venkatesh & 2006). An individual’s EUT is his/her exposure Davis, 2000; Chau, 2001). The more ability the to the technology as well as the skills and abili- instructor has to apply a computer application, ties that are gained through using a technology the most likely s/he perceives it easy to use and (Thompson et al., 2006). Therefore, EUT may useful, and will eventually use it. In the context of impact instructors’ acceptance of LMS for their e-learning systems, Ball and Levy (2008) found classes. Although empirical quantitative research, a significant effect of self-efficacy on instructors’ such as that of Ball and Levy (2008), found no intention to use. In addition, computer self-efficacy significant impact of EUT on instructors’ intention was found to be significant on learners’ perceived to use LMS, researchers Sumner and Hostetler ease of use (Lee, 2006; Pituch & Lee, 2006; Roca (1999) indicate that current level of computer skills et al., 2006) and learners’ perceived satisfaction and extent of use of computing skills in teaching (Sun et al., 2008). Therefore, we hypothesized: are important for instructors’ acceptance of ICT in education. Likewise, Wan, Fang and Neufeld Hypothesis 3a: An instructor’s computer self- (2007) highlight the importance of technology efficacy is positively associated with per- experience on the learning processes and, con- ceived ease of use of LMS. sequently, learning outcomes. Mahdizadeh et al. Hypothesis 3b: An instructor’s computer self- (2008) reveal that instructors’ prior experience efficacy is positively associated with per- with e-learning may explain their perception of the ceived usefulness of LMS. usefulness of e-learning environments and their Hypothesis 3c: An instructor’s computer self- actual use. Therefore we hypothesized: efficacy is positively associated with use of LMS.

123 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

LMS Characteristics Hypotheses of research on the impact of information quality on instructors’ acceptance of LMS. Some research System Quality Hypotheses was conducted from the learners’ perspective. Roca et al. (2006) measured information quality System quality is essential for the acceptance of of LMS by indicators related to relevance, timeli- any technology, including LMS. Researchers, ness, sufficiency, accuracy, clarity, and format, and such as Bailey and Pearson (1983), DeLone and proved that information quality was directly sig- McLean (1992), and Seddon (1997) highlight nificant for learners’ satisfaction and indirectly for the impact of system quality on technology ac- perceived usefulness. Likewise, Lee (2006) found ceptance and have introduced several ways to content quality was significant for learners’ per- measure it. Instructors’ acceptance of LMS may ceived usefulness. In the Middle East, Al-Busaidi be determined to a great extent by system qual- (2009), in an exploratory study in Oman, indicated ity. The more functionalities, interactivity, and that information quality (sufficiency, accuracy, response of LMS, the better is its acceptance and relevance, timeliness, and understandability) was utilization (Pituch & Lee, 2006). Quantitative highlighted as a determinant of learners’ LMS use. empirical studies found a significant impact of Consequently, we hypothesize that: system characteristics on e-learning acceptance: reliability (Wan et al., 2007; Webster & Hackley, Hypothesis 5a: LMS information quality is 1997); accessibility (Wan et al., 2007); and system positively associated with the instructor’s functionality, interactivity, and response (Pituch & perceived ease of use of LMS. Lee, 2006). Albirini (2006) indicates that instruc- Hypothesis 5b: LMS information quality is tors’ vision of technology impacts their attitudes positively associated with the instructor’s toward the use of ICT in education. Therefore, perceived usefulness of LMS. we hypothesized that: Hypothesis 5c: LMS information quality is positively associated with the instructor’s Hypothesis 4a: LMS system quality is positively use of LMS. associated with the instructor’s perceived ease of use LMS. Service Quality Hypotheses Hypothesis 4b: LMS system quality is positively associated with the instructor’s perceived Service quality refers to the quality of support usefulness of LMS. services provided to the system’s end-users. In- Hypothesis 4c: LMS system quality is positively structors’ acceptance of LMS may, to some extent, associated with the instructor’s use of LMS. be related to the quality of the support services. Common measurements of service quality are Information Quality Hypotheses tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman et al., 1988; Kettinger Information quality is also important for instruc- & Lee, 1994). Few studies have investigated the tors’ acceptance of LMS, and refers to the per- impact of service quality on LMS acceptance. For ceived output produced by the system. Information instance, Roca et al. (2006) assessed service quality quality with great accuracy, relevance, timeliness, by indicators related to responsiveness, reliability, sufficiency, completeness, understandability, and empathy, and confirmed its direct significance format, and accessibility are important for the on learners’ satisfaction and indirect significance acceptance of an information technology (Bailey of perceived usefulness in the e-learning context. & Pearson, 1983; Seddon, 1997). There is a lack Thus, we hypothesized that:

124 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Hypothesis 6a: LMS service quality is positively Hypothesis 7a: Management support is positively associated with the instructor’s perceived associated with the instructor’s perceived ease of use of LMS. ease of use of LMS. Hypothesis 6b: LMS service quality is positively Hypothesis 7b: Management support is positively associated with the instructor’s perceived associated with the instructor’s perceived usefulness of LMS. usefulness of LMS. Hypothesis 6c: LMS service quality is positively Hypothesis 7c: Management support is positively associated with the instructor’s use of LMS. associated with the instructor’s use of LMS.

Organization Characteristics Incentives Hypotheses Hypotheses Motivators, in terms of incentives, are important Management Support Hypotheses factors for instructors’ acceptance to integrate LMS in their teaching. Incentives can be “non- Management support is a key factor for the ac- trivial” monetary and non-monetary incentives. ceptance of any organizational initiative. Senior E-learning research lacks the assessment of managers’ open approval and endorsement of incentives on LMS acceptance. Motivators or LMS adoption promote instructors’ adoption incentives for instructors can be enforced by us- and acceptance of LMS. Managers may support ing the technology as a factor in nomination for an LMS by encouraging instructors to adopt it a teaching award, promotion, and tenure (Sumner and identify a clear vision of the objective of & Hostetler, 1999). These incentives’ policies the LMS and how it is aligned with the univer- push instructors to adopt and utilize LMS for sity vision. Little research has investigated the their teaching. Therefore, we hypothesized that: impact of management support on instructors’ acceptance of LMS. However, in the e-learning Hypothesis 8a: An incentive policy is positively context, senior managers should clearly identify associated with the instructor’s perceived the goal of LMS for the university curriculum ease of use of LMS. (Sumner & Hostetler, 1999). This managers’ Hypothesis 8b: An incentive policy is positively support assures instructors that using LMS is part associated with the instructor’s perceived of the organization’s culture and is useful and usefulness of LMS. encourages them to adopt and use the system. Hypothesis 8c: An incentive policy is positively Managers are recognized as a high authority (Ali, associated with the instructor’s use of LMS. 1990); thus, instructors’ adoption and acceptance of LMS may be associated with the endorsement Training Hypotheses of their senior managers. Management support of end-users significantly improves computer usage Providing end-users with training is important, as (Igbaria, 1990). Facilitating conditions, including training improves instructors’ adoption of LMS administrative support, indirectly affect teachers’ and enhances the perceived ease of use of LMS, il- acceptance of technology in education (Teo, 2009). lustrates its potential usefulness, and encourages its Consequently, we hypothesized that: use in teaching. Limited research has investigated the impact of training on instructors’ acceptance of LMS, which can be in the form of workshops, online tutorials, courses, and seminars (Sumner &

125 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Hostetler, 1999). Facilitating conditions, including Continuous Supplementary training, indirectly affect teachers’ acceptance of Use Intention Hypotheses technology in education (Teo, 2009). Thus, we hypothesized: The intention to use the technology is significantly determined by users’ perceived ease of use and Hypothesis 9a: Training is positively associated perceived usefulness (Venkatesh & Davis, 2000). with the instructor’s perceived ease of use The higher the instructors’ perceived ease of use of LMS. of LMS, perceived usefulness of LMS, and actual Hypothesis 9b: Training is positively associated use, the more likely it is that they will continue with the instructor’s perceived usefulness to use it. Continuous intention to e-learning use of LMS. is determined by perceived usefulness and satis- Hypothesis 9c: Training is positively associated faction (Hayashi, Chen, Ryan, Wu, 2004). Thus, with the instructor’s use of LMS. we hypothesized:

Usage and Future Hypothesis 11a: The instructors’ perceived ease of Intention Hypotheses use of LMS is positively associated with their continuous supplementary use intention. Perception and Usage Hypotheses Hypothesis 11b: The instructors’ perceived usefulness of LMS is positively associated A technology’s perceived ease of use and perceived with their continuous supplementary use usefulness are found to be a significant determinant intention. of the intention to use the technology (Venkatesh Hypothesis 11c: The instructors’ supplementary & Davis, 2000). The higher the instructors’ per- use of LMS is positively associated with their ceived ease of use and perceived usefulness of continuous supplementary use intention. LMS, the higher the actual use. Perceived ease of use is also a significant determinant of users’ Future Pure Use Intention Hypotheses perceived usefulness of a technology (Venkatesh & Davis, 2000). Pituch and Lee (2006) found Many organizations begin their LMS adoption as a learners’ perceived ease of use of LMS to sig- supplementary tool to traditional teaching, hoping nificantly affect perceived usefulness. Therefore, that this supplementary adoption will eventually we hypothesized: promote the pure use of LMS for distance education. Perceived ease of use, perceived usefulness, Hypothesis 10a: Instructors’ perceived ease of and actual use may have an important impact on use of LMS is positively associated with continuous intention for supplementary use and their use of LMS. intention for pure use of the LMS for education. Hypothesis 10b: Instructors’ perceived usefulness When instructors believe that LMS is easy, useful, of LMS is positively associated with their and can be utilized for supplementary purposes, use of LMS. they are more likely to adopt it purely for distance Hypothesis 10c: Instructors’ perceived ease of education. Technology perceived ease of use and use of LMS is positively associated with perceived usefulness are found to be significant their perceived usefulness of LMS. determinants of the intention to use the technology (Venkatesh & Davis, 2000). Perceived ease of use, perceived usefulness, and supplementary

126 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

use are significant determinants of learners’ use percent, about average; and only 6 percent were of e-learning for distance education (Pituch & below average. The majority, about 59 percent, has Lee, 2006). Thus we hypothesized: used the LMS for classes for three years or more; 30 percent have used it for one to two years; and Hypothesis 12a: The instructors’ perceived ease 11 percent have used it for less than one year. The of use of LMS is positively associated with majority of the instructors, about 55 percent, have their pure use intention. experience with only Moodle LMS; 9 percent have Hypothesis 12b: The instructors’ perceived use- experience with only WebCT LMS; 31 percent fulness of LMS is positively associated with have experience with both WebCT and Moodle; their pure use intention. and 5 percent have experience with Blackboard. Hypothesis 12c: The instructors’ supplementary use of LMS is positively associated with Research Questionnaire their pure use intention. The questionnaire was distributed to SQU instructors. An invitation email was sent to instructors to METHODOLOGY complete the study questionnaire either online or on an attached MS Word document. A reminder Participants’ Profile was sent two weeks after the initial invitation. Most of the instructors filled the questionnaire This study included 82 instructors from Sultan online (about 95 percent of them).Only five per- Qaboos University (SQU), the first and only cent of instructors completed the questionnaire public university in Oman. Since its launch, SQU as a hard copy. has gone through many technology develop- The questionnaire included the constructs to ments: it adopted WebCT and later switched to be measured for quantitative analysis, along with the open-source Moodle application. Instructors demographic questions (e.g., gender, age, degree, can voluntarily adopt LMS to supplement their LMS usage experience, work experience, and traditional classes. job title). Construct measurements items were The instructors were from different colleges in phrased according to a five–point Likert scale the university and with different demographics. (1= strongly disagree; 2=disagree; 3=Neutral; About 62 percent were male and 38 percent were 4= agree and 5=strongly agree). To statistically female. About 5 percent were assistant lecturers, 27 evaluate the study framework, 28 indicators were percent were lecturers, 50 percent were assistant used. Tables 1 and 2 show the total indicators professors, 13 percent were associate professors, used for each construct. The LMS characteristic and 5 percent were full professors. The instruc- constructs (system quality, information quality, tors’ age varied from 20s to above 50s: about 8 and service quality) were adopted and modi- percent were in their 20s, 26 percent were in their fied from Roca et al. (2006) and Pituch and Lee 30s, 16 percent in their 40s, and 32 percent were (2006); the individual characteristics constructs 50 or over. Almost 44 percent had less than six (computer anxiety, self-efficacy, and technology years of work experience, 30 percent had less experience) were adopted from Ball and Levy than 11 years, 16 percent had less than 16 years, (2008); while the organizational characteristics’ 7 percent had less than 21 years, and 2 percent constructs (management support, incentives, and had more than 20 years. Most indicated that their training) were self-developed, based on Sumner computer skills were above average. Almost 71 and Hostetler (1999). The LMS acceptance percent have above average computer skills; 23 construct (perceived ease of use and perceived

127 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Table 1. Independent constructs measures and loadings

Construct Measures Loading Computer Anxiety 1. I believe that working with computers is very difficult. 0.8913 2. Computers make me feel uncomfortable. 0.9560 3. I get a sinking feeling when I think of trying to use a computer. 0.8698 Technology Experience 1. I feel confident using the e-learning system 0.8465 2. I feel confident downloading/uploading necessary materials from the Internet. 0.8756 3. I feel confident using the chatting and discussion forums. 0.6275 Self Efficacy 1. I could use the e-learning system if I had never used a system like it before. -0.7948 2. I could use the e-learning system if I had only the system manuals for reference. 0.0894 3. I could use the e-learning system if I had seen someone else using it before trying it myself. 0.8652 System Quality 1. The system offers flexibility in teaching as to time and place. 0.7694 2. The system offers multimedia (audio, video, and text) types of course content. 0.7962 3. The response time of the system is reasonable. 0.6025 4. The system enables interactive communication between instructor and students. 0.7737 Information Quality 1. The information provided by the system is relevant for my job. 0.8434 2. The information in the system is very good. 0.8919 3. The information from the e-learning system is up-to-date. 0.8407 4. The information provided by the system is complete. 0.8494 Service Quality 1. The system support services give me prompt service. 0.8425 2. The system support services have convenient operating hours. 0.8410 3. The system support services are reliable. 0.8761 4. The system support services are easy to communicate with. 0.8904 Management Support 1. Senior administrators strongly support the use of e-learning system. 0.9396 2. I get support by department chair or dean on my use of e-learning system. 0.8506 3. My mangers highlight the importance of e-learning system on my curriculum. 0.9188 Incentives 1. The use of e-learning is a factor in the nomination for teaching award. 0.9529 2. The use of e-learning system is a factor in determining promotion. 0.9558 3. The use of e-learning system is a factor in annual elevation of teaching. 0.9611 Training 1. I receive training workshops on how to use e-learning tools. 0.7277 2. I receive on-line manuals on how to use e-learning tools. 0.8759 3. I receive seminars on the use of e-learning tools. 0.8378

128 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

usefulness) were adopted and modified from (Chin, 1998; Chin, 2001). The PLS approach is Venkatesh and Davis (2000), and supplementary a variance-based SEM that assists researchers in use, continuous supplementary use, and future obtaining determinate values of latent variables pure use were adopted and modified according for predictive purposes. The PLS does that by to Pituch and Lee (2006). minimizing the variance of all dependent variables rather than using the model to explain the co-variation of all indicators (Chin, 1998; Chin DATA ANALYSIS & RESULTS and Newsted, 1999). Thus, the model paths are estimated based on the ability to minimize the PLS Analysis Methodology residual variances of the dependent variables. The PLS algorithm uses an iterative process for the Data was analyzed by PLS-Graph 3.0 software. estimation of weights and latent variables scores. PLS (partial least square) is a variance-based The process almost converges to a stable set of structural equation model (SEM) technique that weight estimates. The evaluation of the model is allows path analysis of models with latent variables based on (1) the assessment of the model mea-

Table 2. Dependant constructs measures and loadings

Construct Measures Loading Perceived Ease of Use 1. Using e-learning tools is easy to me. 0.8896 2. E-learning tools are clear and understandable to me. 0.9280 3. I find it easy to get the e-learning system to do what I want it to do. 0.7843 Perceived Usefulness 1. Using e-learning system enables me to accomplish tasks more quickly. 0.8581 2. Using e-learning system improves my performance. 0.9251 3. Using e-learning system increases my productivity. 0.9255 4. Using e-learning system enhances the effectiveness on the job. 0.8958 5. Using e-learning system gives me greater control over my work. 0.8831 Supplementary Use 1. I use the e-learning system as many occasions as possible to supplement my teaching. 0.9165 2. I use the e-learning system on regular basis to supplement my teaching. 0.9047 3. I frequently use the e-learning system to supplement my teaching. 0.9051 4. I use the e-learning system to share/seek course information. 0.7357 5. I use the e-learning system to communicate with students 0.8259 Continuous supplementary Use Intention (CUI) 1. I will frequently use e-learning system to do a teaching task. 0.8871 2. I will use e-learning system on regular basis to supplement my classes in the future. 0.8523 3. I will always try to use the e-learning system to do a teaching task whenever it has a useful feature. 0.8915 Pure Use Intention (PUI) 1. I plan to teach purely online courses for distance learners. 0.9249 2. I will use e-learning system to teach purely online courses. 0.9627 3. I plan to teach purely online courses in as many occasions as possible. 0.9430

129 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

surements by assessing their validity, reliability, 2 show the model constructs’ measurements and and discriminant validity, (2) the analysis of the loading. Table 3 shows that the study constructs’ paths of the structural model (Chin, 1998). Table reliability and AVE are above the recommended 1 and Table 2 show the independent and dependant levels for all the constructs except self-efficacy. constructs’ measures and loading respectively. Therefore, the self-efficacy construct was dropped from the model evaluation. Constructs Validity and Reliability To achieve the discriminant validity of the constructs, Fornell and Larcker (1981) suggest The reliability and the validity are two criteria that the square root of AVE of each construct used by researchers to evaluate the applicabil- should exceed the correlations shared between ity of their measurements to their investigated the constructs and other constructs in the model. model. Reliability refers to the consistency of the The discriminant validity is used to ensure the measures (indicators) of a specific latent variable; differences among constructs (Barclay, Higgins whereas, validity refers to how well the concept is & Thompson, 1995; Chin, 1998). Table 4 shows defined by the measures (Hair, Anderson, Tatham that the model constructs satisfy that rule, as the & Black, 1998; Crano & Brewer, 2002). With PLS, square root of the AVE (on the diagonal) is the reliability of the measurements was evaluated greater than the correlations with other constructs. by internal consistency reliability, and the validity Thus, all the model constructs have a satisfactory was measured by the average variance extracted discriminant validity construct. (AVE), which refers to the amount of variance a latent variable captures from its indicators. AVE Model Evaluation and Paths Analysis was developed by Fornell and Larcker (1981) to assess construct validity. The recommended level for With PLS, R-square values are used to evaluate internal consistency reliability is at least 0.70, and the predictive relevance of a structural model is at least 0.50 for AVE (Chin, 1998). Tables 1 and for the dependent latent variables, and the path

Table 3. Constructs reliability and validity

Construct Total Items Reliability AVE Computer Anxiety (CA) 3 0.932 0.822 Technology Experience (TE) 3 0.831 0.626 Self Efficacy (SE) 3 0.504 0.267 System Quality (SQ) 4 0.827 0.547 Information Quality (IQ) 4 0.917 0.734 Service Quality (SvQ) 4 0.921 0.744 Management Support (MS) 3 0.930 0.817 Incentives (IN) 3 0.970 0.915 Training (TR) 3 0.856 0.666 Perceived Ease of Use (PEU) 3 0.902 0.756 Perceived Usefulness (PU) 5 0.954 0.806 Supplementary Use (USE) 5 0.934 0.740 Continuous Supplementary Use Intention (CUI) 3 0.909 0.769 Pure Use Intention (PUI) 3 0.961 0.890

130 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

coefficients are used to assess the effects of the analysis between the exogenous independent independent variables(Chin, 1998). The signifi- constructs (instructors’ characteristics, LMS’s cance of the model paths was checked based on characteristics, and organization’s characteristics) their t-values. and the endogenous dependent construct (LMS Table 5 shows the R2 values of the endogenous supplementary use, LMS perceived ease of use, dependent constructs. The model explains 58 LMS perceived usefulness), and, consequently, percent of variance in instructors’ usage of LMS; future intention (continuous supplementary use 52.2 percent of variance in instructors’ perceived intention and pure use intention). The statistical usefulness of LMS; and 42.4 percent of variance in significant of the paths’ coefficients was measured instructors’ perceived ease of use of LMS. In ad- by T-values with at least 95 percent confidence dition, instructors’ usage of LMS, their perceived level. usefulness of LMS, and their perceived ease of The analysis showed that the instructor’s use of LMS explains 55.3 percent of variance in characteristics, the LMS’s characteristics and the their continuous supplementary use intention, but organization’s characteristics to some extent have only 22.9 percent of their pure use intention of impact on the instructor’s acceptance and use of LMS. Table 5 also shows the paths’ coefficients LMS; see Table 5. First, instructors’ computer

Table 4. Construct’ correlations and discriminant validity

Construct CA TE SQ IQ SvQ MS IN TR PEU PU USE CUI PUI Computer Anxiety 0.907 (CA) Technology Expe- -0.124 0.791 rience (TE) System Quality -0.072 0.202 0.740 (SQ) Information -0.069 0.188 0.675 0.857 Quality (IQ) Service Quality -0.024 0.076 0.407 0.689 0.863 (SvQ) Management 0.175 0.013 0.257 0.207 0.206 0.904 Support(MS) Incentives 0.224 -0.196 0.144 0.133 0.140 0.447 0.957 (IN) Training 0.024 -0.001 0.235 0.343 0.347 0.220 0.283 0.816 (TR) Perceived Ease of -0.362 0.313 0.418 0.436 0.377 0.182 0.042 0.314 0.869 Use (PEU) Perceived Useful- -0.175 0.230 0.614 0.381 0.176 0.223 0.175 0.204 0.546 0.898 ness (PU) Supplementary -0.178 0.168 0.469 0.370 0.146 0.379 0.223 0.436 0.531 0.614 0.960 Use (USE) Continuous sup- -0.328 0.347 0.507 0.353 0.194 0.195 0.167 0.358 0.583 0.647 0.645 0.877 plementary Use Intention (CUI) Pure Use Inten- 0.005 0.091 0.147 0.058 -0.024 0.058 0.094 0.148 0.324 0.467 0.351 0.431 0.943 tion (PUI)

131 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Table 5. Model Evaluation & Paths Analysis

Construct CA TE SQ IQ SvQ MS IN TR PEU PU USE PEU -0.340G 0.212 E 0.173C 0.068 0.150C 0.106 0.006 0.182D (0.424) (h1a) (h2a) (h4a) (h5a) (h6a) (h7a) (h8a) (h9a) (0.522) -0.033 0.029 0.508G 0.056 0.008 0.014 0.104B 0.002 0.300G (PU) (h1b) (h2b) (h4b) (h5b) (h6b) (h7b) (h8b) (h9b) (h10c) USE -0.088A 0.003 0.051 0.115 B 0.012 0.203G 0.001 0.238G 0.139E 0.324G (0.580) (h1c) (h2c) (h4c) (h5c) (h6c) (h7c) (h8c) (h9c) (h10a) (h10b) CUI 0.238F 0.320 F 0.322G (0.553) (h11a) (h11b) (h11c) PUI 0.077 0.376G 0.080 (0.229) (h12a) (h12b) (h12c) A. Significant at 95% |B. Significant at97.5% |C. Significant at 99% |D. Significant at99.5% E. Significant at 99.75% | F. Significant at 99.9% | G: Significant at99.95% anxiety negatively impacts their perceived ease usefulness (β of 0.324, p-value= 0.0005; h10b) of use (Beta -β = -0.340, p= 0.0005, hypothesis of LMS positively associated with their actual h1a) and supplementary use of LMS (β = -0.088, use of LMS, and instructors’ perceived ease of p= 0.05; h1c). Second, instructors’ experience use of LMS positively associated with their per- with the technology only significantly impacts ceived usefulness of LMS (β = 0.300, p= 0.0005; their perceived ease of use (β = 0.212, p = 0.0025; ; h10c). Eleventh, instructors’ perceived ease of h2a). Third, the study was unable to assess the use of LMS (β = 0.238, p = 0.001; h11a), perceived impact of instructors’ self efficacy on their use usefulness (β = 0.320, p = 0.001; h11b), and cur- and acceptance of LMS because of the low reli- rent actual use (β = 0.322, p = 0.0005; h11c) are ability and validity of the construct (self-efficacy) positively associated with their continuous supple- measures; thus, hypotheses H3a, H3b, H3c were mentary use intention. Twelfth, instructors’ per- not tested. Fourth, system’s quality significantly ceived usefulness of LMS is only positively as- impacts instructors’ perceived ease of use (β = sociated with their pure use intention of LMS for 0.173, p = 0.01; h4a); and perceived usefulness distance education (β = 0.376, p = 0.0005; h12b). (β = 0.508, p= 0.0005; h4b) of LMS. Fifth, infor- Thus, hypotheses h1b, h2b, h2c, h4c, h5a, h5b, mation quality significantly impacts instructor’s h6b, h6c, h7a, h7b, h8a, h8c, h9b, h12a and h12c actual use of LMS (β = 0.149, p= 0.025; h5c). were not significant as indicated in Table 5. Sixth, service quality significantly impacts only instructors’ perceived ease of use of LMS (β = 0.150, p = 0.01; h6a). Seventh, management sup- DISCUSSION & CONCLUSION port significantly impacts instructor’s actual use of LMS (β = 0.203, p=0.0005; h7c). Eight, incen- Discussion of Findings tives policy significantly impacts instructors’ and Implications perceived usefulness of LMS (β = 0.104, p= 0.025; h8b). Ninth, training significantly impacts the LMS include several tools that provide academic instructors’ perceived ease of use (β = 0.182, p = and training institutions an efficient and effec- 0.005; h9a) and actual use of LMS (β = 0.238, p= tive means to support distance education and 0.0005; h9c). Tenth, instructors’ perceived ease supplement their traditional teaching. Moreover, of use (β = 0.139, p= 0.0025; h10a) and perceived LMS enable these institutions to capture their

132 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

educational materials and preserve them for of LMS. Instructors’ use of LMS is impacted by a future reuse. This study examined the impact of combination of several factors: their perception of instructors’ characteristics (computer anxiety and LMS (perceived usefulness of LMS and perceived technology experience); LMS’ characteristics ease of use), organizational characteristics (train- (system quality, information quality, and service ing and management support), LMS characteristics quality); and an organization’s characteristics (information quality), and instructors’ individual (management support, incentives, and training) characteristic (computer anxiety). Other factors on the acceptance of e-learning (supplementary impact system use indirectly through perceived use, perceived usefulness, and perceived ease usefulness and perceived ease of use. Furthermore, of use) and, consequently, their future intention the results indicated that instructors’ continuous of supplementary use and pure use of LMS for intention of LMS supplementary use is determined distance education in an academic institution. by their current use, perceived usefulness, and The results showed that instructors’ individual perceived ease of use, while future intention of characteristics, LMS’ characteristics, and an orga- LMS pure use for distance education is determined nization’s characteristics have various impacts on only by their perceived usefulness of LMS. instructors’ use of LMS, their perceived usefulness LMS is promising for developing countries, of LMS, and their perceived ease of use. First, re- as they provide tools to efficiently build human garding the instructors’ individual characteristics, resources. This study offered significant findings the study found that instructors’ computer anxiety for researchers and practitioners. First, it compre- impacts their perceived ease of use of LMS and hensively examined the critical factors influencing their actual use; whereas instructors’ technology instructors’ use of LMS. The study demonstrates experience impacts their perceived ease of use that the use of LMS is determined by users’ indi- of LMS. Second, regarding the LMS’s charac- vidual characteristics, LMS characteristics, and an teristics, the study found system quality impacts organization’s characteristics. Thus, users, tech- instructors’ perceived ease of use and perceived nology, and an organization’s characteristics have usefulness of LMS, information quality impacts various and significant impacts on instructors’ instructors’ actual use of LMS, whereas, service acceptance of LMS. Few studies have examined quality impacts instructors’ perceived ease of instructors’ acceptance of LMS; the literature also use of LMS. Third, regarding the organization’s lacks the investigation of organizational charac- characteristics, the study found that manage- teristics on LMS acceptance. Second, the study ment support impacts instructors’ actual use of shows that investigating these direct impacts on LMS, incentives policy impacts the instructors’ LMS use is worthwhile. Third, the study shows perceived usefulness of LMS, whereas training that although instructors currently use LMS, the impacts instructors’ perceived ease of use of LMS. perceived ease of use and perceived usefulness of In other words, instructors’ perceived ease of LMS impact their continuous intention to use LMS use of LMS is determined by instructors’ char- for supplementary purposes; only their perceived acteristics (computer anxiety and technology usefulness of LMS impacts their intention to use experience) and an organization’s characteristics LMS purely for distance education. Few studies of training, and LMS’ system and service qual- have examined the link between instructors’ use ity. Instructors’ perceived usefulness of LMS is of LMS as a supplementary and a pure tool for impacted by system quality, its perceived ease distance education. of use, and incentives policy; thus, none of the The study also provided useful insights for instructors’ individual characteristics has a direct practitioners (instructors and academic institu- significant impact on their perceived usefulness tions). Organizations, especially in the Middle

133 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

East where computer and Internet literacy is not ACKNOWLEDGMENT as high as in developed countries, should provide training to improve users’ technology experience This study was granted by Sultan Qaboos Uni- and lessen their computer anxiety. In addition, versity in Oman organizations should adopt high-quality LMS (in terms of system quality, information quality, and service quality) to promote their adoption and use REFERENCES by instructors. In addition, management support and incentives are important to improve instruc- Aczel, J. C., Peake, S. R., & Hardy, P. (2008). tors’ use of LMS and their perceived usefulness, Designing capacity-building in e-learning respectively. The findings suggest that instructors’ expertise: Challenges and Strategies. Comput- computer anxiety plays a significant negative role ers & Education, 50, 499–510. doi:10.1016/j. on their perceived ease of use and actual use of compedu.2007.07.005 LMS, therefore, providing a good training and Al-Busaidi, K. A. (2009). Evaluating the use of service quality are necessary to improve instruc- e-learning system as a supplementary tool at an tor’s perceived ease of use of LMS. Also, orga- academic institution. International Journal of nizations should ensure that management support Internet Education, 4, 153. is in place to encourage instructor’s use of LMS. Furthermore, since instructors’ intention to pure Albirini, A. (2006). Teachers’ attitudes toward e-learning depends on their perceived usefulness information and communication technologies: of LMS, ensuring high system quality and attrac- The case of Syrian EFL teachers. Computers & tive incentives policies, as they are determinants Education, 47, 373–398. doi:10.1016/j.compe- of perceived usefulness, are vital for pursuing a du.2004.10.013 pure e-learning for distance education. Ali, A. J. (1990). Management theory in a transitional society: The Arab’s experience. Interna- Limitations and Future Research tional Studies of Management and Organization, 20(3), 7–35. This study has few limitations. First, the sample was from one academic institution in Oman and Bailey, J., & Pearson, S. (1983). Development one learning management system; more research of a tool for measuring and analyzing computer can be conducted in several organizations using user satisfaction. Management Science, 29(5), different LMS in different countries to improve the 530–545. doi:10.1287/mnsc.29.5.530 generalization of the findings. Second, the study Ball, D., & Levy, Y. (2008). Emerging educational assessed LMS usage from instructors’ perspec- technology: Assessing the factors that influence tive; further research may assess learners’ and instructors’ acceptance in information systems and organizations’ acceptance. Third, this study was other classrooms. Journal of Information Systems unable to assess the impact of self-efficacy; new Education, 19(4), 431–443. measurements might be developed to improve its reliability and validity across different countries. Bandura, A. (1977). Self-efficacy: Toward a unify- Moreover, future research could also examine in ing theory of behavioural change. Psychological detail the benefits of LMS for instructors and the Review, 84(2), 191–215. doi:10.1037/0033- critical factors influencing organizations’ deploy- 295X.84.2.191 ment of LMS.

134 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Barclay, D., Higgins, C., & Thompson, R. (1995). Chin, W. (1998). The partial least square approach The partial least squares (PLS) approach to casual to structural equation modelling. In Marcoulides, modelling: Personal computer adoption and use as G. A. (Ed.), Modern Methods for Business Re- an illustration (with commentaries). Technology search (pp. 295–336). London, UK: Lawrence Studies, 2(2), 285–309. Erlbaum Associates. Beatty, B., & Ulasewicz, C. (2006). Online teach- Chin, W. (2001). PLS Graph user’s guide, version ing and learning transition: Faculty perspectives on 3.0. Houston, TX: Bauer College of Business, moving from blackboard to the Moodle learning University of Houston. management system. TechTrends, 50(4), 36–45. Chin, W., & Newsted, P. R. (1999). Structural doi:10.1007/s11528-006-0036-y equation modeling analysis with small samples Blackboard. (2009). Engaging learners for engag- using partial least squares. In Hoyle, R. H. (Ed.), ing learning. Retrieved on September 7th, 2009, Statistical strategies for small sample research (pp. from http://www.blackboard.com/Teaching- 307–341). Thousand Oaks, CA: Sage Publications. Learning/Learn-Platform.aspx. Compeau, D., & Higgins, C. (1995). Computer Browne, T., Jenkins, M., & Walker, R. (2006). A self-efficacy: Development of a measure and initial longitudinal perspective regarding the use of VLEs test. Management Information Systems Quarterly, by higher education institutions in the United King- 19, 189–211. doi:10.2307/249688 dom. Interactive Learning Environments, 14(2), Crano, W., & Brewer, M. (2002). Principles and 177–192. doi:10.1080/10494820600852795 methods of social research. Mahwah, NJ: Law- Burnett, R., & Marshall, D. P. (2003). Web theory: rence Erlbaum Associates Publishers. An introduction. New York, NY: Routledge Press. Davis, F. D. (1989). Perceived usefulness, per- Burniske, R. W., & Monke, L. (2001). Breaking ceived ease of use, and user acceptance of infor- down the digital walls. Albany, NY: State Uni- mation technology. Management Information Sys- versity of New York Press. tems Quarterly, 13, 319–339. doi:10.2307/249008 Cavus, N., & Momani, A. M. (2009). Computer DeLone, W., & McLean, E. (2003). The Delone aided evaluation of learning management sys- and Mclean model of Information Systems suc- tems. Procedia Social and Behavioral Sciences, cess: A 10-year update. Journal of Management 1, 426–430. doi:10.1016/j.sbspro.2009.01.076 Information Systems, 19(4), 9–30. Ceraulo, S. (2005). Benefits of upgrading to an DeLone, W. H., & McLean, E. R. (1992). Informa- LMS. Distance Education Report, 9, 6–7. tion Systems success: The quest for the dependent variable. Information Systems Research, 3, 60–95. Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, doi:10.1287/isre.3.1.60 J. H. (2008). A virtual learning environment for part-time MASW students: An evaluation of the Doll & Torkzadeh, G. (1988). The measurement WebCT. Journal of Teaching in Social Work, 28, of end user computing satisfaction, IS Quarterly, 87–100. doi:10.1080/08841230802179027 12(2), 259-274. Chau, Y., Patrick, K., & Hu, J. (2001). In- El Tartoussi, I. (2009). Networked readiness in the formation Technology acceptance by indi- United Arab Emirates, The 2nd Annual Forum on vidual professionals: A model comparison E-learning Excellence in the Middle East 2009, approach. Decision Sciences, 32(4), 699–718. Dubai. UAE. doi:10.1111/j.1540-5915.2001.tb00978.x

135 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Falvo, D. A., & Johnson, B. F. (2007). The use Kneght, M., & Reid, K. (2009). Modularizing of learning management systems in the United Information Literacy training via the Blackboard States. TechTrends, 51(2), 40–45. doi:10.1007/ eCommunity. Journal of Library Administration, s11528-007-0025-9 49, 1–9. doi:10.1080/01930820802310502 Fornell, C., & Larcker, D. (1981). Evaluating struc- Lee, Y. (2006). An empirical investigation into tural equation models with unobservable variables factors influencing the adoption of an e-learning and measurement error. JMR, Journal of Market- system. Online Information Review, 30(5), 517– ing Research, 18, 39–50. doi:10.2307/3151312 541. doi:10.1108/14684520610706406 Hair, J., Anderson, R., Tatham, R., & Black, W. Liaw, S., Huang, H., & Chen, G. (2007). Sur- (1998). Multivariate data anlaysis. Upper Saddle veying instructor and learner attitudes toward e- River, NJ: Prentice Hall. learning. Computers & Education, 49, 1066–1080. doi:10.1016/j.compedu.2006.01.001 Hawkins, B. L., & Rudy, J. A. (2007). Educause Core Data Service. Fiscal Year 2006 Summary Mahdizadeh, H., Biemans, H., & Mulder, M. Report. CO, USA: Educause. (2008). Determining factors of the use of e-learning environments by university teachers. Comput- Hayashi, A., Chen, C., Ryan, T., & Wu, J. (2004). ers & Education, 51, 142–154. doi:10.1016/j. The role of social presence and moderating role compedu.2007.04.004 of computer self efficacy in predicting the continuance usage of e-learning systems. Journal of Naidu, S. (2006). E-learning a guidebook of prin- Information Systems Education, 15(2), 139–154. ciples, procedures and practices, 2nd revised edition, Commonwealth Educational Media Center Igbaria, M. (1990). End-user computing effective- for Asia (CEMCA), New Delhi, India. ness: A structural equation model. Omega, 18(6), 637. doi:10.1016/0305-0483(90)90055-E National Centre for Educational Statistics. (2003). Distance education at degree-granting postsec- Igbaria, M., Guimaraes, T., & Davis, G. B. (1995). ondary institutions: 2000-2001 U. Washington, Testing the determinants of microcomputer us- DC: S. Department of Education. age via a structural equation model. Journal of Management Information Systems, 11(4), 87–114. Parasuraman, A., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for Kelly, T., & Bauer, D. (2004). Managing intellec- measuring customer perceptions of service quality. tual capital-via e-learning-at Cisco. In Holsapple, Journal of Retailing, 64(1), 12–40. C. (Ed.), Handbook on Knowledge Management 2: Knowledge directions (pp. 511–532). Berlin, Paulsen, M. F. (2003). Experiences with learning Germany: Springer. management systems in 113 European institutions. Journal of Educational Technology & Society, Kettinger, W. J., & Lee, C. C. (1994). Perceived 6(4), 134–148. service quality and user satisfaction with the In- formation Services function. Decision Sciences, Piccoli, G., Ahmad, R., & Ives, B. (2001). Web- 25(5/6), 737–765. doi:10.1111/j.1540-5915.1994. based virtual learning environments: A research tb01868.x framework and a preliminary assessment of effectiveness in basic IT skill training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989

136 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Pituch, K., & Lee, Y. (2006). The influence of Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, system characteristics on e-learning use. Com- D. (2008). What drives a successful e-learning? puters & Education, 47, 222–244. doi:10.1016/j. An empirical investigation of the critical fac- compedu.2004.10.007 tors influencing learner satisfaction. Computers & Education, 50, 1183–1202. doi:10.1016/j. Raaij, E., & Schepers, J. (2008). The acceptance compedu.2006.11.007 and use of a virtual learning environment in China. Computers & Education, 50, 838–852. Teo, T. (2009). Modelling technology acceptance doi:10.1016/j.compedu.2006.09.001 in education: A study of pre-service teachers. Com- puters & Education, 52, 302–312. doi:10.1016/j. Rainer, R. K., Turban, E., & Potter, R. E. (2007). compedu.2008.08.006 Introduction to Information Systems: Supporting and transforming business. NJ: Wiley. Thompson, R., & Compeau, R., Deborah, & Higgins, C. (2006). Intentions to use Information Reiser, R. A., & Dempsey, J. V. (Eds.). (2002). Technologies: An integrative model. Journal of Trends and issues in instructional design and Organizational and End User Computing, 18(3), technology. Upper Saddle River, NJ: Pearson 25–47. doi:10.4018/joeuc.2006070102 Education. Venkatesh, V., & Davis, F. (2000). A theoretical Roca, J., Chiu, C., & Martinex, F. (2006). Un- extension of the technology acceptance model: derstanding e-learning continuous intention: An Four longitudinal. Management Science, 46(2), extension of the technology acceptance model. 186–204. doi:10.1287/mnsc.46.2.186.11926 International Journal of Human-Computer Stud- ies, 64, 683–696. doi:10.1016/j.ijhcs.2006.01.003 Wan, Z., Fang, Y., & Neufeld, H. (2007). The role of Information Technology in technology- Saady, A. (2005). E-learning curve, ITP Technol- mediated learning: A review of the past for the ogy, Retrieved from www.ITP.net. future. Journal of Information Systems Education, Schein, E. (1985). Organizational culture and 18(2), 183–192. leadership. San Francisco, CA: Jossey-Bass. Webster, J., & Hackley, P. (1997). Teaching effec- Seddon, P. B. (1997). A respecification and ex- tiveness in technology-mediated distance learning. tension of the DeLone and McLean model of IS Academy of Management Journal, 40(6), 1282. success. Information Systems Research, 8(3), doi:10.2307/257034 240–253. doi:10.1287/isre.8.3.240 Woods, R., Baker, J., & Hopper, D. (2004). Hybrid Simonson, M., Maurer, R., Montag-Torardi, M., structure: Faculty use and perception of Web- & Whitaker, M. (1987). Development of a stan- based courseware as a supplement to face-to-face dardized test of computer literacy and a computer instruction. The Internet and Higher Education, anxiety index. Journal of Educational Computing 7(4), 281–297. doi:10.1016/j.iheduc.2004.09.002 Research, 3(2), 231–247. doi:10.2190/7CHY- World Bank. (2010). Learning management sys- 5CM0-4D00-6JCG tem. World Bank Institute. Retrieved on April 7th, Sumner, M., & Hostetler, D. (1999). Factors 2010, from http://web.worldbank.org influencing the adoption of technology in teaching. Journal of Computer Information Systems, 40(1), 81.

137 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Yildirim, S., Temur, N., Kocaman, A., & Goktas, Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, Y. (2004). What makes a good LMS: An analytical J. H. (2008). A virtual learning environment for approach to assessment of LMS. In Proceedings part-time MASW students: An evaluation of the of the 5th International Conference on Information WebCT. Journal of Teaching in Social Work, 28, Technology Based Higher Education and Training, 87–100. doi:10.1080/08841230802179027 Istanbul, Turkey (pp. 125–130). Garrison, R., & Vaughan, H. (2008). Blended Yueh, H.-P., & Hsu, S. (2008). Designing a learn- learning in higher education: Framework, ing management system to support instruction. principles and guidelines. San Francisco, CA: Communications of the ACM, 51(4), 59–63. Jossey-Bass. doi:10.1145/1330311.1330324 Ge, X., Lubin, I. A., & Zhang, K. (2010). An investigation of faculty’s perceptions and experiences when transitioning to a new learning management ADDITIONAL READING system. Knowledge Management & E-Learning: An International Journal, 2(4), 43–62. Aggarwal, A., & Bento, R. (2000). Web-based education. In Aggarwal, A. (Ed.), Web-based Graf, S., Kinshuk, & Liu, T.-C. (2009). Supporting learning and teaching technologies: Opportuni- teachers in identifying students’ learning styles in ties and challenges (pp. 1–16). London, UK: Idea learning management systems: An automatic stu- Group Publishing. dent modelling approach. Journal of Educational Technology & Society, 12(4), 3–14. Al-Busaidi, K., & Al-Shihi, H. (2010). Instructors’ acceptance of learning management systems: A Hu, P. J., Chau, P. Y., Clark, T. H., & Ma, W. W. theoretical framework. Communications of IBIMA (2003). Examining technology acceptance by [Open Source Journal], Vol. 2010, Article ID school teachers: A longitudinal Study. Informa- 862128, 10 pages. tion & Management, 41, 227–241. doi:10.1016/ S0378-7206(03)00050-8 Al-Rawajfih, K., Fong, S., & Idros, S. (2010). Effects of principals’ support on teachers’ self- Kim, S. W., & Leet, M. G. (2008). Validation of efficacy in integrating e-learning in the Jordanian an evaluation model for learning management sys- Discovery Schools. Modern Applied Science, tems. Journal of Computer Assisted Learning, 24, 4(9), 147–151. 284–294. doi:10.1111/j.1365-2729.2007.00260.x Aydin, C. C., & Tirkes, G. (2010). Open source Kluever, R. C., Lam, T. C. M., & Hoffman, E. R. learning management systems in e-learning and (1994). The computer attitude scale: Assessing Moodle. In Proceedings of Education Engineering changes in teachers’ attitudes toward computers. (EDUCON), 2010 (pp. 593–600). IEEE. Journal of Educational Computing Research, 11(3), 251–256. doi:10.2190/484T-CPGX- Bailey, C. J., & Card, K. A. (2009). Effective peda- EUHG-QW8P gogical practices for online teaching: Perception of experienced instructors. The Internet and Higher Kritikou, Y., Demestichas, P., Adamopoulou, E., Education, 12(3–4), 152–155. doi:10.1016/j. Demestichas, K., Theologou, M., & Paradia, M. iheduc.2009.08.002 (2008). User profile modeling in the context of Web-based learning management systems. Jour- nal of Network and Computer Applications, 31, 603–627. doi:10.1016/j.jnca.2007.11.006

138 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Laurillard, D. (2002). Rethinking univer- Russell, M., Bebell, D., O’Dwyer, L., & O’Connor, sity teaching: A conversational framework K. (2003). Examining teacher technology use: for the effective use of learning technologies Implications for preservice and inservice teacher (2nd ed.). New York, NY: Routledge/Falmer. preparation. Journal of Teacher Education, 54(5), doi:10.4324/9780203304846 297–310. doi:10.1177/0022487103255985 Lonn, S., & Teasley, S. D. (2009). Saving time Selim, H. M. (2007). Critical success factors in innovating practice: Investigating perceptions for e-learning acceptance: Confirmatory factor and uses of learning management systems. Com- models. Computers & Education, 49, 396–413. puters & Education, 53, 686–694. doi:10.1016/j. doi:10.1016/j.compedu.2005.09.004 compedu.2009.04.008 Teo, T. (2011). Modeling the determinants of Martin, K., Quigley, M. A., & Rogers, S. (2010). pre-service teachers’ perceived usefulness of Implementing a learning management system e-learning. Campus-Wide Information Systems, globally: An innovative change management 28(2). doi:10.1108/10650741111117824 approach. IBM Systems Journal, 44(1), 125–145. Toral, S. L., Barrero, F. J., Torres, M. R., Gallardo, doi:10.1147/sj.441.0125 S., & Lillo, A. J. L. (2005). Implementation of McGill, T., Klobas, J., & Renzi, S. (2011). LMS a Web-based educational tool for digital signal use and instructor performance: The role of processing teaching using the technological ac- task-technology fit. International Journal on E- ceptance model. IEEE Transactions on Education, Learning, 10(1), 43–62. 48(4), 632–641. doi:10.1109/TE.2005.853074 Moore, M. G. (1989). Three types of interaction. Tucker, A. (2010). Effective practices in e-learn- American Journal of Distance Education, 3(2), ing: An online instructor and learner perspective. 1–6. doi:10.1080/08923648909526659 In J. Sanchez & K. Zhang (Eds.), Proceedings of World Conference on E-Learning in Corporate, Nagar, S. (2010). Study of learning management Government, Healthcare, and Higher Education systems and its effects on distance education. 2010 (p. 1251). International Journal of Educational Administra- tion, 2(2), 323–327. Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal Ngai, E., Poon, J., & Chan, Y. (2007). Empirical of Educational Management, 14(5), 216–223. examination of the adoption of WebCT using doi:10.1108/09513540010344731 TAM. Computers & Education, 48(2), 250–267. doi:10.1016/j.compedu.2004.11.007 Waheed, M., & Jam, A. (2010). Teacher’s intention to accept online education: Extended TAM Ocak, M. (2011). Why are faculty members not Model. Interdisciplinary Journal of Contemporary teaching blended courses? Insights from faculty Research in Business., 2(5), 330–344. members. Computers & Education, 56, 689–699. doi:10.1016/j.compedu.2010.10.011 Wang, S.-C. (2010). University instructor perceptions of the benefits of technology use in Ramayah, T., Ahmad, N., & Lo, M. (2010). The e-learning. In Proceedings of Computer and Elec- role of quality factors in intention to continue us- trical Engineering, (ICCEE ’09). 2nd International ing an e-learning system in Malaysia. Procedia Conference (pp. 580-585). - Social and Behavioral Sciences, 2(2) 5422-5426.

139 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Wang, W., & Wang, C. (2009). An empirical study Supplementary LMS Use: The use of learning of instructor adoption of Web-based learning sys- management system tools to support the traditional tems. Computers & Education, 53(3), 761–774. face-to-face meetings. doi:10.1016/j.compedu.2009.02.021 Pure LMS Use: The use of learning management system for pure e-learning environment. Zhu, C., Valcke, M., & Schellens, T. (2009). A cross- LMS Success Factors: Factors that determines cultural study of online collaborative learning. the success of LMS deployment Multicultural Education & Technology Journal, LMS’s Characteristics: Factors that are re- 3(1), 33–46. doi:10.1108/17504970910951138 lated to the system quality, information quality and service support quality of LMS. Instructor’s Characteristics: Factors that are KEY TERMS AND DEFINITIONS related to the instructor’s individual characteristics such as self efficacy, computer anxiety, technol- Learning Management System: A system ogy experience etc. that provides a platform to support e-learning Organization’s Characteristics: Factors that activities such as communication, collaboration, are related to the organization such as management learning and information/ knowledge transfer. support, training programs and incentives policies.

140 Section 3 Trends and Challenges 142

Chapter 7 A Comparative Study on LMS Interoperability

José Paulo Leal CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal

Ricardo Queirós CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal

ABSTRACT A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study.

INTRODUCTION field this topic is extremely important since there is the need for all systems that typically compose Interoperability is the ability of different computer an eLearning environment to communicate and systems, applications or services to communicate, share data consistently. share and exchange data, information and knowl- The LMS plays a central role in any eLearning edge in a precise, effective and consistent way architecture. Choosing an LMS can be a chal- (Martínez & Navarra, 2007). In the eLearning lenging task for an organization. Several studies have been conducted to analyse and evaluate DOI: 10.4018/978-1-60960-884-2.ch007 these types of systems from pedagogical and in-

stitutional perspectives (Pantel, 1997; Britain & LMS EVOLUTION Liber 1998). However, we are not aware of any study to evaluate the interoperability of LMS with The evolution of eLearning in the last decades other systems typically found in an educational has staggering, from the early monolithic systems institution. developed for specific learning domains to new A major issue in LMS interoperability is the systems featuring reusable tools that can be ef- eLearning standardization. The concept of course, fectively used virtually in any eLearning course. student, educational resource, summary or grade These types of systems evolved from Content must be formally described in order to be shared Management Systems (CMS). The CMS was among all the systems in an educational institution. introduced in the mid-1990s mostly by the online For instance, the difficulty to reuse of a course publishing industry. This type of system can be in schools with LMS from different vendors (or defined as a data repository that also includes even from the same vendor) is an apt example of tools for authoring, aggregating and sequencing the problems found currently in the majority of content. The main goal of these tools is to simplify the LMS. These interoperability issues affect the the creation and administration of online content flexibility of the teaching-learning process and (Nichani, 2009). CMS are focused on content with lead to a decrease of end user satisfaction and the main purpose to store information and provide learning success. access to it. CMS content is organized in small In this chapter we make a comparative study of self-contained pieces of information to improve the LMS support for interoperability. This study reusability at the content component level. These is part of an effort to select an LMS on which to content components when used in the learning base the development of eLearning systems inte- domain are called “learning objects” (LO) and grating heterogeneous components. We chose two the systems that manage them are called Learning LMS vendors - Moodle and Blackboard - since Content Management Systems (LCMS). combined they a have a significant share of the Nowadays, an LMS plays a central role in LMS market and they follow different approaches any eLearning architecture and can be defined to LMS development, namely open source and as software application for the administration, commercial. We analyse the interoperability fea- documentation, tracking, reporting of training tures in these LMS split in two facets reflecting programs, classroom and online events, and train- the broad classes of systems of a typical LMS ing content (Ellis, 2009). Typically it is used by operational environment. These broad classes two types of users’ groups: learners and teach- are Learning Content Management Systems and ers. The learners can use the LMS to plan their Academic Management Systems. learning experience and to collaborate with their This chapter starts by tracing the evolution colleagues; the teachers can deliver educational of LMS. We proceed with the selection of the content and track, analyze and report the learner systems representative of the LMS universe and evolution within an organization. There are open of a methodology for comparing them based in source systems, such as Moodle, Sakai, .LRN or interoperability facets. The following two sec- Dokeos, and commercial systems such as WebCT/ tions analyse separately the learning management Blackboard or Desire2Learn. content facet and the academic management facet. They all feature general tools for delivering For each facet we categorize and identify the most content and for recreating a learning context. From representative system, the existing standards and a course/discipline perspective they provide tools the interoperability issues regarding the com- for handling assignments, managing chat rooms munication with the LMS. In the final section and forums, evaluating multiple-choice tests and we draw conclusions on the results of this study. quizzes, among others. From a learners’ manage-

143 A Comparative Study on LMS Interoperability

ment perspective they provide tools for keeping which can be used, reused or referenced during grade books, managing groups of students, and technology supported learning”. browsing logs. Ashford-Rowe and Malfroy (2009) As every kind of software, LMS continue to organize these tools in four groups, namely: evolve to meet market demands. In relation to interoperability the main trend for the next LMS • Content - Unit/Course online, Lecture and generation is service-oriented architectures (SOA) Tutorial notes, Media (i.e. lectopia, pod- (Dagger & O’Connor & Lawless & Walsh & Wade, cast, videocasts), links to scholarly infor- 2007). In these architectures LMS expose their mation (readings), links to content resourc- functions as services and consume services from es (i.e. websites), interactive resources their operational environments, improving their (.swf .fla .flv and other file types); interoperability with other eLearning systems. In • Communication - Chat, Announcements, fact, the last few years brought us several initia- Discussion Board, Email, Blogs and tives (Smythe, 2003; Wilson & Blinco & Rehak, Forums; 2004) to adapt SOA to eLearning. These initia- • Collaboration - Wikis, Virtual Classroom tives, commonly named eLearning frameworks, and Voice-based communication; have the same goal: to provide flexible learning • Assessment - Quizzes, Reflective learn- environments for learners worldwide. Usually ing journals, Portfolios, Grades, Surveys, they provide a set of open interfaces to numerous Practice activities and past exams. reusable services organized in genres or layers that can be combined in service usage models Recently the eLearning community started (Queirós & Leal, 2010). valuing more the interchange of course content Other trends result from new market demands and learners’ information, which led to the defini- such as Web 2.0, Talent Management, Mobile tion of standards for eLearning content sharing Learning, “Software as a Service” and Open and interoperability. Standards can be viewed as Source Software. With the recent appearance of “documented agreements containing technical Web 2.0 tools and the popularity of social network- specifications or other precise criteria to be used ing tools like Facebook and Twitter, there has been consistently as guidelines to ensure that materials a great demand to use similar tools in the LMS to and services are fit for their purpose” (Nichani, enhance the communication among teachers and 2009). In the eLearning context, standards are students. Talent Management software systems generally developed for the purposes of ensuring are an extension of traditional human resource interoperability and reusability in systems and of management systems. Some researches (Bersin the content and meta-data they manage. In this & Howard & O’Leonard & Mallon, 2009) shows context, several organizations (e.g. IMS GLC, that in 2009 more than 70% of large companies IEEE, ISO/IEC, ADL) are developing specifica- have an LMS already and almost 1/3 of these tions and standards (e.g. IMS CP, IMS CC, IMS companies are considering replacing or upgrading DRI, LOM, SCORM) in the last years (Dagger & these systems with integrated talent management O’Connor & Lawless & Walsh & Wade, 2007). systems (Levensaler & Laurano, 2009). With These specifications are closely related with the more students working at distance, there has been learning object concept as context independent, also a strong demand to make eLearning appli- transportable and reusable pieces of instruction cations accessible through mobile devices (e.g. that are digitally managed and delivered (Rehak Smartphones, PDA) know as Mobile Learning & Mason, 2003). There are other definitions for or m-learning. Using LMS Software as a Service Learning Objects (LO). Rehak & Mason (2003) (SaaS) schools can relieve the financial burden of define a learning object as: “a digitized entity maintaining their LMS by outsourcing the host-

144 A Comparative Study on LMS Interoperability

ing service. Commercial LMS (e.g. Blackboard, The Figure 1 shows that the first major LMS WebCT) have dominated the education market in adopted a commercial development model but previous years, but as costs increase, schools and since the beginning of this century there has been companies are now looking for other options such a shift towards open source systems. In fact, this as open-source solutions (e.g. Moodle, Sakai) that shift was already recognized as a trend in LMS are financially more attractive. development (Davis & Carmean & Wagner, 2009). In spite of the growing popularity of open source, commercial systems are still relevant and they COMPARISON FRAMEWORK must be included in any representative sample of LMS. The goal of this work is to analyse and compare In these two categories we decided to select LMS interoperability features. Given the number the most popular systems taking as reference the of LMS vendors it would be impracticable to study available data on global LMS usage (Davis & them all. Therefore we selected two LMS that we Carmean & Wagner, 2009). As part of this study consider representative of the LMS universe. This we conducted a survey on eLearning systems usage selection is based on their prominence in the LMS on Portuguese higher education institutions. We market and the fact that they cover the open source received responses from 20 different institutions and commercial development models. and the results for LMS usage are shown in Figure Interoperability is in general a complex concept 2. The two most popular LMS in these institu- that can be analysed in multiples perspectives and tions follow the global trend, which reinforces this is surely the case with LMS. To organize our our choice of the reference systems for our study. study we identified two broad classes of systems We decided to focus our study on Moodle and that usually integrate the operational environment Blackboard. We chose them since they represent of the LMS. Thus, we considered two facets in the two main development models used by LMS LMS interoperability, regarding communication vendors (open source and commercial); and com- and data sharing with these classes. bined they have a significant share on the LMS market (33.2% on the international market (Davis Learning Management Systems & Carmean & Wagner, 2009) in 2009 and 80% on our own recent survey). The following para- A good number of LMS that were developed in graphs provide an overview of the selected sys- the past fifteen years are still in use and under tems. active development. For the purpose of our study Moodle (version 1.9.9 - 8th June 2010) is a we must concentrate on a few systems that are free and open-source LMS written in PHP and representative of the LMS universe in terms of created by Martin Dougiamas. Its name is an their characteristics and market share. acronym for Modular Object-Oriented Dynamic A simple categorization of this type of systems Learning Environment. In early January of 2010, is according to their development model. There Moodle had a user-base of 46,624 registered sites are fundamentally two: open source systems, with 32,464,992 users in 3,161,291 courses in 209 such as Moodle, Sakai, .LRN or Dokeos; and countries and in more than 75 languages (Cole commercial systems such as WebCT/Blackboard & Foster, 2007). The most common functions or Desire2Learn. Figure 1 presents a timeline of of Moodle are the course information and docu- the development of several initiatives grouped by mentation, documents repository, announcements, their development model. synchronous and a synchronous communication

145 A Comparative Study on LMS Interoperability

Figure 1. Timeline of development of major LMS

Figure 2. LMS usage in Portuguese higher education institutions

146 A Comparative Study on LMS Interoperability

(email, chat room, discussion forum) and assign- LMS, each corresponding to a different facet in ments. LMS interoperability. We identified also a layer Blackboard (version 9.1 - 1th April 2010) was of infrastructural systems and services that are developed by Blackboard Inc. in 1997 and is an domain independent but that play an important online proprietary virtual learning environment role in LMS interoperability. system that is used by over 3700 educational For the purpose of this study, the broad institutions in more than 60 countries. In Feb- classes of systems that we identified as part of ruary 2006, the virtual learning environment the operational environment of an LMS are the called WebCT (Course Tools) was acquired by following: Blackboard Inc. (Blackboard, 2005) and, as part Learning Content Management Systems of the acquisition terms, the Blackboard brand (LCMS) are used for the development, manage- was assumed until now. ment and publishing of digital learning content (e.g. Learning Objects) that the LMS delivers. Interoperability Facets Examples of these systems are the Learning Ob- ject Repositories, e-Portfolio Systems, Authoring The interoperability features of a system reflect Tools, Specialized Evaluators and others. the operational environment where it is expected Academic Management Systems (AMS) are to be deployed. The operational environment used for managing academic data information of an LMS includes different systems and ser- of an educational institution. Typical features of vices with which it may have to communicate these systems are the management of courses, and exchange data. As depicted in Figure 3 we classes and students, the enrolment of students in identified two broad classes of systems that usu- courses, the submission of summaries and grades ally integrate the operational environment of an by teachers, among others.

Figure 3. LMS interoperability facets

147 A Comparative Study on LMS Interoperability

Apart from these facets the LMS is supported Evaluators or Quizzes. In the following sub-sub- by infrastructure services providing basic func- sections some of these system types are detailed. tions that are not specific to eLearning, such as directory services for authentication and authoriza- Learning Object Repositories tion or printing services. We consider also as part of this infrastructure the web or application server, A repository of learning objects can be defined the database engine and the operative system. In as a “system that stores electronic objects and many cases this infrastructural layer is used for meta-data about those objects” (Holden, 2004). implementing ad hoc interoperability solutions. The need for this kind of repositories is grow- In the following sections the selected systems ing as more educators are eager to use digital are analysed and compared regarding these two educational contents and more of it is available. facets. We categorize and identify the remote Learning object repositories can be organized in systems in each facet, the existent standards and free (CAREO, POOL, CLOE, EducaNext, ARI- the interoperability issues regarding LMS com- ADNE) and payed (Sentient Learning, Harvest munication with those systems. There is a huge Road Hive, Learn Exact, Luminas) (JORUM team, asymmetry among these facets and this is reflected 2006). One of the best known repositories of LOs in the structure of the following sections. Of the is Merlot (Multimedia Educational Resource for two facets, the first has a larger number of systems Learning and Online Teaching) which provides and mature standards. The systems in second facet pointers to online learning materials and includes are mostly home-grown with few and immature a search engine. standards to regulate both content (e.g. academic The Jorum Team made a comprehensive records, course forms, grades, summaries) and survey (2006) of the existing repositories and communication. noticed that most of these systems do not store actual LOs. They just store meta-data describing LOs, including pointers to their locations on the LEARNING CONTENT Web, and sometimes these pointers are dangling. MANAGEMENT FACET Most of the current repositories are specialized search engines of LOs and with little support for The Learning Content Management facet focuses interact with specialized eLearning systems, such on the interoperation with systems that provide as evaluation engines and experimentation envi- pedagogical content and services delivered by the ronments. These systems require both complete LMS. We start by identifying their main types, interoperability and specific Metadata. They need followed by the existing standards for content and service oriented repositories of learning objects, communication, and ending with an example of fully compliant with the existing interoperability system integration in this facet. standards, and supporting new definitions of learning objects for specialized domains. An example System Types of a specialized repository of LO is crimsonHex (Leal & Queirós, 2009), the repository devel- The content delivered by an LMS can be created, oped as part of the EduJudge project to act as a obtained, gathered or evaluated in several types of programming problem repository service to the systems such as Learning Objects Repositories, Evaluation Engine (EE) and the LMS. E-Portfolio systems, Authoring Tools, Specialized

148 A Comparative Study on LMS Interoperability

ePortfolio Systems Authoring Tools

An electronic portfolio is a digital collection of The growing popularity of learning objects lead student work (artefacts) usually managed in ePort- to the development of specialized editors sup- folio systems and displayed for specific audiences porting eLearning metadata. These tools, either and purposes. The ePortfolios systems usually open source, freeware or commercial, export the include (or link) a repository where students or- content to SCORM packages and other formats ganize their artefacts typically for the purpose of such as IMS CP, IMS CC, HTML, PPT, PDF and assessment. The benefits of an ePortfolio system Flash. The most important authoring tools can in an educational institution are shared by students be grouped by their development model such as: and teachers. Students are able to reflect on their educational experiences and showcase their work • Open Source: eXe, Xerte, ScenariChain in a repository. Teachers are able to evaluate the Opale and LOMPad; student progress and provide concrete evidence • Freeware: Hot Potatoes, MyUdutu, MOS of the students’ learning. Solo, Reload and Courselab; Helen C. Barrett (Barret, 2008) organizes the • Commercial: Camtasia, Captivate, ePortfolio tools in two categories: individual and QuizCreator, PPT2Flash, PowerQuizPoint. institutional. Both are presented in Table 1. In the individual category we can use author- The majority of the authoring tools support ing tools to author portfolios offline (requires web multiple application profiles. RELOAD is argu- server space to publish online) or web services to ably the most mature of these projects and is create online and publish a presentation portfolio available both as a standalone Java application allowing interactivity (Web 2.0). In the institu- and as an Eclipse IDE plugin. It supports a broad tional category we can use a software-server where range of metadata formats but cannot be extended an institution installs on their own server to pro- to support specialized formats. The SHAME vide space for hosting portfolios or hosted ser- project (2006) - Standardized Hyper Adaptable vices that an institution adopts (no server required) metadata Editor – stands out from the rest since that host portfolios. it is actually a metadata editing and presentation In the survey we conducted on Portuguese framework for RDF metadata with support for high education institution no one indicated to be all kind of metadata based on a previous map- using an ePortfolio system. This fact allows us to ping for the RDF syntax. Some of these tools are conclude that the dissemination of these tools in specialized in a certain type of multimedia format the educational institutions, at least in Portugal, (e.g. Captivate for video) or activities (e.g. Hot is still low. Potatoes for quizzes) and are the best place for the users to create the respective content.

Table 1. ePortfolio tools by categories

Individual Institutional Authoring tools Web Services Software – Server Hosted Services Mozilla Composer Google Docs Elgg Digication Dreamweaver Zoho Writer Mahara iWebfolio Microsoft Office WikiSpaces OSPI Epsilen Adobe Acrobat Moofolio, MyStuff (embedded in GoogleApps for Education Movie Maker Moodle)

149 A Comparative Study on LMS Interoperability

Specialized Evaluators Packaging (IMS CP, 2007). An IMS CP learning object assembles resources and meta-data into a Examples of eLearning systems that provide distribution medium, typically an archive in zip content can be drawn from different domains. At format, with its content described in a manifest the heart of a system with automatic evaluation file in the root level. The manifest file - named resides an Evaluation Engine (EE). This is an apt imsmanifest.xml -adheres to the IMS CP schema example of a specialized eLearning service, per- and contains the following sections: forming a specific task and reusable in different scenarios. An EE can supply its services not only • Metadata: describes the package as a to LMS but also to other specialized application whole; services, such as quizzes and contest management • Organizations:describes the organization systems. Desktop based applications also fit in of the content within a manifest; this approach. • Resources:contains references to resourc- This model of combining specialized services es (files) needed for the manifest and meta- can be extended to competitive learning in other data describing these resources; domains such as business training, for instance. • Sub-manifests: defines sub packages. In this domain teachers use business simulation games to improve the strategic thinking and de- The manifest uses another standard - the IEEE cision making skills students in particular areas Learning Object Metadata (IEEE LOM, 2002) - (e.g. finances, logistics, and production). Through to describe the learning resources included in the these simulations students compete among them, package (c.f. Sub-subsection 4.2.2). Recently, as they would in a real world companies. A busi- IMS Global Learning Consortium proposed the ness simulation service fulfils a role similar to IMS Common Cartridge (IMS CC, 2010) that that of the EE in programming exercises and it adds support for several standards (e.g. IEEE also requires a repository containing specialized LOM, IMS CP, IMS QTI, IMS Authorization LO describing simulations. An example of a Web Service) and its main goal is to shape the evaluation engine is the UVA Online Judge EE future regarding the organization and distribution (Regueras & Verdú & Castro & Pérez & Verdú, of digital learning content. 2008), the EE developed as part of the EduJudge project to act as an evaluator of programming Metadata problems submitted by students. The content of LO packages is described by meta- Standards data. Its purpose is to support the interoperability and reusability of learning objects. As mentioned In this subsection we introduce several standards previously, the IMS CP manifest contains four related to learning objects. We structured these sections and is precisely metadata that provides an standards in four groups: packaging, metadata, overall description of the package. Metadata can organization and communication. be used to describe file features in the Resource section. In the manifest the Metadata element is Packaging used at two levels: package (overall description of the package) and resource (description of the Packaging is crucial to store eLearning material resource and contained files). In both cases meta- and reuse it in different systems. The most widely data information usually follows the IEEE LOM used content packaging format is the IMS Content schema. The IEEE LOM is a data model used to

150 A Comparative Study on LMS Interoperability

describe a learning object. The model is organized used to design pedagogical activities and articulate in several categories that cover general data, the sequencing of instructions. By default, it uses a such as title and description, technical data such tree-based organization of learning items pointing as object sizes, types and durations, educational to the resources (assets) included in the package. characteristics and intellectual property rights, However, other standards could be accommodated among many others. in this section, such as IMS Simple Sequencing These categories are very comprehensive and (IMS SS) and IMS Learning Design (IMS LD). cover many facets of a LO. However, LOM was These specifications aims to provide to the teachers designed for general LO and does not to meet the mechanisms for coordination of the educational requirements of specialized domains. For instance, instructions based on students’ profile making the there is no way to assert the role of specific re- instruction more dynamic and flexible. sources. Fortunately, IMS CP was designed to be The IMS LD specification is a meta-language straightforward to extend through the creation of for describing pedagogical models and educational application profiles. The term Application Profile goals. Several IMS LD-aware tools are available generally refers to “the adaptation, constraint, and/ as players (e.g. CopperCore, .LRN) and authoring/ or augmentation of a metadata scheme to suit the export tools (e.g. Reload, LAMS). The IMS SS is needs of a particular community”. A well know a specification used to describe paths through a eLearning application profile is the Sharable collection of learning activities. The specification Content Object Reference Model (SCORM, 2009) declares the order in which learning activities are to that extends IMS CP with more sophisticated se- be presented to a learner and the conditions under quencing and Contents-to-LMS communication. which a resource is delivered during an eLearning The IMS GLC is also responsible for another instruction. Despite all these specifications, the application profile, the Question & Test Interop- design of more complex adaptive behaviour is erability (QTI) specification. QTI describes a still hard to achieve. data model for questions and test data and, since version 2.0, extends the LOM with its own meta- Communication data vocabulary. QTI was designed for questions with a set of pre-defined answers, such as multiple The standardization of the learning content it is choice, multiple response, fill-in-the-blanks and not enough to ensure interoperability, which is a short text questions. major user concern with the existing systems. The There are other metadata specifications, such definition of common protocols and interfaces as, the Dublin Core metadata, which provides a for the communication among systems is also an simpler and a more loosely-defined set of elements issue that the major eLearning interoperability useful for sharing metadata across heterogeneous initiatives (e.g. NSDL, POOL, OKI, EduSource, systems. At the present, the Dublin Education IMS) try to address. As an illustration we present Working Group is extending the Dublin Core for the communication guidelines defined by IMS, the specific needs of the education community. arguably the most developed ones in this category. The IMS Learning Tools Interoperability (IMS Organization LTI) provides a uniform standards-based extension point in LMS allowing remote tools and content Learning objects can be organized in items and an to be integrated into LMS. The main goal of the organization defines a path through those items. LTI is to standardize the process for building links The IMS CP specification includes a manifest between learning tools and the LMS. The LTI has section called Organizations. This section can be 3 key concepts (Gilbert, 2009): the Tool Provider,

151 A Comparative Study on LMS Interoperability

the Tool Consumer and the Tool Profile. The ence LMS with the learning content management Tool Provider is a learning application that runs systems usually found in educational institutions. in a container separate from the LMS. Publishes An integration example between a LMS and one one or more tools through the Tool Profiles. The of these systems is also presented. Tool Profile is an XML descriptor that describes how a tool integrates with a tool consumer. It is State of Art composed by information about the tool metadata, vendor information, resource and event handlers The integration with eLearning content manage- and menu links. The Tool Consumer publishes a ment systems can be implemented on the LMS Tool Consumer Profile (XML descriptor of the data or business layer. In the former the integration Tool Consumer’s supported LTI functionality that uses the import / export features of both system is read by the Tool Provider during deployment), and relies on the support of common formats. In provides a Tool Consumer Runtime and exposes the later the integration relies on the existence of the LTI services. compatible web services in both systems. The IMS Digital Repositories Interoperability Data integration is the simplest and most (IMS DRI) specification deals with the commu- popular form of integration in content manage- nication with a specific eLearning system: the ment. For instance, the RELOAD authoring tool repository. Within eLearning, repositories are can be used to create learning objects in SCORM used to store, manage and share LO. One of such format and Blackboard supports and imports efforts was the IMS Digital Repositories (IMS SCORM packages. Table 2 lists some of the most DRI). The IMS DRI specification was created important eLearning content standards and speci- by the IMS Global Learning Consortium (IMS fications defined in the last years by educational GLC) and provides a functional architecture and organizations. For each standard we present the reference model for repository interoperability. LMS support status. The IMS DRI provides recommendations for com- The studied LMS support almost all the LO mon repository functions, namely the submission, package standards with exception of the recent search and download of LOs. It recommends the IMS CC that is only partially supported. In rela- use of web services to expose the repository function to the design and sequencing of learning tions based on the Simple Object Access Protocol activities standards are not yet supported by these (SOAP) protocol, defined by W3C. Due to their LMS, probably due to their complexity. growing popularity other web service interface Data integration assumes an important role in flavours, such as Representational State Transfer the LMS interoperation with system types that do (REST) (Fielding, 2000), should be considered, not require a tight integration, as is the case with since they are not excluded from the recommen- authoring tools. For instance, the Hot Potatoes dation. This will improve interoperability with system enables the creation of quizzes - interactive systems that adhere to a more informal style of multiple-choice, short-answer, jumbled-sentence, development. crossword, matching/ordering and gap-fill exercises - in HTML format. Moodle includes an Integration activity that imports the quiz (HTML file) previously generated in the Hot Potatoes system. It In the majority of the cases an LMS integrates an should be noted that although Moodle supports organization infrastructure in conjunction with the QTI format for quizzes described previously, other systems. In the following sub-subsections we Hot Potatoes cannot export in this format. present the interoperability features of the refer-

152 A Comparative Study on LMS Interoperability

Table 2. Reference LMS support of eLearning content standards

Moodle Blackboard IMS CP yes yes SCORM yes yes IMS CC partial partial IMS QTI yes yes IMS LD no no IMS SS no no

It is possible also to integrate an eLearning tool technology allows third parties to develop modules with an LMS on the business layer. For instance, using the Building Blocks API. For instance, the the IMS Learning Tools Interoperability (IMS company Verbena Consulting LLC created a LTI) provides a uniform standards-based exten- building block that provides a search user interface sion point in LMS allowing remote tools to be that allows searching in the MERLOT repository integrated into LMS. Although this specification and returns matching results along with the meta- is still not explored by the major LMS vendors, data for each learning object. obtaining the certified support for IMS LTI is already a major milestone in their development Example of Integration plan. Another integration approach is through Application Programming Interface (API). The In this subsection we illustrate the use of the com- LMS include APIs to allow developers to extend munication APIs in Moodle, arguably the most their predefined features through the creation of popular LMS nowadays. Concretely we present plugins. Table 3 enumerates the approaches used the new file APIs of Moodle 2.0 and how it was by the selected LMS to address the interoperability used for implementing a plug-in for crimsonHex issues regarding the integration with the system repositories (Leal & Queirós, 2009). The beta types referred in subsection 4.1. version of Moodle 2.0 includes support for dif- Moodle version 2.0 (due in September 2010) ferent types of repositories. Two APIs are already includes several APIs to enable the development available to enable the development of plug-ins of plugins by third parties to access repositories by third parties systems, including: and portfolios (c.f. the following sub-sub-section). Blackboard uses the Building Blocks technology • Repository API for browsing and retriev- to cover the integration issues with other systems. ing files from external repositories; A Building Block is simply a web application that • Portfolio API for exporting Moodle con- runs on the Blackboard application server. This tent to external repositories.

Table 3. Integration APIs in reference LMS

Moodle Blackboard Repositories Repository API Building Blocks API E-Portfolios Portfolio API Building Blocks API Evaluators OPAQUE ws no

153 A Comparative Study on LMS Interoperability

We chose the Repository API since it is the In terms of programming effort we spent half a day most stable of the two. It is organized in two parts: to produce approximately 100 new lines of code. Administration, for administrators to configure For Moodle each repository is just a hierarchy their repositories, and; File picker, for teachers of nodes. This allows Moodle to construct a stan- to interact with the available repositories. dard browse interface. The repository server must Figure 4 presents the file picker GUI of the provide: crimsonHex plug-in. On the left panel are listed the available repositories as defined by the admin- • a URI to download each node (e.g. a LO); istrator. Two crimsonHex repository instances are • a list of nodes (e.g. LO and collections) un- marked with label 1. Label 2 marks the default der a given node (e.g. collection). listing of the selected repository. Pressing the “Preview” link marked with 3 presents a preview In addition to these requirements, a repository of the respective LO. Pressing the “Search” link can optionally support authentication, provide pops-up a simple search form, marked as 4. For additional metadata for each node (mime type, federated search in all available crimsonHex resize, related files, etc.), describe a search facility positories is used the text box marked as 5. The or even provide copyright and usage rules. development of this plug-in was straightforward. Each feature of the plug-in is implemented by a method in a PHP class. A typical method includes:

Figure 4. crimsonHex plugin interface

154 A Comparative Study on LMS Interoperability

a repository invocation (SOAP or REST), the should be noted that most of these evolved from parsing of its response (using a PHP function to home grown systems and are in use in different parse the XML data), a selection of the pertinent schools from the same university or polytechnic data (using XPath) and an iteration over the new institute. In some cases spin-offs were created to results (for instance, populating an array with the develop and commercialize these systems but the relevant data). size of these companies cannot be compared with those developing other types of systems related to eLearning, such as LMS. ACADEMIC MANAGEMENT FACET Standards In this section we analyse the Academic Manage- ment facet. The main system type on this facet is An AMS manages different kinds of information. the Academic Management System (AMS). An The concept of course, student, summary or grade AMS aggregates all the information regarding should be described formally in order to be shared administrative, financial, technical or scientific among all the systems included in a educational processes usual in educational institutions. Ex- institution. As far as we know, there are few stan- amples of these processes are the enrolment of dards that formalize these content types and how students in courses, the management of grades or they are communicated to allow the AMS to share the payment of fees. This interoperability facet is data with other systems. Two know-standards are not as mature as the one analysed on the previous the IMS Learner Information Services (IMS LIS) section and there are still few standards available. and the IMS Learner Information (IMS LIP). This fact burdens the integration of academic man- The IMS Learner Information Services (IMS agement systems with LMS that must resort to ad LIS) is the definition of how systems manage the hoc solutions based on the infrastructural layer. exchange of information that describes people, groups, memberships, courses and outcomes Integration within the context of learning. The IMS LIS, like its predecessor (IMS Enterprise specification), is Unlike in content management, there is a sole type focused on the connection between an LMS and of system in this facet - the AMS - and apparently an AMS. with very few vendors. We were not able to find The IMS Learner Information (IMS LIP) in the literature any study on AMS usage. For this specification addresses the interoperability of reason our analysis is based on the use of AMSs internet-based Learner Information systems with by Portuguese higher education institutions as LMS. It describes mainly the characteristics of a reflected in the survey we conducted for this study. learner. The learner information is a collection of As mentioned before, the questionnaire in- information about a learner (individual or group quired on vendors of different types of eLearning learners) or a producer of learning content (cre- systems in use at each institution. We received ators, providers or vendors). responses from 20 different institutions and the The former is partial supported by Moodle and results for AMS are presented in Figure 5. its implementation in Blackboard is in develop- This data shows that no system is clearly ment. The latter has no support in either LMS. preferred by Portuguese educational institutions. The choices are divided by the systems SIGA, SIGARRA, SOPHIA and Web on Campus. It

155 A Comparative Study on LMS Interoperability

Figure 5. AMS usage

Integration Integration usually includes at least one web application, and these are typically designed based There is an obvious gain in integrating AMS and on the well known three-tier architectural pattern. LMS: avoiding the duplication of processes. For There is a potential for integration in any the three instance, course management is required in both classical tiers: presentation, logic and data. systems and with a tight integration it can be The portal strategy integrates at the presenta- performed in just one of them. Several processes tion tier, providing and unified web interface to can be performed in only one side and reflected in a number of independent subsystems, including the other such as: course management, enrolment eLearning systems. The major advantage of this of students, grades management, summaries man- strategy is the fact that it gives users a sense of agement, exams schedule, absences management. unity, sometimes at the cost of compromising In general, educational institutions use ad hoc consistency. solutions to implement this type of integration. Feature sharing is integration at the logic tier The most common strategies are: and is becoming increasingly popular as more systems expose their functionality using web services. • Portals: aggregating content from multiple Moreover, there are a number of infrastructural sources with a common presentation layer; services, using or not web services, which can be • Database Replication: different applica- exploited by eLearning systems. User authentica- tions but sharing content; tion based in directory services, such as LDAP, is • Features Share: presentation indepen- an apt example of this type of integration. dent but sharing some features (e.g. Finally, integration may occur at the data tier, authentication). and partial database replication is arguably most common example. For instance a LMS may im-

156 A Comparative Study on LMS Interoperability

port data on students, courses and student enrol- in Moodle and Blackboard. This specification ment in courses from administrative systems to promises to be a major step towards content avoid the burden of entering this data manually. interoperability among eLearning systems. Mean- These integration models are usually combined. while, to integrate LMS with content management For instance, a portal that provides and unified systems we must resort to system specific APIs. presentation may also adhere to a single sign-on An example of using Moodle 2.0 Repository API mechanism shared with other services. was presented to illustrate this type of integration. On the academic management facet there are no AMS system standing out from the crowd and CONCLUSION most of those in use, at least in Portuguese higher education institutions, are home grown systems. This chapter presents a comparative study on LMS Standards in this facet are few and immature and interoperability. Given size of this category we not widely supported by existing AMS systems. As focused on a couple of representative systems - a consequence, the integration of LMS and AMS Moodle and Blackboard - since combined they relies on infrastructure services. We presented a represent a significant market share and cover set of integration strategies that are commonly both the commercial and open source develop- used for implementing these ad hoc integrations. ment models. This study is part of an effort to select an LMS We proposed a framework for analysing LMS on which to base the development of eLearning interoperability by distinguishing two different systems integrating heterogeneous components. facets in the way theses systems communicate Unfortunately, from that viewpoint we cannot with their operational environment: learning con- conclude on the superiority of any of the analysed tent management and academic management. We systems. characterized the types of systems that communicate with the LMS trough each facet. Standards are the corner stone of interoperability. Thus we REFERENCES made a comprehensive presentation of the existing standards. We completed the analysis with Ashford-Rowe, K., & Malfroy, J. (2009). E- illustrations of system integration for each facet. learning benchmark report: Learning manage- The main conclusion of this study is that there ment system (LMS) usage. is still a long road ahead in LMS interoperability. Barret, H. C. (2008). Categories of ePortfolio In general it is not straightforward to connect an tools, technical report. Retrieved on December LMS to another system. A lot of work has already 13, 2010, from http://electronicportfolios.com/ been done in defining standards but many of them categories.html. are supported neither by the LMS nor by the system that surround them. The content management Bersin, J., Howard, C., O’Leonard, K., & Mallon, facet is much more developed then the academic D. (2009). Learning management systems. Bersin management facet. Content formats, especially & Associates. those of learning objects, are already mature and Blackboard.com. (2005). Blackboard & WebCT widely supported by the analysed systems. The Announce Agreement to Merge. Retrieved on notable exception is the recent Content Cartridge December 13, 2010, from http://investor.black- of IMS that is not yet supported, as is not its board.com/phoenix.zhtml?c=177018&p=irol- companion specification - the Learning Tools newsArticle&ID=767025. Interoperability - that is still being implemented

157 A Comparative Study on LMS Interoperability

Britain, S., & Liber, O. (1998). A framework IEEE LOM. (2002). IEEE Standard for Learning for pedagogical evaluation of virtual learn- Object Metadata IEEE 1484.12.1-2002. Retrieved ing environments. Retrieved on December 13, on November 12, 2010, from http://www.ieeeltsc. 2010, from http://www.leeds.ac.uk/educol/docu- org/standards/1484-12-1-2002/. ments/00001237.htm. IMS CP. (2007). IMS Content Packaging v1.2 Cole, J., & Foster, H. (2007). Using Moodle - Final specification. Retrieved on November 12, Teaching with the popular open source course 2010, from http://www.imsglobal.org/content/ management system. Sebastopol, CA: O’Reilly packaging/. Community Press. JORUM team. (2006). E-learning repository Common Cartridge Specification, I. M. S. (2010). systems research watch. Retrieved on December Version 1.0 Final Specification. IMS Global 13, 2010, from http://www.jorum.ac.uk/docs/pdf/ Learning Consortium Inc. Retrieved November Repository_Watch_final_05012006.pdf. 12, 2010, from http://www.imsglobal.org/cc/ Leal, J. P., & Queirós, R. (2009). CrimsonHex: A index.html. service oriented repository of specialised learning Dagger, D., O’Connor, A., Lawless, S., Walsh, E., objects. In ICEIS 2009, 11th International Confer- & Wade, V. (2007). Service oriented e-learning ence on Enterprise Information Systems. platforms: From monolithic systems to flexible Levensaler, L., & Laurano, M. (2009). Talent services. IEEE Internet Computing Special Issue management systems 2010. Bersin & Associates. on Distance Learning. Martínez, J. Á., & Navarra, P. L. (2007). Con- Davis, B., Carmean, C., & Wagner, E. D. (2009). tent interoperability on e-learning platforms: The evolution of the LMS: From management to Standardization, digital libraries, and knowledge learning - Deep analysis of trends shaping the management, Revista da Universidad y Sociedad future of e-learning. Sage Road Solutions. LLC. del. Conocimiento. Ellis, R. K. (2009). Field guide to learning man- Nichani, M. (2009). LCMS = LMS + CMS agement systems, ASTD learning circuits. [RLOs] – How does this affect the learner? The Fielding, R. (2000). Architectural styles and the instructional designer? Retrieved on December design of network-based software architectures. 13, 2010, from http://www.elearningpost.com / Phd dissertation. articles/archives/ lcms_LMS_cms_rlos. Gilbert, T. (2009). Leveraging Sakai and IMS Pantel, C. (1997). A framework for comparing LTI to standardize integrations. In 10th Sakai Web-based learning environments. Master’s the- Conference Pearson Education. sis, School of Computing Science, Simon Fraser University, Canada. Holden, C. (2004). What we mean when we say “repositories”, user expectations of repository Queirós, R., & Leal, J. P. (2010). E-learning systems, Academic ADL Co-Lab. Retrieved on frameworks: A survey. In INTED2010 Proceed- December 13, 2010, from http://www.hewlett.org/ ings (pp. 1345-1354). NR/rdonlyres/158FC043-A56F-43C6-ABA7- EB9A62656FCB/0/RepoSurvey2004-1.pdf.

158 A Comparative Study on LMS Interoperability

Regueras, L. M., Verdú, E., Castro, J. P., Pérez, M. Alario-Hoyos, C., Asensio-Pérez, J. I., Bote- A., & Verdú, M. J. (2008). Design of a distributed Lorenzo, M. L., Gómez-Sánchez, E., Vega-Gor- and asynchronous system for remote evaluation of gojo, G., & Ruiz-Calleja, A. (2010). Integration students’ submissions in competitive e-learning. In of external tools in virtual learning environments: International Conference on Engineering Educa- Main design issues and alternatives. Proceedings tion (ICEE ‘08). of the 10th International Conference on Advanced Learning Technologies, ICALT, Sousse, Tunisia Rehak, D., & Mason, R. (2003). Engaging with (pages 384-388). the learning object economy . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable Apostolopoulos, T. K., & Kefala, A. (2003). An approach to e-learning (pp. 22–30). London, e-learning service management architecture. In UK: Kogan Page. Proceedings of the 3rd IEEE International Confer- ence on Advanced Learning Technologies, Athens, SCORM. (2004). 4th Ed. Specification. Retrieved Greece (pp. 140-144). on November 12, 2010, from http://www.adlnet. gov/Pages/Default.aspx. Aroyo, L., Dolog, P., Houben, G., Kravcik, M., Naeve, A., & Wild, F. (2006). Interoperability . SHAME. (2006). SHAME - Standardized Hyper In Personalized adaptive learning. Educational Adaptable Metadata Editor. Retrieved on Decem- Tecnhnology & Society. ber 13, 2010, from http://kmr.nada.kth.se/shame. Barret, H. (2010). Electronic portfolios in STEM Smythe, C. (2003). IMS abstract framework - A - What is an electronic portfolio. Retrieved from review. IMS Global Learning Consortium, Inc. http://www.scribd.com/doc/40206175/E-Portfo- Wilson, S., Blinco, K., & Rehak, D. (2004). An lio-Definition e-learning framework - Paper prepared on behalf Bohl, O., Scheuhase, J., Sengler, R., & Winand, of DEST (Australia), JISC-CETIS (UK), and U. (2002). The shareable content object reference Industry Canada. model (SCORM)-a critical review, Proceedings of the International Conference on Computers in Education (pages 950-951). ADDITIONAL READING Bryden, A. (n. d.). Open and global standards for Aguirre, S., Salvachúa, J., Fumero, A., & Tapiador, achieving an inclusive information society. A. (2006). Joint degrees in e-learning systems: A Casella, G., Costagliola, G., Ferrucci, F., Polese, Web services approach. Collaborative computing: G., & Scanniello, G. (2007). A SCORM thin client Networking, applications and worksharing. architecture for e-learning systems based on Web Al-Smadi & Gutl. (2010). SOA-based architecture services. [Hershey, PA: IDEA Group Publish- for a generic and flexible e-assessment system. ing.]. International Journal of Distance Educa- In EDUCON’10. tion Technologies, 5(1), 13–30. doi:10.4018/ jdet.2007010103 Alario, C., & Wilson, S. (2010). Comparison of the main alternatives to the integration of external Donello, J. (2002). Theory & practice: Learning tools in different platforms, ICERI2010 Proceed- content management systems. ings (pp. 3466-3476).

159 A Comparative Study on LMS Interoperability

Earl, T. (2005). Service-oriented architecture - Nichani, M. (2009). LCMS = LMS + CMS Concepts, technology and design. Upper Saddle [RLOs] – How does this affect the learner? The River, NJ: Prentice Hall. instructional designer? Retrieved from http:// www.elearningpsot.com/articles/archives/lcms_ Eckerson, W. (1995). Server architecture: Achiev- LMS_cms_rlos ing scalability, performance and efficiency in client server applications. Open Information Rehak, D. R., & Mason, R. (2003). Keeping the Systems, 19(1). learning in learning objects . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable Friesen, N. (2005). Interoperability and learning approach to e-learning (pp. 22–30). London, objects: An overview of e-learning standardiza- UK: Kogan Page. tion. Interdisciplinary Journal of Knowledge and Learning Objects. Tastle, J., White, A. & Shackleton, P. (2005), E-learning in higher education: The challenge, Girardi, R. (2004). Framework para coordenação e effort, and return of investment, International mediação de Web services modelados comolearn- Journal on ELearning. ing objects para ambientes de aprendizado na Web. Williams, J., & Goldberg, M. (2005). The evolution Hall, B. (2003). Learning management systems of e-learning. [Global.]. Universitas, 21. and learning content management systems de- mystified. Harasim, L. (2006). History of e-learning: Shift KEY TERMS AND DEFINITIONS happened, The International Handbook of Virtual Learning Environments. Springer. API: Application Programming Interface is Harman, K., & Koohang, A. (2007). Learning a set of rules that a program must follow to ac- objects: Standards, metadata, repositories, and cess the services provided by other program that LCMS. Informing Science Institute, Edição de implements that API. Informing Science. eLearning: A new form of learning based on technology. A model of teaching and learning Hatala, M., Richards, G., Eap, T., & Willms, J. based on the online environment, leveraging the (n. d.). The Interoperability of learning object capabilities of the Internet for communication and repositories and services: Standards, implementa- content distribution tions and lessons learned. Proceedings of the 13th ePortfolio: A web collection of electronic international World Wide. evidence packaged and managed by a user. These Holden, C. (2004). What we mean when we say evidences are stored in an ePortfolio tool with “repositories”, User expectations of repository a twofold set of features: students organize its systems. In: Academic ADL Co-Lab. achievements and teachers use them to assess the work and evolution of students. Malita, L. (2009). E-portfolios in an educational Interoperability: Ability of systems to inter- and ocupational context. Procedia - Social and Be- operate in a uniform way. The interoperability havioral Sciences, 1(1), 2312-2316. doi:10.1016/j. among systems is usually achieved using standards sbspro.2009.01.406. and specification on content and communication. LO (Learning Objects): Units of instructional content that can be used, and most of all reused, on web based eLearning systems. Usually it’s a

160 A Comparative Study on LMS Interoperability

set of files including a XML manifest wrapped role in the eLearning realm coordinating and in- with resources and packaged in a ZIP file. tegrating a set of features by third-party systems. LMS: System used mainly by teachers and Metadata: Data about data. Metadata will students to manage the teaching/learning process. allow systems to index, search and retrieve files Nowadays these type of systems occupy a central based on semantic data.

161 162

Chapter 8 Mobile Learning Management Systems in Higher Education

Demetrios G. Sampson University of Piraeus & Centre for Research and Technology Hellas, Greece

Panagiotis Zervas University of Piraeus & Centre for Research and Technology Hellas, Greece

ABSTRACT Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle.

INTRODUCTION of teaching and learning, offering meaningful learning experiences that bare the potential to During the last years, several studies have been address the shortcomings of traditional classroom- reported that Technology-Supported Education based learning (Catherall, 2004; Tham & Werner, can be effectively used in Higher Education (HE) 2005; Bonk & Graham, 2006; Mayes et al, 2009). for enhancing and enriching the traditional ways Technology-Supported Higher Education can offer a number of services to all relevant actors in HE DOI: 10.4018/978-1-60960-884-2.ch008 (namely, students, tutors, administrators and the

educational organization) such as: (a) transparent reaching a larger and globally dispersed audience access to courses’ materials and activities, as well beyond traditional campuses. as, participation to synchronous and asynchro- On the other hand, the widespread ownership nous group-based course activities, (b) on-line of mobile devices and the growth of mobile com- submission, marking and feedback on students’ munications industry has offer the potential for assignments, (c) effective students’ course activi- the provision of new services including, internet ties tracking, (d) efficient student enrollment and access without place and device constraints, monitoring, and (e) on-line delivery of courses interpersonal and group communication with- offered by an educational organization, reach- out place and time restrictions and sharing of ing students outside campus. These services can digital content in any format (text, image, audio enhance the opportunity of learning to anyone, and video) (Naismith et al., 2005). These new anywhere and at anytime without place and time services offered by mobile devices can be used restrictions and promote wider participation, re- for educational purposes aiming to enhance tra- moving the traditional barriers to Higher Education ditional classroom-based and/or desktop-based studies (Laurillard, 2005). web-facilitated educational experiences. This has Learning Management Systems (LMS) are lead to educational initiatives that investigate the among the most widely used applications in potential of mobile devices as the technological Technology-Supported Education, providing a means to change the way that students interact convenient way to organize and deliver educa- with their tutors, their classmates, the learning tional and training e-services in formal educational material, the administration services and the en- settings (Weller, 2007; Cole & Foster, 2007). LMS vironment of their educational institute. This is are now considered as mainstream applications important towards supporting the continuation of for the organization, management and delivery these interactions not only outside the classroom, of on-line courses in Higher Education, since but also beyond desktop restrictions towards to a they enable efficient planning, implementation, truly constant and instant access from anywhere. administration, tracking and reporting of educa- (Kukulska-Hulme & Traxler, 2005). tional and training activities (Kim & Bonk, 2006). Within this context, providing access to LMS LMS provide to students facilities for enrolling via mobile devices bares the potential to enhance to on-line courses, accessing lecture notes and Technology-Supported Higher Education and supportive course material, communicating with achieve additional benefits for students, tutors, their classmates and their tutors through online administrators and the educational organization. discussions, participating to on-line assessments, More precisely, mobile LMS (mLMS) can pro- as well as, monitoring their progress and grades. vide to students new opportunities for accessing Moreover, LMS enable tutors to organize their courses’ materials and for communicating with courses’ syllabus and teaching material, gather, their classmates and their tutors beyond desktop grade and provide feedback to students’ assign- restrictions. Tutors can use the mobile devices, so ments, track students’ progress and participation to as to instantly and continuously monitor issues their courses and communicate with their students related with their courses (i.e. timely submission for answering their questions. Furthermore, LMS of students’ assignments, questions that have provide to the administrators efficient ways for submitted from the students etc) while they are assigning tutors to the courses and administrating on the move. System administrators can carry out enrolled students. Finally, LMS are offering to basic system support tasks even without access the Higher Educational Institutions an effective to their desktops. Finally, the Higher Educational way for on-line delivery of their courses towards

163 Mobile Learning Management Systems in Higher Education

Institutions are able to make their on-line courses ganization, delivery and administration of on-line truly accessible to their students. courses (Weller, 2007). Over the past years, LMS As a result, the issue of developing mobile have attracted the attention of Higher Education LMS is important for the deployment of feasible community, since their available features can mobile supported educational services in Higher simulate the features of classroom-based teaching Education. In this book chapter, we target ad- (Watson & Watson, 2007). Nowadays, LMS are dressing this issue by studying and applying the considered as mainstream applications for most Mobile Web Best Practices 1.0 (proposed within HE Institutions (Catherall, 2004; Tham & Werner, the World Wide Web Consortium) (Rabin & Mc- 2005; Weller, 2007) and they can offer important CathieNevile, 2006) to a widely used existing benefits to all relevant actors in Higher Education LMS, namely, the Moodle, so as to be accessible (namely, students, tutors, administrators and the via mobile devices. The chapter is structured as fol- educational organization), which are presented lows: Following this introduction, the background below (Bouhnik & Marcus, 2006; Daniels, 2009): section discusses Learning Management Systems and their benefits in Higher Education, the mobile • Students are able to enroll to on-line cours- devices and their anticipated advantages in the es and have access to lecture notes and educational process, the international guidelines courses’ supportive material at anytime for developing web applications and/or content and from anywhere. They are also able to for mobile devices, as well as related works and study specific components of an on-line studies in the field of mLMS development. The course in a more flexible way beyond the next section presents the main design consider- restrictions imposed by the traditional ations for the development of the proposed mo- classroom. Additionally, students are able bile Moodle system, demonstrates how the W3C to continue interaction and communication Mobile Web Best Practices have been employed with their classmates and tutors beyond in the development of mobile Moodle by giving campus physical presence, through on-line examples of our implementation and presents the course activities. Finally, LMS provide validation process followed, so as to ensure con- students with opportunities for participat- formance of the developed mobile Moodle with ing in on-line assessments and receiving the current W3C Mobile Web Best Practices 1.0. instant feedback for their performance, as Finally, we discuss future and emerging trends in well as opportunities for monitoring their the field of mobile LMS development, as well as individual progress and building their own our concluding remarks. e-portfolios of academic achievements. • Tutors are able through the LMS to organize and manage their courses’ syllabus, BACKGROUND upload and share courses’ materials with their students, as well as, gather and grade Learning Management Systems more efficiently students’ assignments. Moreover, LMS offer effective ways for Learning Management Systems (LMS), also re- tutors to communicate and interact with ferred to as Course Management Systems (CMS) their students and to provide feedback to or Virtual Learning Environments (VLE), are them, as well as to monitor their students’ web-based applications (that is, they run on a academic progress and access their grades, server and are accessed by using a web browser) so as to provide additional guidance and designed to support planning, implementation, or-

164 Mobile Learning Management Systems in Higher Education

Table 1. Key features of LMS to the key actors in HE

Features Students Tutors Administrators Educational Organization Upload and share course materials √ √ Access and download lecture notes and course sup- √ portive material Participation to online synchronous and/or asynchronous √ √ activities Create online assessments √ Participation to on-line quizzes/surveys √ Access to online grade book √ √ √ Gather and review assignments √ Create and manage course syllabus √ √ Students and Academic Personnel management and √ √ administration Courses management and administration √ √ Students’ progress tracking √ √ √ √ Summative students’ statistics √ √

support to students that are facing difficul- There are many different LMS available, which ties with their courses. could be divided in two main categories: com- • Administrators can manage more efficient- mercial and open source. Commercial LMS in- ly the administrative tasks related with the clude: Blackboard/WebCT (http://www.black- on-line course organization, such as moni- board.com), Desire2Learn (http://www. toring students’ enrollment, managing stu- desire2learn.com/) and Lotus (http://www.lotus. dents’ records, as well as tutors allocation com). On the other hand, open source LMS include: to the available on-line courses. Moodle (http://moodle.org), ATutor (http://www. • Educational organizations have concrete atutor.ca/) OLAT (http://www.olat.org) and Sakai benefits from LMS by extending the audi- (http://www.sakaiproject.org/). ence of their courses and by increasing the For the purpose of our work, we selected flexibility of studying and teaching at these Moodle (http://moodle.org) as the LMS to be institutions removing the barriers of physi- customized in a manner that it can be effec- cal presence at campus and/or classrooms. tively accessed by mobile devices following the Furthermore, LMS can offer efficient guidelines of W3C Mobile Web Best Practices management of HE institutions Human 1.0. Moodle is an open source environment that Resources (academic and administrative) allows the development of new functionalities and support the implementation of quality and/or customization of the existing ones and it is management plans. offered at no cost. It provides compatibility with current learning technology specifications such as Table 1 summarizes key features of LMS to the SCORM (http://www.adlnet.gov/scorm/) and IMS key actors (namely, students, tutors, administra- CP (http://www.imsglobal.org/content/packag- tors), as well as to the educational organization ing/). Moodle has a large and active community (Weller, 2007; Cole & Foster, 2007). of users (including educational organizations that

165 Mobile Learning Management Systems in Higher Education

use it as their LMS, tutors that deploy it for their Designing Web Applications individual courses, as well as, web developers and/or Content for Mobile Devices who are continually modifying and expanding its source code). This offers a solid setting of Typically, mobile devices have a number of tech- large scale deployment and an environment for nical restrictions (such as limited memory, low community-based further development. screen resolution, insufficient processor power, low network bandwidth and limited input possibili- Mobile Devices ties), which affect both the web content and the web applications that they can handle. Furthermore, Mobile devices, also referred to as handheld there are not available yet globally agreed standard devices or handheld computers, are pocket-sized ways for the design considerations of mobile web computing devices with capabilities for internet applications. To this end, the World Wide Web connectivity (either via a wireless network or a Consortium (W3C) has proposed an initial set of mobile communication network), typically hav- guidelines for Mobile Web Best Practices 1.0 that ing a display screen with touch input and/or a can be taken into consideration when developing mini keyboard (Naismith et al., 2005). Based on web content and/or web application for mobile this definition and for the purpose of our work, devices (Rabin & McCathieNevile, 2006). The examples of mobile devices are: the personal main objective of these guidelines is to improve digital assistants (PDAs), the smart phones and the user experience when web content and ap- the mobile phones. During the last years, mobile plications are accessed from different mobile devices have been used for educational purposes devices. The W3C Mobile Web Best Practices and there are many studies reporting the benefits 1.0 guidelines are grouped under the following that they are offering to the educational process, main sections (Rabin & McCathieNevile, 2006): which could be summarized below (Sharples, 2006; Pettit & Kukulska-Hulme, 2007; Corbeil • Overall Behavior: This section includes & Valdes-Corbeil, 2007; Faux et al, 2006): general guidelines related to the delivery of web content and/or web application to • Mobile devices can facilitate students to be accessed via mobile devices. continue learning activities, initiated inside • Navigation and Links: This section in- the traditional classroom, outside the class- cludes guidelines related with the defini- room through their constant interaction tion of the structure and the navigation and communication with their classmates model of a web page or a web site so as to and/or their tutors without place, time and match the technical limitations of mobile device restrictions. devices (such as display size, input mecha- • Mobile devices can enable tutors to interact nisms, etc). with their students while on the move, to • Page Layout and Content: This section organize, update and monitor their courses, includes guidelines related with the user’s their course activities and their teaching perception of the delivered content and pro- material from anywhere and at anytime. vides recommendations for the language • Furthermore, mobile devices can offer im- used in its text and the spatial relationship portant facilities in including contextual between its constituent components. and situated learning activities in tradition- • Page Definition: This section includes al on-line course. guidelines related with technical aspects on how elements (such as tables, images

166 Mobile Learning Management Systems in Higher Education

and video) should be presented in mobile device taking advantage of its particular technical web content. characteristics, its main limitation is that a differ- • User Input: This section includes guide- ent version of the stand-alone application should lines related with user input taking into be developed for every different type of mobile consideration mobile devices restrictions, device. This can be cost-effective only for certain such as lack of pointing devices and stan- cases where a mobile device has a significant dard keyboards for text entry, as well as, market penetration. mobile devices alternative input mecha- The second approach, which was adopted also nisms, such as through touch screens. in our work, includes the customization of the original LMS, so as to enable access via mobile Related Work: Designing Mobile devices without device-specific restrictions. An Learning Management Systems important design consideration of this approach is that the customization of the mobile LMS should The review of related works in the field of Mobile conform to the W3C Mobile Web Best Practices, Learning Management Systems design reveals that so as to ensure that mLMS can be accessed by there are two main approaches. The first approach any type of mobile device. An initial work based is device-dependent and involves the design (in on this approach has been proposed by Houser accordance to the particular targeted mobile device & Thornton (2005), who customized limited technical characteristics) and the development of Moodle features (such as polls, quizzes, wikis stand alone applications that can be installed to and forums), in order to provide access to these a specific mobile device, so as to provide access features through mobile phones. Another work to the course data of the original LMS. The main following this design approach has been proposed commercial LMS (used by a number of Higher by Cheung, Stewart and McGreal (2006), who in- Education Institutions worldwide) are following vestigated the customization of Moodle for use via this approach and provide stand-alone applications mobile devices within the Athabasca University of the original LMS, Typical examples of such in Canada. The limitation of these efforts is that mobile LMS are: Blackboard Mobile (http://www. they do not claim conformity of the developed blackboard.com/Mobile/), Litmos Mobile (http:// mLMS with W3C Mobile Web Best Practices and, www.litmos.com/mobile-learning/) and Meridian moreover, that the customized Moodle features Mobile LMS (http://www.meridianksi.com/prod- are limited. In our work, we propose the design ucts/mobile_lms/). Moreover, this approach has and the implementation of a server-based mobile been also adopted from some development groups version of Moodle with selected features suit- that are working with open source LMS (mainly able for Higher Education that follows the W3C Moodle). Typical examples are stand-alone ap- guidelines for Mobile Web Best Practices 1.0. plications of Moodle for specific mobile devices such as Apple’s iPhones or smartphones that run the Google Android operating system. There are THE MOBILE MOODLE also cases of Higher Education Institutions that they developed and use stand-alone applications Design Considerations of Moodle, such as the Polytechnic University of Catalonia (Forment & Guerrero, 2008) and In order for a Learning Management System to be University of Belgrade (Minović et al., 2008). designed to support access via mobile devices, we Although the aforementioned design approach should take into account the following design con- can achieve optimized performance for a specific siderations (Keegan, 2005; Sampson et al, 2008):

167 Mobile Learning Management Systems in Higher Education

• Students, tutors and administrators should device then the same information is presented to be able to access the LMS via any type of him/her in an appropriate customized manner. mobile device. The source files of Moodle that have been • Students should be able to enroll in and at- customized in order to create the mobile version tend courses via their mobile device, check of the system are the following: for new course material, upload assignments, send questions to their tutors and • Files for defining content format: that is, communicate with their classmates and tu- the CSS (Cascading Style Sheets) files that tors through discussion forums. contain all the appropriate declarations for • Tutors should be able to conduct basic content format (font size, font color etc.). course management tasks via their mo- • Files for defining content presentation: that bile device such as: monitor their students’ is, the basic source files (PHP or HTML progress, identify and download newly files) that define content presentation (i.e. uploaded students’ assignments, answer information displayed in an array form). students’ questions and communicate with • Files for calling features functions: Mainly their students through discussion forums. PHP files that contain calls of Moodle fea- • Administrators should be able to carry out tures functions. For example in this type basic administrative tasks such as: add new of files the feature of calendar is called. users to the mLMS and allow enrollment If we want to remove the calendar in the of new students to available courses. mobile version of Moodle, then we should delete the corresponding part of code that According to the aforementioned consid- calls the calendar feature. It is important erations we decided about the features that the to note that we can remove also the calen- mobile version of Moodle should support. Every dar feature from the administrative part of course in Moodle consists of resources, activities, Moodle, but this would have impact on the and blocks (Cole & Foster, 2007). Resources PC version of the system because, as men- provide the appropriate tools for adding content tioned before, the two versions share the such as web pages and links to web sites while same database. Thus, any changes in one activities provide the appropriate tools to add version would simultaneously change the course activities such as forums, quizzes, and other version, too. assignments. Blocks are some additional features for course support such as a list of all participants System Implementation or a calendar. Table 2 presents the main features of the proposed Mobile Moodle In order to customize Moodle to support access The implementation of our Mobile Moodle via mobile devices we applied a set of guidelines consists of two sub-systems, the PC version and selected from Mobile Web Best Practices 1.0, the Mobile version, which communicate through which was discussed in the background section. a common database in order for users to have Table 3 presents the selected guidelines, as well as access to the same course material; by identifying their description on how they can be implemented. the device used to access it (PC or mobile device). An implementation example of Guideline 1 is As presented in Figure 1, when a user accesses that a user can type the URL “ask4reasearch.info/ Moodle through a laptop/desktop PC then the moodle” instead of having to type “http://www. information is presented to him/her ordinarily. ask4reasearch.info/moodle/pda” which is the full When a user accesses Moodle through a mobile path for accessing the mobile Moodle version.

168 Mobile Learning Management Systems in Higher Education

Figure 1. The adopted mobile Moodle approach

Figure 2(a) represents an implementation Figure 2(b) represents an implementation example example of Guidelines 2 and 4. The basic naviga- of Guidelines 5 and 7. Scrolling is limited to only tion of the website should be placed on the top of one direction (vertical) (implementing Guideline the page and preferably on a single line. Moodle 5). The fonts and colors used (defined in CSS by default uses an appropriate menu at the top files of the theme adopted) provide sufficient of each page conformant with Guideline 2, so contrast of text colors and background color so no further adaptation has been made in relation that information is visible and readable from users to this guideline. Guideline 4 that indicates the (implementing Guideline 7). use of clear language is of particular importance Figure 3(a) represents an implementation for mobile delivery to assist users determine example of Guidelines 6, 10 and 13. The user whether information is of interest to them. Thus, should not have to scroll significantly to find the the information presented in the system is given primary content of the page. Therefore the most in clear and simple language (i.e. a link to lecture important information (i.e. a forum for posting material) while headings are used to group similar questions related to the course), should precede items in order for a user to distinguish information information that is not so important (i.e. list of presented in a page (i.e. an area that presents the courses that a user is enrolled) (implementing Course Activities) (implementing Guideline 4). Guideline 6). Informing the browser in advance

169 Mobile Learning Management Systems in Higher Education

Table 2. Key features of proposed server-based Mobile Moodle

Features Description Insert a label Insertion of textual description with the possibility of image input. Compose a text page Creation of a simple page of text. Compose a web page Formatting options for a web page including more that text Link to a file or a Creation of hyperlinks to files uploaded to Moodle web site or to web sites outside Moodle. Resources website Display a Directory Organizing files already uploaded to the system in directories. Choice A simple poll (multiple choice question). Forum Threaded discussion board. Glossary Dictionary of terms. Quiz A web-based quiz with a variety of question types, such as multiple choice, true/false, short answer and matching.

Activities Assignments Advanced uploading of files: Learners can upload to the system a specific number of files (the maximum number of files that can be uploaded is defined by the tutor). Upload a single file: Same functionality with “Advanced uploading of files” only that a single file is allowed to be uploaded. Online text: Learners answer to a question placed by the tutor in simple text format. Offline activity: The tutor defines the assignment without the possibility for file uploading. Administration Contains all the administration tools of a course such as students’ registration, grades, course backup etc. Courses A list of all courses in which a student has enrolled. Activities A list of hyperlinks with all the activities added to the course (i.e. forum, assignment). Blocks Search Forums Search based on keywords in all the existing forums in the course. Calendar A calendar for displaying various course or site events. Participants Link to a list of all participants in the current course. about the size of an image avoids the re-flow of or not (i.e. a link leading to course material). Also a page when the browser is handling it. Thus, in it would be helpful to identify the implications of Mobile Moodle the explicit definition of the im- following a link by denoting the size of the tar- ages’ size is given (i.e. height=”16px” and geted file (i.e. 250KB) and/or its format (i.e. PDF) width=”16px”) (implementing Guideline 10). (implementing Guideline 3). It is also helpful to Given the input limitations of mobile devices, the use features of the markup language (i.e. HTML) interface of the system must minimize user input to indicate logical document structure in a page. as far as possible. Thus, in current implementation Moreover, tables do not work well on limited size of mobile Moodle limited typing is required (i.e. screens and by putting for example, navigational typing a keyword to find and access a certain links into them may result for the user to scroll thread in a forum) while it is possible for users to both horizontally and vertically to see possible select items using the device’s navigation keys navigational choices. A simple and easy to use (i.e. select Questions Forum) (implementing alternative to tabular representation, which at the Guideline 13). Figure 3(b) represents an imple- same time can indicate clear document structure, mentation example of Guidelines 3, 8 and 9. It is is the use of ordered and/or unordered HTML important to provide concise and descriptive link lists. For example two headings each one followed text to help users decide whether to follow a link

170 Mobile Learning Management Systems in Higher Education

Table 3. The W3C Mobile Web Best Practices guidelines implemented in Mobile Moodle

Guidelines Description 1. URIs of Site Entry Points Keep the URIs of site entry points short. 2. Navigation Bar Provide only minimal navigation at the top of the page.

Links 3. Link Target Identification Clearly identify the target of each link and note the target file’s format unless you know the

Navigation and device supports it. 4. Page Content Ensure that content is suitable for use in a mobile context, use clear and simple language and limit content to what the user has requested. 5. Scrolling Limit scrolling to one direction, unless secondary scrolling cannot be avoided. 6. Navigation Bars etc. (Extra- Ensure that material that is central to the meaning of the page precedes material that is not. neous material) 7. Color Ensure that information conveyed with color is also available without color and ensure that Page Layout and content foreground and background color combinations provide sufficient contrast. 8. Structural Elements Use features of the markup language to indicate logical document structure. 9. Tables Do not use tables unless the device is able to support them. Do not use nested tables. Do not use tables for layout and where possible, use an alternative to tabular presentation. 10. Image Size Specify the size of images in markup and resize images at the server. 11. Measures Do not use pixel measures and do not use absolute units in markup language attribute values and style sheet property values. Page Definition 12. Style Sheets Use style sheets to control layout and presentation, unless the device is known not to support them, organize documents so that if necessary they may be read without style sheets and keep style sheets small.

- 13. Input Keep the number of keystrokes to a minimum, avoid free text entry where possible and provide In put User pre-selected default values where possible.

Figure 2. Implementation example of W3C mobile Web best practices guidelines (i)

171 Mobile Learning Management Systems in Higher Education

by a number of links represented as an un-ordered .navbar { list (implementing Guidelines 8 and 9). margin-right: 1em; Finally, an implementation example of Guide- width: 100%; lines 11 and 12 is given below. Style information padding: 0em; may be contained in an externally linked style height: 25em; sheet if the device supports use of style sheets. border-width: 1em; Current implementation demands that the device border-style: solid; used to access mobile Moodle supports usage of } style sheets (implementing Guideline 12). More- over, avoiding pixel and absolute measures to Validation Process define the layout allows the browser to adapt the content presented to fit in the display. An excep- In order to validate the conformance of the de- tion to this rule is the situation in which an image veloped mobile Moodle with the W3C Mobile has been specifically sized for a particular display Web Best Practices 1.0, we used an automated as mentioned in Guideline 10 (implementing validation tool provided by W3C, namely, the Guideline 11). The example below defines the mobileOK Checker (http://validator.w3.org/mo- layout of the navigation bar used in mobile bile/). This tool performs various tests on a web Moodle where the width of the bar is given in page to determine its level of mobile-friendliness percentage format (i.e. width: 100%) and all the by conforming it to the W3C Mobile Web Best other style elements are relatively sized (i.e. height: Practices 1.0. When a web page passes all tests 25em). then it is characterized as mobileOK page and it is considered suitable to be accessed by any mobile device. In our case, all different web pages of the

Figure 3. Implementation example of W3C Mobile Web Best Practices guidelines (ii)

172 Mobile Learning Management Systems in Higher Education

developed mobile Moodle were validated with • LMS are used as a ‘one-size-fits-all’ ser- the mobileOK Checker and for all them the tool vice without taking into consideration indi- returned with mobile conformance higher than vidual student learning styles, knowledge 98%. For those pages that the tool did not return level, goals and interests. In such situation, with full mobile conformance, we corrected the all learners receive same activities and ma- source files that correspond to these pages based terial, regardless of their varying pre-exist- on the recommendations provided by the tool. ing knowledge and experience. This process was repeated, so as to achieve full • LMS are promoted as ways to manage stu- conformance to W3C Mobile Web Best Practices dents, rather than to promote rich, interac- 1.0 for all web pages of mobile Moodle. tive experiences.

Within this framework, introducing adaptive FUTURE RESEARCH DIRECTIONS techniques to mobile LMS, so as to provide students with educational material and activities that In our work, we propose the design and the fits their learning styles, knowledge level, goals implementation of a server-based mobile ver- and interests, is attracting increased research at- sion of Moodle with extended features suitable tention. An initial study towards the development for Higher Education that follows the W3C of an Adaptive Mobile LMS has been proposed by guidelines for Mobile Web Best Practices 1.0. Park et al (2008) and introduces the application This is important for the deployment of feasible of an artificial intelligence technique to a mobile mobile-supported educational services in Higher educational device in order to provide a Learning Education. However, it is only an initial step for Management System that is adaptive to students’ the future evolution of mLMS. An emerging trend learning styles (Park et al, 2008). in developing mobile LMS is to exploit current mobile devices enhanced features like position, acceleration sensors or cameras, towards detect- CONCLUSION ing users’ intentions and gathering of contextual information, aiming to provide him/her with per- Technology-Supported Education exploits the sonalized situated context-aware educational potential of web technologies for enhancing and services on-demand. An initial study towards the enriching the traditional ways of teaching and development of a mobile LMS, which gathers learning, offering meaningful learning experiences contextual information and reasoning on them, has that bare the potential to address the shortcomings been recently proposed by Lehsten et al (2010). It of traditional classroom-based learning in which enables personalized mobile learning experience tutors and students must physically participate with location-sensitive lecture streaming, campus in the learning process. LMS are widely used navigation, and ubiquitous features of the whole web-based applications in Technology-Supported university computing infrastructure (Lehsten et Education, providing a convenient way to organize al, 2010). and deliver educational and training e-services in Another trend in developing mobile LMS aims formal education settings such as Higher Educa- to overcome identified limitations of LMS, such tion. On the other hand, the widespread ownership as (Zhang et al, 2004; Liu, 2004): of mobile devices and the potential benefits that they can provide to the learning process has attracted the attention of both researchers and prac-

173 Mobile Learning Management Systems in Higher Education

titioners in Technology-Supported Education. This Corbeil, J. R., & Valdes-Corbeil, M. E. (2007). has lead to mobile devices-supported educational Are you ready for mobile learning? EDUCAUSE initiatives that investigate the potential to change Quarterly, 30(2). the way that students interact with their tutors, their Daniels, P. (2009). Course management systems classmates, the learning material, the administra- and implications for practice. International tion services and the environment of their edu- Journal of Emerging Technologies & Society, cational institute, supporting the continuation of 7(2), 97–108. these interactions not only outside the classroom, but also beyond desktop restrictions towards to a Faux, F., McFarlane, A. E., Roche, N., & Facer, truly instant access from anywhere. As a result, the K. (2006). Handhelds: Learning with handheld development of mobile LMS (mLMS) is important technologies. Bristol, UK: Futurelab. for the deployment of feasible mobile-supported Forment, M., & Guerrero, J. C. (2008, April 11- educational services in Higher Education. Within 13). Moodlbile: Extending Moodle to the mobile this context, in this book chapter, we propose the on/offline Scenario. Proceedings of the IADIS design and the implementation of a server-based International Conference on Mobile Learning, mobile version of Moodle with selected features Algarve, Portugal. suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0. Houser, C., & Thornton, P. (2005). Poodle: A course-management system for mobile phones. In Proc. of the 4th IEEE International Workshop on REFERENCES Wireless and Mobile Technologies in Education (WMTE ’05). IEEE Computer Society. Bonk, C. J., & Graham, C. R. (2006). The handbook of blended learning. San Francisco, CA: Pfeiffer. Keegan, D. (2005). Mobile learning: The next generation of learning. Distance Education In- Bouhnik, D., & Marcus, T. (2006). Interaction in ternational. distance-learning courses. Journal of the American Society for Information Science and Technology, Kim, K., & Bonk, C. J. (2006). The future of 57(3), 299–305. doi:10.1002/asi.20277 online teaching and learning in higher education: The survey says. EDUCAUSE Quarterly, 29(4). Catherall, P. (2004). Delivering e-learning information services in higher education. Oxford, UK: Kukulska-Hulme, & A., Traxler, J. (2005). Mobile Chandos Publishing. learning – A handbook for educators and trainers. UK: Taylor & Francis. Cheung, B., Stewart, B., & McGreal, R. (2006) Going mobile with MOODLE: First steps. IADIS Laurillard, D. (2005). E-learning in higher educa- International Conference on Mobile Learning tion. In Ashwin, P. (Ed.), Changing higher educa- 2006, Dublin, Ireland. tion: The development of learning and teaching. London, UK: RoutledgeFalmer. Cole, J., & Foster, H. (2007). Using Moodle: Teaching with the popular open source course management system, (2nd Edition). Sebastopol, CA: O’ Reilly Community Press

174 Mobile Learning Management Systems in Higher Education

Lehsten, P., Zender, R., Lucke, U., & Tavangarian, Sampson, D., Papapaulou, P., Zervas, P., & Pa- D. (2010, March 29-April 7). A service-oriented panikou, C. (2008). Supporting higher education approach towards context-aware mobile learning teaching through wireless and mobile devices: management systems. 8th IEEE International The case study of mobile Moodle. In Proc. of Conference on Pervasive Computing and Com- the AACE E-Learn 2008-World Conference on E- munications Workshops (PERCOM Workshops) Learning in Corporate, Government, Healthcare, (pp 268-273). and Higher Education. Liu, Y. (2004). Faculty development and CMS. In Sharples, M. (Ed.). (2006). Big issues in mobile Proceedings of Society for Information Technology learning. Report of a workshop by the Kaleido- in Teacher Education International Conference, scope Network of Excellence Mobile Learning 2004, 2409–2412. Initiative. Nottingham, UK: University of Not- tingham. Mayes, J. T., Morrison, D., Bullen, P., Mellar, H., & Oliver, M. (Eds.). (2009). Transforming higher Tham, C. M., & Werner, J. M. (2005). Designing education through technology-enhanced learning. and evaluating e-learning in higher education: A York, UK: Higher Education Academy. review and recommendations. Journal of Lead- ership & Organizational Studies, 11(2), 15–26. Minović, M., Štavljanin, V., Milovanović, M., doi:10.1177/107179190501100203 & Starčević, D. (2008). Usability issues of e- learning systems: Case-study for Moodle learning Watson, W. R., & Watson, S. L. (2007). An argu- management system. In Meersman, R., Tari, Z., & ment for clarity: What are learning management Herrero, P. (Eds.), OTM 2008 Workshops, Lecture systems. What are they not, and what should they Notes in Computer Science 5333 (pp. 561–570). become? TechTrends, 51(2), 28–34. doi:10.1007/ Heidelberg, Germany: Springer. s11528-007-0023-y Naismith, L., Lonsdale, P., Vavoula, G., & Shar- Weller, M. (2007). Virtual learning environments: ples, M. (2005). Literature review in mobile tech- Using, choosing and developing your VLE. UK: nologies and learning. NESTA Futurelab Series. Routledge. Park, H., Baek, Y. K., & Gibson, D. (2008). De- Zhang, D., Zhao, J., Zhou, L., & Numamaker, J. sign of an adaptive mobile learning management (2004). Can e-learning replace classroom learn- system. User interface design and evaluation for ing? Communications of the ACM, 47(5), 75–78. mobile technology, I, 286-301. doi:10.1145/986213.986216 Pettit, J., & Kukulska-Hulme, A. (2007). Go- ing with the grain: Mobile devices in practice. Australasia Journal of Educational Technology, ADDITIONAL READING 23(1), 17–33. Amin, A. H. M., Mahmud, A. K., Abidin, A. I. Z., Rabin, J., & McCathieNevile, C. (Eds.). (2006). Rahman, M. A., Iskandar, B. S., & Ridzuan, P. D. Mobile Web best practices 1.0. (W3C Proposed (2006). M-learning management tool development Recommendation). Retrieved on August 25, 2010, in campus-wide environment. Issues in Informing from http://www.w3.org/TR/mobile-bp/ Science and Information Technology, 3, 423–434.

175 Mobile Learning Management Systems in Higher Education

Andronico, A., Carbonaro, A., Colazzo, L., & Dalsgaard, C. (2006). Social software: E-learning Molinari, A. (2004). Personalisation services for beyond learning management systems. European learning management systems in mobile settings. Journal of Open Distance and E-Learning. Re- International Journal of Continuing Engineer- trieved August 31, 2010, from http://www.eurodl. ing Education and Lifelong Learning, 14(4-5), org/materials/contrib/2006/Christian_Dalsgaard. 353–369. doi:10.1504/IJCEELL.2004.005726 htm Armatas, C., Holt, D., & Rice, M. (2005, Decem- Davide, T., & Roberto, B. (2007). A platform ber 4-7). Balancing the possibilities for mobile to support anytime, anywhere, just-for-me m- technologies in higher education. In Balance, learning. Paper presented at the Seventh IEEE fidelity, mobility: Maintaining the momentum? International Conference on Advanced Learning Proceedings ASCILITE 2005, Brisbane. Technologies (ICALT 2007), Nigata, Japan. Boticario, J.G., & Santos, O.C. (2007). An open Driscoll, M., & Carliner, S. (2005). Advanced IMS-based user modelling approach for devel- Web-based training strategies: Unlocking instruc- oping adaptive learning management systems. tionally sound online learning. San Francisco, Journal of Interactive Media in Education. CA: Pfeiffer. Chao, I. T. (2008). Moving to Moodle: Reflec- Evans, C. (2008). The effectiveness of m-learning tions two years later, EQ, EDUCAUSE Quarterly, in the form of podcast revision lectures in higher 31(3), 46–52. education. Computers & Education, 50(2), 491– 498. doi:10.1016/j.compedu.2007.09.016 Chen, C.-M., & Hsu, S.-H. (2008). Personalized intelligent mobile learning system for supporting Goh, T., & Kinshuk. (2006). Getting ready for effective english learning. Journal of Educational mobile learning – Adaptation perspective. Jour- Technology & Society, 11(3), 153–180. nal of Educational Multimedia and Hypermedia, 15(2), 175–198. Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning Herrington, A., & Herrington, J. (Eds.). (2006). management systems on university teaching and Authentic learning environments in higher educa- learning. Tertiary Education and Management, tion. Hershey, PA: Information Science Publishing. 11(1), 19–36. doi:10.1080/13583883.2005.996 Järvelä, S., Näykki, P., Laru, J., & Loukkanen, T. 7137 (2007). Structuring and regulating collaborative Corlett, D., Sharples, M., Bull, S., & Chan, T. learning in higher education with wireless net- (2005). Evaluation of a mobile learning organizer works and mobile tools. Journal of Educational for university students. Journal of Computer Technology & Society, 10(4), 71–79. Assisted Learning, 21(3), 162–170. doi:10.1111/ Kadyte, V. (2004). Learning can happen any- j.1365-2729.2005.00124.x where: A mobile system for language learning. In Cow, R., Santos, I. M., LeBaron, J., McFadden, Attewell, J., & Savill-Smith, C. (Eds.), Learning T. A., & Osborne, F. C. (2010). Switching gears: with mobile devices (pp. 73–78). London, UK: Moving from e-learning to m-learning. MERLOT Learning and Skills Development Agency. Journal of Online Learning and Teaching, 6(1), Kim, S. H., Mims, C., & Holmes, K. P. (2006). 268–278. An introduction to current trends and benefits of mobile wireless technology use in higher education. AACE Journal, 14(1), 77–100.

176 Mobile Learning Management Systems in Higher Education

Kinshuk, Kommers, P., & Sampson, D. (2004). Wang, H. (2008). Implement and deploy mobile Adaptivity in Web and mobile learning services learning in open universities - The many promises [Editorial]. International Journal of Continuing and challenges ahead. In C. Bonk et al. (Eds.), Pro- Engineering Education and Lifelong Learning, ceedings of World Conference on E-Learning in 14(4/5), 313–317. Corporate, Government, Healthcare, and Higher Education 2008 (pp. 1340-1343). Chesapeake, Liaw, S. (2008). Investigating students’ perceived VA: AACE. satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard Zurita, G., & Nussbaum, M. (2004). A constructiv- system. Computers & Education, 51, 864–873. ist mobile learning environment supported by a doi:10.1016/j.compedu.2007.09.005 wireless handheld network. Journal of Computer Assisted Learning, 20(4), 235–243. doi:10.1111/ Parsons, D., Ryu, H., & Cranshaw, M. (2007). j.1365-2729.2004.00089.x A design requirements framework for mobile learning environments. Journal of Computers, 2(4), 1–8. doi:10.4304/jcp.2.4.1-8 KEY TERMS AND DEFINITIONS Rekkedal, R., & Dye, A. (2007). Mobile distance learning with PDAs: Development and testing Learning Management System (LMS): of pedagogical and system solutions supporting Also referred to as Course Management Systems mobile distance learners. International Review (CMS) or Virtual Learning Environments (VLE), of Research in Open and Distance Learning, are web-based applications (that is, they run on 8(2), 1–26. a server and they are accessed by using a web Sampson, D., Gotze, K., & Zervas, P. (2007). De- browser) designed to support planning, implemen- livering IMS learning design activities via mobile tation, organization, delivery and administration devices. In Proc. of the 7th IEEE International of web-based courses Conference on Advanced Learning Technologies Mobile Device: Also referred to as handheld (pp. 765-769), IEEE Computer Society. device or handheld computer, it is a pocket-sized computing device with capabilities for internet Shen, R., Wang, M., Gao, W., Novak, D., & Lin, connectivity (either via a wireless network or a T. (2009). Mobile learning in a large blended mobile communication network), typically having computer science classroom: System function, a display screen with touch input and/or a miniature pedagogies, and their impact on learning. IEEE keyboard. Examples of mobile devices include: Transactions on Education, 52(4), 538–546. tablet PCs, personal digital assistants (PDAs), doi:10.1109/TE.2008.930794 smart phones and mobile phones. Vovides, Y., Sanchez-Alonso, S., Mitropoulou, Mobile Devices Supported Education: The V., & Nickmans, G. (2007). The use of elearning support of traditional ways of teaching and learn- course management systems to support learning ing through the use of mobile devices strategies and to improve self regulated learn- Mobile Learning Management System ing. Educational Research Review, 2(1), 64–74. (mLMS): A Learning Management system that doi:10.1016/j.edurev.2007.02.004 can be accessed from students, tutors and administrators via mobile devices.

177 178

Chapter 9 Enhancing Electronic Examinations through Advanced Multiple- Choice Questionnaires

Dimos Triantis Technological Educational Institution of Athens, Greece

Errikos Ventouras Technological Educational Institution of Athens, Greece

ABSTRACT The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems (LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented.

INTRODUCTION staff and learning material (Chu, Young, Ngai, Cun, Pearl, & Macario, 2010; Ellis, 2009a). This Learning Management Systems (LMS) ideally “all encompassing” characteristic provides an support the whole cycle of activities related to extensive set of features that are available to the the interactions between instructors and students, users of a LMS. Educational institutions and cor- as well as their interactions with administrative porations can use features such as certification of

DOI: 10.4018/978-1-60960-884-2.ch009

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

participants, compliance tracking, reporting and assessment and testing tools used by LMS. In statistical processing of educational activities, the light of the above, research concerning in- management approval, authoring of material and novative assessment techniques, incorporated content management, as well as assessment ans into electronic examination modules, might be testing. In the present chapter focus will be placed an efficient way to further the adoption of LMS. on the assessment and testing features of a LMS Electronic examinations, as part of computer- and, more specifically, on variants of electronic aided testing systems, enhance student assessment examination methods. An electronic examination procedures in a variety of ways. For example, is an assessment item delivered by the LMS, while the time allocated to scoring the answers of the the development, management and publishing of students and the administrative burden related to the items that the electronic examination consists the registration of the examination grades might of are usually tasks of a Learning Content Man- be drastically reduced or even eliminated. Self- agement System (LCMS), a software application examination material can be provided, which can that is either incorporated into a LMS or closely be used on a distance-based learning framework interacts with it (Feldstein, 2004). The use of by the students (Tsiakas, Stergiopoulos, Nafpak- electronic examinations methods, in the context titis, Triantis, & Stavrakas, 2007). The automated of LMS, as computer-based learning and evalua- extraction of statistical indicators of the students’ tion items, created in the framework of a LCMS performance can also be straightforwardly incor- and managed by a LMS, should be considered for porated to the software implementing and manag- enriching and strengthening the existing assessing the electronic examination (Mattheos et al., ment schemes, while at the same time construct- 2008; Van der Linden & Glas, 2000). One of the ing innovative assessment techniques (Ganguli basic features of a LMS is that it enables to keep et al., 2006; Triantis, Anastasiadis, Tsiakas, & track of the performance of students across the Stergiopoulos, 2007). various types of training activities that the students As stated above, LMS are not directly involved are engaged in. The existence and management in producing the contents provided to the users of quantitative indicators, in the framework of a of the systems. Nevertheless, the whole “super- LMS, might enable the detection by the teacher structure” that a LMS provides to an educational of topics whose assimilation is not satisfactory organization that seeks to benefit from an auto- by the students. In turn, the knowledge that spe- mated system for managing educational processes, cific material has been deficiently learned by the is based on a set of foundations, among which examinees might focus remedial action in subse- are the assessment and testing tools incorporated quent teaching periods of the examined courses into the LMS. The American Society for Train- (Buchanan, 2002; Bull, Conole, Davis, White, & ing and Development (ASTD) conducted two Sclater, 2002). Such actions might be integrated in surveys, in 2009 (189 participants) and 2010 (342 a LMS, by modifying the study material provided participants), on users of LMS. When asked to to the students, in both printed and electronic form, rank the most valuable features of LMS, in both using a LCMS, and by consecutively keeping track years, among 15 features, assessment and testing of the modifications to the students performance ranked 1st with 59.3% of participants ranking it indicators induced by the changes that have been most valuable feature in 2009 and 54.5% in 2010 introduced. Despite of those benefits, the extent (Ellis, 2009b; Ellis, 2010). In 2009 the feature that of the usefulness of electronic examinations is ranked 2nd was content management, with 48.4%, still under investigation, since a number of mat- and in 2010 reporting, with 51.5%. Electronic ters have to be taken into consideration for each examination modules are an important part of specific application, such as the familiarization of

179 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

the students with information technology (IT) and evident for a student who does not possess adequate the burden imposed on the teacher for producing knowledge on the topic addressed in the questions both the examination forms (questionnaires) and of the pair. The questions’ answers are graded in the related electronic study material. Additionally, pairs, providing a bonus (penalty) if both (only the possible overcomplexity of the educational one) answers of the pair are correctly answered. software that both examinees and teachers should The objective of the research presented in this use must be taken into account and reduced as chapter is to advance the investigation comparing much as possible, so as not to hamper the famil- the paired MCQs grading scheme to the traditional iarization of users with the electronic systems constructed response (CR) test, taken as a stan- (Buchanan, 2002; DeBord, Aruguete, & Muhlig, dard, as well as their statistical relationship to the 2004; Mattheos et al., 2008; Thomas, 2003). To most simple grading scheme of MCQs, i.e., the overcome these problems, the LCMS must be positive scoring rule, according to which students easy to use for the content developer and built in gain points for correct answers and have no losses a way to produce user-friendly interfaces for the for wrong answers and omissions. The variation various electronic examination tools. of the parameters controlling the bonus/penalty Multiple-Choice Questions (MCQs) tests are mechanism of the scoring rule is investigated. In inherently suited to and extensively used in elec- this way, indications could be provided that the tronic examination systems, due to the facility to examination method based on paired MCQs might extract the final score of the examinee in a fully constitute a reliable tool for the evaluation of the automatic mode (Mattheos et al., 2008; Van der performance of the students, enhancing electronic Linden & Glas, 2000). Nevertheless, drawbacks examination use. of the MCQs examination method, mainly related to the influence of guessing in choosing an answer, hamper the extension of their use (Scharf BACKGROUND & Baldwin, 2007). In order not to be constrained by the problems related to the most commonly Electronic examinations can significantly augment used MCQs grading schemes, examinations based the speed of testing large number of students, on the more traditional method of constructed if the scoring of the test can be done automati- response questions (CRQs), where, for example, cally through an electronic examination platform, questions are answered by developing a set of without need of judgment application by a human subjects in the form of short essays, are still scorer. Tests being comprised of statements that largely used and often preferred to MCQs. The have to be classified as true or false and MCQs preference to CRQs, which are not well suited for tests are such examination methods (Bush, 2006; electronic examinations, since the grading process Freeman & Lewis, 1998; Scharf & Baldwin, 2007). cannot be usually automated, relies on the fact MCQs usually belong to the “single-best answer” that in CRQs the examiner has the possibility to scheme, that requires the examinee to indicate the check more fully the knowledge of the student most appropriate single response, from a set of re- and especially the way that he/she developed the sponses/answers that are proposed to the examinee, subject under question. In order to alleviate those per question (McCoubrie & McKnight, 2008). A problems of MCQs tests, a specific kind of MCQs much less frequently used variation consists in examination method has been proposed, which using the “elimination procedure” proposed by uses sets of pairs of MCQs, referred as “paired” Coombs et al. (1956), in which examinees are MCQs method (Ventouras, Triantis, Tsiakas, & asked to eliminate as many incorrect answers as Stergiopoulos, 2010). MCQs in each pair concern they can. the same topic, but this similarity should not be

180 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

The benefits of MCQs tests have been ex- multiple-choice (MC) item. It has been shown tensively analyzed (Bush, 2006; McCoubrie & that, in addition to the obvious ability of MCQs McKnight, 2008; Race, 2005; Scharf & Baldwin, to test simple memory skills, such as power of 2007). Apart from the obvious time-reduction information retention and retrieval, MCQs tests benefits for the examiner, in testing large number can investigate complex abilities and understand- of students, and the fact that the examinee’s score ing (Scharf & Baldwin, 2007; Manning & Dix, is not dependent on his/her writing speed skill, 2008; Curzon, 1997). a broad range of student knowledge acquisition In the framework of research concerning for the field being examined might be tested in a parameters used as markers of the reliability and short period of time, provided that a sufficiently validity of examination methods (Anastakis, Co- large question bank is correctly used in construct- hen, & Reznick, 1991; Bennett, Rock, & Wang, ing the MCQs test. The question bank should be 1991; Bridgeman, 1991; Evans, Ingersoll, & constructed in a way that all the teaching units/ Smith, 1966; McLean, Dauphinee, & Rothman, chapters/modules comprised in the course that 1988; Wainer, Wang, & Thissen, 1994; Wass, will be tested are covered by a number of MCQs, Wakeford, Neighbour, & Van der Vleuten, 2003), that MCQs of varying difficulty are created per there exist substantial indications that MC scores teaching unit and that the number of questions in provide higher reliability and are as valid as scores the data bank is large enough so that repetition of extracted from examinations based on the CRQs questions in consecutive tests will be avoided as method (Wainer & Thissen, 1993; Lukhele, This- much as possible. Furthermore, ensuring correct- sen, & Wainer, 1994). It could have been expected ness of the use of the data bank implies that for a that those indications would help in promoting the specific examination, questions will be selected use of MCQs tests in most educational settings from the data bank in a way to cover all teaching where CRQs are still used, especially taking into units, in a way proportionate to the relative sig- account the drawbacks of the CRQs examination nificance of each unit in the course, with questions method concerning the fact that subjects that might of varying difficulty existing in the test for each be examined cannot cover a significant amount of unit. In electronic examinations using MCQs, the material taught during the courses and, more there is the possibility to provide as feedback the importantly in an electronic examination setting, correct answer to the student, immediately after the inherent inability of introducing automated he/she answered each MCQ item. The provision grading in essay-like responses to questions. of immediate on-screen feedback (Scharf & Bald- Nevertheless, CRQs tests are still widely used, and win, 2007; Race, 2005) might enable examinees this might not only be due to their main advantage, to reflect on the procedure they used to reach a namely that they provide to the examiner the pos- wrong answer and therefore avoid consecutive sibility to check the path taken by the student to missteps, provided there is adequate time to re- develop the subject under question and reach the flect on the questions. In the more common case requested answers, but also due to the variety of where the examinee is required to submit his/her drawbacks that plague MCQs. answers to all the questions of the test, without One category of drawbacks of MCQs tests intermediate feedback, the automated nature of the concerns the requirements imposed on the per- marking makes possible the provision of prompt son who will construct the question bank for feedback to the examinee, after the termination the MCQs test. The MCQs examination method of the examination, concerning both the overall relies heavily on the quality of the question bank. test grade and information for correct and incor- As mentioned above, this concerns the need of a rect answers that were given by the examinee per large question bank, so as to cover all the teaching

181 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

units/chapters/modules of the course/material to the examination. More generally, the behavior of be examined, in a way proportionate to the extent examinees in MCQs tests using mixed-scoring of each unit and with the existence of questions rules might be influenced by factors that affect of varying difficulty per unit. The time burden decision making under uncertainty. In turn, this to produce such an extended bank might often might produce variance in the test scores that is result in poorly written or overlapping questions related to the expectations of the examinees and (Bush, 2006). If higher-order thinking skills are not to the knowledge that is tested (Bereby-Meyer, to be consistently checked, the time requirements Meyer, & Flascher, 2002; Bereby-Meyer, Meyer, for constructing the MC items might increase the & Budescu, 2003). Other scoring rules that have effort and expertise needed. For example, in order been used include the order-of-preference scheme to test the aptitude to solve complex problems (Bush, 1999; Wood, 1991). According to this, the and exercises, it has been proposed to fragmen- MCQs used belong to the “single-best-answer” tize the initial problem into questions, so that the scheme, i.e., there is only one correct answer per reasoning path that a student would take could question and examinees have to assign an order of be checked (Stergiopoulos, Tsiakas, Triantis, & preference to every answer for each question. The Kaitsa, 2006a). Obviously, it is a challenge to score per MC item is dictated by the preference construct a variety of such “linked” questions, they assign to the correct answer. Another MCQs per teaching unit. examination variant requires the examinee to as- The other category of drawbacks of MCQs sociate a level of confidence with each selected tests concerns issues related to the scoring answer, so that their degree of certainty in answer- rules that are used for grading the MC items of ing is monitored (Pressley, Ghatala, Woloshyn, the test. The simplest grading scheme uses the & Pirie, 1990; McKenzie, Wixted, Noelle, & “positive-grades-only” scoring rule, according to Gyurjyan, 2001). In this kind of scoring method a which students collect positive points for correct wrong answer that is associated with a high degree answers, while no losses are incurred for wrong of certainty is penalized much more than if the answers or omissions. The influence of guessing wrong answer was related to lack of confidence. in choosing an answer results in collecting some Still another variant are the so-called “liberal” tests partial scores in the final score, by answering (Bush, 2006), which enable the choice of more questions by chance, without possessing knowl- than one answer per question, if the examinee is edge of the questioned material. For remedying uncertain which is the correct answer. It might be this problem a variety of marking schemes for assumed that the examiner might be able to test written MC tests have been introduced. In this the level of knowledge of the examinee since the perspective, “mixed-scoring” rules are used, i.e., examinee has the possibility to select between a rules in which students gain points for correct lot of alternative responses (Bush, 1999). Such answers and lose points for incorrect answers and, alternative MCQs examination variants present to in a stricter variant of the scoring rule, also for the person who will construct the tests the problem omissions (Scharf & Baldwin, 2007). These rules of formulating a suitable grading scheme, since the penalize guessing and seem to drastically reduce variability in selections that an examinee might the gaining of scores through guesswork. On the choose when answering a question is much greater other hand, the use of such rules have been shown than in simple “single-best-answer” schemes. Fur- to dissuade examinees from answering a question thermore, the use of these testing methods might for which they might possess an intermediate or distract the examinee from focusing on answering low level of knowledge, concerning especially correctly the questions, when his/her confidence those examinees who anticipated a high grade in to the answer given is requested, or when having

182 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

to decide to select alternative answering tactics and 46.24 (S.D.=20.37), respectively. Another in “liberal” tests (Bush, 2006). interesting finding of that study was that, when the In a previous work, an alternative MCQs ex- CRQs method was compared to the PSR-MCQs amination method has been presented (Ventouras method, in an another examination where no paired et al., 2010). According to this method, the MC MCQs were used, the statistically significant dif- items given in the test constituted pairs, referred ference between the scores of the CRQs and the as “paired” MCQs. MCQs in each pair concerned PSR-MCQs method was greatly reduced when the same topic, but this similarity was not evident the set of students comprised only those students for a student who did not possess adequate knowl- who got a score greater than 70. Specifically, the edge on the topic addressed in the questions of the difference of the mean scores of the two methods pair. The questions’ answers were graded in pairs, was reduced from 12.42 (p=0.001) for the whole providing a bonus (penalty) if both (only one) an- set of students to 5.37 (p=0.047) for the reduced swers of the pair were correctly answered. If both students set. It seems that the higher-grade effect answers were wrongly answered no marking was of the PSR-MCQs method was reduced for that collected by the examinee. The use of this scor- set of students. A possible reason for that might ing rule aimed at penalizing guesswork, without be that students who got a higher score than oth- inducing the “hampering” effects produced by ers, were, on average, better prepared and quali- the negative marking part of the commonly used fied for the examination, and therefore did not mixed-scoring schemes. In that study a paired apply, or greatly reduced, guessing in selecting MCQs test was given to 63 examinees, in the the answers to the MCQs. Consequently, there framework of a computer-based learning system. are indications that the methodologies used for The same examinees took also a CRQs examina- alleviating the positive grade bias induced by tion. Students were examined in a PC laboratory the positive-grades-only scoring rule, are more room, using an electronic examination platform important for reducing the amount of students, (details of the platform are given in the following who by guesswork could take a “pass” note (in “Methodology of the Examination” section). The the case of that study 50), although their level results of the paired MCQs examination, when of knowledge acquisition, recall and/or reason- using the pair-wise scoring rule, were statistically ing capabilities was not sufficient for taking the indistinguishable with the grades produced by examination with success. the CRQs examination method, when made to the same sample of students, οn the same topics and with the same levels of difficult. Both the COMPARISON OF CONSTRUCTED- results of the paired MCQs examination, when RESPONSES QUESTIONS using the pair-wise scoring rule, and the CRQs AND PAIRED MULTIPLE- examination results differed significantly from CHOICE QUESTIONS those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored Methodology of the Examination the pairing of MCQs (referred to as the PSR- MCQs examination method). Specifically, with In order to expand the investigation of the com- score range (0-100), the mean of the scores of parison of the paired MCQs examination method the examinees for the PSR-MCQs method was to the CRQs examination method (taken as a gold 58.25 (Standard Deviation, S.D.=18.65), while standard) and the PSR-MCQs method, mainly con- the mean for the paired MCQs and the CRQs cerning the effects of the parameters controlling examination methods were 45.95 (S.D.=20.28) the bonus/penalty mechanism of the scoring rule

183 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

of the paired MCQs method, in the examination of that application is incorporated in a wider period of February 2010, a course was selected, in effort to further disseminate the use of PCs in order to compare the three examination methods. the educational process through the creation of The examination of the course was implemented an institution-wide LMS (Triantis et al., 2007; according to the following procedure. Tsiakas et al., 2007; Stergiopoulos et al., 2006a; Stergiopoulos et al., 2006b). The LMS uses the The Examined Course and eClass platform, developed and provided by the the Sample of Students Greek Academic Network (GUnet Asynchronous eLearning Group, 2010), based on the Claroline The course that was selected for comparing the open-source Course Management System (Claro- scores of the three examination methods was an line, 2010). “e-examination” is mainly a managing introductory electronics course entitled “Elec- and editing tool that enables the teacher to create tronic Physics”, which belongs to the group of core electronic assessment tests that can be used either background courses and is taught at the Electron- for examining students or for self-assessment by ics Department of the School of Technological the students. The electronic tests can be displayed Applications of the Technological Educational by a web browser, ensuring their portability across Institution (T.E.I.) of Athens. The course contains PC platforms. The user interface consists of a four teaching units. 30 students participated in a series of questions, which can be either of the CRQs examination and a MCQs examination. MC or the CR type (Stergiopoulos et al., 2006a; Their answers to the MCQs examination were Triantis, Stavrakas, Tsiakas, Stergiopoulos, & scored according to the two different scoring vari- Ninos, 2004). ants mentioned in the Background section, i.e., For the CRQs examination, a set of three test the pair-wise scoring rule and the positive-grades- subjects was created. Two subjects contained only scoring rule. All students had previously 3 CRQs and one subject contained 2 CRQs. been given information about the examination Therefore, the examinee had to answer 8 CRQs. procedure, both in its CRQs and MCQs form and The CRQs were short essay subject development were familiarized with the electronic examination questions, of either theoretical or exercise form. platform used in the MCQs examination, so that The distribution of CRQs was designed so as to they would not be distracted from the presenta- cover each teaching unit proportionally. For the tion format and the requirements for the use of MCQs examination, a database was constructed the electronic system. Additionally, they had containing a large number (N=400) of MCQs, available related study material, in both printed covering the whole range of the subject material and electronic polymorphic form (Stergiopoulos of the course. By using “e-examination”, a first set et al., 2006a). of MCQs {qa1, qa2, …, qak} (k=40) was randomly selected from the database, taking care to cover

Formulation of the Questions each teaching unit proportionally. A weight wai and Examination Procedure was assigned to each question i=1,…,k, depending on its level of difficulty. With those questions as

The CRQs examination was taken by the students reference, a second set of new MCQs {qb1, qb2, …, in its traditional paper-and-pencil form. The qbk} (k=40) was composed, with each question qbi

MCQs examination took place in a PC laboratory possessing a similarity to question qai (i=1,…,k), room, through the “e-examination” application constituting a pair of MCQs, according to the developed at the Technological Educational In- following rationale: a) both questions referred stitution of Athens (T.E.I.-A.). The development to the same topic and b) the knowledge of the

184 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

correct answer for question qai, from a student Scoring Methodology who had proceeded to a systematic study and was cognizant of the topic, implied the knowledge For the MCQs, 2 scores m1 and m2 were computed. of the correct answer for qbi and vice versa. The Score m1 was computed as follows: for each distribution of the weights was as follows: 31 MCQs pair i=1,…,40, the “paired” partial score questions had weight 1, 31 questions had weight pi was computed as 1.5, 16 questions had weight 2 and 2 questions had weight 3. For most pairs, the questions of the pi=( q aiw ai + q biw bi )()1 + kbonus (1.a), pair had the same weight, i.e., it was wai=wbi. In some cases though, the weight of the two paired if both q and q were correct, in which case questions differed slightly (i.e., by 0.5). This was ai bi q =q =1, or done because, for some topics, it was difficult to ai bi create a pair of questions that both respected the p=( q w + q w)()1 − k (1.b), two requirements (a) and (b) stated above and i ai ai bi bi penalty were also absolutely equal in their level of difficulty. The presentation of the 2k=80 questions if either qai or qbi was correct, in which case it that the students had to answer in the PC screen was either qai=1 and qbi=0, or qai=0 and qbi=1, or was designed so that the questions were given with a random sequence, taking care that each pi = 0 (1.c), question qbi was presented after a lapse of at least 10 questions after the presentation of question q . ai if both q and q were incorrect, in which case Questions were automatically given through the ai bi q =q =0. “e-examination” software system. ai bi Parameters k and k were variables that In the first phase of the examination, students bonus penalty controlled the bonus and penalty mechanism, were given the CRQs. In the second phase of the respectively, of the scoring rule. It should be examination, the MCQs were given to students, noted that in the study of Ventouras et al. (2010) in the PC laboratory room. After the end of the k and k had fixed values of 0.25 and 0.5 pre-determined MCQs examination duration time, bonus penalty respectively and it was w =w for every i. a results’ page was produced for each student. ai bi The total score m1, with maximum value equal The results’ page included all questions with the to 10, was then computed as: correct answer and the indication of whether it was correctly or wrongly answered, as well as the 40 final score. One copy was given to the student and ∑ pi i=1 one to the examiner, for processing the scores. m1 = 40 ⋅10 (2) Care was taken so that the MCQs were, overall, ∑()1 +kbonus() w ai + w bi of equivalent level of difficulty with the questions i=1 posed to the examinees in the CRQs examination. This enabled the comparison of the scores that Therefore, to produce score m1, a bonus is would be achieved after the students would have given to the student if he/she answered correctly given their answers in the two forms of examina- both questions of the MCQs pair (qai, qbi) and a tions, i.e., the one using MCQs and the CRQs. penalty if he/she answered correctly only one question of the pair. Therefore, through this scoring algorithm, the final score extracted corresponds to the paired MCQs examination method.

185 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Score m2, with maximum value equal to 10, Statistical Evaluation Rationale was computed according to The comparison of the paired MCQs examination 40 method to the CRQs examination method and the (q w + q w ) ∑ ai ai bi bi PSR-MCQs examination method has as an objec- m2 = i=1 ⋅10 (3) 40 tive to provide indications for accepting the paired ()w+ w ∑ ai bi MCQs examination method as an alternative for i=1 CRQs examinations, according to the rationale stated in the Background section. Such an indi- where cation might be provided, if the scores obtained through the paired MCQs, using the pair-wise scor- 1 if the question was correctly answered qai = [ ing method, are statistically indistinguishable from 0 if the questiion was wronlgy answered the scores obtained from the CRQs examination (4) method. Therefore, the hypothesis H0 to be tested in comparing the paired MCQs, the PSR-MCQs and and the CRQs examination methods could be 1 if the question was correctly answered stated as: “The mean of the distribution of scores q = [ bi 0 if the questiion was wronlgy answered m1, obtained using the paired MCQs examination (5) method, the mean of the distribution of scores m2, obtained using the PSR-MCQs examination This method of scoring ignores any similarities method, and the mean of the distribution of scores existing between the questions of the pair and does m3, obtained using the CRQs examination method, not impose a penalty to the student by negative are equal”. If hypothesis H0 is rejected, i.e., the marking for incorrect answers. Therefore, through overall differences between the three means are this scoring algorithm, the final score extracted significant, then post-hoc pair wise comparisons, corresponds to the PSR-MCQs examination with adjustment for multiple comparisons, should method. be used, in order to check the three “secondary” Finally, for the CRQs examination category, hypotheses, namely H0(paired MCQs to PSR-MCQs) (i.e., “The each of the eight questions j=1,…,8, was graded, mean of the distribution of scores m1, obtained using the paired MCQs examination method is with minimum question grade gj being 0 and maximum grade varying from 0.4 to 2.5, according equal to the mean of the distribution of scores to the importance of the respective question. The m2, obtained using the PSR-MCQs examination sum of the maximum grades was 10. The overall method”), H0(paired MCQs to CRQs) (i.e. “The mean of CRQs examination score, m3, was extracted as the distribution of scores m1, obtained using the the sum of the question grades: paired MCQs examination method is equal to the mean of the distribution of scores m3, obtained 8 using the CRQs examination method”), and m3 = g j (6) ∑ H0(PSR-MCQs to CRQs) (i.e., “The mean of the distribu- j=1 tion of scores m2, obtained using the PSR-MCQs examination method is equal to the mean of the In all examination categories 5.0/10.0 was distribution of scores m3 obtained using the CRQs the minimum score required for passing the examination method”). examination.

186 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Scoring Results and comparisons between each of the examination Statistical Evaluation methods, assessed with post-hoc Bonferroni pair wise comparisons at 0.01 level of significance. In Table 1 the descriptive statistical values re- Significant differences existed between m1 and lated to the three examination methods used in m2 (p<0.001) and between m2 and m3 (p<0.001). the examination are presented, i.e., the mean Therefore the secondary hypotheses values (m1,, m 2 m3 ) and standard deviation H0(paired MCQs to PSR-MCQs) and H0(PSR-MCQs to CRQs) had to (S.D.) of the scores m1, m2 and m3 taken from be rejected at p=0.001 level of significance. If the the three methods for the whole set of students level of significance of the post-hoc test is set to examined, and for two sub-sets of students, per 0.05, then the difference between m1 and m3 examination method: for those students who reaches significance, albeit at a barely adequate passed the examination (score >= 5.0) and for level of p=0.043. those who failed the examination. Furthermore, These results indicate that the paired MCQs the number and the percentage of students belong- examination method with bonus/penalty adjusting to each sub-set is given. A value of ment (resulting in score m1) is statistically equivalent to the CRQs examination method kbonus=kpenalty=kcommon=1/3 was chosen. The Kol- mogorov-Smirnov goodness-of-fit test showed (resulting in score m3), if a strict criterion (p<0.01) that the distributions of the scores m1, m2 and is selected for rejecting the secondary null hy- m3 were each consistent with a normal distribu- pothesis H0(paired MCQs to CRQs). Both methods differ tion. The same held for the two sub-sets of students. significantly from the MCQs, which do not use For the whole set of students who took the a negative marking “penalty” procedure, i.e., the examination, repeated-measures ANOVA with PSR-MCQs examination method (resulting in one within-subjects factor (method of examina- score m2). Even in the case where a lax criterion tion, three levels) indicated that the within-subject of p<0.05 was chosen, so as to reject the second- effect was significant (F1.266,36.725=273.975, ary null hypothesis H0(paired MCQs to CRQs), the difference p<0.001, degrees of freedom were corrected for between m1 and m2 and between m2 and m3, as non-sphericity according to the Greenhouse- quantified by the absolute differences of the means |m1− m 2 | Geisser procedure). Therefore, the hypothesis H0 of their distributions (i.e., =0.92 and to be tested, that the overall differences between |m2− m 3 |=0.86, respectively, see Table 1), is the means of the scores m1, m2 and m3 are not much higher than the difference between m1 and significant, has to be rejected at p=0.001 level of m3 (i.e., |m1− m 3 |=0.06). Therefore, present significance. ANOVA was followed by planned results provide support fort the following infer-

Table 1. Descriptive statistics of the three examination methods

Set of students m1 (mean, S.D.) m2 (mean, S.D.) m3 (mean, S.D.) Number (percentage) of students per examination method Paired PSR- CRQs MCQs MCQs Whole set (n=30) 5.09 (1.75) 6.01 (1.59) 5.15 (1.77) Set of students who passed success- 6.65 (1.09) 6.69 (1.24) 6.5 (1.20) 14 (47%) 22 (73%) 16 (53%) fully the examination (score ≥ 5.0) Set of students who failed the 3.72 (0.77) 4.15 (0.59) 3.60 (0.71) 16 (53%) 8 (27%) 14 (47%) examination

187 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

ences: (i) that the evaluation of the students with ing the regression line of m2 to m3 (Figure 2) PSR-MCQs gives greater success rates and scores with the regression line of m1 to m3 (Figure 3). than both the CRQs examination method and the The regression of m1 to m3 was chosen to be paired MCQs examination method and (ii) that expressed by an equation of the form y=ax and the bonus/penalty marking scheme of the paired the resulting parameters that were computed, i.e., MCQs makes the MCQs examination method a=0.9992 and R2=0.9945, comforted inference scoring results much similar to that obtained (ii) stated above. The alleviation of the positive through the CRQs examination method. This bias grade bias seems to be achieved through the bonus/ of the PSR-MCQs method to higher scores can penalty pair-wise scoring rule of the paired MCQs also be deduced from the inspection of Figure 1 examination method. It should be stressed that and the regression line of m2 to m3 (Figure 2). this improvement is gained while avoiding ex- One of the most probable explanations that could plicit negative marking for incorrect answers, so be put forward as reason for this characteristic of that examinees might not be dissuaded from an- the PSR-MCQs method is the absence of any swering questions for which they might possess mechanism to correct the influence of guesswork an intermediate level of knowledge. on selecting the correct answers. This is in agree- The above results, concerning the comparison ment with the results presented in Ventouras et of the CRQs and the two MCQs examination al. (2010). The introduction of bias by the PSR- methods, stood also when the comparison was MCQs method, due to the lack of correcting for restricted to the two sub-set of students. In the the “guesswork effect”, seems to be removed by statistical evaluations, in order to keep the valid- the paired MCQs examination method with bonus/ ity of the repeated-measures ANOVA methodol- penalty adjustment, as indicated also by compar- ogy, the groups were composed with those students

Figure 1. Number of students graded in the respective score ranges, per examination method. Dotted bars: Paired MCQs examination method. Black bars: PSR-MCQs examination method. Gray bars: CRQs examination method.

188 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Figure 2. Regression line of score of PSR-MCQs (m2) to score of CRQs (m3)

who failed or passed at the CRQs examination ferences existed between m1 and m2 and between method. For both groups of students, repeated- m2 and m3. measures ANOVA with one within-subjects fac- In the present study the effects of varying tor (method of examination, three levels) indi- parameter kcommon were investigated, concerning cated that the within-subject effect was significant, the “fit” of the scores of examinees in the paired therefore the null hypothesis H0 that was tested MCQs and the CRQs examination methods. This in comparing the paired MCQs, the PSR-MCQs fit can be quantified using parameters a and R2, and the CRQs examination methods had to be which are related to the regression line of m1 to rejected, and for both students’ groups, at a high m3. Their best possible value is 1. As can be seen degree of significance (p<0.001). Again, with the from Figure 4, R2 remained at very high levels (> level of significance set to 0.01, significant dif- 0.99) for 0.27≤ kcommon ≤0.47. As can also be

Figure 3. Regression line of score of paired MCQs (m1) to score of CRQs (m3)

189 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

2 seen from Figure 4, the function of R to kcommon tuations, if the value of the parameter controlling is smoothly varying, having a global maximum the bonus/penalty process is varied. 2 R =0.9950 at kcommon=0.37. The same characteristics of a smooth evolution are presented by parameter a, which, as can be seen in Figure 5, FUTURE RESEARCH DIRECTIONS for the range of kcommon values that were investigated (0.23 to 0.47) varied from 1.0337 to 0.9587, Based on the results presented in this chapter, having the values nearest to 1 between kcommon=0.32 future research should firstly include the thorough and kcommon=0.33. Another metric whose relation investigation of the effects of using the “bonus to the variation of kcommon was investigated was giving” and penalizing parameters kbonus and kpenalty, the mean of the square root of the sum of squared respectively, which control the scoring procedure differences (RMS value) of the scores that the for the paired MCQs examination method. In students took in the CRQs and the paired MCQs the work of Ventouras et al. (2010) two differ- 30 1 2 ent values for k and k were used, with ε = m3− m 1 bonus penalty examinations: ∑()j j . Met- k =0.5k , while at the novel results presented 30 j=1 bonus penalty in this chapter a common value k for both ric ε also describes, in an overall way, how well common k and k was used. Although in both cases the grades provided by the paired MCQs exami- bonus penalty the introduction of the paired MCQs procedure nation method come close to the grades of the “attracted” the MCQs examination scoring much CRQs method. This “closeness” is a goal of the closer to the CRQs “gold standard”, with a clear introduction of the paired MCQs method. The differentiation to the positive bias producing best value for ε would be 0. As can be seen from PSR-MCQs method, when a k was used, Figure 6, the function of ε to k is smoothly common common there remained a level of statistically significant varying having a minimum at k =0.34. The common difference (p=0.043) between the paired MCQs smoothness of the variation of the three param- and the CRQs examination methods, which did eters, in relation to k , provides an indication common not exist in the results of Ventouras et al. (2010). that the proposed bonus/penalty grading scheme Therefore, it would be interesting, for refining of the paired MCQs does not present abrupt fluc- the scoring rule, to investigate the effects of us-

2 Figure 4. Variation of R (of the regression of m1 to m3) as related to k(=kcommon)

190 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Figure 5. Variation of a (of the regression of m1 to m3) as related to k(=kcommon)

ing different values for kbonus and kpenalty, by testing of the above investigations to a greater number schemes where the ratio of kbonus to kpenalty could be of students and more than one course would add either augmented (reaching 1 as was the case for value to the results presented. the novel results of the present chapter, or even Another area of future research could be the surpassing it) or reduced. The use of ratios kbonus/ use of the paired MCQs examination method in kpenalty<1 is expected to make the effects of the conjunction with the MCQs examination vari- penalizing procedure more pronounced. It would ant, presented in the Background section, which be interesting to see whether this strengthening requires the examinees to associate a level of “protects” the scoring against the influence of confidence with each selected answer so that their guessing better than the equalization of kbonus and degree of certainty in answering is monitored. kpenalty. It should also be stressed that the application As stated above, in this kind of scoring method

Figure 6. Variation of ε (mean of the square root of the sum of squared differences of m1 and m3) as related to k(=kcommon)

191 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

a wrong answer that was associated with a high the IMS QTI compatibility and capabilities of degree of certainty is penalized more severely than the Claroline platfrom, upon which the LMS if the wrong answer was related to low levels of was built, are not utilized. Nevertheless, due confidence. If such a scheme would be used in to the characteristics of the data model of the conjunction with the paired MCQs bonus/penalty specification, basic features of the e-examination procedure for the paired questions, a second pe- application inherently fit classes of the IMS QTI nalizing factor would be introduced and it would specification. For example, each MCQ item and be interesting to explore the influence of the two the choice of response to each item fit the “single penalizing parameters to the alleviation of the assessmentItem” and the “choiceInteraction” positive grade bias introduced by the PSR-MCQs classes of the specification. On the other hand, method. The additional resources that examinees the extent of compatibility of the novel features need to devote to making the confidence judgments of the proposed MCQs tests, i.e., the selection raise concerns over possible negative effects to of “paired” questions from an item bank and the the performance of the examinees, but there exist grading of the responses, to the model described indications that the addition of such questions in by IMS QTI specification is a matter that needs the examinations do not penalize performance investigation. Among other topics, future work (Carvalho, 2009). should clarify whether the pair of questions will An important topic related to electronic be combined into a “composite” item, although examination software, is its ability to create ap- this should not be made evident to the examinee, plications (i.e., electronic examination platforms) and whether the “generalized response processing that can both be easily accommodated/combined model” of the specification has to be extended to with already existing tools and provide for future include the paired grading scheme, pertaining to developments that will follow common structured the “responseRule” abstract class of the model formats. With this aim in scope, in the last decade, and its derived and associate classes (IMS Global the IMS Question & Test Interoperability (QTI) Learning Consortium Inc., 2006b). The proposed specification has been developed by the IMS electronic examination scheme will surely benefit Global Learning Consortium. if it is re-developed in a form compatible to the IMS QTI specification. Specifically, the question IMS QTI describes a data model for the rep- bank and grading procedures, upon which the resentation of question and test data and their electronic examination is based, will be smoothly corresponding results reports. Therefore, the handled and incorporated into LMS supporting the specification enables the exchange of this item, specification and this will consequently enable test, and results data between authoring tools, the wide dissemination of its use in LMS. item banks, test constructional tools, learning systems, and assessment delivery systems. (IMS Global Learning Consortium Inc., 2006a). CONCLUSION

The software used in the work described in The use of electronic examinations in the frame- the present chapter has been developed indepen- work of LMS provides a valuable tool for educa- dently of the IMS QTI specification. Therefore, tors, since they enable the automated quantification the developed MCQs are currently “locked” to and processing of students’ assessment. Grading the specific e-examination application developed and test results tracking, accompanied with per- at T.E.I.-A. This application provides “examina- formance indicators, can be smoothly merged in tion packages” to the institution-wide LMS, but the learning management and instructors-students

192 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

interactions and procedures that the LMS supports. should be no overlap between questions and that MCQs are inherently suited for electronic exami- the difference between choices to select must nations. Therefore, it can be conjectured that the be clear to a sufficiently prepared student. In adoption of LMS will be enhanced, if electronic the case of the proposed method, these require- examinations based on MCQs could overcome ments are stressed a little further, since, for each problems related to their use in classroom practice. topic that needs to be examined, a pair of ques- Simple positive scoring rules (PSR) of the tions of equivalent level of difficulty should be MCQs introduce bias in the scores gained by constructed, in such a way that the fact that each the examinees, while mixed-scoring rules might MCQ of the pair concerns the same topic should introduce “hampering” effects to examinees, who not be evident to a student, who does not possess are not sufficiently confident of their knowledge adequate knowledge on the topic addressed in the on certain topics. The paired MCQs scoring questions of the pair. By judiciously planning and methodology has been proposed as an alternative allocating the initial human capital resources that scoring rule. The relation of the paired MCQs would be needed for the creation of such paired and PSR-MCQs test performance scores to the question banks, it is conjectured that electronic scores of the examinees using the CRQs method, MCQs examinations, used as “building blocks” in the results presented above, as well as in pre- of combined LCMSs and LMS, will improve the vious research, provide indications about the proper quantification of the level of knowledge superiority of the scoring process adopted in the and competences of the examinees. Taking into paired MCQs, as compared to the PSR-MCQs consideration that automated assessment and examination method. The grading process of testing, as well as handling of grade information the paired MCQs method does not introduce a in the form of performance metrics, is one of positive grade bias, as is the case with the PSR the most valuable features available to users of method. Furthermore, with paired MCQs there is LMS, the improvement of available electronic no need of negative markings for each wrongly examination tools, such as those presented in this answered question. The elimination of significant chapter, might contribute in enhancing the value drawbacks of existing MCQs scoring methods of LMS to the users and, thereby, promoting the might enable the unimpeded exploitation of the wider adoption of LMS. main advantage of MCQs-based examinations, concerning speed and objectivity in grading and the extent of subjects that can be examined in a REFERENCES single examination session. MCQs, being an important part of examination Anastakis, D. J., Cohen, R., & Reznick, R. K. methods, and, in their electronic form, of LMS, it (1991). The structured oral examination as a should nevertheless be stressed that their initial method for assessing surgical residents. American introduction into the assessment procedure of an Journal of Surgery, 162, 67–70. doi:10.1016/0002- educational institution incurs a significant amount 9610(91)90205-R of workload to those who will construct the ques- Bennett, R. E., Rock, D. A., & Wang, M. (1991). tion bank, from which the MCQs will be selected Equivalence of free-response and multiple-choice for each examination session. This is due to the items. Journal of Educational Measurement, 28, requirements for the question bank, irrespective 77–92. doi:10.1111/j.1745-3984.1991.tb00345.x of the grading scheme that will be used, i.e., that it should be sufficiently large to cover the whole range of the material to be examined, that there

193 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Bereby-Meyer, Y., Meyer, J., & Budescu, D. V. Chu, L. F., Young, C. A., Ngai, L. K., Cun, T., (2003). Decision making under internal uncer- Pearl, R. G., & Macario, A. (2010). Learning tainty: The case of multiple-choice tests with management systems and lecture capture in the different scoring rules. Acta Psychologica, 112, medical academic environment. International 207–220. doi:10.1016/S0001-6918(02)00085-9 Anesthesiology Clinics, 48, 27–51. doi:10.1097/ AIA.0b013e3181e5c1d5 Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. (2002). Prospect theory analysis of guessing in Claroline Consortium. (2010). Claroline docu- multiple choice tests. Journal of Behavioral Deci- mentation. Retrieved December 28, 2010, from sion Making, 15, 313–327. doi:10.1002/bdm.417 http://doc.claroline.net/en/index.php/Main_Page. Bridgeman, B. (1991). Essays and multiple-choice Coombs, C., Milholland, J., & Womer, F. (1956). tests as predictors of college freshman GPA. The assessment of partial knowledge. Educational Research in Higher Education, 32, 319–332. and Psychological Measurement, 16, 13–37. doi:10.1007/BF00992895 Curzon, L. B. (1997). Teaching in further educa- Buchanan, T. (2002). Online assessment: Desirable tion: An outline of principles and practice (5th or dangerous? Professional Psychology, Research ed.). London, UK: Cassell. and Practice, 33, 148–154. doi:10.1037/0735- DeBord, K. A., Aruguete, M. S., & Muhlig, J. 7028.33.2.148 (2004). Are computer-assisted teaching methods Bull, J., Conole, G., Davis, H. C., White, S., & effective? Teaching of Psychology, 31, 65–68. Sclater, N. (2002). Rethinking assessment through doi:10.1207/s15328023top3101_13 learning technologies. In A. Williamson, C. Gunn, Ellis, R. K. (2009a). Field guide to learning man- A. Young, & T. Clear (Eds.), Proceedings of the agement systems. Learning circuits, American 19th Annual Conference of the Australasian society for training and development. Retrieved Society for Computers in Learning in Tertiary December 4, 2010, from http://www.astd.org/ Education (ASCILITE2002) (Vol. 1, pp. 8-11). NR/rdonlyres/12ECDB99-3B91-403E-9B15- Auckland, New Zealand: UNITEC Institute of 7E597444645D/23395/LMS_fieldguide_20091. Technology. pdf Bush, M. (1999). Alternative marking schemes Ellis, R. K. (2009b). Learning management sys- for on-line multiple choice tests. Paper presented tems 2009. Learning circuits and e-learning news, at the 7th Annual Conference on the Teaching of American society for training and development. Computing, Belfast. Retrieved August 14, 2010, Retrieved December 4, 2010, from http://www. from http://www.caacentre.ac.uk/resources/bib- astd.org/LC/2009/0509_LMS2009.htm. liography/biblio2.shtml. Ellis, R. K. (2010). Learning Management Systems Bush, M. E. (2006). Quality assurance of multiple- 2010. Learning Circuits and E-Learning News, choice tests. Quality Assurance in Education, 14, American Society for Training and Development. 398–404. doi:10.1108/09684880610703974 Retrieved December 4, 2010, from http://www. Carvalho, M. K. F. (2009). Confidence judg- astd.org/LC/2010/0910_LMSsurvey.htm. ments in real classroom settings: Monitoring Evans, L., Ingersoll, R. W., & Smith, E. J. (1966). performance in different types of tests. Inter- The reliability, validity and taxonomic structure of national Journal of Psychology, 44, 93–108. the oral examination. Journal of Medical Educa- doi:10.1080/00207590701436744 tion, 41, 651–657.

194 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Feldstein, M. (2004). What’s important in a Manning, S., & Dix, A. (2008). Identifying stu- learning content management system. eLearn dents’ mathematical skills from a multiple-choice Magazine. Retrieved December 28, 2010, from diagnostic test using an iterative technique to mini- http://www.elearnmag.org/subpage.cfm?section mise false positives. Computers & Education, 51, =tutorials&article=4-1 1154–1171. doi:10.1016/j.compedu.2007.10.010 Freeman, R., & Lewis, R. (1998). Planning and Mattheos, N., Stefanovic, N., Apse, P., Attstrom, Implementing Assessment. London, UK: Kogan R., Buchanan, J., Brown, P., & Walmsley, A. D. Page. (2008). Potential of Information Technology in dental education. European Journal of Dental Edu- Ganguli, S., Pedrosa, I., Yam, C.-S., Appignani, cation, 12(Suppl. 1), 85–91. doi:10.1111/j.1600- B., Siewert, B., & Kressel, H. Y. (2006). Part I: 0579.2007.00483.x Preparing first-year radiology residents and assessing their readiness for on-call responsibilities. McCoubrie, P., & McKnight, L. (2008). Single Academic Radiology, 13, 764–769. doi:10.1016/j. best answer MCQs: A new format for the FRCR acra.2006.02.057 part 2a exam. Clinical Radiology, 63, 506–510. doi:10.1016/j.crad.2007.08.021 GUnet Asynchronous eLearning Group (2010). Open e-class asynchronous e-learning platform - McKenzie, C. R. M., Wixted, J. T., Noelle, D. Platform description. Greek Academic Network. C., & Gyurjyan, G. (2001). Relation between Retrieved December 28, 2010, from http://eclass. confidence in yes-no and forced-choice taks. gunet.gr/manuals/OpeneClass23_en.pdf. Journal of Experimental Psychology. General, 130, 140–155. doi:10.1037/0096-3445.130.1.140 IMS Global Learning Consortium. (2006a). IMS question and test interoperability overview, McLean, L. D., Dauphinee, W. D., & Rothman, A. version 2.1 public draft (revision 2) specifica- (1988). The oral examination in internal medicine tion. Retrieved December 5, 2010, from http:// of the Royal College of Physicians and Surgeons www.imsglobal.org/question/qtiv2p1pd2/im- of Canada: A reliability analysis. Annals of the sqti_oviewv2p1pd2.html Royal College of Physicians and Surgeons of Canada, 21, 510–514. IMS Global Learning Consortium. (2006b). IMS question and test interoperability assessment test, Pressley, M., Ghatala, E. S., Woloshyn, V., & Pirie, section, and item information model. Version 2.1 J. (1990). Sometimes adults miss the main ideas public draft revision 2 specification. Retrieved De- and do not realize it: Confidence in responses to cember 5, 2010, from http://www.imsglobal.org/ short answer and multiple-choice comprehen- question/qtiv2p1pd2/imsqti_infov2p1pd2.html sion questions. Reading Research Quarterly, 25, 232–249. doi:10.2307/748004 Lukhele, R., Thissen, D., & Wainer, H. (1994). On the relative value of multiple-choice, Race, P. (2005). Making Learning Happen. Lon- constructed response, and examinee-selected don, UK: Sage. items on two achievement tests. Journal of Scharf, E. M., & Baldwin, L. P. (2007). As- Educational Measurement, 31, 234–250. sessing multiple choice question (MCQ) doi:10.1111/j.1745-3984.1994.tb00445.x tests - A mathematical perspective. Active Learning in Higher Education, 8, 31–47. doi:10.1177/1469787407074009

195 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Stergiopoulos, C., Tsiakas, P., Kaitsa, M., Triantis, Tsiakas, P., Stergiopoulos, C., Nafpaktitis, D., D., Fragoulis, I., & Ninos, C. (2006b). Methods Triantis, D., & Stavrakas, I. (2007). Computer of electronic examination applied to students as a tool in teaching, examining and assessing of electronics. Comparison of results with the electronic engineering students. In Proceedings conventional (paper-and-pencil) method. In J. of the International Conference of Computer as a A. Moinhos Cordeiro, V. Pedrosa, B. Encarna- Tool (IEEE-EUROCON 2007) (pp. 2490-2497). ção, & J. Filipe (Eds.), Proceedings of the 2nd Van der Linden, W. J., & Glas, C. A. W. (2000). International Conference on Web Information Computerized adaptive testing: Theory and prac- Systems and Technologies: Internet Technology tice. Heidelberg, Germany: Springer. / Web Interface and Applications(WEBIST 2006) (pp. 305-311). Setubal, Portugal: INSTICC Press Ventouras, E., Triantis, D., Tsiakas, P., & Stergio- poulos, C. (2010). Comparison of examination Stergiopoulos, C., Tsiakas, P., Triantis, D., & Kait- methods based on multiple-choice questions and sa, M. (2006a). Evaluating electronic examination constructed-response questions using personal methods of students of electronics. Effectiveness computers. Computers & Education, 54, 455–461. and comparison to the paper-and-pencil Method. In doi:10.1016/j.compedu.2009.08.028 Proceedinsg of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Wainer, H., & Thissen, D. (1993). Combining Computing (SUTC 2006) (Vol. 2, pp. 143-151). multiple-choice and constructed-response test Los Alamitos, CA: IEEE Computer Society. scores: Toward a Marxist theory of test construction. Applied Measurement in Education, Thomas, P. G. (2003). Evaluation of electronic 6, 103-1 18. marking of examinations. In D. Finkel, (Ed.), Proceedings of the 8th ACM Annual Conference Wainer, H., Wang, X.-B., & Thissen, D. on Innovation and Technology in Computer Sci- (1994). How well can we equate test forms ence Education (ItiCSE 2003) (pp. 50-54). New that are constructed by examinees? Journal York, NY: ACM. of Educational Measurement, 31, 183–199. doi:10.1111/j.1745-3984.1994.tb00442.x Triantis, D., Anastasiadis, C., Tsiakas, P., & Stergiopoulos, C. (2007). Asyncronous teaching Wass, V., Wakeford, R., Neighbour, R., & Van methods and electronic assessment applied to der Vleuten, C. (2003). Achieving acceptable students of the module “Physics of semiconduc- reliability in oral examinations: An analysis of tor devices”. Journal of Materials Education, the Royal College of General Practitioners mem- 29, 133–140. bership examination’s oral component. Medical Education, 37, 126–131. doi:10.1046/j.1365- Triantis, D., Stavrakas, I., Tsiakas, P., Stergiopou- 2923.2003.01417.x los, C., & Ninos, D. (2004). A pilot application of electronic examination applied to students Wood, R. (1991). Assessment and testing: A of electronic engineering: Preliminary results. survey of research. Cambridge, UK: Cambridge WSEAS transactions on advances in engineering University Press. education, 1, 26-30.

196 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

ADDITIONAL READING Philip, A., & Arnold, M. M. (2007). How many questions should I answer? Using bias profiles Downing, S. M. (2003). Validity: On meaning- to estimate optimal bias and maximum score ful interpretation of assessment data. Medical on formula-scored tests. The European Jour- Education, 37, 830–837. doi:10.1046/j.1365- nal of Cognitive Psychology, 19, 718–742. 2923.2003.01594.x doi:10.1080/09541440701326121 Downing, S. M. (2004). Reliability: On the Schubert, S., Schnabel, K. P., & Winkelmann, reproducibility of assessment data. Medical A. (2009). Assessment of spatial anatomical Education, 38, 1006–1012. doi:10.1111/j.1365- knowledge with a ‘3-dimensional multiple choice 2929.2004.01932.x test’ (3D-MC). Medical Teacher, 31, e13–e17. Haladyna, T. M. (2004). Developing and validating doi:10.1080/01421590802334325 multiple-choice test items. Mahwah, NJ: Lawrence Scouller, K. (1998). The influence of assess- Erlbaum Associates. ment method on students’ learning approaches: Hautau, B., Turner, H. C., Carroll, E., Jaspers, K., multiple-choice question examination versus as- Parker, M., & Krohn, K. (2006). Differential daily signment essay. Higher Education, 35, 453–472. writing contingencies and performance on major doi:10.1023/A:1003196224280 multiple-choice exams. Journal of Behavioral Scouller, K. M., & Prosser, M. (1994). Students’ Education, 15, 259–276. experiences in studying for multiple choice ques- Kennedy, P., & Walstad, W. B. (1997). Combin- tion examinations. Studies in Higher Education, ing multiple-choice and constructed-response test 19, 267–279. doi:10.1080/0307507941233138 scores: An economist’s view. Applied Measure- 1870 ment in Education, 10, 359–375. doi:10.1207/ Tan, L. T., & McAleer, J. J. A. (2008). The intro- s15324818ame1004_4 duction of single best answer questions as a test Leung, S. F., Mok, E., & Wong, D. (2008). The of knowledge in the final examination for the impact of assessment methods on the learning fellowship of the Royal College of Radiologists of nursing students. Nurse Education Today, 28, in Clinical Oncology. Clinical Oncology, 20, 711–719. doi:10.1016/j.nedt.2007.11.004 571–576. doi:10.1016/j.clon.2008.05.010 Muijtjens, A. M. M., van Mameren, H., Hoogen- Tarrant, M., Knierim, A., Hayes, S. K., & Ware, boom, R. J. I., Evers, J. L. H., & van der Vleuten, C. J. (2006). The frequency of item writing flaws P. M. (1999). The effect of a ‘‘don’t know’’ option in multiple-choice questions used in high stakes on test scores: Number-right and formula scor- nursing assessments. Nurse Education Today, 26, ing compared. Medical Education, 33, 267–275. 662–671. doi:10.1016/j.nedt.2006.07.006 doi:10.1046/j.1365-2923.1999.00292.x Traub, R. (1993). On the equivalence of the traits Norton, J. M. (1996). A comparison of methods for assessed by multiple choice and constructed- dealing with troublesome examination questions. response tests. In Bennett, R. E., & Ward, W. C. Advances in Physiology Education, 271, 55–60. (Eds.), Construction versus choice in cognitive measurement (pp. 29–44). Hillsdale, NJ: Lawrence Erlbaum Associates.

197 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Valdivia, R., & Nussbaum, M. (2009). Using Zeidner, M. (1987). Essay versus multiple choice multiple choice questions as a pedagogic model type classroom exams: The students’ perspec- for face-to-face CSCL. Computer Applications in tive. The Journal of Educational Research, 80, Engineering Education, 17, 89–99. doi:10.1002/ 352–358. cae.20196 Van der Watering, G., Gijbels, D., Dochy, F., & van der Rijt, J. (2008). Students’ assessment KEY TERMS AND DEFINITIONS preferences, perceptions of assessment and their relationships to study results. Higher Education, Constructed-Response Questions: A ques- 56, 645–658. doi:10.1007/s10734-008-9116-6 tion that requires the respondent to create his/her own response. Wallace, M., & Williams, R. L. (2003). Multiple- Electronic Examinations: A formal test choice exams: Explanations for student choices. involving answering questions and/or solving Teaching of Psychology, 30, 136–139. exercises using electronic media. Walubo, A., Burch, V., Parmar, P., Raidoo, D., Multiple-Choice Question: A question that Cassimjee, M., & Onia, R. (2003). A model for is accompanied by a list of possible responses, selecting assessment methods for evaluating medi- from which the respondent selects the answer(s). cal students in African medical schools. Academic Scoring Rules: The procedures used in the Medicine, 78, 899–906. doi:10.1097/00001888- process of assigning a score for evaluating the 200309000-00011 performance in a test.

198 199

Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility

Lourdes Moreno Universidad Carlos III de Madrid, Spain

Ana Iglesias Universidad Carlos III de Madrid, Spain

Rocío Calvo Universidad Carlos III de Madrid, Spain

Sandra Delgado Universidad Carlos III de Madrid, Spain

Luis Zaragoza News Service, Radio Nacional de España, Spain ABSTRACT Currently, the great majority of institutions of higher education use Learning Content Management Systems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects.

The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle1. In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents.

DOI: 10.4018/978-1-60960-884-2.ch010

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Disability Standards and Guidelines for Learning Management Systems

INTRODUCTION universal access and, at the same time, promote correct technological growth (Fichten, C.S., 2009). Information and Communication Technology Moreover, evaluations of the accessibility of these (ICT) plays a key role in people’s daily lives LMS and LCMSs and the certification of their (Rößling G. et al, 2008), a fact that is equally true compliance with accessibility standards should of people with and without disabilities. Over the also be required. past few years in the education sector and, more In the following section of this chapter, specific specifically, in institutions of higher education, technologies, accessibility standards and previ- Learning Management Systems (LMS) and Learn- ously published work regarding LMS accessibility ing Content Management Systems (LCMSs) have is discussed at length. In the third section, a new become extremely popular pedagogical tools for method for the evaluation of the compliance of teachers and students. Such is the current popular- an LMS with previously examined accessibility ity of LMS in these institutions, in fact, that LMS standards is proposed. This evaluation method are oftentimes the only tool given to students for is then put into practice in the fourth section for communicating with peers and teachers or for ac- the Moodle LMS whose accessibility, specifically cessing particular learning resources. Therefore, for visually-impaired individuals, is tested by an the negative impact of an inaccessible LMS on the accessibility expert and a visually-impaired end- learning experience of students with disabilities user (with the assistance of JAWS screen reader). would be large indeed. In order to provide equal Finally, the fifth section presents briefly general opportunities to all students, it is necessary to conclusions from the case study as well as areas improve the learning environment by removing all for future research. barriers to accessibility. LMS, LCMSs and their learning contents should be available to all students and teachers, including those with disabilities and BACKGROUND regardless of their particular accessibility needs. In the effort to make software completely ac- For the present study, we have considered a wide cessible to all types of users, it must be taken into variety of previous published works on accessibil- account that certain individuals require the use ity standards and regulations, LMS incorporating of Assistive Technologies (ATs) such as screen accessibility requirements into their design, as readers, refreshable Braille displays, speech well as studies of LMS accessibility evaluation synthesizers, magnifiers, adaptable keyboards or methods. With regard to this last point, the user- voice recognition software in order to see, hear, centered design (UCD) approach is considered move or interact with the system and its contents. and developed here. In addition to covering the widest range of user abilities, software should also take user prefer- E-Learning and Accessibility ences and learning styles (e.g., visual, auditory Standards or tactile) into account. The development of software in this way would allow all users, not just In order to make educational resource applications individuals with disabilities, to universally benefit and web sites universally accessible for all users, from system contents (Moreno, L. et al, 2008). not only educational standards like the Sharable Therefore, in order to ensure the achievement of Content Object Reference Model (SCORM), but this goal in the context of institutions of higher also accessibility standards like the Instructional learning, it is necessary to design and develop LMS Management System (IMS) guidelines for devel- and LCMSs according to standards that facilitate oping accessible learning applications2, the World

200 Disability Standards and Guidelines for Learning Management Systems

Wide Web Consortium (W3C) standards3, and the at the same time, support the production of ac- Web Accessibility Initiative (WAI) accessibility cessible content as specified in WCAG. We use guidelines4 must be followed. More specifically, ATAG 2.0 in sections three and four of this study. in order to ensure the accessibility of authoring tools like LMS or LCMSs, WAI Web Content Ac- Accessibility Metadata and Models cessibility Guidelines (WCAG) (W3C, 2008) and Authoring Tool Accessibility Guidelines (ATAG) Considering adaptability as an additional resource (W3C, 2010 a) should be closely followed, as well. of accessibility, some learning platforms are The WCAG guidelines are the most important based on the use of metadata, such as that of the components of the WAI. They are considered Dublin Core Adaptability Statement and IMS the official standard in the European Union and Global Learning Consortium5. With respect to are referenced in most legislation worldwide. this adaptability approach, the most important Other important initiatives exist, as well, such as educational standards are ISO/IEC JTC1/SC36/ U.S. legislation (29 U.S. Code §794d) requiring WG7 “Access for All” and the IMS AccessForAll conformance with U.S. Section 508 technical Metadata Specification (AccMD)6. standards, some of which relate specifically to web accessibility. Although less extensive, these User-Centered Design standards are nevertheless quite similar to the WCAG and studies exist in the accessibility lit- The user-centered design (UCD) approach is a erature that map the section 508 with the WCAG. process in which the needs and preferences of While most current regulatory frameworks still end-users of a product are taken into account reference the WCAG version 1.0, in December when that product is designed. The approach is 2008 the WCAG 2.0 was published as a new W3C particularly useful to consider in evaluations of Recommendation. the accessibility of LMS, and its inclusion in the According to the W3C, the ATAG Recommen- evaluation process is given methodological sup- dation is the method for evaluating accessibility port by standards such as ISO 9241-210:2010 requirements in LMS. The ATAG directs web (which replaced ISO 13407:1999)7 . In order to designers and developers in producing accessible fully consider the access-specific characteris- output (i.e., web pages) that meets standards and tics of persons with disabilities and the diverse guidelines. Moreover, they provide guidance for contexts of use that may create barriers to web prompting the content author (i.e., the authoring accessibility (Newell, AF. & Gregor, P., 2000), tool user) for accessibility related information. the UCD approach may be further extended to Furthermore, they provide ways of checking and that of Inclusive Design (Henry, S. 2007). The correcting inaccessible content, and integrating WAI indicates following this approach in the accessibility into the overall “look and feel” of the accessibility evaluation process (W3C, 2010 c). final software. Finally, these guidelines provide The evaluation process presented in this chapter help and documentation to make the authoring tool uses this approach, promoting the participation itself accessible for people with disabilities. While of end-users with disabilities as well as acces- the version ATAG 2.0 is has been developed (as sibility experts. a working draft) to be compatible with WCAG 2.0, ATAG 1.0 continues to be the current W3C Recommendation. According to ATAG 2.0, the LCMS user interface should be accessible and,

201 Disability Standards and Guidelines for Learning Management Systems

Assistive Technology For the evaluation presented later in this study, a JAWS screen reader was used. JAWS Disabled users interact with computers through is a screen reader which is able to recite aloud Assistive Technologies (ATs) and, as mentioned almost everything encountered on a web screen. earlier, it is essential to take ATs into account It is compatible with the Microsoft Windows when the UCD approach is followed in the evalu- operating system and IBM Lotus Symphony. It ation process. Many different types of ATs exist uses an adjustable synthetic voice to give feedback to respond to the diversity of needs of users with in the form of aural translations of user keyboard disabilities. An example of an AT for users with vi- commands, as well as of text/graphics depicted sion impairment is a screen reader which describes on the Internet. Any text, graphics or links are in audio the visible information appearing on a automatically recited by JAWS at variable speeds computer screen. For blind users, some specific and volumes. JAWS was chosen here due to its User Interface (UI) features are required, such as wide use, configuration options according to the providing a full control of interface elements, as needs and preferences of each user, and versatility well as easy and rapid navigation via keyboard. in achieving better functioning and monitoring of If web sites are designed in accordance with WAI various applications. Guidelines, the accessibility requirements for ATs will be included. In such cases, therefore, ATs should successfully interpret and relay back to the LMS AND ACCESSIBILITY user the semantic mark-up included in the web pages with WAI-ARIA (W3C, 2010 d). LMS Accessibility Studies The case study presented later in this chapter considers an LMS accessed with a screen reader. Some LMS or LCMSs such as Moodle, dotLRN16 Screen readers, such as JAWS8 or Window-Eyes9, or Atutor17, are designed with consideration for are commercial, special-purpose software pro- accessibility standards and problems. Most often, grams. The Linux Screen Reader10, the Orca screen such systems are based on learning standards like reader11 for the GNOME platform and the NVDA SCORM and IMS and on accessibility standards screen reader for Windows12 are free alternatives like ATAG and WCAG. Moreover, some important to these commercial products. Additionally, initiatives carried out in the area of accessibility Fire Vox13 is a free extension to the Firefox web and online learning systems have also been de- browser that provides screen reading functional- veloped (Martin, L et al, 2007), (Iglesias, A. et al, ity, the HearSay web browser14 is a standalone 2009). Other LCMSs found in the literature include self-voicing web browser, and aiBrowser15 is a Dokeos18, Docebo19, Sakai20, and Blackboard21. self-voicing web browser attempting to make For the evaluation of the accessibility of an multimedia web content accessible to blind users. LMS presented in section four below, Moodle Nevertheless, due to the expense of commercial was chosen due to the fact that it is one of the screen readers and the general lack of knowledge most widely-used LMS in the world (the Moodle. in the non-visually-impaired community about org website indicates that there are currently over their importance or the availability of free alter- 49,000 registered Moodle sites in 210 countries natives, screen readers are seldom installed on with over 34 million total users). Developed computers not normally and predominantly used at the Open University of the United Kingdom by blind individuals. which includes close to 9,300 individuals with disabilities among the total number of students

202 Disability Standards and Guidelines for Learning Management Systems

enrolled, Moodle is an open-source, flexible problems present in three different virtual learning software focused on providing accessibility to environments – Moodle, dotLRN and Blackboard all users. The software offers a large spectrum – is conducted. Nevertheless, the study is rather of available tools including, but not limited to, limited in that it only takes into account some course resources, assignment modules, blogs, examples of web pages and only evaluates a small chats, forums, glossaries, interaction, SCORM subset of Moodle tasks with respect to an equally packages, surveys, quiz modules, and wikis. small subset of WCAG 1.0 checkpoints. In Santos Due to the complexity of LMS, very few ac- et al. (Santos, O.C. et al, 2007), results obtained cessibility evaluations have been conducted to in the study of an e-learning framework with an date. Of the evaluations conducted, however, the initial end-user evaluation are presented. problems identified were often similar to those of Additionally, very few studies exist in the lit- static web sites. These problems are compounded erature that consider the use of screen readers by by a poor navigational structure, color contrast the visually-impaired. In (Buzzi C. et al, 2009), problems and other significant problems related the compliance of Moodle software with WCAG to a mark-up which conflicts with user agents 1.0 and the use of WAI-ARIA Suite is evaluated; and ATs. however, and as in Power et al., only a small subset Furthermore, only a few studies have been of Moodle tasks is taken into account. Finally, the undertaken to determine the general types of study does not check for Moodle’s compliance accessibility barriers common to most LMS and with the ATAG. considered as authoring tools. Most of these studies To our knowledge, no evaluation studies of present the following two particular drawbacks. the accessibility of LMS or LCMSs exist using First, since the majority of these studies are the ATAG. This paper tries to address the limita- based on survey and interview data rather than on tions of previous studies, therefore, by presenting empirical results, it has been difficult to make a an evaluation process for the Moodle LCMS as convincing case at institutions of higher learning accessed with a JAWS screen reader. Moreover, for improved accessibility awareness, training and the study is based on accessibility criteria in in- procedures. Second, while all of the studies found structional design and universal design principles present evaluations with respect to the WCAG, (Elias, T., 2010). none do the same with respect to the ATAG. Regarding particular studies, in Kalnins-Cole LMS Accessibility Evaluation & Peters (Kalnins-Cole, T. & Peters, D., 2007) Methodology an evaluation of dotLRN is presented along with recommendations for development changes that The ATAG offer a resource for evaluating the ex- would make dotLRN compliant with the W3C tent to which accessibility requirements are met by international accessibility standards. Results from authoring tools. An LMS or LCMS is an authoring the study show that dotLRN nearly possesses tool for building educational software, so ATAG Level A compliance (according to the international should be taken into account if the software is to standards) and that full Level A compliance would be accessible to all students. be easy to achieve. The study identifies Level As discussed in the prior sub-section, while a AAA compliance as a desirable long-term goal. number of studies of evaluation methodologies for In Power et al. (Power, C. et al., 2010), an the WCAG can be found in the literature and the initial empirical investigation into accessibility WAI, such is not the case with the ATAG. To our

203 Disability Standards and Guidelines for Learning Management Systems

knowledge, a formal evaluation methodology for automated software tools. The principal authoring tools needs to be proposed. It is the main advantage offered by manual testing is that aim of the present paper: to offer a new proposal it covers a broader range of accessibility of accessibility evaluation methodology for LMS. provisions than that considered by auto- The new proposal presented in this paper mated testing. For example, manual test- extrapolates the WCAG evaluation methodology ing can check if the alt attribute accurately from WAI (W3C, 2010 c) to be applied while taking describes the purpose of its corresponding into account the ATAG guidelines. It is important image or contains typical default texts such to note that ATAG 2.0 includes testable success as ‘‘image’’. criteria, techniques, and references to WCAG 2.0, • End-User Testing. This technique is car- so such an extrapolation is possible. ried out by real-world end-users. Logically, While software tools exist that allow for the the technique focuses on the end-users automatic checking of the WCAG conformance and how well specific technical solutions level, no such tools are available for the ATAG. match their needs in a specific context. Even with the existing tools, however, there are This testing technique easily complements many aspects of accessibility that can only be the other techniques previously described. evaluated manually. Thus, the combination of manual and automatic methods, together with The goal of accessibility standards is to isolate end-user testing following the UCD approach, accessibility problems and to define provisions guarantees the detection of barriers to accessibil- for avoiding them. Some accessibility provisions, ity as well as the later implementation of more but not all, can be detected by using automated effective accessibility solutions. testing. Moreover, it is important to identify the In general, three types of accessibility testing underlying causes of an accessibility problem. techniques exist (Abou-Zahra, S. 2008) which Sometimes accessibility problems exist with obtain the best results when combined in such a web content, but other times they originate with way that capitalizes on the specific advantages of the browser, ATs or even the user’s inability to each. These three techniques are the following: handle the tool. With the use of these different techniques, a • Automated Testing. This technique is car- particular method for the evaluation of authoring ried out by software tools without the need tools as LMS is proposed and described in the for human intervention. In automated test- following section. ing, the syntactic structure of web content is analyzed (e.g., checking for alt attributes Accessibility Evaluation in HTML elements). The testing Method for Lms technique is quite cost-effective. However, an important drawback is that it only ad- This chapter presents a new method of LMS dresses a subset of the accessibility provi- evaluation based on WAI recommendations. Fol- sions set out by most standards. lowing the consideration of the many different • Manual Testing. This technique is car- existing techniques for manual and automated ried out by human evaluators who could testing, it was concluded that a modified version be experts or novices, depending upon the of the WCAG evaluation process would be most nature of the accessibility test. In practice, appropriate here. This modified evaluation method most tests carried out by human evaluators includes the following steps and elements: are also frequently guided or supported by

204 Disability Standards and Guidelines for Learning Management Systems

• Set the ATAG 2.0 conformance level for • Manual evaluation. Complete the follow- which the evaluation is to be carried out. ing steps: • Identify web pages and tasks to be select- ◦◦ Examine a selection of web pages ed in the evaluation process, checking the using ATAG 2.0. As an additional remain functionalities and contents of the source for the checklist offered in the LMS with manual testing by people most official documentation for the ATAG, likely to enter the site. In LMS, these peo- this chapter has defined agile guides ple can be classified into three groups or for the application of the ATAG, profiles: administrator, teacher and student. which are described in greater detail • Combine and apply the distinct evalua- in the following section. tion methods listed below and described in ◦◦ Use several user agents to browse and greater detail in the following sub-sections: check the following: ◦◦ ATAG Expert Evaluation Method ▪▪ Turn off images and check (with automatic and manual testing) whether the appropriate alterna- ◦◦ Expert Evaluation Testing tive text is available. ◦◦ End-User Testing ▪▪ Turn off the sound and check that • Draw conclusions with a discussion of the the audio content is still avail- types of problems encountered, as well as able through text equivalents. the best practices to be continued or ex- ▪▪ Use the browser controls to vary tended. Indicate the method used to iden- font-size, checking that the font tify the problems discussed and finally, to size changes on the screen ac- recommend follow-up steps to be taken for cordingly and that the page is every accessibility barrier found during the still usable with larger font sizes. process (e.g., markup validation and other ▪▪ Test with different screen reso- tests). This would help to test for full con- lutions (i.e., by setting the appli- formance and address any problem identi- cation window to less than the fied previously. largest available size) in order to check that horizontal scrolling is ATAG Expert Evaluation Method not required. ▪▪ Change the display color to The method demonstrates compliance with the grayscale and check whether the ATAG 2.0 Working Draft and the WCAG 2.0 color contrast is adequate. Recommendation. The method comprises auto- ▪▪ Unplug the mouse and check mated and manual tests, the latter of which being that all links and form controls supported by automatic tools. The steps of the can be accessed by tabbing method are as follows: through them. Additionally, check that the links clearly indi- • Automated evaluation. Use two general cate what they lead to. accessibility evaluation tools at least and ▪▪ Read over the pages and check note any problems indicated by the tools. whether the text is clear and These tools only exist for checking the simple. WCAG.

205 Disability Standards and Guidelines for Learning Management Systems

▪▪ Use a text browser (e.g., Lynx) in the order given in the documentation was quite and check if the information heavy-handed. The evaluation process was quite available through the GUI and long and tedious and required the repetition of text browsers is the same. Also, particular tasks at distinct moments. The perfor- check if the information in the mance of these tasks could be easily improved, text browser is presented in a with the definition of concrete application paths meaningful order if read serially. and guidelines. ◦◦ Summarize the results. Using the Working Draft of July 8th, 2010 (W3C, 2010 a) an agile guide for the ATAG was As discussed in previous sections of this elaborated and is presented below in Table 1. As chapter, the ATAG establish the accessibility can be observed, each of the different groups guidelines for the design of web content author- presented are applied sequentially. ing tools. These guidelines are organized into The process described above has been created two parts which specify the following: (1) the to optimize and make more agile the accessibil- authoring tool should be accessible and usable for ity evaluation process of authoring tools. Relative authors with disabilities and (2) the authoring tool to evaluations carried out prior to those discussed should permit, support, promote and guarantee the in this chapter, the application of these agile guides production of accessible web content. Similar to resulted in a considerable reduction in evaluation the WCAG, the ATAG establish three different times and a greater understanding of the guidelines. levels of conformance: Level A, Level AA and Level AAA. Expert Evaluation Testing For the elaboration of agile guides for accessibility evaluation process, an analysis of official The method includes manual testing carried out documentation for the ATAG was carried out. Re- by evaluators (1) with intimate knowledge of how sults revealed that the application of the guidelines people with disabilities use the web and (2) who

Table 1. Agile guidelines to evaluate ATAG 2.0

GROUP DESCRIPTION SPECIFIC GUIDELINES (ATAG) Part A: Make the authoring tool accessible for the user interface. Group #1: Necessary for the evaluation of conformity with the A.1.1.1, A.3.1.2, A.3.2.1, A.3.2.2, A.3.3.1, A.4.1.1, A.4.1.2, Content accessibility guide- WCAG. The first guidelines to be followed if the LMS A.1.1.2, A.3.1.3, A.4.1.3, A.1.1.3, A.3.1.4 lines authoring tool is to be accessible. Group #2: Guidelines corresponding to the evaluation of accessibility A.2.1.1, A.3.1.1 Accessibility standards: requirements concerning the authoring tool user interface Clients Group #3: Guidelines corresponding to the evaluation of accessibility A.1.2.1, A.2.2.2, A.2.2.3, A.2.2.1 Accessibility standards: requirements for non-web tools. Non-web tools Group #4: Guidelines corresponding to the evaluation of accessibility A.2.3.1, A.3.2.2, A.3.2.3, A.3.4.1, A.3.4.2, A.3.4.3, A.3.4.4, Accessibility in configuration requirements regarding the configuration and structure of A.3.5, A.3.6.1, A.3.6.3, A.3.6.3, A.3.7.2, A.3.7.1 & structure the tool. Group #5: Guidelines related to accessibility documentation facilitated A.4.2.1, A.4.2.2 Documentation accessibility by authoring tool. Part B:Support the production of accessible content. Group #6: The same ATAG 2.0 guidelines are included and sequentially applied to groups with attention paid to the conformity level Production of accessible divisions A, AA and AAA. content

206 Disability Standards and Guidelines for Learning Management Systems

can accurately identify problems related to user and using shortcut keys, are able to avoid acces- interaction. Similar to walkthroughs and heuristic sibility barriers identified by accessibility experts. evaluations in the field of usability engineering, Additionally, accessibility experts may discover expert evaluators anticipate the issues that end- particular accessibility barriers previously passed users may encounter in the content. In this method, over unnoticed by end-users due to the latter’s manual evaluation is conducted using simulation extensive experience with screen reader software. (e.g., testing keyboard access, the same view in several user agents) and screening techniques Evaluation Method: Checking (e.g., testing accessibility using a screen reader). Compliance with ATAG 2.0

End-User Testing In this sub-section, the main results obtained following the execution of the evaluation method are Testing here is carried out by real-world end-users. presented. The evaluation of the compliance of the It includes the evaluation of accessibility with ATs Moodle LMS with ATAG 2.0 presented here was (W3C, 2010 b). User testing distinguishes between performed by two accessibility experts supported formal and informal checks, with the former usu- by the automatic tools TAW22, AChecker23, and ally being carried out by professionals following the Web Accessibility Toolbar (AIS for Internet well-established procedures (surveys, interviews Explorer24 and the Web Developer extension for …) and the latter by non-experts. Firefox25). In the evaluation, a direct installation In the following section, the method outlined of the official Moodle version 1.9 packages was here is used in a practical example of LMS ac- made, and all basic web pages resulting from the cessibility evaluation. installation were analyzed. While the inclusion of all possible content in Moodle web pages was not tested here, some different types were included CASE STUDY: EVALUATION OF and tested in order to verify Moodle’s ability to ACCESSIBILITY IN MOODLE LMS create accessible content (Part B of the ATAG). A more detailed description of this evaluation is This section presents the method and the results presented below. obtained from an evaluation of the accessibility of In Part A of ATAG 2.0 Working Draft, all the Moodle LMS. The evaluation of the LMS is success criteria must present authoring tool user performed with respect to the ATAG and the use interface-related accessibility problems, this main of the JAWS screen reader AT resource. that the authoring tool user interface must conform Following the method described in the previous to WCAG. In Part B, all success criteria must section, sub-section 4.1 examines the evaluation present accessible web content production issues. method using the ATAG, while the sub-section 4.2 The result obtained in the accessibility evalu- presents an expert evaluation and user testing using ation of Moodle indicates that the lower acces- both JAWS resource and combining the results in sibility level “A” of ATAG is not achieved. In order to obtain tighten conclusions. In this case, relation to the evaluation of compliance of Moodle the combined results improve the conclusions with WCAG 2.0, if Conformance Level “A” is due to the fact that the problems and difficulties obtained, then the success criteria A.1.1.1 of founded by end-users and experts accessing a ATAG is fulfilled, but the A.1.1.2 and A.1.13 are product via an AT are not the same.. For example, not. Figure 1 and Figure 2 show the several levels end-users accessing an LMS via JAWS, due to of conformance resulting from the evaluation of their experience using non-accessible web-sites Moodle with respect to ATAG 2.0 and WCAG 2.0

207 Disability Standards and Guidelines for Learning Management Systems

Figure 1. Number of ATAG (Part A and B) guidelines: fulfilled (Yes), did not fulfill (No) and did not apply (N/A)

Figure 2. Number of WCAG guidelines: fulfilled (Yes), did not fulfill (No) and did not apply (N/A)

One feature obtained of evaluation of the It is important to emphasize that Moodle in compliance of Moodle with WCAG 2.0 is that in the assistant form with the help of users is able the user interface of the Moodle authoring tool to avoid and correct mistakes. If an input error for non-text content present to the user, Moodle is automatically detected, the presenting the ac- shows an alternative text that serves the same cessibility error is identified and then the error is purpose (Success Criteria 1.1.1. WCAG 2.0 described to the user in text as shown in Figure 3. [Level A) is fulfilled). In the same way, labels or instructions are The functionality of the Moodle interface is provided when content requires user input (Suc- operable through a keyboard (Success Criteria cess Criteria 3.3.1 and 3.3.2 of WCAG 2.0 2.2.1 of WCAG 2.0 [Level A] is fulfilled and [Level A] is fulfilled). However, if an input error Success Criteria 3.3.1 of ATAG 2.0 [Level A] is is automatically detected and suggestions for fulfilled too). correction are known, the suggestions are not

208 Disability Standards and Guidelines for Learning Management Systems

Figure 3. Assistance in Moodle interface for forms: Help users correct mistakes

provided to the user (Success Criteria 3.3.3 of than by using CSS styles. Consequently, some WCAG 2.0 [Level AA] is not met). WCAG 2.0 guidelines are not fulfilled. Ad- Moodle uses tables for the layout for the main ditionally, if a user wants to add an image, the content of the web page. However, the content is editor allows the user to include an alternative understandable when it is read line to line, so there description which satisfies point 1.1.1 (Level are no accessibility problems in this case. Addi- A) of WCAG 2.0. Nevertheless, Moodle does tionally, there is a mechanism to bypass blocks of not provide warning advices if the user does not content that are repeated on multiple web pages include an alternative description for the image. (see Figure 4 the link: “Skip to main content”). Moodle does not prevent the user from making The most important accessibility barrier found accessibility errors, so it does not satisfy the B.2.2 during the accessibility evaluation of Moodle’s and B.2.3 guidelines from ATAG 2.0. taking into account ATAG 2.0 is the failure to Other observed feature is about “lang” HTML meet Guideline B.2.5: “Assist authors with ac- attribute, Moodle WYSIWYG editor include cessible templates and other pre-authored content it indicating the language of the system’s con- (Level AA)”. Moodle offers multiple types of tents. Nevertheless, Moodle incorrectly uses the themes (i.e., templates) to assign, but it does not

HTML element, using it with the provide a default theme with an accessible inter- sole intention of achieving a visual effect and not face in compliance with WCAG 2.0 Level AA. for the real aim of the tag, which is to identify a Therefore, B.2.5.8 is not fulfilled. Additionally, text block as a quote. It does not comply with the Moodle does not provide support for creating 3.1.2 (Level AA) success criteria of WCAG 2.0. accessible web content. Therefore some guidelines One of the most important problems related to of Part B of ATAG 2.0 are not fulfilled. the use of JavaScript in Moodle is that when it is Moodle provides a default WYSIWYG editor. disabled in the user agent, it is not possible to use The main problem found in the default editor used the WYSIWYG editor. As an alternative, Moodle by Moodle is that the colors and fonts are included gives a text area to edit the content. However, in HTML with the HTML element, rather although a text area for plain text is an alternative
209 Disability Standards and Guidelines for Learning Management Systems
Figure 4. Mechanism in Moodle to bypass blocks of content that are repeated on multiple web pages
for editing contents, users usually make mistakes Moodle packages. Moodle 1.9 and JAWS 10.0 due to their lack of knowledge of accessibility versions were used here. guidelines resulting in an editing process which A testing procedure with a list of tasks was may not be accessible (2.3.3 of ATAG cannot be completed in the evaluation process, checking the fulfilled). For instance, by disabling the use of main functionalities and contents of Moodle. Both its default editor, Moodle permits one to choose end-users and the accessibility expert interacted the background color of the editor’s text area, an with Moodle – with the expert checking the list of interesting option if it were necessary to achieve tasks with and without JAWS – and in the same an ideal contrast of colors. manner so that interaction results across the LMS In the present evaluation, Moodle demonstrates would be comparable. only partial accessibility and does not comply Appendix 1 shows a table (Table 2) with the with the WCAG or ATAG. Accessibility strate- tasks evaluated in column 3. This table presents gies must, therefore, be included in the design of the type(s) (i.e., profile) of user with the ability to future versions in order to obtain an LMS that is execute the task (e.g., administrator [A], teacher more accessible for all users. [T], student [S]). With regard to this final category, while all these tasks can be performed by Expert Evaluation and User Testing a Moodle user with an administrator profile, users with teacher or student profiles may execute only The evaluation of the accessibility of Moodle a subset of tasks. In the evaluation of accessibility, was performed in July of 2010 by an accessibil- the complete set of Moodle functionalities com- ity expert. End-user testing was conducted by prising the tasks that every user can perform was a visually-impaired person (since childhood) checked. Also, this table shows the functionality and habitual user of JAWS. Both the expert and group in other column and the name of the task end-user made a direct installation from official evaluated in another column. The fourth column offers brief descriptions of accessibility problems
210 Disability Standards and Guidelines for Learning Management Systems
where they arose. And finally, the last column • E3: The user is redirected to another page shows if the individual tasks could be executed without prior warning. using JAWS and, in affirmative cases, whether • E4: The look and feel of Moodle changes such executions involved accessibility problems in some tasks. (Yes*) or no accessibility problems (Yes). In • E5: Tables are used for layout. cases where accessibility problems using JAWS • E6: Text images are used to convey prevented the successful execution of the task, a information. different symbol (No) is used. As opposed to the • E7: Information regarding how to com- related work previously described, the accessibil- plete a task is confusing for the user or dif- ity analysis presented in this study was performed ficult to obtain. on all basic web pages resulting from the direct • E8: Some text in English appears even installation of Moodle technology. This allowed us when Spanish is the selected language of to assess Moodle’s full capacity for accessibility the tool. for visually-impaired users. • E9: There is no button allowing the user to As was briefly discussed earlier in the study, cancel the operation. there is sometimes an overlap between barriers • E10: Tables are not well structured, creat- to accessibility and usability problems (Moreno, ing problems for the screen reader. L. et al, 2009), entailing cognitive barriers to the • E11: There are no page or table headings. successful use of the virtual learning environ- • E12: The many rows in the table make the ment. Some major usability problems directly table difficult to read and require the user affecting accessibility have been detected here to memorize the table structure. and have been included in Table 2 together with • E13: Text appears that can only be modi- accessibility barriers. fied with Windows tool and it do not access Following the exhaustive evaluation process, by keyboard. it is quite clear that the combination of accessibil- • E14: Some text descriptions are incorrect. ity expert and end-user evaluation is extremely • E15: The application does not check data helpful for the detection of the principal acces- inserted into the form, and the errors mes- sibility problems present in Moodle. Furthermore, sages are not useful for the users, making many of these accessibility problems such as the it difficult for users to imagine the cause of lack of descriptive tags and section headings for particular problems. web pages and forms, the dependency upon the • E16: The screen reader does not always non-accessible Windows editor for long strings in read the text correctly. the forms (e.g., descriptions), the changes to the look and feel of different tasks and the redirec- Table 2 presents the detail of all these errors tion to other web page or the refreshing of web including the tasks were they are found. page without previously notifying the user are As it can be observed, the tool contains many repeated in multiple Moodle tasks. In sum, the different obstacles to accessibility, the most com- errors detected are those presented below: monly produced of which being E4 and E1. With these errors, the user may be confused since the • E1: Not all text and combo boxes contain appearance of the website is not always the same associated descriptive texts. and the content is not always clear. The least fre- • E2: Pages refresh without previously con- quently produced errors from the list are E12, E14, sulting the user. E15 and E16. Thus, while important, these errors
211 Disability Standards and Guidelines for Learning Management Systems
are nevertheless relatively insignificant insofar the contents of the LMS and their accessibility. as they appear only once in the tool. Therefore, it is very important to follow accessibil- It is very important to understand that these ac- ity strategies in Moodle module sites. Lastly, this cessibility problems can seriously and negatively study has pinpointed specific contexts in which a affect any type of Moodle user (i.e., administrators, particular user group cannot access certain features teachers, and students), keeping the user from and contents of the LMS. correctly and completely accessing the LMS.
FUTURE RESEARCH CONCLUSION One of the most consistent problems with modern Learning Management Systems (LMS) and learning management systems is their failure to Learning Content Management Systems (LCMS) comply with international standards for acces- are nowadays necessary in institutions of higher sibility. There is a clear need for fully accessible education and should be accessible for users with LMS to be made available, so that educational and without disabilities. For this reason, accessibil- institutions can meet the needs of all students, ity standards must be considered. In this chapter, including those with disabilities. Future LMS technology and disability standards for LMS have development should be required to meet specified been examined in detail. In order to fully develop levels of accessibility and future LMS research and use a resource to evaluate one such system, should define methodological frameworks to be the most important LMS-related standards such used to achieve this accessible development. Fur- as ATAG and WCAG have been described. In thermore, additional work must be done to develop addition, the UCD approach has been introduced accessibility policies that can be implemented by and recommended in order to take into account institutions of higher learning. In this way, not only the participation of users with disabilities in the may the needs of individuals with disabilities be LMS accessibility evaluation process. completely addressed, but the greater university In addition to the review of related literature, a community may also be educated about these new general method for evaluating the accessibil- needs in the process. ity of an LMS based on automatic, expert and end- user testing has been proposed. The combination of several techniques has returned more complete ACKNOWLEDGMENT result in this experimentation. With the objective of presenting a practical application of the method, This research work was supported by the Re- a case study focusing on the Moodle LMS and search Network MA2VICMR (S2009/TIC-1542) visually-impaired end-users (i.e., those requiring (see www.mavir.net/), GEMMA (TSI-020302- screen readers) has been carried out and analyzed. 2010-141) and SAGAS (TSI-020100-2010-184) The results obtained from the case study research projects. demonstrate that the Moodle LMS is only partially, not completely accessible. Moreover, the evaluation shows that an LMS or LCMS does not REFERENCES usually support the task of authoring, reusing or re-purposing contents as well as virtual spaces for Abou-Zahra, S. (2008). Web accessibility evalu- student interaction. For instance, Moodle does not ation. Human-computer interaction series, 2008, offer sufficient support to teachers, despite the fact Part II, 79-106. that it is they who are ultimately responsible for
212 Disability Standards and Guidelines for Learning Management Systems
Buzzi, C., Buzzi, M., & Leporini, B. (2009). Moreno, L., Martínez, P., & Ruiz-Mezcua, B. Accessing e-learning systems via screen reader: (2009). A bridge to Web accessibility from the us- An Example . In Jacko, M. J. A. (Ed.), Human- ability heuristics, 5th Annual Usability Symposium Computer Interaction, Part IV, HCII 2009, LNCS USAB 2009. Usability & HCI eInclusion, LNCS 5613 (pp. 21–30). 5889 (pp. 290-300). Berlin, Germany: Springer. Elias, T. (2010). Universal instructional design Newell, A. F., & Gregor, P. (2000). User sensitive principles for Moodle, Canada International Re- inclusive design: In search of a new paradigm. view of Research in Open and Distance Learning, CUU 2000 First ACM Conference on Universal 11(2). May – 2010. Usability, 2000. Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwoj- Power, C., Petrie, H., Sakharov, V., & Swallow, D. ka, C., Barile, M., Nguyen, M. N., & Wolforth, J. (2010). K. virtual learning environments: Another (2009). Disabilities and e-learning problems and barrier to blended and e-learning. Miesenberger solutions: An exploratory study. Journal of Edu- et al. (Eds.), ICCHP 2010, Part I, LNCS 6179 cational Technology & Society, 12(4), 241–256. (pp. 519–526). Henry, S. L. (2007). Just ask: Integrating acces- Rößling, G., Joy, M., Moreno, A., Radenski, A., sibility throughout design. Madison, WI: ET\ Malmi, L., Kerren, A., & Iturbide, J. Á. (2008). Lawton. Available at www.uiAccess.com/justask/ Enhancing learning management systems to better support computer science education (pp. Iglesias, A., Moreno, L., Revuelta, P., & Jiménez, J. 142-166). Paper presented at the SIGCSE 2008, (2009). APEINTA: A Spanish educational project 4, Nov. 2008. aiming for inclusive education in and out of the classroom. ITiCSE ‘09, Paris, France (p. 393). Santos, O. C., Boticario, J. G., Del Viso, A. F., New York, NY: ACM, NY. De La Cámara, S. P., Sánchez, C. R., Gutiérrez & Restrepo, E. (2007). Basic skills training to Kalnins-Cole, T., & Peters, D. (2007). Towards disabled and adult learners through an accessible an accessible learning management system. In- e-learning platform. Lecture Notes in Computer ternational Journal of Continuing Engineering Science (including subseries Lecture Notes in Education and Lifelong Learning, 17(1), 84–96. Artificial Intelligence and Lecture Notes in Bio- doi:10.1504/IJCEELL.2007.013232 informatics) (LNCS 4556, part 3, pp. 796-805). Martin, L., Gutiérrez-Restrepo, E., & Barrera, World Wide Web Consortium. (W3C) (2008). C. Rodríguez-Ascaso1, A., Santos, Olga C. & Web accessibility initiative (WAI), Web Content Boticario, J. G. (2007). Usability and accessibility Accessibility Guidelines (WCAG) 2.0. Available evaluations along the e-learning cycle, IWWUA - at http://www.w3.org/TR/WCAG20 / WISE 2007 Workshops, LNCS Volume 4832/2007, Nancy, Francia (pp. 453—458). World Wide Web Consortium. (W3C) (2010). Web Accessibility Initiative (WAI), Authoring Tool Moreno, L., Iglesias, A. M., Castro, E., & Martinez, Accessibility Guidelines (ATAG) 2.0. Available at P. (2008). Using accessible digital resources for http://www.w3.org/TR/ATAG20/ teaching database design: towards an inclusive distance learning proposal. ITICSE 2008, Madrid, World Wide Web Consortium. (2010). WAI-ARIA Spain (pp. 32–36). New York, NY: ACM. Overview. Available at http://www.w3.org/WAI/ intro/aria
213 Disability Standards and Guidelines for Learning Management Systems
World Wide Web Consortium. (W3C). (2010 b). Dunn, S. (2003). Return to SENDA? Implement- Involving users in evaluating Web accessibility. ing Accessibility For Disabled Students In Vir- Available at http://www.w3.org/WAI/eval/users. tual Learning Environments In UK Further And html Higher Education. Retrieved September 1, 2004, from personal Web site: http://www.saradunn. World Wide Web Consortium. (W3C). (2010 c). net/VLEreport/ WAI, evaluating Web Sites for accessibility: Over- view. Available at http://www.w3.org/WAI/eval/ Henry, S. L. (2007). Just Ask: Integrating Acces- sibility Throughout Design by, Lulu.com. February 2007, ISBN-10: 1430319526. Available at http:// uiaccess.com/accessucd/about.html ADDITIONAL READING Iglesias, A., Moreno, L., Jiménez, J., & Revuelta, W3C, WAI, Authoring Tool Accessibility Guide- P. (2010). Evaluating the Users’ Satisfaction Using lines (ATAG) Overview (2010). Available at http:// Inclusive Initiatives in Two Different Environ- www.w3.org/WAI/intro/atag.php ments: The University and a Research Conference. W3C, WAI, Introduction to “How People with Dis- Computers Helping People with Special Needs, abilities Use the Web” (2010). Available at http:// 12th International Conference, ICCHP 2010, www.w3.org/WAI/intro/people-use-web.php Vienna, Austria, July 14-16, 2010. LNCS, 2010, Volume 6179/2010, 591-594 W3C, WAI, Web Content Accessibility Guidelines (WCAG) Overview. (2010). Available at http:// Isaila, N. & Nicolau, I. (2010). Promoting com- www.w3.org/WAI/intro/wcag.php puter assisted learning for persons with disabilities. Procedia - Social and Behavioral Sciences. 2(2),, W3C, Web Accessibility Initiative (WAI). (2010). 4497-4501. Available at http://www.w3.org/WAI/ Kelly, B., Phipps, L., & Howell, C. (2005). Abascal, J., Arrue, M., Fajardo, I., & Garay, N. Implementing a holistic approach to e-learning (2004). The use of guidelines to automatically accessibility. In: ALT-C 2005, 6-8 September verify Web accessibility. Universal Access in the 2005, Manchester, UK. Information Society, 3(1), 71–79. doi:10.1007/ s10209-003-0069-3 Mankoff, J., Fait, H., & Tran, T. (2005). Is your Web page accessible? A comparative study of Alan, F. Newell & Peter Gregor (2000). User methods for assessing Web page accessibility for sensitive inclusive design in search of a new the blind. In Proceedings of the SIGCHI confer- paradigm. In Proceedings on the 2000 conference ence on Human factors in computing systems on Universal Usability (CUU ‘00) (pp. 39-44). (CHI ‘05). ACM, New York, NY, USA, 41-50. New York, NY: ACM. NCAM. (2006). CPB/WGBH National Center for Cann, C., Ball, S., & Sutherland, A. (2003). To- Accessible, Media (NCAM), Accessible Digital wards accessible virtual learning environments., Media Guidelines. Available at http://ncam.wgbh. TechDis Web site. Available at http://www.jis- org/invent_build/web_multimedia/accessible- ctechdis.ac.uk/techdis/home digital-media-guide Cook, A. M., & Polgar, J. M. (2007). Cook and Hussey’s Assistive Technologies: Principles and Practice. Mosby.
214 Disability Standards and Guidelines for Learning Management Systems
Pearson, E., & Koppi, T. (2002). Inclu- system development that focuses specifically on sion And online learning opportunities: De- making systems usable. It is a multi-disciplinary signing For accessibility. Association for activity.” In UCD, all “development proceeds Learning Technology Journal, 10(2), 17–28. with the user as the center of focus.” doi:10.1080/0968776020100203 Web Accessibility: Web accessibility means that people with disabilities can use the Web. Preiser, W. F. E., & Ostroff, E. (Eds.). (2001). More specifically, Web accessibility means that Universal design handbook. New York, NY: people with disabilities can perceive, understand, McGraw-Hill. navigate, and interact with the Web, and that they Rice, W. (2006). Moodle E-Learning Course can contribute to the Web. Web accessibility also Development. Packt Publishing. benefits others, including older people with changing abilities due to aging. Rose, D., & Meyer, A. (2000). Universal design WYSIWYG: This is an acronym for “What for learning. Journal of Special Education Tech- You See Is What You Get”. A WYSIWYG view nology, 15(1), 67–70. displays (to authors) the content being edited in Seale, J. (2006). E-learning and disability in higher a way that is very similar to how it will appear education: accessibility research and practice. to end users. UK: Taylor & Francis. Vanderheiden, G. C. (2009). Accessible and usable ENDNOTES design of Information and Communication Tech- nologies. In C. Stephanidis (Ed.), The Universal 1 Moodle, a Learning Management System Access Handbook. UK: Taylor & Francis Group. (LMS). Available at http://moodle.org/ ISBN: 0-8058-6280-3. Pp: 3-1, 3-26. 2 IMS Global Learning Consortium. Available at http://www.imsglobal.org/ 3 W3C: The World Wide Web Consortium. KEY TERMS AND DEFINITIONS Available at http://www.w3.org/ 4 WAI: Web Accessibility Initiative from Assistive Technology: Software (or hard- W3C. Available at http://www.w3.org/WAI/ ware), separate from the authoring tool, that 5 Dublin Core Metadata Initiative. Available provides functionality to meet the requirements of at http://dublincore.org/ users with disabilities. Some authoring tools may 6 IMS AccessForAll Meta-data. Available also provide direct accessibility features at http://www.imsglobal.org/accessibility/ Authoring Tool: Any software, or collection of accmdv1p0/imsaccmd_bestv1p0.html software components, that authors can use to create 7 ISO (International Organization for Stan- or modify web content for use by other people. dardization). Available at http://www.iso. Template: A content pattern that is filled in by org/iso/home.html authors or the authoring tool to produce content 8 Freedom Scientific - JAWS Screen Reading for end users (e.g., document templates, content Software. Available at http://www.freedom- management templates, presentation themes). scientific.com/products/fs/jaws-product- Often templates will pre-specify at least some page.asp authoring decisions. 9 Window-Eyes. Available at http://www. User-Centered Design Process (UCD): “Hu- gwmicro.com/Window-Eyes/ man-Centred design is an approach to interactive
215 Disability Standards and Guidelines for Learning Management Systems
10 Linux Screen Reader. Available at http:// 18 Dokeos, an Open Source E-Learning. Avail- live.gnome.org/LSR able at http://www.dokeos.com/es 11 Orca. Available at http://live.gnome.org/ 19 Docebo, an Open Source E-Learning plat- Orca form. Available at http://www.docebo.org/ 12 NonVisual Desktop Access (NVDA) . Avail- doceboCms/ able at http://www.nvda-project.org/ 20 Sakai Project, an Open Source suite of learn- 13 Fire Vox: A Screen Reading Extension for ing. Available at http://sakaiproject.org/ Firefox. Available at http://www.firevox. 21 Blackboard. Available at http://www.black- clcworld.net/ board.com/ 14 HearSay project. Available at http://www. 22 TAW, a tool for the analysis of Web sites. cs.sunysb.edu/~hearsay/ Available at http://www.tawdis.net/ingles. 15 ACTF, Accessibility Internet Browser for html?lang=en Multimedia (aiBrowser). Available at http:// 23 AChecker, an open source Web accessibil- www.eclipse.org/actf/downloads/tools/ ity evaluation tool. Available at http://www. aiBrowser/index.php atutor.ca/achecker/ 16 dotLRN, An open source E-learning. Avail- 24 AIS, Web Accessibility Toolbar. Available able at http://www.dotlrn.org/ at http://www.visionaustralia.org.au/ais/ 17 ATutor, Learning Content Management toolbar/ System. Available at http://www.atutor.ca/ 25 Firefox Accessibility Extension. Available at http://firefox.cita.uiuc.edu/
216 Disability Standards and Guidelines for Learning Management Systems
APPENDIX 1: TABLE 2 WITH SUMMARY OF EXPERT AND USER EVALUATIONS
This table presents the type(s) (i.e., profile) of user with the ability to execute the task (e.g., administrator [A], teacher [T], student [S]), the functionality group, the name of the task evaluated. The fourth column offers brief descriptions of accessibility problems where they arose. And finally, the last column shows if the individual tasks could be executed using JAWS and, in affirmative cases, whether such executions involved accessibility problems (Yes*) or no accessibility problems (Yes). In cases where accessibility problems using JAWS prevented the successful execution of the task, a different symbol (No) is used.
Table 2. Summary of expert and user evaluations
User Profile Functionality (group) Task Name Errors Can it be completed? A/T/ S General Login user E1 Yes* A/T/S General Change language Moodle E1/E2 Yes* A Users/ Authentication Manage authentication E8 /E10 Yes* A Users/ Authentication Email-based self-registration E5 Yes* A Users/ Authentication No login E7 Yes* A Users/ Authentication Manual accounts E5 Yes* A Users/Accounts Browse list of users E7/E5/E9/E10 Yes* A Users/Accounts Bulk user actions E2/E11 Yes* A Users/Accounts Add a new user E6/E8/E11/E13 Yes* A Users/Accounts Upload users -- Yes A Users/Accounts Upload user pictures -- Yes A Users/Accounts User profile fields E3/E7/E13 Yes* A Users/Permissions Define roles E7/E8/E13 Yes* A/T Users/Permissions Assign system roles E1/E9 Yes* A Users/Permissions User policies E8/E9 Yes* A/T* Courses Add /Edit courses E4/E13 Yes* A Courses Enrollments E9/E11 Yes* A/T/S Courses Participants -- Yes A/T Courses Backup -- Yes A/T Courses Restore a course E5/E7/E9/ E10/E11 No A/T Courses Import E4/E5 Yes* A/T Courses Reset course E4 Yes* A Grades My preferences grader report E1/E3/E4/E6/ E7/E11 Yes* A/T/S Grades/View Overview report E1/E4 No A/T Grades/View Grader report E1/E4 Yes* A/T/S Grades/View User report E1/E4/E10 Yes* A/T Grades/Categories and Items Simple view E1/E4/E10 Yes* A/T Grades/Categories and Items Full view E1/E4/E8/E10/ E12 Yes* A/T Grades/Scales View E1/E4/E10/E13 Yes* A/T Grades/Letters View E1/E4/E16 Yes* A/T Grades/Letters Edit E1/E4 Yes*
continued on following page
217 Disability Standards and Guidelines for Learning Management Systems
Table 2. Continued
User Profile Functionality (group) Task Name Errors Can it be completed? A/T Grades/Import CSV file E1/E4/E9 Yes* A/T Grades/Import XML file E1/E4 Yes* A/T Grades/Export To Open doc spreadsheet / E1/E4/E9 Yes* Plain text file/Excel spdsht/ XML file A/T Reports Filter logs E1/E4 Yes* A/T Reports Activity report E4/ E14 Yes* A/T Reports Participation report E4/ E8/E11 Yes* A/T Questions Questions bank E3/E8/E13 Yes* A/T Reports Live logs from the past hour E2 No A/T Questions Import E4/E7 Yes* A/T Questions Export E4/E9 Yes* A/T Files List of files E1/E4/E7/E10/ E11 Yes* A/T Files Upload a file E3/E4/E8/E11 Yes* A/T Files Make a folder E1/E11/E15 Yes* A/T Groups Create group E4/E6/E11/E13 Yes* A/T Groups Delete group E4/E11 Yes* A/T Groups Add/Remove users E1/E4 Yes* A/T/S New event New event E11/E13 Yes* A/T/S Export calendar Export calendar E11 Yes* A/T Forums Add / Edit a new topic E1/E4/E11/E13 Yes* A/T Forums Delete topic E4 Yes* A/T Forums Reply E1/E3/E4/E11 Yes* A/T/S Profile Change password E4/E8/E11 Yes* A/T/S Profile Edit profile E4/E8/E11/E13 Yes*
218 219
Chapter 11 The Technological Advancement of LMS Systems and E-Content Software
Dorota Dżega West Pomeranian Business School, Poland
Wiesław Pietruszkiewicz SDART Ltd, UK
ABSTRACT This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomeranian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines.
INTRODUCTION to make conduction of the classes that use the distance-learning methods and techniques pos- The use of e-learning in higher education in sible. Every school that offers this kind of studies Poland is regulated by a set of legal acts. The must have at its disposal a well-trained teaching most important one among them is the order staff prepared to conduct classes by the means of concerning conditions which should be fulfilled blended-learning. Moreover, school must assure:
DOI: 10.4018/978-1-60960-884-2.ch011
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. The Technological Advancement of LMS Systems and E-Content Software
• An access to IT systems and the software used presentation engines. In each section we will which enables the synchronous and asyn- discuss relevant issues and results of observations chronous interaction between students and that shaped the process of LMS evolution. Hence, e-tutors. the chapter delivers information meaningful for • Didactic materials prepared in an electron- the practice of e-learning management. ic version. • The possibility of personal consultations with e-tutor at school. EXPERIENCE • The regular verification of students’ progress, knowledge and skills, including con- The West Pomeranian Business School is one of ducting examinations at the end of each the first universities in Poland, which uses methods course at school. and techniques of distance learning. Chronologi- • The regular verification of tutors’ activity. cally, e-learning experience in West Pomeranian • The trainings for students on the e-learning Business School (ZPSB as a University official platform. abbreviation) were as follows:
The above-mentioned list neither impose on • In 2006 started first blended-learning study, schools what kind of software they have to use, the first LMS was Oracle iLearning™. nor it limits the form of IT systems usage (for • In 2007 LMS was changed from Oracle iL- example: the purchase of the license, the lease, earning to Moodle. etc.). The number of classes conducted with the • In 2008 started own production of e-learn- usage of distance-learning methods and tech- ing content and e-books. niques is limited to 60% of the entire amount of • In 2009 started e-learning postgraduate classes planned for each specialization and level studies and e-learning business courses. of education, which is defined by core curriculum, • In 2010 started e-tutoring in science cours- excluding the practical and laboratory classes. As es (for example: mathematics, statistics, far as the postgraduate studies and trainings are econometrics). concerned, there are not any special legal regulations – it is possible to use e-learning methods The future plans include introduction of virtual without limits. practice. Currently, ZPSB is offering three main The main idea of this chapter is to present the products using the technology of e-learning. These process of evolution for LMS using an example products are presented in Figure 1. of West Pomeranian Business School. The chapter ZPSB e-students may achieve Bachelor degree will explain the requirements for different types and Master degree. During their studies they re- of studies and how they influenced the Learning ceive help in form of e-Tutoring associated with Management Systems. The next part of chapter their courses. After a Bachelor degree or Master will include information about different technolo- degree they can continue study at e-Postgraduate gies and software that were and are currently used studies. The graduate and postgraduate students in the e-learning process. The last part of chapter may take part in e-courses (business-oriented free will introduce SPE (SDART Presentation Engine) courses). Additionally, the traditionally learning being an innovative e-learning presentation engine students may also take a part in the e-courses as developed as an Rich Internet Application by SD- well and they receive an access to e-content, and ART Ltd and this software was designed to over- they may expect help via e-Tutoring. After come the problems observed for the previously graduation students may continue learning at
220 The Technological Advancement of LMS Systems and E-Content Software
Figure 1. e-Learning products offered in West Pomeranian Business School
Master degree or Postgraduate studies. At the of the product. The process of preparing teaching moment in ZPSB resource there are: materials in the form of e-contents generates the highest costs, and is characterized by the highest • e-Content (e-learning content for over 100 workload. The costs of this process are mainly courses) and e-books (over 30 titles). associated with the purchase of the copyright of • e-Postgraduate studies (e-learning content teaching materials, salary of the production team for 18 courses). and the costs of adjustment. • e-Business courses (e-learning content for It is worth noting that the quality of teaching 8 business courses). materials play a very important role in the implementation of the educational process. Educational These resources will be analyzed in the fol- process is the preparation and implementation lowing sections. of the course, during which there is transfer of The preparation and implementation of each knowledge, learning and check of knowledge. At product is a complex operation and requires the this stage, available functionality of LMS is very involvement of specialized personnel and tech- important. The last process is the process associ- nology. Typically, this action is a complex of four ated with providing an adequate level of quality processes: the process of preparing e-learning of individual products. Not only technology and platform, the process of preparing teaching ma- publishing teaching materials are examined, but terials, educational process and quality assurance also the quality of teachers’ work. processes. The process of e-learning platform preparation consists in selecting the appropriate LMS system and configure it according to the needs
221 The Technological Advancement of LMS Systems and E-Content Software
Educational Process in the area of Polish higher education. The full e-learning path is realized in the case of post- The selection of LMS require a prior analysis of the graduate studies and business courses. The final e-learning educational process, to choose a proper part of the preliminary phase is the end-users e-learning products and matching technologies. training of LMS. The e-learning platform trainings The common elements are: the registration pro- are the fixed element of at the beginning of stud- cedure for studies and courses, the documentation ies. They take place in the computer room and of studies (courses) and students’ (participants’) last at least for two hours. Alongside with train- results. Therefore in the overall educational pro- ings done for students, at the beginning of each cess a set of three phases may be distinguished: course special trainings of e-tutors are held – they the preliminary phase, the intermediate and the learn how to conduct e-learning classes and how final one. The Figure 2 presents a detailed analysis to make use of all tools offered by the e-learning of these phases. platform. In the case of the full e-learning path The differentiation of the educational process the training of end-users is through the e-materi- appears in its preliminary phase. At this stage als. blended-learning and full e-learning differs. The The main part of the educational process is blended-learning path is realized for the diploma held in the intermediate phase. In the blended- studies (where students get a Bachelor or Master learning path there is a division of courses into degree) and it is closely dependent on law in force two parts: the one realized in the traditional way
Figure 2. Educational process of blended-learning and e-learning paths
222 The Technological Advancement of LMS Systems and E-Content Software
and the other realized by means of the LMS. This in the field of organization, but also in the area article contains the detailed description of the of used technology and the technologies used by use of the e-learning platform in the educational ZPSB will presented in the following section. system. According to law established in Poland all exams have to take place via the traditional E-Learning Technologies way at school. Such a requirement doesn’t exist in the case of the postgraduate studies and short In this section we will analyze the LMS changes, courses. Thus, in the e-learning path the inter- their reasons and the functional requirements for mediate phase includes learning, finishing the LMS relating to different type of e-learning studies. course through the on-line test or exam and the The first LMS system applied by ZPSB for the preparation of the final thesis. When it comes to online learning was Oracle iLearning platform. the business courses, the intermediate phase only The learning materials have been created with use includes learning and consultations with e-tutor. of various technologies such as Adobe Flash™, The final phase is also diverse in regards to the Adobe FlashPaper™ and PDF creator. Currently extent of the use of e-learning technology in the all the e-learning products are based on Moodle educational process. In the case of blended-learn- LMS and it was configured for the presentation of ing path the personal contacts between student and e-learning materials using Adobe Learning Suite™ his/her thesis’ tutor, as well as diploma exam are packet and SDART Presentation Engine. It must emphasized. In the case of postgraduate studies, be explained why a major technological change on the other hand, the only activity that takes was done and Oracle platform was replaced by place in the final phase is the thesis evaluation. Moodle LMS. Oracle iLearning was a closed The short courses are even more organizationally solution with limited capabilities (comparing simplified and the only thing that takes place in to modules available for Moodle LMS) and due the final phase is final on-line exam through the proprietary character it wasn’t possible to extend LMS with the usage of built-in test module. it by developing additional features adjusting the Taking this description into consideration, it platform to meet the University requirements. is obvious that the blended-learning path is more Hence, Moodle being Open Source software was complicated than the e-learning path as far as the selected. organization is concerned. On the contrary, the full Figure 3 presents the overall process of evo- e-learning path requires more commitment into lution of LMS systems as well as technologies precise and faultless preparation of the resources relating to the production of e-content that took that end-users have at their disposal. The same place in e-learning services offered by ZPSB. observation applies to the e-tutors’ engagement The solution (1) was an archetype for the future into the educational process – the more involved products development. Solutions (2) and (3) are e-tutors are, the more effective blended-learning combined into a hybrid of the new LMS system path is. Along with this commitment comes the and small modifications of the technology produc- necessity of computer knowledge, suitable skills ing e-content. The solution (4) is the most mature for conducting on-line classes, such as the skill of the presented solutions. The LMS system and of motivating students to work by means of the technology producing e-content fulfils all the computer and e-learning platform, the ability of needs and requirements established by West Po- talking with students on chats without losing their meranian Business School. The change of the attention, the regularity in checking of the students’ LMS system was required by the need to enhance progresses. The realization the above-mentioned the functionality of e-learning platform as well educational process requires diversity not only as expansion of the flexibility when modules are
223 The Technological Advancement of LMS Systems and E-Content Software
Figure 3. The development of LMS and e-content
prepared by the developers. The Moodle platform optional and unavailable, depends on the organi- is one of the most popular LMS platforms used zation of studies imposed by the law and internal by the Polish universities. One of the major ben- University regulations. efits of a web platform is that it does not require In case when the e-studies are delivered in any installation steps from its users (students, blend-learning form, each registered LMS user teachers, business participants). An access to the has an access to the central service as well as to platform is fully distributed and delivered via the courses one should attended. The central Internet browsers. Benefiting the modular archi- service contains activities such as: announcements tecture of the Moodle platform, it is possible to from and online chat with the e-Learning Centre adjust its functionality according to the needs of and news section. Each course published on the the bespoke product. The Table 1 contains infor- e-learning platform is divided into different ac- mation about the used functionality of the Moodle tivities (on-line, off-line) relating to the learning platform accordingly to the different types of and communication. courses. The presence of activities i.e. obligatory,
224 The Technological Advancement of LMS Systems and E-Content Software
Table 1. Basic functionality of learning management system on different e-learning products
Activity of e-learning Functionality e-Studies e-Postgraduate e-Business courses „Traditional” studies studies Learning e-learning content obligatory obligatory obligatory obligatory e-books obligatory obligatory obligatory obligatory Additional teaching optional optional optional optional materials Additional course optional unavailable unavailable optional pages Glossary obligatory unavailable unavailable unavailable Knowledge check Quiz optional obligatory obligatory optional Assignments optional unavailable unavailable optional WIKI optional unavailable unavailable optional Games (crosswords, optional unavailable unavailable optional Sudoku) Discussion via forum obligatory optional optional unavailable Communications Chat obligatory unavailable unavailable unavailable Forum optional obligatory obligatory optional eMail (internal email) obligatory obligatory optional optional Tutor’s announcement obligatory obligatory obligatory optional Internal messengers optional optional optional optional
Within the activities relating to the learning Motivation for students can be improved by: it is possible: To deliver lectures with use of: • delivering a moderated discussion forums relating to the subject, where students as • the studies using multimedia teaching ma- well as teachers can follow up a conversa- terials provide for each course in SCORM tion and exchange opinions; form. Additionally users have a fast ac- • delivering the workshops based on the ex- cess to the teaching materials in a printable change of files, where students can assess form (e-books); and mark each other’s work; • design of the additional pages for the • enabling content entries to the dictionary; course, where it is possible to publish ad- • gaming solutions, these are very attrac- ditional teaching materials relating to the tive form testing the basic knowledge or subject; the combined activities such as dictionar- • by building of the dictionary containing ies and tests. The examples of games are keywords for the course. This allows to de- crosswords and Sudoku. fine expressions relating to the course subject, which may be done by the students or To test the learning progress using: teachers (teachers additionally have a right to accept and approve the keywords added • the evaluation of posts made by the students by the students). on discussion forums. Only specifically re-
225 The Technological Advancement of LMS Systems and E-Content Software
lating announcements from the period of es: the central service webpages, the main time may be selected for the evaluation; page of course as well as in the calendars • micro testing relating to a particular sub- of students and lecturers. ject or part of the material, typically tests • Publishing of the lecturers’ announce- cover one module. It is possible to build ments. This is a place where public an- a database of questions (with division into nouncements about main issues can be different categories) for each course and distributed, it is recommended to use it to implement following questions within mainly for the general announcements e.g. the e-learning platform: single questions About the sessions and conditions relating or multiple choice questions, true/false to passing a particular exam. questions, questions with selections of the • Real-time conversations (or leaving a mes- answers (to choose a relation between a sage) using a communicator implemented couple), calculations, essay questions. All on the platform. This is a direct one-to-one the types of questions mentioned above are communication. accompanied with explanations, the vari- • E-mail exchange, within each of the cours- ants of answers or their number of points es it is possible to create internal e-mail and rules to calculate the students’ marks groups. This is a part of the e-learning allowing to formula and the conclusion for platform. students; • it is possible to create a home work and The lecturers as well as the students have mark teamwork using a Wiki mechanism, capability to publish additional materials. The where the articles are build by multiple co- e-learning platform supports text files, spread- operating people; sheets, images, audio and video. A user can choose • it is possible to give online space for the preferred language of interface: Polish, English student, place where generated work can or German. be placed or exchanged. For the postgraduate studies delivered in the e-learning form each registered user has an ac- Finally, an electronic logbook for each course cess to the course published on LMS system. The may be created. Each task, apart of the number of postgraduate students learn the course materials points, can be commented. This is very important individually without any supervision. The tutor for the students to fully understand the course and role is restricted only to consultations and answer- received marks. ing students’ questions. Because the tests at the end As a part of the communication between the of course are delivered in the e-learning form it lecturer and students it is possible to use follow- was required to develop a database of the questions ing features: aimed at logical thinking and ability to link the facts and observations. The main reason for that • Real-time conversation delivered by the was to restrict the effects of repetitive knowledge online chats. This form of communication absorption. That demands the additional efforts is rather restrictive as it’s time-consuming during the tests creation and extended instructions and a minimal length is imposed, accord- in the online help. ing to the number of hours available for the The answers and evaluation of the online course. The information about the time for tests is done fully with use of the LMS System. chats sessions is announced in three plac- Information about the achieved marks is passed
226 The Technological Advancement of LMS Systems and E-Content Software
directly to the students and employees of cus- Inserting math symbols in WYSIWG HTML tomer department. In a very similar manner, the editor is very intuitive and involves a mechanism examination results are delivered after the busi- of drag&drop for items or it could be done by ness e-learning course. inserting the current items at current cursor position. In addition to the graphical equation editor it is also possible to insert any math equation in THE EXTENSION OF LMS the LaTeX form. The developed module for audio/video chat The functionality mentioned in the previous sec- contains three forms of video communication (or tion was not sufficient and herein we will describe audio-only when user does not have a webcam): additional LMS extensions developed to provide the features requested by users. Among the ad- • Individual interview – an interview can ditional LMS modules, which were reported by take place only between two users (P2P users as desired ones were: Equation Editor, Audio/ technology is used). Video Chat and Whiteboard. These modules were • Group discussion – everyone can speak not available for Moodle LMS or their capabili- at any time, hence it is a solution suitable ties did not match the requirements. Therefore, rather for a group with a small number of these three modules were developed by West members. Pomeranian Business School. • Moderated group discussion – the modera- The WYSIWG (What You See Is What You tor has special privilege comparing to the Get) editors available for Moodle did not offer rest of the members and primarily controls equation relating features. The developed Equation entire conversation and gives a chance Editor allows one to construct any mathematical to speak to persons who are waiting in a expression and then insert it into the course content, queue. The main difference between the at any place where it is possible to edit the text group conversation and moderated group using an HTML WYSIWG editor. The developed conversation is that in a moderated conver- module contains an editor, where a following set sation only 2 users may speak simultane- of symbols and operators are used: ously and one of them is the moderator.
• Basic binary operator such as multiplica- Interview audio/video is always initiated in tion, addition, subtraction, equality, mem- a similar way as a text conversation. The Audio/ bership sets, factorial, etc. Video Chat Module has been developed using • Special visual fields: such as neighboring Laszlo Systems OpenLaszlo™ technology. expressions, fractions, roots and exponen- The whiteboard module is a solution designed tials, superscripts, subscripts, equations, to support the cooperation of several users logged matrices, etc. on LMS and using one course. As part of this • Brackets and parentheses: square, triangu- activity it is possible to draw geometric objects lar, etc. and lines with a various thickness and different • Math functions: trigonometric, logarith- colors, to write with a pencil or to edit the text by mic, statistical functions, etc. a special editor. The effects of group work can be • Integrals, boundaries, sums, differentials, saved as an image file and send in e-mails to the etc. cooperating users. • Greek letters. The basic configuration of Moodle does not • Arrows (as binary operators). offer the functionality of global management • Other symbols and accents. of user groups from the main site. It’s possible
227 The Technological Advancement of LMS Systems and E-Content Software
only to manage the groups from the level of each • Course program – „List of content” have course. This situation causes a major obstacle for all the modules including sessions. system administrators requiring a time-consuming • Course introduction – definition of the activity to prepare each of course. Hence, the aims of the course, introduction of the ba- LMS functionality was experimentally extended sic information about the subject, descrip- by introducing an additional feature to Moodle tion of the course entry requirements, in- – superior group. The global group management cluding general knowledge of the student module allows administrator to: and skills from the other courses. • Modules include: an introduction to mod- • Define the superior groups. ule (general explanation about the subject • Define the sub-groups. of the module, explanation what student • Assign users to the sub-groups. will learn during the course), the main part • Assign the sub-groups to the courses. divided into sessions (containing a theoretical and practical part), a subject sum- The further development of LMS functionality mary (conclusions about the main aims of depends on the verified needs requested by the the module). end-users and system administrators. • Dictionary – expressions and definitions used within the teaching materials and ma- Multimedia Teaching Materials terials relating to the subject. • Recommended literature – a list of most The most important part of the e-learning course important literature positions as well as ad- is the e-content as well as e-books embedded in ditional reading suggestions relating to the the LMS system and these elements will be discourse subject. cussed in this section by presenting their structure • Set of questions and tasks as a part of self and process of production, both being the results testing activity, including questionnaires of observations, analyzes and adjustments that (single answers multiple answers), true/ toke place since 2006. That’s why the e-learning false questions, tasks with linked coupled materials creation process should be considered answers, schemas with controllable order as a priority by the teaching organization in the of the answers. activities relating to the online studies. The team • Open questions to control understanding. responsible for the production of the e-learning • Additional materials (presentations, legal materials must be continuously looking for solu- acts, etc). tions allowing saving time and reduced the cost • Preliminary test from the knowledge of producing materials. learned during the course. The multimedia teaching materials are produced in compliance with rules relating to gen- Additionally, the multimedia teaching materi- erating e-content (Horton, 2006) with use of the als contain images, animations and links to the programming packages such as: Adobe Flash™, web pages. The quantity of materials is related to Adobe Learning Suite™, Macromedia Flash Pa- the amount of our hours defined for the course. per™, PDF Creator, SDART Presentation Engine. On average typical course contains 5 modules, Currently all the multimedia teaching materials each module contains multiple sessions. Every were prepared and standardized in the form of: course has on average 140 slides and approx. 20 questions or tasks for self testing. Multimedia teaching materials are not the only source of
228 The Technological Advancement of LMS Systems and E-Content Software
information presenting the subject. In Table 2 a team responsible for the multimedia materials detailed content of materials is presented and this includes: multimedia teacher, proof-reader, mul- list is provided for the authors. timedia experts, computer graphic designer. All The authors of the teaching materials are the multimedia materials, before they are released typically lecturers running the classes. Prior to to the students access, are thoroughly verified: the preparation of e-content all authors ought to the original materials are analyzed by the multi- be instructed. The instruction contains explanation media expert, after corrections they are passed to of the rules and directions for the creation of e- a proof-reader then are tested during the produc- learning materials, screens and their contents. The
Table 2. Checklist for e-content authors
The organizational structure of course Amount of screens Number of screens Course program 1 screen (obligatory) Introdution to course 1 screen (obligatory) Introduction to module 1 1 screen (obligatory) Session 1.1 Session 1.2 25-30 screens for module Module 1 ……….. Self-check short quiz for module 1 4-5 screens Cases and examples for module 1 min. 10 screens Summary of module 1 1 screen (obligatory) Introduction to module 2 1 screen (obligatory) Session 2.1 Session 2.2 25-30 screens for module Module 2 ……….. Self-check short quiz for module 2 4-5 screens Cases and examples for module 2 min. 10 screens Summary of module 2 1 screen (obligatory) Introduction to module … 1 screen (obligatory) Session 3.1 Session 3.2 25-30 screens for module Module … ……….. Self-check short quiz for module … 4-5 screens Cases and examples for module … min. 10 screens Summary of module … 1 screen (obligatory) Summary of course 1 screen (obligatory) Literature 1 screen (obligatory) Glossary min. 10 keywords Trial quiz 10 screens Additional materials filenames Database of on-line exam quiz 30 screens
229 The Technological Advancement of LMS Systems and E-Content Software
tion are finally tested by the authors. The detailed The LMS system is constantly used during the tests are as follow: whole process of e-contents creation. During the Introduction phase: introduction phase, the main features of LMS that are used are ones for communication and publi- • selection of the author; cation of the materials. In the secondary phase, • instruction about the rules and directions the main focus is on the part of system enabling for the creation of the e-learning materials, exchange of the files and communication between screens and its contents; people involved. The file exchange improves • work on the basic version of the scenarios the organization and access to the materials as for the course. well as distribution of the corrected versions in a real-time. The final tests are also run using LMS Secondary phase: and at the end created content is delivered to the users via LMS. • preparation of the materials by the author: One of the most important quality features for ◦◦ preparation of the first module of the the teaching materials is that they shall be up- course, to-date. Delivering the latest/up-to-date content ◦◦ consultation about the first version of requires an additional effort, not only from the module with multimedia teacher, author of materials, but also from all the team ◦◦ corrections for the first version of involved in their production. Hence, it is essential module based on the suggestions from to ensure the efficiency of technology used. The multimedia teacher, leading to devel- efficiency influences both costs and time related opment of the rest of the modules, to the development of course. The materials gener- ◦◦ approval of the materials by the mul- ated with use of the Flash technology and manual timedia teacher, preparation of the documents for the FlashPaper ◦◦ language corrections, as well as e-books is extremely time-consuming. ◦◦ approval of the materials after lan- For the post graduate and short courses it was re- guage corrections, quired to prepare materials only in the e-learning ◦◦ approval of the materials to form with a minimum input from the tutor into the production. teaching process. It was essential to find a more • production of the materials: efficient technology. The selected technology was ◦◦ preparation of the course contents, SDART Presentation Engine. Its functionality will ◦◦ preparation of the tests, be presented in one of the following sections of ◦◦ preparation of the graphics and this chapter. animations, ◦◦ preparation of the e-books, ◦◦ elements are joined together, QUALITY ASSURANCE FOR ◦◦ preparation of the packet SCORM, TRAINING PROCESS ◦◦ beta tests. The essential activity of each business process is The final phase: the quality assurance. This section will present this aspect of e-learning management by explain- • tests done by the author; ing the guideline used to provide the high quality • corrections of the materials; of educational services. The evaluation of the • deployment of the materials into LMS. students’ activity is regularly made by e-tutors
230 The Technological Advancement of LMS Systems and E-Content Software
or e-Learning Centre staff. The reports generated Stage 2 by the LMS seem to be enough for this purpose. The Moodle platform delivers detailed informa- The evaluation of the courses and e-tutors – every tion about course logs (e.g. visited places, time element gets marks from 1 to 5. The frequency of entering each element of the course). It also of the opinion survey on courses and e-tutors: 4 allows filtering of the logs according to the activ- times during an academic year. The structure of ity within the course (for example: the number of form is as follows: posts added on the forum, the number of browsing e-content, the number of browsing the additional Part 1: The Course’s Essential materials, the number of quiz attempts, the number and Formal Evaluation of sent files, etc.). In the case of students such information is sufficient. However, the observa- • To what extent did classes meet your rep- tion of e-tutors’ behavior is a more difficult task. resentations and expectations towards the This difficulty lies in limited possibilities of the course? e-tutors’ work evaluation only on the basis of the • Do you think the presented material is reports generated by the LMS. A detailed survey of clear and comprehensible? the e-learning quality was conducted by (Swedish • Do you think the presented material is up- National Agency for Higher Education, 2008). to date? To assure the training quality the three staged • Do you find yourself active on the e-learn- opinion survey was introduced. Students fill in the ing platform during the course? forms either in the electronic version published in • How do you assess your cooperation with the LMS or in the traditional way (on paper) - of other students during the course? students’ choice. Part 2: The E-Tutor’s Evaluation Stage 1 • How do you assess the e-tutor’s commit- The survey carried out among first-year students ment while conducting the course on the - this one is usually a sort of introduction to e- e-learning platform? learning platform training. We can find following • How do you assess the regularity of con- questions there: tacts between the e-tutor and the students – the e-tutor’s availability during the con- • How did you know about our studies? sultation hours? • What factors proved to be crucial for you • How do you assess the regularity of con- while enrolling in e-learning studies? tacts between the e-tutor and the students • What are your representations and expec- (out of the consultation hours) – for ex- tations concerning these studies? ample the waiting time for the answer on • What are, in your opinion, the advantages the forum? of e-learning studies in the comparison • How you assess the e-tutor’s activities with traditional studies? oriented to make students work systemati- • Do you think studies using the Internet like cally (does s/he lead the discussions on the this are the future? forum and chats, does s/he use the group- work tools)? • To what extent does the tutor make use of the tools available on the e-learning plat-
231 The Technological Advancement of LMS Systems and E-Content Software
form, excluding forums and chats (for education path and the organizational aspect. The example dictionaries, quizzes and on-line students may pass filled survey forms either in the exercises, added files, the presentation of traditional way or by e-mail. The processed results websites)? are passed to students, e-tutors and the school authorities as a feedback information. Part 3: Additional Remarks To incorporate the other information, important in the quality assurance process, it was planned to What Information Would You Like To Pass To The verify the results of questionnaires with the prog- E-Tutor, That Is To Say What Is Praiseworthy And ress of learning. This could protect against unfair What Should Be Necessarily Changed? opinions and problems reported by students having problems with a particular course. Moreover, to Stage 3 fill an informational gap also tutors will be questioned. This process will be done automatically The survey on the functionality of the e-learning via Decision Support System, which in this case platform and the activity of the e-Learning Cen- will be supporting the e-learning management, tre. The following elements are the subject of particularly quality assurance. the survey: The system of the opinion survey adopted herein delivers its basics functions. However, when • The main advantages and disadvantages of it comes to the questionnaire survey one ought to the ZPSB e-learning platform, e.g. the intu- make allowance for the students’ subjectivity in itiveness, the selection of the activities, the their attitude toward the courses and the e-tutors. failure rate. Please point out the elements Therefore, at present the research is being carried that are praiseworthy and those that should out on the creation of the e-tutors’ evaluation be improved. Please point out the elements mechanism which would use the methods of the that you find difficult to understand or mas- artificial intelligence. Data used in this solution ter. Please point out the places/activities/ originate mostly from the LMS. elements that could be more intuitive in Comparing the teachers to students it must your opinion. be noted that is difficult to evaluate tutors’ per- • The main advantages and disadvantages formance. In our other researches we proposed of the interface, e.g. the color combina- and explained how data mining can be used to tion of the user interface, the legibility of observe teachers and to support the management the graphic elements, the way of text parts’ of e-learning platform (Pietruszkiewicz, &Dżega, presentation and functionality of multime- 2010) and (Dżega, &Pietruszkiewicz, 2010). This dia didactic material, e.g. is the navigation process contained five major steps: in the material intuitive enough?. • The main advantages and disadvantages • Generation of reports with events regis- of the dealing with students’ problems and tered for users. matters. This evaluation refers to the qual- • Conversion of events sets to time series. ity of service provided by the e-Learning • Calculation of activity attributes. Centre staff. • Grouping users with a similar activity attributes. Thanks to this three-staged evaluation system • Identification of users with ‘active’, ‘mod- the complex monitoring of training process is erate’ and ‘passive’ profiles. possible, the one that takes into account both the
232 The Technological Advancement of LMS Systems and E-Content Software
In the mentioned article we have proposed The first step of SPE development was a the measures of activity and tested in for Moodle technological analysis that was required to find LMS, that were used by data mining algorithms to an appropriate technological solution for materi- group users with similar activity profiles. In our als’ construction that should allow to keep their opinion the LMS should have more data processing high visual quality and reduce or even remove the and decision supporting features as the increas- problems mentioned above. Therefore, SDART ing complexity of LMS causes that this process decided to develop Presentation Engine as RIA is harder to observe and manage using standard (Rich Internet Application) and keep the courses functionality offered by nowadays LMS systems. content in a form of styled XML-based files. In this form the software layer was separated from Novel Approach to presentation layer and it was planned that update Presentation of E-Content of presentation would be easy, fast and possible using any XML editor. As it was presented in the previous sections, The RIA was selected as this hybrid approach ZPSB used various software solutions to present allows to use rich features of web pages – com- the e-learning courses, however they caused some paring to traditional desktop applications, having problems that needed to be addressed during the also the functionality of applications – comparing latest stage of e-learning technological upgrade – to ordinary web pages. Therefore RIA is a hybrid SPE e-content presentation software described in solution that: this section. The first issue was the time-consuming modifications of courses that involved edition of • Has a simple installation or even usually graphical files containing screens used in courses. does not require any installation steps. Due to it, the modifications couldn’t be done with- • Is safe to use, as the service is less vulner- out an image editor or Flash authoring tool and able to threats than the software requiring time required to create or correct a single screen installation. was expected to be reduced. Easy modifications • Is easily available everywhere and at any- are essential as typing errors or material updates time for all users, only if they have an ac- must be frequently done to maintain the quality cess to a machine connected to internet. of courses. • Is a universal solution being hardware or The other important problem was a graphical operating system independent. representation of text on screens effecting their • Could be easily updated as all changes are size. While this wasn’t decreasing the visual immediately available for all users. quality of presentation but caused that search • Is more ergonomic than web pages, as most functionality was impossible. Hence, users had task could be done on client-side without to navigate through materials using menu with- constant reloads of web pages. out a possibility to look for a particular phrase • Has graphical user interface (GUI) more interesting them. Another important problem was attractive than a standard HTML-based insufficient mechanism of zooming, while this service. feature is necessary to make the materials avail- • Is easier to develop than pure HTML- able to people with the sight problems. The last based service. important issue related to an expected improve- • Might process the data, present them in ment of e-learning courses production All of these various forms and conduct an interaction problems were analyzed and solved during design with user. and development of SDART Presentation Engine (SPE later) that was created by SDART Ltd.
233 The Technological Advancement of LMS Systems and E-Content Software
However, RIA has some disadvantages and We have established a list of the most important their particular effects on a developed application quality factors and were looking for a technology must be analyzed. Thus, we had to remember RIA: that would:
• Has additional requirements for software • Have a high speed and fast start-up time. i.e. plug-ins or frameworks. • Offer multi-platform support. • Run with a lower speed than desktop ap- • Have prototyping capabilities. plications, as their code is interpreted and • Not require any unpopular plug-in. converted into a native code on client • Make it possible to be developed by Rapid machine. Application Development tools. • Forbid access to critical resources to main- • Have easy customized GUI components. tain security. • Offer attractive GUI. • Are fully dependent on server and network, any problem with speed or stability For this particular project, Adobe Flex™ of them causes problems with RIA. technology was selected as the best matching requirements for e-learning presentation engine, More information about RIA quality was especially as at runtime it requires Flash plug-in presented in (Pietruszkiewicz, &Dżega, 2009). being the most popular web plug-in, has advanced The mentioned advantages and disadvantages RAD (Rapid Application Development) tools were analyzed and addressed during RIA technol- (Flex Builder and Flex plug-in for Visual Studio) ogy selection and SPE design. The technology and GUI components are easily customized. selection is an essential step for each software project as the selected technology defines the Production Process shape of developed software. Currently there for SPE Software exists several RIA technologies on market – Oracle Java™, Adobe Flash™, AJAX, Adobe As it was mentioned it the previous sections, the Flex™, Laszlo Systems OpenLaszlo™, Oracle production of e-learning content involves a few JavaFX™ and Microsoft Silverlight™ being the steps. For the e-learning materials being produced most popular ones. using SPE these steps are (Figure 4):
Figure 4. The stages of production for SDART Presentation Engine
234 The Technological Advancement of LMS Systems and E-Content Software
• Scenarios production – provided by the au- and deployed also might be simply done by a re- thors constitute a base of e-learning edition of XML file. This is a very easy process production. and neither SCORM have to be re-packaged • Parallel production of media files and (XML replacement in archive is fine) nor any XML – at this step scenarios are converted other file apart of XML must be changed. The to XML and media files (usually graphics) low maintenance effort for e-learning courses are prepared from sketches or descriptions was also a important target to achieve during SPE provided by authors. design&development. • XML testing and adjustments – involve joining of XML with media files and their Effects of SPE Deployment testing, some elements on slides may be and Requirements presented in different forms e.g. large images that do not fit into a screen together The critical analysis of solutions used by ZPSB with text could be changed into a thumb- done during SPE development resulted in a soft- nail button showing them on a separate ware having a few distinguishable advantages popup window only via simple change of that will be analyzed in this section. First of all, it one image attribute in XML configuration does not require sophisticated software to create file. e-learning courses or to modify them. The main • PDF generation – this step is done automat- course part i.e. XML configuration file may be ically by a SPEEG program being a part of edited by any XML editor or even standard text SPE (the advantages of this program will editor (practically a syntax coloring is sufficient). be explained in the following section) that As the courses may use various media files i.e. using XML and media files produces PDF JPG, PNG or SWF they also may be produced by e-book. many freely available graphical tools. • SCORM packaging – it is the last of stage If we consider XML itself, it might said that of production and involves creation of this form of presentation is human readable and SCORM metafiles and SCORM compres- could be easily understood by the staff produc- sion to a ZIP archive. ing e-learning courses. It is also easy to construct these files as XML schema might be filled by text After the production SCORM package might copied from the scenarios provided by authors. be deployed on LMS. Moreover, commonly to all XML files, the content The most frequent changes to e-learning is separated from the formatting. It is possible content that need to be done during materials pro- to give a special formatting to the elements of duction involve only edition of XML file. It must presentation but the basic formatting (font, styles, be noted that usually, for the other presentation sizes) is provided by engine according the mean- engines, the production must be fully synchronized ing of these elements. and text and images cannot be prepared paral- Additionally, the courses created with SPE are lelly. This is a problem when images are created easy to modify. In the result they may also be easily by other person than the one preparing XML. In updated and re-deployed. This feature is impor- SPE media materials can be embedded in differ- tant during evaluation of courses as small typing ent ways, hence it is possible to decide about it errors are a common situation and presentation during testing&adjustments phase. mechanism should allow to easily correct them The changes of course (typos, formatting, up- and re-deploy new versions of materials. Using date etc.) that occur after SCORM was produced SPE courses might be re-edited by modifying
235 The Technological Advancement of LMS Systems and E-Content Software
XML and replacing them in SCORM packages. well as a basic one from many cost-free editors Analyzing time require to produce the e-learning available or even a text editor with syntax coloring courses with the previous presentation engines it (a popular solution among the web-programmers). was calculated that for ZPSB case SPE reduced Another required tool is a graphical editor or SWF time of production by 70%. Further, the process producer. SPE uses bitmap images or vector clips of modification was also significantly reduced. in SWF format. Bitmap images might be produced The generation of files is easy to learn as to by many editors including free editors like e.g. format text HTML tags may be used. From the popular GIMP, while SWF might be produced by functional point of view a textual representation Flash authoring tools e.g. Adobe Flash™ IDE, of course and its visual generation by an RIA SWiSH or exported from OpenOffice suite. engine may be extended by search and zooming The high cost of software to create e-learning features. This first feature means that users may courses is a common problem of e-learning author- look for materials interesting them, while the ing software. SPE does not have such requirements second one is important for people that does not because this issue was considered during the its see standard fonts clearly. SPE was designed to design and development and it was conducted not avoid a common mistake of RIA i.e. fixation of only to provide a high functionality of SPE but fonts’ size that causes problems of people with also to make e-learning production faster, easier sight problems. and less expensive. SPE also was build in a way ensuring that The runtime requirements are practically en- different types of materials might be embedded. sured by each desktop computer. It must provide They include images as well as animations, audio a web browser equipped with a Flash plug-in, and video files. Using SWF files it is also possible which is available for all major browser. SPE is to use interactive elements build for a particular also hardware and Operating System independent, presentation e.g. games or puzzles. therefore may be used on any modern computer. The additional part of each course is its PDF e-book being preferred in some situation by users SPE Functionality and Structure e.g. by users wanting to print the materials or to read them on devices without Internet connection On Figure 5 a SPE screenshot and e-book were (during the journey). SPE was designed to generate shown. Both were taken from the same XML these files automatically by SPEEG (SPE E-book course file, while screen version is generated on- Generator) being an application converting XML line by SPE each time users watches slides and course files into PDF e-books. Comparing to the e-book was generated off-line and added to the previous presentation engines used by ZPSB, learning materials. For the obvious reasons PDF which required a manual generation of PDF files, does not contain any interactive elements and SPEPG does it in a few minutes. primarily can be used to read or printing materials. The technical requirements to use SPE could The main form of e-learning course are interactive be divided into two groups i.e. those relating to SPE materials available via web browser. the content production and those required to run The online materials presented on the screen- e-learning course. Considering the first group, shot 5a. reveal a few parts of application: the production of e-learning courses using SPE does not require any unpopular or costly software • Navigation section 1– allowing users to tools. It might be done using standard and freely move forward and backward. available software. The necessary tool is any XML editor. It might be a sophisticated XML editor as
236 The Technological Advancement of LMS Systems and E-Content Software
Figure 5. The example of SPE-based e-learning materials – online course (a) and offline e-book (b)
237 The Technological Advancement of LMS Systems and E-Content Software
• Course structure 2– this easy accessible clickable course structure. The third one searches course structure allows users to jump di- for user-defined phrase and lead user to the found rectly to a desired section. screens. All of these types of navigation are re- • Main presentation area 3 – containing the quired for different kinds of users or even the same presentation i.e. text, graphics or interac- users taking the course at various moments. Some tive elements, this area is also used to show users prefer learning courses from the first to last the test questions. screen, the others learn courses during multiple • Search window 4a or search field 4b with short sessions and there also exists users interested the search results – the search field is al- only in screens with information relating to their ways visible, while the search window ap- demand (search phrase). pears after search was started and contains During runtime SPE loads XML course file, navigable list of screens for which a phrase while images are load on demand i.e. only if they looked for was found, this window keeps are required by a slide presented to user. Consider- to be open until user decided to close it, ing the low size of vector graphics (primarily used hence it is possible to perform a fast verifi- on screens) it effect the start-up time of course cation of found screens without the neces- which is very fast and necessary files are also sity to re-run searching multiple times. downloaded shortly. Comparing to the systems • Zooming panel 5 – allowing to increase/ with a graphical text representation, the method decrease font size. used by SPE is faster as text contained in XML is • Test exam 6 – is a short exam with inter- available immediately, thus user does not observe active tests with a summary at the end of any reload gaps between the screens. exam. Figure 6 contains two subfigures with UML diagrams relating to the SPE functionality (a) and The structure of SPE presentation window was structure (b). As presented on use case diagram designed to fulfill the most fundamental software SPE supports the functionality for e-learning quality factors relating to the user’s experience courses including: with the software i.e.: • Navigation – user may navigate through • Visual simplicity – users should not be screens (next, previous) or go to a particu- forced read a manual to use software, if it lar section. is not really required. • Presentation – information stored in e- • Intuitiveness – design software in an intui- learning content are presented by SPE on tive way. screens. • Ergonomic – make the functions easily • Knowledge testing – done using test accessible. screens during the learning progress, al- • Functionality – ensure that available func- lowing users to examine correct answers, tions will be the ones required by user. the other way is an preliminary tests evaluating users’ answers and returning a sum- The different ways of course navigation are mary without information about correct possible in SPE. The first one – a full navigation ones. assumes that user will read screen after screen and • Search – user may search for a particular is available using standard ‘next’ and ‘previous’ phrases and go to found screens. buttons. The second one allows users to navigate • Zooming – changing presentation by ad- directly to a selected section and is done using justing the font size for presented material.
238 The Technological Advancement of LMS Systems and E-Content Software
Figure 6. Use case (a) and deployment (b) UML diagrams for SPE
These features were observed or reported by for developers but form user’s perspective it ir- users as the most useful ones and required to fully relevant, because SPE works as compiled engine use e-learning materials. that could be deployed on any server or client. For The second diagram on Figure 6 explains how users the more interesting technologies are ones SPE-based course might be deployed and run from relating to the production of e-learning content. LMS. ZPSB uses SPE on Moodle LMS but as The basic information about e-learning structure this software is LMS-independent, SPE might be are represented by a XML file used by SPE (its used also with other LMS. The only requirement structure and capabilities will discussed in the next for LMS is that such system must use SCORM section). HTML was used as a formatting language packaged content, which an obvious requirement for text. It is important feature, because HTML for any mature LMS. is a very popular language and its knowledge SPE uses popular web technologies in a way is belongs to the basic skill of people working that ensures high quality of e-learning courses with web technologies. Thus, materials might be and keeping their development simple and im- constructed even by a beginning users even with mediately understandable to all people with any a limited web development experience. The flash web-development experience. A proprietary and technology is used at run-time by SPE as this rare solutions are significant problem for vari- Flex application was compiled into a Flash code ous e-learning systems, that too often reject the and according to surveys Flash reaches 99% of developers’ experience. Internet enabled desktops (June 2010). Due to it The technologies used by SPE are: Flex, XML, SPE might be used with a greater confidence than HTML, Flash and standard multimedia files. As any presentation engines using different run-time it was mentioned earlier Flex is RIA technology technologies as they are less popular or isn’t a based on MXML language., while MXML is a homogeneous on various browsers or Operating XML application mixed with ActionScript pro- Systems as Flash is. However, in the future Flash gramming language (more generally ECMAS- superiority might change as JavaScript is promoted cript). However, it is important technical feature as a dominant part of HTML 5 but considering
239 The Technological Advancement of LMS Systems and E-Content Software
the past experience with web standardization it might even take a few years until all browsers
Text with popup element 1 nologies used by SPE include raster graphics and ...
PDF generated automatically from XML-based Another line of text with popup element 2 is a support for graphic hardware resulting in a ...
high speed of visual effects like glowing, blurring,
shades etc which were used in SPE and positively Popup 1 text influence users’ web experience. To explain the advantages of RIA-based e- Popup 2 text learning presentation engine using XML as a representation for courses we will briefly analyze features of XML application used by SPE. There are four main types of screens: The third group uses screens like one below:
• Textual screens with or without multimedia materials e.g. course content screen, A question to answer introductions and typical content screen. • Interactive presenting screens with pop- self-check knowledge. An example of the first type of screen is below: An example of screen from the last group is below:
Text example ¹:
bullet point 1,
Explanation
bullet point 2,
of element 1
bullet point 3 with italics Explanation style.
of element 2
... And later bolded text
1. Author Title, As it can be seen screens have an easy to Publisher ... understand structure and are very informative (a typical characteristic of tagged languages). Ad- ditionally, the content may include a subset of For the second group of screens an example was HTML formatting tags being the most popular provided below: and important ones e.g. italics, bold, anchors, breaks or fonts. Using even a simple text editor
240 The Technological Advancement of LMS Systems and E-Content Software
with a syntax coloring the person editing course extended by various interactive clips enlarging its may understand parts of the presentation and fill interactive capabilities and engaging users more slides templates. The textual course representa- with the courses. tion is approx. ten times smaller than the size of At the moment it is possible to adopt SPE to a bitmap converted text. new visual style via replacement of background Considering the future extensions, the XML- images and this feature is planned to be further based representation of courses is also safer extended and introduced in the configuration part solution as this is an universal form that might of XML file. Hence, SPE will be even more easy be authored by various tools as well as processed to adapt to a new style of curses. by different programming languages. Even in Another important feature of new generation situation when SPE will be rewritten using a e-learning courses is intelligence. Therefore, new coming technologies (e.g. HTML5) it will SPE will use Artificial Intelligence methods to be possible to use old XML files or update XML analyze the progress of learning and recommend courses to a new versions schema via conversion the important materials for user e.g. materials that scripts. This problem should be remembered by could be forgotten be user from the last time he/ each organization deploying a new software as she was learning or materials relevant to semantic the data usually live longer than the software search not only matching given keywords. AI will that was used to create it. Moreover, XML isn’t a be also used to analyze users’ behavior during the proprietary format and there are several technolo- usage of presentations and to improve courses by gies extending it e.g. search language XPath or emphasizing the important parts or restructuring transformation language XSLT. the courses. The experiments over adaptive e- learning process were performed by (Liu, &Yang, SPE Further Development 2005) proposing APeLS system, by (Rossi, 2009) proposing e-learning joined with Web 3.0 or using Considering the features presented in the last sec- SCO approach as it was presented by (Brkovic, tion it is possible to say that SPE-based e-learning Milosevic, &Krneta 2006). courses, incorporating different technologies, are The last possible area of further development synergic examples of a modern web application are increased features for handicapped people. featuring the important aspect relating to the qual- They will include a voice synthesizer and in- ity of web services i.e. flexibility, easy modifica- creased visual accessibility e.g. via switchable tion, universality or not limiting the future usage high contrast. The analysis of e-learning environ- of generated data. ment from the perspective of students with sight However, in the future SDART Presentation disabilities was done by (Drigas, Koukianakis, Engine will be further developed by the additional &Papagerasimou 2006). features, giving SPE even more functionality. It is planned to add new screens, especially more interactive test slides allowing users to self evalu- CONCLUSION ate learned knowledge. The embedded multimedia materials might be currently included in SPE as The technical advancement and changes in the life SWF files, but it is planned to introduce new tags style increase the popularity and significance of that will be used to represent audio and video files. methods and techniques of distance education. As As SPE could be used for various e-learning tasks e-learning becomes more popular also the demands i.e. in the higher education, corporate education and requirements relating to it are changing. The or government educational projects, SPE will be users expect courses to be tailored to their needs
241 The Technological Advancement of LMS Systems and E-Content Software
and IT services supporting their process of learn- mechanisms, supporting users in the process of ing. Therefore LMS software must be adjusted to learning, being a move towards Web 3.0 applied fulfill users’ expectation. These features relate in e-learning; and by using technologies making to users’ perspective, but e-learning is also very e-learning even more available and convenient, demanding activity form the business perspec- moving it from the desktop computers or laptops tive. For the universities the production process to mobile devices. Both of these areas are inves- of e-learning materials becomes more and more tigation by ZPSB and SDART Ltd. complex, time and resources consuming task, therefore it is essential to use software and technologies offering the high quality for produced REFERENCES materials, keeping the production process to be simple and not expensive. Brkovic, M., Milosevic, D., & Krneta, R. (2006). This chapter presented the process of evolution SCos for adaptive e-learning. 28th International for LMS and e-content software used by West Conference on Information Technology Interfaces Pomeranian Business School. At the beginning (pp. 29-34). Cavtat/Dubrovnik. chapter explained the requirements for different Drigas, A., Koukianakis, L., & Papagerasimou, types of studies and how they influenced the shape Y. (2006). An e-learning environment for non- of LMS systems. Later, the chapter analyzed dif- traditional students with sight disabilities. 36th ferent technologies and software that were used in Annual Frontiers in Education Conference, San the e-learning process. In its last part the chapter Diego, CA (pp. 23-27). presented SPE e-content software being an innovative e-learning presentation engine developed in Dżega, D., & Pietruszkiewicz, W. (2010). The form of Rich Internet Application by SDART Ltd support of e-learning platform management by The content of each section included observations the extraction of activity features and clustering and obstacles encountered at different stages of based observation of users. In S. Konstantopoulos, LMS deployment and evolution. S. Perantonis, V. Karkaletsis, C. Spyropoulos, & The material presented herein does not consti- G. Vouros (Eds.), Artificial Intelligence: Theories, tute the final step of LMS evolution as we started Model and Applications-SETN 2010 (pp. 315- the investigation over the further areas of LMS 320). LNAI 6040. Heidelberg, Berlin, Germany: extensions. It is planned to develop a new version Springer-Verlag. of presentation engine SPE e.g. to incorporate ar- Horton, W. (2006). E-learning by design. San tificial intelligence features to make the e-learning Francisco, CA: John Wiley & Sons, Inc. content self-adjustable. The AI features are also planned to be used in the module supporting the Liu, H., & Yang, M. (2005). QoL guaranteed ad- management of LMS, as the complexity of this aptation and personalization in e-learning systems. process i.e. number of users, observable activi- IEEE Transactions on Education, 48(4), 676–687. ties and their character, make this process hard doi:10.1109/TE.2005.858398 to perform without a supporting software. This Pietruszkiewicz, W., & Dżega, D. (2009). The module will preprocesses the data, search for the practical aspects of rich Internet application usage patterns and deliver easily understandable development and quality factors: RIA–based comments. decision support system. Quality assessment in In our opinion the further development of Web 2009 (Vol. 561). CEUR. Learning Management Systems should and will be done in two major directions – by using intelligent
242 The Technological Advancement of LMS Systems and E-Content Software
Pietruszkiewicz, W., & Dżega, D. (2010). An Boticario, J. G., & Santos, O. C. (2007). An open application of data mining in the management of IMS-based user modelling approach for devel- e-learning platform . In Korczak, J. (Ed.), Business oping adaptive learning management systems. informatics. Data mining and business intelligence Journal of Interactive Media in Education, 2007 (pp. 60–70). Wrocław, PA: Publishing House of (2). Retrieved on August 12, 2010, from http:// Wrocław University of Economics. citeseerx.ist.psu.edu/viewdoc/download?doi=10 .1.1.101.3799&rep=rep1&type=pdf Rossi, P. G. (2009). Learning environment with elements of artificial intelligence. Journal of E- Carliner, S., & Shank, P. (2008). The e-learning learning and Knowledge Society - English Version, handbook: A comprehensive guide to online learn- 5(1), 193-199. ing. San Francisco, CA: John Willey & Sons, Inc. Swedish National Agency for Higher Education. Cavus, N. (2010). The evaluation of learning man- (2008). E-learning quality: Aspects and criteria agement systems using an artificial intelligence for evaluation of e-learning in higher education. fuzzy logic algorithm. Advances in Engineering Report 2008:11 R, PA: Swedish National Agency Software, 41(2), 248–254. doi:10.1016/j.adveng- for Higher Education, Stockholm. Retrieved on soft.2009.07.009 July 30, 2010, from http://www.hsv.se/downloa Clark, R. C., & Mayer, R. E. (2008). E-learning d/18.8f0e4c9119e2b4a60c800028057/0811R.pdf and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Willey & Sons, Inc. ADDITIONAL READING Cole, J. (2008). Using Moodle: Teaching with the Allen, M. W. (2007). Designing successful e- popular open source course management system. learning, Michael Allen’s online learning library: Sebastopol, CA: O’Reilly Media, Inc. Forget what you know about instructional design Ćukušić, M., Alfirević, N., & Granić, A. (2010). and do something interesting. San Francisco, CA: E-learning process management and the e-learning John Willey & Sons, Inc. performance: Results of a European empirical Berggren, A., Burgos, D., & Fontana, J. M. (2005). study. Computers & Education, 55(2), 554–565. Practical and pedagogical issues for teacher adop- doi:10.1016/j.compedu.2010.02.017 tion of IMS learning design standards in Moodle Del Puerto Paule Ruiz, M., Jesús Fernández LMS. Journal of Interactive Media in Education, Díaz, M., & Ortín Sole, F. (2008). Adaptation in 2005(02). current e-learning systems. Computer Standards Bersin, J. (2004). The blended learning book: & Interfaces, 30(1-2), 62–70. doi:10.1016/j. Best practices, proven methodologies, and les- csi.2007.07.006 sons learned. San Francisco, CA: John Willey Gaeta, M., Orciuoli, F., & Ritrovato, P. (2009). & Sons, Inc. Advanced ontology management system for Black, E. W., Dawson, K., & Priem, J. (2008). personalised e-Learning. Knowledge-Based Data for free: Using LMS activity logs to measure Systems, 22(4), 292–301. doi:10.1016/j.kno- community in online courses. The Internet and sys.2009.01.006 Higher Education, 11(2), 65–70. doi:10.1016/j. iheduc.2008.03.002
243 The Technological Advancement of LMS Systems and E-Content Software
Hannon, J., & D’Netto, B. (2007). Cultural Romero, C., Ventura, S., & García, E. (2008). diversity online: student engagement with Data mining in course management systems: learning technologies. International Journal Moodle case study and tutorial. Computers of Educational Management, 21(5), 418–432. & Education, 51(1), 368–384. doi:10.1016/j. doi:10.1108/09513540710760192 compedu.2007.05.016 Horton, W., & Horton, K. (2003). E-learning Tools Rosenberg, M. J. (2001). E-Learning: Strategies and Technologies: A consumer’s guide for trainers, for Delivering Knowledge in the Digital Age. New teachers, educators, and instructional designers. York, NY: McGraw-Hill. Indianapolis: William Horton Consulting, Inc. Shen, R., Han, P., Yang, F., & Yang, Q. (2003). Kritikou, Y., Demestichas, P., & Adamopoulou, Data mining and case-based reasoning for dis- E. (2008). User Profile Modeling in the context tance learning . Journal of Distance Education of web-based learning management systems. Technologies, 3(1). Journal of Network and Computer Applications, Tang, T.Y., & McCall,a G. (2002), Student mod- 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 elling for a web-based learning environment: A Lewis, B. A., MacEntee, V. M., DeLaCruz, S., et al. data mining approach, [in:] Eighteenth National (2005). Learning Management Systems Compari- Conference on Artificial Intelligence, American son. Proceedings of the 2005 Informing Science Association for Artificial Intelligence, Menlo Park. and IT Education Joint Conference. Retrieved Watson, W. R., & Watson, S. L. (2007). An Ar- August 02, 2010, from http://www.ultimedia. gument for Clarity: What are Learning Manage- co.uk/upload/E-Learning%20and%20LMS%20 ment Systems, What are They Not, and What Comparisons/Learning%20Management%20 Should They Become? TechTrends, 51(2), 28–34. Systems%20Comparison%20P03f55Lewis.pdf doi:10.1007/s11528-007-0023-y Oztekin, A., Kong, Z. J., & Uysal, O. (2010). Zajac, M. (2009). Using learning styles UseLearn: A novel checklist and usability evalu- to personalize online learning. Campus- ation method for eLearning systems by criticality Wide Information Systems, 26(3), 256–265. metric analysis. International Journal of Indus- doi:10.1108/10650740910967410 trial Ergonomics, 40(4), 455–469. doi:10.1016/j. ergon.2010.04.001 Rice, W. (2007). Moodle Teaching Techniques: KEY TERMS AND DEFINITIONS Creative Ways to Use Moodle for Constructing Online Learning Solutions. Birmingham: Packt Blended Learning: A mixed form of learning Publishing. where traditional learning is used together with Rice, W. (2008). Moodle 1.9 E-Learning Course e-learning form. Development: A complete guide to successful E-Content: A set of materials containing learning using Moodle. Birmingham: Packt course information and exercises. Publishing. LMS: Software system used to manage the process of learning.
244 The Technological Advancement of LMS Systems and E-Content Software
RIA: Web-based application with advanced SPE: SDART Presentation Engine being a interactivity and rich visual style. software used to present e-content. SCORM: A common standard of data storage XML: A tagged language used to represent data used to represent universal courses in e-learning structures with logical meaning of its elements. platforms.
245 Section 4 Case Studies 247
Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education
Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal
Ana Cláudia Rodrigues NID-RH, ESEIG, Portugal
Carla Teixeira Lopes Faculdade de Engenharia da Universidade do Porto, Portugal
Paulo Coelho de Oliveira ISEP, Portugal
Ricardo Queirós KMILT, ESEIG, Portugal
Mário Pinto KMILT, ESEIG, Portugal
ABSTRACT The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies.
DOI: 10.4018/978-1-60960-884-2.ch012
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Differences in Internet and LMS Usage
INTRODUCTION university: Australia (Kennedy, Dalgarno, Sturt, Bennett, Gray, Waycott, Judd, Bishop, Maton, In recent years, Learning Management Systems Krause & Chang, 2009), United Kingdom (Conole, (LMS) has been assumed as the cornerstone of Laat, Dillon & Darby, 2006), USA (Walker and the learning environments wide spread in the Jorn, 2007). majority of the academic institutions. The central Since these LMS platforms present oppor- role occupied by the LMS in a typical architecture tunities previously unavailable for academic of a learning environment forces us to study its institutions to apply their financial, logistical, and structure and its use by all users, whether they instructional resources (Uzunboylu, Ozdamli & are teachers or students, in the teaching/learning Ozcinar, 2006), knowing the students perceptions process. will allow, not only to enhance Polytechnic of In order to improve the LMS usage as a tool Porto teachers’ perceptions of what has the most to support teaching and learning in an effective impact on their students, but also, to define poli- way, and since LMS are delivered by Internet, it cies regarding eLearning platforms as a support is important to know the Internet students’ usage to traditional education delivery and understand habits, concerning the Internet and LMS usage. needs to be improved to undertake on distance This information will allow us to adopt the most and blended learning courses offers. appropriate learning and pedagogical strategies, The remainder of this paper is organized as according the students’ profile, characteristics and follows. Section 2 traces the evolution of the preferences. It is also relevant to know whether eLearning systems, highlighting the LMS. In these habits of Internet and LMS usage are in line the following section we begin by describing the with the following studies presented in this section. research methodology used for the study such as This study aims to characterize higher educa- data collection and population and sample. Then, tion students’ of Polytechnic of Porto regarding we present the data analysis and discussion. The their behavior on the Internet and eLearning last section focuses on the main contributions of platform, and it seeks to investigate how students this work, more precisely, the results of this study use the information and communication technolo- and a perspective of future research. gies in their learning activities. Based on these, we propose suggestions to improve LMS usage what will be of interest to the definition of orga- STATE OF THE ART nizational policies. The motivation for this work came from the Learning Management Systems heterogeneity of students’ profiles found in the Polytechnic of Porto and the need to enhance Electronic Learning (eLearning) can be defined the teachers’ pedagogical strategy based on the as the delivery of educational content via any students’ profile. The study is focused on gender, electronic media, including the Internet, satel- age and scientific field. These criteria were cho- lite broadcast, audio/video tape, interactive TV, sen based on several studies regarding LMS and CD-Rom and others (Tastle, White, & Shack- Internet and due to the variety of Polytechnic of leton, 2005). Despite some efforts to improve Porto’s students characteristics. remote education (Harasim, 2006), the genesis Several studies across the world have been of eLearning can be traced with the development combining the characterization and analysis of of network communication in the late 1960s, students’ Internet usage patterns and perceptions more precisely, with the invention of email and of how technologies could be used in learning at computer conferencing. These innovations con-
248 Differences in Internet and LMS Usage
tribute to the collaboration between teachers and majority of the academic institutions. Typically, students and initiate a new education paradigm they have two types of users’ groups: learners shift (Williams & Goldberg, 2005). During the and teachers. Learners can use the LMS to plan 1980s and 1990s, there was a significant growth their learning experience and to collaborate with in the number of students studying part-time and their colleagues; teachers can deliver educational also in non-traditional learners, such as, typical content and track, analyze and report the learner 18 to 24 years old students seeking the university evolution within an organization. demand and women’s returning to the workforce after child rearing (Williams & Goldberg, 2005). LMS in Higher Education The growth in lifelong learning has made the educational institutions to seek for flexible edu- Higher education institutions have been chal- cation delivery to satisfy these non-traditional lenged by the emergence of new information and students. By the end of the century, this delivery communication technologies (ICT), in the last few has been accentuated by the emergence of new years. Among various technologies, eLearning forms of distance delivery based on information and LMS have been widely adopted at higher and communication technologies advances, such education, motivating researchers and teachers as, the Internet. to study its influence in the learning process (e.g. In their first generation, eLearning systems Misko, Choi, & Lee, 2004; Gras, 2005). In fact, were developed for a specific learning domain Mayadas (2007) states that in 2006 there were and had a monolithic architecture. Gradually, these 3,5 million students enrolled in at least one on- systems evolved and became domain-independent, line course in United States (US); this was about featuring reusable tools that can be effectively used 20% of all university students in the US. Since virtually in any eLearning course. The systems then until now, online learning is been growing that reach this level of maturity usually follow 20% a year. Today, New York University enrolls a component-oriented architecture in order to almost 130.000 online students in LMS platforms facilitate tool integration (Leal & Queirós, 2010). (NYU, 2010). Different kinds of component based eLearning A vast number of LMS that provide integrated systems target specific aspects of eLearning, such services exists nowadays, like for instance, We- as student or course management. One such case bCT, Blackboard, Centra, Learning Space and is the LMS that aims to simplify the management Moodle (Botturi, Cantoni, & Tardini, 2006; Babo of learning within an organization (Harman & & Azevedo, 2009). Within these eLearning sys- Koohang, 2007).This type of system allows tems, Moodle assumes an important role (Babo students to plan their learning and collaboration & Azevedo, 2009). It is an open source solution, activities with colleagues, while teachers may as- offers a set of scalable tools and services that sociate educational content and monitor, analyze support communication and collaboration across and report progress of their students. Most LMS’s students and teachers and there are a lot of compo- are structured around courses rather than courses’ nents that we can add to the platform (Georgouli, content thus, they only support reusability at the Skalkidis, & Gerreiro, 2008). course level, where many learners can enroll In early January of 2010, Moodle had a user- on a single course. LMS also don’t support the base of 46,624 registered sites with 32,464,992 creation of instructional content. This “issue” users in 3,161,291 courses in 209 countries and in implies the use of third part content creation tools. more than 75 languages (Moodle, 2010). Accord- Despite these issues, the LMS are the cornerstone ing to some authors (Monsakul, 2007; Moodle, of the learning environments wide spread in the 2010), the following are the most common func-
249 Differences in Internet and LMS Usage
tions of LMS’s that we can find in Moodle: i) and online submission have been widely used by course information and documentation; ii) docu- students and teachers. On the other hand, only ment repository; iii) announcements; iv) discus- 44% of the respondent noticed positives changes sion board; v) external links; vi) synchronous and in their learning process after the adoption of asynchronous communication (email, chat room, Moodle platform. discussion forum); vii) assignments; ix) users pro- Several studies point to different Internet usage files. There is also some plug-ins that we can add habits, depending on the criteria discussed: male to Moodle, in order to install other functionalities students predominate in the use of Internet; female like, for instance, hot potatoes and Sharable Con- students prefer contents based on science learning tent Object Reference Model (SCORM) features. environment while male tend to prefer distracting According to Landsberger (2004), LMS’s relevant contents; the Internet usage increases by more features in higher education are about: i) course- 10 percentage points in each age group between organizing functions, such as electronic and 2006 and 2009; the number of messages sent and multimedia contents, evaluation and assessment received is clearly greater in men than women tools; ii) dynamic tools to facilitate the learning (Kubora et al., 2008). Introducing gender as a process, such as synchronous and asynchronous criterion in studies in the ICT fields consistently discussion groups; iii) student’s environments to show a male bias towards Internet use (Kubora enhance collaboration and cooperation between et al., 2008; Madell & Muncer, 2004; Nachmias, them. With its various functionalities, LMS’s Mioduser, & Shemla, 2001; Sink, Stob & Tanigu- serve different learner’s characteristics, different chi, 2008). And, especially in an education setting learning styles and outcomes (Landsberger, 2004; and, in order to get equity among graduates facing Monsakul, 2007). the labor market, gender is an issue that can’t be To understand all these changes and to see how dismissed (Madell & Muncer, 2004). students are dealing with them, several studies Age is another criterion that is very much were made to characterize Internet usage patterns considered in ICT studies. In recent years there and LMS implementations in higher education: has been widespread interest in studying young e.g. testing for a variety of strategies in an eLearn- people’s relation with ICT. Tapscott (1999) intro- ing platform (Oliveira, Oliveira, Souza, & Costa, duced, in the late 90’s, the term Net generation to 2006); other findings point the need to LMS characterize a whole new population born after support solving problems and critical thinking 1980 that grew in an environment where techno- functionalities (Andronico, Carbonaro, Colazzo, logical artifacts and digital culture were a part of Molinari, Ronchetti & Trifonova, 2004; Mon- their everyday life. The computer, Internet, cellular sakul, 2007); still others conclude that teaching phone, digital camera, digital games and social and learning pedagogy should be modified to the networks are, among others, examples of how new student’s typical profile, which includes the present is digital technology in this generation’s capacity of learning anytime, anywhere, across world, in such a way that these are not viewed as multi-platforms and with multi-tasking (Kubora, new technologies but as tools that have always Terashima, Nakahashi, & Morioka, 2008). been there (Prensky, 2001). This generation tends Botturi, Cantoni, &Tardini (2006) conducted to work and learn, unlike traditional approach, a survey at University of Lugano in order to developing some particular characteristics: mul- evaluate students’ usage and satisfaction with titasking, visual learning, hypertext, short mes- eLearning platform, Moodle. The survey indicates sages, and compact information, among others that 94,3% of the respondents is satisfied with the (Kennedy et al., 2009; Simões, 2009). Burlea platform usage, but only communication forums (2008) found in her study, that the older students
250 Differences in Internet and LMS Usage
(from 46-64 years old) found more difficulties ers and, systematically found that Informatics of individually using the facilities offered by the department teachers used Moodle more and in a information system. For example, Kennedy et al. more complex manner than teachers from other (2009) found slight differences between genera- scientific fields. tions of students and teaching staff in their study, as they said a little unexpectedly. As Polytechnic Research Methodology of Porto is known for having a large percentage of students who are (or has been in any moment of To characterize Internet and LMS usage patterns their lives) professionals (as opposed to students and their role in Higher Education, a descriptive that are continuing their studies after high school case study with students from Polytechnic of Porto without going through the labor market), and these was conducted. students are older, it was found to be interesting In Polytechnic of Porto, all courses are deliv- to study the behavior and perceptions of different ered in presential classes and students must enroll generation students, in the same learning context, in a minimum of classes (except for the working towards Internet and LMS platform supporting students that have a special authorization). LMS face-to-face classes. (Moodle) is used as a support to presential classes In the study promoted by The Australian and, it is not mandatory. Learning and Teaching Council (Kennedy et al. The students’ usage patterns were analyzed 2009) they considered “a key aspect of the Inves- in two components: Internet usage and LMS in tigation stage” (p.17) to analyze the discipline in Education. which they were studying: students were asked Internet usage patterns is characterized by type to nominate the course and subject in which they (e.g.: location, frequency, type of use, motivation) were responding to the questionnaire and these and by the underneath communication tools (e.g.: were used to classify students into the discipline social networks, chat, email in frequency, activi- categories, according to the Australian Standard ties, benefits). The role of LMS in education deals Classification of Education Codes. Their assump- with items such as: types of functionalities, their tion was to consider how the students’ experiences frequency of use and their perceived importance. with technology might differ as a function of the field they were studying. They did not found Data Collection relevant statistical differences between discipline areas, this is, they reported around the same level Data was collected through an online question- of technology use in all five discipline areas (Arts, naire distributed to students of five Polytechnic Sciences or others). However, clear differences of Porto’s schools: ESEIG2, ESTSP3, ESTGF4, between the three universities students’ engaged ISCAP5, and ISEP6. This instrument was chosen in the study (Kennedy et al. 2009). In Polytechnic because it allows the collection of a large number of Porto there are also very different study areas of answers at a reasonable cost. The questionnaire and, there is the common sense perception and, had twenty-seven questions, five of which were to some extent, empirical evidence, that students open. In the first week of June of 2009 a pilot test that engaged in Informatics courses tend to use was carried out with 15 students from various LMS more than others, since their teachers do it programs and academic years of ESEIG. This (Rodrigues, Pinto, Queirós, 2010). Rodrigues et test validated the questionnaire’s objectivity, un- al. (2010) studied Moodle sophistication usage, derstanding and also the web form’s accessibility. using Janossy & Hover’s (2008) model in one of In the second week of June 2009 began Polytechnic of Porto schools (ESEIG1) by teach- the dissemination of the questionnaire, in the
251 Differences in Internet and LMS Usage
various schools of Polytechnic of Porto. The an- In Figure 1, it is visible the respondents distri- nouncement was the same in all schools and was bution according to the school where they study. done by: announcements on the school website, According to the Ministry of Science, Technol- announcements on the school learning manage- ogy and Higher Education (DGES, 2010), Portu- ment system, email messages sent to students gal has ten study fields: Science (SCI); Health and teachers (so they would ask their students to (HEA); Technology (TEC); Agriculture and answer the questionnaire) and requests to teach- Natural Resources; Architecture, Arts and Design ers, who used computers in classrooms to free (AAD); Education; Law and Social sciences class time so that students could complete the (LSS); Economics (ECO); Humanities (HUM); questionnaire. The questionnaire answers were Physical Education, Sports and Show’s Arts. In collected anonymously, from June 5th until the Figure 2 it is possible to see the distribution of end of July. In July, the schools’ academic services the population and the sample by scientific field. were contacted in order to gather descriptive data As observed the majority of the students (popula- about the population. tion and sample) attend courses in the field of Technology (TEC-51%), Economics (ECO-23%), Population and Sample Heath (HEA-13%) and Law and Social Sciences (LSS-9%) and so, the analysis by scientific fields Similar to what is done in other studies (e.g. Walker will be based on these main fields. & Jorn, 2007), the case study methodology was Most of the students are undergraduate adopted for this study. Students from five Poly- (94.6%), 5.2% are graduate students and students technic of Porto’s schools compose the universe from technological specialization programs are of this study. The study included students from 0.2% of this sample. Concerning gender, 51% of technological specialization programs and also students are female and 49% male. from undergraduate and graduate programs. From As expected, in the population there are more the 1416 obtained answers, 1397 were considered students born in 1980 and after (86%) than born valid (11% of Polytechnic of Porto’s population). until 1980 (14%)7. However, the percentage of older students (over 29 years old) is large because
Figure 1. Sample distribution across schools
252 Differences in Internet and LMS Usage
Figure 2. Sample distribution across scientific fields
Polytechnic of Porto’s schools also host working which differences were statistically significant students, which are older than ordinary students. through hypothesis tests. Our analysis considered The proportion of working students in the popula- three students features: age, gender and scientific tion is 25%, 32% of which answered the survey. field. In the first two, defined only by two groups, The student’s sample regarding open questions we have used the proportion test using the Chi- analysis was defined considering all students who square distribution in nominal variables and the answered all the open questions (n=121). The Mann-Whitney test in ordinal variables. In the sample consisted of 65 female students (53,7%) scientific field feature, we have decided to select and 56 male students (46,3%). Considering age only the four most representative fields, namely groups, there are 102 students (84,3%) born in Health, Technology, Law and Social Sciences, and or after 1980 and 19 (15,7%) born until 1980. Economics. To study the differences between sci- Finally considering the scientific fields there are entific fields, we have used the Chi-square test on 26 students (21,5%) from Health, 52 students nominal variables. On ordinal variables, we have (42,1%) from Technology, 13 students (10,7%) applied the Kruskal-Wallis test to detect differ- from Law and Social Sciences, 27 students (22,3%) ences between the 4 fields and, when differences from Economics and 3 students (1,7%) that didn’t were found, we have applied the Mann-Whitney answered what was the scientific field that they test between pairs of scientific fields adjusting the belonged. significance level to the number of tests performed. The open questions analysis was done using QSR NVivo 7. To analyze the significant differ- DATA ANALYSIS ences found in NVivo7, we applied the proportion hypothesis test in age and gender, and the Chi- The data analysis was carried out on two major square test in the scientific field. components: characterization of Internet usage, and the role of Internet and LMS in education. In both cases, we have started by a descriptive analysis of the sample and, then, we studied
253 Differences in Internet and LMS Usage
Internet Usage forums (p=0.000), instant messaging (p=0.000), social networks (p=0.000), wikis (p=0.000), games Due to the nature of data this section was divided (p=0.000). Only blogs are used similarly by male into two sub-sections: Type of Internet usage and and female in both students’ generations (Babo Communication tools usage. et al., 2010a). Scientific field is also statistically related with Type of Internet usage the usage of email (χ2=52.12, p=0.000), search engines (χ2=33.67, p=0.000), forums (χ2=9.13, The majority of students access the Internet at p=0.000), blogs (χ2=9.13, p=0.028), instant home (91%) and at school (80%). Only 24% say messaging (χ2=38.61, p=0.000), social networks they have mobile Internet access. Most students (χ2=48.41, p=0.000), wikis (χ2=54.70, p=0.000) access the Internet several times a day (65%), and games (χ2=14.77, p=0.002). A descriptive 32% connect daily and the remaining 3% con- analysis shows that health students tend to use more nect monthly, weekly and even more rarely. On search engines, instant messaging and blogs than average, 51% of students are connected to the students from other scientific fields. Technology Internet between 1 and 3 hours per day (Rodrigues students use more email, forums, games and wikis et al., 2010). and students from Law and Social Sciences use We found that male students stay connected for more social networks than students from other longer periods of time (p=0.013) (Babo, Lopes, scientific fields. Rodrigues, Pinto & Oliveira 2010a). Students According to Table 1, presented below, the born in 1980 and afterwards stay connected for main reasons for Internet usage are research longer periods of time (w=103902, p=0.000). An concerning class works/study (94%), accessing explanation for this factor could be the professional documents in Moodle or another LMS (77%) duties of the older students, which prevent them and email exchange (77%) (Rodrigues et al., to a more lasting usage (Babo, Lopes, Rodrigues, 2010). An overview of Internet usage’s main Pinto, Queirós & Oliveira, 2010b). Analyzing by reasons by gender (Babo et al., 2010b) shows scientific field we found that students enrolled that female students statistically use the Internet in Technology degrees stay connected for longer more than male students to research concerning periods of time than Health (w=64028, p=0.019) class work/study, email, connect with friends and and Economics (w=143266, p=0.000) students. access friends’ social networks. Male students use On average, 48% of the time spent on the Internet theZ Internet more than female students to read is for personal leisure (the mode is 50%). The newspapers, magazines and portals, download most used Internet tools are email (95%), search music or films, play games, use forums (post or engines (92%), instant messaging (58%) and social read) and shopping. In general, it is noticeable networks (52%). that female students use the Internet to study, and Students’ gender is related to the usage of some to socialize with others. While male students are Internet tools. Male students statistically use more more engaged in leisure activities and enrolled instant messaging (p=0.007), forums (p=0.009), in activities by themselves that do not depend on games (p=0.000) and wikis (p=0.000) and female other people (Babo et al., 2010b). students, statistically, use more social networks With the two independent samples t-test, it (p=0.000) and search engines (p=0.039) (Babo et was also verified that youngest students use more al., 2010b). Students’ generation is also related Internet based tools (p=0.000) than students born with the use of Internet tools. Younger students use before 1980 (Babo et al., 2010a). more email (p=0.000), search engines (p=0.003),
254 Differences in Internet and LMS Usage
Table 1. Gender, age and scientific field differences regarding Internet usage’s main reasons
Gender Age Scientific Fields Internet Usage’s Total Older Younger Main Reasons F M HEA TEC LSS ECO (Born<1980) Born >1980) Researching 96% 90% 97% 92% 97% 93% concerning class 94% works/ study p 0.000 p 0.008 Accessing docu- ≅ 85% 73% 65% 80% ≅ 82% 72% ment in Moodle or 77% another LMS p 0.001 p 0.000 82% 73% 87% ≅ 70% 83% 72% ≅ 80% 80% Email 77% p 0.000 p 0.000 p 0.001
Connecting with 78%≅ 66% 8% ≅ 63% 84% 70% ≅ 76% 64% 72% friends p 0.000 p 0.000 p 0.000
Reading newspa- 43%≅ 62% 68% ≅ 49% 35% 56% ≅ 58% 49% pers, magazines 52% and portals p 0.000 p 0.000 p 0.000
Downloading 44%≅ 57% 17% ≅ 55% 52% 57 % ≅ 43 % 40 % 50% music or films p 0.000 p 0.000 p 0.000
Searching info ≅ 56% ≅ 42% ≅ (other than school 47% issues) p 0.000 Sharing informa- 3% ≅ 25% tion (documents, 28% music, films, etc.) p 0.000
Accessing friends’ 30% 18% 2% ≅ 21% 27% 17% 39% 27% 24% social networks p 0.000 p 0.000 p 0.000
14%≅ 24% 1% ≅ 17% 14% 22% ≅ 14% 18% Gaming 19% p 0.000 p 0.000 p 0.019
Using forums 6% ≅ 31% 2% ≅ 15% 7% 27% ≅ 10% 11% 17% (post or read) p 0.000 p 0.000 p 0.000
7% ≅ 17% 2% ≅ 9% 4% 16% ≅ 8% 12% Shopping 12% p 0.000 p 0.000 p 0.000 Using Blogs (post 9% ≅ ≅ ≅ or visit others) Searching new 9% 10% friends/aquain- 6% tances p=0.028 (The absence of p-values means there are no significant differences between the groups.)
Data overview shows that both generations networks, play games, and participate in fo- of students use the Internet for different reasons rums. Older students use the Internet mainly for (Babo et al., 2010a). Younger students use it for visiting the eLearning platform to look for new contacting with friends, download music and documents, email, read the news in newspapers, movies, share information (documents, music, magazines and portals, to get information about movies, etc.), visit friend’s webpage in social various themes not connected with their studies,
255 Differences in Internet and LMS Usage
and to shop. Non working students, namely, the of social networking is minor among the older younger students at IPP, do not shop (shop in a generation: only 54% have a profile in a social lesser extent) because they cannot afford it (Kwak, network. Finally, the possession of a profile in a Fox & Zinkhan, 2002). social network is also related with the scientific In general this descriptive analysis shows that, field of the student (χ2=18.67, df=3, p=0.000). on one hand, younger students are more enrolled More than 74% of the students from Health, Law in Web 2.0’s activities like socialization and shar- and Social Sciences and Economics degrees have ing. On the other hand, students born before 1980 a profile in a social network while 69% of Tech- engage in activities typical of the Web 1.0 using nology students’ have such a profile. the Internet as a mean of getting information, more Table 2 shows the most engaged activities in than to interact with others, to share information, social networks by the students: viewing photos namely personal information (Babo et al., 2010a; (47%), knowing what my friends are doing/what Kennedy, 2009). is happening (36%) and commenting (35%) are Through Table 1 we can see that almost all the most reported activities in social networks. reasons for Internet use are statistically associ- An overview of the data (Babo et al., 2010b) ated with the scientific field. The exceptions are: and Table 2 above, shows that female students searching for information not related to school are, with statistical evidence, the ones most en- subjects, sharing information, using blogs and gaged in activities developed in social networks: searching for new friends. The trend shows that view photos, try to find out what their contacts health students use the Internet more than other are doing, make comments, talk with other people students to: research concerning class work/study, and share information (photos, videos, music). email and connect with friends. Technology stu- Male students admit more than female students dents use more the Internet to download music that they meet new people, searching for new and movies, play games, use forums and shopping. friends and acquaintances. Either, male students Law and Social Sciences students use more the are meeting other male students or, female students Internet to: research concerning class work/study, are less frank about their online activity, another visit the eLearning platform to look for new docu- explanation could be, female students are not ments, read the news in newspapers, magazines actively trying to meet new people, thus do not and portals and accessing friends’ social networks. acknowledge this activity. Among younger students, it is observed a more Communication Tools Usage frequent use of social networks. In social networks, the younger students are statistically different In general, 74% of Polytechnic of Porto students from the older in what regards the activities: see have a profile in a social network. Of these, 91% photos (p 0.000), find what their contacts are have a profile in HI58, followed by Facebook doing (p 0.000), do comments (p 0.000), talk (23%). About 33% of the students use social net- with other ≅people (p=0.003) and, share information works to make new friends. Nevertheless, there (p 0.000):≅ photos, films, music. Older≅ students is a statistically significant (p=0.002) difference statistically update their CV (p 0.000) more than between gender: 78% of the female students have the≅ younger generation and, search for interesting a profile in a social network, while only 70% of information other than school subjects≅ (p 0.000). the male students have such a profile (Babo et Through Table 2 we can see that almost all the al., 2010b). Regarding generation there are also activities in social networks use are statistically≅ differences: 77% of the younger students have associated with the scientific field. The exceptions a profile in a social network. The importance are: search for information not related to school
256 Differences in Internet and LMS Usage
Table 2.Gender, age and scientific field differences regarding most engaged activities in social networks
Activities most Gender Age Scientific Fields developed in Social Total Older Younger F M HEA TEC LSS ECO Networks (Born<1980) Born >1980) 54% 39% 22% 51% 60% 42% 55% 45% Viewing photos 47% p 0.000 p 0.000 p 0.000
Knowing what my 41%≅ 31% 56% ≅ 77% 44% 32% ≅ 44% 35% friends are doing/what 36% is happening p 0.000 p 0.000 p 0.000 44%≅ 26% 16% ≅ 39% 49% 28% ≅ 43% 34% Commenting 35% p 0.000 p 0.000 p 0.000
Finding interesting ≅ 18% ≅ 8% ≅ things(search infor- 33% mation – other than p 0.000 school issues) ≅ Chatting with other 35% 27% 29% 28% 34% 39% 31% people p 0.000 p 0.004
Sharing information 30%≅ 19% 14% 26% 34% 20% ≅ 33% 22% (documents, photos, 24% videos, music) p 0.000 p 0.000 p 0.000 Meeting new people 10%≅ 19% 24% ≅ 32% 10% 17% ≅ 8% 19% (search new friends/ 15% aquaintences) p 0.000 p 0.008 p 0.004
Informing my friends ≅ 28% ≅ 32% ≅ 15% what I am doing p 0.000 Searching for informa- 9% ≅ tion about companines 17% 7% 5% 8% 12% 10% Updating CV 8% p 0.000 p 0.040
Setting personal 27% ≅ 34% ≅ meeting with network 3% contacs p 0.033 Sending messages 0% ≅ The absence of p-values means there are no significant differences between the groups subjects, search for information about companies, In order to connect synchronously, 98% of the setting personal meetings and sending messages. students use MSN while Skype and Google Talk The trend shows that health students use more the have small shares (15% and 11%, respectively). social networks to: view photos, try to find out However, male students are more diverse in their what their contacts are doing, make comments tools’ choice (Google Talk, p=0.001 and Skype, and share information. The Law and Social Sci- p=0.021). Females are much more loyal to MSN ences students use the social networks to: try to hardly using any other synchronous tool (p=0.028). find out what their contacts are doing and update Data shows (Babo et al., 2010b) that 28% of the their CV. Finally, Economics students use the students use instant messaging several times a social networks to talk with other people and try day and 34% use it at least daily. Male students to meet new friends. are more frequent users (p=0.004).
257 Differences in Internet and LMS Usage
Instant messaging is used by 28% of the group and scientific field will be reported. Table 3 students several times a day and 34% use it at analysis the statistical differences encountered by least daily. In what refers to differences between gender and age group in: a) the students’ access generations, younger students are more frequent LMS platform; b) Students’ evaluation of usability users (p=0.000). of the LMS platform and c) Students’ evaluation Email is used more than instant messaging. of the importance of the LMS platforms in their According to data (Babo et al., 2010b), 52% of learning process. students use email several times a day and 39% Analyzing Table 3, by gender, male students use it at least once a day. Even though there are not visit more often the LMS platform than female statistical gender differences regarding whether students (65% of male students visit the LMS students use email, when it comes to the frequency platform in a daily basis against 57% of female of email usage male students use email more often students), a difference that is statistically signifi- (p 0.000): 59% of male students use it several cant (p=0.006). It is important to notice there are times a day against 44% of the female students. no statistical differences between the two age In ≅terms of scientific field we found that, regard- groups regarding the frequency of visits to LMS ing to the use of email, there are some statistical platforms. differences between the several fields considered. When it comes to usability it is important to Health students use it less than Technology stu- say that no gender or age statistically significant dents (w=5718, p=0.000) and less than Law and differences were found related to the difficulty in Social Sciences students (w=13623, p=0.001). the use of LMS platforms. Economics students use it less than Technology Analyzing the importance of LMS platforms students (w=116093, p=0.000) and less than Law in students’ learning process it is important to and Social Sciences students (w=27952, p=0.001). refer that more than 77% of the students consider From the data collected we can state that the ma- LMS platforms to have an important role in their jority of students use the email several times a day learning process. If we associate this result with with special importance to Technology (58,3%) the fact that most students access LMS platforms, and Law and Social Sciences (57,3%) students. at least once a day, in order to help students’ learn- Unlike the observed with the results of Inter- ing teachers should improve and promote the use net’s session time, social networks and instant of these platforms. messaging usage, older students do a larger use In terms of age group we found that older of email (71% use it several times a day against students feel that this kind of platforms are more 48% of the youngest ones; p 0.000). This might important to their learning process than younger be happening due to the fact that older students students, a difference that is statistically signifi- usually are also workers and≅ can have email con- cant (p=0.008). This result is consistent with the nected all day allowing them to check it several fact that most of the older students are employed, times a day. attend fewer classes and need the LMS platforms to be updated, inquire teachers and colleagues Role of Internet and and access documents on the contents covered LMS in Education in classes. No differences were found in terms of gender. The same analysis was made concerning In this section we will analyze the use of the scientific fields (Table 4). Internet and LMS and their role in education. A Through Table 4 we can see/observe that the descriptive analysis of the sample will be made major differences encountered are between Health and significant statistical differences in gender, age students and students from other scientific fields.
258 Differences in Internet and LMS Usage
Table 3. Statistical differences by gender and age group regarding students’ access to LMS platform, students’ evaluation of LMS usability’s platform, and the importance of those platforms to students’ learning process
Male Born >1980 Access < Access NDF p-0.006
Female Usability NDF Usability NDF Born<1980 Importance NDF Importance > P=0.008 (NDF equals Non Differences Found. The left operands of the signs are the row header. The right operands are the column header.)
In terms of access to LMS platforms Health stu- Social Sciences students (w=11299, p=0.000) and dents have the lowest access to these platforms Economics students (w= 30598, p=0.000). Fi- what is consistent with the fact that Health students nally, and in concordance with the previous results, have more difficulties when using LMS platforms it is important to say that only 59% of Health (only 63% of them say that the use of the LMS students say the LMS platform is important to the platform is easy against more that 78% of the learning process against more than 79% for stu- other scientific fields). This last result is statisti- dents from other scientific fields. This result is cally significant since the Wilcoxon test shows statistically significant since the Wilcoxon test Health students find it harder to use than Technol- shows Health students find it less important than ogy students (w=51506, p=0.000), than Law and Technology students (w=47211, p=0.000), Law
Table 4. Statistical differences by scientific field in students’ access to LMS platforms and evaluation concerning the usability of LMS platforms and the importance of those platforms learning process
HEA TEC LSS ECO Health Access < < < w=43761 w=11348 w=26457 p=0.000 p=0.000 p=0.000 Usability < < < w=51506 w=11299 w=30598 p=0.000 p=0.000 p=0.000 Importance < < < w=47211 w=9300 w=26772 p=0.000 p=0.000 p=0.000 Technology Access NDF NDF Usability NDF NDF Importance < NDF w=35959 p=0.004 Law & Access NDF Social Sciences Usability NDF Importance > w=26923 P=0.005
259 Differences in Internet and LMS Usage
and Social Sciences students (w=9300, p=0.000) The trend shows that forums’ are more used and Economics students (w=26772, p=0.000). by Technology students’ teachers and wikis and According to the students in this study, the tools blogs are more used by Law and Social Sciences most used by teachers for educational purposes are: field teachers. These results may be related to the email (88%), Search Engines (36%) and Forums tools that students themselves say they use more (23%), as illustrated in Table 5. frequently: email, search engines and Forums Analyzing the students’ response we can find (Babo et al., 2010b). No statistical differences significant differences in their perceptions about were found in age groups. the tools used by their teachers. Only the use of Regarding the open question where students search engines by teachers is not associated with were asked about what can be done to facilitate or the scientific field. The proportion of students enhance their learning regarding LMS platforms, stating that the teachers use the email is larger in five dimensions emerged from the qualitative health students. This can explain why health analysis: a) to be used by all teachers (27%); b) students are not so supporters of LMS platforms update the content available (25%); c) to use a as students from other scientific fields. As stated wide variety of functionalities (16%); d) to be by students, search engines are the second most used to increase the interaction between teachers used tool by their teachers. Considering gender, and students (9%); e) make easier their access female students, have the perception that their and use (9%) (Table 6). teachers use search engines more than male stu- Describing results by gender we can state that dents (40% against 33%, p=0.008). Tools such as female students find that the two most important forums (25% against 21%, χ2=3.2482, df=1, actions to be performed in the LMS platform to p=0.036), blogs (9% against 6%, p=0.032) and facilitate their learning process are its use by all Wikis (11% against 7%, p=0.008) are, in male teachers (40%) and content’s update (27%). For students’ opinion, more used by their teachers. male students the two most important actions are
Table 5. Internet’s tools used by teachers for school purposes
Gender Age Scientific Fields Internet tools used by Total Older Younger teachers F M HEA TEC LSS ECO (Born<1980) Born >1980) 89% 87% 86% 88% 95% 89% 80% 83% Email 88% χ2=27.85, df=3, p=0.000 40% 33% 34% 37% 43% 36% 37% 32% Search engines 36% χ2=5.71,df=1 P=0.008 21% 25% 21% 23% 7% 28% 20% 26% forums 23% χ2=3.25,df=1 χ2=45.30,df=3,p=0.000 p=0.036 7% 11% 7% 9% 1% 13% 18% 4% wikis 9% χ2=5.90,df=1 χ2=57.09,df=1,p=0.000 P=0.008 6% 9% 11% 7% 2% 9% 10% 8% blogs 8% χ2=3.43,df=1 χ2=13.44,df=3,p=0.004 p=0.031 (The absence of p-values means there are no significant differences between the groups.)
260 Differences in Internet and LMS Usage
content’s update (31%) and a wider variety of more important to female students than male stu- functionalities (29%). dents and has statistical relevance (χ2=7.24, df=1, Analyzing by age we can state that for older p=0.004). For all the other dimensions considered students it is important to enlarge the variety of and demographic variables there weren’t found functionalities used in LMS platforms (33%) and statistical differences and this means even if there to use them to increase the interaction between are some tendencies, all of students have similar students and teachers (20%). Younger students opinion about what could enhance their learning think it is important to spread the use of LMS regarding Moodle. platforms to every teacher (34%). We would Many students state that it would be important like to note that none of the older students refers that the LMS platform was used by all teachers difficulties to access or to use LMS platforms. because it allows in a single place to have all the Finally, analyzing by scientific fields we can information related to a course and stay updated. state that “to be used by all teachers” is the most To illustrate this opinion we transcribe an answer: important action to be taken to almost all scientific fields (Health, Law and Social Sciences and …it was important that ALL teachers saw the Economics). The exception is the Technology field platform as something important for the aca- degree students’. For Technology field students’ demic community because it allows students to the most important action is “update the contents stay updated and know the material taught in available”. This action also was considered im- class… (S221) portant to Health and Economics. Statistical analysis of these five dimensions Students, also, think that teachers should use and the demographic variables considered in this more LMS’s functionalities like quizzes, forums, study there weren’t found any differences with blogs to improve the learning process and they the exception for the first dimension “To be used also think it is important to promote interaction by all teachers”. This dimension was considered
Table 6. What measures could be used in LMS platforms to facilitate or enhance your learning?
What measures could Gender Age Scientific Fields be used in LMS platforms to facilitate Total Older Younger F M HEA TEC LSS ECO or enhance your (Born<1980) Born >1980) learning? 39% 14% To be used by all 27% 2 16% 29% 31% 14% 54% 33% teachers χ =7.24,df=1 p=0.004 Update the content 25% 26% 23% 21% 26% 27% 25% 15% 26% available To use a wide variety 16% 11% 21% 26% 14% 15% 17% 23% 11% of functionalities Make their access/use 9% 8% 11% 0% 11% 15% 8% 15% 4% easier To be used to increase the interaction between 9% 12% 5% 16% 8% 8% 8% 15% 8% students and teachers (The absence of p-values means there are no significant differences between the groups.)
261 Differences in Internet and LMS Usage
between teachers and students. The next two (Kennedy et al., 2009), in which 95,8% of the transcripts illustrate these ideas: students have access to Internet. Such facts could indicate that teachers should exploit the use of …Do diagnostic tests, promote the use of blogs teaching and learning methodologies supported or forums, to share questions and those questions on the Internet. be answered by teacher for all students… (S252) Students that stay connected for longer periods of time are male, born until 1980 (also found by Williams and Goldberg, 2005) and studying Create a kind of msn that would allow teachers to Technology or Law and Social Sciences. talk in real time with students in order to answer Reading email and search engines are the most to student’s questions (S247) used Internet tools by students, furthermore, data shows that there are no significant differences Finally students have the opinion that access between genders. Male students use more Internet to LMS platforms must be improved and must tools (wikis, forums, games, instant messaging), be easier to use. The next transcript reflects this with exception for social networks and search opinion, although it doesn’t make any specific engines than female students. These results are, suggestion: again, in line with the study of the Australian universities (Kennedy et al., 2009) in which the Easier access and more precise instructions for most used Internet tools are email, search engines use… (S110) and instant messaging. Concerning age gaps it is a fact that younger students use more email, search engines, forums, RESULTS DISCUSSION instant messaging, social networks, wikis, and games. Only blogs are used similarly by both The results presented in this study are very im- students’ generations. portant for two reasons. First, they show the state Scientific field is related with the usage of of Internet and LMS platforms usage for higher Internet tools. Students from Health statistically education learning proposes in a very important use more search engines, instant messaging and Portuguese tertiary level education institution, Blogs than students from other scientific fields. allowing an analysis and comparison of this im- Students from Technology use more email, fo- portant topic across countries. Second, they are rums, games and wikis than students from other important to the LMS policies to be defined at scientific fields. Finally, students from Law and Polytechnic of Porto. Social Sciences degrees use more social networks Since LMS are available on the Internet it than students from other scientific fields. was considered important to know the Internet When students were asked about the reasons students’ usage habits. for accessing the Internet, most of them answered The data collected and analyzed in this study that the main reasons are academic. They use it show the state of Internet usage and LMS per- to carry out research to do class work or to study. ceptions at Polytechnic of Porto. From the data Other reasons mentioned by females are be- analysis, we can state that 91% of the students ing connected with friends and access to friends’ have Internet access at home and 80% at school. social networks. Conversely, men prefer to use the They also show that 90% of students’ access In- Internet to read newspapers, download music or ternet every day. These results are in agreement videos and play games. The reason for this behav- with a study made by an Australian Universities ior differences can be explained by psychological
262 Differences in Internet and LMS Usage
differences between the two genders. Women tend The differences found were regarding the to exhibit a more social behavior than men, who frequency of LMS access: male students access it usually have a more individual attitude. These more frequently then female students. This find- results are in agreement with a study made by ing is consistent with the fact that male students Madell and Muncer (2004). stay connected to Internet for longer periods of Both generations of students use the Internet time (e.g. Minhsein, 2005; Misko et al., 2004; for different reasons (Babo et al., 2010a). Younger and even in Japan: Kubora et al., 2001). Next students use it to contact with friends, download studies will allow us to see if the gender gap will music and movies, share information (documents, decrease, as found in other international studies music, movies, etc.), visit friend’s webpage in (e.g. Misko et al., 2004). social networks, play games, and participate in fo- In what comes to age differences, it was found rums. Hence, older students use the Internet mainly that older students found LMS more important to to visit the eLearning platform to look for new their studies than younger ones. This finding goes documents, email, read the news in newspapers, accordingly with the increase of non-traditional magazines and portals, to get information about students (over 24 years old) in higher education various themes not connected with their studies, studies (Williams & Goldberg, 2005). The growth and to shop, since, unlike the younger students, in lifelong learning has made the educational in- they have the economic means that enable that stitutions to seek for flexible education delivery attitude(Kwak et al., 2002). and support so they can better host and address In general, this study’s data shows that, on these news students’ needs. one hand, younger students are more enrolled in Regarding scientific fields the access to the activities of Web 2.0 like socialization and shar- LMS platform results are in accordance with the ing. And, on the other hand, students born before general findings across Polytechnic of Porto except 1980 engage in activities typical of the Web 1.0 for the Heath degrees’ students. The perceptions using the Internet as a mean of getting informa- of the LMS ease of use by Heath students may tion, more than to interact with others, to share explain its lower use: generally they have more information, namely personal information. This difficulty in using the LMS platform when com- finding is consistent with several studies conducted paring with students from other scientific fields. in this area (Kennedy et al., 2009). In general, students find it easy to use Moodle The role of Internet and LMS in education and no differences were found neither between was analyzed in five different perspectives: fre- gender and age groups. Heath degrees’ students quency, difficulty and importance of its usage to have an issue using LMS: they use it less, they the students learning, the kind of functionalities find it more difficult to and they don’t find it as were used by teachers (according to the students important for their studies as students in other perceptions) and suggestions from students in fields. These might be explained by the low use order to improve LMS usage for their learning of Moodle by teachers: the less teachers use it, process. Differences between gender and two age the less it used by students, the less they find it groups (students born before 1980 and in 1980 or important and the less they train their abilities in after) as well as the scientific field of their degree using it, thus they find it more difficult. were analyzed. Regarding students’ perceptions about the tools More than 60% of Polytechnic of Porto’s stu- used by their teachers the use of email is unanimous dents checks out the LMS platform at least daily for gender, age group and for all of the scientific which is similar to what happens in the University fields but for Heath degrees’ students. Students’ of Minnesota: 59% (Walker & Jorn, 2007). perception is that email is the most common In-
263 Differences in Internet and LMS Usage
ternet tool used by their teachers. This result can know what Internet tools teachers are using, we also help to explain why Health students are not should find the original data from the LMS system so supporters of LMS platforms as other students administrator, since the perceptions of students from different scientific fields. may not be reliable. Supporting these findings in an eLearning From the suggestions students made regarding readiness evaluation study carried out in Poly- what could be done to improve or enhance their technic Heath School (Lopes, 2007), it was found learning regarding LMS platforms, emerged five that student’s access to computers and Internet, dimensions: (1) to be used by all teachers, (2) one of the major concerns, was due to a lack of update the content available, (3) to use a wider technological infrastructures. This study allowed variety of functionalities, (4) to be used to increase to identify the professors’ need of ICT training the interaction between teachers and students and and technological support (72,4% said they had (5) make access and use easier. no previous experience with eLearning) and Suggestions from students’ point out that a acknowledge professors interest and openness lot of teachers still do not use the LMS platform towards e-learning (e.g. 69% value lectures access and, from the ones that use them, some don’t have in the LMS platform). the (content) updated content, which implies that The students’ perceptions of the Internet tools students would like to base their studies more on most used by teachers are email (88%), search the LMS platform. These are the two features engines (36%) and forums (23%). Botturi et al. female students point out primarily, which goes (2006) study at University of Lugano also found in accordance with previous results found in this similar frequent Internet tools used by teachers study, which state that female students use more (perceived by students). Even though students Internet to study than male students. Some other have classes with the same teachers, we found suggestions, request for sophistication, saying that gender differences in the perceptions of the teach- more functionalities should be used, including the ers Internet tools usage. No difference between interaction between teachers and students. Older age groups was found. students make this request in bigger proportion. Analyzing by scientific field the forums’ are This finding reinforces the importance that older most used by the teachers of Technology degree students give to LMS platforms as a mean of learn- students. Wikis and blogs are more used by Law ing. Male students also advocate the use of other and Social Sciences teachers. These results may functionalities in the LMS, which is consistent be related to the tools that students themselves say with previous findings in this study. “To be used they use more frequently: email, search engines by all teachers” is the most important action to and Forums (Babo et al., 2010b). In realizing that be taken to almost all scientific fields (Health, students’ perceptions about the tools used by their Law and Social Sciences and Economics). The teachers may be influenced by their own personal exception is Technology field, whose students’ experience led us to two different kinds of conclu- agreed that the most important action is “update sions: empirical finding and new course of action. the contents available”, followed by Health and Students tend to say that their teachers use the Economic students. tools they, themselves use. Along with data from On one hand there was demand for more the first part of this study characterizing students’ sophisticated use of LMS platforms, but on the Internet usage we can conclude, that male students other hand, a smaller percentage of students (11%) use more diversified Internet tools, than female request for more help and direction to access and students. Regarding the course of action we should use the LMS platform. These differences show dif- follow in future researches, if we want to really ferent levels of LMS users abilities, which suggest
264 Differences in Internet and LMS Usage
that making training or a specialist available for of the most addressed issues about enhance eL- students would be welcomed. To increase LMS earning is the need to “be used by all teachers”, usage some institutional rules should be used and “ update the content available”. It suggests along to with training and support to teachers, in the need of a more pro-active attitude by teachers order to increase the use of the ones who still do concerning the LMS usage. not use or use it barely and to assist the advanced Another important issue that we could extract users to sophisticate their usage even more. from the data collected and discussed is the variety of Internet tools used by students, according to their scientific field. It is possible that the student CONCLUSION profile (interests, demands requirements, relationships, etc.) could influence the type of tools and The presented study focused on one of the first tasks done on the Internet. Thus, it is important studies in Polytechnic of Porto about Internet and to adapt eLearning strategies and learning ap- LMS usage, as well as the role of LMS in higher proaches according the student scientific field education as a tool to support the learning process and its profile. Students from scientific fields The data collected was analyzed, considering three like law, technologies, health or others, have dif- dimensions: gender, age and scientific fields, in ferent profiles, different cultures and different order to provide a more detailed picture of the approaches in their learning processes. The LMS student population of Polytechnic of Porto. methodologies and strategies must be centered on The results could support the idea that it is the student needs, according to their skills and important to promote active learning strategies, their learning context. across students and teachers, which improve the Following the example of the University of learning process. The results clearly suggest the Minnesota (Walker & Jorn, 2007), we intend to implementation of methodologies and learning carry out a bi-annual analysis of the habits of tools (LMS) based on the Internet, since students Internet and LMS usage among students of the have acquired consistent habits in using the In- Polytechnic of Porto to study its development, ternet and LMS platforms. According to the data as well as the evolution of “new technologies” collected and analyzed, the majority of students usage in teaching and learning activities. This have skills and competences to use Internet tools information may have a relevant role in defining and LMS facilities, in order to improve their learn- technologies planning and thus, to the success of ing process. In fact, almost all students access higher education proposes. the Internet daily, regardless of scientific field, In future studies, we aim to study the habits gender or age. of LMS usage for teaching: what Internet tools, However, some results could indicate the need LMS activities and type of learning objects are to introduce some changes in the actual paradigm, available to promote learning and interaction particularly in respect of: i) the usage of Internet among teachers and their students. In future stud- tools by teachers, shifting from traditional e- ies, even though “what” is being used is always mails to collaborative platforms (Web 2.0), such important, it is our plan to focus on the “how” wikis, blogs, and forums. A significant number of LMS are being used and perceived by students students, particularly the younger ones, use these as well as teachers. tools daily so it suggests their ability to explore With all this data, it will be possible to establish these tools in an learning environment; ii) a greater orientation guidelines for the use of technologies in usage of the LMS platform, particularly in some IPP and Higher Education institutions, in general. scientific areas, such as health field. In fact, one
265 Differences in Internet and LMS Usage
REFERENCES Conole, G., de Laat, M., Dillon, T., & Darby, J. (2006). JISC LXP: Student experiences of Andronico, A., Carbonaro, A., Colazzo, L., Mo- technologies: Final Report: Joint Information linari, A., Ronchetti, M., & Trifonova, A. (2004). Systems Committee. Retrieved October 7, 2010, Designing models and services for learning man- from http://www.jisc.ac.uk/. agement systems in mobile settings . In Mobile and Ubiquitious Information Access (Vol. 2954, pp. Georgouli, K., Skalkidis, I., & Gerreiro, P. (2008). 524–532). Berlin-Heidelberg, Germany: Springer- A framework for adopting LMS to introduce Verlag. doi:10.1007/978-3-540-24641-1_8 e-learning in a traditional course. Journal of Educational Technology & Society, , 227–240. Babo, R., & Azevedo, A. (2009). Learning management systems usage on higher education Gras, M. (2005). Internet usage and digital divide institutions. In Proceedings of 13th IBIMA Confer- among Spanish university students. Revista Media ence - Knowledge Management and Innovation e Jornalismo - A Europa e os Media:14(8)(1). in Advancing Economies: Analyses & Solutions Harasim, L. (2006). History of e-learning: Shift (pp. 883 – 889). happened. The international handbook of virtual Babo, R., Lopes, C., Rodrigues, A., Pinto, M., learning environment. Berlin-Heidelberg, Ger- Queirós, R., & Oliveira, P. (2010a). Gender dif- many: Springer-Verlag. ferences in Internet usage habits - A case study Harman, L., & Koohang, A. (2007). Learning in higher education. 14th International Business objects: Standards, metadata, repositories, and Information Management Association, Istanbul, LCMS. Information Sciense Institute, Informing Turkey. Sciense. Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Janossy, J., & Hover, T. (2008). Proposed model Queirós, R., & Oliveira, P. (2010 b). Compari- for evaluating C/LMS usage. In K. McFerrin et son of Internet usage habits in two generations al. (Eds.), Proceedings of Society for Information of higher education students - A case study. 2th Technology & Teacher Education International International Conference on Computer Supported Conference 2008 (pp. 2979-2986). Chesapeake, Education, Valencia, Spain. VA: AACE. Botturi, L., Cantoni, L., & Tardini, S. (2006). Kennedy, G., Dalgarno, B., Sturt, C., Bennett, Introducing a Moodle LMS in higher education: S., Gray, K., Waycott, J., & Judd, T. …Chang, R. The e-courses experience in Ticino. Retrieved on (2009). Educating the Net generation. A handbook June 21, 2010, from http://www.elearninglab.org/ of findings for practice and policy. Australia: wp-content/ uploads/ 2010/ 04/ jelks_ecourses_ Australian Learning and Teaching Council. paper.pdf Kubora, K., Terashima, K., Nakahashi, Y., & Mo- Burlea, A. (2008). The complexity of an e-learning rioka, H. (2008). Analyzing learning environments system: A paradigm for the human factor, The of distance education in Japan, distance learning Inter-Networked World: ISD theory, practice and and the Internet, Tokyo, Japan. education (Vol. 2, pp. 267–278). New York, NY: Springer-Verlag. Kwak, H., Fox, R. & Zinkhan, G. (2002). What products can be successfully promoted and sold via the Internet? Journal of Advertising Research, Jan.-Feb.
266 Differences in Internet and LMS Usage
Landsberger, J. (2004). Thoughts on convergence Nachmias, R., Mioduser, D., & Shemla, A. (2001). in instructional settings. TechTrends, 48, 6–8. Information and Communication Technologies doi:10.1007/BF02763348 usage by students in an Israeli high school: Equity, gender, and inside/outside school learning issues. Leal, J. & Queirós, R. (2010). From e-learning Education and Information Technologies, 6(1), systems to specialised e-learning services. Chap- 43–53. doi:10.1023/A:1011367212148 ter of EduJudge project book - A new learning paradigm: Competition supported by technology. NYU. (2010). New York University. Retrieved from http://www.nyu.edu/ Lopes, C. (2007). Evaluating e-learning readiness in a health sciences higher education institution. Oliveira, P., Oliveira, C., Souza, F., & Costa, N. In M. Baptista Nunes & M. McPherson (Eds.), (2006). Teaching stategies to promote active learn- Proceedings of the IADIS International Confer- ing in higher education: Current develepments ence e-Learning 2007 (vol. 1, pp. 59-67). in technology-assisted education, FORMATEX 2006, (pp. 636-640). Retrieved on September7, Madell, D., & Muncer, S. (2004). Gender dif- 2010, from http://www.formatex.org/ micte2006/ ferences in the use of the Internet by English pdf/ 636-640.pdf secondary school children. Social Psychol- ogy of Education, 7, 229–251. doi:10.1023/ Prensky, M. (2001). Digital Natives, Digital Im- B:SPOE.0000018552.85903.4d migrants. MCB University Press. Mayadas, F. (2007). Online education: Worldwide Rodrigues, A., Pinto, M., & Queirós, R. (2010). and moving quickly. In Technology- Enabled Grau de Preparação e Utilização do E-learning Education (pp. 57-60), Proceeding of the 2007 no Ensino Superior. Estudo de Caso na ESEIG- MIT LINC Conference, Dubai, UAE. IPPorto. Setúbal, Portugal: Conferência Investi- gação e Intervenção em Recursos Humanos. Minhsein, H. (2005). Integrating Internet into traditional education: A practical study. Simões, L. & Gouveia, L. (2009). Geração Net, Web 2.0 e Ensino Superior. Cadernos de Estudos Misko, J., Choi, J., & Lee, I. (2004). E-learning Mediáticos, 21-32. in Australia and Korea: Learning from practice. Korea Research Institute for Vocational Educa- Sink, C., Sink, M., Stob, J., & Taniguchi, K. tion & Training. Retrieved on June 19, 2010, (2008). Further evidence of gender differences in from http://www.ncver.edu.au/ research/ core/ high school–Level computer literacy - . Chance, cp0306.pdf 21(1), 49–53. doi:10.1007/s00144-008-0011-9 Monsakul, J. (2007). Learning management sys- Tapscott, D. (1999). Growing up digital: the rise of tems in higher education: A review from faculty the next generation. New York, NY: McGraw-Hill. perspective, 4th International Conference on E- Tastle, J., White, A., & Shackleton, P. (2005). learning for Knowledge-Based Society, Bangkok, E-learning in higher education: The challenge, Thailand. effort, and return of investment. International Moodle. (2010). Current Moodle statistics, Journal on E-Learning, 241–251. Moodle.org. Retrieved on May 27, 2010, from http://moodle.org/stats.
267 Differences in Internet and LMS Usage
Uzunboylu, H., Ozdamli, F., & Ozcinar, Z. (2006). Casella, G., Costagliola, G., Ferrucci, F., Polese, An evaluation of open source learning manage- G., & Scanniello, G. (2007). A SCORM thin client ment systems according to learners tools. Online architecture for e-learning systems based on Web Submission. Retrieved December 28, 2010, from services. [Hershey, PA: IDEA Group Publish- http://www.eric.ed.gov/ ERICWebPortal/ content- ing.]. International Journal of Distance Educa- delivery/ servlet/ ERICServlet?accno =ED494265 tion Technologies, 5(1), 13–30. doi:10.4018/ jdet.2007010103 Walker, J. & Jorn, L. (2007) Net generation students at the University of Minnesota, Twin Cities Dagger, D., O’Connor, A., Lawless, S., Walsh, E., Student Educational Technology Survey 2007. & Wade, V. (2007). Service-oriented e-learning platforms: From monolithic systems to flexible Williams, J., & Goldberg, M. (2005). The evolution services . IEEE Internet Computing, 11(3), 28–35. of e-learning. [Global.]. Universitas, 21. doi:10.1109/MIC.2007.70 Davis, B., Carmean, C., & Wagner, E. D. (2009). The evolution of the LMS: From management to ADDITIONAL READING learning - Deep analysis of trends shaping the Al-Smadi & Gutl. (2010). SOA-based architecture future of e-learning. Sage Road Solutions. LLC. for a generic and flexible e-assessment system. Donello, J. (2002). Theory & practice: Learning In EDUCON’10. content management systems. Alario, C., & Wilson, S. (2010). Comparison of Ellis, R. K. (2009). Field guide to learning man- the main alternatives to the integration of external agement systems. ASTD Learning Circuits. tools in different platforms, ICERI2010 Proceed- ings, (pp. 3466-3476). Friesen, N. (2005). Interoperability and learning objects: An overview of e-learning standardiza- Apostolopoulos, T. K., & Kefala, A. (2003). An tion. Interdisciplinary Journal of Knowledge and e-learning service management architecture. In Learning Objects. Proceedings of the 3rd IEEE International Con- ference on Advanced Learning Technologies (pp. Hall, B. (2003). Learning management systems 140-144), Athens, Greece. and learning content management systems de- mystified. Bohl, O., Scheuhase, J., Sengler, R., & Winand, U. (2002). The shareable content object reference Harasim, L. (2006). History of e-learning: Shift model (SCORM)-A critical review, Proceedings happened - The international handbook of virtual of the International Conference on Computers in learning environments. Germany: Springer. Education (950-951). Harman, K., & Koohang, A. (2007). Learning Bryden, A. (n. d.). Open and global standards for objects: Standards, metadata, repositories, and achieving an inclusive information society. LCMS - Informing Science Institute. Edição de Informing Science. Burlea, A. (2008). The complexity of an e-learning system: A paradigm for the human factor: The Hatala, M., Richards, G., Eap, T., & WilLMS, Inter-Networked world - ISD Theory, Practice J. (n. d.). The interoperability of learning object and Education, 2, 267-278. New York, NY: repositories and services: Standards, implementa- Springer-Verlag. tions and lessons learned. Proceedings of the 13th international World Wide.
268 Differences in Internet and LMS Usage
Holden, C. (2004). What we mean when we say KEY TERMS AND DEFINITIONS “repositories.” User expectations of repository systems. In: Academic ADL Co-Lab. JORUM eLearning: A new form of learning based on team (2006). E-learning repository systems re- technology. A model of teaching and learning search watch. Retrieved December 13, 2010, from based on the online environment, leveraging the http://www.jorum.ac.uk/ docs/ pdf/ Repository_ capabilities of the Internet for communication and Watch_ final_ 05012006.pdf. content distribution LMS: System used mainly by teachers and Kennedy, G., Dalgarno, B., Sturt, C., Bennett, students to manage the teaching/learning process. S., Gray, K., Waycott, J., & Chang, R. (2009). Nowadays these type of systems occupy a central Educating the Net generation. A handbook of find- role in the eLearning realm coordinating and in- ings for practice and policy. Australia: Australian tegrating a set of features by third-party systems. Learning and Teaching Council. LO (Learning Objects): Units of instructional Martínez, J. Á., & Navarra, P. L. (2007). Con- content that can be used, and most of all reused, tent interoperability on e-learning platforms: on web based eLearning systems. Usually it’s a Standardization, digital libraries, and knowledge set of files including a XML manifest wrapped management. Revista da Universidad y Sociedad with resources and packaged in a ZIP file. del Conocimiento. Metadata: Data about data. Metadata will allow systems to index, search and retrieve files Nichani, M. (2009). LCMS = LMS + CMS based on semantic data. [RLOs] – How does this affect the learner? The Net Generation: For the majority of the au- instructional designer? Retrieved from http:// thors the “Net generation” are all the people born www.elearningpsot.com/articles/archives/lcms_ after 1980. These people access to new technolo- LMS_cms_rlos. gies as they have always existed showing higher Rehak, D., & Mason, R. (2003). Engaging with capacity of use and self-efficacy. the learning object economy . In Littlejohn, A. Web 2.0: A new Web paradigm enhancing (Ed.), Reusing online resources: A sustainable the bidirectional communication between client approach to e-learning (pp. 22–30). London, (using a web browser) and web servers. This new UK: KoganPage. paradigm augments the capacity for the user to product, collaborate, share and recommend using Tastle, J., White, A. & Shackleton, P. (2005). Web 2.0 tools. E-learning in higher education: The challenge, Web 2.0 Tools: Tools that proliferate on the effort, and return of investment, International Web that enhance the production, collaboration Journal on ELearning. and sharing of documents. Some examples of these Walker, J. & Jorn, L. (2007). Net generation stu- tools are the Wikis, Blogs and Social Networks. dents at the University of Minnesota. Twin Cities Student Educational Technology Survey 2007. ENDNOTES Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, , 21. 1 http://www.eseig.ipp.pt 2 http://www.eseig.ipp.pt/ 3 http://www.estsp.ipp.pt/ 4 http://www2.estgf.ipp.pt/
269 Differences in Internet and LMS Usage
5 http://www.iscap.ipp.pt/ in 2010, in our opinion, Facebook would be 6 http://www.isep.ipp.pt/ the most used social network. 7 These numbers do not include ESTSP. 8 Hi5 is a social networking website (http:// hi5.com/) very popular among Portuguese young people. If the study was conducted
270 271
Chapter 13 LMS Adoption at the University of Genova: Ten Years After
Maura Cerioli University of Genova, Italy
Marina Ribaudo University of Genova, Italy
Marina Rui University of Genova, Italy
ABSTRACT The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy.
INTRODUCTION buy goods, etc. The Web – the largest cognitive artefact ever built – offers an easy access to data, The advent of the Web in the mid-1990s has dra- their sharing, and collaborative elaboration, and matically changed the way people communicate, this new opportunity has also an important impact look for information, produce and share textual and on education. Indeed, the different forms of com- multimedia content, access to (online) services, munication, cooperation, and knowledge building, which are central in modern theories on education, DOI: 10.4018/978-1-60960-884-2.ch013
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. LMS Adoption at the University of Genova
are not anymore limited to classroom activities. model to measure LMS usage. The five levels But, they can be moved online thanks to an easy span from Level 0, corresponding to no use of the access to network technologies. To prompt these LMS, to Level 4 in which students’ activities are new forms of interaction, in the last two decades, recorded and made accessible to all students for schools, universities, and companies have started review, or made available in real time to remote offering LMS (Learning Management Systems) to participants. Surprisingly, this classification does the different actors involved in education. not include collaborative group activities, which The adoption of LMS at the university level indeed started to appear in the recent years. In this may follow two approaches, most often with metric, the University of Genova is at Level 3. In the early adopters evolving from the first to the addition to course syllabus and reading materials next. From the one hand, single departments (Level 1), assignments and assessments (Level 2), or degree programs may develop or customize the students also find forums for asynchronous their own LMS, offering students and teachers a communication and online tests to evaluate their supplement to the traditional educational process. preparation, although with different numbers This approach leads in time to a plethora of dif- depending on their curricula. ferent systems coexisting in the same institution, After a short analysis of background literature often using different technologies, maintained in Section 2, this chapter describes in Section 3 by separate groups of technical staff; thus, it is the adoption of the central LMS software platform hardly scalable. This was, indeed, the situation at the University of Genova, from the early ex- for the case study of the University of Genova, periences to the present. Data extracted from the up to 2004. On the other hand, the adoption and log files of the LMS are presented together with management of a LMS may be centralized for all a project, focussing on instructional design and the substructures. At the University of Genova, ICT supported learning. In Section 4 the findings such a centralized service, called AulaWeb, was obtained by two surveys administered respectively introduced in the academic year 2004/05 and is to faculty members and students are discussed. currently supporting education for all the Faculties Finally, Section 5 presents conclusions and future of the University, as this chapter will illustrate. research directions. Measuring the achievements of an institution in implementing technology enhanced learning, being completely online or a blend of face-to-face BACKGROUND and Web-based activities, is difficult. Indirect and partial information may be obtained by a Learning Management Systems have become systematic analysis of LMS usage. Indeed, if an important resource for Higher Education the activities are taken online, the kind of tools Institutions willing to modernise their curricula used and the interactions between students and and teaching methods as well as to widen their system give a rough measure of involvement. audience by attracting new students, geographi- Unfortunately, according to (Janossy, 2008), even cally remote or adults, who could not otherwise quantitative analyses of LMS usage are generally attend traditional lectures. This is witnessed by difficult to obtain, since those available rely on the rich literature on e-learning, ranging from incomplete log data or ad hoc surveys. One of the proposal of different pedagogical models, to the major contributions of this chapter is pre- the description of LMS technical development, senting a large amount of data about the usage adoption, qualitative and quantitative usage, to of LMS across all the Faculties of the University the analysis of success case studies at the school, of Genova. Janossy (2008) proposes a five levels university or company levels.
272 LMS Adoption at the University of Genova
Although researchers in the field of education of evaluating students are receiving low to moder- claim that technology enhanced learning should ate use, particularly with quizzes. Tools in other not focus on technologies but on learners, this categories are receiving very little use. does not necessarily always happen. As pointed out, for example, in (McGee & Diaz, 2007), “tech- Considering the research described in these cat- nologies are often selected for arbitrary reasons egories, it appears that, as a group, instructors that have little to do with teaching and learning are in an early phase of CMS adoption. Arguably, or the students they serve” (p. 38). Research by they are using a CMS in familiar ways, such as Dror (2008) supports the same claim: transmitting information or completing grades. (Malikowski et al, 2007, p.167) It is naïve and would be a mistake to think, let alone take for granted, that using technology per As we will discuss later in this chapter, the se will enhance learning. In fact, using technology case study of the University of Genova fits these often hinders learning because it does not match conclusions. and fit well the cognitive system. (p. 218) The adoption of a centralized LMS for the whole university poses many challenges. At the When the traditional learning material is simply technical level, the administration of the single transformed into digital form and presented to software platform is demanding, because of its the learners, they remain passive and not really large community of users; at the organizational engaged in the process of knowledge acquisition. level, proposals, decisions and strategies need In contrast, when learners are active and moti- to be shared and clearly communicated to the vated, participating and interacting, then learning university community; at the educational level, is maximised. In this case study, we will see how faculty members need to be informed of the at the University of Genova the homogeneity of advantages they might achieve adopting ICT for technological support is not entailing similarity education, and those who are convinced need to of learning models across the different programs. be trained; at the learning level, even if today’s Malikowski, Thompson and Theis (2007) students are “technology addicted”, or Digital proposed a model to study LMS not only from a Natives (Prensky, 2001), they do not really seem technological point of view but also considering accustomed to use technology for learning. For the pedagogical aspects. The authors select five example, McNaught, Lam and Ho (2009) analyse categories - transmitting course content; evaluat- the use of a wide range of digital technologies, ing students; evaluating courses and instructors; showing that students are indeed “digitally ready” creating class discussions; and creating computer but “there is no strong empirical evidence that based instruction – and describe them introduc- students are committed to eLearning, that is, to ing the LMS features for each category but also using the technology-based tools they know how to providing recommendations from an educational use with the intention of maximizing their learning point of view. They compare the use of LMS experience and outcomes” (p. 655). (called CMS in their work) in different American Success factors in implementing e-learning institutions and summarize their results as follows: programs are deeply investigated in the literature. Volery and Lord (2000) observe that re-imple- The most prominent use of a CMS is to transmit mentation of conventional models borrowed from information to students. A category showing mod- passive face-to-face transmission does not work erate use is creating class discussions, primarily in the Internet based setting and can bring only asynchronous discussions. Tools in the category minor improvements. Three critical success fac-
273 LMS Adoption at the University of Genova
tors are identified in online delivery: the ease of building knowledge and the network is no lon- access to the technology, the instructor attitudes ger a mere tool for content distribution but it is towards students and his technical competences, rather a facilitator for the interaction among the and the students’ previous use of the technology. participants involved in the educational process. O’Neill, Singh and O’Donogue (2004) discuss the Downes (2005) coined the term e-learning 2.0 issues associated with the global competition of to denote the need of changes in the pedagogical Higher Education that, thanks to the development models in the first decade of the new millennium. of new delivery and communication methods, has extended beyond national boundaries. Tradi- The model of e-learning as being a type of content, tional universities have opportunities to change produced by publishers, organized and structured educational methods, and those who will miss into courses, and consumed by students, is turned the appointment will become less competitive. on its head. Insofar as there is content, it is used Samarawickrema and Stacey (2007) cluster the rather than read— and is, in any case, more likely factors that enable and impede the adoptions of to be produced by students than courseware au- a LMS into five categories: institutional reasons; thors. And insofar as there is structure, it is more technology reasons; influence of colleagues and likely to resemble a language or a conversation network; influence of research and the literature; rather than a book or a manual. (Downes, 2005) and personal reasons. Time constraints, heavy workload, rewards structures, professional devel- Dalsgaard (2006) suggests that it is neces- opment, policies at the university and faculty levels sary to move e-learning beyond LMS. These are all issues that should be carefully addressed software platforms still maintain the main role when deciding to adopt a LMS. MacKeogh & of course administrators (e.g., managing students Fox (2009) provide similar conclusions present- enrolments, exams, assignments, course materi- ing drivers and barriers that could emerge when als, etc.), but the active learning process can be trying to embed e-learning in the educational effectively obtained by using educational social strategies. Some of these barriers like the lack software such as individual collaborative tools to of funding, of strategies at the university level, build knowledge and social networks to engage and of recognition in terms of career for the time in different forms of communication. spent in teaching activities are also preeminent Successful case studies exist within this line of in this case study. research, mostly experiences involving research- The discussion above poses the emphasis ers in the educational and technological fields. For on LMS in a context that we can associate with instance, Ullrich, Borau, Luo, Tan, Shen, and Shen the Web 1.0 that emerged in the mid-1990s and (2008) analyse technological principles of Web 2.0 expanded access to (open) information. More from a learning perspective and two case studies, recently, new research activities have emerged respectively based on micro-blogging and social laying their roots in the so-called Web 2.0, spread bookmarking, are presented. To the best of our out in the past few years starting from the seminal knowledge, the advanced experiences presented presentation of O’Reilly (2005). If Web 1.0 was in the literature involve groups of early adopters about consuming information, Web 2.0 is about and the transition from the more traditional use of producing and sharing it. Tools such as blogs, LMS to real e-learning 2.0 is still in its infancy. wikis, social networks, tagging systems, and Moving from the small scale of the early adopters mashups are examples of user-centric applications to the large scale of the whole institutional level that emphasize users’ participation and collabora- will be a slow process. tion. The user becomes active in the process of
274 LMS Adoption at the University of Genova
TEN YEARS OF EXPERIENCE forums. Such pioneering efforts involved mainly some faculty members - the early adopters - of The University of Genova is a traditional university Computer Science, Engineering and Pedagogical offering face-to-face courses. However, in the last Sciences. On the other hand, almost at the same ten years, it has made several efforts to improve time, the University joined two consortia, Tethys the quality of its educational process by the in- (http://www.tethys-univ.org/) and Nettuno (http:// troduction of ICT support. These attempts began www.consorzionettuno.it/), both promoting the as personal initiatives by individual faculty. But, development of distance learning. Though the they quickly scaled up to the level of Departments, involvement was officially at the level of the and finally of the overall University. overall institution, the activities on these two The many centralized and somehow top- networks have been mostly left to the volunteer- down initiatives notwithstanding, the University ing commitment of individual research groups. of Genova still misses an official policy in ICT The first really centralized effort took place educational-support, badly hindering its adoption. in 2001, when the University participated in the Without such a policy, the level of commitment national project CampusOne. This project was in terms of funding and personnel is unclear, focused on two main goals: supporting reorgani- making effective planning impossible. More- zation of the degree structure in the spirit of the over, as rewards for faculty members spending Bologna Declaration, and promoting technologi- their time on ICT-supported teaching, so far on a cal innovation within the educational process. A totally voluntary base, cannot be expected, their specific committee was appointed to work on this enthusiasm is inevitably dumped. Thus, the lack latter action. At that moment, many universities in of a central policy in ICT educational-support is Europe were facing the “buy-or-build dilemma” a main strategic mistake, as discussed in litera- and a survey conducted in more than one hundred ture (see e.g., (Samarawickrema & Stacey, 2007; European institutions (Paulsen, 2003) revealed MacKeogh & Fox, 2009)) and also pointed out the adoption of 52 different commercial and 35 during a certification procedure undergone by the self-developed LMS. On the open source side, University of Genova on the extent of ICT support Claroline (http://www.claroline.net/) seemed (project UNIQUe, http://unique.europace.org/). the most eligible, and it was taken in consider- The results, based on peer review, were mostly ation for a while. But, in the end, the committee good, but for the lack of a strategy of eLearning. decided to build from scratch, because the avail- For this topic, the need of implementing a central able platforms seemed to be not fully SCORM policy plan on e-learning to be integrated into the compliant (http://www.adlnet.gov/Technologies/ general University quality plan has been stressed, scorm/default.aspx), while this was perceived, at together with the need of increasing the focus on that time, as a strong requirement. Moreover, the the quality of communication. prospective users judged the available platforms insufficiently appealing. Then, each Faculty ap- Past pointed a few members as representative users in the user-centered development of a centralized Since the late nineties, the University of Genova service for ICT support to education. As require- has hosted and facilitated a certain amount of ments addressed the teachers’ needs, the platform spontaneous experiences in the field of ICT was expected to be extremely convenient to use supported learning. On the one hand, a few in- by them, hopefully leading to a large number of dividual research groups experimented with the adopters. distribution of material via Web, and the use of
275 LMS Adoption at the University of Genova
In retrospect, the approach was naïve: first of the final choice was driven by two main concerns: all, students were not included in the stakehold- stability and low-cost. Indeed, the end users made ers; secondly, including the requirements of all clear that further changes of platform in the next Faculties inevitably forced the development of a few years would not have been acceptable, and general purpose platform, no more appealing for the extremely limited economical resources were any specific user than the already existing ones; absorbed by the service technical support. Thus, finally, and most relevantly, the code production the committee decided to move to Moodle (Cole exceeded the available resources, so that the tool & Foster, 2007), because its large community was not completed within the project, making its of active developers guaranteed that it would be adoption problematic, even as an experiment. supported in the years, and because it was free Besides these “official” initiatives, some of charge. Moreover, as Moodle is open source, other platforms were adopted (or their use was tailoring of, and additions to, the platform could increased) by individual courses or programs, be seamlessly integrated, providing a natural mostly following a blended learning style, since way to address the specialized requirements by the activity around ICT supported learning was the individual end users. In retrospect, the choice quite lively in those years. made was sound. Indeed, nowadays Moodle is still These early (official and unofficial) experi- maintained, its diffusion is growing to the point of ences consistently showed both that students both competing with the giant of the LMS market greatly appreciated the tool support, and that the BlackBoard (http://www.itcnetwork.org) and hav- (non negligible) effort required on the teacher ing being adopted by leaders of distance learning, side had a good return of investment in terms of such as The Open University (http://learn.open. students’ participation and satisfaction. ac.uk/mod/oublog/view.php?user=27287). The Moodle-based centralized service was Present called AulaWeb (Ribaudo & Rui, 2009). Following the same policy already adopted At the end of CampusOne, in 2004, the Univer- for CampusOne, joining AulaWeb for faculty sity nominated a new committee to supervise the members has been, and still is, exclusively volun- adoption of a sustainable centralized LMS for all tary. In particular, the production of educational Faculties. From the very beginning, it was evident contents is left to the initiative of the individuals, that completing and maintaining the current cus- within the scope of their capabilities and time tom made LMS was not feasible due to limited availability. Moreover, a few professors from resources. Hence, the first task of the committee each Faculty accepted to act as supplementary was to select a standard platform to be adopted. administrators, to alleviate the burden of the The requirements collected from the end users undersized technical staff (initially two people, were mostly basic (support for forums, document currently just one) devoted to AulaWeb. These repositories, polling, quizzes and wikis); thus, all administrators filter the needs of their users, and the analyzed LMS met them. On the other hand, interpret the peculiarities of their own area of some users had a broad variety of very special- education. Initially, they had also the task of dis- ized requirements which no single platform was seminating the use of the platform among their simultaneously able to satisfy. Since the choice of faculty members, while currently they just provide supporting only some of them, privileging some the first level of help desk. The technical staff users over others, would be politically unaccept- takes care of the more advanced levels, manages able, the committee elected to ignore the specific and updates the platform. Moreover, the staff requirements in the selection process. Therefore, encouraged Moodle adoption by a few seminars
276 LMS Adoption at the University of Genova
on its tools to facilitate the use of the platform by minority, and most of those available are poor. ICT beginners. These seminars were a concrete This is, however, unavoidable, unless the efforts help to limit the a-priori rejection of the platform voluntarily spent on AulaWeb have a place in the and overcome the heterogeneity of the involved evaluation of faculty members, as they ask, or their users – different in terms of education, age, and use of AulaWeb is made mandatory; either action command of ICT technologies. can be taken only on the basis of a central policy In a short time all the Faculties of the University and should burst the use of AulaWeb, extending were present on the portal, although with different its users’ base to the students of currently unsup- numbers of online courses and active users. The ported courses. participation had a significant growth, as shown in Figure 1: the number of activated courses, in AulaWeb in 2010 the histogram on the left, increased quickly and, as a consequence, the number of enrolled students In August 2010, more than two third of the stu- grew up continuously. dents (26280 out of 36835) were enrolled, along In August1 2009, AulaWeb underwent a struc- with the majority of faculty members (844 out of tural reorganisation; all users were reset and only 1518, that is, 55.6%). AulaWeb hosted about 120 the active ones registered again. It is interesting degree programs out of 143, for 1125 different to note that, in a short time, the number of students subjects. During the week there was an average enrolled on the new portal, depicted by the dashed of 3800-4000 distinct users’ accesses in the 24 line in the diagram on the right of Figure 1, reached hours, with more than 100 users simultaneously the same size of those already registered on the connected, showing the vitality of its community. old portal. The data become, however, less encouraging This fact witnesses that AulaWeb has now when we investigate the use of the different Moodle reached a number of users close to saturation, at tools. Figure 2 shows the use of AulaWeb: data are least in the current setting. In order to significantly split by considering all the Faculties and the two improve the use of ICT for education, a central “technological” subgroups formed by Computer policy is strongly needed. Indeed, many students, Science (part of the Faculty of Science) and ICT in their questionnaires, presented in Section 4, Engineering (part of the Faculty of Engineering). complain that the courses on AulaWeb are a The left histogram represents the percentages of
Figure 1. Growth of AulaWeb
277 LMS Adoption at the University of Genova
online courses and online students weighted with courses, because those subjects are of interest for respect to the overall number of courses and stu- any kind of curricula. dents of the University. The last two data in the So far, we have discussed the quantitative series (the rectangles with thick borders) show impact of AulaWeb. Let us now analyze the of- the same percentages when considering the whole fered courses from a qualitative viewpoint. As University, thus providing the average trend. It shown on the right of Figure 2, content repository is clear from the histogram that some Faculties courses, that is, those offering only downloading are more “active” than others. If we consider the of educational material, are still (in percentage) courses, then we have, as it could be expected, a the large majority, with the Faculty of Law offer- clear “winner” in the Computer Science program, ing only such kind of courses. Only in two cases, followed by the Faculties of Economics and En- Architecture and Computer Science, the percent- gineering. The numbers of students are not even age of blended courses, that is those offering roughly proportional to those of courses; for not only educational material, but also thematic instance, Sciences and Engineering swap places. forums and, in many cases, also other types of This phenomenon is due to the average number online activities, exceeds that of content reposi- of students per course in different Faculties. In- tories. Some Faculties also offer organizational deed, in many cases the most populated courses courses, giving only support for administration of were the first to move to AulaWeb, because their traditional courses, like for instance registrations large number of students made them difficult to to in-presence examinations2. manage by traditional means. So, in the Faculties In more detail, data collected in Figure 3 show where few courses moved to AulaWeb the average the activities on AulaWeb normalized by the of- number of students is higher. Moreover, since each ficial number of courses for each Faculty. The last student has many courses, the number of students is columns in the table show the average values and naturally logarithmic over the number of courses. standard deviation. Here and in the following data Notice that the percentages of students enrolled comparisons, we consider as worth of notice values on AulaWeb for Foreign Languages and Com- differing from the average by more than the stan- puter Science are higher than 100%. This is due dard deviation, and show with a gray background to the fact that, in both cases, also students from those below, and in a larger bold font those over other Faculties are attending some of their such limits. The significance of this statistical
Figure 2. AulaWeb Usage and course typology
278 LMS Adoption at the University of Genova
analysis is somewhat limited by the small number age of courses actually offering those activities of points in the series and their large spread, as (AW-Log series). The data for students and teach- shown by standard deviation measures close or ers come from recent questionnaires, described in even larger than the average. However, some data detail in Section 4. The AW-Log data show that are outstanding by any standard, like for instance content distribution is indeed predominant (more the number of resources for the Computer Science than 7 times the second higher value), followed by program, which is four times the average and more comparatively similar values for activities strongly than twice the second best value. related to the organization of traditional courses, Some results, like the preeminence of the (i.e., assignments and reservations3), as well as to Computer Science program, or the better use of communication (i.e., discussions in subject related quiz and thematic forum by the Faculty of Educa- forums4). Activities involving collaboration (i.e., tion, which is also consistently over average on wiki and glossaries) are by far less extensively all activities, or, vice versa, the poor quality of used. By comparing the different series in the the Faculty of Literature were to be expected. histogram, the users seem to acknowledge more Some other achievements, like for instance the interaction on all activities, but resource download- very good use of assignment and quiz by the ing, than it could be expected by the percentage Faculty of Foreign Languages, are mostly due to of courses offering them (series AW-Log). We the individual efforts of some enthusiastic early interpret this discrepancy as a confirmation that adopters, who not only use the tools, but also students and teachers using advanced features disseminate the good practice among their col- are more involved in AulaWeb, more familiar leagues. with it, hence more willing to participate into the Finally, Figure 4 compares how students questionnaires. Thus, we assume these statistics (S-Quest series) and teachers (T-Quest series) to be somehow biased. Another interesting point perceive their activities on AulaWeb to the percent- is that students’ involvement both in thematic
Figure 3. Activities on AulaWeb (empty cells stand for 0)
279 LMS Adoption at the University of Genova
forums and wikis is much lower than teachers’ University teachers, in fact, rarely come into participation, while for the other features they are contact with educational models, as they are comparable. These data confirm the many teach- required to be expert only on their own subject, ers’ complaints that students are not eager to share their career is dependent mostly on their research and build knowledge and that often collaborative achievements and courses on teaching techniques activities are ignored. are not part of any program but those on Education. Thus, they usually come up with personal varia- The WEL Project tions of the educational process they experienced as students, mostly based on in-presence lessons. As discussed so far, from the very beginning the For a large majority the course design still consists use of AulaWeb has seen the download of teach- of selecting the topics to be covered and ordering ing material as prevalent activity. In the winter them in a logical stream, without planning activi- of 2007/08 a quick survey showed that the main ties to better convey the relevant aspects of each cause of the scarce use of more sophisticated tools topic, or to involve and evaluate students. was not, in most cases, the “fear of technology”. Therefore, in April 2008 the action WEL Indeed, thanks to the basic dissemination made by (Web Enhanced Learning) was promoted, to the AulaWeb staff, the needed knowledge was, in improve the quantity and especially the quality fact, available to all interested faculty members. of Web-supported teaching in the University. Its The main problem appeared to be their teaching main goal was devising a model for the transfer style, mostly based on in-presence lessons and of instructional design knowledge and skills to totally content-driven. They did not feel the need subject-oriented university teachers. for technical support of collaborative activities The WEL course, delivered by ITD (the Insti- simply because such kind of activities was not tute for Didactic Technologies of CNR) experts part of their teaching process. (Puddu, Repetto & Vallarino, 2008), was driven by hands-on projects: each participant was indeed
Figure 4. Activities on AulaWeb
280 LMS Adoption at the University of Genova
required and helped to redesign his/her course, In the second phase (May-June 2008) two including some form of collaborative activity. individual face-to-face meetings with CNR co- WEL has been organized into three distinct designers took place. Each participant had been phases. asked to plan her course in terms of macro-design In the first phase (May 2008) two plenary (i.e., goals and objectives definition, evaluation lectures showed to university teachers (1) an criteria, types of activities, first version of the overview on the use of ICT technologies for edu- course guide) and micro-design (i.e., providing a cational methods and (2) the instructional design detailed storyboard of the course with a descrip- methodology they were going to follow to redesign tion of the modules, planned activities, initial their teaching strategies (see, e.g., Trentin, 2001). messages for the virtual class, activity scheduling). About 100 faculty members attended those In parallel, on demand technical course have sessions and, among them, 48 decided, at the been organized for small groups of participants. end of the second lecture, to commit themselves Various software tools have been presented, in- to the project. The ICT competencies claimed by cluding some for content production, graphic and the participants were mostly average (64%), with video editing, as well as some Moodle modules. a minority of experts (22%) and a few beginners The third and last phase started in September (13%). The large majority (82%) had previous 2008 with the last face-to-face meeting with CNR experiences in using LMS and almost 50% had co-designers: a sort of audit to check the work already used AulaWeb, though mostly without done during the summer to create the learning any (formal) formation. materials/strategies based on the design phase. There was a high variety of subjects, from The third step ended with the launch of the revised humanistic to scientific disciplines; the distribu- first semester courses. tion of participants with respect to their Faculty, A final plenary meeting was organized in normalized by the number of staff members of February 2009 to share the experiences of the each Faculty is shown in Figure 5. first semester courses, and take into account the
Figure 5. Distribution of faculty members in WEL
281 LMS Adoption at the University of Genova
lessons learned to slightly adjust those of the is still given to resources distribution, something second semester. has changed when considering collaborative ac- It is worth noting that the participation was tivities. Indeed, we can observe that the usage of (again) on a voluntary base. The participants’ tools like wikis and glossaries has increased in number has been limited for two reasons. On the 2008/09, in most cases with a percentage of growth one hand the instructional design experts were able much larger than that of the courses and the to individually train only a few faculty members. other resources. Moreover, these numbers suggest On the other hand, even interested professors were that also the approach to the use of the tools is not sufficiently encouraged into participation, be- improving. For instance, not only the number of cause of the lack of recognition, and insufficient wikis has increased of almost 75%, but the num- communication (both problems due to the missing ber of pages and versions has dramatically grown, policy for e-learning). Indeed, among the respon- about 4 times the percentage of the wikis them- dents of the questionnaire T-Quest, described in selves! Thus, we can infer that not only do we Section 4, 181 (out of 210) answered that they have a larger number of wikis, but also that they did not participate in the WEL project, because: are used differently, with more online activities “Did not have time” (92, that is, 43.81%), “Did going on. not know about the project” (45, that is, 21.43%), In general, all the teachers involved in the “Thought it was not useful” (20, that is, 9.52%). WEL experiment reported an improvement of While the remaining 24 (11.43%) did not answer. their courses in the number of active participants, Information about WEL disseminated through passed exams and even average scores. Moreover, “word of mouth” within the faculty members most of the professors acknowledged that their community since the majority of the respondents teaching styles had changed, and this was, in fact, of T-Quest answered that they might consider par- reflected by the numbers on AulaWeb. The findings ticipating to a second edition of the course, with presented here are consistent with other analysis 59“Yes” (28.10%), and 97“Maybe” (46.19%), from the literature. For instance, the interested while only 35 (16.67%) answered “No”, and the reader can see (Yueh & Hsu, 2008), where the remaining 19 (9.05%) did not answer. authors describe the adoption process of a LMS, Data shown in Table 1 compare aggregate built by an internal team, and the support offered numbers extracted from AulaWeb logs and can to professors. Though they are not using Moodle, be used to get some insights on the changes in the the approach followed to increase acceptance and use of AulaWeb in pre- and post-WEL periods. usage of their platform is remarkably similar to Comparing the data in Table 2, it is quite clear that adopted in Genova. Their results, showing that, though the lion’s share of the online activity that most frequently used functions are indeed the
Table 1. Pre- and post-WEL
07-08 08-09 07-08 08-09 Courses 822 1,016 +23.60% Resources 10,622 14,954 +40.78% Quizzes 135 199 +47.41% Forums 1,296 1,452 +12.04% Wikis 51 89 +74.51% - Threads 7,917 10,133 +27.99% - Pages 302 1,115 +269.21% - Posts 24,628 30,856 +25.29% - Versions 1,899 8,201 +331.86% Assignments 692 949 +37.14% Glossaries 5 12 +140.00%
282 LMS Adoption at the University of Genova
basic ones, and that supporting professors with more than in actual limitations of the tools, or a team of instructional specialists has increased insufficient technical skills of the students. This the use of their LMS, are in accordance with the experience seems to be quite in contrast with the findings of the WEL project. main claims in the literature that students are Comparing the numbers of 2008/09 to those used to, and insist on, sharing knowledge, as for of 2009/10, we can see that courses, resources instance in (Prensky, 2007). However, it seems and forums have slightly increased (respectively to be perfectly coherent with the experiments in from 1016 to 1125 (+11%), from 14954 to 18464 (Ebner et al., 2008), where optional wikis, with (+23.5%), and from 1452 to 1478 (+2%)). On the the only reward of being allowed to use them at contrary, quizzes, wikis, assignments and glos- examination time, were left totally empty, and saries have decreased (respectively from 199 to with the observation in (O’Neill, 2005) that un- 147 (-26%), from 89 to 62 (-30%), from 949 to less there is a prized reward for it, and that clear 863 (-9%), and from 12 to 6 (-50%)). Part of this responsibilities are attributed to individuals, wikis failure may be due to the update of the platform, are simply not used by students. and the resulting cleanup of obsolete data. But, Moreover, on the unofficial students’ sites, as courses, resources and forums have neverthe- the personal notes are always provided as docu- less increased, still we can identify a difficulty ments, sometimes even at payment, and never with the more advanced features, which is not collaboratively developed. Thus, it seems quite surprising. Indeed, lacking a follow-up of WEL clear that students are simply not interested in (the promised second edition never started), cooperative efforts. Sharing knowledge means to any reward for the faculty members, and a clear them sharing individually developed documents. appreciation from the students (who deserted Collaborative activities are successful only if collaborative activities as much as possible), the participation is compulsory, though after being enthusiasm of teachers deflated, and they went (forcedly) involved, students acknowledge the back to their (bad) practices. activity value, agreeing that they reached a level of knowledge much deeper than that they could (Unwilling) Collaborative Activities have acquired working the same time in isolation, and that the social aspects were actually enjoyable. It is worthwhile pointing out that during the WEL experience, the students were required to participate into some collaborative activities in order to CASE STUDY: AULAWEB AND pass the exams, and those activities were mostly WEL FOR THE COMPUTER successfully, contributing to the improvement of SCIENCE CURRICULA AulaWeb usage. But, some optional collaborative activities went mostly ignored. For instance, Faculty members of the degree program in Com- several courses proposed optional wikis for the puter Science heavily use AulaWeb from its first collaborative production of notes, or collections activation, after having been prominent in a few of solved exercises, which had to be abandoned earlier experimentations. Currently, the Computer for lack of critical mass. The students, even Science program is by far the best user of AulaWeb, those of Computer Science who should not have and this could be related to its large participation experienced any technical problem, candidly in WEL, with 13.8% against 2.9% on the overall admitted that “they had no time for activities not University. In particular, two mandatory courses directly rewarded in terms of grading”. Thus, it expected to be taken in the first semester of the appears that the crucial failure was in motivations, third and last year of the first cycle were part of the
283 LMS Adoption at the University of Genova
experiment: Programming Advanced Techniques both SE and TAP monitor students’ efforts, not (TAP for short) and Software Engineering with to exceed the expected workload. Moreover, the project (SE for short). Together, these exams take deadlines of the activities of the different courses up 18 out of the 30 credits of the semester. The for the same year are globally planned to avoid other 12 credits are evenly distributed between conflicts. Hence the interferences should be null another mandatory course, Enterprise Knowledge or very limited and the average full time student for the IT Professional (EK for short), and an should be able to manage all the courses at the open choice from a list. Thus, the analysis of the same time, while it was not so. results for those courses, and those sharing the For instance, in 2008/09 less than half (in semester with them, is significant to get an insight percentage) of the student successfully took EK on interferences and global trends. exam, if compared to 2007/08. It is quite clear that In (Cerioli & Ribaudo, 2009) the details of this improving courses, by encouraging the participa- experience are illustrated. Here we just recall the tion of the students in interesting activities during key points to analyze the impact of WEL. the semester, has a negative side effect on the other TAP and SE address similar topics, requiring courses, which have a choice between partially basically the same capabilities to be success- losing their audience (like EK) or planning some fully taken. Both have a major project, which activity of their own to capture the students, add- the students find hard to pass, and several pre- ing to the student burden and competing for their paratory activities, received by students, at best, time (like TAP adversus SE). with lukewarm interest. Moreover, the teams of The competition among courses is due to a teachers delivering both courses were almost peculiarity of the Italian system. Indeed, in Italy totally overlapped in 07/08 and 08/09. However, the concept of passing from a year to the next before WEL, TAP and SE were differently orga- does not exist. Students simply have to eventually nized: SE was strictly structured in a stream of collect positive marks in all the courses in their collaborative activities, while TAP was loosely curriculum in an arbitrary number of academic proposing individual tasks. Accordingly, the years. Hence, even mandatory courses may be changes introduced following WEL, were different almost indefinitely postponed. Moreover, each in nature: SE improved only the ICT support to year the same exam may be taken by each student its well-established organization, while TAP was several times (an average of 6-12 possibilities is totally restructured around a set of asynchronous common all over Italy). Thus, many students take collaborative activities. an exam with an incomplete preparation, thinking In 07/08 there were almost twice as many that if they are not lucky the first time, they can students passing the project and the overall exam still try it again (and again...) in a very short time, of SE than of TAP (in percentage). After the slightly improving their preparation, till they are WEL-driven redesign, in 08/09 the percentages able (or lucky enough) to pass it. of successes for SE increased marginally, while These two mechanisms were established to those for TAP more or less doubled, leveling the facilitate the students, by letting them organize difference with SE. themselves. But, it is quite clear that they are, Though both courses were successful, during paradoxically enough, causing the students to lose the semester the teachers of other courses having a lot of efforts. Indeed, they encourage the students the same expected audience gave negative feed- not to plan, focus and deliver, but to overreach and backs. The student participation was decreasing, struggle. Many students are not able, nor forced with peaks in occasion of deadlines for mandatory by the regulations of the University, to complete activities of SE and TAP. It is worth noting that the exam of some year before passing to the next.
284 LMS Adoption at the University of Genova
So they add a full workload of new exams, hop- Teachers Feedback (T-Quest) ing of regaining the lost time. They start to work on all courses, old and new. But, when the effort During the summer 2009, a short survey (just required becomes too much, they drop some 14 questions, some of them open-ended), in the courses, starting from those where they invested following called T-Quest, was carried out among less, that is, those without intermediate activities. faculty members to get their opinions on AulaWeb It is worth noting that the interferences due to (and, marginally, on WEL). For about one month the WEL-driven redesign are of the same nature as it was possible to submit feedback, and in the end those experienced in the past every time a course 210 valid answers were received (12.9% of the proposed any activity during the lesson time while faculty members, 23.5% if we consider only those others limited themselves to standard lessons. enrolled to AulaWeb as teachers in 2009). The So the conflict is between courses with/without answers were not categorized by Faculty, so that activities during the term, not between courses there is no way to verify if they are representative using/ignoring technological support. However, of the actual ratio of staff. the current trend toward a more participative and Faculty members were asked about their use technologically supported learning model will (or lack of use) of AulaWeb, and those using the amplify the problem, introducing a divide between platform were also solicited for their judgment on modern and traditional courses, till the latter will its ease of use. Only 11 out of 210 teachers (about be completely extinct. The delicate point will be 5%) said they do not use AulaWeb. Out of the to find a balance among the different competing others 199 participants, 67.84% find the platform courses to let students participate in all of them. optimal or good to use, the remaining 32.16% find it fair, satisfactory or scarce. Those who were not Feedback on AulaWeb totally satisfied left 45 comments, complaining about the rigidity of the user interface; the weak Any online service needs to be evaluated by support for group management; the high number its users to understand its acceptance. Hence, of steps necessary to perform simple operations we have planned and administered two online like uploading a single file. surveys for the community of AulaWeb users, Almost two third (134 out of 199, that is, using the Moodle feedback. Both questionnaires 67.34%) stated that they do not you use AulaWeb were proposed to the overall population, by email only for uploading material for lectures, and fur- invitations through the official university mailing ther detailed their usage of the tools available on lists, and feedback submission was not enforced. AulaWeb, as already shown in Figure 4. Thus, the received feedbacks are given by mo- For the other 65, who consider AulaWeb as tivated volunteers, who can be assumed to have a mere repository for their notes and slides, the answered at the best of their capabilities. On the motivation for not using the rich array of available other hand, the surveys had to be light in order tools was equally distributed among the following to encourage participation, so that in many cases answers: “I have no time”, “I do not know how just one choice among a list of possibilities has to use them”, “I do not think they are useful for been proposed, in place of a group of standard my subject”, “I have tried to use them, but then Likert scaled questions, one for each item in the I gave up”. list. This approach obviously hinders any statisti- The final part of T-Quest was about the (dis) cal analysis. advantages introduced in the teaching style by the adoption of AulaWeb, if any. The majority of the 158 comments left recognized among the
285 LMS Adoption at the University of Genova
advantages the ease of distribution of the teaching Students Feedback (S-Quest) material and the increase of communication and interaction with the class, as well as the possibil- Analogously to the T-Quest, during the summer ity of complementing and enriching face-to-face 2010, an online survey composed of 28 questions lessons by asynchronous communications. But, (single and multiple choice, plus some open-ended faculty members observed an increase in their questions), in the following called S-Quest, was workload for preparing the digital material and launched among the students. S-Quest has been for timely answering forum posts. Moreover, available for about two months and 1397 valid some perceived AulaWeb as too complex to be answers have been collected, slightly less than used by beginners. 4% (9%) of the students enrolled in the University According to teachers, most of the students (AulaWeb). seem satisfied although some faculty members Students could choose not to answer specific stress that the reason is that they have easy access questions, so that the numbers of answers for the to the teaching material while it is generally dif- possible choices (and their percentage) very sel- ficult to involve them in any form of interactive dom sum up to the number of respondents (100%), activity. They also observe that full-time working though in the following we list for each question students are the most approving, since the platform the results of all possible answers. allows them to (virtually) take part to the lectures. The distribution of answers among Faculties, Finally, to the question “Do you think that an weighted by the number of students officially adequate training on the use of ICT technolo- enrolled to them, sees the Faculty of Sciences gies for education could improve your teaching (including the Computer Science program) at the activity?”, half of the respondents, that is, 103 top, followed by those of Education, and Foreign (49.05%) said “Yes” and another 71 (33.81%) Languages, and then by those of Engineering said “Maybe”, giving 72% of positive responses, (counting also the technological group), and while only 5.71% answered “No” (and the last Economics, as shown in Figure 6. 11.43% did not answer at all).
Figure 6. Percentage of students’ answers
286 LMS Adoption at the University of Genova
In Figure 7, the sample is characterized in to take exams, suggesting a possible market for terms of age, kind of degree and working position. e-learning. Since our University does not provide these data When asked about their frequency of access about students, it is not possible to know if the to AulaWeb, 19.18% answer every day, 39.73% answering sample is representative, under these two or three times per week, 23.91% less than respects, of the overall population. once per week, 6.23% less than once per month, 28.42% of the respondents claims optimal and 9.31% only for exams. The vast majority skills in the use of the computer, 50.61% good (86.54%) connect from home, 7.02% from the skills, 18.04% sufficient knowledge and only university laboratories, with a small percentage about 2% declare themselves not enough skilled (3.08%) accessing from work or other means, like to take full advantage of ICT supported education. internet café (2.29%). The majority of students (66.71%) go to the uni- The vast majority (82.39%) easily located versity every day, followed by 15.75% answering their courses online, but 194 (13.89%) were not two or three times per week, 3.65% less than once able to find them and many of them blamed their in a week, 1.50% less than once in a month. These failure on the lack of courses for their program, data, with more than 80% of frequently presents, which agrees totally with the situation shown by confirm the vocation of Genova University as our previous analysis for some Faculties. face-to-face educational institution. However, the The next questions investigate the ease of use of 8.59% of students is going to the university only AulaWeb. Its interface does not pose any problem for more than half of the respondents (59.84%)
Figure 7. Characterization of students’ sample
287 LMS Adoption at the University of Genova
while it is considered too crowded by 11.31%, too since those teachers who do not use AulaWeb still static and impossible to personalize by 8.59%, maintain some Web pages and therefore, from the “antique” by 10.02%, leaving a small percentage student’s point of view, multiple URLs are neces- of 4.65% to the option difficult to browse. The sary to find all the relevant information. online interaction is generally judged to be fine Independently from its actual use, we have (by 58.87%), but not efficient (i.e., requires too asked students whether they think that “The use many clicks) for 16.39% of the students, too slow of AulaWeb could improve their higher education”, for 8.38%, too repetitive for 8.16%. and we got a 4.01% of “No” answers, followed A significant percentage of students (11.88%) by a 19.90% of “Not noticeable”, a 30.21% of do not expect any interaction with teachers and “Fairly”, and 42.16% of “Yes”. even when it was expected, it has been null for 8.23% of the students, scarce for 28.20%, quite frequent for 29.35%, very frequent for only 4.94%. CONCLUSION AND FUTURE Thus, apparently communication between teachers DEVELOPMENT and students still need improvement. Answers to the question “Has the use of This chapter has reviewed the last ten years of AulaWeb enriched the formative offer of your experience on ICT supported learning at the Uni- course?” are shown in Figure 8. Many of those versity of Genova. The adoption of a centralized having a negative viewpoint left comments, LMS has been presented, discussing the followed clustering on two major complaints: the sub- process, the achievements, a formative project jects present on AulaWeb are not enough, and on instructional design techniques organized for many teachers use AulaWeb only for uploading faculty members, and the feedbacks provided by (sometimes after a long delay) a digital version teachers and students. of the material available in the past at the student The numbers presented here clearly show that office. Any form of interaction or collaboration is AulaWeb has reached its limits. It seems indeed lacking. Finally, in some cases, now finding the difficult to think of further improvements without teaching material has become even more difficult significant changes in the overall organization.
Figure 8. Improvement of the formative offer
288 LMS Adoption at the University of Genova
Some contradictions emerge. The service itself that have emerged by the feedbacks provided by should be enhanced, but the size of the staff barely our community of users. manages the current situation, and the University A comparison of our case study with similar is disinclined to spend. At the same time, students research in Europe would help to clarify the ask for a larger number of (traditional) courses to negative impact of permissive examination poli- be supported also online, while faculty members cies. We need to collect data from educational seek recognition of their online activity. The systems where the exams of one year have to be University, as institution, needs to improve its passed before starting the next or, at least, can- rating and financial income, increasing the num- not be taken several times a year, and compare bers of students, by catching also remote users the student results and LMS usage in those and and improving its services, so that expanding the our systems. Thus, we could understand if the ICT support to education becomes crucial, but the competition among courses we have discussed University is not disposed to put high priority on actually impairs the adoption of collaborative the definition of an e-learning policy. activities in education. The financial side of the problem is amplified by the current general economic crisis: the short- age of resources to invest in e-learning prevents REFERENCES the student numbers, and hence the corresponding revenue, from increasing, in a vicious circle. Cerioli, M., & Ribaudo. M. (2009). Great is the The claim of faculty members for recognition enemy of good: Is perfecting specific courses of their online activity is exasperated by the fact harmful to global curricula performances? 4th that nowadays their evaluation is based mostly on European Conference on Technology Enhanced research production, and marginally on officially Learning (EC-TEL 2009) (LNCS 5794). Berlin- recognized activities. So, in terms of working Heidelberg, Germany: Springer. career, if in the past educational efforts paid very Cole, J., & Foster, H. (2007). Using Moodle. poorly, in the current settings spending time on Teaching with the popular open source course teaching activities is totally detrimental unless management system. O’Reilly Community Press. officially recognized. Free online edition. Despite all the difficulties, providing online support for education is a must for a 21st Century Dalsgaard, C. (2006). Social software: E-learning University aspiring to be world-class. Hence, the beyond learning management systems. European definition a policy for e-learning, to be submitted Journal of Open, Distance and E-learning. to, and get approved by the central authority, is one Downes, S. (2005). E-learning 2.0, eLearn, 2005 of the first priorities of the committee responsible (10). for e-learning and AulaWeb. Once established the policy, it will be interesting – and necessary - to Dror, I. E. (2008). Technology enhanced learning: deeply reanalyze user activities and their per- The good, the bad, and the ugly. Pragmatics & ception of the service to understand its impact. Cognition, 16(2). Amsterdam, Netherlands: John Meanwhile, the service needs to be maintained Benjamins Publishing Company. and improved to meet the users’ expectations Ebner, M., Kickmeier-Rust, M., & Holzinger, that have emerged from the questionnaires. This A. (2008). Utilizing Wiki-Systems in higher affects the technical side of the management of education classes: A chance for universal access? the platform since new Moodle modules or their Springer Universal Access in Information Society updating might be required to solve the problems International Journal, 7(4).
289 LMS Adoption at the University of Genova
Janossy, J. (2008). Proposed model for evaluating Prensky, M. (2001). Digital natives, digital im- C/LMS faculty usage in higher education institu- migrants. On the Horizon - The Strategic Planning tions. Conference MBAA 2008. Resource for Education Professionals, 9(5). MacKeogh, K., & Fox, S. (2009). Strategies for Prensky, M. (2007). Changing paradigms – from embedding e-learning in traditional universities: “being taught” to “learning on your own with Drivers and barriers. Electronic Journal of e- guidance”. Educational Technology. July-August. Learning, 7(2). Puddu, F., Repetto, M., & Vallarino, E. (2008). The Malikowski, S. R., Thomson, M. E., & Theis, WEL initiative: A model of Web enhanced learn- J. G. (2007). A model for research into course ing for faculty members. Journal of E-learning management systems: Bridging technology and and Knowledge Society, 4(3). learning theory. Journal of Educational Comput- Ribaudo, M., & Rui, M. (2009). AulaWeb, Web- ing Research, 36(2). doi:10.2190/1002-1T50- based learning as a commodity - The experience 27G2-H3V7 of the University of Genova. 1st International McGee, P., & Diaz, V. (2007). Wikis and podcasts Conference on Computer Supported Education and blogs! Oh, my! What is a faculty member (CSEDU 2009). INSTICC Press. supposed to do? EDUCAUSE Review, 42(5). Samarawickrema, G., & Stacey, E. (2007). Adopt- McNaught, C., Lam, P., & Ho, A. (2009). The ing Web-based learning and teaching: A case study digital divide between university students and in higher education. Distance Education, 28(3). teachers in Hong Kong. Proceedings ASCILITE doi:10.1080/01587910701611344 2009, Auckland, New Zealand. Trentin, G. (2001). Designing Online Courses. The O’Neill, K., Singh, G., & O’ Donoghue, J. (2004). Web in Higher Education: Assessing the Impact Implementing e-learning programmes for higher and Fulfilling the Potential. The Haworth Press. education: A review of the literature. Journal of Ullrich, C., Borau, K., Luo, H., Tan, X., Shen, L., Information Technology Education, 3. & Shen, R. (2008). Why Web 2.0 is good for learn- O’Neill, M. E. (2005). Automated use of a Wiki ing and for research: Principles and prototypes. for collaborative lecture notes. SIGCSE Bull, 17th International World Wide Web Conference 37(1). New York, NY: ACM. (WWW 2008). O’Reilly, T. (2005). What is Web 2.0: Design pat- Volery, T., & Lord, D. (2000). Critical suc- terns and business models for the next generation cess factors in online education. International of software. Retrieved from http://www.oreillynet. Journal of Educational Management, 14(5). com/pub/a/oreilly/tim/news/2005/09/30/what-is- doi:10.1108/09513540010344731 web-20.html (Dec. 2010). Yueh, H.-P., & Hsu, S. (2008). Designing a Paulsen, M. F. (2003). Experiences with learning learning management system to support in- management systems in 113 European institu- struction. Communications of the ACM, 51(4). tions. Journal of Educational Technology & doi:10.1145/1330311.1330324 Society, 6(4).
290 LMS Adoption at the University of Genova
ADDITIONAL READING Instructional Design: The practice of maximizing the effectiveness, efficiency and appeal Anderson, P. (2007). What is Web 2.0? Ideas, of instruction and other learning experiences. technologies and implications for education. JISC Roughly speaking, it consists of determining the Technology and Standards Watch. Bristol, UK: skills and needs of the learner, the end goal of JISC. Retrieved December 28, 2010 from http:// instruction, and designing activities to assist in www.jisc.ac.uk/media/documents/techwatch/ the transition. tsw0701b.pdf Learning Management System: Commonly Hassan, M. S. (2007). Critical success factors abbreviated as LMS, are software applications for for e-learning acceptance: Confirmatory factor the administration, documentation, delivery, and models . Computers & Education, 49(2). reporting of training programs. Online Surveys: a series of questions proposed Mahmud, K., & Gope, K. (2009). Challenges of through a computer for the purpose of gathering implementing e-learning for higher education in information from respondents. least developed countries: A case study on Ban- Web Enhanced Learning: Combines tradi- gladesh. International Conference on Information tional face-to-face courses with pedagogically and Multimedia Technology (ICMT 2009). sound uses of Web technology into the design and delivery of the course. Morgan, G. (2003). Faculty use of course management systems. Research study from the EDU- CAUSE center for applied research. Retrieved from http://www.educause.edu/ECAR/Faculty- ENDNOTES UseofCourseManagementSy/158560 1 In September the courses for the next aca- Woods, R., Baker, J. D., & Hopper, D. (2004). demic year are created, and students who Hybrid structures: Faculty use and perception of have passed the exams during the summer Web-based courseware as a supplement to face- start to withdraw from the corresponding to-face instruction. The Internet and Higher Edu- courses. Thus, August is the best moment cation, 7(4). doi:10.1016/j.iheduc.2004.09.002 for data collection on one year. Moreover, in August there is no educational activity going on, so that it is also the best choice for technical updates. KEY TERMS AND DEFINITIONS 2 In the Italian system each exam is given, and may be taken by the same student, many Blended Learning: Combines traditional times per year, so that the organizers need to face-to-face courses with computer-mediated know how many students are to be expected education. Learners and teachers work together in order to choose appropriate classrooms/ within and outside the class to improve the quality laboratories. of learning and teaching. 3 The former are mostly used to collect es- Collaborative Activities: Two or more people says and projects autonomously produced learning or attempting to learn something together by (groups of) students, while the latter to using methodologies and environments in which register for in-presence exams. learners actively engage in a common task. 4 Forum news is omitted from the analysis, eLearning: Policy: courses of action, regu- since each Moodle course has one by default. latory measures, laws, and funding priorities concerning ICT support to educational processes.
291 292
Chapter 14 Effective Use of E-Learning for Improving Students’ Skills
Lorenzo Salas-Morera University of Córdoba, Escuela Politécnica Superior, Spain
Antonio J. Cubero-Atienza University of Córdoba, Escuela Politécnica Superior, Spain
María Dolores Redel-Macías University of Córdoba, Escuela Politécnica Superior, Spain
Antonio Arauzo-Azofra University of Córdoba, Escuela Politécnica Superior, Spain
Laura García-Hernández University of Córdoba, Escuela Politécnica Superior, Spain
ABSTRACT The educational system promoted by the European higher education area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academical performance.
DOI: 10.4018/978-1-60960-884-2.ch014
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Effective Use of E-Learning for Improving Students’ Skills
INTRODUCTION the activities required: attendance at theoretical and practical classes, private study, Over the last few years, major regulatory changes completion of tests, preparation of reports, have been introduced at European level, leading to tracking and participating in the forum. an overhaul not only of the structure of university C. The teacher needs to be able to attend – ap- qualifications but also of teaching methods, with propriately and at a suitable pace – to all the the incorporation of more active methodologies students in the time available. in which the role played by students in the learn- D. The students need to receive prompt and ing process has expanded significantly. Teaching reliable feedback on the results of the class methods involving fluid and effective interaction assessments. between teacher and student, and amongst the E. The members of the group need to interact students themselves, aimed at facilitating the proactively between themselves and with exchange of opinions and general information, their teacher, raising questions and generat- streamlining the tutorial system and encouraging ing productive contributions to the discus- group collaboration, have thus acquired particular sion forums, constructing a knowledge base importance. that is critical and robust. The results presented here are drawn from a six-year experiment in Industrial Engineering and The results of this experiment were generally Software Engineering courses. Using a range of highly positive. Specifically, students rated very teaching techniques, the aim of these courses was highly the exchange of information through the to enhance the active and responsible participation forum, while the role of the teacher as motivator of students in the learning process, and to enhance and moderator was regarded as crucial. Similarly, the acquisition not only of subject-related knowl- online quizzes were very positively viewed by edge but also of certain general skills appropriate students, who – in exchange for investing a mod- to the study of engineering. To this end, modified erate but timely amount of effort – were able to teaching methods were incorporated into four reap considerable academic rewards. Finally, it modules of the Engineering Projects section, in was seen as essential that the general scheduling the last year of the course, replacing traditional of the students’ work be well coordinated in terms teaching methods based on teacher-centred classes of the course as a whole, and realistic in terms of and examinations with an alternative based on vari- the amount of effort required, in order to avoid ous techniques such as asynchronous discussion the kind of backlog that would hinder the assigned forums, group tutoring, collaborative learning, tasks being properly completed. online quizzes and peer assessment. To attain this objective, the following secondary considerations had to be addressed (Salas-Morera, Berral-Yerón, BACKGROUND Serrano-Gómez & Martínez-Jiménez, 2009; Lan & Yang, 2009): E-learning tools are being widely incorporated into the teaching-learning process as a back- A. Students need to be motivated to work up and complement to conventional university regularly and follow the correct sequence classes, as well as in distance-learning institu- of activities set out in the syllabus. tions (Yau, Lam & Cheung, 2009). These tools, B. Overall student working time needs to be among which Blackboard, WebCT and Moodle appropriate to the credits assigned for the figure prominently, help teachers to make learn- subject, adopting a realistic approach to all ing more attractive, dynamic and participatory by
293 Effective Use of E-Learning for Improving Students’ Skills
virtue of forums, virtual tutorials, blogs, social Nevgi, Virtanen & Niemi (2006) note that networks, wikis, surveys, online quizzes etc. learning through the Web is a social process (Kozaris, 2010). Thus, it is no longer enough to involving not only individual but also social simply expose students to course content; there and group skills in the collaborative creation of needs to be interaction between students and knowledge. They therefore recommend a pack- teacher, among the students themselves and with age consisting of assessment or self-assessment the learning community and its learning resources tests, asynchronous discussions, tutoring sets and (Banks, 2003). For their part, the students receive learning diaries which provide the user group with information immediately, both in terms of content the tools required for real, profound collaborative and activities, as well as the results of assessments, learning. This context prompts a major change making this type of tool highly efficient (Lahwal, in the teacher’s role: compared to the traditional Amaimin, & Al-Ajlan, 2009). Such tools enable educational system in which the teacher was teachers to maintain a tighter control over the pace merely required to transmit information, his/her of the group, adapting the degree of difficulty and role has now become much more complex – and the introduction of content to the general progress more demanding – in that the teacher is expected of the group, and allowing the students to organise to act as advisor, content facilitator, resource their own work both individually and as a team provider, technician and social animator (Zhao, & (De Laat, Lally, Lipponen, & Simons, 2007). Jiang, 2008). In the same way, there are additional However, using e-learning tools brings its own difficulties in managing students’ groups and in- difficulties: first it requires a high level of com- teractions when on-line collaborative activities are mitment on the part of the academic authorities, programmed (Tuparova & Tuparov, 2010; Vivek who need to devote sufficient resources in terms & Ansari, 2010). of hardware and technical staff to guarantee that Asynchronous discussion forums are a very such tools are available with enough flexibility and useful tool for encouraging the critical dimen- efficiency (McPherson & Nunes, 2008); secondly, sion of learning; students interactions, both in order to get the most out of these technologies, among themselves and with the instructor, yield teachers as well as students need to exhibit certain a synergy in the approach to preparing, sharing skills that they may not necessarily possess at the and understanding information, which fosters a outset (De Laat & Lally, 2003; Kirkwood, 2009). fuller understanding of the material to be studied In this way, one of the most important issues using (Erlin, Yusof & Rahman, 2009). Yet the use of e-learning tools is usability in terms of whether a activities of this sort has certain drawbacks, due product can be used by specified users to achieve mainly to a lack of active student participation specified goals with effectiveness, efficiency and in the e-learning platform, a lack of quality in satisfaction in a specified context of user (accord- the contributions, and the use of the forum as a ing with the international standard ISO 9241-11). means of exchanging information unrelated to Kakasevski, Mihajlov, Arsenovski & Chungurski the subject (Ertmer, Richardson, Belland, Camin, (2008), analysed standard modules of Moodle Connolly, Coulthard, Lei & Mong, 2007; Lan & giving useful information for administrators, Yang, 2009; So, 2009). Mazzollini & Maddison teachers and students on how to improve effective (2003), examining the role of the teacher as forum usage of this system. Kirner, Custódio & Kirner director, report that instructor intervention tends to (2008) analysed intuitiveness, operationability, generate more student satisfaction than peer par- efficiency of use, learnability, attractiveness, and ticipation; increased instructor participation also user satisfaction, concluding that Moodle has a helps to shorten discussions, without necessarily satisfactory usability level. impairing their final quality. It is therefore essential
294 Effective Use of E-Learning for Improving Students’ Skills
to decide beforehand what goals the forum seeks be borne in mind when using forums as teach- to achieve – e.g. active student participation, prob- ing tools include the evaluation method selected lem-solving, deeper conceptual understanding of and the ratio of the teacher effort involved to the subject under study, exchange of information the effectiveness of the assessment in terms of – since these goals should govern the level and learning objectives. Dennen (2008) highlights style of teacher participation. Similarly, Andresen the importance of evaluating student participa- (2009) emphasises that online discussion forums tion in forums, but questions the efficacy of the do not constitute a way of eradicating or substi- evaluation systems used, arguing that while one tuting the teacher, but on the contrary demand a may readily measure the length and the number of higher level of attention than traditional classes, messages posted by each student, these measures something that needs to be acknowledged by the do no necessarily reflect the learning achieved. academic authorities. However, despite the fact Whilst online discussion is valuable in that it that the forums have been criticised in the past encourages a sense of social connection, students for their lack of quality and low levels of student cannot be expected to have sufficient rhetorical, participation, Andresen notes that these problems analytic and argumentation skills to fully engage have diminished in recent years. According to in certain types of activities. Dennen suggests no Rovai (2007), it is essential to bear in mind a fewer than four different methods of assessment: number of factors if an effective strategy is to be devised that will allow online discussion forums A. Participation measures: this cannot strictly be to be exploited to the full: motivating students to considered a method of assessing learning, take part in discussions, for example by awarding since a relationship between the quantity of them a mark; leaving students in no doubt from messages and learning cannot be assumed. the outset of the course that their participation B. Message content and quality measures: much is expected; providing opportunities for socio- more reliable, but more difficult to measure emotional discussions, which give rise to a strong and requiring more effort on the part of the sense of community; and ensuring that discussions teacher (something that must also be borne in remain content-oriented and task-oriented. At the mind). A simple way of doing it is to classify same time, teachers need to exhibit an attitude that the messages as on-topic or off-topic, but this fosters the social dimension, avoiding becoming would prevent the possibility of awarding a the centre of attention themselves and encouraging nuanced score, while exposing students to equality in terms of the culture, gender and status the temptation of submitting mini-essays, of their students. Taking a similar approach, Hew thereby interfering with the interaction. & Cheung (2008) also note the lack of participation C. Holistic measurements: simultaneously tak- on the part of the students and the poor quality ing into account both quality and quantity of their contributions, and propose various strate- engenders a positive feedback loop encour- gies through which the teacher can attract student aging learning more than mere participation. participation: e.g. by giving his own opinions and D. Asking for short reports on the students’ experience; by starting discussion threads, asking experiences in the discussion processes. questions about specific aspects of the subject; by appraising student input; by suggesting new Another tool widely used as a complement directions for the discussion; by thanking students to e-learning strategy is e-assessment, whose for their posts; and by clarifying specific issues. chief advantages are a considerable reduction in Given that participation in online forums needs marking time for the teacher and the opportunity to be evaluated, other major considerations to for students to schedule their individual efforts,
295 Effective Use of E-Learning for Improving Students’ Skills
completing the tests at a variety of times and loca- Furthermore, it is very important that profes- tions. There are doubts, however, about the true sors pay attention, not only on students’ academical efficacy of e-assessments. The following ques- performance and knowledge acquisition, but on tions have been raised: do e-assessments place a complete curriculum design which must take an additional burden of stress on the students? into consideration the practical skills needed for Do students possess adequate ICT skills to tackle the students’ professional development (Zhou, these assessments successfully? Does this type of 2004; Barrella, Simmons & Buffinton, 2006). examination conform with the students’ learning Thus, considerable efforts have been made over style and with the type of technology most com- recent decades, to define the skills needed by monly used in the workplace? (Whitelock, 2009). engineering students in order to find the right job Angus & Watson (2009) argue that compliance and ensure career development, favouring those with three basic criteria is essential in order for curricula that focus on the acquisition not only online assessments to be genuinely effective and of knowledge but also of skills and competences. assist in improving learning: a) online assessment Rehman, Said & Al-assaf (2009) stress the need methods must be an effective proxy for traditional for coherent development of students’ skills over methods; b) students must be at least indifferent the whole curriculum. In the case of engineering to whether online or traditional methods are used; and technology, as well as the relevant local and and c) online methods should provide better out- national regulations, the Criteria for Accrediting comes than traditional methods. They conclude Engineering Programs (ABET, 2009) must also that exposure to regular low-mark online testing be taken into account. Among such skills the fol- significantly improves student learning as mea- lowing stand out: sured by a final proctored exam. In a similar vein, Kibble (2007) and Dobson (2008) investigate A. An ability to apply knowledge of mathemat- whether student participation in low-mark online ics, science, and engineering quizzes enhances final results, concluding that B. An ability to design and conduct experiments, students either agree or strongly agree that online as well as to analyse and interpret data tests constitute a suitable assessment method and C. An ability to design a system, component, or that they have a positive effect on the final mark. process to meet desired needs within realistic Kibble (2007) also stresses that the best assessment constraints prompted by economic, environ- strategy is to use different methods simultaneously, mental, social, political, ethical, health and such as online tests, group assignments and final safety, manufacturability, and sustainability examinations, with the aim of encouraging differ- considerations. ent ways of reflecting on content. Savander-Ranne, D. An ability to function in multidisciplinary Lundén & Kolari (2008) designed a course in teams which students were exposed to a learning cycle E. An ability to identify, formulate, and solve comprising taught classes, pre-lecture assign- engineering problems ments, student seminar presentations and concept F. An understanding of professional and ethical tests aimed at encouraging student participation, responsibility with the consequence that the students displayed a G. An ability to communicate effectively substantial improvement in their communication H. The broad education necessary to under- and problem solving skills, as well as an improve- stand the impact of engineering solutions ment in general learning outcomes. in a global, economic, environmental and societal context
296 Effective Use of E-Learning for Improving Students’ Skills
I. A recognition of the need for, and an ability asynchronous discussion forums, groups tutorials to engage in, life-long learning and practical sessions, as well as theory classes. J. A knowledge of contemporary issues Table 1 sets out the knowledge and skills involved K. An ability to use the techniques, skills and in the subject, their connection to the activities modern engineering tools necessary for carried out, and the importance of these activities engineering practice. for the final mark. The last column shows the number of activities intended to foster a particular skill, and the last row shows the number of skills METHODS needed for a particular activity. According to the European Credit Transfer The aim was to adapt the teaching methodology System (ECTS) and the Spanish regulation (Min- used in the modules of the University of Córdoba’s isterio de Educación Cultura y Deporte, 2003), Engineering Projects section to the model set out the average student should be able to pass the set by the European Higher Education Area. The subjects for a given academic year with a total modules in question are worth 4.5 ECTS credits workload (in Spain) of between 1500 and 1600 (European Commission, 2009) and the contents hours, which has to include all the activities to be relate to Project Management. According to the undertaken by the students: classes, practical ses- syllabus, the following student skills should be sions, study, homework, tutorials, seminars etc. developed: An academic year of 60 credits involves a workload per credit of between 25 and 30 hours. When A. Adaptation to new situations drawing up the teaching programmes for their B. Analysis of client requirements subjects, teachers have to take a range of factors C. Ability to analyse and synthesise into consideration such as: a) the activities to be D. Ability to apply knowledge in practice undertaken by the students should be directed not E. Ability to manage information only towards the acquisition of knowledge, but F. Ability to organise and plan also towards the development of the appropriate G. Oral and written communication skills skills, b) the student workload should be realistic, H. Estimating and programming work practicable in the time available in the semester I. Design methods and consistent with the credits awarded for the J. New technologies (ICT) material and c) it is essential that everyone knows K. Strategic organisation and planning from the outset what tasks they have to do, when L. Problem solving they have to do them and their bearing on the M. Decision taking final mark, which is why a weekly task schedule was drawn up. The first challenge when schedul- In order to acquire these abilities, a combined ing activities was therefore to ensure that students strategy was devised based on a variety of teaching were objectively able to complete the tasks tools, both face-to-face and non face-to-face using within the total number of hours allotted to the Moodle, such that in combination they obliged subject. For that purpose, during the initial ECTS the students to study the theoretical aspect of the implementation phase, students were asked to subject, while also requiring them to exercise complete weekly surveys in order to ascertain the targeted skills (Salas-Morera, Berral-Yerón, whether the number of working hours assigned Serrano-Gómez & Martínez-Jiménez, 2009). to each activity was realistic, and whether it was Specifically, activities comprised: online quizzes, consistent with the requirements of the course as
297 Effective Use of E-Learning for Improving Students’ Skills
Table 1. Activities undertaken in relation to skills and percentage weighting for the final mark
GROUP ONLINE PRACTICAL FINAL FORUM TUTORIALS QUIZZES SESSIONS EXAMINATION (10%) (12.5%) (25%) (12.5%) (40%) Theoretical contents X X X X X 5 Adaptation to new situations X X X 3 Analysis of client requirements X X X 3 Ability to analyse and synthesise X X X 3 Ability to apply knowledge in X X 2 practice Ability to manage information X X 2 Ability to organise and plan X X X X 4 Oral and written communica- X X X X 4 tion skills Estimating and programming X X X X 4 work Design methods X X X 3 New technologies (ICT) X X X X 4 Strategic organisation and X X X X X 5 planning Problem solving X X 2 Decision making X X X X 4 10 10 6 10 10 a whole. Initially, 4.5 ECTS credits were assigned and problems with the whole group. Interspersed to each subject, so that the activities scheduled between these sessions were five practical sessions should require a total student workload of between (P.S.), conducted in small groups, whilst students 112 and 135 hours. were simultaneously encouraged to take part in The activities and sequence scheduled for the asynchronous discussion forum (A.D.F.), start- the course were as follows: theoretical classes ing threads on any aspect of the subject matter in the lecture theatre for the whole group (75-90 that took their interest, with a view to fostering students, depending on the year); practical classes cooperation both amongst themselves and with in the laboratory or computer room in groups of the teacher. Six online quizzes (O.Q.) were set, at up to 25 students; an online forum devoted to uniform intervals throughout the semester, testing the topics studied in the module with the free familiarity with the material covered. After each and simultaneous participation of all students; set of two quizzes, a tutorial (T) was scheduled for fortnightly feedback on forum activity through the whole student group. Finally, forum activity group tutorials of up to 25 students; periodical was tracked through five group tutorials (G.T.), completion of online quizzes; and feedback on in which the students were asked to prepare for quiz outcomes in whole-group tutorials. their classmates a summary of forum postings, Figure 1 shows the general organisation of the together with further information and a critical scheduled activities and their interconnections: the opinion on the postings, which in turn led to new subject was divided into three theoretical blocks discussion. Given the importance of a realistic (T.B.) in which the teachers tackled both theory workload, at the start of the course a calendar
298 Effective Use of E-Learning for Improving Students’ Skills
of activities was published on Moodle and was the course as it progresses. The students have to updated and adjusted on a weekly basis. shows complete six quizzes, two per theoretical block, the breakdown of anticipated hours of work for in such a way that the first quiz covers the first each of the scheduled activities, in accordance half of each block while the second covers all the with the number of credits assigned to the subject material studied up until that point, not just in that matter (112-135). particular block but from the start of the semester. The details of the activities were as follows: Whenever a quiz comes round, the students have four days in which they are free to complete it Online Quizzes at any time, on condition that once they open it they have to finish it within 40 minutes. The total The main aim of the Moodle online quizzes is to time allotted for preparing for the quiz, revising encourage students to study on a regular basis, the necessary concepts and completing it was two at a rhythm that will allow them to keep up with hours. Quizzes comprised ten to fifteen questions
Figure 1. General flowchart of scheduled activities and links between them
299 Effective Use of E-Learning for Improving Students’ Skills
Table 2. Distribution of students’ workload
Assigned time Assigned time Face-to-face activities Individual and virtual activities (hours) (hours) Whole group activities Online quizzes and preparation (O.Q.) 12 Theoretical Blocks & Problem Solving (T.B.) 33 Asynchronous Discussion Forums (A.D.F.) 10 Tutorial Sessions (T.) 6 Studying 30 Groups of up to 25 students Homework 10 Practical Sessions (P.S.) 12 Personal Tutoring 1 Total individual and virtual activities Group Tutorial Sessions (G.T.) 9 63 estimated time Total face-to-face time assigned 60 Final Examination 5 Students’ total working time (60+63+5) 128 which might involve multiple choice, matching Asynchronous Discussion and numerical answers – if the latter, a certain Forum and Group Tutorials margin of error was allowed. Each question was awarded a maximum score of one point, with a The asynchronous discussion forum proved to be penalty for every incorrect option chosen. Each among the most productive of the tools tested in quiz comprised several blocks of questions, set these courses, in relation to the effort required. All jointly by all the teachers; questions varied in group members were required to register on the difficulty, and there were sufficient questions to forum, and messages were automatically tracked ensure that when randomly assigned to students, by e-mail, so that all users were immediately and each student would have a different quiz. Questions almost effortlessly informed of new developments were also programmed to appear in random order, as they appeared. The main aim of the forum was and choices for each question were also presented to maintain contact and keep the group’s attention randomly. The quizzes were not particularly dif- focused on the themes raised by the subject matter. ficult, and their content related to concepts that The contact is extremely fluid and effective, and were basic to the subject; the students therefore helps communication extend much further than in needed to study and revise these concepts at least the face-to-face sessions, enabling students to raise with the same frequency that the teacher scheduled doubts and make constructive contributions. For the quizzes, with great benefits for the develop- their part, teachers were able to take stock of the ment of other activities. Once the students had way knowledge is developing within the group, completed the two quizzes for each block, the allowing them to direct their students’ work from teacher obtained from Moodle a file containing a distance in a way that was virtually unnotice- a statistical analysis of student answers, which he able. The teacher’s role in regard to student forum then analysed in order to identify the main weak- postings was twofold: when a student asked a nesses; afterwards, a 2-hour tutorial was held with question or put forward a subject for debate, the the whole group, in which the tested material was teachers kept to the sidelines in the hope that the discussed, paying special attention to those areas students would come up with an answer between in which the poorest results were obtained. themselves, as freely and spontaneously as possible, intervening to make minor suggestions or encourage others to join in. Secondly, if the teacher noticed any drop-off in forum postings
300 Effective Use of E-Learning for Improving Students’ Skills
over a certain period, he suggested new subjects students at the beginning of the year. There were for discussion, thus encouraging students to seek five discussion sessions, organised as follows: information on additional aspects of the subject. At the beginning of the year there was a one- Subjects suggested by teachers included: ACM/ hour session with each group of up to 25 students. IEEE Computing Curricula, ethical regulations In this session students were asked to introduce in the engineering professions, and professional themselves to other members of the group, explain- activities in general. ing their reasons for choosing their degree subject It was considered important not to base evalua- and outlining their academic career to date. There tions of forum activity on the number of contribu- was also an initial survey in which students were tions from each student, as this would encourage asked about the extent to which they believed they a flood of low-quality, irrelevant contributions; have already acquired the skills targeted by the the students were informed of this fact. Thus the module, as well as the importance they attached evaluation of forum content, although it required to these skills for their future careers. more work on the part of the teaching staff, was Four two-hour sessions were held with each holistic in nature, awarding a higher score to the of the groups of up to 25 students, distributed most relevant contributions and lower scores to uniformly throughout the semester. Prior to each the least relevant. Although Moodle gives the session, one of the four subgroups was asked to possibility of evaluating forum contributions in- monitor the forum and prepare a handout with dividually, an external spreadsheet was used, so an outline of the most important themes, which each student was evaluated for this aspect globally they discussed with the teaching staff. Once the at the end of the semester. teacher had approved the summary as an adequate Forum activity did not take place in isolation and properly organised reflection of the topics from the rest of the activities in the course, but emerging in the forum, the subgroup prepared rather as a reinforcement of all the other activities, a Microsoft Office or OpenOffice presentation, in its role as the default communication tool. This which was also sent to the teacher for approval. role was further enhanced by what are known as Finally, in the group tutorial session, they gave group tutorials (G.T.). The aim of these tutorials the presentation, which lasted no more than 40 was to enable students to work together on the minutes, to the rest of the group; the remainder of subjects discussed in the forum over the previous the 2-hour session was devoted to a group discus- weeks. To do this, students needed to make use of sion of the subjects addressed in the presentation. their analytical and synthetical skills, their ability Assessment of this session was carried out jointly to manage information, their ability to express by the teacher (50%) and the other students in themselves clearly in speaking and writing, and the group (50%). The following points were as- their planning and organisational skills. As in the sessed: presentation quality; appropriateness of collective tutorials for the quizzes, the pooling of material (joint ratings for all subgroup members); views on the subjects discussed in the forum helped oral expression during the presentation; and oral to highlight the most important concepts and, expression during the discussion (the last two often, to address mistakes which might otherwise points were rated individually). have been impossible to detect. For the purposes of this activity, the main group was divided into Other Assessable Activities four groups of up to 25 students, and in turn each of these was sub-divided into four subgroups of The remaining assessable activities comprised four to five students. Group and subgroup divi- six two-hour practical sessions in the laboratory sions were made on the basis of affinity between or computer room in small groups. Prior to each
301 Effective Use of E-Learning for Improving Students’ Skills
session, a handout for the session was published on the average student spent over 18 hours, rather Moodle, which dealt with problem-solving using than a total of 12 hours, doing the 6 quizzes. In specific software (for example Microsoft Project future, therefore, the time to be spent on each quiz or OpenProject). In the course of the week follow- may need to be revised from two to three hours. ing each activity, the students uploaded a short By contrast, the time students claimed to spend on report onto Moodle regarding the results of the the forum approached the ten hours anticipated, practical session. The assessment was completed i.e. some 40 minutes per week. Moreover, the with a final examination. average weekly time remained extremely uniform over the 15 weeks of the course. Weekly study time maintained a steady pattern of around three RESULTS AND DISCUSSION hours in the first weeks, subsequently rising to a maximum of seven hours as the final examination To evaluate the experiment the following tools approached; similar findings were observed for were used: weekly surveys of student workloads; the time spent writing reports, which increased analysis of quiz results; analysis of forum activ- in the weeks when reports were to be handed in. ity; academic result of group tutorials; surveys Students spent strikingly little time on personal on perceived competences before and after the tutorials. There may have been several reasons course; survey of students’ opinions on all subject- for this: timetables for classes and other activities related activities; comparison of overall academic were so demanding that students had no time to performance. make individual appointments with the teacher; moreover, the growing use of virtual tutorials via Student Workload Surveys the forum contributed to a decline in individual tutorials, which were only requested by students Over the last four years, weekly surveys were car- with specific problems that they did not wish to ried out through Moodle in order to determine the share with the rest of the class via the discussion actual student workload in each of the activities. forum. Every week, students were asked to indicate: 1) The difference between the times anticipated time spent revising and doing the online quiz; 2) in the initial schedule and those actually reported time spent on the discussion forum; 3) time devoted by the students was very striking. The 104 hours to private study; 4) time spent drafting reports; devoted to non face-to-face activities, plus the 60 and 5) time spent attending personal tutorials. devoted to face-to-face activities, gave an actual Participation in these surveys was completely total of 164, compared to a total time of 128 hours. voluntary, and had no impact on the overall mark. This difference between the real and actual times Table 3 shows the average results for the explains why the students constantly complained weekly surveys carried out during the academic about the workload they had to sustain, not only years 2006-2007 to 2009-2010. With regard to in this module but also in all the others making revising for and doing quizzes, the fact that dates up the subject, and may account, at least in part, were scheduled beforehand meant that time spent for the high failure rate observed in these subjects. on revising for and actually doing the quizzes was relatively uniform throughout the course, Analysis of Quiz Results although at times the student workload for this activity increased considerably. In the initial subject Table 4 shows the results of the quizzes from schedule, each quiz was intended to take a total of 2006-2007 to 2009-2010. The most remarkable 2 hours; by contrast, survey results showed that finding was the difference between the number
302 Effective Use of E-Learning for Improving Students’ Skills
of matriculated students (between 90 and 64) and the number of students actually completing TOTAL 18.58 9.30 58.63 17.53 0.24 104.29 the quizzes. Every year, some students failed to complete a single quiz, something which certainly
AVG. merits detailed study in its own right. The number
1.24 0.62 3.91 1.17 0.02 6.95 of “efficient” students, i.e. students that completed at least four of the six quizzes was 77, 57, 64 and
15 54 respectively. It was noticeable that the aver- 0.93 0.48 7.43 1.67 0.00 10.51 age mark in each year (out of a maximum score of ten) was relatively high, as was the number of 14
1.26 0.66 6.78 2.34 0.00 11.03 students passing the activity, particularly in comparison with the number of students participating,
13 suggesting that this activity makes a significant 2.32 0.64 5.43 1.57 0.01 9.96 contribution to the subject pass rate. The activity’s contribution to overall learning outcomes will be
12 examined later.
2.01 0.99 5.21 2.11 0.00 10.32 With regard to the existence of significant differences among years results, ANOVA analy- 11
0.71 0.58 3.94 1.77 0.05 7.04 sis have been made both, for the overall quizzes scores (O.Q.S.) and for every individual quiz; and 10
0.72 0.58 3.69 1.17 0.06 6.21 significant differences were only found in quizzes nº4 and nº6 of the forth year with respect to the 9
1.46 0.48 3.69 0.92 0.01 6.56 others. Nevertheless, there is not apparent reason for this differences, more when O.Q.S. are simi-
8 lar. 1.16 0.65 3.39 1.58 0.01 6.80
7 Analysis of Forum Activity 1.03 0.54 3.96 1.63 0.02 7.17
Changes in forum activity between academic 6 years 2004-2005 and 2009-2010 are charted in 1.11 0.74 3.02 0.58 0.00 5.45 Table 5, which shows that student participation
5 in the forum increased from 2007-2008, the first 1.58 0.62 3.45 0.82 0.06 6.53 year that this activity was included in the overall
4 mark; a significant increase was noticed not only 1.36 0.75 3.06 0.88 0.01 6.05 in the number of threads started by the students
3 but also in the number of responses, indicating 1.98 0.55 3.20 0.29 0.01 6.02 that the awarding of a mark greatly encouraged 2
0.92 0.54 1.96 0.12 0.02 3.56 participation. Comparing the number of threads started with
1 the number of participating students of the previ- 0.04 0.52 0.42 0.08 0.00 1.07 ous section gave averages of 0.47, 1, 1.55 and - - - 1.33 threads started per student, which tended to reinforce the claim that the forum generated a WEEK Nº Online quiz prep and zes aration Forum Studying Homework Personal tu torial TOTAL
Table 3. Average weekly time devoted to each activity in the academic years 2006-2007 2009-2010 Average 3. Table growing interest among students over the years.
303 Effective Use of E-Learning for Improving Students’ Skills
Table 4. Quiz results for the four study years
YEAR 06/07 90 STUDENTS YEAR 07/08 64 STUDENTS 1 2 3 4 5 6 O.Q.S. 1 2 3 4 5 6 O.Q.S. N 83 84 80 77 76 71 90 56 62 50 53 55 59 64 MAX 10,00 8,50 9,60 8,80 9,30 10,00 8,83 8,30 10,00 9,60 9,30 9,30 9,00 7,97 MIN 1,00 2,10 3,00 2,80 2,30 4,00 0,00 2,20 2,60 1,70 3,30 3,80 2,00 0,00 MEDI- 7,50 5,65 7,55 6,00 8,60 8,00 6,37 6,30 6,05 7,20 6,10 8,60 7,00 6,38 AN MEAN 7,34 5,56 7,25 6,06 7,55 7,79 6,02 6,08 5,88 6,77 6,09 7,85 7,08 5,77 PASSES 71 52 76 56 61 70 73 45 45 42 41 48 56 47 S.D. 2,01 1,40 1,44 1,52 2,17 1,17 2,07 1,60 1,63 1,70 1,43 1,77 1,59 1,86 YEAR 08/09 78 STUDENTS YEAR 09/10 70 STUDENTS 1 2 3 4 5 6 O.Q.S. 1 2 3 4 5 6 O.Q.S. N 62 60 59 63 63 63 78 65 64 59 54 56 55 70 MAX 8,40 9,30 9,70 9,30 9,70 9,00 8,38 8,40 9,60 8,80 9,80 9,60 10,00 7,90 MIN 1,00 2,40 2,80 1,50 3,90 3,00 0,00 2,20 0,80 2,70 0,10 4,00 4,00 0,00 MEDI- 5,70 6,30 8,00 8,00 7,60 7,00 6,00 5,70 5,90 6,50 4,74 8,20 8,00 5,55 AN MEAN 5,58 6,02 7,50 7,03 7,38 7,03 5,34 5,54 5,69 6,50 4,80 7,80 7,82 5,32 PASSES 46 41 52 47 58 61 54 39 41 51 21 50 54 45 S.D. 1,55 1,80 1,71 2,26 1,38 1,22 2,71 1,42 2,09 1,33 2,40 1,48 1,17 1,67
Similarly, comparison of the number of answers targeted by the module, leading to contributions with the number of participating students in each that gradually become more and more relevant. year yielded figures of 1.18, 3.23, 7.50 and 7.57, further supporting this hypothesis. Academic Results for The quality of contributions also improved over Group Tutorials time, with the students themselves realising that off-topic or repetitive responses were unlikely to The group tutorial activity brought together other receive good marks. Thus the forum proved a use- activities in the module through the questions ful tool in focusing students’ attention on the topics emerging on the forum. It also helped to foster the most important skills in the area, such as ability
Table 5. Discussion forum activity, 2004-2005 to 2009-2010
YEAR 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 Subjects started by the teacher (A) 25 54 26 35 46 55 Total number of replies to subjects started by the teacher (B) 48 136 33 52 193 210 Replies per subject started by the teacher (A/B) 1.92 2.52 1.27 1.49 4.20 3.82 Subjects started by students (C) 56 80 36 57 99 72 Total number of replies to subjects started by students (D) 140 198 91 184 480 409 Replies per subject started by students (C/D) 2.50 2.48 2.53 3.23 4.85 5.68
304 Effective Use of E-Learning for Improving Students’ Skills
to analyse and synthesise, to manage information, Similarly, the teacher’s score was compared and to organise and plan work, as well as speaking to the mark finally awarded to the students. Here and writing skills. Understandably, therefore, this there was no statistically significant difference in activity was given core status when organising the two years being investigated (M1=7.21; tuition. Moreover, all students were given shared SD1=1.67; M2=6,97; SD2=1.19; t(116)=0.95, responsibility in the evaluation of tutorials. The p=0.35); (M1=7.08; SD1=1.29; M2=7.38; following were assessed: presentation quality; SD2=0.76; t(99)=-1.56, p=0.12), indicating that appropriateness of material (joint ratings for the students’ participation did not have a signifi- all subgroup members); oral expression during cant influence on the mark. the presentation; and oral expression during the discussion (the last two points are rated individu- Self-Assessed Skills Surveys at the ally). Both the teacher and the other students in Beginning and End of the Course the group assessed all members of the subgroup on each of the aspects mentioned above, and the In order to ascertain the extent to which the experi- overall mark was derived (50%-50%) from the ment improved students’ skills, and determine the teacher’s score and the students’ score. As Figure importance the students attached to these skills for 2 shows, scores awarded by the teacher covered their professional careers, surveys were carried out a wider range, whilst student scores tended to be in 2008-2009 and 2009-2010, at the beginning and concentrated in a narrower range between six and end of the year, using a scale of 1-5. The students nine points. The first question that tends to arise were asked to rate the importance they attached when peer assessment is used is whether students to these skills for their future professional careers are more lenient or more demanding with their and the extent to which they thought they had al- colleagues than the teacher is, and whether there ready acquired them (Table 6). The most striking is a significant difference between scores obtained finding was that in both two years there was an in this way compared to scoring performed solely across-the-board increase in students’ perceptions by the teacher. An independent-samples t-test of their own skills, indicating that working method was conducted in order to know whether there achieved its goals to a considerable degree, at least are statistically significant differences between in terms of students’ perceptions. At the outset, the scores assigned by teacher and scores assigned by competence considered by students as being least students in 2008-2009 and 2009-2010. In the first important for their professional career was Design year a significant difference was found between Methods; the student grading of this competence scores assigned by teacher (M1=7.21; SD1=1.67) by the end of the year had increased more than and by students (M2=6.73; SD2=0,90); t(98)=2.03, for any other competence; in the second year, it p=0.045, so students were more demanding was the competence recording the second-greatest with their classmates than the teacher was. In increase in student grading. This could be due to 2009-2010, however, the hypothesis of equality the fact that across the rest of the curriculum for of means was accepted (M1=7.08; SD1=1.29; this degree a good deal of emphasis was given
M2=7.30; SD2=0.55; t(83)=-1.25, p=0.21), so to technical aspects of engineering and to related the student score not being significantly higher problem-solving, but hitherto there had been no than the teacher-awarded score. Thus no reliable overview of design in engineering and related conclusions can be drawn from these findings, projects. Meanwhile, the skill showing least im- suggesting that a further sample needs to be taken provement in both years was New Technologies in order to reach firmer conclusions. (ICT), possibly because the students considered that they had already developed considerable abili-
305 Effective Use of E-Learning for Improving Students’ Skills
Figure 2. Group tutorial scores in 2008-2009 and 2009-2010
ties in this area over the course of their studies and Students’ Opinions Regarding that it was now difficult for them to show further the Value of the Experiment improvement. Finally, in both years the subject considered a priori most important by students From 2004-2005 onwards, students were asked was Decision-Making, thus reflecting the general at the end of the semester to grade the statements approach of the degree, which is strongly oriented listed in Table 7. After ANOVA tests, significant towards project performance and management. differences between years were found in cases When the means at the start and at the end of marked with bold characters and shading in each year were statistically compared by means Table 7. The most striking finding was the ef- of t-test, significant differences were found in that fect of including participation in the discussion cases marked with bold characters and shading forum (items 5 and 6) as part of the final mark in Table 6, what clearly remarks the success of in the last three years. The significant increase in the experience in this aspect. the student’s interest in making interesting and constructive contributions to discussion can only be due to this factor. Over the same period, there was a similar increase in the students’ perception of teacher availability (9), which again may be largely due to forum activity being evaluated. The
306 Effective Use of E-Learning for Improving Students’ Skills
Table 6. Student perception of skills at the beginning and end of each year
ACADEMIC YEAR 2008-2009 ACADEMIC YEAR 2009-2010 IMPORTANCE START OF END OF YEAR IMPORTANCE START OF END OF YEAR YEAR YEAR COMPETENCE mean SD mean SD mean SD mean SD mean SD mean SD Adaptation to new 4.23 1.03 3.54 0.86 3.63 0.73 4.47 0.65 3.05 0.67 3.93 0.54 situations Analysis of client re- 4.23 0.82 3.46 0.81 3.79 0.72 4.47 0.68 2.62 0.87 3.78 0.68 quirements Ability to analyse and 4.12 0.82 3.54 0.71 3.68 0.73 4.32 0.68 2.93 0.66 3.93 0.60 synthesise Ability to apply 4.04 0.96 3.42 0.86 3.97 0.80 4.52 0.62 3.18 0.75 4.00 0.77 knowledge in practice Ability to manage 4.08 0.80 3.24 0.78 3.74 0.68 4.27 0.71 3.12 0.69 3.85 0.65 information Ability to organise 4.27 0.87 3.73 0.92 4.08 0.89 4.57 0.59 2.90 0.84 4.26 0.70 and plan Oral and written com- 4.23 0.99 3.38 0.98 3.82 0.98 4.60 0.72 2.83 0.83 3.63 0.82 munication skills Estimating and pro- 4.08 0.80 3.62 0.85 3.82 0.78 4.38 0.74 2.87 0.87 4.07 0.81 gramming work Design methods 3.58 0.70 3.04 0.72 3.66 0.70 4.02 0.72 2.47 0.68 3.70 0.85 New technologies 4.23 0.99 3.50 1.10 3.56 0.84 4.45 0.70 3.12 0.85 3.52 0.88 (ICT) Strategic organisation 4.27 0.92 3.42 0.76 3.74 0.84 4.32 0.65 2.70 0.70 3.89 0.83 and planning Problem solving 4.04 1.15 3.54 0.86 3.87 0.72 4.60 0.56 3.18 0.77 3.81 0.67 Decision making 4.27 1.15 3.73 1.00 4.00 0.91 4.65 0.58 3.17 0.85 4.04 0.81 other eye-catching features were the high values During this period there was a marked drop in the assigned by students to the questionnaires carried number of students passing the study subjects. out periodically (8) and the utility of learning This led to an overhaul of teaching methods in resources available in Moodle (7). The remainder 2004-2005, taking advantage of the implementa- of the topics in the survey attracted high scores, tion of the ECTS Pilot Plan. For the first time, all although there were no significant differences the activities to be carried out by students (rather between academic years. than just attendance at lectures) were scheduled in detail; the programme was fine-tuned by weekly Academic Performance feedback as the course progressed. The aim was to ensure that all students were kept constantly Final student marks for the last ten academic years informed of the activities scheduled for the fol- are shown in Figure 3 under three headings: passed, lowing weeks, thus enabling them to plan their failed and missed. In the first period, 2000-2001 work in good time. At the same time, a discus- to 2003-2004, traditional teaching methods were sion forum was included on Moodle, although used, with theoretical-practical classes and final this forum did not initially contribute to the final examinations as the only method of evaluation. mark; finally, online quizzes were scheduled.
307 Effective Use of E-Learning for Improving Students’ Skills
Table 7. Student views on scheduled activities, 2004-2005 to 2009-2010
Statement 04-05 05-06 06-07 07-08 08-09 09-10 1.- Information on subject teaching and assessment 3.98 3.92 3.75 3.92 4.11 4.29 methods was sufficient and proved useful 2.- I think that what I learned in this subject will be 3.89 3.88 4.11 3.96 4.18 4.19 useful to me in my professional life 3.- The problems studied helped to imagine real-life 3.98 3.94 4.09 3.98 3.95 4.16 practical situations 4.- The problems studied helped to reinforce theoreti- 4.27 4.47 4.36 4.35 4.47 4.33 cal concepts 5.- The forum was useful for sharing information and 3.00 3.65 2.84 4.15 4.30 4.13 expressing views on subject content 6.- The forum was useful for making concepts clearer 3.94 3.63 3.18 4.05 4.00 4.00 7.- The learning resources available on Moodle were 4.25 4.47 4.09 4.29 4.23 4.23 useful 8.- The quizzes were useful as a way of keeping up 4.04 4.04 4.16 4.08 4.20 4.06 with the subject 9.- The teacher was available for queries (albeit 3.96 4.50 4.02 4.15 4.39 4.58 online/by e-mail) 10.- The teacher encouraged class participation 3.79 4.16 4.09 4.00 4.07 4.13
In the first year of implementation of the ECTS it as a way of answering questions and clarifying credits system, there was a notable increase in grey areas, since a single reply served to dis- pass rate as a positive consequence of the de- seminate information and explanations among tailed scheduling but mainly as a consequence all students, instantly. Prior to that, participation of teachers analysing their students’ workloads, in the forum had been considered supplementary something that up until then had not been taken and voluntary, when in reality it had gradually into account in time-tabling. However, over the become indispensable. As soon as forum activity next two years (2004-2005 and 2005-2006) the was included in the final mark, all the activities number of failures again rose, perhaps because required of the students were incorporated into more teachers changed their teaching methods, the assessment, allowing teachers to calculate leading students to complain about the large the students’ workload more accurately, and en- number of tasks they were expected to perform abling students to adapt their efforts to reflect the simultaneously, and about the lack of coordination percentage of the final mark represented by each among subjects. Given this situation, and with the activity. The result was a substantial increase in aim of encouraging even greater participation in forum participation, as well as an enhancement of the forum, an activity that had proved extremely its value, thanks to an improvement in the quality useful in the judgement of both the teachers and of student postings. There was again an increase the students themselves, from 2007-2008 10% in the number of passes in that academic year, of the overall mark was allotted to assessment of confirming that the changes had been appropriate. participation in the forum. The forum came to be Finally, in 2008-2009 and 2009-2010 the group one of the most useful and valued activities, both tutorial activity was added, in which groups of up for the students, who appreciated its immediacy to 25 students discussed topics raised in the forum and ease of use, and for the teachers, who valued in preceding weeks. The aim of this activity was
308 Effective Use of E-Learning for Improving Students’ Skills
to foster the basic skills required by graduates, perceived the accessibility of their teachers, even as well as linking with other course activities; if such accessibility was only online. the introduction of this activity prompted a new Teacher participation in the forum is impor- increase in pass levels. tant and needs to fulfil a dual role: first it should motivate students to participate and exchange information, expressing appreciation for contribu- CONCLUSION tions and encouraging others to take part; secondly it should be used to put forward new discussion In recent years a combination of teaching strate- topics when forum activity abates. Students are gies including asynchronous discussion forums, seen to be more diffident about participating online quizzes and group tutorials, has been when it is the teacher who starts a discussion; it implemented in Industrial Engineering and Soft- is therefore advisable for the teacher to keep to ware Engineering degree courses with the aim the sidelines, intervening only to correct mistakes of improving some students’ skills needed in and encourage general participation rather than Engineering, such as ability to organise and plan, actually leading discussions. oral and written communication skills, estimating Weekly monitoring of student workloads has and programming work, design methods, strategic shown that the average effort expended has exceed- organisation and planning, problem solving and ed that anticipated by the teachers in their schedul- decision making, among others. These strategies ing in every one of the years under investigation. have prompted greater skills acquisition, as well This finding lends support to arguments in favour as enabling students to regulate their workload. of making future schedules more realistic, some- Surveys carried out at the end of each year thing that should also have a positive influence on showed that students considered the forum a very overall results. Weekly monitoring of scheduling useful means of exchanging information in general has also proved very useful in motivating students and clarifying concepts included in the course to keep up with the activities in a comprehensive content. Participation in the forum, and also the fashion, completing them in time, as planned by quality of postings, rose substantially from the their teacher. However, while the activities as a moment that assessment of this activity started whole are designed to help the students gradu- to contribute to the overall mark for the course. ally acquire the knowledge and skills needed for Perhaps as a result of greater use of the forum, the subject, it was found – every year – that the there was also an improvement in the way students number of hours of study increased substantially in the final weeks before the examination. It may
Figure 3. Final student marks as a function of the introduction of new teaching and evaluation methods
309 Effective Use of E-Learning for Improving Students’ Skills
therefore be necessary to pay more attention to Future prospects of research deal with improv- this aspect, in order to enable students to work ing activity planning, making it more realistic with more effectively and spread their workload more the students’ workload, analysing the real effect of uniformly over the time available. co-evaluation, not only on the final marks but also In all study years, students reacted positively on the students’ attitude in facing the academic towards completing the online quizzes, indicating activities, and finally, searching for new ways of that quizzes helped them to keep up to date in the evaluating skill acquisition. subject. Teachers displayed considerable regard for this tool in terms of the ratio between effort expended and results obtained; a few hours of REFERENCES correctly-focussed work at critical junctures was found to help consolidate subject content as well ABET. (2009). Criteria for accrediting engineering as contributing to the progress of both theoretical programs. Accreditation Board for Engineering and practical classes. Moreover, because quizzes and Technology, 29. Retrieved November 22, are scheduled far enough in advance and can be 2010, from http://www.abet.org/Linked%20 completed within a flexible time frame, they nei- Documents-UPDATE/Criteria%20and%20PP/ ther interfere with the normal development of the E001%2010-11%20EAC%20Criteria%201-27- teaching material nor take up students’ attention 10.pdf beyond what is absolutely essential. Andresen, M. A. (2009). Asynchronous discussion Group tutorials are a core activity in the new forums: Success factors, outcomes, assessments, teaching methodology. They enable the work of and limitations. Journal of Educational Technol- the group to focus on the topics discussed in the ogy & Society, 12(1), 249–257. forum, and they actively contribute to the development of skills such as the ability to manage Angus, D. S., & Watson, J. (2009). Does regular information, to analyse and synthesise, as well as online testing enhance student learning in the enhancing oral and written expression and other numerical sciences? Robust evidence from a competences. Peer assessment of this activity con- large data set. British Journal of Educational solidates the responsibility students feel towards Technology, 40(2), 255–272. doi:10.1111/j.1467- themselves as well as towards their colleagues, 8535.2008.00916.x even though they tended to award marks within a Banks, S., Goodyear, P., Hogson, V., & Mc- narrow band between 6 and 9 points. However, in Connell, D. (2003). Introduction to the special the two study years, peer scores was not decisive issue on advances in research on networked for the final marks obtained, given that there were learning. Instructional Science, 31, 1–6. no significant differences between the final marks doi:10.1023/A:1022583918064 and those awarded by the teachers alone, which indicates that the assessment system needs to be Barrella, E., Simmons, J., & Buffinton, K. (2006). revised in the future. Professional engineering education best practice According to students, skills improved sub- study for 1st-year, multi-disciplinary courses. stantially in each study year. The skills showing ASEE Annual Conference and Exposition, Con- most improvement were Design Methods and ference Proceedings (10). Organisation and Planning Ability; this is con- De Laat, M., & Lally, V. (2003). Complexity, sistent with the general philosophy of the content theory and praxis: Researching collaborative of the Projects module and the way the activities learning and tutoring processes in a networked were planned. learning community. Instructional Science, 31, 7–39. doi:10.1023/A:1022596100142
310 Effective Use of E-Learning for Improving Students’ Skills
De Laat, M., Lally, V., Lipponen, L., & Simons, R. Kibble, J. (2007). Use of unsupervised online J. (2007). Online teaching in networked learning quizzes as formative assessment in a medical communities: A multi-method approach to study- physiology course: Effects of incentives on stu- ing the role of the teacher. Instructional Science, dent participation and performance. Advances in 35, 257–286. doi:10.1007/s11251-006-9007-0 Physiology Education, 31, 253–260. doi:10.1152/ advan.00027.2007 Dennen, V. P. (2008). Looking for evidence of learning: Assessment and analysis methods for Kirkwood, A. (2009). E-learning: You don’t online discourse. Computers in Human Behavior, always get what you hope for. Technol- 24, 205–219. doi:10.1016/j.chb.2007.01.010 ogy, Pedagogy and Education, 18(2), 107–121. doi:10.1080/14759390902992576 Dobson, J. L. (2008). The use of formative online quizzes to enhance class preparation and scores on Kirner, T. G., Custódio, C. A., & Kirner, C. (2008). summative exams. Advances in Physiology Educa- Usability evaluation of the Moodle system from tion, 32, 297–302. doi:10.1152/advan.90162.2008 the teachers’ perspective. IADIS International Conference e-Learning (pp. 371-378). Erlin, Y. N., & Rhaman, A.A. (2009). Students’ Interactions in Online Asynchronous Discussion Kozaris, I. A. (2010). Platforms for e-learning. Forum: A Social Network Analysis. International Analytical and Bioanalytical Chemistry, 397(3), Conference on Education Technology and Com- 893–898. doi:10.1007/s00216-010-3587-x puter, 25-29. Lahwal, F., Amaimin, M., & Al-Ajlan, A. (2009). Ertmer, P. A., Richardson, J. C., Belland, B., Perception cultural impacts: Principles for trainers’ Camin, D., Connolly, P., Coulthard, G., & Mong, skills for e-learning. NCM 2009 - 5th International C. (2007). Using peer feedback to enhance the Joint Conference on INC, IMS, and IDC (Article quality of student online postings: An exploratory number 5331619, pp. 986-993). sStudy. Journal of Computer-Mediated Com- Lan, Y. F., & Yan, C. L. (2009). A practical ap- munication, 12, 412–433. doi:10.1111/j.1083- proach to encourage students’ participation in 6101.2007.00331.x asynchronous online discussions based on expec- European Commission. (2009). ECTS users’ tancy theory. International Conference on Virtual guide. Luxembourg: Office for official publi- Environments, Human-Computer Interfaces and cations of the European communities, p. 60. Measurements Systems (Article number 5068907, Retrieved July 19, 2010, from http://ec.europa. pp. 271-276). eu/education/lifelong-learning-policy/doc/ects/ Mazzolini, M., & Maddison, S. (2003). Sage, guide_en.pdf guide or ghost? The effect of instructor interven- Hew, K. F., & Cheung, W. S. (2008). Attracting tion on student participation in online discussion student participation in asynchronous online dis- forums. Computers & Education, 40, 237–253. cussions: A case study of peer facilitation. Comput- doi:10.1016/S0360-1315(02)00129-X ers & Education, 51, 1111–1124. doi:10.1016/j. McPherson, M. C., & Nunes, J. M. (2008). Criti- compedu.2007.11.002 cal issues for e-learning delivery: What may seem Kakasevski, G., Mihajlov, M., Arsenovski, S., & obvious is not always put into practice. Journal Chungurski, S. (2008). Evaluating usability in of Computer Assisted Learning, 24(5), 433–445. learning management system Moodle. Proceed- doi:10.1111/j.1365-2729.2008.00281.x ings of the ITI 2008 30th Int. Conf. On Information Technology Interfaces (pp. 613-618).
311 Effective Use of E-Learning for Improving Students’ Skills
Ministerio de Educación Cultura y Deporte. Tuparova, D., & Tuparov, G. (2010). Manage- (2003). REAL DECRETO 1125/2003, de 5 de ment of students’ participation in e-learning septiembre, por el que se establece el sistema eu- collaborative activities. Procedia Social and Be- ropeo de créditos y el sistema de calificaciones en havioral Sciences, 2, 4757–4762. doi:10.1016/j. las titulaciones universitarias de carácter oficial y sbspro.2010.03.764 validez en todo el territorio nacional. Boletín Ofi- Vivek, S., & Ansari, Y. (2010). Collaborative cial del Estado. Nº 224, 34355-34356. Retrieved learning issues in synchronous online interac- November 6, 2010, from http://www.boe.es/boe/ tions. International Conference on Technology dias/2003/09/18/pdfs/A34355-34356.pdf for Education (T4E), 5-8. Nevgi, A., Virtanen, P., & Niemi, H. (2006). Sup- Whitelock, D. (2009). Editorial: E-assessment: porting students to develop collaborative learning Developing new dialogues for the digital age. skills in technology-based environments. Brit- British Journal of Educational Technology, 40(2), ish Journal of Educational Technology, 37(6), 199–202. doi:10.1111/j.1467-8535.2008.00932.x 937–947. doi:10.1111/j.1467-8535.2006.00671.x Yau, J., Lam, J., & Cheung, K. S. (2009). A review Rehman, H. U., Said, R. A., & Al-assaf, Y. (2009). of e-learning platforms in the age of e-learning An integrated approach for strategic development 2.0. 2nd International Conference on Hybrid of engineering curricula: Focus on students’ design Learning and Education (ICHL 2009), LNCS skills. IEEE Transactions on Education, 52(4), 5685 (pp. 208-217). 470–481. doi:10.1109/TE.2008.930508 Zhao, J., & Yiang, Y. (2008). A case study on col- Rovai, A. P. (2008). Facilitating online discussions laborative e-tutoring. Proceedings of 2008 IEEE effectively. The Internet and Higher Education, International Symposium on IT in Medicine and 10, 77–88. doi:10.1016/j.iheduc.2006.10.001 Education (Article number 4743903, pp. 439-444). Salas-Morera, L., Berral- Yerón, J., Serrano- Zhou, Z. (2004). From classrooms to the real Gómez, I., & Martínez-Jiménez, P. (2009). An engineering world: The training program in the assessment of the ECTS in software engineer- microelectronics research center at Georgia Tech. ing: A teaching experience. IEEE Transactions IEEE Transactions on Education, 47, 188–120. on Education, 52(1), 177–184. doi:10.1109/ doi:10.1109/TE.2003.822262 TE.2008.921801 Savander-Ranne, C., Lundén, O. P., & Kolari, S. (2008). An alternative teaching method for ADDITIONAL READING electrical engineering courses. IEEE Transac- tions on Education, 51(4), 423–431. doi:10.1109/ An, H., Shim, S., & Lim, K. (2009). The effects TE.2007.912500 of different instructor facilitation approaches So, H. J. (2009). When groups decide to use on students’interactions during asynchronous asynchronous online discussions: Collaborative online discussions. Computers & Education, 53, learning and social presence under a voluntary 749–760. doi:10.1016/j.compedu.2009.04.015 participation structure. Journal of Computer As- Armellini, A., & Aiyegbayo, O. (2010). Learning sisted Learning, 25, 143–160. doi:10.1111/j.1365- design and assessment with e-tivities. British Jour- 2729.2008.00293.x nal of Educational Technology, 41(6), 922–935. doi:10.1111/j.1467-8535.2009.01013.x
312 Effective Use of E-Learning for Improving Students’ Skills
Badcock, P. B., Pattison, P. E., & Harris, K. L. Hew, K. F., & Cheung, W. S. (2010). Higher-level (2010). Developing generic skills through univer- knowledge construction in asynchronous online sity study: A study of arts, science and engineer- discussions: An analysis of group size, duration ing in Australia. Higher Education, 60, 441–458. of online discussion, and student facilitation doi:10.1007/s10734-010-9308-8 techniques. Instructional Science, 17. Ben Ammar, M., Neji, M., Alimi, A. M., & Ileh, M. T., & Saleh, I. (2009). Case based reason- Gouardères, G. (2010). The affective tutoring ing approach of tutoring in e-Ieaming platform. system. Expert Systems with Applications, 37, International Conference on Multimedia Comput- 3013–3023. doi:10.1016/j.eswa.2009.09.031 ing and Systems -Proceedings (Art. no. 5256711, pp. 150-157). Benson, A., Lawler, C., & Whitworth, A. (2008). Rules, roles and tools: Activity theory and the Jassó, J., Milani, A., & Pallottelli, S. (2008). comparative study of e-learning. British Journal Blended e-learning: Survey of on-line student of Educational Technology, 39(3), 456–467. assessment. 19th International Conference on doi:10.1111/j.1467-8535.2008.00838.x Database and Expert Systems Application (pp. 626-630). Bye, L., Smith, S., & Monghan Rallis, H. (2009). Reflection using an online discussion Khalid, Sh. U., Basharat, A., Shahid, A. A., & Has- forum: Impact on student learning and satisfac- san, S. (2009). An adaptive e-learning framework tion. Social Work Education, 28(8), 841–855. to supporting new ways of teaching and learning. doi:10.1080/02615470802641322 International Conference on Information and Communication Technologies (Art. Nº 5267175, Calvani, A., Fini, A., Molino, M., & Ranieri, pp. 300-306). M. (2010). Visualizing and monitoring effective interactions in online collaborative groups. Brit- Lan, W., Jiacheng, X., & Xueli, Q. (2009). Explo- ish Journal of Educational Technology, 41(2), ration of Moodle-based collaborative learning for 213–226. doi:10.1111/j.1467-8535.2008.00911.x the deaf (pp. 145–147). International Forum on Computer Science-Technology and Applications. Choi, H., & Kang, M. (2010). Applying an activity system to online collaborative group Manouselis, N., Vuorikari, R., & Van Assche, work analysis. British Journal of Educational F. (2010). Collaborative recommendation of e- Technology, 41(5), 776–795. doi:10.1111/j.1467- learning resources: An experimental investigation. 8535.2009.00978.x Journal of Computer Assisted Learning, 26(4), 227–242. doi:10.1111/j.1365-2729.2010.00362.x Dermo, J. (2009). E-assessment and the student learning experience: A survey of student Michinov, N., Brunot, S., Le Bohec, O., Juhel, J., perceptions of e-assessment. British Journal & Delaval, M. (2011). Procrastination, participa- of Educational Technology, 40(2), 203–214. tion, and performance in online learning environ- doi:10.1111/j.1467-8535.2008.00915.x ments. Computers & Education, 56, 243–252. doi:10.1016/j.compedu.2010.07.025 Dringus, L. P., & Ellis, T. (2010). Temporal transitions in participation flow in an asynchronous Moreno, L., Gonzalez, C., Castilla, I., Gonzalez, discussion forum. Computers & Education, 54, E., & Sigut, J. (2007). Applying a constructivist 340–349. doi:10.1016/j.compedu.2009.08.011 and collaborative methodological approach in engineering education. Computers & Education, 49, 891–915. doi:10.1016/j.compedu.2005.12.004
313 Effective Use of E-Learning for Improving Students’ Skills
Nicol, D. (2007). Laying a foundation for lifelong of teaching and learning is based on transparent learning: Case studies of e-assessment in large communication amongst teachers and students 1st-year classes. British Journal of Educational involved with a course with the aim of facilitating Technology, 38(4), 668–678. doi:10.1111/j.1467- the general learning process. 8535.2006.00657.x Co-Evaluation: Evaluation in which each student participating in an academical activity, Qi, C. X., Qi, Z. F., Sang, J. S., & Sun, Y.-J. (2010). is evaluated not only by the professor, but by Web-based collaborative learning system. Inter- each one of his/her group partners too. The aim national Conference on Networking and Digital of this technique is reinforcing the responsibility Society (pp. 557-560). of the students in their own knowledge process Rubio, R., Martín, S., & Morán, S. (2010). Col- by means of emitting critical judgments about laborative Web learning tools: Wikis and blogs. the others’ work. Computer Applications in Engineering Education, Collaborative Learning: Learning and teach- 8(3), 502–511. doi:10.1002/cae.20218 ing technique based on creating knowledge in a group of students by means of sharing activities Summers, M., & Volet, S. (2010). Group work and materials. These activities can include collab- does not necessarily equal collaborative learn- orative writing, group projects, problem solving, ing: Evidence from observations and self-reports. and debates, among others. European Journal of Psychology of Education, E-Learning: Transfer of skills and knowledge 25(4), 473–492. doi:10.1007/s10212-010-0026-5 my means of computers and Internet applications. Vighnarajah, Luan, W. S., & Bakar, K. A. (2009). It involves Web-based learning, computer-based Qualitative findings of students’ perception on learning, virtual classroom and digital collabora- practice of self-regulated strategies in online tion. community discussion. Computers & Education, Educational Technology: Practice of fa- 53, 94–103. doi:10.1016/j.compedu.2008.12.021 cilitating students learning by using appropriate technological processes and resources including Wang, Q. (2010). Using online shared workspaces software, hardware and Internet applications and to support group collaborative learning. Comput- activities. ers & Education, 55, 1270–1276. doi:10.1016/j. Engineering Education: Activity of teaching compedu.2010.05.023 basic and specialized knowledges needed for doing Wang, Q., & Woo, H. L. (2010). Facilitating professional practice of engineering. coordination in the collaborative learning pro- Group Tutoring: In the context of this chap- cess. 2nd International Conference on Education ter group tutoring is an activity in groups up to Technology and Computer (Art. nº. 5529271, 25 students used to discuss about the concepts V1-181-V1-185). of asynchronous discussion forum. It involves students doubts, further readings and general group knowledge construction, and it’s useful not only for making more comprehensive analysis of KEY TERMS AND DEFINITIONS subject’s concepts, but to reinforce speaking, writing, analyzing and synthesizing students’ skills. Blended Learning: Learning and teaching environment in which e-learning and e-tutoring, among others, are mixed with face-to-face activities. This combination of different models
314 315
Chapter 15 Strategies of LMS Implementation at German Universities
Carola Kruse Technische Universität Braunschweig, Germany
Thanh-Thu Phan Tan Technische Universität Braunschweig, Germany
Arne Koesling Leibniz Universität Hannover, Germany
Marc Krüger Leibniz Universität Hannover, Germany
ABSTRACT In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms.
DOI: 10.4018/978-1-60960-884-2.ch015
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Strategies of LMS Implementation at German Universities
INTRODUCTION As academic structures are very diverse from nation to nation and bring along different terms Using a central learning management system for affected people and institutions, we find it (LMS) in academic terms is subject to some necessary to clarify the terms and concepts we are strategies that are following the universities’ using here regarding German universities. Please, goals on several levels such as for educational refer to the section “Key Terms & Definitions” or administrative reasons. In this article, we are at the end of this article for more information. focussing on the different strategies at German universities on how to implement an LMS into everyday university life. The originating mo- TWO APPROACHES OF LMS tive is that an LMS that is intensely used by the IMPLEMENTATION IN GERMAN academic staff and students can also set a path EDUCATIONAL INSTITUTIONS to improve teaching and learning at university, either in regard of organisation or quality. We The use of new media, including LMS, was at understand under the intensity of LMS usage least until 2004 rather subject to time-limited both the quantity (How many people are using projects at many universities in Germany (Klei- the LMS?) as well as the quality (At what extend mann & Wannemacher, 2004). Although there and how diverse is the LMS used?). On the basis already existed initiatives to fold up technology of three case studies based upon implementation enhanced learning (TEL) activities within univer- processes at three universities in Germany, we sities some years before (e.g. “NewMediaNet”1 show different ways to anchor LMS usage within at the University of Freiburg), besides projects, an educational institution. the continuous use of new media was narrowed to The article is divided into two main parts. First, single departments. This becomes apparent in the we introduce general aspects of LMS implementa- succeeding nationwide calls for project proposals tion at German universities starting off with the issued by the Federal Ministry of Education and description and discussion of two approaches Research (in short BMBF) “Neue Medien in der concerning the implementation process (top-down Bildung” and “Neue Medien in der Bildung 2”. As and bottom-up approach). Then, we are going in the first call demanded project proposals for single to detail by showing how these approaches affect eLearning-units (Bundesministerium für Bildung the decision making on one LMS, the setting up und Forschung, 2000), the BMBF recognized that of services, the process of change management as it did not go any further than that: no processes well as the development of networking and col- were initiated to implement eLearning structures laboration. In the second part, we are focussing into the universities. For that reason the second on our cases by presenting the common ground call demanded project proposals to integrate eL- concerning strategy goals and LMS choice. Sub- earning into the universities sustainably (DLR, sequently, we are going into detail by recounting 2009). TEL related systems were operated due to how three universities in Hannover, Osnabrück and the interests and the engagement of single persons Braunschweig have gone through the process of and in some cases attracted larger user groups. But implementation. Finally, we are comparing these those bottom-up approaches were limited since experiences to the general information of the first they were not reflecting the needs of the university part and are making suggestions for forthcoming as a whole. However, particularly around that time, processes. some universities already identified the potential of new media and launched activities to provide
316 Strategies of LMS Implementation at German Universities
centralized services for the use of TEL systems. Pros and Cons of the Often an LMS is implemented as a starting point Two Approaches for TEL at university. While the selection of an appropriate LMS can be done by extensive evalu- From our experiences and observations in the ations and strategy considerations of technical German university landscape, there are diversified aspects like costs, performance, adaptability to advantages and disadvantages for both of these corporate identity and usability aspects within approaches. The top-down approach involves a this top-down strategy, a broad acceptance of risk that the focus on LMS is predominantly set the chosen solution regarding all concerned user on administrational processes than the didactical groups (teachers, students) cannot be predicted possibilities. For example, the academic staff is safely in advance. eased by organisational procedures through the Kerres (2005) distinguishes between two LMS but not supported in questions about how to approaches of organisation development for integrate an LMS within their course. Furthermore, integrating new media in universities. While the it is more likely that some people are very much minimal change on the one hand tries to do only displeased with the compulsory LMS usage – small changes in structures and processes to ensure probably another one than they have been using acceptance of the teachers, the active approach before. Also, it is possible that an LMS does not on the to other hand makes use of a change man- fulfil the demands of the different teaching and agement to induce extensive modifications to the learning cultures at the faculties. The sequential organisational structure. Depending on the attitude nature of engineering courses demands other of the key administration as well as its engagement features than the holistic nature of the humanities. towards the use of a common LMS, it evolves in For example, in mechanical engineering content a more or less profound change of the university is based on facts, laws and rules that often cannot landscape. Either it is at one period of time un- be questioned. In TEL, this content can easily decided whether to implement an LMS at all or be transformed into a Web Based Training or which LMS to choose to maintain university ad- learning quizzes. In comparison, the students of ministration and courses. In this case, it is possible a philosophy course build and discuss theses that that a faculty or even a singular department feels can be viewed and argumented from different per- the need for setting up an LMS for itself during spectives. In this case, regarding TEL, it would be that period of uncertainty. Often other faculties necessary to supply communication means such or departments notice how helpful an LMS can as discussion boards or chats. So, if the LMS does be for daily work and start to participate in using not offer the needed features, it is most likely that the LMS so the number of users rises gradually. one or the other faculty is dissatisfied with its use. This is what we call bottom-up approach. The However, the top-down approach also com- other direction is, when the key administration prises advantages. It ensures a high frequency of pushes the integration of LMS in the university usage which potentially maintains even the use so the faculties have to use the LMS for some of didactical possibilities of the LMS, because processes – often administrative processes – and the academic staff might become interested in hopefully recognize the benefits an LMS can didactical applications after they have got in touch provide. We call this top-down approach. through administrative use and widespread provision. It brings along centrally organized services to professionally support a large number of users with a wide range of services (e.g. the universities of Trier (2010), Frankfurt/Main (2010) or
317 Strategies of LMS Implementation at German Universities
Duisburg-Essen (2010) have one central service mean to adapt to several LMS structures with unit for all LMS users). As the LMS implementa- different interfaces and to overlook several data tion is headed by the key administration, the users sources and probably hold several user logins and can expect a sustainable operation of the system passwords, depending on the IT infrastructure of and support. For the academic staff as well as the the university. For example, at the Technische students, it is a relief that they can depend on one Universität Braunschweig, we have evaluated LMS only and do not have to change from one to that several students had to use up to six different the other system depending on the courses they LMS in their courses (Fleck, 2010). No matter give or attend. Singular institutions that would how many LMS are operated at the university, have operated an LMS on their own otherwise the bottom-up approach also means that the LMS do not have to put up with technical and support is not intended to be linked to other IT systems questions. such as the university management information The bottom-up approach takes the risk of hav- system or library system. ing miscellaneous LMS each with only a subset of Of course, this approach also has its advantages. users at one university. While each LMS demands Since the LMS was chosen based on specific de- technical operation and supporting services it is mands, it ensures users to have a concrete will to obvious that several LMS produce redundant use the LMS in their university courses. From our workload. Should there later on be a decision observations, the effect is that if not the number to use a single LMS the change management but the intensity of usage is relatively high as well will be difficult to merge different LMS users as the rate of self-organisation. Users do not have and data to one. Considering the practical use, to be convinced and are more likely to be satisfied for academic staff and students several LMS with the system. Therefore, they often recommend
Table 1. Overview of the two approaches
Approach Bottom-Up Top-Down Proceeding One faculty or one department starts using an The key administration pushes the integration of an LMS LMS, afterwards other faculties/departments into the university start participating Advantages LMS is chosen based on user demands. Users have High frequency of usage. Academic staff might get interested concrete will and request to use the LMS. High in the didactical possibilities during administrative usage. intensity of usage in courses. High satisfaction Other IT systems used at the university can be connected with the chosen system refinement participation to the LMS support and authentication can be organised by the users is higher centrally from the beginning. Students can use one single LMS and not different ones in their subjects. Disadvantages Low number of users. Coexisting of several LMS Focus is possibly on administrational processes. Didacti- in different faculties or departments. Redundant cal possibilities might play a minor role. People might be workload for supporting the several LMS. If displeased by the reason that one LMS is forced when they one LMS is chosen to be the general one at that have used another one before. The LMS mirght not fulfil university: difficulties to merge the different us- the special needs of some academic staff or departments. ers and data to one LMS. Several login data for Lower intensity of usage in courses academic staff and students. Users might have to handle with different LMS interfaces. Other IT systems are most likely not to be connected to the several LMS Rate of usage low usage/few users in the beginning, can high/many users from the beginning increase fastly Kind of usage Probably more didactical Probably more administrative
318 Strategies of LMS Implementation at German Universities
the usage to others so that there is not as much • didactical advantages (like online exercis- need for marketing activities. The participation in es, tutorials, learning diaries) evaluation and refinement processes of the system • administrative possibilities (like partici- and organisation is also usually high. pant lists, dividing participants into groups, Considering all these pros and cons that only configuring the whole system based on display a part of arguments and examples, it is one’s needs) obvious that one approach is not superior to the • technical possibilities (like stable opera- other. In the contrary, experiences in Germany tion, adaptability to individual means, in- show that the universities tend have gone through teroperability with other IT systems, data a mixture of both during changing phases. The protection) extend of the two approaches varies from university to university, as mentioned above, depending These parameters are assessed variously at dif- on the attitude of the key administration towards ferent institutions or rather universities. Probably a common LMS and also depending on the suc- the parameters are weighted differently depending cessfulness of singular institutions to apply a on whether the top-down approach or the bottom- specialized LMS to their individual needs. We up approach occurs at the certain university. can state that the decision or at least the ap- Considering the top-down approach, the proval of one central LMS by the key administra- technical and the administrative possibilities are tion is required to establish a sustainable univer- given more attention due to the fact that the key sity-wide solution. Experiences have proofed that administration follows the goal to unify the ad- one central LMS is necessary for several reasons: ministrative processes at the university. In the case The academic staff and students can rely on of a bottom-up approach, the didactical und com- one central system and do not have to switch from municational advantages may be more relevant one to the other. The operation and support of due to the fact that the students’ and academic one LMS is more efficient and avoids redundant staffs’ needs take centre stage and both groups are workload. In most cases, it is possible to link the more interested in didactics and communication. LMS with other central IT systems so that it can According to the weighting of the approaches, a be used more effectively in adminstrative terms decision is based upon the different possibilities as well as in terms of the diversity of teaching and of the LMS. After this decision is made, there learning. The university can distinguish itself from are subsequent decisions implied concerning others by creating a disctinctive image with the the tangible implementation of the LMS and the LMS. In the following, we are going to discuss services that can be provided. the processes of choosing one central LMS.
Choosing a Central LMS SERVICES AND CHANGE for a University MANAGEMENT
Baumgartner et al. (2002) consider the following When introducing new structures and processes parameters to be the critical ones for the decision in the university to integrate new media, there on LMS usage: are some basic services that need to be provided. Generally, we distinguish between central and lo- • communication possibilities (like internal cal services. In the following, when talking about messages, chat, discussion boards) services and change management, we mean central services and change management which have a
319 Strategies of LMS Implementation at German Universities
closer connection to the top-down-approach than • Students to the bottom-up one. The services are intended • Academic Staff to ease LMS usage for the students, the academic • Institutions (Departments, Institutes, staff and the university administration and give Schools etc.) them the skills to embrace the use of this media • Key administration and administrational within their activities. services Kubicek et al. (2004) identified eight fields of activities to consider when setting up TEL in an The activity field tasks are related to these educational institution. It is the services business stakeholders in a dissimilar way. In the following to make sure that these tasks are performed well table, the fields and stakeholders are matched. The in order to support all persons concerned in an fields’ relevance for the particular stakeholder is optimal way. marked with the symbols +, o and -. “+” means high or very high relevance • Integration to university administration “o” means middle-rate relevance (neither high • Integration into curricula nor low) • Qualification and pedagogic support, “-“ means low or no relevance consulting It is important to keep the demands and aims • Development of content of these groups in mind when taking a look at the • Operation of systems, technical service activity fields. Some of these them are the same and support for different stakeholders such as the need for • Marketing security, privacy or availability, but many of them • Quality Management are particular for one of the stakeholders. For • Strategy Development example, Students want to get necessary information One can identify four types of stakeholders easily. Within a university there are many sources in a university context. The acceptance of these of information and it can be difficult to find out affected groups is most essential to the work of where to get which information. In the course of the integrating new media. Bologna process (European Commission, 2010), the universities are restructuring from diploma to
Table 2. Overview of fields of activity and the different stakeholders
Students Academic Staff Institutions Key administration and services Integration to university administration - - o + Intgration into curricula o o o + Qualification and pedagogic support, o + o - consulting Development of content o + - - Operation of systems, technical service + + + + and support Marketing + + o + Quality Management + + + + Strategy Development - - o +
320 Strategies of LMS Implementation at German Universities
bachelor and master degree programmes. This their courses replaced by pure online learning makes it complicated for the students’ orientation (Kaltenbaek, 2009). due to their tight timetables. An LMS can provide Institutions want to stand out within the uni- a central information source and planning tool for versity as there is often a competition between their study life. the several departments offering the best courses Also, they need possibilities to communicate and being most innovative. A basic need of all with the academic staff and fellow students. New departments is to organize their person and course media offer many communication channels, so data most efficiently and effectively. Due to the the students expect to be able to use them within fact that there is a lot of these data, it has to be the university as well. In addition students have clear what data can be found in which place and to use every possibility to learn and to exercise. is transferred to which other systems at what time. Having to go through many exams and to render In the same way key administration and ad- many course achievements, students need some ministrational services need to have all IT systems space to learn alone or collaboratively. These integrated into a network within the university and spaces can also be provided virtually in addition well-defined data sources (Igel, 2009). Although to the courses they attend and the things they do the leadership is aware of the institutions’ com- at the library or at home. petition, it wants all institutions to cooperate in Furthermore, some students are interested in order to have a functioning unit. Since university mobile learning as well as the use of new tech- teaching gradually becomes more important it nologies. They expect the university to provide wants an improvement of teaching as well. Al- state-of-the-art digital media, and they are entitled together, the goal is to satisfy the academic and to do so due to the fact that they are paying high administrational staff and to ensure a high quality admission fees for their studies. standard for the whole university. Besides the Academic staff has to use the time for teaching competition amongst academic staff as well as effectively. As having several functions within a institutions, there is also a competition between university – doing research as well as preparing universities in the country or internationally to and holding several courses – academic staff has have the best reputation. to organize its time well. Since university teach- It is obvious that Kubicek’s et al. activity-fields ing becomes gradually more important (Stifter- cover the needs and aims of the different groups verband für die Deutsche Wissenschaft, 2010), unequally. Some of them concern all groups, for an increasing number of academic staff wants to example the operation of systems or qualification, improve their courses. At the moment, there is a others are more important for a particular group, rising number of initiatives initiated by the Federal for example the development of content for the Ministry of Education and Research to improve teachers. Regarding the top-down and bottom-up teaching and learning at universities in Germany approach, it is clear that the needs and aims as (Stifterverband für die Deutsche Wissenschaft, well as the activity fields which are related to the Bundesministerium für Bildung und Forschung, affected groups become more important in every Winde, 2010).To achieve that goal, the academic special case. For example, considering the bottom- staff needs to organize a lot of material for the up approach, students and teachers are most likely courses, including several subjects. The mate- the groups that are more affected because they are rial has to be sorted and to be filed well. Even the first persons that act in that case. So at first, though more and more academic staff members their needs and aims take centre stage and so do want to refine the material for the courses and to the co-related activity fields. Otherwise, consider- integrate online units, they do not want to have ing the top-down approach, the needs and aims of
321 Strategies of LMS Implementation at German Universities
key administration and administrational services or other external institutions. From a technical as well as the institutions are the groups that have point of view, there is on the one hand network- to be brought into the focus and therefore other ing and collaboration focussing on the LMS activity fields play a decisive role. itself and its environment and on the other hand Just as every university needs the services, it the networking and collaboration concerning the also has to apply a change management when im- interrelationship between the LMS and other IT plementing new media. It covers several decision systems and their environments. processes as well as the coordination of different In the following, a number of parties are listed fields. It starts with an evaluation of the state of that possibly network and/or collaborate in one the art, a strategy on convincing other LMS users way or the other at a university and beyond: to change the system, and goes on to planning the linking of other important IT systems as well as • Central service units within the university, establishing an operation and support unit with a e.g. the university library with an online qualification concept for several thousand people. library system, the computer centre, the Other fields of change management concern the public information office responsible for involvement of stakeholders as well as the strategy the university’s website), and others of further development (Seufert, 2008). As many • The university administration as most of universities are undergoing similar processes, an the university administrations have an ad- information exchange between them is necessary ministrating system for matriculation data for an effective progress. This complex process or staff directories shows that there is some networking and col- • The faculties as institutions within a laboration on multiple levels. university • The teachers and the students within the Network and Collaboration university teaching different subjects and also from different universities teaching First of all, it is necessary to differentiate between the same subject networking and collaboration. Networking stands • The key administration, especially the for the cooperation of different parties whereas vice president responsible for teaching and each party has its own set of tasks and decision learning power. A network is established to make use • Other universities using this special LMS of the different expertises and to exchange that or having experiences with implementing know-how in the perspective of a special issue of LMS common interest. Collaboration of different parties • Other universities having didactical expe- on the other side serves to share their tasks and rience in content production and/or the de- decisions in order to reach a common goal. Both velopment of TEL courses networking and collaboration should not be viewed • Concerning LMS with open source code: seperately. One can be the grounds for the other developers from other universities, compa- at certain stages. Both have the characteristic to nies or with private interest set free synergetic effects. Concerning the LMS implementation there is How networking and collaboration works shall the need for networking and collaboration on an be demonstrated with the following two examples. internal and external level. We understand internal as the level within a university and external as the connection between two or more universities and/
322 Strategies of LMS Implementation at German Universities
The Task Force on Technology university-specific functions in generic ways to let Enhanced Teaching and the community benefit absorbs a great deal of time Learning (Network) because of discussions, compromises and finally reprocessing the software-code. Nevertheless, by Since an LMS is implemented for the use in the use of plugins with a specialised functionality teaching and learning, it is obvious against the spectrum, the Stud.IP community experienced an background of a diversity of teaching and learning innovative way of developing and sharing ad- cultures that it holds a large variety of applica- ditional functions needed for university-specific tion scenarios. In order to convince professors, requirements. Plugins add functionality to the base academic staff and students of the advantages, system, keeping the addition of this functionality the service units have to be aware of this wide optional. In the field of didactics and change man- range. This is why in Northern Germany, about agement, the universities are able to profit by the ten universities from different federal states have experiences of the other universities, especially the built a task force to develop and exchange special TEL interested staff and later the central institu- issues on Technology Enhanced Teaching and tions that are responsible for the LMS. Learning. Normally, the service units do not have identical profiles so that they often vary in exper- Strategy Goals at Lower tise and services. So, it is of a great profit when Saxon Universities one university displays its experiences in online- lecture production for example. Other universities After we have discussed the two approaches and can give critical feedback and have a practical their impact on different aspects such as LMS example about the pros and cons of this special choice, service and change management as well as service. Furthermore, the universities can decide networking and collaboration, we will now make to collaborate concerning this special service in the shift to our case studies. At our universities order to make quicker advancements (Phan Tan (Universities of Braunschweig, Hannover and & Prey, 2010; Phan Tan & Krüger, 2009). Osnabrück, all are situated in the German federal state Lower Saxony) that we are representing in The Developer Community of an this article, we mainly follow four strategic goals: Open Source lMS (Collaboration) The first goal is to achieve a high penetration of LMS usage by attracting the faculty staff. In the case of an LMS like Stud.IP2, those develop- Secondly, the main argument for using an ers are employed by the universities using Stud. LMS instead of traditional means of course ad- IP having the task to adapt Stud.IP to the needs of ministration and moderation is in the main point that specific university. A core group of develop- the efficient and effective use of LMS tools. It is ers steers the developments of all programmers to absolutely plausible for a teacher to use the LMS some extend resulting in benefits for every partici- instead of a paper/pencil list when it comes to pating university. However since those developers maintain attendance lists or administer learning primarily represent the needs of a single university, groups within a course. Further administrative this redirection of effort to maintain one single advantages such as the distribution of digital monolith software product is a non-trivial task. documents and bibliographies are as convincing as On one hand, dearly needed maintenance tasks communicative advantages when using discussion for the software is not covered properly, since the boards, chats and wikis to collect, facilitate and focus always is on the addition of new functions. summarize the students’ contributions. On the other hand, the pace of implementing the
323 Strategies of LMS Implementation at German Universities
Thirdly, it is intended to take the high penetra- a strict course structure, Stud.IP offered courses tion of LMS usage as a basis for a larger diversity that were connected to teachers, professional of teaching and learning through media as well groups and faculties. The teachers were enabled as an optimised way of traditional teaching and to create online course rooms with a few clicks learning supported by an LMS. Examples of diver- and therefore it required little experience for the sity are for instance online and blended learning system itself. Primarily, the functionalities were formats, student oriented collaborative learning, limited to file upload and download, participants subject/university comprehensive courses or lists, discussion boards and literature lists and international courses. therefore focussed on the support of attendance Last but not least, the use of an LMS is also a tuition. In other words, Stud.IP is not yet natively contribution to the Bologna process. Especially, capable of creating WBT or playing SCORM comprehensive and international courses draw the modules while other LMS were more eligible to universities together and are a great opportunity support distance learning and commercial further for exchange. education. It is closer to the traditional nature of There are different ways to reach these strat- teaching and learning at German universities than egy goals. To exemplify this fact we regard three some other LMS and therefore a low-threshold universities in Lower Saxony, Germany, that technical application for teachers and students. implemented Stud.IP as LMS and also take part Moreover, Stud.IP was targeted to the concerns in the Stud.IP network but with different foci. of students right from the start: Each course that a student is subscribed to, is entered into a calendar, The Decision on “Stud.IP” at so are changes in dates for lectures. Besides the Lower Saxon Universities file upload and download functionality and the simple to use communication with teachers, this The possibility and decision to install and offer function is most popular among students and eases the LMS Stud.IP was often based on the fact that their study life. Altogher, eleven universities are Stud.IP is Open Source software and beyond the using Stud.IP in Lower Saxony. Only very few effort for installation without further costs in test- made a clear decision against it. In the following, ing phases. For both approaches, this given fact we describe what is necessary for implementing was ideal for a start. Furthermore, the universi- an LMS into the university and why it is obvious ties in Lower Saxony started networking early in to collaborate especially in this process. 2002 (the so-called ELAN (E-Learning Academic Network Niedersachsen (2010)) projects, funded by the Ministry of Science and Culture3) and CASE-STUDIES noticed the associated universities’ experienced benefits by using Stud.IP. This was a crucial Technische Universität argument for many universities to decide on this Braunschweig (TU BS) LMS, too. Again, it counts for both approaches, for it was possible to exchange experiences on At the Technische Universität Braunschweig, different levels. Stud.IP was introduced in 2004 during the ELAN The significant differences to other LMS be- projects, when Braunschweig was amongst other came evident: The structure of Stud.IP was more tasks responsible for the didactical support of the suited to represent the organisational structure of manyfold TEL projects in the network. German educational institutions and to tackle their The system was tentatively installed by the needs. For example, while other LMS followed universities’ central service unit Kompetenz-
324 Strategies of LMS Implementation at German Universities
zentrum Hochschuldidaktik für Niedersachsen The Technische Universität Braunschweig adds (KHN), the Centre of Excellence for Lower authority to the students suggestions and requests Saxon Higher Education, as a consequence of with regard of their quality assurance (Faßbender, its media didactical consulting activities. The Miosge 2010). The usage of Stud.IP is seen as one faculty of construction engineering started the of the activities making a contribution to quality use of the system and funded the support done at improvement at the university. KHN after the successful introduction of Stud.IP. Considering networking and collaboration This bottom up constellation (support at KHN, within the university, it was a major advantage financial funding from a university department) that the 5-year-period of bottom-up develop- was continued until 2009. Within this period, ment lasted this long, as the networking activi- word-of-mouth recommendation took place and ties with the faculty of construction engineering the user community grew up to over 10,000 users, and the IT centre were built on solid grounds of although there was no proactive advertisement common interest. The top-down desicion for a for the system. Since 2006, a cooperation with university-wide system was another push for the the university IT centre was started, because other administrative systems to collaborate. For the increased payload of the system required an example, special plugins are being programmed enhanced technical service and more suitable in comprehensive teamwork to enhance library localization. Furthermore, the KHN developed functions and provide a single sign on for the IT several didactical concepts for using this LMS. systems. A task force with representatives from all Those concepts were the results from didactical IT systems is exchanging experiences, problems workshops where the participants – professors and demands on regular terms in order to further and academics – prepared their own technology develop technical and organisational questions enhanced teaching. With the help of Stud.IP, the that relate to all IT systems. On external terms, the teaching and learning programme was extended TU Braunschweig is both member of the network by online- and blended-learning components. on technology enhanced teaching and learning as Therefore, the participating university teachers well as the developer core group mentioned above. were able to actively experience the didactical As the KHN is offering its teaching and learning integrated use of Stud.IP in the learning process. programme to all academic staff in Lower Saxony, In autumn 2009, after a comparative evalua- too, it raises the comprehensive awareness for tion with several other LMS, there was a decision didactical questions on TEL. for a university-wide usage of Stud.IP at the TU Braunschweig covers about 13,500 students, Braunschweig and its interlink to administrative 240 professors and almost 3,000 additional teach- systems (university management information ers. Currently, there are 12,000 user registered system, library, content management system, in the system (11,000 students, 1,000 teachers, and portals). Since that point of time, human administrators, and tutors) of which more than resources and adequate technical hardware has 8,300 users (or 7,500 students) have been active been provided from central university funding to during the last semester in the summer of 2010 with facilitate support, services and operation of the nearly 2,000 active courses and 4,700 documents. system. The KHN is still responsible for supervision, therefore this is a clear commitment to a Leibniz Universität Hannover (LUH) didactical focal point in the utilization of the LMS. The students demand the usage of Stud.IP in their The first Stud.IP installation at the Leibniz Uni- courses in the meantime due to the fact they got versität Hannover was set up in 2004 at L3S Re- used to the advantages for their learning process. search Center, a central institution at the university
325 Strategies of LMS Implementation at German Universities
concerned with digital learning and knowledge. the financial support of this so-called e-learning After the Stud.IP system was provided by L3S, service department (“elsa”) has been taken over the Leibniz Universität Hannover experienced a by the university itself. “elsa” is also member of surprisingly high demand. The information that the the two networks mentioned above. Stud.IP system was available spread very quickly Against the background that the LUH has within the university. Therefore, without requiring 20,600 students, 308 professors and over 2,200 further advertisement for Stud.IP, several teachers additional teachers the following figures can be used it within their lectures. given: During the last semester in the summer So, the decision to use Stud.IP as LMS at the of 2010, there were 14,700 student users online LUH resulted from a high demand by the teachers, at least one time within the last 30 days. In ad- not from a systematic comparative evaluation of dition, 1,100 professors and teachers as well as LMS. It is believed so far that this development is 600 tutors are using Stud.IP. Up to 150 users a good basis of decision making. Not IT experts, are concurrently online at the same time and 1.7 administrative officers, executive comittees or million documents have been downloaded in the TEL experts did take decisions in advance, but winter semester 2009/2010. the teachers themselves. Hannover is sure that they provided a system giving the most benefits Universität Osnabrück (UOS) in daily use for teachers and students this way. Since the midyear 2005, the widespread use of The Stud.IP System has been operated at Univer- Stud.IP at the LUH has been systematically fos- sität Osnabrück since 2003. The decision of the tered, however generously funded by the Federal university leadership to use Stud.IP as a manda- Ministry of Education and Research. Therefore, tory LMS for the management of all university several measurements, information events, train- courses and other purposes was early and expected. ings, interlinks of Stud.IP to HIS-LSF (Hoch- Stud.IP has therefore been used university-wide schulInformationsSystem, 2010) (the university since 2004. For example, the course catalogue management information system in which course and staff directory of the complete university is data is administrated) and ILIAS (another LMS, generated from Stud.IP and provided as PDF on but with a stress on content production) (ILIAS, the website. Stud.IP is also used for the course 2010), systematic further development within the room management of the university as well as all developer community, didactical orientation of electronic application procedures. The university trainings were performed.Within an extensive TEL established a department called “Zentrum für strategy, Stud.IP was established as the official Informationsmanagament und virtuelle Lehre der LMS of LUH in summer 2008. In the meantime, Universität Osnabrück (virtUOS)” responsible Stud.IP is used for administrative matters as well for the implementation and maintenance of TEL as for supporting university courses. Stud.IP can systems in an organisational and technical manner. be seen as well-established now. Osnabrück was right from the start a location at Considering networking and collaboration which the use of an LMS was supported strongly within the university, it is obvious that after the by a top-down approach, so the human resources induction phase at L3S it was mainly due to the for support are mostly provided by central uni- funded project that led to university-wide col- versity funds. laboration. So, it was possible to set up a profound Services with a didactical focus like trainings and elaborate central team that had the task to and consulting are also done, but they are not a focal spread and gather all information and processes point of the Osnabrück range of services regarding within the university. After the funding phase, Stud.IP. This seems to be due to the fact that the
326 Strategies of LMS Implementation at German Universities
executive board demands that Stud.IP supports However, even though Stud.IP was introduced the university administration on principle terms. top-down it is not only used as a course catalogue “virtUOS” is a central service unit consider- only, when an adequate service is provided. This ing most administrative systems at the university, is indicated by the following figures: There are therefore it is not as much necessary to collaborate 34,427 lectures with 247,852 documents in the with other institutions as the other two universi- system, 3,507 literature lists, 445,848 lecture-dates ties. Due to this matter of fact, it is possible to and 8,705 news. make shortcut decisions and carry them out almost instantly. This makes it possible to collaborate on more elaborate terms, such as developing TEL THE CASE STUDY UNIVERSITIES: based introductions to the library coorporately AN OVERVIEW with the library. Like the universities in the other two case studies presented, Osnabrück is highly Table 3 shows an extract of the case studies, integrated in a network. The focal point of Uni- focussing on the key features with respect to the versität Osnabrückis to play a leading role in the two approaches. Stud.IP development within the Stud.IP developer community. The introduction of Stud.IP as a central unver- CONCLUSION sity-wide LMS is also illustrated by the amount of users. The Universität Osnabrück covers about Each case study in this article was a blend of the two 9,300 students, 214 professors and over 1,100 approaches. While at the Leibniz Universität Han- additional teachers. During the last semester in nover the top-down and the bottom-up approach the summer of 2010, almost 8,400 students and came about equal, at the Universität Osnabrück about 3,400 other university members (professors the top-down approach was realised more than and academic staff) are actively using the system. the bottom-up. Quite contrary to Hannover, the
Table 3. Overview of key features on the LMS implementation at three universities
TU Braunschweig Leibniz Universität Universität Osnabrück (13,500 students) Hannover (9,300 students) (20,600 students) Strategy/Approach More bottom-up than top-down Equally top-down and bottom- More top-down than bottom- up up Implementation first 2004 first 2004 first 2003 university-wide 2009 (imple- university-wide 2008 (imple- university-wide 2004 mentation process in progress) mentation process completed) Focus of use didactical hybrid administrative Funding institutional funding from 2004 institutional funding from 2004 third-party project funding to 2009 to 2005 from 2003 to 2004 central university funding since third-party project funding central university funding 2009 from 2005 to 2008 since 2004 central university funding since 2008 Institution responsible for within the university, teaching within the university, e- within the university, e- service and learning centre learning centre learning centre Number of student users 7,500 (57%) 14,700 (71%) 8,400 (90%)
327 Strategies of LMS Implementation at German Universities
TU Braunschweig was determined more by the the resources for keeping the LMS running at all bottom-up approach than by the top-down ap- three universities are paid by the central university proach, because the university leadership came to funding in the meantime. In addition, every uni- the decision to have Stud.IP as the central LMS for versity has institutionalised the offer for didacti- the whole university at a rather late point of time. cal, technical and other kinds to provide services As one can see in the table above, the three either by instructing an existing institution within universities implemented Stud.IP almost even the university that was already concerned with though not quite at the same time. So the net- supporting the LMS and its users before without work pictured earlier in this article was founded having the official instruction – or by founding a along these developments offering a platform of new specialized institution. At all universities, we exchange at a very early stage of growth, whereas experienced the necessity to have such a central one university was able to make the step from a institute to bring the affected persons and institutes project to an established system earlier than the together and to make commonly gainful decisions. others. Simultaneously, the network encouraged We assume that the approaches have been not only the described but also other Lower Saxony successful, since all three universities have an universities to use Stud.IP as an LMS in a manner above average intensity of usage. Neverthe- of a domino effect. Altogether, the networking got less, a continuous service offer is mandatory to essential for the further development and estab- guarantee the proceeding of the use of the LMS. lishment of Stud.IP at most universities. At the moment there is a basic support for user Nevertheless or even therefore, each univer- requests and mentoring and a course programme sity described in the three case studies generated with lessons in getting the basic information about a main focus of LMS use which depends on the the LMS across to the teachers, students and the approach that determined the implementation administrative staff. process. While the Universität Osnabrück has an administrative focus due to the fact the LMS was implemented mostly via top-down approach and OUTLOOK the university leadership preferred using the LMS to support the university administration, the TU For the further etablishment of Stud.IP, we have Braunschweig uses the LMS to support the faculty pegged the following tasks as the logical steps that staff and improve the lectures that way because have to be done. There are mainly didactical, but the bottom-up approach was the superior one for also technical tasks. the implementation and the academic staff was Enlarging training offerings and the didacti- able to bring in their requirements and aims very cal mentoring to supply specialised needs like early. Since the Leibniz Universität Hannover had realising several didactical settings including a top-down approach equivalent to the bottom-up online work sheets etc. Currently, the trainings approach, the usage of the LMS has a hybrid char- just give an introduction to the LMS usage by acter, for the faculty staff as well as the university teaching the participants to use the basic LMS leadership were able to bring in their requirements features like uploading files or managing the list and aims at an equal share. For the next step, it is of their course participants. Another important task appropriate to analyze if the approach by which is appointing TEL coaches to support individual an LMS is integrated into university is related to ideas and teachers in realising their individual the long-term kind of usage of the system. teaching ideas. For these means, the universities As being deducted from the fact that each are collaborating by exchanging trainers for spe- case study includes both of the two approaches, cial topics or by offering comprehensive courses.
328 Strategies of LMS Implementation at German Universities
The next important technical step is to interlink Bundesministerium für Bildung und Forschung. Stud.IP with more of the other systems. As many (2000). Förderprogramm Neue Medien in der of the universities in Germany use a university Bildung. Lehr- und Lernsoftware. Retrieved on management information system, Hannover and December 20, 2010, from http://www.bmbf.de/ Braunschweig as well as three other universities pub/neue_medien_bildung.pdf are working on a middleware in the so-called “Pro- Bundesministerium für Bildung und Forschung. Liza” project. The goal is to develop an interlink (2000). Qualitätspakt Lehre. Retrieved on De- between Stud.IP and HIS-LSF (see the Hannover cember 20, 2010, from http://www.bmbf.de/ case above) in order to pass through the basic data de/15375.php about courses and academic staff so that it is not necessary to fill in the data at different sources and Deutsches Zentrum für Luft- und Raumfahrt produce redundant workload. The middleware is (DLR). (2009). Projektträger im DLR – Neue programmed collaboratively and the universities medien in der bildung, Retrieved on December 20, are exchanging experiences in the organisation of 2010, from http://www.dlr.de/pt/desktopdefault. the implementation process. The finished product aspx/tabid-5882/9540_read-18559/ shall be applicable to other universities using the Elan (n. d.). E-Learning Academic Network two systems as well. ProLiza is another example Niedersachsen. E-Learning Academic Network for the effectiveness of a good working network. Niedersachsen. Retrieved on August 17, 2010, In the end, it seems that networking as a very from http://www.elan-niedersachsen.de important strategy to foster LMS usage. By the exchange of experiences and evaluation results, the European Commission. (2010). The bologna universities are able to adapt the good experiences process - Towards the european higher educa- and validated results and to avoid mistakes and tion area. Retrieved on August 30, 2010, from extrawork. As the universities are not developing http://ec.europa.eu/education/higher-education/ straightforwardly but more in terms of circles, doc1290_en.htm the past has shown that each university can profit Faßbender, H., & Miosge, C. (2010). Technische form the other, no matter if it has started off first Universität Braunschweig. Hervorragende as an early adopter or if it has jumped on the Qualität und viel Raum für eigene Ideen. In bandwagon lateron. Winde, M. (Ed.), Von der Qualitätsmessung zum Qualitätsmanagement. Praxisbeispiele an Hoch- schulen (pp. 48–57). Essen, Germany: Edition REFERENCES Stifterverband. Baumgartner, P., Häfele, H., & Maier-Häfele, K. Fleck, W. (2010, August 24). Evaluationsergeb- (2002). E-learning praxishandbuch: Auswahl nisse von studierendeninterviews zu Stud.IP, von lernplattformen; Marktübersicht - funktionen Sommer 2010. Unpublished manuscript. Goethe - fachbegriffe. Innsbruck, Austria: Studienverl. Universität Frankfurt am Main. E-Learning Cen- Bundesministerium für Bildung und Forschung. ter am HRZ. Retrieved August 30, 2010, from (2000). Exzellenzinitiative. Retrieved on Decem- http://www.rz.uni-frankfurt.de/internet/elearning/ ber 20, 2010, from http://www.bmbf.de/de/1321. index.html php HochschulInformationsSystem. (n. d.). Lehre, Studium, Forschung. Retrieved August 30, 2010, from http://www.his.de/abt1/ab10
329 Strategies of LMS Implementation at German Universities
Igel, C. (2009). Learning-management an hoch- Phan Tan, T.-T., & Prey, G. (2010). Programm zur schulen. In Issing, L. J., & Klimsa, P. (Eds.), didaktischen e-kompetenzentwicklung in Nied- Online-Lernen. Handbuch für Wissenschaft ersachsen. Zeitschrift für hochschulentwicklung, und Praxis (pp. 207–215). München, Germany: 5(1). Retrieved from http://www.zfhe.at Oldenbourg. Seufert, S. (2008). Innovationsorientiertes bil- ILIAS. (n. d.). ILIAS Information Center. Retrieved dungsmanagement: Hochschulentwicklung durch August 30, 2010, from http://www.ilias.de sicherung der nachhaltigkeit von e-learning. Wi- esbaden, Germany: VS Verlag für Sozialwissen- Kaltenbaek, J. (2009). Hochschule online - Online schaften / GWV Fachverlage GmbH Wiesbaden. lehren und lernen in der Hochschule. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch Stifterverband für die Deutsche Wissenschaft. (n. für Wissenschaft und Praxis (pp. 367–388). d.). Exzellenz in der Lehre. Aktuelle Programme München, Germany: Oldenbourg. und Initiativen. Rertieved December 20, 2010, from http://www.stifterverband.de/ Kerres, M. (2005). Strategieentwicklung für die nachhaltige implementation neuer medien in der Stifterverband für die Deutsche Wissenschaft. hochschule. In T. Pfeffer (Ed.), Medien in der (2010). Lehre neu denken. Die Zukunft des Wissenschaft, Vol. 32. Handbuch Organisation- akademischen Lehren und Lernens. Retrieved sentwicklung. Neue Medien in der Lehre: Dimen- December 20, 2010, from http://www.stifter- sionen, Instrumente, Positionen (pp. 147–162). verband.info/publikationen_und_podcasts/po- Münster, Germany: Waxmann. sitionen_dokumentationen/lehre_neu_denken/ lehre_neu_denken.pdf Kleimann, B., & Wannemacher, K. (2004). E- learning an deutschen hochschulen: Von der Universität Duisburg-Essen (u. d.). Zentrum für projektentwicklung zur nachhaltigen Implemen- Hochschul- und Qualitätsentwicklung: Kompe- tierung. Hannover, Germany: HIS. tenzentwicklung in Studium & Lehre. Retrieved August 30, 2010, from http://zfh.uni-duisburg- Kubicek, H., Breiter, A., Fischer, A., & Wiede- essen.de/elearning wald, C. (2004). Organisatorische einbettung von e-learning an deutschen hochschulen. Re- Universität Trier (u. d.). Koordinationsstelle E- trieved on August 30, 2010, from http://www. Learning. Retrieved August 30, 2010, from http:// ifib.de/publikationsdateien/MMKH_Endberi- www.uni-trier.de/index.php?id=33362 cht_2004-05-26.pdf Winde, M. (Ed.). (2010). Von der Qualitätsmes- Phan Tan, T.-T., & Krüger, M. (2009). Hoch- sung zum Qualitätsmanagement. Praxisbeispiele schulübergreifende weiterbildung zum e-learning an Hochschulen. Essen, Germany: Edition in niedersachsen. In Apostolopoulos, N. (Ed.), Stifterverband. Grundfragen multimedialen lehrens und lernens. Bildungsimpulse und bildungsnetzwerke: Tagungsband GML 2 2009, 12. - 13. März 2009 (pp. 69–80). Berlin, Germany: Univ.-Verl. der TU Univ.-Bibliothek.
330 Strategies of LMS Implementation at German Universities
ADDITIONAL READING Bertelsmann-Stiftung. (2001). Studium online: Hochschulentwicklung durch neue Medien (2. Albrecht, R. (2001). Lehren und lernen mit neuen Aufl.). Gütersloh, Germany: Verl. Bertelsmann- medien: Plattformen, Modelle, Werkzeuge. Mün- Stiftung. ster, Germany: Waxmann. Bremer, C., & Kohl, K. Eleonora. (2004). E-learn- Anderson, T. (2004). Theory and practice of ing-strategien und e-learning-kompetenzen an online learning. Athabasca, Canada: Athabasca hochschulen. Bielefeld, Germany: Bertelsmann. University. Bruns, B., & Gajewski, P. (2000). Multimediales Apostolopoulos, N. (Ed.). (2009). Grundfragen lernen im Netz: Leitfaden für entscheider und multimedialen lehrens und lernens: Bildungsim- planer (2. Aufl.). Berlin, Germany: Springer. pulse und bildungsnetzwerke: Tagungsband GML 2 2009, 12. - 13. März 2009. Berlin, Germany: Dick, E. (2000). Multimediale Lernprogramme Univ.-Verl. der TU Univ.-Bibliothek. und telematische Lernarrangements: Einführung in die didaktische Gestaltung. Nürnberg, Germa- Appelrath, H.-J., Boles, D., Kleinefeld, N., Mar- ny: BW Bildung und Wissen Verl. und Software. cos, I., Reil, D., Runge, M., & Willer, S. (2006). Einsatz des open-source-lernmanagementsystems Eileen O’ Donnell. (2010). E-learning can im- Stud.IP zur Unterstützung der Präsenzlehre der prove learning: Preparing students for work. Universität Oldenburg. In C. Hochberger (Ed.), Saarbrücken, Germany: LAP Lambert Academic GI-Edition Proceedings: Vol. 94. Informatik 2006: Publishing. Informatik für Menschen; Beiträge der 36. Jahres- El-Ghareeb, H. (2010). Evaluation of service tagung der Gesellschaft für Informatik e.V. (GI); oriented architecture in e-learning: Integrating 2. bis 6. Oktober 2006 in Dresden (pp. 53–58). Bboth LMS and UMIS. Saarbrücken, Germany: Bonn, Germany: Ges. für Informatik. LAP Lambert Academic Publishing. Appelrath, H.-J., & Schulze, L. (Eds.). (2009). Auf ELAN. (2005). ELAN: Bericht zur Förderphase dem Weg zu exzellentem e-learning: Vernetzung ELAN I (1.10.2002 - 31.12.2004). Oldenburg, und Kooperation der Hochschullehre in Nieder- Germany: Bibliotheks- und Informationssystem sachsen. Münster, Germany: Waxmann. der Univ. Ballin, D., & Brater, M. (1996). Handlungsori- Ellis, R. A., & Goodyear, P. (2010). Students’ entiert lernen mit multimedia: Lernarrangements experiences of e-learning in higher education: planen, entwickeln und einsetzen. Nürnberg, Ger- The ecology of sustainable innovation. New York, many: BW Bildung und Wiss. Verl. und Software. NY: Routledge. Baumgartner, P., & Payr, S. (2001). Studieren und Freiburg, U. (n. d.). Servicestelle e-learning. Forschen mit dem Internet. Innsbruck, Austria: Retrieved August 17, 2010, from http://www. Studienverl. rz.uni-freiburg.de/services/elearning Behrens, U. (2001). Teleteaching is easy!?: Fry, H. (2008). A handbook for teaching and learn- Pädagogisch-psychologische Qualitätskriterien ing in higher education: Enhancing academic und Methoden der Qualitätskontrolle für Tele- practice (3rd ed.). New York, NY: Routledge. teaching-Projekte (2. Aufl.). Landau, Germany: Verl. Empirische Pädagogik (Univ., Diss.--Trier, 1999.).
331 Strategies of LMS Implementation at German Universities
Grace, S., & Gravestock, P. (2009). Inclusion and Lee, M. J. W., & McLoughlin, C. (2010). Web diversity: Meeting the needs of all students. New 2.0-based e-learning: Applying social informatics York, NY: Routledge. for tertiary teaching. Hershey, PA: Information Science Reference. doi:10.4018/978-1-60566- Häfele, H., & Maier-Häfele, K. (2005). 101 294-7 e-learning seminarmethoden: Methoden und strategien für die online- und blended learning Pfeffer, T. (Ed.). (2005). Handbuch organisa- seminarpraxis (2. Aufl.). Bonn, Germany: Man- tionsentwicklung: Neue medien in der lehre: agerSeminare Verl.-GmbH. Dimensionen, instrumente, positionen. Münster, Germany: Waxmann. Hamm, I. (1998). Hochschulentwicklung durch neue Medien: Erfahrungen - Projekte - Perspe- Rebensburg, K. (Ed.). (2009). Grundfragen mul- ktiven: mit einer bestandsaufnahme über multime- timedialen lehrens und lernens: Bildungsimpulse dia-projekte an deutschen hochschulen (2. Aufl.). und bildungsnetzwerke. Tagungsband. Berlin, Gütersloh, Germany: Verl. Bertelsmann-Stiftung. Germany: Universitätsverl. der TU. Hochberger, C. (Ed.). (2006). Informatik 2006: Salmon, G. (2004). E-tivities - der schlüssel zu Informatik für menschen: Beiträge der 36. Jahres- aktivem online-lernen. Zürich, Switzerland: Orell tagung der gesellschaft für informatik e.V. (GI):2. Füssli. bis 6. Oktober 2006 in Dresden. Bonn, Germany: Schenkel, P. (2000). Qualitätsbeurteilung multi- Ges. für Informatik. medialer lern- und informationssysteme: Evalu- Holten, R., & Nittel, D. (2010). E-learning in hoch- ationsmethoden auf dem prüfstand. Nürnberg, schule und weiterbildung: Einsatzchancen und Germany: BW Bildung und Wiss. Verl. und erfahrungen. Bielefeld, Germany: Bertelsmann. Software. Issing, L. J., & Klimsa, P. (Eds.). (2009). Online- Schulmeister, R. (1997). Grundlagen hypermedi- lernen: Handbuch für wissenschaft und praxis. aler lernsysteme: [Theorie - Didaktik - Design] München, Germany: Oldenbourg. (2., aktualisierte Aufl.). München, Germany: Oldenbourg. Kerres, M. (2001). Multimediale und telemediale lernumgebungen: Konzeption und entwicklung Schulmeister, R. (2006). E-learning: Einsichten (2., vollst. überarb. Aufl.). München, Germany: und aussichten. München, Germany: Oldenbourg. Oldenbourg. doi:10.1524/9783486593815 doi:10.1524/9783486595062 Kleimann, B., Özkilic, M., & Göcks, M. (n. d.). Schulmeister, R., & Wessner, M. (2001). Virtuelle Studieren im Web 2.0: Studienbezogene Web‐ und universität, virtuelles Lernen. München, Ger- e‐learning‐dienste (HISBUS‐Kurzinforma- many: Oldenbourg. doi:10.1524/9783486598926 tion Nr. 21). Stud.IP e.V. (n. d.). Stud.IP - Die plattform für Krüger, M., & von Holdt, U. (2007). Neue medien lehre, lernen & verwaltung. Retrieved August in vorlesungen, seminaren & projekten an der 30, 2010, from http://www.studip.de leibniz Universität Hannover: Tagungsband zur e-Teaching- und e-Science-Tagung 2007. Aachen, Germany: Shaker.
332 Strategies of LMS Implementation at German Universities
Watkins, R. (2009). E-learning companion: A ENDNOTES student’s guide to online success (3rd ed.). Boston, MA: Cengage Wadsworth. 1 “NewMediaNet” is now called “Servicestelle E-Learning” (Universität Freiburg, 2010). Yang, H. Hao, & Yuen, S. Chi-Yin. (2010). 2 Stud.IP is an open source LMS that has been Handbook of research on practices and outcomes developed by a group of students in Germany. in e-learning: Issues and trends. Hershey, Pa.: Stud.IP is a short form for “Studienbeglei- Information Science Reference. tender Internetsupport von Präsenzlehre” Zwingenberger, A. (2009). Wirksamkeit multime- (students accompanying internet support in dialer lernmaterialien: Kritische bestandsauf- face-to-face teaching and learning), http:// nahme und metaanalyse empirischer evaluations- www.studip.de (Stud.IP e.V., 2010),. Since studien. Münster, Germany: Waxmann (Techn. its initial start in 2000, it has been established Hochsch., Diss.--Aachen, 2008.). at twenty-something universities in Germany during the past ten years. (Appelrath et al., 2006) 3 ELAN stands for e-learning academic network in Lower Saxony (ELAN, 2010).
333 Strategies of LMS Implementation at German Universities
EXPLANATION OF KEY TERMS
In this article, we use the term “university” as a comprehensive term for all kinds of universities that exist in Germany, ranging from general over technical to applied sciences and teaching schools. In general, they are structured hierarchically:
The head of universities is a group of key administrators (president, vice presidents, provost).
Next to several central service units (library, computer centre, centre for teaching and learning) there are a number of faculties (of engineering, life sciences, economics, humanities, etc.). Each faculty is divided up into single units called departments, institutes or schools. They can be of different sizes and levels and can be summed up as institutions. Concerning the affected people, each institution holds a professorship.
In this article, we regard associate and assistant professors also as professors. Then there is the academic staff comprised of senior lecturers, lecturers and instructors. There is a small number of academic staff members who are not involved in teaching, e.g. researchers from third-party funds projects or scholarship holders. However, for an easier handling in this article, we use the term academic staff as an overall term for the teaching staff including the professors.
Last but not least, there is the administrational staff working in non-academic positions. Considering the term technology enhanced learning (TEL), we use it as the application of information, communication and learning technologies in teaching and learning. It covers the whole range of formats from the addition to traditional teaching over hybrid teaching and learning up to virtual learning environments.
334 335
Compilation of References
ABET. (2009). Criteria for accrediting engineering pro- Alexander, S., & Hedberg, J. (1994). Evaluating tech- grams. Accreditation Board for Engineering and Technol- nology-based learning: Which model? In Beattie, K., ogy, 29. Retrieved November 22, 2010, from http://www. McNaught, C., & Wills, S. (Eds.), Multimedia in higher abet.org/Linked%20Documents-UPDATE/Criteria%20 education: Designing for change in teaching and learn- and%20PP/E001%2010-11%20EAC%20Criteria%20 ing (pp. 233–244). Amsterdam, Netherlands: Elsevier. 1-27-10.pdf Ali, A. J. (1990). Management theory in a transitional Abou-Zahra, S. (2008). Web accessibility evaluation. society: The Arab’s experience. International Studies of Human-computer interaction series, 2008, Part II, 79-106. Management and Organization, 20(3), 7–35.
Aczel, J. C., Peake, S. R., & Hardy, P. (2008). Designing Alsop, G., & Tompsett, C. (2002). Grounded theory as an capacity-building in e-learning expertise: Challenges approach to studying students’ uses of learning manage- and Strategies. Computers & Education, 50, 499–510. ment systems. ALT-J Research in Learning Technology, doi:10.1016/j.compedu.2007.07.005 10(2), 63–76. doi:10.1080/0968776020100207
Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining Anastakis, D. J., Cohen, R., & Reznick, R. K. (1991). association rules between sets of items in large databases, The structured oral examination as a method for assess- Proceedings of the 1993 ACM SIGMOD International ing surgical residents. American Journal of Surgery, 162, Conference on Management of Data, Washington, DC. 67–70. doi:10.1016/0002-9610(91)90205-R (pp.207-216). Anderson, T., Rourke, L., Garrison, D. R., & Archer, Albirini, A. (2006). Teachers’ attitudes toward informa- W. (2001). Assessing teaching presence in a computer tion and communication technologies: The case of Syrian conferencing context. Journal of Asynchronous Learning EFL teachers. Computers & Education, 47, 373–398. Networks, 5(2). doi:10.1016/j.compedu.2004.10.013 Andresen, M. A. (2009). Asynchronous discussion forums: Al-Busaidi, K. A. (2009). Evaluating the use of e-learning Success factors, outcomes, assessments, and limitations. system as a supplementary tool at an academic institu- Journal of Educational Technology & Society, 12(1), tion. International Journal of Internet Education, 4, 153. 249–257.
Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Andronico, A., Carbonaro, A., Colazzo, L., Molinari, A., Coming to terms: how researchers in learning and literacy Ronchetti, M., & Trifonova, A. (2004). Designing models talk about knowledge. Review of Educational Research, and services for learning management systems in mobile 61(3), 315–343. settings. In Mobile and Ubiquitious Information Access (Vol. 2954, pp. 524–532). Berlin-Heidelberg, Germany: Springer-Verlag. doi:10.1007/978-3-540-24641-1_8 Compilation of References
ANGEL Learning (2008). The ANGEL learning manage- Bailey, J., & Pearson, S. (1983). Development of a tool ment suite. for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545. doi:10.1287/ Angus, D. S., & Watson, J. (2009). Does regular online mnsc.29.5.530 testing enhance student learning in the numerical sciences? Robust evidence from a large data set. British Ball, D., & Levy, Y. (2008). Emerging educational Journal of Educational Technology, 40(2), 255–272. technology: Assessing the factors that influence in- doi:10.1111/j.1467-8535.2008.00916.x structors’ acceptance in information systems and other classrooms. Journal of Information Systems Education, Ardichvili, A., & Yoon, S. (2009). Designing integra- 19(4), 431–443. tive knowledge management systems: Theoretical considerations and practical applications. Advances Bandura, A. (1977). Self-efficacy: Toward a unifying in Developing Human Resources, 11(3), 307–319. theory of behavioural change. Psychological Review, doi:10.1177/1523422309337593 84(2), 191–215. doi:10.1037/0033-295X.84.2.191
Argyle, M., & Dean, J. (1965). Eye-contact, distance and Banks, S., Goodyear, P., Hogson, V., & McConnell, D. affiliation. Sociometry, 28, 289–304. doi:10.2307/2786027 (2003). Introduction to the special issue on advances in research on networked learning. Instructional Science, Ashford-Rowe, K., & Malfroy, J. (2009). E-learning 31, 1–6. doi:10.1023/A:1022583918064 benchmark report: Learning management system (LMS) usage. Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares (PLS) approach to casual modelling: Avgeriou, P., Papasalouros, A., Retalis, S., & Skordala- Personal computer adoption and use as an illustration kis, M. (2003). Towards a pattern language for learning (with commentaries). Technology Studies, 2(2), 285–309. management systems. Journal of Educational Technology & Society, 6(2), 11–24. Barrella, E., Simmons, J., & Buffinton, K. (2006). Professional engineering education best practice study Babo, R., & Azevedo, A. (2009). Learning manage- for 1st-year, multi-disciplinary courses. ASEE Annual ment systems usage on higher education institutions. Conference and Exposition, Conference Proceedings (10). In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Barret, H. C. (2008). Categories of ePortfolio tools, Analyses & Solutions (pp. 883 – 889). technical report. Retrieved on December 13, 2010, from http://electronicportfolios.com/categories.html. Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, R., & Oliveira, P. (2010a). Gender differences in Internet Baturay, M. (2008). Analysis of a learning management usage habits - A case study in higher education. 14th Inter- system model. E-Journal of New World Sciences Acad- national Business Information Management Association, emy, 3(3), 464–477. Istanbul, Turkey. Baumgartner, P., Häfele, H., & Maier-Häfele, K. (2002). E- Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, learning praxishandbuch: Auswahl von lernplattformen; R., & Oliveira, P. (2010 b). Comparison of Internet usage Marktübersicht - funktionen - fachbegriffe. Innsbruck, habits in two generations of higher education students - Austria: Studienverl. A case study. 2th International Conference on Computer Beatty, B., & Ulasewicz, C. (2006). Faculty perspectives Supported Education, Valencia, Spain. on moving from blackboard to the moodle learning man- Baggaley, J. (2008). Where did didtance educa- agement system. TechTrends, 50(4), 36–45. doi:10.1007/ tion go wrong? Distance Education, 29(1), 39–51. s11528-006-0036-y doi:10.1080/01587910802004837
Bagui, S. (2006). An approach to mining crime patterns. International Journal of Data Warehousing and Mining, 2(1), 50–80. doi:10.4018/jdwm.2006010103
336 Compilation of References
Beatty, B., & Ulasewicz, C. (2006). Online teaching Blackboard. (2009). Engaging learners for engaging and learning transition: Faculty perspectives on moving learning. Retrieved on September 7th, 2009, from from blackboard to the Moodle learning management http://www.blackboard.com/Teaching-Learning/Learn- system. TechTrends, 50(4), 36–45. doi:10.1007/s11528- Platform.aspx. 006-0036-y Blackboard.com. (2005). Blackboard & WebCT An- Beetham, H. (2009). E-portfolios in post-16 learning nounce Agreement to Merge. Retrieved on December in the UK: Developments, issues and opportunities. 13, 2010, from http://investor.blackboard.com/phoenix. Retrieved October 2, 2009, from http://www.jisc.ac.uk/ zhtml?c=177018&p=irol-newsArticle&ID=767025. uploaded_documents/eportfolio_ped.doc Bonk, C. J., & Graham, C. R. (2006). The handbook of Bennett, R. E., Rock, D. A., & Wang, M. (1991). blended learning. San Francisco, CA: Pfeiffer. Equivalence of free-response and multiple-choice Botturi, L., Cantoni, L., & Tardini, S. (2006). Introducing items. Journal of Educational Measurement, 28, 77–92. a Moodle LMS in higher education: The e-courses experi- doi:10.1111/j.1745-3984.1991.tb00345.x ence in Ticino. Retrieved on June 21, 2010, from http:// Bereby-Meyer, Y., Meyer, J., & Budescu, D. V. (2003). www.elearninglab.org/wp-content/uploads/2010/04/ Decision making under internal uncertainty: The case jelks_ecourses_paper.pdf of multiple-choice tests with different scoring rules. Bouhnik, D., & Marcus, T. (2006). Interaction in distance- Acta Psychologica, 112, 207–220. doi:10.1016/S0001- learning courses. Journal of the American Society for 6918(02)00085-9 Information Science and Technology, 57(3), 299–305. Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. (2002). doi:10.1002/asi.20277 Prospect theory analysis of guessing in multiple choice Boulakfouf, S., & Zampunieris, D. (2008). Integrating tests. Journal of Behavioral Decision Making, 15, Web 2.0 features into a learning management system. 313–327. doi:10.1002/bdm.417 Conference on e-Learning 2008. Retrieved December Berlanga, A. J., Sloep, P. B., Brouns, F., Bitter-Rijpkema, 29, 2010, from http://publications.uni.lu/record/1695/ M. E., & Koper, R. (2008). Towards a TENCompetence files/DZ_Pages%20from%20ecel08-cd1.pdf e-portfolio. [iJET]. International Journal of Emerging Bredo, E. (1994). Cognitivism, situated cognition and Technologies in Learning, 3, 24–28. Deweyian pragmatism. Philosophy of Education. Re- Bersin, J., Howard, C., O’Leonard, K., & Mallon, D. trieved June, 2010, Online at http://www.ed.uiuc.edu/ (2009). Learning management systems. Bersin & As- EPS/PES-Yearbook/94_docs/BREDO.HTM] sociates. Bridgeman, B. (1991). Essays and multiple-choice tests as Beyer, H., & Holtzblatt, K. (1998). Contextual design, predictors of college freshman GPA. Research in Higher defining vustomer-centred systems. MA, USA: Morgan Education, 32, 319–332. doi:10.1007/BF00992895 Kaufmann Publishers, Inc. Britain, S., & Liber, O. (1998). A framework for pedagogi- Black, E. W., Dawson, K., & Priem, J. (2008). Data for cal evaluation of virtual learning environments. Retrieved free: Using LMS activity logs to measure community in on December 13, 2010, from http://www.leeds.ac.uk/ online courses. The Internet and Higher Education, 11, educol/documents/00001237.htm. 65–70. doi:10.1016/j.iheduc.2008.03.002 Brkovic, M., Milosevic, D., & Krneta, R. (2006). SCos Blackboard (2010). Blackboardlearn+ ANGEL edi- for adaptive e-learning. 28th International Conference on tion. Retrieved on April 19, 2010, from http://www. Information Technology Interfaces (pp. 29-34). Cavtat/ blackboard.com/Teaching-Learning/Learn-Resources/ Dubrovnik. ANGEL-Edition.aspx
337 Compilation of References
Browne, T., Jenkins, M., & Walker, R. (2006). A lon- Bush, M. (1999). Alternative marking schemes for on- gitudinal perspective regarding the use of VLEs by line multiple choice tests. Paper presented at the 7th An- higher education institutions in the United Kingdom. nual Conference on the Teaching of Computing, Belfast. Interactive Learning Environments, 14(2), 177–192. Retrieved August 14, 2010, from http://www.caacentre. doi:10.1080/10494820600852795 ac.uk/resources/bibliography/biblio2.shtml.
Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Buzzi, C., Buzzi, M., & Leporini, B. (2009). Accessing Design and delivery of integrated learning systems: Their e-learning systems via screen reader: An Example. In impact on student achievement and attitudes. Journal of Jacko, M. J. A. (Ed.), Human-Computer Interaction, Part Educational Computing Research, 21(4), 475–486. IV, HCII 2009, LNCS 5613 (pp. 21–30).
Buchanan, T. (2002). Online assessment: Desirable or Byam, N. (1995). The emergence of community in dangerous? Professional Psychology, Research and computer-mediated communication. In Jones, S. G. (Ed.), Practice, 33, 148–154. doi:10.1037/0735-7028.33.2.148 Cybersociety. Newbury Park, CA: Sage.
Bull, J., Conole, G., Davis, H. C., White, S., & Sclater, N. Carvalho, M. K. F. (2009). Confidence judgments in real (2002). Rethinking assessment through learning technolo- classroom settings: Monitoring performance in different gies. In A. Williamson, C. Gunn, A. Young, & T. Clear types of tests. International Journal of Psychology, 44, (Eds.), Proceedings of the 19th Annual Conference of 93–108. doi:10.1080/00207590701436744 the Australasian Society for Computers in Learning in Catherall, P. (2004). Delivering e-learning information Tertiary Education (ASCILITE2002) (Vol. 1, pp. 8-11). services in higher education. Oxford, UK: Chandos Auckland, New Zealand: UNITEC Institute of Technology. Publishing. Bundesministerium für Bildung und Forschung. (2000). Cavus, N., & Momani, A. M. (2009). Computer aided Exzellenzinitiative. Retrieved on December 20, 2010, evaluation of learning management systems. Procedia So- from http://www.bmbf.de/de/1321.php cial and Behavioral Sciences, 1, 426–430. doi:10.1016/j. Bundesministerium für Bildung und Forschung. (2000). sbspro.2009.01.076 Förderprogramm Neue Medien in der Bildung. Lehr- und Ceraulo, S. (2005). Benefits of upgrading to an LMS. Lernsoftware. Retrieved on December 20, 2010, from Distance Education Report, 9, 6–7. http://www.bmbf.de/pub/neue_medien_bildung.pdf Cerioli, M., & Ribaudo. M. (2009). Great is the enemy Bundesministerium für Bildung und Forschung. (2000). of good: Is perfecting specific courses harmful to global Qualitätspakt Lehre. Retrieved on December 20, 2010, curricula performances? 4th European Conference on from http://www.bmbf.de/de/15375.php Technology Enhanced Learning (EC-TEL 2009) (LNCS Burlea, A. (2008). The complexity of an e-learning system: 5794). Berlin-Heidelberg, Germany: Springer. A paradigm for the human factor, The Inter-Networked Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. World: ISD theory, practice and education (Vol. 2, pp. H. (2008). A virtual learning environment for part- 267–278). New York, NY: Springer-Verlag. time MASW students: An evaluation of the WebCT. Burnett, R., & Marshall, D. P. (2003). Web theory: An Journal of Teaching in Social Work, 28, 87–100. introduction. New York, NY: Routledge Press. doi:10.1080/08841230802179027
Burniske, R. W., & Monke, L. (2001). Breaking down the digital walls. Albany, NY: State University of New York Press.
Bush, M. E. (2006). Quality assurance of multiple-choice tests. Quality Assurance in Education, 14, 398–404. doi:10.1108/09684880610703974
338 Compilation of References
Chapman, B. (2010). LMS KnowledgeBase 2010: In- Colace, F., & De Santo, M. (2003). Evaluating online depth profiles of 90+ learning management systems, with learning platforms: A case study. Proceedings of the 36th custom comparison across 200+ features. Brandon Hall Hawaii International Conference on System Sciences. Research, Retrieved on November 9, 2010, from http:// Retrieved December 29, 2010, from http://www.hicss. www.brandon-hall.com/publications/lmskb/ lmskb.shtml hawaii.edu/HICSS36/HICSSpapers/ETWBE03.pdf Commonwealth of Learning (2004) LMS Evaluation Tool Cole, J., & Foster, H. (2007). Using Moodle - Teaching User Guide, 1.2, 3waynet Inc. and the Commonwealth with the popular open source course management system. of Learning. Sebastopol, CA: O’Reilly Community Press. Chau, Y., Patrick, K., & Hu, J. (2001). Information Tech- Common Cartridge Specification, I. M. S. (2010). Version nology acceptance by individual professionals: A model 1.0 Final Specification. IMS Global Learning Consortium comparison approach. Decision Sciences, 32(4), 699–718. Inc. Retrieved November 12, 2010, from http://www. doi:10.1111/j.1540-5915.2001.tb00978.x imsglobal.org/cc/index.html. Cheung, B., Stewart, B., & McGreal, R. (2006) Going Compeau, D., & Higgins, C. (1995). Computer self- mobile with MOODLE: First steps. IADIS International efficacy: Development of a measure and initial test. Conference on Mobile Learning 2006, Dublin, Ireland. Management Information Systems Quarterly, 19, 189–211. Chin, W. (2001). PLS Graph user’s guide, version 3.0. doi:10.2307/249688 Houston, TX: Bauer College of Business, University of Connolly, T. M., & Stansfield, M. H. (2007b). From e- Houston. learning to online games-based e-learning: Implication Chin, W., & Newsted, P. R. (1999). Structural equation and challenges for higher education and training. In Li, F. modeling analysis with small samples using partial least (Ed.), Social implications and challenges of e-business. squares. In Hoyle, R. H. (Ed.), Statistical strategies for Hershey, PA: Idea-Group Publishing. small sample research (pp. 307–341). Thousand Oaks, Connolly, T. M. & Stansfield, M. H. (2007a). From CA: Sage Publications. e-learning to games-based e-learning. International Chin, W. (1998). The partial least square approach to Journal of Information Technology and Management, structural equation modelling. In Marcoulides, G. A. (Ed.), 26(2/3/4), 188-208. Modern Methods for Business Research (pp. 295–336). Conole, G., de Laat, M., Dillon, T., & Darby, J. (2006). London, UK: Lawrence Erlbaum Associates. JISC LXP: Student experiences of technologies: Final Chu, L. F., Young, C. A., Ngai, L. K., Cun, T., Pearl, R. Report: Joint Information Systems Committee. Retrieved G., & Macario, A. (2010). Learning management systems October 7, 2010, from http://www.jisc.ac.uk/. and lecture capture in the medical academic environ- Coombs, C., Milholland, J., & Womer, F. (1956). The ment. International Anesthesiology Clinics, 48, 27–51. assessment of partial knowledge. Educational and Psy- doi:10.1097/AIA.0b013e3181e5c1d5 chological Measurement, 16, 13–37. City & Guilds. (2009). How are centres using ePortfolios. Corbeil, J. R., & Valdes-Corbeil, M. E. (2007). Are you Retrieved November 12, 2009, from http://www.cityand ready for mobile learning? EDUCAUSE Quarterly, 30(2). guilds.com/44912.html Cradler, J. (2008). Research on e-learning. Learning and Claroline Consortium. (2010). Claroline documentation. Leading with Technology, 3(5), 54–57. Retrieved December 28, 2010, from http://doc.claroline. net/en/index.php/Main_Page. Crano, W., & Brewer, M. (2002). Principles and methods of social research. Mahwah, NJ: Lawrence Erlbaum As- sociates Publishers.
339 Compilation of References
Curyer, S., Leeson, J., Mason, J., & Williams, A. (2007). DeBord, K. A., Aruguete, M. S., & Muhlig, J. (2004). Developing e-portfolios for VET: Policy issues and in- Are computer-assisted teaching methods effective? teroperability. Australian flexible learning framework. Teaching of Psychology, 31, 65–68. doi:10.1207/ Retrieved October 16, 2009, from http://e-standards. s15328023top3101_13 flexiblelearning.net.au/docs/vet-eportfolio-report-v1-0. DeLone, W., & McLean, E. (2003). The Delone and Mclean pdf model of Information Systems success: A 10-year update. Curzon, L. B. (1997). Teaching in further education: Journal of Management Information Systems, 19(4), 9–30. An outline of principles and practice (5th ed.). London, DeLone, W. H., & McLean, E. R. (1992). Information UK: Cassell. Systems success: The quest for the dependent variable. Dabbagh, N. (2005). Pedagogical models for e-learning: Information Systems Research, 3, 60–95. doi:10.1287/ A theory-based design framework. International Journal isre.3.1.60 of Technology in Teaching and Learning, 1(1), 25–44. Dennen, V. P. (2008). Looking for evidence of learn- Dagger, D., O’Connor, A., Lawless, S., Walsh, E., & Wade, ing: Assessment and analysis methods for online dis- V. (2007). Service oriented e-learning platforms: From course. Computers in Human Behavior, 24, 205–219. monolithic systems to flexible services. IEEE Internet doi:10.1016/j.chb.2007.01.010 Computing Special Issue on Distance Learning. DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E- Dalsgaard, C. (2006). Social software: E-learning beyond learning in Organizations. Journal of Management, 31(6), learning management systems. European Journal of Open, 920–940. doi:10.1177/0149206305279815 Distance and E-learning. Deshpande, M., & Karypis, G. (2002). Using conjunction Daniels, P. (2009). Course management systems and im- of attribute values for classification. Proceedings of the 11th plications for practice. International Journal of Emerging International Conference on Information and Knowledge Technologies & Society, 7(2), 97–108. Management, McLean, VA (pp. 356-364).
Davenport, T. H., & Prusak, L. (1998). Working knowledge: Deutsches Zentrum für Luft- und Raumfahrt (DLR). How organizations manage what they know. Boston, MA: (2009). Projektträger im DLR – Neue medien in der Harvard Business Review Press. bildung, Retrieved on December 20, 2010, from http:// www.dlr.de/pt/desktopdefault.aspx/tabid-5882/9540_ Davis, F. D. (1989). Perceived usefulness, perceived read-18559/ ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, Dey, A. K., & Abowd, G. D. (1999). Towards a better 319–339. doi:10.2307/249008 understanding of context and context-awareness. GVU Technical Report GIT-GVU-99-22. College of Comput- Davis, B., Carmean, C., & Wagner, E. D. (2009). The ing, Georgia Institute of Technology, Atlanta, Georgia. evolution of the LMS: From management to learning - Deep analysis of trends shaping the future of e-learning. Dillenbourg, P. (2008). Integrating technologies into Sage Road Solutions. LLC. educational ecosystems. Distance Education, 29(2), 127–140. doi:10.1080/01587910802154939 De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring Dillon, P., Wang, R., & Tearle, P. (2007). Cultural discon- processes in a networked learning community. Instruc- nection in virtual education. Pedagogy, Culture & Society, tional Science, 31, 7–39. doi:10.1023/A:1022596100142 15, 153–174. doi:10.1080/14681360701403565
De Laat, M., Lally, V., Lipponen, L., & Simons, R. J. Dobson, J. L. (2008). The use of formative online quizzes (2007). Online teaching in networked learning communi- to enhance class preparation and scores on summative ties: A multi-method approach to studying the role of the exams. Advances in Physiology Education, 32, 297–302. teacher. Instructional Science, 35, 257–286. doi:10.1007/ doi:10.1152/advan.90162.2008 s11251-006-9007-0
340 Compilation of References
Doll & Torkzadeh, G. (1988). The measurement of end Ellis, R. K. (2009). Field guide to learning management user computing satisfaction, IS Quarterly, 12(2), 259-274. systems, ASTD learning circuits.
Dorninger, C., & Schrack, C. (2007). E-portfolio’s in Ellis, R. K. (2009a). Field guide to learning manage- education-learning tools or means of assessment? In ment systems. Learning circuits, American society for Proceedings of the International Computer-Aided Learn- training and development. Retrieved December 4, 2010, ing Conference (ICL2007), Villach, Austria. from http://www.astd.org/NR/rdonlyres/12ECDB99- 3B91-403E-9B15-7E597444645D/23395/LMS_field- Downes, S. (2005). E-learning 2.0, eLearn, 2005 (10). guide_20091.pdf Drigas, A., Koukianakis, L., & Papagerasimou, Y. (2006). Ellis, R. K. (2009b). Learning management systems 2009. An e-learning environment for non-traditional students Learning circuits and e-learning news, American society with sight disabilities. 36th Annual Frontiers in Education for training and development. Retrieved December 4, 2010, Conference, San Diego, CA (pp. 23-27). from http://www.astd.org/LC/2009/0509_LMS2009.htm. Dror, I. E. (2008). Technology enhanced learning: The Ellis, R. K. (2010). Learning Management Systems 2010. good, the bad, and the ugly. Pragmatics & Cognition, Learning Circuits and E-Learning News, American Society 16(2). Amsterdam, Netherlands: John Benjamins Pub- for Training and Development. Retrieved December 4, lishing Company. 2010, from http://www.astd.org/LC/2010/0910_LMS- Dżega, D., & Pietruszkiewicz, W. (2010). The support survey.htm. of e-learning platform management by the extraction of Erlin, Y. N., & Rhaman, A.A. (2009). Students’ Interac- activity features and clustering based observation of users. tions in Online Asynchronous Discussion Forum: A Social In S. Konstantopoulos, S. Perantonis, V. Karkaletsis, C. Network Analysis. International Conference on Education Spyropoulos, & G. Vouros (Eds.), Artificial Intelligence: Technology and Computer, 25-29. Theories, Model and Applications-SETN 2010 (pp. 315-320). LNAI 6040. Heidelberg, Berlin, Germany: Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Springer-Verlag. Connolly, P., Coulthard, G., & Mong, C. (2007). Using peer feedback to enhance the quality of student online Ebner, M., Kickmeier-Rust, M., & Holzinger, A. (2008). postings: An exploratory sStudy. Journal of Computer- Utilizing Wiki-Systems in higher education classes: A Mediated Communication, 12, 412–433. doi:10.1111/ chance for universal access? Springer Universal Access j.1083-6101.2007.00331.x in Information Society International Journal, 7(4). Eteokleous-Grigoriou, N. (2009). Instilling a new learning, EDUCAUSE. (2005).Retrieved on February 11th, 2011, work and communication culture through systemically from E-Portfolios Web page, URL: http://www.educause. integrated technology in education. Systems Research and edu/ELI/Archives/EPortfolios/5524 Behavioral Science, 26, 707–716. doi:10.1002/sres.983 El Tartoussi, I. (2009). Networked readiness in the United European Commission. (2009). ECTS users’ guide. Lux- Arab Emirates, The 2nd Annual Forum on E-learning embourg: Office for official publications of the European Excellence in the Middle East 2009, Dubai. UAE. communities, p. 60. Retrieved July 19, 2010, from http:// Elan (n. d.). E-Learning Academic Network Nieder- ec.europa.eu/education/lifelong-learning-policy/doc/ects/ sachsen. E-Learning Academic Network Niedersachsen. guide_en.pdf Retrieved on August 17, 2010, from http://www.elan- European Commission. (2010). The bologna process - niedersachsen.de Towards the european higher education area. Retrieved Elias, T. (2010). Universal instructional design principles on August 30, 2010, from http://ec.europa.eu/education/ for Moodle, Canada International Review of Research in higher-education/doc1290_en.htm Open and Distance Learning, 11(2). May – 2010.
341 Compilation of References
Evans, L., Ingersoll, R. W., & Smith, E. J. (1966). The Folajimi, Y. (2009). Transforming distant pedagogical reliability, validity and taxonomic structure of the oral learning to Web based collaborative system: An Intelli- examination. Journal of Medical Education, 41, 651–657. gent Tutoring Systems Architecture. In T. Bastiaens et al. (Eds.), Proceedings of World Conference on E-Learning Falvo, D. A., & Johnson, B. F. (2007). The use of learning in Corporate, Government, Healthcare, and Higher Edu- management systems in the United States. TechTrends, cation 2009 (pp. 1618–1622). Chesapeake, VA: AACE. 51(2), 40–45. doi:10.1007/s11528-007-0025-9 Forment, M., & Guerrero, J. C. (2008, April 11-13). Faßbender, H., & Miosge, C. (2010). Technische Uni- Moodlbile: Extending Moodle to the mobile on/offline versität Braunschweig. Hervorragende Qualität und viel Scenario. Proceedings of the IADIS International Confer- Raum für eigene Ideen. In Winde, M. (Ed.), Von der ence on Mobile Learning, Algarve, Portugal. Qualitätsmessung zum Qualitätsmanagement. Praxis- beispiele an Hochschulen (pp. 48–57). Essen, Germany: Fornell, C., & Larcker, D. (1981). Evaluating structural Edition Stifterverband. equation models with unobservable variables and measurement error. JMR, Journal of Marketing Research, 18, Faux, F., McFarlane, A. E., Roche, N., & Facer, K. (2006). 39–50. doi:10.2307/3151312 Handhelds: Learning with handheld technologies. Bristol, UK: Futurelab. Franklin, T., & Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Joint Feldstein, M. (2004). What’s important in a learning Information Systems Committee (JISC) report. Retrieved content management system. eLearn Magazine. Retrieved August 8, 2010, from http://ie-repository.jisc.ac.uk/148/1/ December 28, 2010, from http://www.elearnmag.org/ web2-content-learning-and-teaching.pdf subpage.cfm?section=tutorials&article=4-1 Freeman, R., & Lewis, R. (1998). Planning and Imple- Feldstein, M., & Masson, P. (2006). Unbolting the chairs: menting Assessment. London, UK: Kogan Page. Making learning management systems more flexible. E- learn magazine. Retrieved from http://www.elearnmag. Froehlich, T. J. (1994). Relevance reconsidered - To- org/subpage.cfm?section=tutorials&article=22-1 on wards an agenda for the 21st century: Introduction to April 19, 2010. special topic issue on relevance research. Journal of the American Society for Information Science Ameri- Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwojka, can Society for Information Science, 45(3), 124–134. C., Barile, M., Nguyen, M. N., & Wolforth, J. (2009). doi:10.1002/(SICI)1097-4571(199404)45:3<124::AID- Disabilities and e-learning problems and solutions: An ASI2>3.0.CO;2-8 exploratory study. Journal of Educational Technology & Society, 12(4), 241–256. Fu, X., & Wang, L. (2005). Data dimensionality reduction with application to improving classification performance Fielding, R. (2000). Architectural styles and the design of and explaining concepts of data sets. International Journal network-based software architectures. Phd dissertation. of Business Intelligence and Data mining, 1(1), 65-87. Fleck, W. (2010, August 24). Evaluationsergebnisse Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). Advanced von studierendeninterviews zu Stud.IP, Sommer 2010. ontology management sytem for personalised e-learning. Unpublished manuscript. Goethe Universität Frankfurt Knowledge-Based Systems, 22, 292–301. doi:10.1016/j. am Main. E-Learning Center am HRZ. Retrieved August knosys.2009.01.006 30, 2010, from http://www.rz.uni-frankfurt.de/internet/ elearning/index.html Ganguli, S., Pedrosa, I., Yam, C.-S., Appignani, B., Siewert, B., & Kressel, H. Y. (2006). Part I: Preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Academic Radiology, 13, 764–769. doi:10.1016/j.acra.2006.02.057
342 Compilation of References
García, E., Romero, C., Ventura, S., & Calders, T. (2007). Gras, M. (2005). Internet usage and digital divide among Drawbacks and solutions of applying Association Rule Spanish university students. Revista Media e Jornalismo mining in learning management systems. Proceedings of - A Europa e os Media:14(8)(1). the International Workshop on Applying Data mining in Grasha, T. (1990). The naturalistic approach to learning e-Learning, Crete, Greece (pp. 13-22). styles. College Teaching, 38(3), 106–113. Garrison, R. (2000). Theoretical challenges for distance Gray, K., Chang, S., & Kennedy, G. (2010). Use of social education in the 21st century: A shift from structural to Web technologies by international and domestic under- transactional issues. International Review of Research in graduate students: Implications for learning and teaching Open and Distance Learning, 1(1), 1–17. in Australian universities. Technology, Pedagogy and Edu- Garrison, R. (1997). Computer conferencing: The post- cation, 19(1), 31–46. doi:10.1080/14759390903579208 industrial age of distance education. Open Learning, Grosseck, G. (2009). To use or not to use Web 2.0 in higher 12(2), 3–11. doi:10.1080/0268051970120202 education? Procedia Social and Behavioral Sciences, 1, Garrison, D. R., & Vaughan, N. D. (2007). Blended 478–482. doi:10.1016/j.sbspro.2009.01.087 learning in higher education: Framework, principles, Gunawardena, C. (1995). Social presence theory and and guidelines. San Francisco, CA: Jossey-Bass. implications for interaction and collaborative learning in Gaspay, A., Dardan, S., & Legorreta, L. (2008). Distance computer conferences. International Journal of Educa- learning through the lens of learning models: New outlets tional Telecommunications, 1(2/3), 147–166. for innovation. Review of Business Research, 8(4), 53–62. Gunawardena, C., & Zittle, F. (1997). Social presence as Georgouli, K., Skalkidis, I., & Gerreiro, P. (2008). A a predictor of satisfaction within a computer mediated framework for adopting LMS to introduce e-learning in conferencing environment. American Journal of Distance a traditional course. Journal of Educational Technology Education, 11(3), 8–26. doi:10.1080/08923649709526970 & Society, 227–240. Gunawardena, C. N. (1993). The Social context of online Gerring, J. (2007). Case study research: Principles and education. In Proceedings of the Distance Education practices. Cambridge, UK: Cambridge University Press. Conference, Portland, Oregon.
Gilbert, T. (2009). Leveraging Sakai and IMS LTI to Gunawardena, C., Carabajal, K., & Lowe, C. A. (2001). standardize integrations. In 10th Sakai Conference Critical analysis of models and methods used to evaluate Pearson Education. online learning networks. Education Resources Informa- tion Center (ERIC). Retrieved December 29, 2010, from Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, http://eric.ed.gov/PDFS/ED456159.pdf K. (1991). A comprehensive study of the effects of an integrated learning system. Indiana State University. GUnet Asynchronous eLearning Group (2010). Open e-class asynchronous e-learning platform - Platform Ginsberg, R., & McCormick, V. (1998). Computer use in description. Greek Academic Network. Retrieved De- effective schools. Journal of Staff Development, 19(1), cember 28, 2010, from http://eclass.gunet.gr/manuals/ 22–25. OpeneClass23_en.pdf. Goffman, E. (1978). Behavior in public places. Notes Guri-Rosenblit. (2005). ‘Distance education’ and ‘e- on the social organization of gatherings (4th ed.). New learning’: Not the same thing. Higher Education, 49, York, NY: Free Press. 467–493. doi:10.1007/s10734-004-0040-0 Graf, S., & Kinshuk (2006). Considering learning styles in Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). learning management systems: Investigating the behavior Multivariate data anlaysis. Upper Saddle River, NJ: of students in an online course. Proceedings of the 1st IEEE Prentice Hall. International Workshop on Semantic Media Adaptation and Personalization, Athens, Greece (pp. 25-30).
343 Compilation of References
Haldane, A. (1998). The convergence of e-learning Holden, C. (2004). What we mean when we say “reposito- and knowledge management: Fusion or confusion? ries”, user expectations of repository systems, Academic People development in a knowledge economy, 2. Learn- ADL Co-Lab. Retrieved on December 13, 2010, from ing Futures Ltd., Retrieved on April 26, 2010 from http://www.hewlett.org/NR/rdonlyres/158FC043-A56F- http://www.learningfutures.co.uk/LFdocs/ MKelle- 43C6-ABA7-EB9A62656FCB/0/RepoSurvey2004-1. her_06-01-03_15-29-27.pdf. pdf.
Hall, J. (2003). Assessing learning management systems. Holmberg, B., & Ortner, G. E. (1991). Research into dis- Chief learning officer. Retrieved on April 19, 2010, from tance education. Frankfurt am Main, Germany: Peter Lang. http://www.clomedia.com/features/2003/January/91/ Holmberg, B. (1995). Theory and practice of distance index.php. education, 2nd edition. London, UK, and New York, NY: Hamuy, E., & Galaz, M. (2010). Information versus com- Routledge. munication in course management system participation. Holmes, B., & Gardner, J. (2006). E-learning concepts Computers & Education, 54, 169–177. doi:10.1016/j. and practice. London, UK: SAGE Publications. compedu.2009.08.001 Holtzblatt, K., & Jones, S. (1994). Contextual inquiry: Harasim, L. (2006). History of e-learning: Shift happened. A participatory design technique for system design. The international handbook of virtual learning environ- In Schuler, D., & Namioka, A. (Eds.), Participatory ment. Berlin-Heidelberg, Germany: Springer-Verlag. design: Principles and practice. Englewood Cliffs, NJ: Harman, L., & Koohang, A. (2007). Learning objects: Prentice-Hall. Standards, metadata, repositories, and LCMS. Informa- Homberg, B. (2005). The Evolution, Principles and Prac- tion Sciense Institute, Informing Sciense. tices of Distance Education (Vol. 11). Hagen, Germany: Hawkins, B. L., & Rudy, J. A. (2007). Educause Core FernUniversität, ZIFF. Data Service. Fiscal Year 2006 Summary Report. CO, Horton, W. (2003). E-learning tools and technologies: A USA: Educause. consumer’s guide for trainers, teachers, educators, and Hayashi, A., Chen, C., Ryan, T., & Wu, J. (2004). The role instructional designers. Indianapolis, IN, USA: Wiley. of social presence and moderating role of computer self Horton, W. (2006). E-learning by design. San Francisco, efficacy in predicting the continuance usage of e-learning CA: John Wiley & Sons, Inc. systems. Journal of Information Systems Education, 15(2), 139–154. Houser, C., & Thornton, P. (2005). Poodle: A course- management system for mobile phones. In Proc. of the Henry, S. L. (2007). Just ask: Integrating accessibility 4th IEEE International Workshop on Wireless and Mobile throughout design. Madison, WI: ET\Lawton. Available Technologies in Education (WMTE ’05). IEEE Computer at www.uiAccess.com/justask/ Society. Hew, K. F., & Cheung, W. S. (2008). Attracting student Hsu, J. (2002). Data mining trends and developments: participation in asynchronous online discussions: A case The key data mining technologies and applications for the study of peer facilitation. Computers & Education, 51, 21st century, Proceedings of the19th Annual Conference 1111–1124. doi:10.1016/j.compedu.2007.11.002 for Information Systems Education (ISECON 2002), San HochschulInformationsSystem. (n. d.). Lehre, Studium, Antonio, TX (Art 224b). Forschung. Retrieved August 30, 2010, from http://www. Huang, S., & Yang, C. (2009). Designing a semantic his.de/abt1/ab10 bliki system to support different types of knowledge and adaptive learning. Computers & Education, 53, 701–712. doi:10.1016/j.compedu.2009.04.011
344 Compilation of References
IEEE LOM. (2002). IEEE Standard for Learning Janossy, J., & Hover, T. (2008). Proposed model for Object Metadata IEEE 1484.12.1-2002. Retrieved on evaluating C/LMS usage. In K. McFerrin et al. (Eds.), November 12, 2010, from http://www.ieeeltsc.org/stan- Proceedings of Society for Information Technology & dards/1484-12-1-2002/. Teacher Education International Conference 2008 (pp. 2979-2986). Chesapeake, VA: AACE. Igbaria, M. (1990). End-user computing effectiveness: A structural equation model. Omega, 18(6), 637. Jaroszewicz, S., & Simovici, D. A. (2004). Interestingness doi:10.1016/0305-0483(90)90055-E of frequent itemsets using Bayesian networks as background knowledge. Proceedings of the 10th ACM SIGKDD Igbaria, M., Guimaraes, T., & Davis, G. B. (1995). Testing International Conference on Knowledge Discovery and the determinants of microcomputer usage via a structural Data mining (pp. 178-186). Seattle, WA. equation model. Journal of Management Information Systems, 11(4), 87–114. JISC. (2008). Effective practice with e-portfolios: Sup- porting 21st century learning. Joint information systems Igel, C. (2009). Learning-management an hochschulen. committee. London, UK: JISC. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 207–215). Jonassen, D. H. (1996). Computer-mediated communi- München, Germany: Oldenbourg. cation: Connecting communities of learners. Computers in the Classroom (158–182). Edgewood Cliffs, NJ: Iglesias, A., Moreno, L., Revuelta, P., & Jiménez, J. (2009). Prentice-Hall, Inc. APEINTA: A Spanish educational project aiming for inclusive education in and out of the classroom. ITiCSE Jones, D. (2004). The conceptualisation of e-learning: ‘09, Paris, France (p. 393). New York, NY: ACM, NY. Lessons an implications. Studies in Learning, Evaluation. Innovation and Development, 1(1), 47–55. ILIAS. (n. d.). ILIAS Information Center. Retrieved August 30, 2010, from http://www.ilias.de JORUM team. (2006). E-learning repository systems research watch. Retrieved on December 13, 2010, from IMS CP. (2007). IMS Content Packaging v1.2 Final http://www.jorum.ac.uk/docs/pdf/Repository_Watch_fi- specification. Retrieved on November 12, 2010, from nal_05012006.pdf. http://www.imsglobal.org/content/packaging/. Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, IMS Global Learning Consortium. (2006a). IMS question T., Richards, G., & Brooks, C. (2007). Using Semantic and test interoperability overview, version 2.1 public draft Web technologies to analyze learning content. Internet (revision 2) specification. Retrieved December 5, 2010, computing, 11(5), 45–53. from http://www.imsglobal.org/question/qtiv2p1pd2/ imsqti_oviewv2p1pd2.html Kakasevski, G., Mihajlov, M., Arsenovski, S., & Chun- gurski, S. (2008). Evaluating usability in learning man- Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). The agement system Moodle. Proceedings of the ITI 2008 role of ICTs in higher education in South Africa: One 30th Int. Conf. On Information Technology Interfaces strategy for addressing teaching and learning challenges. (pp. 613-618). Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa, 10-21 July. Retrieved Kalnins-Cole, T., & Peters, D. (2007). Towards an acces- from http://emerge2006.net. sible learning management system. International Journal of Continuing Engineering Education and Lifelong Learn- Janossy, J. (2008). Proposed model for evaluating C/LMS ing, 17(1), 84–96. doi:10.1504/IJCEELL.2007.013232 faculty usage in higher education institutions. Conference MBAA 2008. Kaltenbaek, J. (2009). Hochschule online - Online lehren und lernen in der Hochschule. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 367–388). München, Germany: Oldenbourg.
345 Compilation of References
Keegan, D. (2005). Mobile learning: The next generation Kettinger, W. J., & Lee, C. C. (1994). Perceived service of learning. Distance Education International. quality and user satisfaction with the Information Ser- vices function. Decision Sciences, 25(5/6), 737–765. Keegan, D. (2002, November). The future of learning: doi:10.1111/j.1540-5915.1994.tb01868.x From eLearning to mLearning. ZIFF Papiere 119. Hagen, Germany: FernUniversitat, Ziff. Kibble, J. (2007). Use of unsupervised online quizzes as formative assessment in a medical physiology course: Kekwaletswe, R. M. (2006). Social presence and context Effects of incentives on student participation and perfor- awareness for knowledge transformation in an m-learning mance. Advances in Physiology Education, 31, 253–260. environment. In Proceedings of Emerge Online Confer- doi:10.1152/advan.00027.2007 ence: Learning Landscapes in Southern Africa (10-21 July). Retrieved from http://emerge2006.net. Kim, S. W., & Lee, M. G. (2008). Validation of an evaluation model for learning management systems. Kekwaletswe, R. M. (2007). Knowledge transformation Journal of Computer Assisted Learning, 24(4), 284–294. in a mobile learning environment: An inquiry of context doi:10.1111/j.1365-2729.2007.00260.x and social presence awareness. PhD thesis. South Africa: University of Cape Town. Kim, K., & Bonk, C. J. (2006). The future of online teaching and learning in higher education: The survey Kekwaletswe, R. M. (2009). Conceptualizing ubiquitous says. EDUCAUSE Quarterly, 29(4). learning through context-aware wired and wireless Web services. In Proceedings of the IADIS Mobile Learning Kirkwood, A., & Price, L. (2005). Learners and learning conference, Barcelona, Spain (25 – 28 February). in the 21st century: What do we know about students’ attitudes and experiences of ICT that will help us design Kekwaletswe, R. M., & Ng’ambi, D. (2006). Ubiquitous courses? Studies in Higher Education, 30(3), 257–274. social presence: Context-awareness in a mobile learning doi:10.1080/03075070500095689 environment. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trust- Kirkwood, A. (2009). E-learning: You don’t always get worthy Computing, Taichung, Taiwan (90-95). Taichung, what you hope for. Technology, Pedagogy and Educa- Taiwan. tion, 18(2), 107–121. doi:10.1080/14759390902992576
Kelly, T., & Bauer, D. (2004). Managing intellectual Kirner, T. G., Custódio, C. A., & Kirner, C. (2008). Us- capital-via e-learning-at Cisco. In Holsapple, C. (Ed.), ability evaluation of the Moodle system from the teachers’ Handbook on Knowledge Management 2: Knowledge perspective. IADIS International Conference e-Learning directions (pp. 511–532). Berlin, Germany: Springer. (pp. 371-378).
Kennedy, G., Dalgarno, B., Sturt, C., Bennett, S., Gray, Kleimann, B., & Wannemacher, K. (2004). E-learning an K., Waycott, J., & Judd, T. …Chang, R. (2009). Educating deutschen hochschulen: Von der projektentwicklung zur the Net generation. A handbook of findings for practice nachhaltigen Implementierung. Hannover, Germany: HIS. and policy. Australia: Australian Learning and Teaching Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, Council. H., & Verkamo, A. I. (1994). Finding interesting rules Kerres, M. (2005). Strategieentwicklung für die nach- from large sets of discovered association rules. Proceed- haltige implementation neuer medien in der hochschule. ings of the 3rd International Conference on Information In T. Pfeffer (Ed.), Medien in der Wissenschaft, Vol. 32. and Knowledge Management (CIKM’94) Gaithersburg, Handbuch Organisationsentwicklung. Neue Medien in Maryland, USA (pp. 401-408). der Lehre: Dimensionen, Instrumente, Positionen (pp. Kneght, M., & Reid, K. (2009). Modularizing Infor- 147–162). Münster, Germany: Waxmann. mation Literacy training via the Blackboard eCom- munity. Journal of Library Administration, 49, 1–9. doi:10.1080/01930820802310502
346 Compilation of References
Kozaris, I. A. (2010). Platforms for e-learning. Ana- Lam, P., Keing, C., McNaught, C., & Cheng, K. F. (2006). lytical and Bioanalytical Chemistry, 397(3), 893–898. Monitoring e-learning environments through analyzing doi:10.1007/s00216-010-3587-x Web logs of institution-wide e-learning platforms. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Who’s Kozma, R. (1987). The implications of cognitive psy- learning? Whose technology? (pp. 429–440). Proceedings chology for computer-based learning tools. Educational of the 23rd Annual Australian Society for Computers in Technology, 27(11), 20–25. Learning in Tertiary Education 2006 conference, Sydney, Kritikou, Y., Demestichas, P., Adamopoulou, E., De- Australia. Retrieved December 29, 2010, from http:// mestichas, K., Theologou, M., & Paradia, M. (2008). www.ascilite.org.au/conferences/sydney06/proceeding/ User profile modeling in the context of Web-based pdf_papers/p62.pdf learning management systems. Journal of Network and Lan, Y. F., & Yan, C. L. (2009). A practical approach to Computer Applications, 31(4), 603–627. doi:10.1016/j. encourage students’ participation in asynchronous online jnca.2007.11.006 discussions based on expectancy theory. International Kubicek, H., Breiter, A., Fischer, A., & Wiedewald, C. Conference on Virtual Environments, Human-Computer (2004). Organisatorische einbettung von e-learning an Interfaces and Measurements Systems (Article number deutschen hochschulen. Retrieved on August 30, 2010, 5068907, pp. 271-276). from http://www.ifib.de/publikationsdateien/MMKH_ Landsberger, J. (2004). Thoughts on convergence in Endbericht_2004-05-26.pdf instructional settings. TechTrends, 48, 6–8. doi:10.1007/ Kubora, K., Terashima, K., Nakahashi, Y., & Morioka, BF02763348 H. (2008). Analyzing learning environments of distance Laudon, K. C., & Laudon, J. P. (1995). Information sys- education in Japan, distance learning and the Internet, tems: A problem-solving approach (3rd ed.). Orlando, Tokyo, Japan. FL: Dryden Press. Kukulska-Hulme, & A., Traxler, J. (2005). Mobile learn- Laurillard, D. (2005). E-learning in higher education. ing – A handbook for educators and trainers. UK: Taylor In Ashwin, P. (Ed.), Changing higher education: The & Francis. development of learning and teaching. London, UK: Kwak, H., Fox, R. & Zinkhan, G. (2002). What products RoutledgeFalmer. can be successfully promoted and sold via the Internet? Leal, J. & Queirós, R. (2010). From e-learning systems Journal of Advertising Research, Jan.-Feb. to specialised e-learning services. Chapter of EduJudge Lahwal, F., Amaimin, M., & Al-Ajlan, A. (2009). Percep- project book - A new learning paradigm: Competition tion cultural impacts: Principles for trainers’ skills for supported by technology. e-learning. NCM 2009 - 5th International Joint Confer- Leal, J. P., & Queirós, R. (2009). CrimsonHex: A service ence on INC, IMS, and IDC (Article number 5331619, oriented repository of specialised learning objects. In pp. 986-993). ICEIS 2009, 11th International Conference on Enterprise Lam, P., McNaught, C., & Cheng, K. F. (2008). Pragmatic Information Systems. meta-analytic studies: learning the lessons from natural- Lee, Y. (2006). An empirical investigation into fac- istic evaluations of multiple cases. ALT-J, 16(2), 61–79. tors influencing the adoption of an e-learning sys- doi:10.1080/09687760802315879 tem. Online Information Review, 30(5), 517–541. doi:10.1108/14684520610706406
347 Compilation of References
Lehsten, P., Zender, R., Lucke, U., & Tavangarian, D. Lopes, C. (2007). Evaluating e-learning readiness in a (2010, March 29-April 7). A service-oriented approach health sciences higher education institution. In M. Bap- towards context-aware mobile learning management tista Nunes & M. McPherson (Eds.), Proceedings of the systems. 8th IEEE International Conference on Pervasive IADIS International Conference e-Learning 2007 (vol. Computing and Communications Workshops (PERCOM 1, pp. 59-67). Workshops) (pp 268-273). Lorenzo, G., & Ittelson, J. (2005). An overview of ePort- Levensaler, L., & Laurano, M. (2009). Talent management folios, Educause learning initiative. systems 2010. Bersin & Associates. Lukhele, R., Thissen, D., & Wainer, H. (1994). On the Levin, D., & Arafeh, S. (2002). The digital disconnect: The relative value of multiple-choice, constructed response, widening gap between Internet savvy students and their and examinee-selected items on two achievement tests. schools. Pew Internet & American Life Project. Retrieved Journal of Educational Measurement, 31, 234–250. December 29, 2010, from http://www.pewinternet.org/~/ doi:10.1111/j.1745-3984.1994.tb00445.x media//Files/Reports/2002/PIP_Schools_Internet_Re- Lytras, M., & Pouloudi, A. (2006). Towards the develop- port.pdf.pdf ment of a novel taxonomy of knowledge management Lewis, B. A., MacEntee, V. M., DeLaCruz, S., Englander, systems from a learning perspective: An integrated ap- C., Jeffrey, T., & Takach, E. …Jason W. (2005). Learning proach to learning and knowledge infrastructures. Journal management systems comparison. Proceedings of the 2005 of Knowledge Management, 10 6), 64-80. Informing Science and IT Education Joint Conference, MacKeogh, K., & Fox, S. (2009). Strategies for embed- Flagstaff, Arizona. ding e-learning in traditional universities: Drivers and Li, W., Han, J., & Pei, J. (2001). CMAR: Accurate and barriers. Electronic Journal of e-Learning, 7(2). efficient classification based on multiple class-Association MacQueen, J. (1967). Some methods for classification rules. Proceedings of the 2001 IEEE International Confer- and analysis of multivariate observations. Proceedings of ence on Data mining (pp. 369-376). the 5th Berkeley Symposium on Mathematical Statistics Liaw, S., Huang, H., & Chen, G. (2007). Surveying and Probability (pp. 281-297). instructor and learner attitudes toward e-learning. Com- Madell, D., & Muncer, S. (2004). Gender differences puters & Education, 49, 1066–1080. doi:10.1016/j. in the use of the Internet by English secondary school compedu.2006.01.001 children. Social Psychology of Education, 7, 229–251. Liu, Y. (2004). Faculty development and CMS. In Pro- doi:10.1023/B:SPOE.0000018552.85903.4d ceedings of Society for Information Technology in Teacher Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Education International Conference, 2004, 2409–2412. Determining factors of the use of e-learning environ- Liu, H., & Yang, M. (2005). QoL guaranteed adaptation ments by university teachers. Computers & Education, and personalization in e-learning systems. IEEE Trans- 51, 142–154. doi:10.1016/j.compedu.2007.04.004 actions on Education, 48(4), 676–687. doi:10.1109/ Malikowski, S. R., Thomson, M. E., & Theis, J. G. (2007). TE.2005.858398 A model for research into course management systems: Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, T. N. Bridging technology and learning theory. Journal of Edu- (2003). A context awareness architecture for facilitat- cational Computing Research, 36(2). doi:10.2190/1002- ing mobile learning. In Proceedings of MLEARN 2003 1T50-27G2-H3V7 (LSDA), London. Manning, S., & Dix, A. (2008). Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives. Computers & Education, 51, 1154–1171. doi:10.1016/j. compedu.2007.10.010
348 Compilation of References
Martin, L., Gutiérrez-Restrepo, E., & Barrera, C. McCoubrie, P., & McKnight, L. (2008). Single best answer Rodríguez-Ascaso1, A., Santos, Olga C. & Boticario, J. MCQs: A new format for the FRCR part 2a exam. Clinical G. (2007). Usability and accessibility evaluations along Radiology, 63, 506–510. doi:10.1016/j.crad.2007.08.021 the e-learning cycle, IWWUA - WISE 2007 Workshops, McDonald, J. (2002). Is “as good as face-to-face” as good LNCS Volume 4832/2007, Nancy, Francia (pp. 453—458). as it gets? Journal of Asynchronous Learning Networks, Martínez, J. Á., & Navarra, P. L. (2007). Content in- 6(2), 10–23. teroperability on e-learning platforms: Standardization, McGee, P., & Diaz, V. (2007). Wikis and podcasts and digital libraries, and knowledge management, Revista da blogs! Oh, my! What is a faculty member supposed to Universidad y Sociedad del. Conocimiento. do? EDUCAUSE Review, 42(5). Mason, R., Pegler, C., & Weller, M. (2004). E-portfolios: McKelvie, G., Dotsika, F., & Patrick, K. (2007). Interactive An assessment tool for online courses. British Journal of business development, capturing business knowledge and Educational Technology, 35(6), 717–727. doi:10.1111/ practice: A case study. The Learning Organization, 14(5), j.1467-8535.2004.00429.x 407–422. doi:10.1108/09696470710762637 Mattheos, N., Stefanovic, N., Apse, P., Attstrom, R., McKenzie, C. R. M., Wixted, J. T., Noelle, D. C., & Buchanan, J., Brown, P., & Walmsley, A. D. (2008). Gyurjyan, G. (2001). Relation between confidence in Potential of Information Technology in dental education. yes-no and forced-choice taks. Journal of Experimental European Journal of Dental Education, 12(Suppl. 1), Psychology. General, 130, 140–155. doi:10.1037/0096- 85–91. doi:10.1111/j.1600-0579.2007.00483.x 3445.130.1.140 Mayadas, F. (2007). Online education: Worldwide and McLean, L. D., Dauphinee, W. D., & Rothman, A. (1988). moving quickly. In Technology- Enabled Education (pp. The oral examination in internal medicine of the Royal 57-60), Proceeding of the 2007 MIT LINC Conference, College of Physicians and Surgeons of Canada: A reli- Dubai, UAE. ability analysis. Annals of the Royal College of Physicians Mayes, J. T., Morrison, D., Bullen, P., Mellar, H., & and Surgeons of Canada, 21, 510–514. Oliver, M. (Eds.). (2009). Transforming higher educa- McNaught, C., Lam, P., & Ho, A. (2009). The digital divide tion through technology-enhanced learning. York, UK: between university students and teachers in Hong Kong. Higher Education Academy. Proceedings ASCILITE 2009, Auckland, New Zealand. Mazza, R., & Milani, C. (2005). Exploring usage analysis McPherson, M. C., & Nunes, J. M. (2008). Critical is- in learning systems: Gaining insights from visualizations. sues for e-learning delivery: What may seem obvious 12th International Conference on Artificial Intelligence is not always put into practice. Journal of Computer in Education (AIED 2005). The Netherlands: Amster- Assisted Learning, 24(5), 433–445. doi:10.1111/j.1365- dam. Retrieved December 29, 2010, from http://linux3. 2729.2008.00281.x dti.supsi.ch/~mazza/Web_area/Pubblicazioni/AIED05/ aied-ws2005.pdf Miller, T., & Hutchens, S. (2009). 21st century teaching technology: Best practicess and effectiveness in teaching Mazzolini, M., & Maddison, S. (2003). Sage, guide or psychology. International Journal of Instructional Media, ghost? The effect of instructor intervention on student 36 3), 255-262. participation in online discussion forums. Comput- ers & Education, 40, 237–253. doi:10.1016/S0360- Minhsein, H. (2005). Integrating Internet into traditional 1315(02)00129-X education: A practical study. McBride, R., & Schostak, J. (1994). What is qualitative research. Retrieved May 7, 2009, from http://www.enquireylearning.net/ELU/Issues/Research/Res1Ch1.html
349 Compilation of References
Ministerio de Educación Cultura y Deporte. (2003). REAL Morgan, G. (2003). Faculty use of course learning systems. DECRETO 1125/2003, de 5 de septiembre, por el que Educause center for applied research reports. Retrieved se establece el sistema europeo de créditos y el sistema December 29, 2010, from http://net.educause.edu/ir/ de calificaciones en las titulaciones universitarias de library/pdf/ecar_so/ers/ers0302/ekf0302.pdf carácter oficial y validez en todo el territorio nacional. Murray, G. R., Riley, C., & Scime, A. (2007, May). A new Boletín Oficial del Estado. Nº 224, 34355-34356. Re- age solution for an age-old problem: Mining data for likely trieved November 6, 2010, from http://www.boe.es/boe/ voters. Paper presented at the 62nd Annual Conference dias/2003/09/18/pdfs/A34355-34356.pdf of the American Association of Public Opinion Research, Minović, M., Štavljanin, V., Milovanović, M., & Starčević, Anaheim, CA. D. (2008). Usability issues of e-learning systems: Case- Nachmias, R., Mioduser, D., & Shemla, A. (2001). Informa- study for Moodle learning management system. In tion and Communication Technologies usage by students Meersman, R., Tari, Z., & Herrero, P. (Eds.), OTM 2008 in an Israeli high school: Equity, gender, and inside/outside Workshops, Lecture Notes in Computer Science 5333 (pp. school learning issues. Education and Information Tech- 561–570). Heidelberg, Germany: Springer. nologies, 6(1), 43–53. doi:10.1023/A:1011367212148 Misko, J., Choi, J., & Lee, I. (2004). E-learning in Aus- Naidu, S. (2006). E-learning a guidebook of principles, tralia and Korea: Learning from practice. Korea Research procedures and practices, 2nd revised edition, Common- Institute for Vocational Education & Training. Retrieved wealth Educational Media Center for Asia (CEMCA), on June 19, 2010, from http://www.ncver.edu.au/research/ New Delhi, India. core/cp0306.pdf Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. Monsakul, J. (2007). Learning management systems in (2005). Literature review in mobile technologies and higher education: A review from faculty perspective, 4th learning. NESTA Futurelab Series. International Conference on E-learning for Knowledge- Based Society, Bangkok, Thailand. National Centre for Educational Statistics. (2003). Distance education at degree-granting postsecondary Moodle. (2010). Current Moodle statistics, Moodle.org. institutions: 2000-2001 U. Washington, DC: S. Depart- Retrieved on May 27, 2010, from http://moodle.org/stats. ment of Education. Moodle. (2010). Moodle Statistics. Retrieved August 25, Nevgi, A., Virtanen, P., & Niemi, H. (2006). Support- 2010, from http://moodle.org/stats/ ing students to develop collaborative learning skills in Moore, M. G. (1993). Theory of transactional distance. technology-based environments. British Journal of Edu- In Keegan, D. (Ed.), Theoretical principles of distance cational Technology, 37(6), 937–947. doi:10.1111/j.1467- education (pp. 33–38). New York, NY: Routledge. 8535.2006.00671.x
Moreno, L., Iglesias, A. M., Castro, E., & Martinez, P. Newell, A. F., & Gregor, P. (2000). User sensitive inclusive (2008). Using accessible digital resources for teaching design: In search of a new paradigm. CUU 2000 First database design: towards an inclusive distance learning ACM Conference on Universal Usability, 2000. proposal. ITICSE 2008, Madrid, Spain (pp. 32–36). New Ng’ambi, D. (2004). Towards a knowledge sharing York, NY: ACM. framework based on student questions: The case for a Moreno, L., Martínez, P., & Ruiz-Mezcua, B. (2009). A dynamic FAQ environment. PhD thesis. South Africa: bridge to Web accessibility from the usability heuristics, University of Cape Town. 5th Annual Usability Symposium USAB 2009. Usability Ngwenyama, O. K., & Lee, A. (1997). Communication & HCI eInclusion, LNCS 5889 (pp. 290-300). Berlin, richness in electronic mail: Critical social theory and Germany: Springer. the contextuality of meaning. Management Information Systems Quarterly, 21(2), 145–167. doi:10.2307/249417
350 Compilation of References
Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating does this affect the learner? The instructional designer? learning management systems: Adoption of hexagonal Retrieved on December 13, 2010, from http://www.el- e-learning assessment model in higher education. Trans- earningpost.com /articles/archives/ lcms_LMS_cms_rlos. forming Government: People, Process and Policy, 3(2), 11–130. Nichols, M. (2003). A theory for elearning. Journal of Educational Technology & Society, 6(2), 1–10. Padmanabhan, B., & Tuzhilin, A. (2000). Small is beautiful: Discovering the minimal set of unexpected patterns. Nonaka, I. (1994). A dynamic theory of organizational Proceedings of the 6th ACM SIGKDD International knowledge creation. Organization Science, 5(1), 14–37. Conference on Knowledge Discovery and Data mining, doi:10.1287/orsc.5.1.14 Boston, MA (pp. 54-63). NYU. (2010). New York University. Retrieved from http:// Pan, G., & Bonk, C. (2007). The emergence of open- www.nyu.edu/ source software in North America. International Review Oliveira, P., Oliveira, C., Souza, F., & Costa, N. (2006). of Research in Open and Distance Learning, 8(3), 1–17. Teaching stategies to promote active learning in higher Pantel, C. (1997). A framework for comparing Web- education: Current develepments in technology-assisted based learning environments. Master’s thesis, School of education, FORMATEX 2006, (pp. 636-640). Retrieved Computing Science, Simon Fraser University, Canada. on September7, 2010, from http://www.formatex.org/ micte2006/pdf/636-640.pdf Parasuraman, A., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for measuring cus- O’Neill, K., Singh, G., & O’ Donoghue, J. (2004). Imple- tomer perceptions of service quality. Journal of Retailing, menting e-learning programmes for higher education: A 64(1), 12–40. review of the literature. Journal of Information Technol- ogy Education, 3. Park, H., Baek, Y. K., & Gibson, D. (2008). Design of an adaptive mobile learning management system. User O’Neill, M. E. (2005). Automated use of a Wiki for col- interface design and evaluation for mobile technology, laborative lecture notes. SIGCSE Bull, 37(1). New York, I, 286-301. NY: ACM. Patriotta, G. (2003). Organizational knowledge in the mak- Ong, S. S., & Hawryszkiewycz, I. (2003). Towards ing: How firms create, use, and institutionalize knowledge. personalised and collaborative learning management sys- Oxford, UK: Oxford University Press. tems. Third IEEE International Conference on Advanced Learning Technologies (ICALT’03). Retrieved December Paulsen, M. (2002). Online education sytems in Scandi- 29, 2010, from http://www.computer.org/portal/web/csdl/ navian and Australian universities_A comparative study. doi/10.1109/ICALT.2003.1215113 International Review of Research in Open and Distance Learning, 3(2), 1–14. O’Reilly, T. (2005). What is Web 2.0: Design patterns and business models for the next generation of software. Paulsen, M. F. (2003). Online education and learning Retrieved from http://www.oreillynet.com/pub/a/oreilly/ management systems. Bekkestua, Norway: NKI. tim/news/2005/09/30/what-is-web-20.html (Dec. 2010). Paulsen, M. F. (2003). Experiences with learning man- Osei-Bryson, K.-M. (2004). Evaluation of decision agement systems in 113 European institutions. Journal trees: A multicriteria approach. Computers & Opera- of Educational Technology & Society, 6(4), 134–148. tions Research, 31(11), 1933–1945. doi:10.1016/S0305- Paulsen, M. F. (2002). Online education systems: Discus- 0548(03)00156-4 sion and definition of terms. Retrieved December 29, 2010, from http://www.porto.ucp.pt/open/curso/modulos/ doc/Definition%20of%20Terms.pdf
351 Compilation of References
Perry, B. (2009, June). Customized content at your fin- Polanyi, M. (1958). Personal Knowledge. Chicago, IL: gertips. Training & Development, 29–31. The University of Chicago press.
Pettit, J., & Kukulska-Hulme, A. (2007). Going with the Power, C., Petrie, H., Sakharov, V., & Swallow, D. (2010). grain: Mobile devices in practice. Australasia Journal of K. virtual learning environments: Another barrier to Educational Technology, 23(1), 17–33. blended and e-learning. Miesenberger et al. (Eds.), ICCHP 2010, Part I, LNCS 6179 (pp. 519–526). Phan Tan, T.-T., & Krüger, M. (2009). Hochschulüberg- reifende weiterbildung zum e-learning in niedersachsen. Prensky, M. (2007). Changing paradigms – from “be- In Apostolopoulos, N. (Ed.), Grundfragen multimedialen ing taught” to “learning on your own with guidance”. lehrens und lernens. Bildungsimpulse und bildungs- Educational Technology. July-August. netzwerke: Tagungsband GML 2 2009, 12. - 13. März Prensky, M. (2001). Digital natives, digital immigrants. 2009 (pp. 69–80). Berlin, Germany: Univ.-Verl. der TU On the Horizon - The Strategic Planning Resource for Univ.-Bibliothek. Education Professionals, 9(5). Phan Tan, T.-T., & Prey, G. (2010). Programm zur di- Pressley, M., Ghatala, E. S., Woloshyn, V., & Pirie, J. daktischen e-kompetenzentwicklung in Niedersachsen. (1990). Sometimes adults miss the main ideas and do Zeitschrift für hochschulentwicklung, 5(1). Retrieved not realize it: Confidence in responses to short answer from http://www.zfhe.at and multiple-choice comprehension questions. Reading Picciano, A. G. (2002). Beyond student perceptions: Is- Research Quarterly, 25, 232–249. doi:10.2307/748004 sues of interaction, presence and performance in an online Puddu, F., Repetto, M., & Vallarino, E. (2008). The course. Journal of Asynchronous Learning Networks, 6(1). WEL initiative: A model of Web enhanced learning for Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based faculty members. Journal of E-learning and Knowledge virtual learning environments: A research framework and Society, 4(3). a preliminary assessment of effectiveness in basic IT skill Queirós, R., & Leal, J. P. (2010). E-learning frameworks: training. Management Information Systems Quarterly, A survey. In INTED2010 Proceedings (pp. 1345-1354). 25(4), 401–426. doi:10.2307/3250989 Quinlan, J. R. (1993). C4.5: Programs for machine learn- Pietruszkiewicz, W., & Dżega, D. (2009). The practical ing. San Francisco, CA: Morgan Kaufmann. aspects of rich Internet application development and quality factors: RIA–based decision support system. Quality Raaij, E., & Schepers, J. (2008). The acceptance and use of assessment in Web 2009 (Vol. 561). CEUR. a virtual learning environment in China. Computers & Edu- cation, 50, 838–852. doi:10.1016/j.compedu.2006.09.001 Pietruszkiewicz, W., & Dżega, D. (2010). An application of data mining in the management of e-learning platform. Rabin, J., & McCathieNevile, C. (Eds.). (2006). Mobile In Korczak, J. (Ed.), Business informatics. Data mining Web best practices 1.0. (W3C Proposed Recommenda- and business intelligence (pp. 60–70). Wrocław, PA: tion). Retrieved on August 25, 2010, from http://www. Publishing House of Wrocław University of Economics. w3.org/TR/mobile-bp/
Pituch, K. A., & Lee, Y. K. (2006). The influence of system Race, P. (2005). Making Learning Happen. London, characteristics on eLearning use. Computers & Education, UK: Sage. 47(2), 222–244. doi:10.1016/j.compedu.2004.10.007 Rainer, R. K., Turban, E., & Potter, R. E. (2007). Introduc- Pituch, K., & Lee, Y. (2006). The influence of system tion to Information Systems: Supporting and transforming characteristics on e-learning use. Computers & Educa- business. NJ: Wiley. tion, 47, 222–244. doi:10.1016/j.compedu.2004.10.007
Polanyi, M. (1966). The tacit dimension. London, UK: Routledge & Kegan Paul.
352 Compilation of References
Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Roca, J., Chiu, C., & Martinex, F. (2006). Understand- Murray, G. R. (2009). Finding “persistent Rules”: Combin- ing e-learning continuous intention: An extension of the ing association and classification results’, Expert Systems technology acceptance model. International Journal of With Applications, 36(3P2), 6019-6024. Human-Computer Studies, 64, 683–696. doi:10.1016/j. ijhcs.2006.01.003 Rakotonirainy, A., Loke, S. W., & Fitzpatrick, G. (2000). Context awareness for the mobile environment. Retrieved Rodrigues, A., Pinto, M., & Queirós, R. (2010). Grau Oct 11, 2009 from ftp://ftp.cc.gatech.edu/pub/gvu/ de Preparação e Utilização do E-learning no Ensino tr/2000/00-18r.ps.Z. Superior. Estudo de Caso na ESEIG-IPPorto. Setúbal, Portugal: Conferência Investigação e Intervenção em Regueras, L. M., Verdú, E., Castro, J. P., Pérez, M. A., Recursos Humanos. & Verdú, M. J. (2008). Design of a distributed and asynchronous system for remote evaluation of students’ Rogers, L., & Newton, L. (2001). Integrated Learning submissions in competitive e-learning. In International Systems -- An ‘open’ approach. International Journal Conference on Engineering Education (ICEE ‘08). of Science Education, 23(4), 405–422.
Rehak, D., & Mason, R. (2003). Engaging with the Rogers, E. M. (2003). Diffusion of innovations (5th ed.). learning object economy. In Littlejohn, A. (Ed.), Reusing New York, NY: Free Press. online resources: A sustainable approach to e-learning Romero, C., Ventura, S., & Garcia, E. (2008). Data min- (pp. 22–30). London, UK: Kogan Page. ing in course management systems: Moodle case study Rehman, H. U., Said, R. A., & Al-assaf, Y. (2009). An and tutorial. Computers & Education, 51, 368–384. integrated approach for strategic development of engi- doi:10.1016/j.compedu.2007.05.016 neering curricula: Focus on students’ design skills. IEEE Romero, C., Ventura, S., & García, E. (2008). Data min- Transactions on Education, 52(4), 470–481. doi:10.1109/ ing in course management systems: Moodle case study TE.2008.930508 and tutorial. Computers & Education, 51(1), 368–384. Reiser, R. A., & Dempsey, J. V. (Eds.). (2002). Trends doi:10.1016/j.compedu.2007.05.016 and issues in instructional design and technology. Upper Roqueta, M. (2008). Learning management systems: A Saddle River, NJ: Pearson Education. focus on the learner. Distance Learning, 5(4), 59–66. Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, Rose, E. (2004). “Is there a class with this content?” J. (2008). Evaluating learning support systems usability WebCT and the limits of individualization. The Journal an empiracle approach. Communication & Cognition, of Educational Thought, 38(1), 43–65. 41(1 & 2), 143–158. Rossi, P. G. (2009). Learning environment with elements Ribaudo, M., & Rui, M. (2009). AulaWeb, Web-based of artificial intelligence. Journal of E-learning and Knowl- learning as a commodity - The experience of the University edge Society - English Version, 5(1), 193-199. of Genova. 1st International Conference on Computer Supported Education (CSEDU 2009). INSTICC Press. Rößling, G., Joy, M., Moreno, A., Radenski, A., Malmi, L., Kerren, A., & Iturbide, J. Á. (2008). Enhancing learning Richardson, J. C., & Swan, K. (2003). Examining social management systems to better support computer science presence in online courses in relation to students’ per- education (pp. 142-166). Paper presented at the SIGCSE ceived learning and satisfaction. Journal of Asynchronous 2008, 4, Nov. 2008. Learning Networks, 7(1), 68–88. Rourke, L., Anderson, T., Garrison, D. R., & Archer, Richardson, H. C., & Ward, R. (2005). Developing and W. (2001). Assessing social presence in asynchronous implementing a methodology for reviewing e-portfolio text-based computer conferencing. Journal of Distance products, The centre for recording achievement (CRA). Education, 14(2). Retrieved October 10, 2009, from http://www.jisc.ac.uk/ uploaded_documents/epfr.doc
353 Compilation of References
Rovai, A. P. (2008). Facilitating online discussions ef- Savander-Ranne, C., Lundén, O. P., & Kolari, S. (2008). fectively. The Internet and Higher Education, 10, 77–88. An alternative teaching method for electrical engineering doi:10.1016/j.iheduc.2006.10.001 courses. IEEE Transactions on Education, 51(4), 423–431. doi:10.1109/TE.2007.912500 Ryder, M., & Wilson, B. (1997). From center to periphery: Shifting agency in a complex technical learning environ- Scharf, E. M., & Baldwin, L. P. (2007). Assessing multiple ments. Paper presented at the Meeting of the American choice question (MCQ) tests - A mathematical perspec- Educational Research Association, Chicago, IL. tive. Active Learning in Higher Education, 8, 31–47. doi:10.1177/1469787407074009 Saady, A. (2005). E-learning curve, ITP Technology, Retrieved from www.ITP.net. Schein, E. (1985). Organizational culture and leadership. San Francisco, CA: Jossey-Bass. Saba, F. (2000). Research in distance education: A status report. International Review of Research in Open and Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). There Distance Learning, 1(1), 1–9. is more to context than location. Computers & Graph- ics, 23, 893–901. doi:10.1016/S0097-8493(99)00120-X Sabry, K., & Barker, J. (2009). Dynamic Interac- tive Learning Systems. Innovations in Educa- Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Testing tion and Teaching International, 46(2), 185–197. terrorism theory with data mining. International Journal of doi:10.1080/14703290902843836 Data Analysis Techniques and Strategies, 2(2), 122–139. doi:10.1504/IJDATS.2010.032453 Salas-Morera, L., Berral- Yerón, J., Serrano-Gómez, I., & Martínez-Jiménez, P. (2009). An assessment of the ECTS Scime, A. (2008). Web page extension of data warehouses. in software engineering: A teaching experience. IEEE In Wang, J. (Ed.), Encyclopedia of Data Warehousing Transactions on Education, 52(1), 177–184. doi:10.1109/ and Mining (2nd ed., pp. 1211–1215). Hershey, PA: Idea TE.2008.921801 Group Reference.
Samarawickrema, G., & Stacey, E. (2007). Adopt- Scime, A. & Murray, G. R. (2007). Vote prediction by ing Web-based learning and teaching: A case study iterative domain knowledge and attribute elimination. in higher education. Distance Education, 28(3). International Journal of Business Intelligence and Data doi:10.1080/01587910701611344 mining, 2(2), 160-176.
Sampson, D., Papapaulou, P., Zervas, P., & Papanikou, Sclater, N. (2008). Large-scale open source e-learning C. (2008). Supporting higher education teaching through systems at open university UK. Research Bulletin EDU- wireless and mobile devices: The case study of mobile CAUSE Center for Applied Research, 2008(12), 1-13. Moodle. In Proc. of the AACE E-Learn 2008-World SCORM. (2004). 4th Ed. Specification. Retrieved on Conference on E-Learning in Corporate, Government, November 12, 2010, from http://www.adlnet.gov/Pages/ Healthcare, and Higher Education. Default.aspx. Santos, O. C., Boticario, J. G., Del Viso, A. F., De La Scribner, S. (1987). Thinking in action: Some character- Cámara, S. P., Sánchez, C. R., Gutiérrez & Restrepo, E. istics of practical thought. In Sternberg, R., & Wagner, (2007). Basic skills training to disabled and adult learners R. (Eds.), Practical intelligence: Nature and origins of through an accessible e-learning platform. Lecture Notes competence in everyday world. Cambridge, UK: Cam- in Computer Science (including subseries Lecture Notes bridge University Press. in Artificial Intelligence and Lecture Notes in Bioinfor- matics) (LNCS 4556, part 3, pp. 796-805). Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean model of IS success. Infor- Saunders, M., Lewis, P., & Thornhill, A. (2007). Re- mation Systems Research, 8(3), 240–253. doi:10.1287/ search methods for business students (4th ed.). Pearson isre.8.3.240 Education Ltd.
354 Compilation of References
Sen, A., Dacin, P. A., & Pattichis, C. (2006). Current So, H. J. (2009). When groups decide to use asynchro- trends in Web data analysis. Communications of the ACM, nous online discussions: Collaborative learning and 49(11), 85–91. doi:10.1145/1167838.1167842 social presence under a voluntary participation structure. Journal of Computer Assisted Learning, 25, 143–160. Seufert, S. (2008). Innovationsorientiertes bildungsman- doi:10.1111/j.1365-2729.2008.00293.x agement: Hochschulentwicklung durch sicherung der nachhaltigkeit von e-learning. Wiesbaden, Germany: Stergiopoulos, C., Tsiakas, P., Kaitsa, M., Triantis, D., VS Verlag für Sozialwissenschaften / GWV Fachverlage Fragoulis, I., & Ninos, C. (2006b). Methods of elec- GmbH Wiesbaden. tronic examination applied to students of electronics. Comparison of results with the conventional (paper-and- SHAME. (2006). SHAME - Standardized Hyper Adapt- pencil) method. In J. A. Moinhos Cordeiro, V. Pedrosa, able Metadata Editor. Retrieved on December 13, 2010, B. Encarnação, & J. Filipe (Eds.), Proceedings of the 2nd from http://kmr.nada.kth.se/shame. International Conference on Web Information Systems Shariq, Z. S. (1999). How does knowledge trans- and Technologies: Internet Technology / Web Interface form as it is transferred? Speculations on the pos- and Applications(WEBIST 2006) (pp. 305-311). Setubal, sibility of a cognitive theory of knowledgescapes. Portugal: INSTICC Press Journal of Knowledge Management, 3(4), 243–251. Stergiopoulos, C., Tsiakas, P., Triantis, D., & Kaitsa, M. doi:10.1108/13673279910303998 (2006a). Evaluating electronic examination methods of Sharples, M. (Ed.). (2006). Big issues in mobile learning. students of electronics. Effectiveness and comparison to Report of a workshop by the Kaleidoscope Network of the paper-and-pencil Method. In Proceedinsg of the IEEE Excellence Mobile Learning Initiative. Nottingham, UK: International Conference on Sensor Networks, Ubiquitous, University of Nottingham. and Trustworthy Computing (SUTC 2006) (Vol. 2, pp. 143-151). Los Alamitos, CA: IEEE Computer Society. Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). A follow- up investigation of “teaching presence” in the SUNY Stifterverband für die Deutsche Wissenschaft. (n. d.). Learning Network. Journal of Asynchronous Learning Exzellenz in der Lehre. Aktuelle Programme und Initia- Networks, 7(2), 61–80. tiven. Rertieved December 20, 2010, from http://www. stifterverband.de/ Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, Canada: Stifterverband für die Deutsche Wissenschaft. (2010). Wiley. Lehre neu denken. Die Zukunft des akademischen Lehren und Lernens. Retrieved December 20, 2010, from http:// Simões, L. & Gouveia, L. (2009). Geração Net, Web 2.0 e www.stifterverband.info/publikationen_und_podcasts/ Ensino Superior. Cadernos de Estudos Mediáticos, 21-32. positionen_dokumentationen/lehre_neu_denken/lehre_ Simonson, M., Maurer, R., Montag-Torardi, M., & Whita- neu_denken.pdf ker, M. (1987). Development of a standardized test of Strivens, J. (2007). A survey of e-pdp and ePortfolio computer literacy and a computer anxiety index. Journal practice in UK higher education. UK: Higher Education of Educational Computing Research, 3(2), 231–247. Academy. doi:10.2190/7CHY-5CM0-4D00-6JCG Sturgess, P., & Nouwens, F. (2004). “Evaluation of online Sink, C., Sink, M., Stob, J., & Taniguchi, K. (2008). Fur- learning management systems. Turkish Online Journal ther evidence of gender differences in high school–Level of Distance Education, 5(3). computer literacy. Chance, 21(1), 49–53. doi:10.1007/ s00144-008-0011-9 Sumner, M., & Hostetler, D. (1999). Factors influencing the adoption of technology in teaching. Journal of Smythe, C. (2003). IMS abstract framework - A review. Computer Information Systems, 40(1), 81. IMS Global Learning Consortium, Inc.
355 Compilation of References
Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. (2008). Thomas, K., Lam, P., & Ho, A. (2009). Knowledge dif- What drives a successful e-learning? An empirical fusion in eLearning: Learner attributes and capabilities investigation of the critical factors influencing learner in an organization. In G. Siemens & C. Fulford, (Eds.), satisfaction. Computers & Education, 50, 1183–1202. ED-MEDIA 2009 (pp.493–497). Proceedings of the 21st doi:10.1016/j.compedu.2006.11.007 Annual World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Ha- Sutherland, R., Facer, K., Furlong, R., & Furlong, J. waii. (pp. 22–26). Chesapeake, VA: Association for the (2000). A new environment for education? The computer Advancement of Computers in Education. in the home. Computers & Education, 34(3-4), 195–212. doi:10.1016/S0360-1315(99)00045-7 Thomas, P. G. (2003). Evaluation of electronic marking of examinations. In D. Finkel, (Ed.), Proceedings of the Swan, K. (2002). Building communities in online 8th ACM Annual Conference on Innovation and Technol- courses: The importance of interaction. Educa- ogy in Computer Science Education (ItiCSE 2003) (pp. tion Communication and Information, 2(1), 23–49. 50-54). New York, NY: ACM. doi:10.1080/1463631022000005016 Thompson, R., & Compeau, R., Deborah, & Higgins, C. Swan, K. (2003). Developing social presence in online (2006). Intentions to use Information Technologies: An discussions. In Naidu, S. (Ed.), Learning and teaching integrative model. Journal of Organizational and End User with technology: Principles and practices (pp. 147–164). Computing, 18(3), 25–47. doi:10.4018/joeuc.2006070102 London, UK: Kogan Page. Trentin, G. (2001). Designing Online Courses. The Web Swedish National Agency for Higher Education. (2008). in Higher Education: Assessing the Impact and Fulfilling E-learning quality: Aspects and criteria for evaluation the Potential. The Haworth Press. of e-learning in higher education. Report 2008:11 R, PA: Swedish National Agency for Higher Education, Triantis, D., Anastasiadis, C., Tsiakas, P., & Stergiopoulos, Stockholm. Retrieved on July 30, 2010, from http:// C. (2007). Asyncronous teaching methods and electronic www.hsv.se/download/18.8f0e4c9119e2b4a60c800028 assessment applied to students of the module “Physics of 057/0811R.pdf semiconductor devices”. Journal of Materials Education, 29, 133–140. Tan, P.-N., Steinbach, M., & Kumar, V. (2006). Introduc- tion to data mining. Boston, MA: Addison Wesley. Triantis, D., Stavrakas, I., Tsiakas, P., Stergiopoulos, C., & Ninos, D. (2004). A pilot application of electronic ex- Tapscott, D. (1999). Growing up digital: the rise of the amination applied to students of electronic engineering: next generation. New York, NY: McGraw-Hill. Preliminary results. WSEAS transactions on advances in Tastle, J., White, A., & Shackleton, P. (2005). E-learning engineering education, 1, 26-30. in higher education: The challenge, effort, and return of in- Tsiakas, P., Stergiopoulos, C., Nafpaktitis, D., Triantis, D., vestment. International Journal on E-Learning, 241–251. & Stavrakas, I. (2007). Computer as a tool in teaching, Taylor, J. (2001). 5th generation distance education. 20th examining and assessing electronic engineering students. ICDE World Conference on Open Learning and Distance In Proceedings of the International Conference of Com- Education (pp. 1-11). Dusseldorf, Germany. puter as a Tool (IEEE-EUROCON 2007) (pp. 2490-2497).
Teo, T. (2009). Modelling technology acceptance in educa- Tuparova, D., & Tuparov, G. (2010). Management of stu- tion: A study of pre-service teachers. Computers & Educa- dents’ participation in e-learning collaborative activities. tion, 52, 302–312. doi:10.1016/j.compedu.2008.08.006 Procedia Social and Behavioral Sciences, 2, 4757–4762. doi:10.1016/j.sbspro.2010.03.764 Tham, C. M., & Werner, J. M. (2005). Designing and evaluating e-learning in higher education: A review and rec- Ullrich, C., Borau, K., Luo, H., Tan, X., Shen, L., & ommendations. Journal of Leadership & Organizational Shen, R. (2008). Why Web 2.0 is good for learning and Studies, 11(2), 15–26. doi:10.1177/107179190501100203 for research: Principles and prototypes. 17th International World Wide Web Conference (WWW 2008).
356 Compilation of References
Underwood, J. (1997). Integrated learning sys- Wagner, N., Hassanein, K., & Head, M. (2008). Who is tems: Where does the management take place? responsible for e-learning success in higher education? A Education and Information Technologies, 2, 275–286. stakeholder’s analysis. Journal of Educational Technology doi:10.1023/A:1018677616969 & Society, 11(3), 26–36.
Universität Duisburg-Essen (u. d.). Zentrum für Hoch- Wainer, H., Wang, X.-B., & Thissen, D. (1994). How schul- und Qualitätsentwicklung: Kompetenzentwicklung well can we equate test forms that are constructed by in Studium & Lehre. Retrieved August 30, 2010, from examinees? Journal of Educational Measurement, 31, http://zfh.uni-duisburg-essen.de/elearning 183–199. doi:10.1111/j.1745-3984.1994.tb00442.x
Universität Trier (u. d.). Koordinationsstelle E-Learning. Wainer, H., & Thissen, D. (1993). Combining multiple- Retrieved August 30, 2010, from http://www.uni-trier.de/ choice and constructed-response test scores: Toward a index.php?id=33362 Marxist theory of test construction. Applied Measurement in Education, 6, 103-1 18. Uzunboylu, H., Ozdamli, F., & Ozcinar, Z. (2006). An evaluation of open source learning management Walker, J. & Jorn, L. (2007) Net generation students at the systems according to learners tools. Online Submis- University of Minnesota, Twin Cities Student Educational sion. Retrieved December 28, 2010, from http://www. Technology Survey 2007. eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ Walther, J. (1994). Interpersonal effects in computer ERICServlet?accno=ED494265 mediated interaction. Communication Research, 21(4), Van Beveren, J. (2002). A model of knowledge ac- 460–487. doi:10.1177/009365094021004002 quisition that refocuses knowledge management. Wan, Z., Fang, Y., & Neufeld, H. (2007). The role of In- Journal of Knowledge Management, 6(1), 18–22. formation Technology in technology-mediated learning: doi:10.1108/13673270210417655 A review of the past for the future. Journal of Information Van der Linden, W. J., & Glas, C. A. W. (2000). Comput- Systems Education, 18(2), 183–192. erized adaptive testing: Theory and practice. Heidelberg, Wang, W., & Wang, C. (2009). An empirical study of Germany: Springer. instructor adoption of Web-based learning systems. Venkatesh, V., & Davis, F. (2000). A theoretical extension Computers & Education, 53, 761–774. doi:10.1016/j. of the technology acceptance model: Four longitudinal. compedu.2009.02.021 Management Science, 46(2), 186–204. doi:10.1287/ Wang, Y., & Chen, N. (2009). Criteria for evalu- mnsc.46.2.186.11926 ation synchonous learning management systems: Ventouras, E., Triantis, D., Tsiakas, P., & Stergiopoulos, Arguments from the distance language classroom. C. (2010). Comparison of examination methods based Computer Assisted Language Learning, 22(1), 1–18. on multiple-choice questions and constructed-response doi:10.1080/09588220802613773 questions using personal computers. Computers & Educa- Wass, V., Wakeford, R., Neighbour, R., & Van der Vleu- tion, 54, 455–461. doi:10.1016/j.compedu.2009.08.028 ten, C. (2003). Achieving acceptable reliability in oral Vivek, S., & Ansari, Y. (2010). Collaborative learning examinations: An analysis of the Royal College of General issues in synchronous online interactions. International Practitioners membership examination’s oral component. Conference on Technology for Education (T4E), 5-8. Medical Education, 37, 126–131. doi:10.1046/j.1365- 2923.2003.01417.x Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Watson, W., & Watson, S. (2007). What are learning Management, 14(5). doi:10.1108/09513540010344731 management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y
357 Compilation of References
Watson, W. R., & Watson, S. L. (2007). An argument for Wood, R. (1991). Assessment and testing: A survey of clarity: What are learning management systems. What research. Cambridge, UK: Cambridge University Press. are they not, and what should they become? TechTrends, Woods, R., Baker, J., & Hopper, D. (2004). Hybrid struc- 51(2), 28–34. doi:10.1007/s11528-007-0023-y ture: Faculty use and perception of Web-based courseware Webster, J., & Hackley, P. (1997). Teaching effectiveness as a supplement to face-to-face instruction. The Internet in technology-mediated distance learning. Academy of and Higher Education, 7(4), 281–297. doi:10.1016/j. Management Journal, 40(6), 1282. doi:10.2307/257034 iheduc.2004.09.002
Weller, M. (2007). Virtual learning environments: Us- World Bank. (2010). Learning management system. World ing, choosing and developing your VLE. UK: Routledge. Bank Institute. Retrieved on April 7th, 2010, from http:// web.worldbank.org Wenger, E. C. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. World Wide Web Consortium. (W3C) (2008). Web ac- doi:10.1177/135050840072002 cessibility initiative (WAI), Web Content Accessibility Guidelines (WCAG) 2.0. Available at http://www.w3.org/ Wenger, E. C., & Snyder, W. M. (2000). Communities of TR/WCAG20 / practice: The organizational frontier. Harvard Business Review, 78(1), 139–145. World Wide Web Consortium. (W3C) (2010). Web Ac- cessibility Initiative (WAI), Authoring Tool Accessibility Whisker, G. (2001). The Postgraduate Research Hand- Guidelines (ATAG) 2.0. Available at http://www.w3.org/ book. Basingstoke, UK: Palgrave MacMillan. TR/ATAG20/ Whitelock, D. (2009). Editorial: E-assessment: Develop- World Wide Web Consortium. (2010). WAI-ARIA Over- ing new dialogues for the digital age. British Journal of view. Available at http://www.w3.org/WAI/intro/aria Educational Technology, 40(2), 199–202. doi:10.1111/ j.1467-8535.2008.00932.x World Wide Web Consortium. (W3C). (2010 b). Involving users in evaluating Web accessibility. Available at http:// Wiener, M., & Mehrabian, A. (1968). Language within www.w3.org/WAI/eval/users.html language: Immediacy, a channel in verbal communication. New York, NY: Appletoncentury-Croft. World Wide Web Consortium. (W3C). (2010 c). WAI, evaluating Web Sites for accessibility: Overview. Avail- Williams, J., & Goldberg, M. (2005). The evolution of able at http://www.w3.org/WAI/eval/ e-learning. [Global.]. Universitas, 21. Yau, J., Lam, J., & Cheung, K. S. (2009). A review of Wilson, S., Blinco, K., & Rehak, D. (2004). An e-learning e-learning platforms in the age of e-learning 2.0. 2nd Inter- framework - Paper prepared on behalf of DEST (Australia), national Conference on Hybrid Learning and Education JISC-CETIS (UK), and Industry Canada. (ICHL 2009), LNCS 5685 (pp. 208-217). Winde, M. (Ed.). (2010). Von der Qualitätsmessung zum Yildirim, S., Temur, N., Kocaman, A., & Goktas, Y. (2004). Qualitätsmanagement. Praxisbeispiele an Hochschulen. What makes a good LMS: An analytical approach to as- Essen, Germany: Edition Stifterverband. sessment of LMS. In Proceedings of the 5th International Winograd, T., & Flores, F. (1986). Understanding Conference on Information Technology Based Higher computers and cognition: A new foundation for design. Education and Training, Istanbul, Turkey (pp. 125–130). Norwood, NJ: Ablex. Yueh, H.-P., & Hsu, S. (2008). Designing a learning Witten, I. H., & Frank, E. (2005). Data mining: Practical management system to support instruction. Communica- machine learning tools and techniques (2nd ed.). San tions of the ACM, 51(4). doi:10.1145/1330311.1330324 Francisco, CA: Morgan Kaufman.
358 Compilation of References
Zhang, D., Zhao, J., Zhou, L., & Numamaker, J. (2004). Can Zhao, J., & Yiang, Y. (2008). A case study on collabora- e-learning replace classroom learning? Communications tive e-tutoring. Proceedings of 2008 IEEE International of the ACM, 47(5), 75–78. doi:10.1145/986213.986216 Symposium on IT in Medicine and Education (Article number 4743903, pp. 439-444). Zhang, H., Almeroth, K., Knight, A., Bulger, M., & Mayer, R. (2007). Moodog: Tracking students’ online Zhou, Z. (2004). From classrooms to the real engineer- learning activities. In C. Montgomerie & J. Seale (Eds.), ing world: The training program in the microelectronics Proceedings of World Conference on Educational Multi- research center at Georgia Tech. IEEE Transactions on media, Hypermedia and Telecommunications 2007 (pp. Education, 47, 188–120. doi:10.1109/TE.2003.822262 4415–4422). Chesapeake, VA: AACE.
359 360
About the Contributors
Rosalina Babo is a Coordinator Professor at the Information Systems Department at School of Ac- counting and Administration of Porto / Polytechnic Institute of Porto (ISCAP/IPP), Portugal. Since the year 2000 she is the head of the Information Systems Department and a member of the university scientific board. She was one of the founders (2006) of CEISE/STI research center and its director until the year 2011. Her primary area of research is eBusiness as well as Usability Evaluation. Under these subjects cross country studies regarding cultural differences is a challenge for her. Also eLearning is one of her strong interests.
Ana Azevedo is a lecturer in Information Systems and Technologies at the School of Accounting and Administration of Porto / Polytechnic Institute of Porto (ISCAP/IPP), Portugal. She is a PhD Student in Information Systems and Technologies at the department of information systems, University of Minho, Guimarães, Portugal. She is a researcher of Algoritmi Research Center and of CEISE/STI research center. She was one of the founders (2006) of CEISE/STI research center and its vice-director until the year 2011. Her primary areas of research interests are Decision Support Systems, Data mining, and Business Intelligence. Her research interest also includes e-Learning. She is a member of the Editorial Review Board of the International Journal of Technology and Human Interaction (IJTHI) and a member of the editorial board of the International Journal of Data Mining and Emerging Technologies.
* * *
Kamla Ali Al-Busaidi is an assistant professor of information systems at Sultan Qaboos Univer- sity in Oman. She received her PhD in management information systems from Claremont Graduate University in California in 2005, MSc in information systems management from Duquesne University in Pennsylvania in 1999, and BSc in information systems from Sultan Qaboos University in Oman in 1997. Her research interests include knowledge management systems, decision support systems, learning management systems and the deployment of information and communication technologies in Arab countries. She has published several conference proceedings and journals, including International Jour- nal of Knowledge Management, Journal of Global Information Technology Management, International Journal of Global Management Studies and Communications of IBIMA. She also served as a reviewer for several international and local journals, and conference proceedings including AMCIS, ICIS, HICSS, IRMA, IEEE & IBIMA. She has been an organizing member for several local and international workshops and conference proceedings. About the Contributors
Hafedh Al-Shihi is an Assistant Professor at the College of Commerce and Economics in Sultan Qaboos University. He is a member of the Association of Information Systems (AIS) and the Special Interest Group on Electronic Government (SIGeGov). He is also a Research Associate in the Center for International Corporate Governance Research at Victoria University, Australia. He has several publications on e-government / m-government, e-learning and technology adoption and dissemination and has attended several local and international conferences. He has been asked to be a keynote speaker in local and international conferences. Latest was in the “SOIE & FIESTA Conference in Tunis 2009” organized by the Tunisian Association for Artificial Intelligence. He acted in the reviewing committee in several international journals and conferences. Latest was the International Journal of Electronic Governance. Currently, he is teaching introductory and major Information Systems (IS) courses to undergraduate students and supervising the Information Systems Group of IS majored students (www.isg-zone.com). He also publishes a weekly article in Oman Newspaper related to IT.
Antonio Arauzo-Azofra received MSc in Computer Science in 2001 and PhD in the field of Ar- tificial Intelligence in 2006 from the University of Granada (Spain). He is currently Assistant Profes- sor of Project Engineering in the Higher Technical College at the University of Córdoba (Spain). His primary areas of research are feature selection applied in classification problems for intelligent data mining in information systems and the development of intelligent user adaptive systems for automated facility layout design. Specialized in project management where he has been lecturing for 6 years, he is in active collaboration on several projects for the improvement of teaching methodologies adapted to the European Higher Education Area. He received the University of Córdoba Social Council award of Educational Innovation in 2008.
Gavin Baxter is a Research Assistant in the School of Computing at the University of the West of Scotland. His research focuses on the implementation of Web 2.0 technologies in organisations for the purposes of organisational learning. His other research areas include: the applicability of blogs as communication and knowledge sharing tools in organisational project-based environments. He has also published works about organisational blogging in the journals: The Learning Organization and IJCEnt. He also researches about the application of Web 2.0 technologies in educational contexts and is presently involved on the ‘Web 2.0 European Resource Centre’ project.
Rocío Calvo, I graduated in Computer Science from Carlos III University of Madrid in the 2008 and studied as a postgraduate in Computer Science and Technology at the same University in 2010. Nowadays, I am working for the Computer Science department of Carlos III University as a researcher. (MoreInformation: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_rcalvo)
Maura Cerioli has been an Associate Professor in Computer Science at the University of Genova since 2000. Her initial research interests were related to the study of formal methods for program specification. More recently she is working in the area of Software Engineering, and is currently undertaking studies on teaching strategies. She has taught in the years a number of subjects, including programming languages, software engineering, advanced programming techniques, data bases. She is the chair of the ICT certification program for the University of Genova and member of the committee designing the new graduate and undergraduate curricula for Computer Science in her Faculty.
361 About the Contributors
Thomas Connolly is Chair of the ICT in Education Research Group at the University of the West of Scotland and is Director of the Scottish Centre for Enabling Technologies and Director for the Centre of Excellence in Games-based Learning. His specialisms are online learning, games-based learning and database systems. He has published papers in a number of international journals as well as authoring the highly acclaimed books ‘Database Systems: A Practical Approach to Design, Implementation, and Management’, ‘Database Solutions’ and Business Database Systems, all published by Addison Wesley Longman. Professor Connolly also serves on the editorial boards of many international journals, as well as managing several large-scale externally funded research projects.
Antonio J. Cubero-Atienza was born in Córdoba, Spain, in 1961. He received the BS degree in Agricultural Engineering from the University of Córdoba, Spain, in 1984, and the PhD in Agricultural Engineering from University of Córdoba, Spain, in 1988. He is a Professor of University of Córdoba, in the Department of Rural Engineering, Area of Projects Engineering. His teaching areas include Project Engineering, Safety and Health at work, between others. Currently, he is Executive Manager of Cór- doba University from June 2010. His primary research areas include noise and vibration in industrial environments, environmental noise, identification models of sound, educational technology and sound quality in biofuels.
Sandra Delgado Marqueta graduated top in Computer Science from the Carlos III University of Madrid in 2009 and studied a postgraduate degree in Computer Science and Technology at the same university in 2010. Currently, She is working in R+D Department in the IT Company working in accessible Web design.
Dorota Dżega received her PhD degree in Computer Science from Szczecin University of Technology (currently West Pomeranian University of Technology) in Szczecin, Poland. She is Assistant Professor at Faculty of Economics and Computer Science, West Pomeranian Business School (Szczecin, Poland) and Vice Dean of e-Learning. Her research and educational activities relate to the project management (especially software projects), data analysis and computer systems in management. She has an experience in the realization of research projects and development projects financed by Polish Ministry of Science and European Union. She is a member of Association of Academic E-learning and The Polish Information Processing Society.
Bob Folden has an EdD in Developmental Education, Instructional Systems and Design. Dr. Folden has been involved in various areas of distance education. Having taught over ITFS, managed a correspondence course program, worked with satellite course delivery and computer managed instruction, he has a personal knowledge of distance education. He led the development of integrated technologies for distance education which integrated satellite delivery of instruction with computerized simulation to train technicians at a distance. His courses at Texas A & M University-Commerce cover business administration and various aspects of management information systems. Most of his courses are taught at a distance using web based technologies. He also consults with a number of external entities to develop and deliver instruction at a distance.
362 About the Contributors
Laura García-Hernández was born in Spain on August 14, 1984. She received MSc in Computer Science in 2007 from the Oberta University of Catalunya (Spain). She works currently as assistant professor of Projects Engineering Area in the Department of Rural Engineering of the Higher Technical College at the University of Córdoba (Spain). Her research interests are in the fields of Facility Layout Design, Interactive Algorithms, Evolutionary Computation, Project Management and Educational Technology. Since 2007, she is in active collaboration on projects for the adaptation of teaching methodologies to the European Space for Higher Education and, in particular, to the ECTS.
Carole Gould is a PhD student in the school of Computing at the University of the West of Scot- land researching Web 2.0 technologies in Learning and Teaching. She is specifically working towards the design of an educational framework that takes account of the growing use of Web 2.0 technologies within both educational and vocational environments. She has a number of publications in this area.
Thomas Hainey is a researcher in the School of Computing at the University of the West of Scot- land specialising in games-based learning and particularly evaluation of games-based learning. He has a number of journal and conference publications in this area.
Ana Iglesias is a member of the faculty of the Computer Science Department of Carlos III University of Madrid, since February 2006. She obtained a degree in Computer Science from Carlos III University of Madrid in 1999, and her PhD in Computer Science from Carlos III University of Madrid (UC3M) in 2004. Since 2000, I worked at the Advanced Databases Group in the Computer Science Department at Carlos III University of Madrid. I am also a member of the Spanish Centre of Caption and Audiodescrip- tion (CESyA). I have been working in several National research projects on Accessibility, Advanced Database Technologies, CASE Environments, Natural Language Processing and Information Retrieval. My research interests include Accessibility, Database Design and Advanced Database Technologies, Adaptive Intelligent Educational Systems, Natural Language Processing and Information Retrieval. (More Information: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_aiglesia)
Ray M. Kekwaletswe is an Associate Professor of Information in the Department of Informatics at Tshwane University of Technology, South Africa. Previously, he was the Manager of Multimedia and Digital Computing at the American University, Washington DC, where he attained the BA, BSc, and an MSc in Computer Information Systems and Technology. He earned his PhD degree in information systems from the University of Cape Town. His research areas include ubiquitous mobile computing, human computer interaction, context and presence awareness, and emerging learning and teaching technologies.
Arne W. Koesling has graduated in computer science at the Carl-von-Ossietzky Universität of Old- enburg, Germany in 2003. Since 2005 he worked for Hanover Medical School supervising the central learning management system ILIAS. From 2005 to 2008 he also worked as a research assistant for the L3S Research Center. He took part in the European TENCompetence project aiming at facilitating lifelong learning. In 2008, he switched to the newly established eLearning Service Department (ELSA). His research activity is currently centered around adaptation and adaptability in regular learning management systems. Arne is a member of the Stud.IP core-group, which is steering the further development of Stud.IP.
363 About the Contributors
Phil Marc Krüger, born 1971, graduated in electrical engineering at the university of applied science in Emden, Germany. He also received a Master of Higher Education (Lehramt für Berufsbildende Schulen) at the Leibniz Universität Hannover. Since 2001, he works as researcher in the field of e-Learning as a lecturer in media- and higher education teaching methods at the L3S Research Center. In 2008, he switches to the newly established eLearning Service Department (ELSA). There, his field of activity comprises consulting and further education of university lecturers, as well as research and development tasks to explore new kinds of teaching. His focal point of work is on video-based learning. Within this topic, he did substantial examinations about self-directed and cooperative learning. You can find more information at http://www.marckrueger.de
Carola Kruse solved her state examination as Primary School Teacher in 2005 at Universität Osnabrück but did not become a teacher after all. She started working at University Osnabrück at the Centre for Information Management and Virtual Teaching (virtUOS) as a counsellor, aiding professors and academic staff with using the LMS Stud.IP and finding access to technology enhanced learning. In addition she did extra-occupational distance learning academic studies in “Adult and further Education” at Technische Universität Kaiserslautern and graduated in 2009. Since 2010 Carola Kruse is working as a researcher at Technische Universität Braunschweig in the Centre of Excellence for Lower Saxon Higher Education. She is responsible for the LMS project management and offers courses for professors and academic staff. Since March 2010 Carola Kruse is member of the Stud.IP Core Group, the commission steering the general further development of Stud.IP.
Paul Lam is an Assistant Professor at the Centre for Learning Enhancement and Research (CLEAR) at The Chinese University of Hong Kong (CUHK). He involves heavily in many teaching and learning research studies and services such as promotion of the outcomes-based approach to learning, the enhancement of teaching and learning spaces, and the use of technology for teaching and learning. Paul Lam’s research interests range widely across several key aspects of teaching and learning, including web-supported teaching and learning, mobile learning, case-based teaching and learning, learners’ characteristics, self and peer assessment, teaching and learning spaces, and English language teaching (ELT).
José Paulo Leal is Assistant Professor at the Department of Computer Science of the Faculty of Sciences of the University of Porto (FCUP) and Associate Researcher of the Center for Research in Advanced Computing Systems (CRACS). His main research interests are eLearning system implementation, structured document processing and software engineering. He has a special interest on automatic exercise evaluation, in particular on the evaluation of programming exercises, and on web adaptability. He has participated in several research projects in his main research areas, including technology transfer projects with industrial partners. He has over 60 publications in conference proceedings, journals and book chapters.
Jack Lee acquired his Master of Philosophy in 2006 and now is a Project Executive working in Centre for Learning Enhancement and Research, The Chinese University of Hong Kong. His research areas involve the correlation between student’s experience and perception on eLearning, and eLearning application and development; usage and analysis of Weblog; student motivation of peer assessment; and
364 About the Contributors
consultation and development of teaching and learning spaces etc. Recently, Jack leads a team of eight graduate students and conducted a campus-wide research project regarding students’ learning style and learning activity; and uses the SEM to model the learning environment of the university.
Judy Lo is an Assistant Computer Officer in the Information Technology Centre of The Chinese University of Hong Kong. She is responsible for developing and maintaining eLearning platforms and services for the university. She suggests and implements enhancements to the eLearning platforms and services to cope with new demands. She provides consultation and technical assistance to help faculty members on eLearning course design, courseware development and use of new technology for teaching and learning at the university. She is also responsible for eLearning promotion at the university and has been involved in major eLearning projects such as eLearning Service@CUHK, Podcast@CUHK and CUHK on iTunes U. She has a BBA in Management Information Systems, an MSc in Computer Science and a Postgraduate Diploma in Education.
Carla Teixeira Lopes is an invited assistant at Faculdade de Engenharia da Universidade do Porto (FEUP). She holds a degree in Informatics and Computing Engineering by FEUP in 1999 and Master in Information Management by FEUP in 2005. Presently, she is also a PhD student in Informatics Engi- neering at the same institution. In the past, she was a Professor in the Biomathematics, Biostatistics and Bioinformatics scientific area of a health-allied higher education institution. Before being a lecturer, she was a Project Manager in Informatics Companies. Her main research interests are Information Retrieval and Human Computer Interaction. E-learning is also one of her general interests.
Carmel McNaught is Director and Professor of Learning Enhancement in the Centre for Learning Enhancement and Research (CLEAR) at The Chinese University of Hong Kong. Since the early 1970s, Carmel has worked in higher education in Australasia and southern Africa in the fields of chemistry, science education, second language learning, eLearning, and higher education curriculum and policy matters. Current research interests include evaluation of innovation in higher education, strategies for embedding learning support into the curriculum, and understanding the broader implementation of the use of technology in higher education. She is actively involved in several professional organizations and is a Fellow of the Association for the Advancement of Computers in Education; is a University Qual- ity Assurance Auditor for both Australia and Hong Kong; is on the editorial board of 12 international journals; and is a prolific author; recent publications and activities can be viewed at http://www.cuhk. edu.hk/clear/people/Carmel.html
Lourdes Moreno, earned a PhD in computer science from Universidad Carlos III de Madrid and Mathematician in the speciality of Computation Sciences for the Universidad Complutense de Madrid. I have worked with several IT companies, in R+D Departments, working in matters about Infometrics (information measurement) especially in web-channel. Since 2002 I work as an Associate Professor and since 2004 as an Assistant Professor at the Advanced Database group in the Computer Sciences Depart- ment at the Universidad Carlos III de Madrid. Presently I belong to the Spanish Centre of caption and Audiodescription (CESYA) working in the accessibility for people with different disabilities. I have developed the doctoral thesis in the Design and development of accessible Web applications. I work in several international research projects and I am co-author of several publications (More Information: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_lmoreno).
365 About the Contributors
Paulo Coelho de Oliveira is an Associate Professor and Lecturer in Electrical & Computer Engineer- ing at the Polytechnic Institute of Porto - Higher School of Engineering (IPP-ISEP. PhD in Didactics of Higher Education from University of Aveiro in 2009, MSc in Mechanic Engineer (Option Fluids and Heat Transfer) in 2001 from Faculty of Engineering, University of Porto and Diploma Degree (4 years) in Physics and Applied Mathematics in 1994 from Faculty of Science, University of Oporto. He is a member of Laboratory of Quality Evaluation in Education at the University of Aveiro. His main research interests are related with science & engineering teaching and evaluation in Higher Education.
Wiesław Pietruszkiewicz received his PhD degree in Computer Science from Szczecin University of Technology (currently West Pomeranian University of Technology) in Szczecin, Poland. He is Tech- nical Director at SDART Ltd a UK-based company in Manchester - being an innovative researching & software developing company, where he is responsible for the creation of systems incorporating scientific components. He also holds a position of Assistant Professor at Faculty of Computer Science and Information Technologies, West Pomeranian University of Technology, where teaches about the software design and programming artificial intelligence & knowledge-based systems. His research interests relate to the software development, intelligent systems and data mining & processing. He is a member of Association for Computing Machinery.
Mário Pinto is an Associate professor in Informatics Department, at the Higher School of In- dustrial Studies and Management (ESEIG), Polytechnic Institute of Porto. PhD in Informatics from PortucalenseUniversity in 2008, MSc in Electrical and Computer Engineering in 2000 from Faculty of Engineering, University of Porto and Diploma Degree (4 years) in Applied Informatics in 1989 from Portucalense University. He is a founding member of KMILT (Knowledge Management, Interactive and Learning Technologies) research group, and researcher at Portucalense University, and member of ISPIM. His main research interests are related with Learning management systems, knowledge management systems and support decision systems.
Brendan P. Post is the Manager of Online Learning Systems at The College at Brockport where he has been a member of the technology staff for over five years. Brendan has been involved in the area of learning management systems for several years, playing an active role in the ANGEL LMS user community. His participation includes multiple presentations at user conferences, founding member of the Western New York ANGEL User Group, and primary developer of software integration between ANGEL and Apple’s iTunes U. Brendan holds a Bachelor of Arts in Computer Science and Mathematics from Messiah College, Grantham, Pennsylvania.
Ricardo Queirós is an Assistant Professor at the School of Industrial Studies and Management (ESEIG) in Vila do Conde, which is responsible for courses in the area of ICT and Programming Languages. He is a PhD student of the Doctoral Program in Computer Sciences in the Faculty of Sciences of the Uni- versity of Porto (FCUP). His scientific activity is related with e-Learning standards and Interoperability, Languages for XML, Architectural Integration with focus on the development of e-Learning Systems. He is an associated member of the Center for Research in Advanced Computing Systems (CRACS) - an INESC-Porto Associated Laboratory - and a founding member of KMILT (Knowledge Management, Interactive and Learning Technologies) research group.
366 About the Contributors
María Dolores Redel-Macías was born in Córdoba, Spain in 1977. She received the BS Degree in Automatic and Electronic from the University of Córdoba, Spain, in 2006. She is currently as Assistant Professor in the Department of Rural Engineering, University of Córdoba, Spain. Her current areas of interest include identification models of sound, sound quality in biofuels. She is collaborating in different educational project about educational technologies, virtual labs and e-learning tools.
Marina Ribaudo is an Associate Professor in Computer Science. She is currently teaching subjects on Network Protocols, Web technologies, Peer-to-Peer systems at the University of Genova. Her initial research interests were related to the study of formal methods (stochastic extensions of Petri nets and process algebras) for the qualitative and quantitative modeling and analysis of concurrent and distributed systems. More recently, she has undertaken studies related to network applications in general (client- server, peer-to-peer, publish-subscribe systems) with particular interest to usability and accessibility issues. From July, 2005, she is one of the members of a group called Advisory Board for the ICT Educational Support, a committee suggesting strategies for the introduction of ICT technologies in education. From February 2009, she is coordinating this group.
Ana Cláudia Rodrigues is an Assistant Professor and lecturer in Human Resources field at School of Industrial Studies and Management (ESEIG), a Polytechnic Institute of Porto’s faculty. Is a PhD student at ISCTE. She was responsible for creating and managing a social network of students and alumni in the Human Resources undergraduate course. She also intervened in the course curriculum change, in order to emphasize new technologies. She has been a member of a research group of Internet and LMS usage at IPP and has published several papers studying this issue. She is also a founding member of a Human Resources research group (NID_RH).
Marina Rui is researcher at the Faculty of Science of University of Genova since 1983. She has a degree in Physical Chemistry and works in Theoretical Chemistry. In the last decade she also dealt with issues related to the use of ICT in education, blended learning and e-learning.In her university, she has been Coordinator of the Committee for the WEB and of the Advisory Board for the ICT Educational Support of which she is member since 2004. Together with M. Ribaudo, she is current representative of the AulaWeb Portal at University of Genova. She is also member of the Organizing Committee of the Italian Moodlemoot.
Lorenzo Salas-Morera received MSc in Agricultural Engineering in 1989 and PhD in the field of Agricultural Engineering in 1993 from the University of Córdoba (Spain). He is a Professor of Univer- sity of Córdoba, in the Department of Rural Engineering, Area of Projects Engineering. He is currently the Director of Academical Organization of Córdoba University and his primary areas of research are engineering design, development of intelligent user adaptive systems for automated facility layout design, projects management and educational technology. Since 2002, he has led several projects for the adaptation of teaching methodologies to the European Higher Education Area and, in particular, to the ECTS. He received the University of Córdoba Social Council award of Educational Innovation in 2008.
367 About the Contributors
Demetrios G. Sampson is an Associate Professor on eLearning at the Department of Digital Sys- tems, University of Piraeus, Greece and a Senior Researcher at the Informatics and Telematics Institute (ITI), Centre for Research and Technology Hellas (CERTH). His main scientific interests are in the areas of Technology-enhanced Learning. He is the co-author of more than 235 publications in scientific books, journals and conferences with at least 820 known citations (h-index: 16). He has received five (5) times the Best Paper Awards in International Conferences on Advanced Learning Technologies. He is Co-Editor-in-Chief of the Educational Technology and Society Journal (impact factor 1.067, 2009), Associate Editor of the IEEE Transactions on Learning Technologies (TLT) Journal, Member of Editorial Board of (18) International Journals and Guest Co-Editor in (16) Special Issues of International Journals. He is a Senior Member of IEEE and the Chair of the IEEE Computer Society Technical Committee on Learning Technology (LTTC).
Anthony Scime is a graduate of George Mason University with an Interdisciplinary Doctorate in Information Systems and Education. Currently, he is an Associate Professor of Computer Science at The College at Brockport, State University of New York. His work in data mining has been published in Expert Systems with Applications, the International Journal of Business Intelligence and Data Min- ing, Data Mining and Knowledge Discovery Technologies, and Public Opinion Quarterly. Idea Group Publishing published his book Web Mining: Applications and Techniques. His current research interests include the Web mining, data mining in the social and behavioral sciences, and computing education.
Tyler Swanger is currently employed with Yahoo! as a Network Operations Engineer. At Yahoo!, Tyler is responsible for monitoring and troubleshooting problems that occur within Yahoo’s network, as well as assist other Yahoo! engineers to quickly resolve customer facing issues. He studied Computer Science and Computational Science at The College at Brockport State University of New York. Tyler’s interests include high performance computing, parallel computing, data mining, and their applications. His current research focuses on data mining algorithms and their computational performance.
Thanh-Thu Phan Tan has studied English Literature and Culture, Pedagogical Sciences as well as Business Studies in Hannover and Liverpool and has graduated from Leibniz Universität Hannover in 2002. After that, she has worked at L3S Research Center in Hannover for the next 5 years. In 2007, she switched to the Center of Excellence in Lower Saxon Higher Education at the Technische Universität Braunschweig. In 2008, she has finished her doctoral degree on e-learning competencies in Higher Education. By now, she has led and co-worked in several projects on technology enhanced learning in Higher Education and is – next to some teaching in that field – implementing the learning management system at the university at the moment.
Dimos Triantis has studied at the University of Athens. He received his BSc in Physics in 1975, the MSc in Electronic Automation in 1980 and his PhD in Solid State Physics in 1983. In 1989 he was appointed Professor in the Department of Electronics of the Technological Educational Institution of Athens, in the discipline of Semiconductor and Dielectric Physics. Since 2007 he has been Honorary Professor Associate of the School of Engineering and Design of Brunel University (UK). He was Head of the Department of Electronics from 1992 to 1997, and for a number of years Subject Area Leader in Electronics. He is Dean of Faculty of Technological Applications from the academic year 2010-11. His
368 About the Contributors
research interests include the areas of Microelectronics and Nanoelectronics, non-destructive testing of materials via electric measurements and new technologies in education. His published research work in international journals and international conference proceedings numbers over 150 papers.
Erricos Ventouras got his Diploma in Engineering from the Department of Electrical and Computer Engineering of the National Technical University of Athens in 1989 and the title of Doctor in Engineering from the same Department in 1994, on Biomedical Engineering. From 1998 he is with the Department of Medical Instrumentation Technology of the Technological Educational Institution of Athens, where he is currently Professor of Biomedical Engineering. He has taught at Graduate Courses at the University of Patras and at the University of Athens. His research interests include electronic examination methods, biosignal data acquisition and processing and tomographic techniques for medical imaging. He has over 100 publications in international scientific journals, book chapters and proceedings of international conferences. He was Scientific Leader of European Union-funded research projects.
Kaitlyn Whitlock graduated from The College at Brockport in Computer Information Systems. She was the Assistant Project Manager of the ANGEL Learning Management System evaluation team in The College’s Library Information Technology Services Department. Currently, she is a Network Operations Engineer at Yahoo! responsible for monitoring and troubleshooting network problems and assisting other Yahoo! engineers in quickly resolving customer issues.
Luis Zaragoza. I have been working as a Journalist for Radio Nacional de España (the Spanish National Radio) in its News Service, since September 2008. There I cover news related to disabilities in its different aspects (technological among others). Born blind in 1978, I obtained a degree in Journalism from Complutense University of Madrid in 2001 and a PhD from that University in 2007. My PhD was published as a book in 2008. I also obtained a degree in Geography and History from the National Uni- versity of Distance Education (UNED) in 2004, and a masters in Management of Political and Electoral Communication from the Autonomous University of Barcelona (UAB) in 2005. I worked during 2007 in Servimedia, a news agency specialized in social subjects. I have published several papers on the history of Communication, the contemporary Spanish history and the history of blind organizations in Spain.
Panagiotis Zervas received a Diploma in Electronics and Computer Engineering from the Technical University of Crete, Greece in 2002 and a Master’s Degree in Computational Science from the Department of Informatics and Telecommunications of the National and Kapodistrian University of Athens, Greece in 2004. Currently, he is a PhD candidate at the Department of Digital Systems, University of Piraeus, Greece and a Researcher at the Informatics and Telematics Institute (ITI) of the Centre for Research and Technology Hellas (CERTH). His main scientific interests are in the areas of Technology-enhanced Learning. He is the co-author of more than 25 publications in scientific journals and conferences. He has received the Best Paper Award in the 2nd International Conference on Intelligent Networking and Collaborative Systems (2010), the Best Short Paper Award in IEEE International Conference on Ad- vanced Learning Technologies (July 2009) and the Best Poster Award in IEEE International Conference on Advanced Learning Technologies (July 2007).
369 370
Index
A B Academic Management System (AMS) 147, 155- behaviorism 4, 26-27 157 Blackboard 17, 54, 59, 95, 109, 111-112, 117-118, accessibility 41, 120, 124, 199-217, 241, 251, 309 127, 135-136, 143, 145, 147, 152-153, 155, Accounting Information System (AIS) 7, 207, 216 157, 165, 167, 177, 202-203, 216, 249, 276, active content 78, 82, 84, 93 293 active discussion 78, 83, 93 blended learning 11-12, 27, 88, 90, 138, 174, 243- active quizzes 93 244, 248, 276, 291, 314, 324, 332 active websites 82, 93 blogs 13, 23, 50, 52, 54, 71, 73, 87, 99, 108, 144, ad hoc surveys 272 203, 254, 256, 260-262, 264-265, 269, 274, administrational staff 321, 334 290, 294, 314 Adobe Flash™ 223, 228, 234, 236 bottom-up approach 315-319, 321, 327-328 Adobe FlashPaper™ 223 buy-or-build dilemma 275 American Society for Training and Development (ASTD) 112, 158, 179, 194, 268 C ANGEL Learning Management System 94-95 ANGEL mining 94 C4.5 algorithm 97, 102 ANGEL users 108 change management 139, 316-319, 322-323 ANOVA 187-189, 303, 306 Chinese University of Hong Kong (CUHK) 75-76, Application Programming Interface (API) 153-154, 82, 87-89 157, 160 Chi-square distribution 253 Apriori algorithm 103-104 Classification Based on Multiple Class-Association Architecture, Arts and Design (AAD) 252 Rules (CMAR) 97, 110 Assistive Technologies (ATs) 200, 202-204, 207, classification mining 98, 107, 113 214 classification rule 102, 113 association mining 94-95, 97-99, 102, 104-105, clustering 94-95, 97-98, 105-108, 113, 242, 288 107, 113-114 clustering rule 113 association rule 98, 102-103, 105, 109, 113 co-evaluation 310, 314 Asynchronous Discussion Forum (ADF) 298, 300, cognitivism 26-27, 45 311, 313-314 collaborative activities 280, 282-284, 289, 291, AulaWeb 272, 276-283, 285-290 294, 312 authoring tools 147-149, 152, 192, 201, 203-204, Collaborative Learning System (CLS) 13, 314 206-208, 213-215, 233, 236 Communities of Practice (CoPs) 55, 71-73 automated evaluation 205 Computer Assisted Instruction (CAI) 2-6 Average Variance Extracted (AVE) 130 Computer-Based Learning (CBL) 51, 70, 118, 179, awareness 28-32, 34-49, 203, 325 183, 314 Computer Managed Instruction (CMI) 3-4, 6 computer self-efficacy 120, 123, 135 Index
Constructed Response (CR) 72, 178, 180, 184, 195 e-Postgraduate Studies 220-221 Constructed Response Questions (CRQs) 180-181, e-Tutoring 220, 312, 314 183-190, 193 European Credit Transfer System (ECTS) 297-298, constructivism 26-27, 55 307-308, 311-312 context awareness 32, 34-37, 39-40, 42-43, 45-47 Evaluation Engine (EE) 148, 150 contextual inquiry 30, 37, 46, 49 evaluation method 199-200, 204-205, 207, 244, 295 Control of Substances Hazardous to Health (COSHH) 62-63 F Correspondence Course Learning (CCL) 4-5 Course Management System (CMS) 12, 22, 24, Faculty Information System (FIS) 7-8, 13 118, 143, 158, 160, 164, 174-175, 177, 184, Federal Communications Commission (FCC) 27 243, 269, 273, 289 female students 250, 253-254, 256, 258, 260-264 Finance Information Systems (FIS) 7-8, 13 D G data mines 94 data mining 92, 94-95, 97-99, 105, 107-113, 232- General Education Program (GEP) 98 233, 243-244 Germany 24-25, 136, 175, 196, 213, 242, 266, 268, decision trees 94-95, 97, 99-102, 110 289, 315-316, 319, 321, 323-324, 329-334 disability 199, 212, 215 Google Talk 257 distance learning 4, 10-11, 16, 23-27, 31, 95, 111, Graphical User Interface (GUI) 6, 154, 206, 233- 122, 137, 158, 177, 213, 220, 244, 266, 275- 234 276, 324 Group Tutoring (GT) 293, 314 E H East Kilbride & District Engineering Group Training Health (HEA) 58, 63, 252-254, 256-262, 264-265, Association (EKGTA) 58-60 267, 296 e-Books 220-221, 225, 228, 230, 236 Higher Education (HE) 9-10, 24-26, 30-31, 38-40, e-Business 70, 221 46, 48, 56-57, 65, 70-73, 90-92, 120, 123, 135, Eclipse IDE plugin 149 137-139, 145-146, 155, 157, 160, 162-165, Economics (ECO) 33, 243, 252-254, 256-261, 264, 167, 173-177, 180-181, 185, 194-195, 197-200, 278, 286, 334 212, 214-215, 219, 222, 231, 241, 243, 247- e-Content 219-221, 223-224, 228-229, 231, 233, 252, 262-263, 265-267, 269, 272, 274, 288- 242, 244-245 292, 297, 300-301, 312-313, 325, 329, 331 educational technology 25-26, 46, 48, 69-73, 109- Humanities (HUM) 85, 252, 317, 334 110, 112, 134, 136, 138, 175-176, 213, 266, Human Resource Information System (HRIS) 8-9 268-269, 290, 310, 312-314 e-Examination 184-185, 192 I e-Learning 6, 11-14, 16, 21, 23-27, 70, 72, 88, 90- immediacy 30, 34, 47, 308 93, 96, 109, 111, 113, 117-121, 123-127, 133- IMS Digital Repositories Interoperability (IMS 140, 157-160, 174-177, 194-195, 199-200, 203, DRI) 144, 152 213-216, 219-245, 264, 266-269, 272-275, 282, IMS Learner Information (IMS LIP) 155 287, 289-295, 311-314, 324, 326, 329-333 IMS Learner Information Services (IMS LIS) 155 eLearning Liaison Person (eLLP) 89 IMS Learning Design (IMS LD) 151, 177, 243 e-Mail 102, 226, 232, 300 IMS Learning Tools Interoperability (IMS LTI) engagement 30, 55, 71, 77, 80-81, 83, 85-86, 93, 151, 153, 158 223, 244, 316-317 IMS Simple Sequencing (IMS SS) 151 engineering education 159, 176-177, 196, 198, 213, Information Communication Technology (ICT) 310, 313-314 5-12, 14, 27, 62, 91-92, 117, 120-121, 123-124, epistemology 27, 33, 52, 57 200, 249-250, 264, 271-273, 275-277, 281, e-Portfolio 70-71, 147-148 284, 286-289, 291, 296-297, 305
371 Index
information quality hypotheses 124 Leibniz Universität Hannover (LUH) 315, 325-328, Instant Messenger (IM) 40, 329, 331-332 332 instructional design 26, 137, 203, 213, 243, 271- Library Information System (LIS) 9, 13, 155 272, 280-282, 288, 291 LMS applications 117-118 Instructional Management System (IMS) 144, LMS evaluation tool 96, 109 149-153, 155, 157-159, 165, 177, 192, 195, LMS logs retrieval and reading system 93 200-202, 215, 243, 311 LMS platforms 28, 42, 224, 248-249, 258-262, Instructional Television Fixed Service (ITFS) 4-6, 264-265 27 LMS’s characteristics 131, 133, 140 Instructor’s Characteristics 131, 140 LMS success factors 140 Integrated Learning System (ILS) 2, 11, 24 internet usage 247-248, 250-251, 253-255, 262, M 264, 266 internetworking 6 male students 250, 253-254, 256-258, 260-264 interoperability 70, 142-145, 147-148, 150-153, manual evaluation 205, 207 155, 157-160, 192, 195, 268-269, 319 MCQs examination method 180-181, 183, 185-188, intimacy 34 190-191 MCQs test 181, 183 metadata 96, 148-154, 158-161, 201, 215, 266, J 268-269 Jorum Team 148, 158, 269 Mobile Learning Management System (mLMS) 162-164, 167-168, 173-175, 177 K modern apprenticeship programme 58 Moodle 17, 26, 54, 59-60, 62-69, 71, 76-77, 80, 82, key administrators 334 84, 92, 105-106, 109, 111-113, 118, 127, 135, Knowledge Management Systems (KMS) 23, 25, 138, 143, 145, 152-155, 157-158, 162, 164- 92, 95 165, 167-176, 199-200, 202-203, 207-213, 215, knowledge sharing 28-30, 32, 34-36, 38-39, 42-46, 220, 223-224, 227-228, 231, 233, 239, 243- 49, 55 244, 249-251, 254, 261, 263, 266-267, 271, Knowledge Transfer Partnerships (KTP) 69 276-277, 281-282, 285, 289, 291, 293-294, 297, 299-302, 307, 311 L Multiple-Choice (MC) 143, 152, 178, 180-184, Laszlo Systems OpenLaszlo™ technology 227 193-198 Law and Social sciences (LSS) 252-254, 256-262, Multiple-Choice Questions (MCQs) 178, 180-193, 264 195-197 Learning 2.0 50 MXML language 239 Learning Content Management System (LCMS) 12- 13, 118, 143, 147, 158, 160, 179-180, 195, 201, N 203, 212, 216, 266, 268-269 nature of activities 79, 93 Learning Management Systems (LMS) 1-2, 7, 9, net generation 250, 266, 268-269 11-19, 22-25, 27-30, 32, 35, 38, 41-43, 45, 48, 50-54, 56-60, 68-69, 72-73, 75-76, 78-79, 81-98, 102, 105, 107-113, 116-127, 131-136, O 138-140, 142-148, 150-153, 155-158, 160-168, Online Assessment (eAssessment) 51, 59, 194, 296 173-179, 184, 192-194, 199-205, 207, 210, Online Quizzes (OQ) 77-78, 83-85, 87, 93, 293- 212-213, 215, 219-224, 226-228, 230-233, 235, 294, 296-299, 307, 309-311 239, 242-244, 247-251, 253-254, 258-269, 271- organization’s characteristics 116-117, 119, 121- 276, 281-283, 288-291, 315-329, 331, 333 122, 131, 133, 140 Learning Objects (LO) 13, 75, 139, 143-144, 147- Overall Quizzes Scores (OQS) 303 152, 154, 157-158, 160, 265-266, 268-269
372 Index
(SCORM) 144, 149, 151-152, 159, 165, 200, P 202-203, 225, 230, 235-236, 239, 245, 250, Partial Least Square (PLS) 129-130, 135 268, 275, 324 PDF creator 223, 228 Short Message Service (SMS) 40, 43 pedagogical strategies 19, 21, 248 Skype 257 pedagogy 19, 23, 26-27, 48, 55, 90-91, 250, 311 social awareness 28-32, 34, 37-43, 45, 48-49 perceived ease of use 116-117, 119-120, 122-127, social presence 28-32, 34-47, 49, 136, 312 131-135 social software 55, 72-73, 87, 176, 274, 289 personalized learning 28, 31, 38, 40 special needs 199, 214 PLS approach 129 SPEEG (SPE E-book Generator) 235-236 Polytechnic of Porto 248, 251-253, 256, 262-263, Standardized Hyper Adaptable Metadata Editor 265 (SHAME) 149, 159 popularity 17, 56, 77, 79, 82, 93, 144-145, 149, Structural Equation Model (SEM) 129, 136 152, 200, 241 Student Information System (SIS) 9, 13 Portable Digital Assistants (PDAs) 37-40, 43, 52, Sultan Qaboos University (SQU) 116, 127, 134 166, 177 Sum of Squared Errors (SSE) 106-107 portfolio 56-57, 73, 149, 153, 159 Supplementary LMS Use 140 portfolio API 153 surveys 51, 77, 89, 144, 179, 203, 207, 239, 272, Practical Sessions (PS) 297-298, 301 285, 291-292, 294, 297, 302, 305, 309 programmed learning 2, 4, 6, 11, 27 system quality hypotheses 124 Programming Advanced Techniques (TAP) 284 PSR-MCQs method 183, 188, 190, 192 T Pure LMS Use 140 TCC classes 98 Technische Universität Braunschweig (TU BS) 315, Q 318, 324-325, 329 QSR NVivo 7 253 Technological Educational Institution of Athens QTI specification 192 (T.E.I.-A.) 178, 184, 192 Question & Test Interoperability (QTI) 150-152, Technology (TEC) 5-8, 10-12, 14-16, 24-28, 39-40, 192 45-48, 50-51, 53-55, 58, 60, 68-73, 88, 91-92, 96-97, 102, 109-110, 112-113, 116-127, 132- R 140, 144, 153, 160, 162, 165, 174-176, 180, 194-196, 200, 202, 211-213, 215, 220-221, RDF metadata 149 223, 227, 230, 234, 239, 242, 250-254, 256, RDF syntax 149 258-262, 264, 266-269, 272-274, 280, 289-291, repository API 153-154, 157 296, 310-314, 316, 323, 325, 334 Resource Description Framework (RDF) 149 template 105, 215 Theoretical Blocks (TB) 298 S Thesis Continuation Credit (TCC) 98 Science (SCI) 24-25, 33, 46, 72, 90, 110, 134, tool consumer profile 152 137-138, 158, 160, 174-177, 196, 213, 220, top-down approach 315, 317, 319, 321, 326-328 243-244, 250, 252, 268, 275, 277-279, 283, T-Quest 279, 282, 285-286 286, 296, 310-311, 313, 324, 332-333 traditional learning 9, 11, 244, 273 scoring rules 182, 193-194, 198 screen reader 200, 202-203, 207, 211, 213, 216 U SDART Presentation Engine (SPE) 219-220, 223, ubiquitous personalized support 28, 49 228, 230, 233-236, 238-242, 245 Universität Osnabrück (UOS) 326-328 self-efficacy 119-121, 123, 127, 130, 132, 134-135, University of Genova 271-273, 275, 288, 290 138, 269 university strategy 315 service quality hypotheses 124 usability 25, 96, 175, 207, 211, 213-214, 244, 258- Sharable Content Object Reference Model 259, 294, 311, 317
373 Index
usage framework 122 Waikato Environment for Knowledge Analysis User-Centered Design Process (UCD) 200-202, (WEKA) 98 204, 212, 215 Web 2.0 50, 89 User Interface (UI) 6, 153, 175, 184, 201-202, 207- Web 2.0 Tools 51, 68-69, 88, 144, 269 208, 232-233, 285 web accessibility 201, 207, 212-216 Users’ Experience with the Technology (EUT) 123 WebCT 17, 25, 76-77, 80, 82, 84, 88, 113, 118, 127, 135, 138-139, 143, 145, 147, 157, 165, 249, V 293 Web Enhanced Learning (WEL) 280-285, 290-291 variance-based SEM 129 WEL Project 280, 282-283 Video Conferencing (VC) 4-6, 27 West Pomeranian Business School (ZPSB) 219- Virtual Learning Environments (VLEs) 12, 53, 90, 221, 223, 227, 232-233, 235-236, 239, 242 92, 135-136, 158-160, 164, 175, 177, 203, 213- What You See Is What You Get (WYSIWYG) 209, 214, 268, 334 215, 227 Virtual Learning (VL) 6, 12, 53, 92, 135-138, 147, Wiki 57, 64-69, 71, 73, 226, 279, 290 158-160, 164, 175, 177, 203, 211, 213-214, World Wide Web Consortium (W3C) 152, 162, 266, 268, 334 164-167, 171-175, 201-204, 206-207, 213-215 vocational training 50-51, 53, 55-58, 60, 69 X W XML (eXtensible markup language) 150, 152, 155, WAI-ARIA Suite 203 161, 233, 235-236, 238-241, 245, 269
374