Bulletin number 32 Delivering and Assessing Learning on Mobile Devices: Report on a pilot project by SQA, May and June 2007 June 2008 Published by the Scottish Qualifications Authority The Optima Building, 58 Robertson Street, Glasgow G2 8DQ Ironmills Road, Dalkeith, Midlothian EH22 1LE www.sqa.org.uk The information in this publication may be reproduced in support of SQA qualifications. If it is reproduced, SQA should be clearly acknowledged as the source. If it is to be used for any other purpose, then written permission must be obtained from the Publishing Team at SQA. It must not be reproduced for trade or commercial purposes. © Scottish Qualifications Authority 2008 SQA is committed to the use of robust evidence in the development and evaluation of policy and its implementation, and carries out or commissions research across a range of topics to support this. The publication of Research Bulletins allows us to disseminate the results of our research activity to practitioners, policy makers, parents, academics and anyone else who has an interest in the key role that qualifications play in economic growth and social inclusion in Scotland. Disclaimer This report is the result of a small-scale research project. The information it contains reflects the opinions and experiences of the researcher and should not be viewed as a comparison of products by the Scottish Qualifications Authority. Products mentioned are examples of devices that could be used for e-learning and e-assessment — there are many others. The inclusion of a product should not be viewed as a recommendation by the Scottish Qualifications Authority, its staff or appointees. Some of these products, including their costs, may also have changed since the report was compiled or published. Contents Executive summary 1 1 Introduction 3 2 Hardware platforms 4 3 Formatting teaching and learning materials 6 4 Using the materials 10 5 Using mobile devices for assessment 12 6 Candidate and teacher experience 14 7 Vendor technologies 15 8 Conclusions and recommendations 17 Executive summary This report presents the findings of a small-scale pilot project undertaken by the Scottish Qualifications Authority between May and June 2007. The remit for the project was to take a preliminary look the prospects for using mobile devices, such as mobile phones and smartphones, for the delivery and assessment of learning. Three devices were selected as broadly representing the types of device in current use for mobile internet access. There are other products which provide the same functions and similar features. The three devices we selected were: ♦ The T-Mobile Vario II MDA: a Windows Mobile-based smartphone. ♦ The Nokia N800 Internet Tablet: for standalone WiFi internet access. ♦ The Nokia 6151 Mobile Phone: a typical budget 3G mobile telephone. The teaching and learning materials selected were for the SQA Internet Safety Unit1. Although these materials render well on full-sized screens, there are several factors that cause problems when trying to display them on mobile devices. These include the requirement for a lot of scrolling, redundant navigational features, the amount of white space, the size of graphics, and the use of add-on technologies, such as Java and Flash. Changes were made to the materials themselves, including removing or reducing graphics, using bolding, avoiding add-ons and keeping pages short. The materials were developed using Course Genie2. Changes were made to the Course Genie settings to simplify navigation, reduce the amount of white space displayed, and to remove the logo from the top of each page. The cumulative effect of these changes was to produce clear and readable pages which can be navigated with a minimal amount of downward scrolling. All the devices tested displayed the materials in a usable format and permitted effective navigation, although the devices with larger screens provided the best user experience. Download speeds were excellent using the latest 3G technology and the slightly older GPRS technology. Excellent results were also obtained using WiFi access. Although there appears to be great potential for using mobile devices for assessment, the mobile assessment aspect of this project was not particularly successful. The online assessments for the SQA Internet Safety qualification are hosted on the SOLAR website, but most of the devices used could not get past the first page, probably because subsequent pages appear to be Flash-based. Only the Nokia N800 Internet Tablet displayed the SOLAR pages correctly. 1 An Intermediate 1 National Unit designed to enable candidates to make safe and legal use of the Internet — http://www.sqa.org.uk/files/nu/F0H510.pdf 2 Course Genie has since been renamed Wimba Create. 1 Course Genie questions, used in the online materials, worked well on devices running the latest Windows Mobile 6, but not on earlier versions. They also worked on mobile phones running the Opera Mini browser. It may be useful to investigate alternative methods for online assessment, such as assessment via text messaging (SMS). Mobile devices may also be suitable for collecting audio and video evidence for a portfolio-based assessment and uploading it to a website, where an assessor or verifier could examine it online. The project included a brief trial of the Mobile version of the SQA Internet Safety materials with a small group of secondary pupils, using relatively modern mobile telephones with GPRS access to the internet via proprietary browsers. The pupils enjoyed the experience and the trial highlighted a number of operational difficulties which will inform future developments. During the project, we encountered two potentially useful vendor-specific technologies: M-Learning and Zoho Challenge. These could usefully be investigated further in future projects. The preliminary study has highlighted three major points: ♦ It is possible, and even relatively easy, to adapt existing learning materials so that they can be displayed on a variety of mobile devices. ♦ Assessment on mobile devices is not nearly as straightforward, largely due to the fact that the mechanisms used make use of technologies, such as Flash, which are poorly implemented on mobile devices at present. ♦ In general, things worked better on the newer and more sophisticated devices than on the older and simpler ones. Future developments would probably be best restricted to specific platforms, particularly devices running the Windows Mobile 6 operating system and mobile phones which support the Opera Mini browser. The outlook for mobile delivery is positive and there is already a large amount of material available which could be converted. The picture regarding mobile assessment is more complex and requires further investigation, but it may improve as technological developments take place. 2 1 Introduction The initial remit for the project was: ♦ Re-format teaching and learning materials for rendering on mobile devices. ♦ Explore suitable formats for rendering materials on mobile devices. ♦ Explore potential of mobile devices for assessment (using the existing SQA Internet Safety item bank). ♦ Research candidates’ and teachers’ views on mobile learning and mobile assessment. ♦ Research candidates’ and teachers’ experience of using the SQA Internet Safety materials and assessments on mobile devices. ♦ Explore vendor technologies to support mobile learning/mobile assessment. The project was very brief and was very much a preliminary look at the prospects and potential pitfalls in this area. Decisions had to be made early in the project about the teaching materials and assessments to be used and the hardware platforms to be covered. The teaching materials selected were those developed recently for the Internet Safety Unit, which had been piloted in the latter half of 2006. These were selected because both teaching and assessment materials were already available in online format. 3 2 Hardware platforms There is a vast range of different types of hardware available, so we decided to restrict coverage to three types of devices. The devices were selected because they broadly represent the types of device in current use for mobile internet access. ♦ The T-Mobile Vario II MDA is an HTC Hermes in T-Mobile livery. Similar phones are available, with slightly different styling, from several other manufacturers.3 It represents the current state-of-the-art in Windows Mobile- based smartphones. At the outset of this study the current release was Windows Mobile 5, but during the study Windows Mobile 6 was released, bringing significant improvements. The operating system and associated software is very similar to that used on many standalone PDAs (ie, those which are not also telephones), so information obtained about this platform should also apply to PDAs. ♦ The Nokia N800 Internet Tablet is a standalone device for internet access. It runs the Maemo operating system, a variant of Linux, and uses the Opera browser. Unlike the other devices tested, it is not a mobile phone, but accesses the internet via Bluetooth or WiFi. It is mainly of interest due to its large screen size (800 x 400 pixels). ♦ The Nokia 6151 Mobile Phone is a typical budget mobile telephone which supports 3G internet access. It is not Windows-based, but has its own proprietary browser. It also runs the Opera Mini browser from Opera Software, which works on a wide range of current phones. The 6151 was recently recommended as a ‘Best Buy’ by Which? magazine. 3 When this report was written the HTC Hermes variants were among only a few Smartphones offering 3G access. At the time of publication there are many others. 4 More about the selected hardware platforms T-Mobile Vario II MDA The Vario II is a compact smartphone with a slide-out QWERTY keyboard. It is 3G and HSDPA enabled, allowing internet access at broadband speeds up to 1.8Mb. It incorporates a 400MHz Samsung processor, 64MB dynamic memory, Microsoft Word and Excel editing, plus a PowerPoint and PDF viewer. Connectivity includes Bluetooth, infrared and USB2.