Development of a Hci Course on the Microsoft Surface
Total Page:16
File Type:pdf, Size:1020Kb
Session S3F Work in Progress - Development of a HCI Course on the Microsoft Surface Aaron Striegel, Dirk Van Bruggen University of Notre Dame, [email protected], [email protected] Abstract - With the advent of multi-touch devices, there exist exciting new opportunities for course development in the area of Human Computer Interfacing (HCI). From the most recent set of devices, the Microsoft Surface offers a set of unique capabilities and challenges for teaching a HCI course. The natural trend of the device towards simultaneous interactions with multiple users as well as the ability to recognnize physical objects via byte tags offer fascinating teaching and laboratory experiences. In this paper, we will discuss our experiences in teaching a Surface-oriented course with regards to lecture and laboratory design. We will also offer insight with respect to student performance, student feedback, comparingn those aspects versus efforts with the Nintendo Wiimote with regards to programming difficulty. Finally, we will conclude with Figure 1 - Microsoft Surface (surface.com) suggestions for future course refinement. target audiences. The size, cost, and mechanisms for Index Terms – Microsoft Surface, Human Computer interaction (touch, byte tags, Surface API) represent a Interface, Course Development. sizeable departure from other gesture-based systems. It is these departures and their subsequent impact on a Human INTRODUCTION Computer Interface course that form the basis for this paper. In the past few years, there has been an explosion of portable In this paper, we describe what we believe to be the first of devices capable of responding to multi-touch input. While its kind in offering a course primarily dedicated towards the notion of touch as an interface is certainly not new to the teaching HCI on the Microsoft Surface. We will present our area of Human Computer Interface (HCI), the proliferation course design philosophy and offer various commentary of low cost, poortable devices has created significant interest with respect to lessons learned from the first offering in the in using touch / gesture-recognition as an input mechanism. course of the Fall of 2009 at the University of Notre Dame. To that end, significant coursework has emerged taking OVERVIEW – SYSTEM INTERFACE DESIGN advantage of the mobile nature of many of the multi-touch devices with the Apple iPhone / iPod Touch and Google With the idea of exploring a truly unique offering by the Android (only recently multi-touch) being two of the driving Department of Computer Science and the University of devices [1-4]. Notre Dame, the idea was conceived to teach a course where In addition to significant new devices in the mobile the new Microsoft Surface would serve as the basis for the space, there has also been a similar rush of new devices in course. The course, to be offered for the first time in the Fall the non-mobile space. Particularly, the Nintendo Wiimote of 2009, would take upper level students from a variety of has spurred significant interest with aspects ranging from disciplines (though primarily Computer Science) through coursework to inexpensive smart boards to medical various concepts of Human Computer Interface with a interfacing / instrumentation. Similar, the most recent primary focus on programming the Microsoft Surface. release of Windows 7 suppporting multi-touch natively Hence, the course was initially advertised in the Spring of purports to usher in new innovation with respect to multi- 2009 as an exploratory course under the following touch design. description: Perhaps one of the more fascinating multi-touch devices to emerge in the past year has been the Microsoft This course will focus on the interface to computing Surface [5] (see Figure 1). Unlike more inexpensive systems, in particular with respect to multi-sensory devices, the Surface takes several entirely different input / output through direct experience in approaches with respect to multi-touch interfacing and the laboratory and project activities. Specific interfaces /978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washinngton, DC 40th ASEE/IEEE Frontiers in Education Conference S3F-1 Session S3F to be covered include traditional graphical user interface (GUI) design, visualization via specialized controls (network graphs, virtual worlds), multimedia (audio, video), and sensory input / output (motion capture, multi-touch, etc.). Further topics include exploration of the effects of scaling / networking on responsiveness, exposure to multiple design choices (web, framework, native), and exposure to multiple contemporary programming languages. Enrollment for the course was capped at twenty students and the order was placed for two Microsoft Surface devices to be delivered in the late spring of 2009. Enrollment quickly Figure 2 – Byte tag – Microsoft Surface filled up and was extended to allow several additional graduate students and interested faculty members to audit / sit in on the course. The initial exposure of the faculty to the Surface was provided through a Surface demonstration unit at the College of Business at the University of Notre Dame. Unfortunately, the College of Business had only minimal experience developing applications on the Surface itself. Hence, a primary task over the spring and summer was to determine the key capabilities of the Surface itself that in turn were also updated by virtue of a Service Pack updated during the spring semester. Although the order of events Figure 3 – Identity tag – Microsoft Surface with respect to course design and electing to offer the course were less than optimal, the investigation of the capabilities for the user to see with a thick plexiglass / plastic protecting did not truly begin in earnest until delivery of the Surface the projection area from the user. Thus, whereas a multi- units until June of 2009. touch device such as the iPod touch requires a human finger to touch and detects the touch via pressure / capacitance, the DEVICE CAPABILITIES Surface relies on the infrared reflection for finger detection. For the order, a total of three Surface units were delivered, The usage of a set of cameras in conjunction with two for the course itself and one for the Office of the projected image contributes to the large footprint of the Information Technology to explore the Surface in terms of Surface but also enables significantly improved capabilities. potential new technology adoption. Two of the units were To start, the Surface itself possesses the capability to track burned with the Commercial kit for a slight discount up to 50 different objects placed on the screen, thus allowing meaning that applications would be developed externally on up to 5 users to place each of their 10 fingers on the screen a workstation and copied to the Surface unit itself. One unit and to track them individually. Moreover, the usage of the was burned with the Development SDK meaning that code camera enables the use of byte tags (see Figures 2 and 3) could be directly written and debugged using Visual Studio whereby objects placed on the Surface can have a separate 2008 on the Surface itself through an external monitor and identifier. The examples from the Surface literature include keyboard. identifying a wine vintage at a winery (via byte code on the At its core, the first key defining feature of the glass) or identifying camera accessories at an electronic Surface is its size. Unlike a touch-enabled monitor or store. Alternatively, the tag could be used to identify a guest laptop, the Surface is quite large requiring multiple strong as part of a hotel kiosk, enabling an interface based on individuals to move the device. In part, the large footprint personal identification. allows the device to serve as a de facto table where The Surface capabilities run as an application under individuals can easily gather around the edges of the device. Windows 32-bit Vista (Surface Shell). Input to the Surface The device can comfortably enable up to 4 individuals to arrives through a custom interface program (Surface Input). work on the device at the various corners or edges of the The Surface itself is modestly powered with a dual-core device. processor, reasonable video card, and reasonable amount of The second key feature of the Surface lies in how memory capable of running moderate applications. touch interactions are detected. Unlike traditional multi- Programming for the Surface can take place using either touch or touch-based devices that tend to be capacitive- XNA (akin to game programming ala OpenGL) or WPF based, the Surface relies on an infrared camera to detect (successor to Windows Forms). Both approaches use C# fingers and objects placed on the table. Images are projected and are developed in Microsoft Visual Studio 2008. An /978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference S3F-2 Session S3F emulation platform called Surface Workstation enables there was a Microsoft Surface community that had been development of Surface-centric applications on Windows running for over a year, access to the Surface Community 32-bit Vista. The emulation provides the capacity to site was restricted to only two user names per Surface unit emulate finger touches using multiple mice as an input. purchased. Thus, while machines in the laboratory were The summer of 2009 was used to explore the enabled with a username / password combination, students capabilities of the Surface itself and to explore which would be unable to register their own username at the capabilities were reasonable for the students to explore as Surface Community website1. part of the class. Two REU students, Omari Holtz and The first critical decision for the course arose from Solomon Adeyemi, assisted conducted an initial exploration the material to cover to maximize the students’ ability to of the Surface capabilities. Three HP multi-touch devices program on the Surface over the course of the semester.