The future of human-computer interaction: overview of input devices Fabrizio Fornari School of Computer Science H´ask´olinn´ıReykjav´ık- Reykjav´ıkUniversity Reykjav´ık,Iceland November, 2012 Abstract We are in 2012 and we are still using the mouse and the keyboard to interact with a computer. We have seen a lot of changes in the world of Computer Science, relating to: performance, design and the way we interact with the computer. Differ- ent input devices have been developed around the computer, starting from the most recent touchscreens, continuing with webcams, microphones, and arriving to the oldest mice and keyboards. The aim of this research is to let the reader imagine a new way to interact with the computer. To reach our purpose, we introduce advanced technologies such as: Speech and Voice Recognition, Electronic Perception, Eye Tracking and Brain Computer In- terfaces. We propose examples of the cited technologies that may change the paradigm that saw, until now, keyboard and mouse as leaders of the input devices. 1 1 Introduction From the computer's birth1, we saw a lot of changes in the world of Com- puter Science. Changes relating to: performance, design and human-computer interaction [49]. A few years ago, the words \input device" evoked in our mind only two specific objects: the keyboard and the mouse - the main instruments used to provide data to a personal computer. Keyboard and mouse are, in fact, two of the first input devices in the history of computer. Nowadays, with the evolution of computers, we have a large set of input de- vices that changed the way we interact with the computer. For example we have: touch-screens, that allow users to interact with a computer using only their hands; webcams and microphones, used together to make video-calls and many others. Although we introduced other input devices we are still using mouse and keyboard, why? We want to provide an overview of input devices to let the reader imag- ine a world without mice and keyboards. We start with the definition of input device and the histories of mouse and keyboard to continue introduc- ing instruments and technologies that may replace the usage of mouse and keyboard such as: the Electronic Perception Technology [70] - the ability of electronic components to form a 3-D map of their surroundings and see what their users are doing. We also refer to Eye Tracking [47] - the abil- ity of a computer to keep track of eye movements of users for developing tasks like the movement of the pointer on a screen or the scroll of a page at the same time that the user reads. We describe a new technology called \Tobii Gaze" [38] - an example of Eye Tracking compatible with the new release of Windows 8 [69]. Other technologies that we discuss are Speech and Voice Recognition [58, 59], the abilities that allow a computer to un- derstand speech and to recognize who is the speaker. We introduce one of the most popular software package for Speech and Voice recognition called 1With the term computer, we refer to a Personal Computer (desktop, laptop, etc.) 2 Dragon NaturallySpeaking [32] and we focus on pros and cons of using our own voice to communicate with a computer. At the end we analyze a different approach to improve the human- computer interaction called Brain Computer Interface (BCI) [35]. The BCI consist in a collaboration between a brain and a device that enables signals from the brain to direct some external activities, such as the control of a cursor. We describe the different types of BCI: Invasive, Partially invasive and Non-invasive focusing on an example of Non-invasive Brain Computer Interface called Emotiv EPOC [14] (winner of the 2012 Advance Global Australian Award for Information and Communication Technologies [11]). For each cited technology, we provide a definition and an example that may bring the reader to discover new ways to interact with the computer. 3 2 Background In this section we provide the definition of input device and describe the history of the keyboard and the mouse to give the reader the opportunity to learn more about these 2 devices. 2.1 Input Device From Wikipedia: \An input device is any peripheral - piece of computer hardware equipment - that we use to provide data and control signals to an information processing system such as a computer or other information appliance." [50] In our case we refer to an input device as a device that converts the user's actions and analog data (sound, graphics, pictures) into digital electronic signals that can be processed by a computer. Are the input devices that let a user to control a computer, to interact with it and to take advantage of the calculation power for which the computer is well-known; a computer without input devices would not allow the interaction with an user. 2.2 Keyboard and Mouse Since the birth of the computer, the input devices have increasingly evolved; nowadays we are used to different types of input devices. Despite of the variety of available input devices, the most used ones until now are the keyboard and the mouse. 4 2.2.1 The Keyboard One of the first developed input devices for computer is the keyboard. A keyboard is a typewriter-style device, which uses an arrangement of but- tons or keys to input letters or numbers into a computer device [43]. The first keyboards were really different from the idea of keyboard that we have nowadays. It derives largely from two devices: teleprinters [62] and key- punches [51]. It was from such devices that modern computer keyboards inherited their layouts. From the 1940s until the late 1960s, typewriters were the main means of data entry for computing, becoming integrated into what were known as computer terminals [44]. The keyboard remained the primary, most integrated computer peripheral well into the era of per- sonal computing until the introduction of the mouse as a consumer device in 1984. However, keyboards remain central to human-computer interac- tion to the present day. Indeed even mobile personal computing devices such as smartphones [57] and tablets [60] adopt the keyboard as an optional virtual, touchscreen-based [63], means of data entry. Nowadays we can see different kind of physical keyboards: Standard, Laptop-size, Thumb-sized, Software-Keyboard, Foldable-Keyboard and Optical Keyboard [43]. Studies such as the one presented in [33] have demonstrated that a key- board is a place where bacteria tend to accumulate. An article by the BBC [31] underlines the danger of using a keyboard. From the sanitary point of view, in a hospital a keyboard can be custodian of very dangerous bacteria because different people touch the same keyboard, even people that have been in touch with dangerous bacteria. This can be a good motivation for trying to find a replacement for the keyboard. 5 2.2.2 The Mouse We introduce the definition and history of mouse taken from Wikipedia [54]. In computing, a mouse is a pointing device that functions by detect- ing two-dimensional motion relative to its supporting surface. Physically, a mouse consists of an object held under one of the user's hands, with one or more buttons. The mouse sometimes features other elements, such as \wheels", which allow the user to perform various system-dependent opera- tions, or extra buttons or features that can add more control or dimensional input. The mouse's motion typically translates into the motion of a pointer on a display, which allows for fine control of a Graphical User Interface (GUI) [48]. The computer mouse has been invented by Douglas Engelbart of Stan- ford Research Center in 1964 and was originally called \X-Y Position Indi- cator" for a display system. Made of wood and featuring a bright red push button, the mouse was more like a matchbox than computer device. Still, the \mouse" (so named because the cord looked like a tail) took off, and new versions popped up in Research and Development labs and in stores as the PC and computing itself became more mainstream. Whereas today's mice are sleek and curved, early models were blocky and bulky. Many compa- nies, including Microsoft, Apple, Hewlett-Packard, and Logitech, produced innovative models from the 1980s until now and the mouse has become even more efficient and comfortable. The types of mice vary by technology as much as color: mechanical mice, gyroscopic mice, 3D mice, optical mice, tactile mice, and most recently, wireless mice and touch mice. During the years, as the mouse evolved, studies have been done to find mouse-related health problems [4, 12, 18, 20, 25]. The results obtained from the studies show how the use of mouse can cause traumas such as Carpal Tunnel Syndrome [22], Tendonitis [1] or Bursitis [2]. If the computer mouse can cause this kind of problems, why do we still use it? 6 3 The Alternatives We want to enlighten the reader on the world of input devices, focusing on devices that we think are the most quoted to replace the use of mouse and keyboard. We start from well-known devices such as touchscreens, micro- phones and webcams to continue with new technologies such as: Electronic Perception Technology and Brain Computer Interface. 3.1 Touchscreen A touchscreen [63] is an electronic visual display that can detect the presence and location of a touch within the display area. The term generally refers to touching the display of the device with a finger or hand. Touchscreens can also sense other passive objects, such as a stylus.