DOCSLIB.ORG

A Natural User Interface and Touchless Interaction Approach on Web Browsing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Natural User Interface and Touchless Interaction Approach on Web Browsing A Natural User Interface and Touchless Interaction Approach on Web Browsing Master of Science Thesis in the Master Degree Programme, Interaction Design & Technologies GEORGIOS LAFKAS Report No: 2013:143 ISSN: 1651-4769 Department of Applied I.T. CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden, 2013 Abstract The aim of this Master’s Thesis is to explore the principles of Natural Interaction and suggest an implementation of a web browsing application that uses them. The research that takes place is examining the ingredients of Natural User Interfaces and Touchless Interaction with a focus on Gestures and Voice. How can those be combined to control an interface? What are the most important things to mind when designing Natural Interfaces and Interactions? These are questions that this report attempts to answer. The final result of the thesis is a prototype web browser application that works only with gestures and voice. The prototype simulates browsing on a number of created web pages. It is implemented using the Microsoft Kinect for XBox sensor and Kinect SDK v1.7 with all the included tools and interaction controls. The browser is undergoing a usability test and the participants are observed and later interviewed to describe their experience. Based on that, the conclusions drawn are that designing an application that uses a Natural Interface needs to provide short and comfortable Gesture and Speech Vocabularies and Feedback mechanisms that inform the user at every point of the interaction. Furthermore, it is concluded that the design of the content for such an application should be different than the traditional design of a webpage for optimal use while responding to implicit input (context- awareness) adds to the naturalness of the interaction. Finally, the setting for using such an application is recommended to be part of a home entertainment system (e.g. smart TV) or public space installation rather than a desktop computer. Keywords: human-computer interaction, natural user interfaces, touchless interaction, gestures, speech recognition, touchless web browsing. 2 Table of Contents Introduction................................................................................................................................................5 Problem................................................................................................................................................5 Goal and aim........................................................................................................................................5 Research question.................................................................................................................................5 Structure...............................................................................................................................................6 Involved Parties....................................................................................................................................6 Background................................................................................................................................................7 User Interfaces......................................................................................................................................7 Computer Interfaces (CI)................................................................................................................7 Command Line Interfaces (CLI)....................................................................................................8 Graphical User Interfaces (GUI)....................................................................................................9 Touch User Interfaces (TUI)...........................................................................................................9 Natural User Interfaces (NUI)......................................................................................................10 Motion Sensing Technology...............................................................................................................11 Mechanical Sensing......................................................................................................................11 Inertial Sensing.............................................................................................................................12 Acoustic Sensing..........................................................................................................................12 Magnetic Sensing..........................................................................................................................12 Optical Sensing.............................................................................................................................12 Radio and Microwave Sensing.....................................................................................................12 Current motion tracking solutions......................................................................................................13 Nintendo Wii.................................................................................................................................13 Playstation Move..........................................................................................................................13 Kinect............................................................................................................................................14 Leap Motion..................................................................................................................................15 Omek Interactive..........................................................................................................................15 Related Work......................................................................................................................................16 Theory......................................................................................................................................................19 Gestures..............................................................................................................................................19 Gesture Recognition.....................................................................................................................21 Speech................................................................................................................................................21 Speech recognition........................................................................................................................22 Semiotics............................................................................................................................................23 Natural Interaction concepts...............................................................................................................24 Intuition and intuitive behavior....................................................................................................24 Channels of communication and multimodality...........................................................................25 Context..........................................................................................................................................25 Ergonomics.........................................................................................................................................26 Methodology............................................................................................................................................27 Overview............................................................................................................................................27 Ideation.........................................................................................................................................27 Requirements................................................................................................................................27 Prototyping...................................................................................................................................28 User test........................................................................................................................................29 Execution.................................................................................................................................................30 Ideation...............................................................................................................................................30 Concept Definition.............................................................................................................................30 Literature Study..................................................................................................................................31 Benchmarking....................................................................................................................................31 Requirements......................................................................................................................................31 Low-fidelity Prototype.......................................................................................................................32 Concept Refinement...........................................................................................................................32
Load more

Recommended publications
  • Infosys to Demonstrate Next-Generation Natural User Interface Cable TV Concepts at the Cable Show’S “Imagine Park”
    PRESS RELEASE Infosys to Demonstrate Next-Generation Natural User Interface Cable TV Concepts at The Cable Show’s “Imagine Park” Leveraging device sensor capabilities and digital set-top boxes will enable rich, interactive television viewing experiences letting consumers “wave goodbye” to their remote controls BOSTON, MAY 22, 2012 – Infosys will showcase new and innovative ways that consumers may experience television in the future via multi-channel natural user interfaces at “Imagine Park,” an interactive, high-profile platform for introducing and demonstrating innovative new products and services at The Cable Show 2012. Rather than using a remote control, Infosys will show and discuss how digital consumers can use voice, facial-recognition and simple gesture-based movements to access their personalized content and interact with the set-top- box of the future. Natural user interfaces (NUI) – touch, voice, video and gesture interfaces – in devices ranging from smartphones and tablets to peripherals such as videogame consoles provide an opportunity for cable companies to deliver personalized interactive customer experiences. “Cable television innovation is crucial to serve the growing number of digital consumers who expect rich, interactive, personalized viewing experiences, and natural user interfaces are the next step in the evolution of TV entertainment,” said Mit Majumdar, head of U.S. cable practice at Infosys. “With our breadth and depth of expertise, Infosys provides cable operators the ability to capture the imagination of their customers by helping them quickly launch new technologies like NUI that allow consumers to access personalized TV with the wave of a hand or voice command.” From the perspective of cable multiple services operators, multi-channel user interfaces will enable service providers to increase customer loyalty, emphasize personalization and help add interactive revenue-generating channels to differentiate their television service offerings.
    [Show full text]
  • Gesture Recognition for Human-Robot Collaboration: a Review
    International Journal of Industrial Ergonomics xxx (2017) 1e13 Contents lists available at ScienceDirect International Journal of Industrial Ergonomics journal homepage: www.elsevier.com/locate/ergon Gesture recognition for human-robot collaboration: A review * Hongyi Liu, Lihui Wang KTH Royal Institute of Technology, Department of Production Engineering, Stockholm, Sweden article info abstract Article history: Recently, the concept of human-robot collaboration has raised many research interests. Instead of robots Received 20 July 2016 replacing human workers in workplaces, human-robot collaboration allows human workers and robots Received in revised form working together in a shared manufacturing environment. Human-robot collaboration can release hu- 15 November 2016 man workers from heavy tasks with assistive robots if effective communication channels between Accepted 15 February 2017 humans and robots are established. Although the communication channels between human workers and Available online xxx robots are still limited, gesture recognition has been effectively applied as the interface between humans and computers for long time. Covering some of the most important technologies and algorithms of Keywords: Human-robot collaboration gesture recognition, this paper is intended to provide an overview of the gesture recognition research Gesture and explore the possibility to apply gesture recognition in human-robot collaborative manufacturing. In Gesture recognition this paper, an overall model of gesture recognition for human-robot collaboration is also proposed. There are four essential technical components in the model of gesture recognition for human-robot collabo- ration: sensor technologies, gesture identification, gesture tracking and gesture classification. Reviewed approaches are classified according to the four essential technical components. Statistical analysis is also presented after technical analysis.
    [Show full text]
  • E-37-V Dome Camera Operation Manual
    E-37-V Dome Camera Operation Manual i Table of Contents 1 Network Config .............................................................................................................. 1 1.1 Network Connection .......................................................................................... 1 1.2 Log in ................................................................................................................ 1 2 Live ................................................................................................................................ 6 2.1 Encode Setup ................................................................................................... 6 2.2 System Menu .................................................................................................... 7 2.3 Video Window Function Option ......................................................................... 7 2.4 Video Window Setup ......................................................................................... 8 2.4.1 Image Adjustment ....................................................................................... 8 2.4.2 Original Size ............................................................................................... 9 2.4.3 Full Screen ................................................................................................. 9 2.4.4 Width and Height Ratio ............................................................................. 10 2.4.5 Fluency Adjustment .................................................................................
    [Show full text]
  • Natural Interaction in Augmented Reality Context
    Natural Interaction in Augmented Reality Context John Aliprantis1, Markos Konstantakis1, Rozalia Nikopoulou2, Phivos Mylonas2 and George Caridakis1 1 University of the Aegean, 81100 Mytilene, Greece {jalip, mkonstadakis, gcari}@aegean.gr 2 Ionian University 49100 Corfu, Greece [email protected], [email protected] Abstract. In recent years, immersive technologies like Virtual and Augmented Reality have been accelerating at an incredible pace, building innovative experiences and developing new interaction paradigms. Current research has widely explored gesture interaction with Augmented Reality interfaces, but usually requires users to manipulate input devices that could be cumbersome and obtrusive, thus preventing them from interacting efficiently with the 3D environment. Therefore, Natural User Interfaces and freehand gesture interaction are becoming more and more popular, improving the user’s engagement and sense of presence, providing more stimulating, user-friendly and non-obtrusive interaction methods. However, researchers argue about the impact of the interaction fidelity in usability and user satisfaction, questioning the level of naturalness that should characterize the interaction metaphors. Current paper proposes different gesture recognition techniques for three basic interaction categories (translation, rotation and scaling) in a Leap Motion Controller - Augmented Reality framework. A prototype is implemented in order to evaluate efficiency and usability of the proposed architecture. Finally, experimental results are discussed. Keywords: Natural interactionAugmented realityLeap motion controller Gesture recognition. 1 Introduction Over the last few years, Augmented Reality (AR) has developed into a cutting edge technology, providing new ways to interact with computer – generated information. By removing the boundaries between physical and virtual, AR has been able to create more engaging experiences, enhancing user’s enjoyment and satisfaction.
    [Show full text]
  • Class -IV Super Computer Year- 2020-21
    s Class -IV Super Computer Year- 2020-21 2. Windows 7 ❖ Focus of the Chapter 1. Windows desktop 2. Desktop icons 3. Start Menu 4. Task bar 5. Files and folders 6. Creating & saving new file/folder 7. Selecting a file/folder 8. Opening a file/folder 9. Renaming a file/folder 10. Deleting a file/folder 11. Moving a file/folder 12. Copying a file/folder 13. Creating a shortcut to a file/folder Keywords • Booting – Loading of the operating system. • Taskbar- The long bar present at the bottom of the desktop • Notification area- The area located on the right side of the taskbar • Folder- A container for storing files and other folders. Introduction Windows 7 is an operating system that Microsoft has produced for use on personal computers. It is the follow-up to the Windows Vista Operating System, which was released in 2006. An operating system allows your computer to manage software and perform essential tasks. It is also a Graphical User Interface (GUI) that allows you to visually interact with your computer’s functions in a logical, fun, and easy way. Interact with your computer’s functions in a logical, fun, and easy way. * The first screen appear after you turn on the power of computer is a desktop • If it is a shared PC; more than one user uses it, or one user with password protected, you will arrive at Welcome Screen Desktop Components 1- Icons: An icon is a graphic image, a small picture or object that represents a file, program, web page, or command.
    [Show full text]
  • Chrome Security
    Browser Privacy Settings Some browsers may occasionally make modifications to privacy settings to protect users from possible unsecured content. Unsecured content is identified with the prefix http:// in the URL and can create mixed content in your Canvas Page. Secured content is identified with the https:// prefix in the URL. If something is not displaying properly in Canvas, it could be the browser is preventing it from showing. You can always click the ‘new window’ icon to open the content in a new window. Or you can ‘allow the content. Below, are images that show how to enable content in both browsers Google Chrome and Mozilla Firefox. Chrome Security Google Chrome verifies that the website content you view is transmitted securely. If you visit a page in your Canvas course that is linked to insecure content, Chrome will display a shield icon in the browser address bar. You can choose to override the security restriction and display the content anyway by clicking the shield icon and then clicking the Load unsafe script button. Chrome Media Permissions Chrome has its own media permission within the browser. To use your computer camera and microphone within any Canvas feature, you will have to approve two permissions: 1. Allow access to Canvas via the Adobe Flash Player Settings. This prompt appears in the center of the video and audio pop-up windows, or the center of the browser if you are accessing Conversations. 2. Allow access to Canvas via Chrome's media permission settings. This prompt appears just below the address bar. Click the Allow button.
    [Show full text]
  • CUSTOMIZING YOUR TOOLBARS in INTERNET EXPLORER, OUTLOOK EXPRESS, WINDOWS and WORD a Seminar Presented by Bill Wilkinson April 2008
    CUSTOMIZING YOUR TOOLBARS IN INTERNET EXPLORER, OUTLOOK EXPRESS, WINDOWS AND WORD A Seminar Presented by Bill Wilkinson April 2008 Internet Explorer 7 Toolbars Internet Explorer 7, the latest version of Microsoft’s Web browser, sports a cleaner look than its predecessor. In fact, some users might wonder where everything went. The main toolbar is reduced in IE7 to just a couple of main buttons in the upper-left corner near the Address bar and a smattering of smaller buttons scattered about. The Menu Bar, a familiar feature in just about every program, was hidden by default in the earliest version IE7, but Microsoft was shamed into returning it to its traditional place. If you downloaded IE7 when it first came out, you will not see the Menu Bar. We will show you how to access it during this seminar session. The IE7 standard install also lacked a visible Links toolbar. Fortunately, Internet Explorer 7 features a streamlined toolbar that you can customize if you find it a little too sparse. The Page button is really a drop-down menu of commands loosely related to viewing the page in your browser window. This is where you can change text size, zoom level, or send pages and links through email. The Copy and Paste commands also ended up here, along with the command for opening a new window . Clicking Tools opens another submenu with a lot of the same commands from the original Tools menu. Here you’ll find the Pop-up Blocker, the Internet Options command, and other settings. Rounding out the IE7 interface are star-shaped buttons on the left side of the screen to add and view Favorites (called the Favorites Center).
    [Show full text]
  • Study of Gesture Recognition Methods and Augmented Reality
    Study of Gesture Recognition methods and augmented reality Amol Palve ,Sandeep Vasave Department of Computer Engineering MIT COLLEGE OF ENGINEERING ( [email protected] ,[email protected]) Abstract: However recognizing the gestures in the noisy background has always been a With the growing technology, we difficult task to achieve. In the proposed humans always need something that stands system, we are going to use one such out from the other thing. Gestures are most technique called Augmentation in Image desirable source to Communicate with the processing to control Media Player. We will Machines. Human Computer Interaction recognize Gestures using which we are going finds its importance when it comes to to control the operations on Media player. working with the Human gestures to control Augmentation usually is one step ahead when the computer applications. Usually we control it comes to virtual reality. It has no the applications using mouse, keyboard, laser restrictions on the background. Moreover it pointers etc. but , with the recent advent in also doesn’t rely on certain things like gloves, the technology it has even left behind their color pointers etc. for recognizing the gesture. usage by introducing more efficient This system mainly appeals to those users techniques to control applications. There are who always looks out for a better option that many Gesture Recognition techniques that makes their interaction with computer more have been implemented using image simpler or easier. processing in the past. Keywords: Gesture Recognition, Human Computer Interaction, Augmentation I.INTRODUCTION: Augmentation in controlling the media player operations. Media player always In Human Computer interaction, the first requires mouse or keyboard to control it.
    [Show full text]
  • Poly Video Mode Administrator Guide
    ADMINISTRATOR GUIDE 3.0.0 | December 2019 | 3725-85854-004A Poly Video Mode Getting Help For more information about installing, configuring, and administering Poly/Polycom products or services, go to Polycom Support. Plantronics, Inc. (Poly — formerly Plantronics and Polycom) 345 Encinal Street Santa Cruz, California 95060 © 2019 Plantronics, Inc. All rights reserved. Poly, the propeller design, and the Poly logo are trademarks of Plantronics, Inc. All other trademarks are the property of their respective owners. Contents Before You Begin................................................................................................6 Audience, Purpose, and Required Skills.............................................................................6 Related Poly and Partner Resources..................................................................................6 Getting Started................................................................................................... 8 Product Overview of Poly Video Systems...........................................................................8 Administrator Features and Capabilities.................................................................. 9 Powering the System On and Off......................................................................................10 Navigating the System...................................................................................................... 10 Access the System Web Interface..........................................................................10
    [Show full text]
  • Internet Explorer 11
    ® Microsoft Internet Explorer 11 Qu ick Reference Card Internet Explorer 11 Window Keyboard Shortcuts Address Bar list arrow Security report General Search Refresh View favorites, feeds and history New Window <Ctrl> + <N> Navigation Home New Tab <Ctrl> + <T> Buttons Address Tab New tab Close Window/Tab <Ctrl> + <W> Bar Open link in New Tab <Ctrl> + <Shift> Tools (Foreground) + click Open link in New Tab <Ctrl> + click (Background) Select Address Bar <Alt> + <D> Find <Ctrl> + <F> Print <Ctrl> + <P> Select All <Ctrl> + <A> Zoom In <Ctrl> + <+> Zoom Out <Ctrl> + <-> Full Screen Mode <F11> Help <F1> Add Favorite <Ctrl> + <D> Refresh <F5> Stop <Esc> Navigation Next item <Tab> Up one screen <Page Up> Scroll Bars Down one screen <Page Down> Home <Alt> + <Home> The Fundamentals Tools <Alt> + <X> Favorites <Alt> + <C> • A web address, also known as a URL (Uniform • To Search the Web: Click the Search button in the Resource Locator) is made up of two main parts: Address Bar, type a keyword or term in the Address Bar View Address Bar <F4> ü http:// (Hypertext Transfer Protocol): the set of and press <Enter>. History rules for exchanging files on the World Wide • To Change the Search Provider: Click the Address Bar Go Forward <Alt> + <à> Web. list arrow in the Address bar and select the provider Go Back <Alt> + <ß> ü Domain name: the unique name that you wish to use from the row of icons along the bottom of Switch to previous Tab <Ctrl> + <Shift> identifies a specific site on the web. the menu. + <Tab> • To Add a Search Provider: Click the Address Bar list • To Open a Web Page: Click the Address Bar and Switch to next Tab <Ctrl> + <Tab> type the address of the Web page.
    [Show full text]
  • Chromebook Cheat Sheet Quick Tips to Keep You Rockin’!
    Chromebook Cheat Sheet Quick Tips to keep you rockin’! 1) Power 2) Connecting to the Internet 3) Logging in 4) Touchpad Navigation 1 finger on touchpad 2 fingers on touchpad at the same time 3 fingers on touchpad at the same time 5) Keyboard CAPS LOCK Print Screen 6) “Top 10” Chromebook Keyboard Shortcuts 10. Take a Screenshot on a Chromebook 9. Open File Manager 8. Open Task Manager 7. Launch Apps on the Shelf 6. Lock the Screen 5. Open the Hotdog 4. Toggle Caps Lock On/Off 3. Logout 2. Mirror Monitor 1. Show Keyboard Shortcut Overlay 7) A bunch more keyboard shortcuts 1) Power Opening the clamshell lid of a Chromebook will turn it on. If it’s already open but powered off, push the power button in the upper-right corner of the keyboard. Be aware that just closing the lid does NOT turn it off, and will slowly drain the battery. To power the Chromebook down, either use the “Shut down” button in the lower-right menu, or push and hold the Power button at the upper-right of the keyboard. 2) Connecting to the Internet Chromebooks are basically just internet browsing devices. If you start up a school Chromebook and get the dinosaur screen - “Can’t connect to internet” - turning it off and turning it back on will usually get it to connect. 3) Logging in & out Chromebooks require a slightly different log on than “regular” computers. Users need to enter their entire school email address (not just initial and last name) and their regular school password.
    [Show full text]
  • Natural User Interfaces and Accessibility
    Chapter 1 Natural User Interfaces and Accessibility he widespread adoption of mobile computing is a computers using actions related to everyday behavior, good thing for librarians who care about access such as touch, gestures, speech, and conversation.1 T for all. That’s because mobile devices make use The goal is to make it easy for humans to understand of natural user interfaces, and those interfaces are and use computers without having to learn compli- making computing easier for people of all ages and cated or abstract ways of doing things. Designers of abilities, as you’ll see in this report. these interfaces aim to create experiences that feel This trend, combined with the move toward multi- just as natural to a novice as to an expert user—and device ecosystems and the emphasis on students as for expert users it can feel like an extension of their creators with mobile apps, means that mobile learning body. is headed in a direction that is empowering for learn- ers of all abilities. There are other trends in mobile learning, but this Putting the Human before report focuses on these three: the Computer: The Move from GUI to NUI • natural user interfaces and accessibility • multi-device ecosystems NUIs are a new branch in the evolution of human- • content creation with mobile devices computer interaction, after GUIs (graphic user inter- faces). GUIs were designed to make computing easier That’s because there are synergies between these to learn than command-line interfaces, where you had trends that offer opportunities for those who care to remember specific commands and type them.
    [Show full text]