DOCSLIB.ORG

Better Mouse Support for Touch-Based Interactions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Better Mouse Support for Touch-Based Interactions MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Fluid DTMouse: Better Mouse Support for Touch-Based Interactions Esenther, A.; Ryall, K. TR2006-001 May 2006 Abstract Although computer mice have evolved physically (i.e., new form factors, multiple buttons, scroll-wheels), their basic metaphor remains the same: a single-point of interaction, with modifiers used to control the interaction. Many of today’s novel input devices, however, do not directly (or easily) map to mouse interactions. For example, when using one’s finger(s) or hand directly on a touchable display surface, a simple touch movement could be interpreted as either a mouse-over or a drag, depending on whether the left mouse button is intended to be depressed at the time. But how does one convey the state of the left mouse button with a single touch? And how does one fluidly switchbetween states? The problem is confounded by the lack of precision input when using a single finger as the mouse cursor, since a finger has a much larger footprint than a single pixel cursor hotspot. In this paper we introduce our solution, Fluid DTMouse, which has been used to improve the usability of touch tables with legacy (mouse-based) applications. Our technique is applicable to any direct-touch input device that can detect multiple points of contact. Our solution solves problems of smoothly specifying and switching between modes, addressing issues with the stability of the cursor, and facilitating precision input. Advanced Visual Interfaces (AVI) This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c Mitsubishi Electric Research Laboratories, Inc., 2006 201 Broadway, Cambridge, Massachusetts 02139 Fluid DTMouse: Better Mouse Support for Touch­Based Interactions Alan Esenther, Kathy Ryall Mitsubishi Electric Research Laboratories 201 Broadway Cambridge, MA 02139 +1­617­621­7564 {esenther, ryall}@merl.com ABSTRACT 1. INTRODUCTION Although computer mice have evolved physically (i.e., new form Since its introduction almost forty years ago, the mouse concept factors, multiple buttons, scroll­wheels), their basic metaphor has been and continues to be a critical part of computer remains the same: a single­point of interaction, with modifiers interfaces. It is arguably the most ubiquitous part of any GUI. used to control the interaction. Many of today's novel input Mice have evolved, now having multiple buttons, scroll­wheels, devices, however, do not directly (or easily) map to mouse and other sensing devices. However, even as today's computing interactions. For example, when using one's finger(s) or hand moves off the desktop and into the environment, the concept of a directly on a touchable display surface, a simple touch movement mouse remains largely the same: a single­point of interaction, could be interpreted as either a mouse­over or a drag, depending with modifiers used to control the interaction. on whether the left mouse button is intended to be depressed at Newer touch­sensitive surfaces such as horizontal tabletops or the time. But how does one convey the state of the left mouse vertical whiteboards can allow users to interact with displayed button with a single touch? And how does one fluidly switch content more naturally and directly than is possible with a between states? The problem is confounded by the lack of computer mouse. Although some can detect simultaneous points precision input when using a single finger as the mouse cursor, of input, most applications, new and existing, are generally since a finger has a much larger "footprint" than a single pixel written assuming that a mouse is being used. While we note that cursor hotspot. In this paper we introduce our solution, Fluid new interaction paradigms are evolving and that we are actively DTMouse, which has been used to improve the usability of touch researching in this very area, we must recognize the importance tables with legacy (mouse­based) applications. Our technique is of providing support for legacy applications in both today’s and applicable to any direct­touch input device that can detect future systems. Such support requires mouse­based input. multiple points of contact. Our solution solves problems of smoothly specifying and switching between modes, addressing One of the challenges of supporting such touch­sensitive surfaces issues with the stability of the cursor, and facilitating precision is to provide a natural mechanism for emulating mouse buttons input. on a surface that only detects touches. This is particularly problematic with “direct­touch” surfaces where the input and Categories and Subject Descriptors output are the same. For direct­input devices, the user is touching H.5.2 [User Interfaces]: Interaction styles (e.g., commands, the actual image of the content, rather than manipulating an menus, forms, direct manipulation). indirect input device such as a mouse or a touchpad off to the side. In these touch­based systems, there is a difficulty in General Terms interpreting a single­touch when used as a replacement for Design, Human Factors. mouse­input; a simple touch movement could be interpreted as either a mouse­over or a drag, depending on whether the left Keywords mouse button is intended to be depressed at the time. But how visual interaction, tabletop interfaces, mouse emulation, multi­ does one convey the state of the left mouse button with a single touch. touch? And how does one fluidly switch between states? For example, users may have to right­click to open a context menu, drag with the left mouse button to re­position content, double­ click to engage a button, or may simply wish to reposition the mouse without any additional input being generated. Permission to make digital or hard copies of all or part of this work for In addition to the mouse button problem, there is a second, personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies confounding problem – the lack of precision input when using a bear this notice and the full citation on the first page. To copy otherwise, or single finger as the mouse cursor. A finger has a much larger republish, to post on servers or to redistribute to lists, requires prior specific "footprint" than a single pixel cursor hotspot, making it permission and/or a fee. awkward, difficult, and at times impossible to interact with GUI AVI '06, May 23­26, 2006, Venezia, Italy. elements designed for mouse interaction. Copyright 2006 ACM 1­59593­353­0/06/0005...$5.00. In this paper we introduce Fluid DTMouse, our solution to the Tablet PCs [6] are also direct­input devices. They rely on a finger­based, direct­touch mouse problem; it has significantly special pen. These devices have the unusual ability to detect enabled the use of touch tables with legacy (mouse­based) “hovering” when the pen is near the surface but not actually in applications. While we have developed our techniques using the contact with the surface. So if the pen is hovering, then the DiamondTouch platform, they are applicable for any direct­touch mouse is simply moved, and if the pen is in contact with the input device that can detect multiple points of contact (e.g., [2] Tablet PC surface then dragging (left­mouse button down) is [11]) or future technologies with this capability. Our solution engaged. Right­clicking is supported by holding a button on the solves problems of smoothly specifying and switching between Tablet PC pen, or holding the stylus down briefly, or by selecting modes, addressing issues with the stability of the cursor, and a “right­click” menu item to indicate that the next tap should be facilitating precision input. Section 2 introduces the mouse interpreted as a right­click. It is the lack of this third state, emulation problems and discusses related work. Section 3 hovering (the first two states being touching or not touching), describes the Fluid DTMouse work and explains how it solves which makes mouse­over support so difficult on most touch key problems. Section 4 wraps up and discusses future and surfaces. For our research we were interested in solutions that ongoing work. would work with fingers, since requiring the use of a special pen or other hardware may not be as convenient – particularly for a 2. BACKGROUND & RELATED WORK large touch surface. It should be noted, however, that another One of the more fundamental challenges for direct­touch input is advantage of using a fine­tipped pen such as the Tablet PC pen is that users may wish to move a mouse cursor across a touch­ that the fine tip does not obscure content as much as a finger sensitive display without engaging any mouse buttons (e.g., does. At any rate, currently Tablet PCs are only available with simply move the mouse over a tooltip). But when a user touches very small surface areas, compared to tabletop and whiteboard a touch­sensitive surface, how is the system to know whether the surfaces. touch was intended to simply move the mouse, or to “drag” with The DiamondTouch [1] “DiamondTouch Mouse” [7] software the mouse by holding down the left mouse button during the supports a right­click by tapping with a second finger.
Load more

Recommended publications
  • Coloriync J43 Chooser {Antral Strip Limltruli Panels; Date 3; ‘Time I»
    US 20120174031Al (19) United States (12) Patent Application Publication (10) Pub. N0.: US 2012/0174031 A1 DONDURUR et a]. (43) Pub. Date: Jul. 5, 2012 (54) CLICKLESS GRAPHICAL USER INTERFACE (52) US. Cl. ...................................................... .. 715/808 (75) Inventors: MEHMET DONDURUR, (57) ABSTRACT DHAHRAN (SA); AHMET Z. SAHIN’ DH AHRAN (SA) The chckless graphical user mterface provldes a pop-up Wm doW When a cursor is moved over designated areas on the (73) Assigneez KING FAHD UNIVERSITY OF screen. The pop-up WindoW includes menu item choices, e. g., PETROLEUM AND MINERALS “double click”, “single click”, “close” that execute ‘When the DH AHRAN (SA) ’ cursor is moved over the item. This procedure eliminates the traditional ‘mouse click’, thereby allowing users to move the cursor over the a lication or ?le and 0 en it b choosin (21) Appl' NO" 12/985’165 among the aforerrrifntioned choices in the? ?le or Zipplicatior'i . bein focused on. The 0 -u WindoW shoWs the navi ation (22) Flled: Jan‘ 5’ 2011 choiges in the form of 1; tgxt,pe.g., yes/no or color, egg, red/ _ _ _ _ blue, or character, such as triangle for ‘yes’ and square for Pubhcatlon Classl?catlon ‘no’. Pop-up WindoW indicator types are virtually unlimited (51) Int, Cl, and canbe changed to any text, color or character. The method G06F 3/048 (200601) is compatible With touch pads and mouses. .5 10a About This Cam pater Appearance 5;‘ Fort Menu ?ptinns System Pm?ler Talk Calculatar Coloriync J43 Chooser {antral Strip limltruli Panels; Date 3; ‘time I» f Favmjltes } Extensions Manager B "Dali?i ?' File Exchange Q Key Caps File Sharing Patent Application Publication Jul.
    [Show full text]
  • Creating Document Hyperlinks Via Rich-Text Dialogs
    __________________________________________________________________________________________________ Creating Document Hyperlinks via Rich-Text Dialogs EXAMPLE: CREATING A HYPERLINK IN EXCEPTIONS/REQUIREMENTS: Enter the appropriate paragraph either by selecting from the Lookup Table or manually typing in the field, completing any prompts as necessary. Locate and access the item to be hyperlinked on the external website (ie through the County Recorder Office): Copy the address (typically clicking at the end of the address will highlight the entire address) by right click/copy or Control C: __________________________________________________________________________________________________ Page 1 of 8 Creating Hyperlinks (Requirements/Exceptions and SoftPro Live) __________________________________________________________________________________________________ In Select, open the Requirement or Exception, highlight the text to hyperlink. Click the Add a Hyperlink icon: __________________________________________________________________________________________________ Page 2 of 8 Creating Hyperlinks (Requirements/Exceptions and SoftPro Live) __________________________________________________________________________________________________ The Add Hyperlink dialog box will open. Paste the link into the Address field (right click/paste or Control V) NOTE: The Text to display (name) field will autopopulate with the text that was highlighted. Click OK. The text will now be underlined indicating a hyperlink. __________________________________________________________________________________________________
    [Show full text]
  • Predicting Search Task Difficulty
    Predicting Search Task Difficulty Jaime Arguello University of North Carolina at Chapel Hill [email protected] Abstract. Search task difficulty refers to a user's assessment about the amount of effort required to complete a search task. Our goal in this work is to learn predictive models of search task difficulty. We evaluate features derived from the user's interaction with the search engine as well as features derived from the user's level of interest in the task and level of prior knowledge in the task domain. In addition to user-interaction fea- tures used in prior work, we evaluate features generated from scroll and mouse-movement events on the SERP. In some situations, we may prefer a system that can predict search task difficulty early in the search session. To this end, we evaluate features in terms of whole-session evidence and first-round evidence, which excludes all interactions starting with the second query. Our results found that the most predictive features were different for whole-session vs. first-round prediction, that mouseover fea- tures were effective for first-round prediction, and that level of interest and prior knowledge features did not improve performance. 1 Introduction Search engine users engage in a wide variety of search tasks. A large body of prior research focused on characterizing different types of search tasks (see Li and Belkin [10]). The motivation behind this prior work is to understand how task characteristics influence search behavior and how search systems can provide customized interactions for different task types. One important search task characteristic is search task difficulty.
    [Show full text]
  • Welcome to Computer Basics
    Computer Basics Instructor's Guide 1 COMPUTER BASICS To the Instructor Because of time constraints and an understanding that the trainees will probably come to the course with widely varying skills levels, the focus of this component is only on the basics. Hence, the course begins with instruction on computer components and peripheral devices, and restricts further instruction to the three most widely used software areas: the windows operating system, word processing and using the Internet. The course uses lectures, interactive activities, and exercises at the computer to assure accomplishment of stated goals and objectives. Because of the complexity of the computer and the initial fear experienced by so many, instructor dedication and patience are vital to the success of the trainee in this course. It is expected that many of the trainees will begin at “ground zero,” but all should have developed a certain level of proficiency in using the computer, by the end of the course. 2 COMPUTER BASICS Overview Computers have become an essential part of today's workplace. Employees must know computer basics to accomplish their daily tasks. This mini course was developed with the beginner in mind and is designed to provide WTP trainees with basic knowledge of computer hardware, some software applications, basic knowledge of how a computer works, and to give them hands-on experience in its use. The course is designed to “answer such basic questions as what personal computers are and what they can do,” and to assist WTP trainees in mastering the basics. The PC Novice dictionary defines a computer as a machine that accepts input, processes it according to specified rules, and produces output.
    [Show full text]
  • Fuzzy Mouse Cursor Control System for Computer Users with Spinal Cord Injuries
    Georgia State University ScholarWorks @ Georgia State University Computer Science Theses Department of Computer Science 8-8-2006 Fuzzy Mouse Cursor Control System for Computer Users with Spinal Cord Injuries Tihomir Surdilovic Follow this and additional works at: https://scholarworks.gsu.edu/cs_theses Part of the Computer Sciences Commons Recommended Citation Surdilovic, Tihomir, "Fuzzy Mouse Cursor Control System for Computer Users with Spinal Cord Injuries." Thesis, Georgia State University, 2006. https://scholarworks.gsu.edu/cs_theses/49 This Thesis is brought to you for free and open access by the Department of Computer Science at ScholarWorks @ Georgia State University. It has been accepted for inclusion in Computer Science Theses by an authorized administrator of ScholarWorks @ Georgia State University. For more information, please contact [email protected]. i Fuzzy Mouse Cursor Control System For Computer Users with Spinal Cord Injuries A Thesis Presented in Partial Fulfillment of Requirements for the Degree of Master of Science in the College of Arts and Sciences Georgia State University 2005 by Tihomir Surdilovic Committee: ____________________________________ Dr. Yan-Qing Zhang, Chair ____________________________________ Dr. Rajshekhar Sunderraman, Member ____________________________________ Dr. Michael Weeks, Member ____________________________________ Dr. Yi Pan, Department Chair Date July 21st 2005 ii Abstract People with severe motor-impairments due to Spinal Cord Injury (SCI) or Spinal Cord Dysfunction (SCD), often experience difficulty with accurate and efficient control of pointing devices (Keates et al., 02). Usually this leads to their limited integration to society as well as limited unassisted control over the environment. The questions “How can someone with severe motor-impairments perform mouse pointer control as accurately and efficiently as an able-bodied person?” and “How can these interactions be advanced through use of Computational Intelligence (CI)?” are the driving forces behind the research described in this paper.
    [Show full text]
  • The Fancytooltips Package∗†
    The fancytooltips package∗y Robert Maˇr´ık [email protected] May 4, 2009 1 Introduction The package fancytooltips is a package for LATEX. The pdf can be created by pdflatex or by latex + dvips + AdobeDistiller1 + Adobe Acrobat2. It allows to create tooltips in a similar way like cooltooltips package, but the tooltip is a page from another PDF file. In this way you can use mathematics, pictures and animations in your tooltips. The resulting PDF file can be used also with free Abobe Reader. The tooltips are activated by clicking the active area on the screen and deac- tivated after closing page or by moving mouse outside the link. You can try the links here (Einstein's formula) and also here (animation { numbers from 1 to 6). You have to use the free Adobe Reader or nonfree Adobe Acrobat to see the effect (xpdf, evince and others fail to work with JavaScripts). For more examples how the presentation may look like see the example.pdf and example-min.pdf files in the examples subdirectory. The buttons are created using eforms.sty which is a part of AcroTeX bundle. 2 Usage 2.1 The file with tooltips The file with tooltips is an ordinary pdf file, one tooltip per page, tooltips should be in the top right corner at the page, in a colored box and the rest of the page should be transparent. If you consider to use movetips option (see below), then every page should have the dimensions equal to the dimen- sions of the colored box with tooltip3.
    [Show full text]
  • Microsoft Excel 2013: Mouse Pointers & Cursor Movements
    Microsoft Excel 2013: Mouse Pointers & Cursor Movements As you move the mouse over the Excel window it changes shape to indicate the availability of different functions. The five main shapes are shown in the diagram below. General pointer for selecting cells singly or in groups Pointer used at bottom right of selection to extend and fill data. Selected cells are shown by means of a heavy border as shown. The extension point is at the bottom right of the border and is a detached square. Insertion point. When pointer is like this you may type in text in this area. You must double click the left mouse button to move the cursor (a flashing vertical line) into the cell area. Insertion and editing can then be done in the normal way. Pointer for menus or moving a selection. When Copying a selection a small cross appears Used where you can change the dimensions of a Row or column. This pointer indicates that you can drag a boundary in the direction of the arrows. 1 What do the different mouse pointer shapes mean in Microsoft Excel? The mouse pointer changes shape in Microsoft Excel depending upon the context. The six shapes are as follows: Used for selecting cells The I-beam which indicates the cursor position when editing a cell entry. The fill handle. Used for copying formula or extending a data series. To select cells on the worksheet. Selects whole row/column when positioned on the number/letter heading label. At borders of column headings. Drag to widen a column.
    [Show full text]
  • Visualizing User Interface Events Event Stream Summarization Through Signs
    Visualizing User Interface Events Event Stream Summarization through Signs Vagner Figueredo de Santana and Maria Cecília Calani Baranauskas Institute of Computing, University of Campinas (UNICAMP), Albert Einstein Av., 1251, Campinas, Brazil Keywords: Semiotics, User Interface Events, Visualization of Usage Data, Log Visualization, Event Icons, Icon Library, Image Library, User Interface Evaluation, Usability Evaluation, Accessibility Evaluation, Web Usage Mining. Abstract: Effective visual representation is related to how people interpret signs created to carry specific information. In the last years many user interface evaluation tools are considering detailed usage data to represent users’ actions. The volume of data gathered is leading developers to represent usage in a summarized way through graphical representations. If visual components used to represent complex data are not effective, then graphics used to summarize data may turn the interpretation of complex terms even harder. This work presents a study about graphical representations for user interface (UI) events and contributes with the validation of usage graph visualization and an open set of signs to support the summarization of client-side logs. The study involved 28 Information Technology specialists, potential users of UI evaluation tools. From the results one expects that evaluation tool developers, evaluators, and Web usage miners can reuse the validated usage graph representation and proposed set of signs to represent usage data in a summarized way. 1 INTRODUCTION these events is called, respectively, client-side logs and server-side logs (Santana and Baranauskas, The evaluation of user interface (UI) is a key task 2010a). when developing information systems and is part of In the last decade, website evaluation tools using a number of Software Engineering development server-side data (i.e., based on Web server logs) processes.
    [Show full text]
  • Vista Wait Cursor Gif
    Vista Wait Cursor Gif 1 / 4 Vista Wait Cursor Gif 2 / 4 3 / 4 С Vista Transformation Pack можете да промените изгледа на Windows XP и да го направите като Windows Vista. Пакета ... vista transformation pack .gif С Vista ... Updated Vista Rainbar's launcher for constant waiting cursor activity issue. On October 31, 2013, claimed that the 's counterpart, a, was more recognized than Windows Vista's wait cursor due to the cultural impact that .... So the blue circle animation that pops up next to the mouse pointer that shows ... and closing running programs in the tray, and the waiting animation ... a problem by performing a clean boot in Windows Vista or in Windows 7.. In your jQuery use: $("body").css("cursor", "progress");. and then back to normal again $("body").css("cursor", "default");.. N. ▻ Non- animated throbbers‎ (1 C, 3 F) ... Ajax loader metal 512.gif 512 × 512; 45 KB ... Cursor Windows Vista.gif 32 × 32; 9 KB ... Wait.gif 40 × 40; 4 KB.. Download GIF Viewer for free. Windows 7/8/10-compatible animated .gif player. A C# program used to visualize and extract frames from .GIF files. (you need .. In this topic you'll learn more about static and animated cursors. ... in different context (Normal Select, Help Select, Working in background, Busy. ... colors, smooth transparency (alpha channel) - Supported by Windows XP, Vista and superior only ... which can be saved using various file formats (BMP, PSD, GIF, JPEG, WMF.. Windows Vista's "busy" cursor? Reply ... This always makes me forget my troubles. http://upload.wikimedia.org/wikipedia/en/3/3d/WaitCursor-300p.gif.
    [Show full text]
  • Clickjacking: Attacks and Defenses
    Clickjacking: Attacks and Defenses Lin-Shung Huang Alex Moshchuk Helen J. Wang Carnegie Mellon University Microsoft Research Microsoft Research [email protected] [email protected] [email protected] Stuart Schechter Collin Jackson Microsoft Research Carnegie Mellon University [email protected] [email protected] Abstract such as a “claim your free iPad” button. Hence, when Clickjacking attacks are an emerging threat on the web. the user “claims” a free iPad, a story appears in the user’s In this paper, we design new clickjacking attack variants Facebook friends’ news feed stating that she “likes” the using existing techniques and demonstrate that existing attacker web site. For ease of exposition, our description clickjacking defenses are insufficient. Our attacks show will be in the context of web browsers. Nevertheless, the that clickjacking can cause severe damages, including concepts and techniques described are generally applica- compromising a user’s private webcam, email or other ble to all client operating systems where display is shared private data, and web surfing anonymity. by mutually distrusting principals. We observe the root cause of clickjacking is that an Several clickjacking defenses have been proposed and attacker application presents a sensitive UI element of a deployed for web browsers, but all have shortcomings. target application out of context to a user (such as hiding Today’s most widely deployed defenses rely on frame- the sensitive UI by making it transparent), and hence the busting [21, 37], which disallows a sensitive page from user is tricked to act out of context. To address this root being framed (i.e., embedded within another web page).
    [Show full text]
  • Pen Computer Technology
    Pen Computer Technology Educates the reader about the technologies involved in a pen computer Fujitsu PC Corporation www.fujitsupc.com For more information: [email protected] © 2002 Fujitsu PC Corporation. All rights reserved. This paper is intended to educate the reader about the technologies involved in a pen computer. After reading this paper, the reader should be better equipped to make intelligent purchasing decisions about pen computers. Types of Pen Computers In this white paper, "pen computer" refers to a portable computer that supports a pen as a user interface device, and whose LCD screen measures at least six inches diagonally. This product definition encompasses five generally recognized categories of standard products, listed in Table 1 below. PRODUCT TARGET PC USER STORAGE OPERATING RUNS LOCAL EXAMPLE CATEGORY MARKET INTERFACE SYSTEM PROGRAMS Webpad Consumer & No Standard Flash Windows CE, Only via Honeywell Enterprise browser memory Linux, QNX browser WebPAD II plug-ins CE Tablet Enterprise No Specialized Flash Windows CE Yes Fujitsu applications memory PenCentra Pen Tablet Enterprise Yes Windows & Hard drive Windows 9x, Yes Fujitsu specialized NT-4, 2000, Stylistic applications XP Pen-Enabled Consumer Yes Windows Hard drive Windows 9x, Yes Fujitsu & Enterprise 2000, XP LifeBook B Series Tablet PC Consumer Yes Windows Hard drive Windows XP Yes Many under & Enterprise Tablet PC development Edition Table 1: Categories of Pen Computers with LCD Displays of Six Inches or Larger Since the different types of pen computers are often confused, the following paragraphs are intended to help explain the key distinguishing characteristics of each product category. Pen Computers Contrasted Webpad: A Webpad's primary characteristic is that its only user interface is a Web browser.
    [Show full text]
  • (12) Patent Application Publication (10) Pub. No.: US 2007/0211071 A1 Slotznick Et Al
    US 20070211071A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2007/0211071 A1 Slotznick et al. (43) Pub. Date: Sep. 13, 2007 (54) METHOD AND APPARATUS FOR (52) U.S. Cl. .............................................................. 34.5/594 INTERACTING WITH A VISUALLY DISPLAYED DOCUMENT ON A SCREEN (57) ABSTRACT READER User interaction of a visually displayed document is pro vided via a graphical user interface (GUI). The document (76) Inventors: Benjamin Slotznick, Mt. Gretna, PA includes, and is parsed into, a plurality of text-based gram (US); Stephen C. Sheetz, Lititz, PA matical units. An input device modality is selected from a (US) plurality of input device modalities which determines the Correspondence Address: type of input device in which a user interacts with to make AKN GUMPSTRAUSS HAUER & FELD a selection. One or more grammatical units of the document L.L.P. are then selected using the selected type of input device. ONE COMMERCE SQUARE Each grammatical unit that is selected is read aloud to the 2005 MARKET STREET, SUITE 2200 user by loading the grammatical unit into a text-to-speech PHILADELPHIA, PA 19103 (US) engine. The text of the grammatical unit is thereby auto matically spoken. Furthermore, a Switching modality is (21) Appl. No.: 11/642,247 selected from a plurality of switching modalities. The (22) Filed: Dec. 20, 2006 Switching modality determines the manner in which one or more Switches are used to make a selection. Using the Related U.S. Application Data selected Switching modality, a user steps through at least Some of the grammatical units in an ordered manner by (60) Provisional application No.
    [Show full text]