United States Patent (19) 11 Patent Number: 4,821,029 Logan et al. (45) Date of Patent: Apr. 11, 1989 54 TOUCH SCREEN COMPUTER-OPERATED 4,475,239 10/1984 van Raamsdonk ...... - - - - - - - 340/707 VIDEO DISPLAY PROCESS AND 4,476,463 10/1984 Ng et al. .................. 340/712 APPARATUS 4,524,421 6/1985 Searby et al. ... 340/706 4,550,221 10/1985 Mabusth ........... ... 340/706 (75) Inventors: James D. Logan, Arlington; Yury 4,621,257 11/1986 Brown ...... ... 340/712 Litvin, Bedford, both of Mass. 4,656,603 4/1987 Dunn ................................... 340/721 73) Assignees: Microtouch Systems, Inc., Middlesex; OTHER PUBLICATIONS The Academy of Applied Science, PC Magazine, "The Tale of the Mouse', vol. 1, No. 10, Boston, both of Mass.; a part interest Feb. 1983, pp. 66–71. (21) Appl. No.: 604,319 Primary Examiner-John W. Caldwell, Sr. (22) Filed: Apr. 26, 1984 Assistant Examiner-Jeffery A. Brier (51) Int. Cl." ............................................... G09G 1/00 Attorney, Agent, or Firm-Rines and Rines; Shapiro and (52) U.S. C. .................................... 340/712; 340/706; Shapiro 340/721; 178/18 57 ABSTRACT (58) Field of Search ............... 340/789, 706, 707, 708, The invention provides a novel computer-operated 340/709, 710, 711, 712, 716, 721, 724; 178/18; touch screen video display system in which the human 273/148 B, DIG. 28 operator by first touching a sub-area or chamber of the 56) References Cited screen containing predetermined graphic information U.S. PATENT DOCUMENTS and then pointing on the screen to a different sub-area 3,189,889 6/1965 Bridgett .............................. 340/707 or chamber where it is desired to display said informa 3,482,241 12/1969 Johnson .......... ... 340/337 tion may cause the processing equipment automatically 3,757,322 9/1973 Barkan et al. ....................... 340/365 to effect the transfer of said graphic information to and 3,911,215 10/1975 Hurst et al. ..... ... 178/18 display of the same at the different sub-area, while also 4,220,815 9/1980 Gibson et al. ... ... 178/18 storing the information in the memory of the processing 4,310,839 1/1982 Schwerdt ... 340/712 equipment. 4,374,381 2/1983 Ng et al. ..... 340/711 4,451,895 5/1984 Sliwkowski ......................... 340/707 10 Claims, 2 Drawing Sheets MEMORY (ROMIRAM & AUXILLIARY) U.S. Patent Apr. 11, 1989 Sheet 1 of 2 4,821,029 s U.S. Patent Apr. 11, 1989 Sheet 2 of 2 4,821,029 RESEAL DATATO A/62 NORMAL STATE RECOGNIZETOUCH (LOCATION OF SUBAREA ORCHAMBER) DISPLAY ERROR MESSAGE (OPTIONAL) SANY DATA ENACTIVE STATE 2 IS SUB-AREA ORCHAMBER EMPTY? MOVEDAANSUB. AREA ORCHAMBERTO ACTIVESTATE SAVELOCATION OF SUB-AREA OR CHAMBER WITH ACTIVE STATE DATA DISPLAYWDEOFEED BACKOF ACTIVE STATE DAAN SUB AREA ORCHAMBER ISSUB-AREA ORCHAMBER EMPTY RESEVIDEOFEEDBACK OF ACTIVE DAATO NORMAL DISPLAY TRANSFERICOPYDATA NACTIVE DATA SUBAREA ORCHAMBER FROMSAVED CHAMBER LOCATION TO PRESENTLY DESIGNATEDSUB-AREA ORCHAMBER 4,821,029 1. 2 In present-day touch screen systems before refer TOUCHSCREEN COMPUTER-OPERATED WIDEO enced, the art has taken the natural course of, in effect, DISPLAY PROCESS AND APPARATUS transferring the function keyboard or keys to the desig nated portions of the touch screen display, but still The present invention relates to computer operated requiring the human operator to understand what is video display systems and processes, being more partic involved in operating these various function keys. This ularly concerned with novel techniques and apparatus technique only transfers the external function keys in for enabling the user to transfer graphic display infor the keyboard to representations of the keys displayed on mation and data to different sub areas, chambers or the graphic display. regions of the display by user-friendly touch techniques 10 What is believed to be one of the break-through fac operable upon the touch screen of the display system. tors of the present invention is the freeing of the opera The basic concept and routines involving touch dis tor from this task of understanding and operating func plays have been well developed in recent years, stem tion keys whether remote from the display or presented ming from early concepts involving pluralities of touch upon the display; and through the novel concept under sensitive contacts adjacent the screen of cathode-ray 15 lying the invention of actually using the touch screen as tube display systems used in data processing and related the self-sufficient control mechanism, obviating the operations (as described, for example, in U.S. Pat. No. necessity for function keys (whether remote or repre 3,482,241) to more sophisticated resistive touch-screen sented on the display) or the knowledge by the operator displays such as those described in U.S. Pat. Nos. of the use of function keys. 3,911,215, 3,757,322, 4,220,815 and 4,374,381. In the 20 The present invention therefore provides a particu present art of transferring data provided on a screen larly user-friendly environment that allows an operator surface, two primary methods are used. The first in to visually, tactilely and graphically designate a desired volves the use of a keyboard or other remote non section of displayed data and to perform automatic graphic device to block or separate a section of data and display and storage operations upon that data without subsequently to perform a transfer, copy or storage 25 having had specific training, and thus removes the asso operation. The difficulty inherent in such a remote sys ciated complexities with learning and operating param tem includes the loss of simplicity between coordinating eters and other non-data distinguishing parameters that key strokes to data output and a lengthy menu type list are not a direct subject of the provided data. of operating parameters that must be used. An object of the present invention, accordingly, is to The second common method for block manipulation 30 provide a new and improved touch screen computer of output data on a screen involves the use of a pointer operated video display and processing system which system such as that commonly referred to as a "mouse', completely obviates the necessity for function keys and as described, for example, in an article entitled "The knowledge of their operation and for external indica Tail of the Mouse', PC Magazine, Vol. 1, No. 10, Feb tors, such as the before-mentioned "mouse', and pro ruary, 1983, p. 66, on. In such a system, a cursor or 35 vides a new dimension in user-friendly function of the other graphic symbol is remotely located to an area touch screen itself wherein the operator, by pointing to designating a portion of the output screen, providing a the desired sub-area where graphic data earlier pointed begin-block parameter. The “mouse' is then moved to to as displayed elsewhere is to be transferred, can auto the end of the data block to be manipulated, and re matically cause this to happen merely by touch of the motely an end-block parameter is located upon the 40 screen at the appropriate subareas. screen. Additionally, remotely, as by a keyboard de The invention thus provides a "seeing and pointing' vice, the data is then transferred, copied or stored as revolutionary and new visual interface for software, noted above. The "mouse' operative systems also allow allowing direct interaction with personal and other for the designation of certain predefined blocks of out computers by simply touching the screen, providing put data to allow block transfers to occur. 45 easier cursor control, menu selection and command With both of the aforementioned methods, remote input without cumbersome key stroke combinations to designation of the visually produced output of data remember or cursor keys to toggle. In addition, the must be achieved by external keyboards and related present invention provides faster operation than indi mechanisms. cated in systems such as the "mouse', or other pointing Underlying the present invention is the breaking with 50 devices, because there is no remote or intermediate the convention previously considered necessary in the device to grasp, orient or manipulate-the user getting art of employing external or remote mechanisms for directly to the desired part of the screen with a single, triggering and initiating the transfer of graphically pres efficient touch motion, and without training. Further ented information from one sub-area of the screen to features reside in the versatility that any type of graphic another desired sub-area, and providing a novel ap 55 display information, icons, words and characters can be proach that enables the human operator to employ only accurately pointed to with the finger; and, of course, elements of the touch screen itself, cooperative with graphic display buttons, dials or switches or the like on associated processing, to enable such transfer. This is the screen may be manipulated like real objects with no effected, moreover, with the very desirable human en moving parts to go wrong or rolling mechanisms to gineering factor that the user may visually locate and 60 stick or fragile components to fail. merely touch a sub-area or region of the display where A further object is to provide a vastly more flexible the user desires the graphic information to be displayed, touch screen process and system that incorporates, to initially identifying such information by touching first gether with the appropriate processing, a self-suffi the sub-area where it is being displayed; and that this ciency of control to the touch screen never before at human touching operation will initiate and control the 65 tained transfer to the desired sub-area of the graphic informa Other and further objects will be explained hereinaf tion then displayed elsewhere -all merely by pointing ter and are more particularly delineated in the appended on the touch screen surface alone.