USOO8902181B2
(12) United States Patent (10) Patent No.: US 8,902,181 B2 Hinckley et al. (45) Date of Patent: Dec. 2, 2014
(54) MULTI-TOUCH-MOVEMENT GESTURES 7,400.316 B2 7/2008 Appleyard et al. FORTABLET COMPUTING DEVICES 7,499,024 B2 3/2009 Johnston, Jr. et al. 7,532,196 B2 5/2009 Hinckley 7,567,242 B2 7/2009 Perkins et al. (75) Inventors: Kenneth P. Hinckley, Redmond, WA 7.703,047 B2 4/2010 Keely et al. (US); Michel Pahud, Kirkland, WA 7,812,826 B2 10/2010 Ording et al. (US); William A. S. Buxton, Toronto (Continued) (CA) (73) Assignee: Microsoft Corporation, Redmond, WA FOREIGN PATENT DOCUMENTS (US) CN 101809581 A 8, 2010 KR 1020010041283. A 5, 2001 (*) Notice: Subject to any disclaimer, the term of this KR 10201 20005417 A1 1, 2012 patent is extended or adjusted under 35 WO 2009084809 A1 T 2009
(21) Appl. No.: 13/367,377 Hinckley, et al., “Sensor Synaesthesia: Touch in Motion, and Motion in Touch.” retrieved at
TUCH MOWMENT OTHER CNTEXT INPUT INPUT INPUT INFORMATION NFORMATION INFORMATION INFORMATION
------GESTURE SIGNATURE(s)
BEHAWOR EXECUTING MODULE 208 US 8,902,181 B2 Page 2
(56) References Cited 2010/0235729 A1 9/2010 Kocienda et al. 2010/0281435 A1 11/2010 Bangalore et al. U.S. PATENT DOCUMENTS 2010/0295781 A1 11/2010 Alameh et al. 2010/0295799 A1 11/2010 Nicholson et al. 7,812,828 B2 10, 2010 Westerman et al. 2010/0298.033 A1* 11/2010 Lee ...... 455,566 7,847,789 B2 12, 2010 Kolmykov-Zotov et al. 2010/0306670 Al 12/2010 Quinn et al. 7.956,847 B2 6, 2011 Christie 2010/03282.27 A1 12/2010 Matejka et al. 7,982,739 B2 T/2011 Pasula 2011 0115741 A1 5/2011 Lukas et al. 8,228,292 B1 T/2012 Ruiz et al. 2011/O163955 A1 7/2011 Nasiri et al. 8,265,705 B2 9, 2012 Lee 2011/O167357 A1 7/2011 Benjamin et al. 8.413,077 B2 4, 2013 Yamamoto et al. 2011/O187651 A1 8, 2011 Whitlow et al. 2003/0076310 A1 4, 2003 Kanzaki et al. 2011 0193788 A1 8/2011 King et al. 2003. O164821 A1 9, 2003 Rezania 2011/O1971.53 A1 8/2011 King et al. 2004/OO12575 A1 1, 2004 Homer et al. 2011/0221777 A1 9, 2011 Ke 2004.?004.7505 A1 3, 2004 Ghassabian 2011/0231796 A1 9/2011 Vigil 2004/O140962 A1 T/2004 Wang et al. 2011/02391.1.0 A1 9, 2011 Garrett et al. 2004/O2O3520 A1 10, 2004 SchirtZinger et al. 2012/0092268 A1 4/2012 Tsai et al. 2005.0024346 A1 2, 2005 Dupraz et al. 2012/00922.69 A1 4/2012 Tsai et al. 2005, 00798.96 A1 4, 2005 Kokko et al. 2012. O154293 A1 6/2012 Hinckley et al. 2005/01796.48 A1 8, 2005 Barabe et al. 2012. O154294 A1 6/2012 Hinckley et al. 2005/0216867 A1 9, 2005 Marvit et al. 2012. O154295 A1 6/2012 Hinckley et al. 2005/0253817 A1 11/2005 Rytivaara et al. 2012. O154296 A1 6/2012 Hinckley et al. 2006, OO12580 A1 1, 2006 Perski et al. 2012/0158629 A1 6/2012 Hinckley et al. 2006, OO26535 A1 2, 2006 Hotelling et al. 2012/0206330 A1 8, 2012 Cao et al. 2006.0109252 A1 5, 2006 Kolmykov-Zotov et al. 2012/0260220 A1* 10/2012 Griffin ...... T15,863 2006, O136840 A1 6, 2006 Keely et al. 2013/0335333 A1 12/2013 Kukulski et al. 2006, O146038 A1 T/2006 Park et al. 2006/0177112 A1 8, 2006 Yang et al. OTHER PUBLICATIONS 2006/O197750 A1 9, 2006 Kerr et al. 2006/O197753 A1 9, 2006 Hotelling Song, et al., “WYSIWYF: Exploring and Annotating Volume Data 2006/0256008 A1 11, 2006 Rosenberg with a Tangible Handheld Device.” retrieved at < (56) References Cited Izadi, et al., “C-Slate: A Multi-Touch and Object Recognition System for Remote Collaboration using Horizontal Surfaces'. Second OTHER PUBLICATIONS Annual IEEE International Workshop on Horizontal Interactive Human-Computer System, Oct. 2007, pp. 3-10. Hinckley, et al., U.S. Appl. No. 12/970,943, filed Dec. 17, 2010, Kendrick, "ChromeTouch: Free Extension for Touch Tables', entitled “Using Movement of a Computing Device to Enhance Inter GigaOM, May 6, 2010, 9 pages. pretation of Input Events Produced When Interacting With the Com Kurtenbach, et al., “Issues in Combining Marking and Direct puting Device. 68 pages. Manipulation Techniques'. In Proceedings of the 4th Annual ACM Hinckley, et al., U.S. Appl. No. 12/970,945, filed Dec. 17, 2010, Symposium on User Interface Software and Technology, Nov. 11, entitled “Detecting and Responding to Unintentional Contact With a 1991, 8 pages. Computing Device. 47 pages. Lester, et al., “Are You With Me?” Using Accelerometers to Deter Hinckley, et al., U.S. Appl. No. 12/970,949, filed Dec. 17, 2010, mine ifTwo Devices are Carried by the Same Person'. In Proceedings entitled “Cooperative Use of Plural Input Mechanisms to Convey of Second International Conference on Pervasive Computing, Apr. Gestures. 46 pages. 21, 2004, 18 pages. Hinckley, et al., U.S. Appl. No. 12/970,939, filed Dec. 17, 2010, Li, et al., “Experimental Analysis of Mode Switching Techniques in entitled “Detecting Gestures Involving Intentional Movement of a Pen-Based User Interfaces'. In Proceedings of the SIGCHI Confer Computing Device. 68 pages. ence on Human Factors in Computing Systems, Apr. 2, 2005, 10 Hinckley, et al., U.S. Appl. No. 12/970,955, filed Dec. 17, 2010, pageS. entitled “Supplementing a Touch Input Mechanism With Fingerprint Mohamed, et al., “Disoriented Pen-Gestures for Identifying Users Detection. 40 pages. Around the Tabletop Without Cameras and Motion Sensors'. Pro Hinckley, et al., U.S. Appl. No. 13/327,794, filed Dec. 16, 2011, ceedings of the First IEEE International Workshop on Horizontal entitled "Gesture Combining Multi-Touch and Movement.” 53 Interactive Human-Computer Systems (Tabletop '06), Jan. 2006, 8 pageS. pageS. Ramos, et al., “Tumble! Splat Helping Users Access and Manipulate Mulroy, “N-Trig Pushes Pen and Multitouch Input', PC World, Occluded Content in 2D Drawings,” retrieved at < (56) References Cited Essl., et al., “Use the Force (or something)—Pressure and Pressure Like Input for Mobile Music Performance.” retrieved at < (56) References Cited Ashbrook, et al., “MAGIC: A Motion Gesture Design Tool.” retrieved at < (56) References Cited Zhou, Hong, Notice of Allowance, U.S. Appl. No. 13/026,058, Jul. 17, 2014, pp. 1-5. OTHER PUBLICATIONS Treitler, Damon, U.S. Office Action, U.S. Appl. No. 13/327,794, Jul. 17, 2014, pp. 1-13. St. SES Figal Office Action, U.S. Appl. No. “First Office Action and Search Report Received in Chinese Patent Figueroa-Gibson, Gloryvid, U.S. Office Action, U.S. Appl. No. Application No. 2011 10427737.8, Mailed Date: Mar. 28, 2014, 12/970,939, Oct. 2, 2014, pp. 1-40. Filed Date: Feb. 7, 2012, 19 Pages. Figueroa-Gibson, Gloryvid, U.S. Office Action, U.S. Appl. No. 12/970,943, Sep. 17, 2014, pp. 1-20. * cited by examiner U.S. Patent Dec. 2, 2014 Sheet 1 of 14 US 8,902,181 B2 ------H - sa ANY APPLICATION REMOTE PROCESSING SOURCE(S) FUNCTIONALITY ANY APPLICATION(S) 116 INTERPRETATION AND BEHAVIOR SELECTION MODULE (IBSM) 114 INPUT INFORMATION DISPLAY MECHANISM 104 TOUCH MOVEMENT INPUT INPUT MECHANISM(S) MECHANISM(s) 108 110 • ACCELER OTHER INPUT OMETER(S) MECHANISM(S) • GYROMETER(S) 112 O ETC. INPUT MECHANISMS 106 FIG. 1 U.S. Patent Dec. 2, 2014 Sheet 2 of 14 US 8,902,181 B2 TOUCH MOVEMENT OTHER CONTEXT INPUT INPUT INPUT INFORMATION INFORMATION INFORMATION INFORMATION GESTURE MATCHING MODULE GESTURE SIGNATURE(S) BEHAVIOR EXECUTING MODULE 2O8 FIG. 2 U.S. Patent Dec. 2, 2014 Sheet 3 of 14 US 8,902,181 B2 LLUSTRATIVE FEATURES FOR USE INDEFINING A GESTURE MTM GESTURE CONTACT LOCATION ON OF CONTACTS ON SCREEN NUMBER OF CONTACTS TYPE OF CONTACT CONTACT ON A SURFACE OTHER THAN THE SCREEN MOVEMENT STATIC POSTURE KIND OF MOVEMENT (E.G., SPATIAL PATH) ROTATING (TILTING, FLIPPING, PIVOTING, TWISTING, ETC.) SLIDING (TRANSVERSE) SHAKING, VIBRATING, ETC. TAPPING DYNAMIC NATURE OF MOVEMENT MAGNITUDE OF MOVEMENT CONTEXT APPLICATION USER ENVIRONMENT FIG. 3 U.S. Patent Dec. 2, 2014 Sheet 4 of 14 US 8,902,181 B2 EXAMPLE I: TILTING 412 EXAMPLE I: 402 TILTING (SIDE 416 PROFILE) 406 404 FIG. 5 U.S. Patent Dec. 2, 2014 Sheet 5 of 14 US 8,902,181 B2 N N 408 EXAMPLE II: FLIPPING (AND ALSO TILTING) 402 EXAMPLE II: FLIPPING (SIDE PROFILE) 406 402 404 FIG. 7 U.S. Patent Dec. 2, 2014 Sheet 6 of 14 US 8,902,181 B2 EXAMPLE lll: 8O2 416 ? 4O2 PIVOTING 404 408 418 t MOVEMENTN-PLANE 4O6 804 4O2 e 408 416 404 418 406 FIG. 8 U.S. Patent Dec. 2, 2014 Sheet 7 of 14 US 8,902,181 B2 EXAMPLE IV: PIVOTING FIG. 9 U.S. Patent Dec. 2, 2014 Sheet 8 of 14 US 8,902,181 B2 408 EXAMPLE V: 402 TWISTING EXAMPLE VI: REAR CONTACT U.S. Patent Dec. 2, 2014 Sheet 9 of 14 US 8,902,181 B2 402 EXAMPLE VI: REAR CONTACT EXAMPLE VIII: BOOKLET s 1302 GESTURES FIG. 13 U.S. Patent Dec. 2, 2014 Sheet 10 of 14 US 8,902,181 B2 1302 EXAMPLE VIII: BOOKLET GESTURES (CONTINUED) 1306 EXAMPLE VIII: BOOKLET GESTURES (CONTINUED) U.S. Patent Dec. 2, 2014 Sheet 11 of 14 US 8,902,181 B2 ILLUSTRATIVE ACTIONS INVOKED BY A GESTURE ACTION CHANGE LAYOUT OF TEMS CHANGE INTERACTION EXPERIENCE CHANGE TOOLBAR SELECTION CHANGE CONTENT ORIENTATION PERFORM A ZOOMING AND/OR SCROLLING OPERATION ACTIVATE AN INTERFACE PAGE ACTIVATE AN APPLICATION, AN OPERATING SYSTEM FUNCTION, ETC. PERFORMACTION ON ADESIGNATED OBJECT PERFORMA SETTING/RESETTING OPERATION PERFORMAN EDITING FUNCTION PERFORMA FLUID CONTROL OPERATION FLIP APAGE, ETC. FIG. 16 U.S. Patent Dec. 2, 2014 Sheet 12 of 14 US 8,902,181 B2 WORKSPACE A WORKSPACE E WORKSPACED WORKSPACE C WORKSPACE B WORKSPACE A TEMA FIG. 18 U.S. Patent Dec. 2, 2014 Sheet 13 of 14 US 8,902,181 B2 RECEIVE TOUCH INPUT INFORMATION FROMA TOUCH INPUT MECHANISM 1902 RECEIVE MOTION INPUT INFORMATION FROMA MOTION INPUT MECHANISM 1904 DETERMINE WHETHER THE TOUCH INPUT INFORMATION AND THE MOTION INPUT INFORMATION INDICATE THAT THE USER HAS MADE AMULTI-TOUCH-MOVEMENT (MTM) GESTURE 1906 PERFORMAN ACTION IN RESPONSE TO THE MTM GESTURE 1908 FIG. 19 U.S. Patent Dec. 2, 2014 Sheet 14 of 14 US 8,902,181 B2 PRESENTATION COMMUNICATION CONDUIT(s) PROCESSING 2022 DEVICE(S) 2006 NETWORK INTER FACE(s) 2020 SYSTEM MEDIA RAM DEVICE(s) 2002 2008 INPUT MODULE(S) COMPUTER-READABLE 2014 MEDIUM EXAMPLES 2010 Eile Sb t s? 27 FIG. 20 US 8,902,181 B2 1. 2 MULTI-TOUCH-MOVEMENT GESTURES This Summary is provided to introduce a selection of con FORTABLET COMPUTING DEVICES cepts in a simplified form; these concepts are further described below in the Detailed Description. This Summary BACKGROUND is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to A tablet computing device commonly allows users to make limit the scope of the claimed subject matter. various gestures by touching the Surface of the device's touchscreen in a prescribed manner. For example, a user can BRIEF DESCRIPTION OF THE DRAWINGS instruct the computing device to execute a panning operation by touching the Surface of the touchscreen with a single finger 10 FIG. 1 shows an illustrative computing device that includes and then dragging that finger across the Surface of the touch functionality for interpreting touch input information in the context of movement input information. screen Surface. In another case, a user can instruct the com FIG. 2 shows an illustrative interpretation and behavior puting device to performa Zooming operation by touching the selection module (IBSM) used in the computing device of Surface of the touchscreen with two fingers and then moving 15 FIG 1. the fingers closer together or farther apart. FIG. 3 enumerates different features that can be used to To provide a robust user interface, a developer may wish to define a multi-touch-movement (MTM) gesture. expand the number of gestures that the computing device is FIGS. 4-12 illustrate behavior and/or static poses associ able to recognize. However, a developer may find that the ated with different types of MTM gestures. design space of available gestures is limited. Hence, the 20 FIGS. 13-15 illustrate behavior and/or static poses associ developer may find it difficult to formulate a gesture that is ated with additional types of MTM gestures. In this case, the Suitably distinct from existing gestures. The developer may computing device on which these gestures are performed create an idiosyncratic and complex gesture to distinguish corresponds to a booklet computing device having two or over existing gestures. But an end user may have trouble more sections. remembering and performing Such a gesture. 25 FIG. 16 enumerates different actions that may be invoked by MTM gestures. SUMMARY FIG. 17 illustrates a transition from a stacked view of a collection of items to an axonometric view of the collection of Functionality is described herein for detecting and items. The transition between views illustrated in FIG. 17 responding to gestures performed by a user using a comput- 30 may be invoked by an MTM gesture. ing device, such as, but not limited to, a tablet computing FIG. 18 illustrates a carousel view of the collection of device. In one implementation, the functionality operates by items. The transition to and/or from the carousel view may be receiving touch input information from at least one touch invoked by an MTM gesture. input mechanism in response to a user touching the comput FIG. 19 shows an illustrative procedure that explains one ing device. The functionality also receives movement input 35 manner of operation of the IBSM of FIGS. 1 and 2. information from at least one movement input mechanism in FIG.20 shows illustrative computing functionality that can response to movement of the computing device. The func be used to implement any aspect of the features shown in the tionality then determines whether the touch input information foregoing drawings. and the movement input information indicate that a user has The same numbers are used throughout the disclosure and performed or is performing a multi-touch-movement (MTM) 40 figures to reference like components and features. Series 100 gesture. In some cases, the user performs an MTM gesture by numbers refer to features originally found in FIG. 1, series grasping the computing device with two hands and establish 200 numbers refer to features originally found in FIG. 2, ing contact with one or more surfaces of the computing device series 300 numbers refer to features originally found in FIG. with those hands. The user then moves the computing device 3, and so on. in a prescribed path, and/or to achieve a prescribed Static 45 posture. For example, the MTM gesture can be defined with DETAILED DESCRIPTION respect to any type of tilting, flipping, pivoting, twisting, sliding, shaking, vibrating, and/or tapping motion. This disclosure is organized as follows. Section A The functionality can invoke any behavior in response to describes illustrative functionality for interpreting gestures determining that the user has performed (or is performing) an 50 made by a user in the course of interacting with a computing MTM gesture. More specifically, in one case, the functional device, including multi-touch-movement gestures which ity performs the behavior after a discrete gesture has been involve touching and moving the computing device. Section performed. In another case, the functionality performs the B describes illustrative methods which explain the operation behavior over the course of the gesture, once it is determined of the functionality of Section A. Section C describes illus that the gesture is being performed. In one case, the function- 55 trative computing functionality that can be used to implement ality can modify a view in response to determining that the any aspect of the features described in Sections A and B. user has performed an MTM gesture. In another case, the This application is related to commonly-assigned patent functionality can invoke any function in response to deter application Ser. No. 12/970,939, entitled, “Detecting Ges mining that the user has performed an MTM gesture. In tures Involving Intentional Movement of a Computing another case, the functionality can perform any type of con- 60 Device.” naming the inventors of Kenneth Hinckley, et al., trol operation, e.g., by using the gesture to specify a param filed on Dec. 17, 2010. eter, a path, a range of values, etc. The control operation may As a preliminary matter, some of the figures describe con affect any designated item in any manner. Still other types of cepts in the context of one or more structural components, behavior can be associated with gestures. variously referred to as functionality, modules, features, ele The above approach can be manifested in various types of 65 ments, etc. The various components shown in the figures can systems, components, methods, computer readable storage be implemented in any manner by any physical and tangible media, data structures, articles of manufacture, and so on. mechanisms, for instance, by Software, hardware (e.g., chip US 8,902,181 B2 3 4 implemented logic functionality), firmware, etc., and/or any although the principles described herein are also applicable to combination thereof. In one case, the illustrated separation of computing devices having Smaller or larger interaction Sur various components in the figures into distinct units may faces, and/or other aspect ratios. The interaction Surface can reflect the use of corresponding distinct physical and tangible also have any shape (for example, the interaction Surface need components in an actual implementation. Alternatively, or in 5 not be rectangular). Further, in Some cases, the computing addition, any single component illustrated in the figures may device 100 can also embody one or more features of other be implemented by plural actual physical components. Alter form factors. For example, in some cases, the computing natively, or in addition, the depiction of any two or more device 100 may also be classified as a game device, a Smart separate components in the figures may reflect different func phone device, an electronic book reader device, various types tions performed by a single actual physical component. FIG. 10 of netbook computing devices (with and without physical 20, to be discussed in turn, provides additional details regard keyboards), various types of laptop computing devices, ing one illustrative physical implementation of the functions multi-screen “booklet devices, and so on. shown in the figures. In Some scenarios, the computing device 100 is large Other figures describe the concepts in flowchart form. In enough so that the user will most often grasp the computing this form, certain operations are described as constituting 15 device 100 with both hands (that is, when holding the com distinct blocks performed in a certain order. Such implemen puting device 100 aloft). The user may then interact with the tations are illustrative and non-limiting. Certain blocks computing device 100 using the thumbs (and/or other fingers) described herein can be grouped together and performed in a of the grasping hands. However, in some scenarios, a user single operation, certain blocks can be broken apart into plu may also grasp the computing device 100 with one hand. The ral component blocks, and certain blocks can be performed in user may then interact with computing device 100 using his or an order that differs from that which is illustrated herein her free hand and/or the grasping hand. (including a parallel manner of performing the blocks). The More specifically, to simplify explanation, many of the blocks shown in the flowcharts can be implemented in any examples presented herein indicate that the user may estab manner by any physical and tangible mechanisms, for lish contact with the computing device 100 using one or more instance, by Software, hardware (e.g., chip-implemented 25 fingers of one of more hands. As used herein, a thumb is logic functionality), firmware, etc., and/or any combination considered a particular type of finger. More generally, the thereof. principles described hereincan be applied to the case in which As to terminology, the phrase “configured to encom the user establishes contact with the computing device 100 passes any way that any kind of physical and tangible func using any hand parts (and/or other body parts), and/or any tionality can be constructed to perform an identified opera 30 implements that are manipulated by the user (such as an tion. The functionality can be configured to perform an active or passive stylus). operation using, for instance, software, hardware (e.g., chip In some implementations, all of the gesture-recognition implemented logic functionality), firmware, etc., and/or any functionality described herein is implemented on the comput combination thereof. ing device 100. Alternatively, at least some aspects of the The term “logic' encompasses any physical and tangible 35 gesture-recognition functionality can be implemented by functionality for performing a task. For instance, each opera remote processing functionality 102. The remote processing tion illustrated in the flowcharts corresponds to a logic com functionality 102 may correspond to one or more server com ponent for performing that operation. An operation can be puters and associated data stores, provided at a single site or performed using, for instance, Software, hardware (e.g., chip distributed over plural sites. The computing device 100 can implemented logic functionality), firmware, etc., and/or any 40 interact with the remote processing functionality 102 via one combination thereof. When implemented by a computing or more networks (not shown), such as the Internet. However, system, a logic component represents an electrical compo to simplify and facilitate explanation, it will henceforth be nent that is a physical part of the computing system, however assumed that the computing device 100 performs all aspects implemented. of the gesture-recognition functionality. The phrase “means for in the claims, ifused, is intended to 45 The computing device 100 includes a display mechanism invoke the provisions of 35 U.S.C. S 112, sixth paragraph. No 104 and various input mechanisms 106. The display mecha other language, other than this specific phrase, is intended to nism 104 provides a visual rendering of digital information on invoke the provisions of that portion of the statute. a display surface of the computing device 100. More specifi The following explanation may identify one or more fea cally, in one case, the computing device 100 includes a single tures as "optional.” This type of statement is not to be inter 50 display Surface. In another case, the computing device 100 preted as an exhaustive indication of features that may be includes plural display Surfaces, e.g., disposed on different considered optional; that is, other features can be considered parts of the computing device 100. The display mechanism as optional, although not expressly identified in the text. 104 can be implemented by any type of display, such as a Finally, the terms “exemplary' or “illustrative' refer to one liquid crystal display, etc. Although not shown, the comput implementation among potentially many implementations 55 ing device 100 can also include other types of output mecha A. Illustrative Computing Device and its Environment of nisms, such as an audio output mechanism, a haptic (e.g., Use vibratory) output mechanism, etc. A.1. Overview The input mechanisms 106 provide input information in FIG. 1 shows an illustrative portable tablet computing response to input events. In one case, the input mechanisms device 100 on which a user can perform gestures. The term 60 106 provide input information in response to input actions tablet computing device is used broadly herein to refer to any performed by a user. A single instance of input information computing device having one or more generally flat touch can have any composition and duration, and can be generated sensitive Surfaces for receiving input from a user. In some in response to one or more input events. examples, the computing device 100 has a rigid and/or flex The input mechanisms 106 can include at least one touch ible and/or adjustable interaction surface that ranges from 5 65 input mechanism 108 which provides touch input informa inches to 12 inches (along its longest side), with an aspect tion when the user makes contact with at least one Surface of ratio typically ranging from approximately 4:3 to 16:9. the computing device 100. For example, in one case, the touch US 8,902,181 B2 5 6 input mechanism 108 can correspond to a touchscreen inter orientation and/or motion of the computing device 100 to face mechanism which provides touch input information Supply the movement input information. For instance, the when it detects that a user has touched a touch-sensitive movement input mechanism 110 can be implemented using display Surface. This type of touch input mechanism can be accelerometers, gyroscopes, magnetometers, vibratory sen implemented using any technology, such as resistive touch sors, torque sensors, strain gauges, flex sensors, optical screen technology, capacitive touch screen technology, encoder mechanisms, laser speckle tracking mechanisms, acoustic touch screen technology, bi-directional touch screen mechanical orientation sensors, and so on. Any movement technology, optical (e.g., infrared beam or in-plane camera) input mechanism 110 can sense movementalong any number technology, and so on. In bi-directional touch screen technol of spatial axes. For example, the computing device 100 can ogy, a display mechanism provides elements devoted to dis 10 incorporate an accelerometer and/or a gyroscope that mea playing information and elements devoted to receiving infor Sures movement along three spatial axes. mation. Thus, a surface of a bi-directional display mechanism FIG. 1 also indicates that the input mechanisms 106 can is also a capture mechanism. include any other input mechanisms 112. Illustrative other In the examples presented herein, the user may interact input mechanisms can include one or more image sensing with the touch input mechanism 108 by physically touching a 15 input mechanisms, such as a video capture input mechanism, display surface of the computing device 100. However, the a depth sensing input mechanism, a stereo image capture touch input mechanism 108 can also be configured to detect mechanism, and so on. Some of the image sensing input when the user has made contact with any other surface of the mechanisms can also function as movement input mecha computing device 100 (which may or may not also provide a nisms, insofar as they can be used to determine movement of display Surface). Such as the back of the computing device the computing device 100 relative to the surrounding envi 100, and/or the sides of the computing device 100, and/or any ronment. Other input mechanisms can include a physical front portions of the computing device 100 which do not keyboard input mechanism, a joystick mechanism, a mouse provide a display Surface (such as a bezel of the computing input mechanism, a Voice input mechanism, and so on. device 100, if it provides one). In addition, in some cases, a In some cases, the input mechanisms 106 may represent user can be said to make contact with a surface of the com 25 components that are integral parts of the computing device puting device 100 when he or she draws close to a surface of 100. For example, the input mechanisms 106 may represent the computing device, without actually physically touching components that are enclosed in or disposed on a housing the Surface. Among other technologies, the bi-directional associated with the computing device 100. In other cases, at touch screen technology described above can accomplish the least Some of the input mechanisms 106 may represent func task of detecting when the user moves his or her hand close to 30 tionality that is not physically integrated with the other com a display Surface, without actually touching it. A user may ponents of the computing device 100. For example, at least intentionally or unintentionally contact a surface of the com some of the input mechanisms 106 can represent components puting device 100 with any part of his or her hand, such as that are coupled to the computing device 100 via a commu with any part of one or more fingers, a palm, a knuckle, etc. As nication conduit of any type (e.g., a cable). For example, one stated above, a thumb is considered as one type offinger. 35 type of touch input mechanism 108 may correspond to a Alternatively, or in addition, the touch input mechanism pad-type input mechanism that is separate from (or at least 108 can correspond to a pen input mechanism whereby a user partially separate from) the display mechanism 104. A pad makes physical or close contact with a surface of the comput type input mechanism is also referred to as a tablet, a digitizer, ing device 100 with a stylus or other implement of any type a graphics pad, etc. (besides, or in addition to, the users hands). Accordingly, the 40 In certain cases, the computing device 100 may correspond term touch input information is also intended to encompass to a booklet computing device 100 that includes two or more input information that may be produced when a stylus makes sections. The booklet computing device can include any type contact (or near-contact) with the computing device 100. of coupling mechanism for coupling the sections together, Further, in some cases, the touch input information may Such as, without limitation, a hinge mechanism. In some include a combination of input information produced by the 45 cases, each section of the booklet computing device includes tip of the stylus and input information produced by any por its own touch input mechanism. For example, each section of tion(s) of the hand which deliberately or inadvertently makes the booklet computing device can include a touchscreen inter contact (or near-contact) with the touch-sensitive surface(s) face mechanism. In addition, or alternatively, each section of of the computing device 100. For example, a user may acci the booklet computing device can include its own movement dently rest or brush a finger, palm, knuckle, etc. against a 50 input mechanism, each for sensing the movement of that touch-sensitive surface while making pen strokes using a section (which may be independent, to some extent, from the stylus. However, to facilitate and simplify description, the movement of other sections). However, in other cases, the explanation will henceforth assume that the user interacts booklet computing device can include a single touch input with the touch input mechanism 108 by physically touching mechanism and/or a single movement input mechanism. its surface with fingers of the hands. 55 In addition, or alternatively, a booklet computing device FIG. 1 depicts the input mechanisms 106 as partially over can include any type of movement input mechanism that lapping the display mechanism 104. This is because at least senses movement of the sections relative to each other. For some of the input mechanisms 106 may be integrated with example, the booklet computing device can include an angle functionality associated with the display mechanism 104. sensor (e.g., a resistive flex sensor, an optical encoder, etc.) This is the case, for example, with respect to a touch interface 60 which detects the angle of a first section relative to a second mechanism because the display Surface of this device is used section. to both display information and provide input information. An interpretation and behavior selection module (IBSM) The input mechanisms 106 also include at least one move 114 performs the task of interpreting the input information. In ment input mechanism 110 for Supplying movement input particular, the IBSM 114 receives at least touch input infor information that describes movement of the computing 65 mation from the touch input mechanism 108 and movement device 100. That is, the movement input mechanism 110 input information from the touch movement input mecha corresponds to any type of input mechanism that measures the nism 110. Based on these instances of input information, the US 8,902,181 B2 7 8 IBSM 114 determines whether the user has made (or is mak kind of gesture. For example, a gesture signature may indicate ing) a recognizable gesture. If a gesture is detected, the IBSM that a gesture X is characterized by a pattern of observations executes behavior associated with that gesture. More specifi A, B, and C. Hence, if the gesture matching module 202 cally, in one case, the IBSM 114 performs the behavior after determines that the observations A, B, and Care present in the a discrete gesture has been completed. In another case, the input information at a particular time, it can conclude that the IBSM 114 performs the behavior over the course of the ges user has performed (or is currently performing) gesture X. ture, once it is determined that the gesture is being performed. Any gesture that is defined with respect to a combination of FIG. 2 provides additional details regarding one implemen two or more contacts and movement is referred to herein as a tation of the IBSM 114. Section A.3 provides additional multi-touch-movement (MTM) gesture. The gesture signa details regarding behavior that can be generated in response 10 ture which characterizes an MTM gesture is referred to as an to detecting that the user has performed (or is performing) a MTM signature. gesture. More specifically, in Some cases, the user can perform an Finally, the computing device 100 may run at least one MTMgesture by grasping the computing device 100 with two application 116 that performs any high-level and/or low-level hands. The user can then use his or her thumbs (and/or other function in any application domain. In one case, the applica 15 fingers and/or other hand parts) to establish two or more tion 116 represents functionality that is stored on a local store contacts on one or more Surfaces of the computing device provided by the computing device 100. For instance, the user 100. In other cases, however, the user may establish the two or may download the application 116 from a remote market more contacts using a singlehand. The user can then move the place system or the like. The user may then run the application computing device 100 in a prescribed manner. 116 using the local computing resources of the computing The user may perform the movement in various ways. In device 100. Alternatively, or in addition, a remote system can Some cases, the user performs the movement by moving the store at least parts of the application 116. In this case, the user computing device 100 over a prescribed path associated with can execute the application 116 by instructing the remote the MTM gesture. Alternatively, or in addition, the user estab system to run it. lishes the movement by advancing the computing device 100 In one case, the IBSM 114 represents a separate component 25 to a prescribed orientation. In the latter case, the gesture is with respect to application 116 that both recognizes a gesture associated with a static posture of the computing device 100, and performs whatever behavior is associated with the ges independent of the path that is used to establish that static ture. In another case, one or more functions attributed to the posture. In still other cases, an MTM gesture can be defined IBSM 114 can be performed by the application 116. For by a combination of one or more static poses and dynamic example, in one implementation, the IBSM 114 can interpret 30 movement of the computing device 100. Section A.2 a gesture that has been performed, while the application 116 describes additional features that can be used to define an can select and execute behavior associated with the detected MTM gesture. gesture. Accordingly, the concept of the IBSM 114 is to be In some cases, the touch input events associated with an interpreted liberally herein as encompassing functions that MTM gesture (which cause the generation of the touch input can be performed by any number of components within a 35 information) occur simultaneously with the movement input particular implementation. events (which cause the generation of the movement input FIG. 2 shows one implementation of the IBSM 114. The information), meaning that the user touches the computing IBSM 114 can include a gesture matching module 202 for device 100 at the same time that he or she moves it. But in receiving various instances of input information. The input other cases, the touch input events can be temporally inter information can include touch input information from the 40 leaved with the movement input events in any manner. For touch input mechanism 108, movement input information example, a touch input event can precede and/or follow a from the movement input mechanism 110, and any other movement input event. Alternatively, or in addition, a sepa input information from any other input mechanisms 112. The rate touch input event and/or movement input event can pre input information can also include context information which cede and/or follow a simultaneous occurrence of a touch input indicates a context in which a user is currently using the 45 event and a movement input event. However, to facilitate computing device 100. For example, the context information explanation, it will henceforth be assumed that the MTM can identify the application that the user is running at the gestures involve simultaneously touching and moving the present time, if any. Alternatively, or in addition, the context computing device 100. information can describe the physical environment in which The gesture matching module 202 can also incorporate the user is using the computing device 100, e.g., by providing 50 noise canceling functionality 206. The noise canceling func details regarding any of ambient light and/or Sound levels in tionality 206 reduces the possibility that the IBSM 114 will the environment; the presence of other proximal computing interpret non-gesture-related behavior as gestures. The noise devices, persons, and/or physical barriers (such as walls and canceling functionality 206 also reduces the possibility that windows) in the environment, etc. Alternatively, or in addi the IBSM 114 will incorrectly interpret gesture-related tion, the context information can identify the user who is 55 behavioras noise, or incorrectly interpretan intended gesture currently interacting with the computing device 100, and so for an unintended gesture, that is, by “confusing these ges O. tures. In one implementation, the gesture matching module 202 In one case, the noise canceling functionality 206 can compares the input information with a collection of gesture define various dead-band thresholds which result in the signatures provided in a data store 204. If the gesture match 60 delayed execution of an MTM gesture. For example, the noise ing module 202 finds a match, it concludes that the user has canceling functionality 206 can delay the execution of a rota performed (or is performing) a gesture that is associated with tion-based MTM gesture (to be described below) until the the matching gesture signature. user rotates the computing device 100 beyond a prescribed More specifically, a gesture signature may provide any threshold angular displacement. This will ensure that the descriptive information which characterizes the touch input 65 IBSM 114 does not interpret small incidental rotations that information and/or motion input information that are typi occur during the normal handling of the computing device cally produced when a user makes (or is making) a particular 100 as an MTM gesture. In another case, the noise canceling US 8,902,181 B2 9 10 functionality 206 can delay the execution of a static posture applied to any surface(s) of the computing device 100. This gesture until the user has maintained that static pose for a may correspond, for instance, to the number offingers that the prescribed period of time. Again, this provision will help user has used to touch a surface of the computing device 100. ensure that the IBSM 114 does interpret transitory positions In addition, or alternatively, the MTM gesture can be defined that occur during handling of the device as MTM gestures. with respect to the type of contact that the user has applied to Alternatively, or in addition, the signatures that describe the the Surface of the computing device. This may correspond, for MTM gestures may incorporate the above-described types of instance, to the type offinger that the user has applied, and/or dead-band features as integral parts of their definition. whether the user has applied a finger rather than a stylus, etc. In addition, or alternatively, the noise canceling function The IBSM 114 can determine the type offinger that has been ality 206 can identify other types of contacts and/or move 10 applied based on any combination of evidence, such as: a ments that are indicative of noise. The noise canceling func detected fingerprint (which can be compared with a set of tionality 206 can then filter this noise out (or otherwise take preregistered fingerprints associated with different fingers account of this noise) when interpreting the input information belonging to a user); a detected finger contact shape; a for the presence of MTM gestures. This filtering operation detected finger contact placement with respect to other finger can occurat any juncture of analysis performed by the gesture 15 and/or hand portion contacts (e.g., a finger placement with matching module 202. For example, the gesture matching respect to a palm contact), and so on. The IBSM 114 can also module 202 can detect and remove noise prior to, during, determine the type of contact that has been performed by and/or after the interpretation of a gesture. analyzing a detected Stylus tip contact with respect to any In addition, or alternatively, the noise canceling function hand portion contact (e.g., a pen contact with respect to palm ality 206 can compare the contacts and/or movements per contact). These contact-related features are mentioned by formed by the user with one or more telltale noise signatures. way of example, not limitation; other implementations can For example, one noise signature may describe a pattern of define gestures with respect to additional types of contact behavior that is indicative of the user picking up or setting related features. down the computing device 100, e.g., without the intent of Some of these contact-related features may also provide making a gesture of any type. The IBSM 114 can disqualify 25 evidence regarding the user's position and/or orientation with the classification of input information as an MTM gesture if respect to the computing device 100. For example, the man the input information also matches such a noise signature, ner in which the user rests his or her palm against a touch regardless of whether the input information otherwise sensitive surface while interacting with the computing device matches the pattern of behavior associated with the MTM 100 may indicate the position of the user with respect to the gesture. Alternatively, or in addition, the signatures that 30 computing device 100. describe the MTM gestures can define (or make reference to) As a second category, an MTM gesture can be defined with the type of noise information that will override the classifi respect to the characteristics of the movement associated with cation of the input information as MTM gestures. the gesture. For example, the MTM gesture can be defined as After detection of a gesture, a behavior executing module a static posture of the computing device 100, regardless of the 208 executes whatever behavior is associated with the 35 path used to establish that static posture. detected gesture. More specifically, as Stated above, in a first In addition, or alternatively, the MTM gesture can be case, the behavior executing module 208 executes a behavior defined with respect to the spatial characteristics of a path at the completion of a gesture. In a second case, the behavior over which the computing device 100 is moved. For example, executing module 208 executes a behavior over the course of the user may rotate one or more parts of the computing device the gesture, starting from that point in time that it recognizes 40 100, e.g., to establishing a tilting, flipping, pivoting, and/or that the telltale gesture is being performed. twisting motion (to be described in greater detail below). Section A.2 (below) describes various types of MTM ges Alternatively, or in addition, the user may move the comput tures. Section A.3 describes various actions that may be ing device 100 in a transverse manner, e.g., to slide it within invoked by the MTM gestures of section A.2. That is, any a plane. Alternatively, or in addition, the user may shake or gesture described in section A.2 can invoke any action 45 vibrate the computing device 100. Alternatively, or in addi described in section A.3. tion, the user may tap the computing device 100, e.g., by A.2. Illustrative Gestures striking the computing device 100 with a finger or multiple FIG.3 enumerates illustrative features that may be used to fingers and/or other implement(s), or by rapping the comput define an MTM gesture. These features are grouped into ing device 100 against Some object (such as a table, a hand, various categories to provide an organized explanation of 50 another tablet device that senses a mutually-corresponding these features. However, these categories are not to be con signal produced by the collision, etc.). Strued as mutually exclusive. For example, a feature may have Alternatively, or in addition, the MTM gesture can be properties which warrant its classification into two or more defined with respect to the temporal aspects of the computing categories. Any entity or entities may define gestures, such as device's movement over its prescribed path, referred to herein device developers, application developers, end users, stan 55 as the temporal-movement characteristics of the gesture. That dards organizations, etc. is, the temporal-movement characteristics define the speed As a first category, an MTM gesture can be defined with (i.e., constant rate), acceleration, and/or deceleration of the respect to the characteristics of the contact associated with the computing device 100 over the course of its movement. In gesture. For example, the MTM gesture can be defined with Some cases, for instance, different gestures can be defined respect to the location or locations on a touch-sensitive dis 60 based on different constant rates at which the user moves the play Surface that the user has touched. In addition, or alterna computing device 100 over the same spatial path. In addition, tively, the MTM gesture can be defined with respect to the or alternatively, different gestures can be defined based on location or locations of other parts of the computing device different ways in which the user accelerates and/or deceler 100 that the user has touched (such as a back surface, side ates the computing device 100 over its path. Surface(s), bezel regions, etc. of the computing device 100). 65 For example, a first gesture and a second gesture can both In addition, or alternatively, the MTM gesture can be defined involve rotating the computing device 100 over the same with respect to the number of contacts that the user has spatial path to perform a scrolling and/or Zooming operation. US 8,902,181 B2 11 12 But the first gesture may involve rotating the computing In one case, the IBSM 114 learns and applies a single set of device 100 at a constant rate while the second gesture may user-specific characteristics corresponding to gestures made introduce one or more spurts of acceleration and/or decelera by a single user, based on the assumption that a single user/ tion into the movement. The second gesture may cause the owner predominately uses the computing device 100. In other IBSM114 to perform scrolling/zooming by larger increments cases, the IBSM 114 can learn and apply different sets of compared to the first gesture. Thus, a user may perform the user-specific characteristics corresponding to different second gesture when he or she wishes to quickly advance to a respective users. In this approach, the IBSM 114 can deter target location or Zoom level. The user may perform the first mine the user who is interacting with the computing device gesture when he or she wishes to advance more slowly to a 100 at any particular moment, and then apply an appropriate target location or Zoom level. 10 set of user-specific characteristics when interpreting the ges In certain cases, the gestures that leverage temporal-move ture-related behavior of that user. The IBSM 114 can deter ment characteristics can be defined with respect to absolute mine the user who is interacting with the computing device speeds, accelerations, and decelerations. That is, if the IBSM 100 by asking the user to manually identify himselfor herself 114 detects movement which matches pre-defined ranges of (e.g., via a log-in routine or the like), and/or by automatically absolute rates/accelerations/decelerations associated with an 15 detecting the user. The IBSM 114 can automatically deter MTM gesture, it will determine that the user has performed mine the identity of the user through any mechanism, e.g., by (or is performing) that gesture. In other cases, these gestures fingerprint analysis, Voice analysis, face-recognition analy can be defined with respect to relative changes of movement. sis, behavioral analysis, etc. These gestures are baseline-agnostic. For example, assume In addition, or alternatively, the IBSM 114 can interpret that a gesture has a profile which is characterized by slow gestures based on information pertaining to the environment rotation, followed by fast rotation, followed by slow rotation. in which the computing device 100 is being used. The envi The IBSM 114 can detect that the user has performed this ronmental information can include, for example, information gesture if the users behavior matches this slow-fast-slow regarding the presence of features in the environment (either pattern, regardless of the absolute rates that are associated fixed or movable). Such as computing devices, persons, fur with the “slow’ rotation and the “fast rotation in this pattern. 25 niture, walls, whiteboards, etc. The environment information An MTM gesture can also be defined with respect to the can also describe the position of these features relative to the magnitude at which any of the above-described motions are computing device 100, e.g., with respect to direction (e.g., as performed. For example, the IBSM 114 can define two dif assessed based on relative angular heading), and/or approxi ferent gestures based, at least in part, on the force at which mate distance, etc. tapping and/or shaking is applied, and so on. 30 The above-described context-related features are set forth These movement-related features are mentioned by way of by way of example, not limitation; other implementations can example, not limitation; other implementations can define define gestures with respect to additional types of context gestures with respect to additional types of movement-related related features. features. As stated, the categories of features described above FIGS. 4-11 illustrate contacts and movements associated are not mutually exclusive. For example, the magnitude at 35 with illustrated gestures. In these examples, the user grasps a which a tapping motion is performed can also be classified as computing device 402 with two hands (404, 406) to establish defining the temporal-movement characteristics of the ges two or more points of contact with one or more surfaces of the ture. computing device 402. In other cases, the user may perform As a third category, an MTM gesture can be defined with the gesture while grasping the computing device 402 with a respect to a context in which the gesture is invoked. For 40 single hand and/or using a single hand to apply the plural example, the IBSM 114 can interpret a particular pattern of COntactS. contacts and movements in a manner which depends on the Further, the figures indicate that the computing device 402 application that a user is interacting with at the present time. includes a front display surface 408 that also functions as a Thus, for example, the IBSM 114 can interpret the same touchscreen interface mechanism. A bezel 410 may surround contacts and movements in two different ways when the user 45 the display surface 408. In some cases, the bezel 410 itself is interacting with two different respective applications. may incorporate a touch-sensitive Surface; in other cases, it In addition, or alternatively, the IBSM 114 can interpreta does not. Further, in other cases, the computing device 402 particular pattern of contacts and movements in a manner can entirely dispense with the bezel 410. In such an imple which depends on the user who is interacting with the com mentation, the display surface 408 may extend to the edges of puting device 100 at any given time. For example, the IBSM 50 a front surface of the computing device 402. 114 can learn the user-specific characteristics of a particular As a final prefatory note, the figures illustrate many MTM MTM gesture by analyzing the manner in which the user gestures in which the user applies two or more points of performs the gesture over a span of time. Alternatively, or in contact to a peripheral region associated with the computing addition, the IBSM 114 can allow the user to manually device 402. (The term “point” refers to a general locus at specify preferences which define the user-specific character 55 which a user touches the computing device 402, also referred istics of the MTM gesture. The IBSM 114 can then interpret to hereinas a touch "contact, rather than a single mathemati the MTM gesture, when performed by this particular user, by cal point.) For example, the peripheral region may encompass making reference to the learned and/or manually-specified the region defined by the bezel 410 and/or the outer portions user-specific characteristics of the gesture. This allows the of the display surface 408. However, an MTM gesture can be IBSM 114 to more accurately determine the type of gesture 60 defined with respect to points of contacts applied at any that the user has performed. For example, a particular user location(s) on any Surface(s) of the computing device 402. may perform a Zooming operation in a slightly idiosyncratic In any of these cases, the fact that the user has applied two manner, e.g., by more aggressively tilting the computing simultaneous points of contact to the touch-sensitive device 100 compared to other users. The IBSM 114 can take surface(s) of the computing device 402 alerts the IBSM 114 the user-specific characteristics of this Zooming gesture into 65 that the user may intend to perform an MTM gesture, as account to more accurately determine when the particular opposed to some other gesture or input action or incidental user has performed the gesture. contact. The IBSM 114 can thereafter interpret any subse US 8,902,181 B2 13 14 quent movement of the computing device 402 as part of the 406). (This movement therefore also qualifies as a tilting MTM gesture, rather than noise (e.g., inadvertent move movement.) Alternatively, the user can rotate the computing ment). In this sense, the application of two or more points of device 402 about an axis 608 which passes along the edge 602 contacts can be considered as triggering an MTM-related or any other axis that passes through any other edge of the state transition. computing device 402. Once again, the flipping MTM gesture Beginning with FIG. 4, this figure shows an MTM gesture can be defined (and distinguished from other MTM gestures) that involves a tilting movement. In this gesture, the user with respect to any of the features enumerated in FIG.3, such grasps the computing device 402 with two hands (404, 406) as, but not limited to: the maximum angular displacement(s) along two opposing edges (412,414) of the computing device produced by the flipping movement; the orientation(s) of the 402 device. More specifically, the two edges (412,414) have 10 computing device 402 at the start and/or end of the flipping respective lengths. The user grasps these edges at respective movement; the temporal-movement characteristics of the points that lie generally in the middle of these lengths. In flipping movement; and/or the location(s) at which the user doing so, the user applies his or her thumbs (416, 418) and/or grasps the computing device 402, and so on. other fingers to a peripheral portion of the touch-sensitive FIG. 8 shows an MTM gesture that involves a pivoting display surface 408. As the display surface 408 incorporates a 15 movement. In this gesture, in state A, the user grasps the touch-sensitive surface, the touch input mechanism 108 pro computing device 402 with two hands (404, 406) along duces touch input information in response to the user's finger diagonally-opposing corners of (802, 804) of the computing COntactS. device 402 device. In doing so, the user applies his or her The user then proceeds to rotate the computing device 402 thumbs (416,418) and/or other fingers to a peripheral portion about an axis 420 that generally passes through the contacts of the touch-sensitive display surface 408. established by the user's hands (404, 406). In some cases, this The user then proceeds to rotate the computing device 402 particular MTM gesture can be defined by a single rotational in a plane that is generally parallel to the display surface 408. Sweep of the computing device 402, e.g., by moving the In some cases, this particular MTM gesture can be defined by computing device 402 in a downward direction away from the a single pivotal Sweep of the computing device 402 in a user, or by moving the computing device 402 in an upward 25 clockwise direction (resulting in state B of FIG. 8) or coun direction towards the user. In this case, the IBSM 114 can terclockwise direction (resulting in state C in FIG. 8). More distinguish between two different gestures based on the direc over, the IBSM 114 can distinguish between two different tion in which the user tilts the computing device 402. In other gestures based on the direction in which the user pivots the cases, an MTM gesture can be defined with respect to two or computing device 402. In other cases, an MTM gesture can be more rotational Sweeps of the computing device 402, e.g., by 30 defined with respect to two or more pivotal sweeps of the moving the computing device 402 first in the downward computing device, e.g., by moving the computing device 402 direction and then in the upward direction. first in the clockwise direction and then in the counterclock More generally, any of the features enumerated in FIG. 3 wise direction. Once again, the pivoting MTM gesture can be can be used to define a tilting MTM gesture (or any other defined (and distinguished from other MTM gestures) with gesture described herein). For example, the tilting MTMges 35 respect to any of the features enumerated in FIG. 3, such as, ture can be defined (and distinguished from other MTM ges but not limited to: the maximum angular displacement(s) tures) with respect to any temporal-movement characteristics produced by the pivoting movement; the orientation(s) of the of the rotational Sweep(s) performed by the user (e.g., rates, computing device 402 at the start and/or end of the pivoting accelerations, decelerations, etc.). In addition, or alterna movement; the temporal-movement characteristics of the tively, the tilting MTM gesture can be defined (and distin 40 pivoting movement; and/or the location(s) at which the user guished from other MTM gestures) with respect to the maxi grasps the computing device 402, and so on. mum angular displacement(s) produced by the tilting FIG.9 shows another gesture that is defined with respect to movement. In addition, or alternatively, the tilting MTM ges a pivoting movement. In this case, however, the user grasps ture can be defined (and distinguished from other MTM ges the computing device at respective intermediary points along tures) based on the location(s) at which the user grasps the 45 the opposing edges (902,904) of the computing device 402. computing device 402. In addition, or alternatively, the tiling rather than at opposing corners (as in the case of FIG. 8). The MTM gesture can be defined (and distinguished from other user then proceeds to rotate the computing device 402 in a MTM gestures) based on the starting and/or ending spatial plane that is generally parallel to the display surface 408. orientation(s) of the computing device 402. But in other FIG. 10 shows an MTM gesture that involves a twisting cases, the tilting MTM gesture can be defined independent of 50 movement. In this gesture, the user grasps the computing the starting and/or ending spatial orientation(s) of the com device 402 with two hands (404, 406) along diagonally-op puting device 402. posing corners (1002, 1004) of the computing device 402. In FIGS. 6 and 7 show an MTM gesture that involves a flip doing so, the user applies his or her thumbs (416, 418) and/or ping movement. In this gesture, the user again grasps the other fingers to a peripheral portion of the touch-sensitive computing device 402 with two hands (404, 406) along two 55 display surface 408. opposing edges (602, 604) of the computing device 402 The user then proceeds to rotate the computing device 402 device. In this case, however, the user grasps the edges (602, in such a manner that the corner 1002 is moved in a first 604) at laterally-adjacent corners of the two edges (602, 604). direction towards a reference point, while the corner 1004 is In doing so, the user applies his or her thumbs (416, 418) moved in a second direction away from the reference point. and/or other fingers to a peripheral portion of the touch 60 For example, if the display surface 408 is facing the user (and sensitive display surface 408 at the two respective laterally thus the user may correspond to the reference point), the user adjacent corners of the display surface 408. can pull the corner 1002 up towards himself or herself; at the The user then proceeds to rotate the computing device 402 same time, the user can push the corner 1004 away from about an axis that generally passes through one of the edges of himself or herself. For the particular positioning of the user's the computing device 402. For example, the user can rotate 65 hands (404, 406) shown in FIG. 10, this type of rotation does the computing device 402 about an axis 606 which passes not correspond to a tilting motion (because it does not occur through the contacts established by the user's hands (404, about an axis which connects the user's contact positions), US 8,902,181 B2 15 16 nor does it correspond to a pivoting movement (because it angular displacement(s) produced by the movement; the ori does not occur in a plane that is parallel to the display Surface entation(s) of the computing device 100 at the start and/or end 408). But for other positions of the hands (404, 406), what is of the movement; the temporal-movement characteristics of considered a twisting movement may also correspond to the movement; and/or the location(s) at which the user grasp some other kind of rotation already described. 5 the computing device 402. Once again, the twisting MTM gesture can be defined (and FIG. 12 shows another back-touching MTM gesture in distinguished from other MTM gestures) with respect to any which the user applies a contact to a back Surface of the of the features enumerated in FIG. 3, such as, but not limited computing device 402. In this case, the user applies the thumb to: the maximum angular displacement(s) produced by the 416 of the hand 404 to touch a point along the periphery of the twisting movement; the orientation(s) of the computing 10 display Surface 408 that is generally midway along a side device 100 at the start and/or end of the twisting movement; edge 1202. The user then places one or more fingers 1204 of the temporal-movement characteristics of the twisting move the other hand 406 on the back of the computing device 402. ment; and/or the location(s) at which the user grasp the com The user then pivots the computing device 402 about a pivot puting device 402, and so on. For example, the user can point defined by the thumb 416, in a plane that is generally perform a twisting movement for the thumb placements 15 parallel to the display surface 408. In this movement, the shown in FIG. 4. The IBSM 114 can interpret this gesture as fingers 1204 of the hand 406 remain in contact with the back a different type of twisting gesture than the twisting gesture surface of the computing device 402. Further, these fingers described above with respect to FIG. 10. 1204 can help guide the pivoting movement. Other back FIG. 11 shows an MTM gesture in which the user applies touching MTM gestures of this kind can be defined based on one or more fingers of the hand 406 to touch a front surface of different ways of touching the computing device 100 and/or the computing device 402. For example, the user may apply moving the computing device 100. one or more fingers to the display surface 408 that is disposed As a general clarification, the explanation above described on the front surface of the computing device 402. Optionally, gestures which include a single class of movement, such as a the user may use the finger(s) of the hand 406 to select an item tilting, flipping, pivoting, twisting, etc. But any MTM gesture of any type on the display surface 408 on which the some 25 can be defined with respect to two or more types of motions gesture-related behavior is to be performed, such as by using that are applied at the same time or in temporal Succession. a single finger to touch the item, or two or more fingers to For example, a gesture can be performed by sliding the com frame a region, etc. The user may then apply one or more puting device 402 in a plane while simultaneously rotating it, fingers of the hand 404 to touch a back surface 1102 of the or by sliding it and then rotating it (or vice versa). In addition computing device 402, thereby holding the computing device 30 or alternatively, any MTM gesture can include shaking move 402 like a tray. More generally stated, the useruses a first hand ment, vibratory movement, tapping-induced movement, and to touch any first surface of the computing device and a so on, or any combination thereof. These non-rotating move second hand (and/or the first hand) to touch any second Sur ments can be applied by themselves or in combination with face of the computing device. In the particular illustrated case other types of movements. To cite one example, the user can of FIG. 11, the first surface and the second surface are oppo 35 perform a particular MTM gesture by holding the computing site in the sense the normal of the first Surface is generally device 402 in the manner shown in FIG.4, while tapping one pointed in an opposite direction to the normal to the second or more fingers on the rear touch-sensitive surface of the Surface. In other cases (not shown), the user may alterna computing device 402. tively, or in addition, touch any of the side surfaces of the FIG. 13 shows a booklet computing device 1302 that computing device 402 and/or any other “non-front 40 includes two sections—namely a first section 1304 and a Surface(s). second section 1306. Although not shown, other booklet The non-front surface(s) that the user touches each include computing devices can include three or more sections. The a touch-sensitive surface, which may or may not be integrated booklet computing device 1302 can use any coupling mecha with a display surface. This feature can be implemented in nism to couple the first section 1304 to the second section any manner, Such as by providing a touch-sensitive display 45 1306, such as, without limitation, a hinge mechanism. Fur Surface on any non-front Surface. Further, in the case of a ther, in Some implementations, the coupling mechanism can booklet computing device (to be described below), a first allow a user to detach the first section 1304 from the second section can be rotated relative to a second section Such that its Section 1306. back Surface faces the back Surface of the second section; in In one implementation, the first section 1304 includes a this configuration, the normal to the first section will point in 50 first display surface 1308, while the second section 1306 an opposite direction to the normal to the second Surface. As includes a second display surface 1310. Further, in some an additional feature, the computing device 402 can option implementations, both the first section 1304 and the second ally include functionality that visualizes the positions of the section 1306 may incorporate touch input mechanisms. For user's fingers that are placed on the back surface 1102 of the example, the first display surface 1308 and the second display computing device 402, e.g., using technology described in 55 surface 1310 may each incorporate touchscreen interface Forelines, et al., “Lucid Touch: a See-through Mobile mechanisms. The booklet computing device 1302 can also Device.” Proceedings of the 20th Annual ACM Symposium on include touch-sensitive surfaces disposed on the back of the User interface Software and Technology, 2007, pp. 269-278. first section 1304 and/or the back of the second section 1306, After establishing the type of contact shown in FIG. 11, the and/or on any other parts of the booklet computing device user can move the computing device 402 in any manner to 60 1302. complete a back-touching MTM gesture (meaning a gesture The user may apply an MTM gesture using the booklet that involves, as one feature, touching a back Surface of the computing device 1302 by touching any touch-sensitive Sur computing device 402). For example, the user can tilt, flip, or face(s) of the booklet computing device 1302. In the merely twist the computing device. In addition, a back-touching illustrative case of FIG. 13, the user grasps the first section MTM gesture can be defined (and distinguished from other 65 1304 with the hand 404, placing his or her thumb 416 on the MTM gestures) with respect to any of the features enumer first display surface 1308. The user grasps the second section ated in FIG. 3, such as, but not limited to: the maximum 1306 with the hand 406, placing his or her thumb 418 on the US 8,902,181 B2 17 18 second display Surface 1310. Again, this represents one of generally parallel to a horizontal plane, as shown in FIG. 7 many possible finger placements. For instance, in another (while simultaneously applying at least two contacts to the case (not shown), the user can apply fingers to the two sec display surface 408). tions (1304, 1306) of a booklet computing device 1302 in the As another example, the IBSM 114 can detect that the user manner shown in FIG. 4 (or any other of the preceding fig has performed a particular static posture gesture by producing ures). Different booklet MTM gestures can be defined with and holding a particular angle between the first section 1304 respect to the above-described types of differing finger place and the second section 1306 of the booklet computing device mentS. 1302 of FIG. 13 (while simultaneously applying two contacts The user may then proceed to move the booklet computing to the booklet computing device 1302). For example, the device 1302 in any manner, e.g., by performing any type of 10 IBSM 114 can detect that the user has performed a first rotating, sliding, shaking, vibrating, and/or tapping move gesture by producing the relatively small angle between sec tions (1304, 1306) shown in FIG. 14, whereas the user has ment described above. Different booklet MTM gestures can performed a second gesture by producing the large hyperex be defined with respect to any of the features enumerated in tend angle between sections (1304, 1306) shown in FIG. 15. FIG. 3, such as by leveraging different paths of movement, 15 In any of these posture-related examples, the IBSM 114 can different starting and/or ending orientations of movement, also consider other environmental factors which indicate the different temporal-movement characteristics of movement, manner in which the user is currently using the computing different maximum displacements produced by the move device 100. ments, and so on. A.3. Illustrative Actions that Maybe Invoked In addition, or alternatively, a booklet MTM gesture may The MTM gestures described above can invoke any type of be defined with respect to movement of the first section 1304 actions. In some cases, a device developer may establish a relative to the second section 1306. For example, the user can nexus between a particular MTM gesture and a resultant dynamically rotate the first section 1304 relative to the second action. This type of MTM gesture is application-agnostic in section 1306 in a first and/or second direction about an axis the sense the IBSM 114 will respond to the MTM gesture in 1312 associated with the coupling mechanism. For instance, 25 the same manner regardless of the application that is being FIG. 14 shows a case in which the user has moved the sections run at the time that the MTM gesture is performed. In another (1304, 1306) closer together relative to the starting state case, an application developer may establish an association shown in FIG. 13. between a particular MTM gesture and a resultant action. In FIG. 15 shows a case in which the user has moved the this case, the IBSM 114 can respond to the same MTM 30 gesture in a manner that depends on the application that is sections (1304, 1306) farther apart relative to the starting state being run at the time that the MTM gesture is performed. In shown in FIG. 13. Indeed, in the case of FIG. 15, the angle addition, or alternatively, the computing device 100 may between the two sections (1304, 1306) exceeds 180 degrees, allow the user to define the action that will be invoked in meaning that the user has hyperextended the sections (1304, response to the execution of a particular MTM gesture. 1306). More specifically, a booklet MTM gesture can be 35 FIG. 16 provides a non-exhaustive list of possible actions defined with respect to a single sweep about the axis 1312, or that may be invoked by different MTM gestures. These with respect to two or more Sweeps, e.g., by decreasing the actions are grouped into various categories to provide an angle between the sections (1304, 1306), followed by increas organized explanation of the actions. However, these catego ing the angle between the sections (1304, 1306). In addition to ries are not to be construed as mutually exclusive. For the above-stated features, different MTM gestures can be 40 example, an action may have properties which warrant its defined with respect to any of: (a) for single-sweep gestures, classification into two or more categories. Further, as men the direction of the single Sweep (that is, depending on tioned above, the IBSM 114 can invoke an action after the whether the user is opening or closing the booklet computing completion of the gesture in its entirety, or in the course of the device 1302); (b) for plural-sweep gestures, the combination user execution of the gesture. of sweeps that define the gesture; (c) whether the first section 45 In a first case, the IBSM 114 can invoke a particular view in 1304 is moved and the second section 1306 is held stationary response to determining that the user has performed (or is (or vice versa), or whether both sections (1304, 1306) are performing) an MTM gesture. For example, the IBSM 114 moved; (d) and/or the temporal-movement characteristics of can change the layout of items on a display Surface in the movement of section(s) (1304, 1306), and so on. response to determining that the user has performed (or is In all of the examples described so far, the MTM gestures 50 performing) an MTM gesture. For instance, consider the were defined with respect to the dynamic movement of the implementation shown in FIG. 17. In a first state, a display computing devices (402, 1302) over a prescribed path. Alter Surface presents a collection of layered items (such as work natively, or in addition, some MTM gestures can be defined spaces) in a stacked manner, Such that the display Surface only with respect to the static posture of the computing devices shows the topmost item (e.g., workspace A). Upon detecting (402, 1302), regardless of the path used to produce the static 55 an MTM gesture, the IBSM 114 can invoke an axonometric posture. For example, consider the case of FIGS. 6 and 7. The view. In this view, the display surface illustrates the individual IBSM 114 may detect that the user performs a first gesture members in the collection of items. For example, the axono when he or she holds the computing device 402 in a generally metric view may simulate the rotation of the stack of items in upright position, as shown in FIG. 6. More specifically, the manner Such that its depth-related dimension in now visible. IBSM 114 may detect that the user performs this gesture 60 The user can then selector otherwise interact with any item in when the user holds the computing device 402 at an angle that the collection of items. In the alternative case of FIG. 18, the exceeds a prescribed threshold angle (relative to a horizontal display Surface presents a collection of items in a carousel plane), for a period of time that exceeds a prescribed threshold type format in response to detecting that the user has per time (while simultaneously applying at least two contacts to formed (or is performing) an MTM gesture. the display surface 408). The IBSM 114 may detect that the 65 In another case, the IBSM 114 can modify an interaction user performs a second gesture when he or she holds the experience in response to the determining that the user has computing device 402 such that the display surface 408 is performed (or is performing) an MTM gesture. The user US 8,902,181 B2 19 20 interaction experience defines any manner in which the com puting devices and/or other target destination(s), or otherwise puting device 100 presents information to the user and/or share the item(s) with the nearby computing device(s). receives input from the user. As a general principle, the IBSM In another case, the IBSM 114 can perform a setting/ 114 can present an interface experience to the user which resetting operation in response to a determination that the facilitates the user's interaction with the computing device user has performed (or is performing) an MTM gesture. For 100 for a current condition which is presumed to affect the example, the IBSM 114 can erase the contents of one or more user. The IBSM 114 infers the current condition of the user workspaces in response to the user flipping the computing based on the gesture that the user has performed (or is per device 402 over and back again in the manner shown in FIG. forming), in optional conjunction with other evidence. For 4. Or the IBSM114 can reset (or set) a form, a control setting, example, the IBSM 114 can conclude that the user may be 10 a game state, etc. in response to the type of movement shown viewing the content on the display Surface at a more oblique in FIG. 4. Alternatively, if the computing device 402 has a angle for the static posture shown in FIG. 7, compared to the display surface on both of front side and back side, this static posture shown in FIG. 6. The IBSM114 can respond by gesture can be executed by just flipping the computing device displaying textual content in a larger font for the scenario of 402 over once, e.g., by an angle of 180 degrees. The user can FIG. 7 (compared to the scenario of FIG. 6), so as to make the 15 then proceed to work on the “back' display surface of the content easier to read in the static posture shown in FIG. 7. computing device 402, which has been cleared by the tilting In another case, the IBSM 114 can also change the options moVement. available to the user in response to determining that the user In another case, the IBSM 114 can control an editing func has performed (or is performing) an MTM gesture. For tion in response to a determination that the user has performed example, the IBSM 114 can provide a first set of toolbar (or is performing) an MTM gesture. For example, assume that buttons for the static posture shown in FIG. 6 and a second set the user is creating a drawing using a drawing application. of toolbar buttons for the static posture shown in FIG. 7. The IBSM 114 can modulate any visual attribute of a graphi In another case, the IBSM 114 can also change the orien cal item (e.g., a thickness, color, hue, etc.) in response to an tation at which content is presented on a display Surface in MTM gesture. For example, the user can designate a graphi response to determining that the user has performed (or is 25 cal item and then perform the tilting movement shown in FIG. performing) an MTM gesture. For example, the IBSM 114 4 to change Some visual attribute of that item in a continuous can change from a landscape mode to a portrait mode in manner; for example, the angular displacement of the com response to the pivoting operation shown in FIG. 8, or vice puting device 402 may define hue, size, or transparency, etc. WSa. assigned to the item in a continuous manner. In another case, the IBSM 114 can perform a Zooming 30 In the particular case of the booklet computing device 1302 operation, panning operation, and/or scrolling operation in of FIG. 13, the IBSM114 can interpreta rapid flip of a section response to any of the MTM gestures described above. For (1304 or 1306) relative to another section as a request to flip example, the IBSM 114 can control a Zoom level in a con a page or chapter, bookmark a page or chapter, delete a page tinuous manner based on an angular displacement produced or chapter, move an item from one section to another, and so by the tilting movement shown in FIG. 4. 35 O. In another case, the IBSM 114 can activate a particular In some of the cases described above, the IBSM 114 can interface page in response to determining that the user has control a designated item in a fluid manner in response to performed (or is performing) an MTM gesture, such a par detecting that a gesture has been performed or is being per ticular home page or the like. formed. For example, in response to the ongoing detection of In another case, the IBSM 114 can invoke a particular 40 the gesture, the IBSM 114 can set a parameter, set a range of application or operating system function (or any other func values, specify a movement path, etc. that affects the item in tion) in response to determining that the user has performed a real-time manner. The item may correspond to an icon or (or is performing) an MTM gesture. The application may other graphical object that appears in a presentation, a region correspond to functionality that is provided at a local location of a presentation, an entire application screen, a tool, a com (with respect to the computing device 100) and/or at a remote 45 mand, and so forth. The user may designate the item in any location. For example, in one case the application may cor manner described above, Such as by touching the item with a respond to locally-stored application program that executes finger, framing the item with two or more fingers, and so on. on a runtime system of the computing device 100. In another To repeat, the actions set forth above are described by way case, the application may correspond to a remote web page or of example, not limitation. Any implementation can associate web service, etc. 50 any invoking behavior with any gesture to achieve any envi In another case, the IBSM 114 can performany action on a ronment-specific objective. designated item in response to determining that the user has B. Illustrative Processes performed (or is performing) an MTM gesture. For example, FIG. 19 shows a procedure 1900 that represents an over the user can touch a particular item on the display Surface of view of one manner of operation of the interpretation and the computing device 100 with one or more fingers, and then 55 behavior selection module (IBSM) 114 of FIGS. 1 and 2. move the computing device 100 in a telltale manner. The Since the principles underlying the operation of the IBSM IBSM 114 may interpret this combination of behavior as a 114 have already been described in Section A, certain opera request to perform some pre-determined behavior on the tions will be addressed in Summary fashion in this section. designed item, such as by deleting or erasing the item, trans In block 1902, the IBSM 114 receives touch input infor ferring the item from a first location to a second location, 60 mation from the touch input mechanism(s) 108 in response to changing the visual appearance of the item, changing the contact made with a surface of the computing device 100. In order of the item with respect to other items and so on. For block 1904, the IBSM 114 receives movement input infor example, in the example of FIG. 11, the user can use the mation from the movement input mechanism(s) 110 in finger(s) of hand 406 to designate an item (or items), with the response to movement of the computing device 100. In block other hand 404 holding the computing device 402 from its 65 1906, the IBSM 114 determines whether the touch input underside. The IBSM114 interprets the gesture as a request to information and the movement input information correspond transfer the designated item(s) to one or more nearby com to a multi-touch-movement (MTM) gesture. As explained in US 8,902,181 B2 21 22 Section A, a user performs an MTM gesture by establishing provisions of the functionality. The functionality can also two or more contacts with a surface of the computing device provide Suitable security mechanisms to ensure the privacy of 100, in conjunction with moving the computing device 100 in the user data (such as data-Sanitizing mechanisms, encryption a prescribed manner. In some cases, the user applies the mechanisms, password-protection mechanisms, etc.). contacts using two hands which are also used to grasp the Further, the description may have described various con computing device 100. In block 1908, the IBSM 114 per cepts in the context of illustrative challenges or problems. forms any behavior that is associated with the MTM gesture This manner of explanation does not constitute an admission that has been detected in block 1906. that others have appreciated and/or articulated the challenges C. Representative Computing Functionality or problems in the manner specified herein. FIG. 20 sets forth illustrative computing functionality 2000 10 Although the subject matter has been described in lan that can be used to implement any aspect of the functions guage specific to structural features and/or methodological described above. For example, the computing functionality acts, it is to be understood that the subject matter defined in 2000 can be used to implement any aspect of the IBSM 114. the appended claims is not necessarily limited to the specific In one case, the computing functionality 2000 may corre features or acts described above. Rather, the specific features spond to any type of computing device that includes one or 15 and acts described above are disclosed as example forms of more processing devices. In all cases, the computing func implementing the claims. tionality 2000 represents one or more physical and tangible What is claimed is: processing mechanisms. 1. A method, performed by a tablet computing device, for The computing functionality 2000 can include volatile and detecting gestures, comprising: non-volatile memory, such as RAM 2002 and ROM 2004, as receiving touch input information from at least one touch well as one or more processing device(s) 2006 (e.g., one or input mechanism, in response to a user touching the more CPUs, and/or one or more GPUs, etc.). The computing computing device; functionality 2000 also optionally includes various media receiving movement input information from at least one devices 2008, such as a hard disk module, an optical disk movement input mechanism, in response to movement module, and so forth. The computing functionality 2000 can 25 of the computing device; perform various operations identified above when the pro determining whether the touch input information and the cessing device(s) 2006 executes instructions that are main movement input information indicate that the user has tained by memory (e.g., RAM 2002, ROM 2004, and/or else performed or is performing a multi-touch-movement where). gesture, More generally, instructions and other information can be 30 where the multi-touch-movement gesture entails: stored on any computer readable medium 2010, including, establishing two or more contacts with a surface of the but not limited to, static memory storage devices, magnetic computing device by a user who grasps the computing storage devices, optical storage devices, and so on. The term device with two hands during use; and computer readable medium also encompasses plural storage moving the computing device to establish a prescribed devices. In all cases, the computer readable medium 2010 35 posture; represents some form of physical and tangible entity. after determining that the user has performed a multi The computing functionality 2000 also includes an input/ touch-movement gesture, evaluating data from one or output module 1712 for receiving various inputs (via input more sensors of the tablet computing device to deter modules 2014), and for providing various outputs (via output mine whether a predefined telltale noise signature rep modules). One particular output mechanism may include a 40 resenting a predefined pattern of behavior has occurred; presentation module 2016 and an associated graphical user and interface (GUI) 2018. The computing functionality 2000 can cancelling implementation of the multi-touch-movement also include one or more network interfaces 2020 for gesture when the predefined telltale noise signature has exchanging data with other devices via one or more commu occurred. nication conduits 2022. One or more communication buses 45 2. The method of claim 1, wherein said two or more con 2024 communicatively couple the above-described compo tacts correspond to two hand contacts placed at two respective nents together. points along a peripheral region of the Surface of the comput The communication conduit(s) 2022 can be implemented ing device. in any manner, e.g., by a local area network, a wide area 3. The method of claim 2, wherein the two hand contacts network (e.g., the Internet), etc., or any combination thereof. 50 are placed on or near two corners of the computing device. The communication conduit(s) 2022 can include any combi 4. The method of claim 2, wherein the peripheral region nation of hardwired links, wireless links, routers, gateway includes two opposing edges have respective lengths, and functionality, name servers, etc., governed by any protocol or wherein the two hand contacts are placed on or near points combination of protocols. that are midway along the respective lengths of the two edges. Alternatively, or in addition, any of the functions described 55 5. The method of claim 1, wherein said two or more con in Sections A and B can be performed, at least in part, by one tacts include at least a first contact placed on a first Surface of or more hardware logic components. For example, without the computing device, and a second contact placed on a sec limitation, illustrative types of hardware logic components ond surface of the computing device, the first Surface and the that can be used include Field-programmable Gate Arrays second Surface having normals pointed in generally opposite (FPGAs), Application-specific Integrated Circuits (ASICs), 60 directions. Application-specific Standard Products (ASSPs), System 6. The method of claim 2, further comprising moving the on-a-chip systems (SOCs), Complex Programmable Logic computing device to establish a prescribed path that corre Devices (CPLDs), etc. sponds to a tilting movement whereby the computing device In closing, functionality described herein can employ vari is rotated about an axis that runs generally through the two ous mechanisms to ensure the privacy of user data maintained 65 hand contacts. by the functionality. For example, the functionality can allow 7. The method of claim 2, further comprising moving the a user to expressly opt into (and then expressly opt out of) the computing device to establish a prescribed path that corre US 8,902,181 B2 23 24 sponds to a flipping movement whereby the computing logic configured to determine whether the touch input device is rotated about an axis that runs generally along an information and the movement input information indi edge of the computing device. cate that the user has made or is making a multi-touch 8. The method of claim 2, further comprising moving the movement gesture by: computing device to establish a prescribed path that corre determining that the user has applied at least two hand sponds to a pivoting movement whereby the computing contacts at two respective points along a peripheral device is rotated in a plane that is generally parallel to the region of a display Surface of the computing device, Surface of the computing device. using two hands that are also used to grasp the com 9. The method of claim 2, wherein the two hand contacts puting device; and correspond to a first hand contact and a second hand contact, 10 determining that the user has rotated the computing and further comprising moving the computing device to device in a prescribed manner while touching the establish a prescribed path that corresponds to a twisting display Surface with said at least two hand contacts; movement whereby the first hand contact is moved towards a after determining that the user has performed a multi reference point and the second hand contact is moved away 15 touch-movement gesture, evaluating data from one or from the reference point. more sensors of the tablet computing device to deter 10. The method of claim 1, wherein the multi-touch-move mine whether a predefined telltale noise signature rep ment gesture is defined, at least in part, based on a manner in resenting a predefined pattern of behavior has occurred; which the computing device is moved over a prescribed path, and as characterized by one or more of a speed of the computing logic configured perform an action in response to deter device; an acceleration of the computing device; and a decel mining that the user has performed a multi-touch-move eration of the computing device. ment gesture unless the predefined telltale noise signa 11. The method of claim 1, wherein the prescribed posture ture has occurred. defines a prescribed tilt of a display Surface of the computing 19. A booklet computing device, comprising: device. 25 at least a first section and a second section; 12. The method of claim 1, wherein the computing device a hinged coupling mechanism for movably coupling the includes at least a first second section and a second section, first section to the second section, each section compris wherein said two or more contacts correspond to two hand ing a separate display device; contacts, a first hand contact placed on a Surface of the first section, and a second hand contact placed on a 30 at least one touch input mechanism configured to receive Surface of the second section, touch input information in response to a user touching wherein said moving involves moving the first section with the first section or the second section, or both the first respect to the second section. section and the second section; 13. The method of claim 1, further comprising invoking a at least one movement input mechanism configured to view in response to determining that the user has performed 35 receive movement input information in response to or is performing a multi-touch-movement gesture, where the movement of the booklet computing device, including: view shows individual items in a layered collection of items. movement applied to both the first section and the sec 14. The method of claim 1, further comprising invoking an ond section; and interaction experience in response to determining that the 40 movement of the first section relative to the second sec user has performed or is performing a multi-touch-movement tion; and gesture, where the interaction experience facilitates interac a gesture matching module configured to: tion with the computing device for a condition that is assumed receive the touch input information from said at least one to affect the user in a current state. touch input mechanism, and the movement input 15. The method of claim 1, further comprising invoking a 45 information from said at least one movement input toolbar selection in response to determining that the user has mechanism; performed or is performing a multi-touch-movement gesture. determine whether the touch input information and the 16. The method of claim 1, further comprising invoking an movement input information indicate that a user has interface page in response to determining that the user has performed or is performing a multi-touch-movement performed or is performing a multi-touch-movement gesture. 50 17. The method of claim 1, further comprising invoking an gesture, application or an operating system function in response to where the multi-touch-movement gesture entails: determining that the user has performed or is performing a establishing two or more contacts with the booklet multi-touch-movement gesture. computing device, using two hands that are also 18. A computer readable storage device for storing com 55 used to grasp the booklet computing device; and puter readable instructions, the computer readable instruc tions providing an interpretation and behavior selection mod moving the sections of the booklet computing device ule (IBSM) when the instructions are executed by one or more relative to each other; processing devices, the computer readable instructions com a noise cancelling mechanism that, after determining that prising: 60 the user has performed a multi-touch-movement ges logic configured to receive touch input information from at ture, evaluates data from one or more sensors of the least one touch input mechanism of a tablet computing booklet computing device to determine whether a pre device, in response to a user touching the computing defined telltale noise signature representing a predefined device; pattern of behavior has occurred; and logic configured to receive movement input information 65 wherein the noise cancelling mechanism cancels imple from at least one movement input mechanism, in mentation of the multi-touch-movement gesture when response to movement of the computing device; the predefined telltale noise signature has occurred. US 8,902,181 B2 25 26 20. The booklet computing device of claim 19, wherein the gesture matching module is configured to detect different multi-touch-movement gestures for different static poses that are maintained for a prescribed period of time that achieve different respective angular separations between the first sec 5 tion and the second section. k k k k k