(12) United States Patent (10) Patent No.: US 9.235,374 B2 Sirpal Et Al

US009235374B2

(12) United States Patent (10) Patent No.: US 9.235,374 B2 Sirpal et al. (45) Date of Patent: Jan. 12, 2016

(54) SMARTPAD IDUAL SCREEN KEYBOARD (58) Field of Classification Search WITH CONTEXTUAL LAYOUT USPC ...... 345/169, 173-179 See application file for complete search history. Applicant: Z124, Grand Cayman (KY) (71) (56) References Cited (72) Inventors: Sanjiv Sirpal, Oakville (CA); Martin Gimpl. Helsinki (FI) U.S. PATENT DOCUMENTS 3,704.950 A 12, 1972 Rosencranz (73) Assignee: Z124, George Town (KY) 6,021,269 A 2/2000 Lewis (Continued) (*) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 FOREIGN PATENT DOCUMENTS U.S.C. 154(b) by 76 days. CN 1896946 1, 2007 (21) Appl. No.: 13/627,191 CN 10.1038737 9, 2007 (Continued) (22) Filed: Sep. 26, 2012 OTHER PUBLICATIONS (65) Prior Publication Data Official Action for U.S. Appl. No. 13/247,581, mailed Sep. 27, 2013 US 2013 FOOT6638A1 Mar. 28, 2013 101 pages. (Continued) Related U.S. Application Data Primary Examiner — Waseem Moorad (60) Provisional application No. 61/539,884, filed on Sep. Assistant Examiner — Sujit Shah 27, 2011. (74) Attorney, Agent, or Firm — Sheridan Ross P.C. (57) ABSTRACT (51) Int. C. G06F 3/4 (2006.01) A multi-display device is adapted to be dockable or otherwise G06F L/I6 (2006.01) associatable with an additional device. In accordance with one exemplary embodiment, the multi-display device is (Continued) dockable with a Smartpad. The exemplary Smartpad can (52) U.S. C. include a screen, a touch sensitive display, a configurable CPC ...... G06F 3/1423 (2013.01); B29D 1 1/00673 area, a gesture capture region(s) and a camera. The Smartpad (2013.01); E05D3/12 (2013.01); G02B 6/0001 can also include a port adapted to receive the device. The (2013.01); G06F 1/1601 (2013.01); G06F exemplary Smartpad is able to cooperate with the device Such I/I616 (2013.01); G06F I/I637 (2013.01); that information displayable on the device is also displayable G06F 1/1641 (2013.01); G06F 1/1692 on the Smartpad. Furthermore, any one or more of the func (2013.01); G06F 3/01 (2013.01); G06F 3/041 tions on the device are extendable to the smartpad, with the (2013.01); G06F 3/048 (2013.01); G06F Smartpad capable of acting as an input/output interface or 3/0412 (2013.01); G06F 3/0481 (2013.01); extension of the Smartpad. Therefore, for example, informa G06F 3/0483 (2013.01); G06F 3/04883 tion from one or more of the displays on the multi-screen (2013.01); device is displayable on the Smartpad. (Continued) 20 Claims, 134 Drawing Sheets

US 9.235,374 B2 Page 2

(51) Int. Cl. 8.732,373 B2 5/2014 Sirpal et al. 2001/00484.82 A1 12/2001 Takemoto G06F 9/44 (2006.01) 2003/0025678 A1 2/2003 Lee et al...... 345,173 G06F 3/048. I (2013.01) 2003. O189594 A1 10, 2003 Jones G06F 3/0483 (2013.01) 2004/0150635 A1 8/2004 Clapper G09G 5/14 (2006.01) 3.339: A 39 St.IncKley B29D II/00 (2006.01) 2006/0033724 A1 2/2006 Chaudhri et al...... 345,173 E05D3/12 (2006.01) 2006/0034042 A1 2/2006 Hisano et al. F2/8/00 (2006.01) 2006, O1835.05 A1 8, 2006 Willrich

G06F 3/01 (2006.01) 398-99. A 3: Nata.goIn et al. ge. Ot 2.8 2006/0248404 A1 1 1/2006 Lindsay et al. (2013.01) 2007/00 16873 A1 1/2007 Lindsay G06F 3/0489 (2013.01) 2007/0024524 A1 2/2007 Lai G06T3/00 (2006.01) 2007, 0079027 A1 4, 2007 Marriott et al.

G09G I/00 (2006.01) 32.82. A 399, St.COWa. G09G 5/00 (2006.01) 2007, 0120762 A1 5, 2007 O'Gorman G09G 5/34 (2006.01) 2007/0120763 A1 5/2007 De Paepe et al. H04M I/02 (2006.01) 2007/0130399 A1 6/2007 Anderson et al. H04N 5/445 (2011.01) 2007. O139719 A1 6, 2007 Gotham et al. H047 72/06 (2009.01) 2007. O198948 A1 8/2007 Toriyama 2007/0216700 A1 9, 2007 Chen et al. H04788/06 (2009.01) 2007/02424.21 A1 10, 2007 Goschin et al. H05K 5/00 (2006.01) 2007.0245256 A1 10, 2007 BOSS et al. H05K 5/02 (2006.01) 2007/0294.263 A1 12/2007 Punjet al. H05K 5/04 (2006.01) 2008.0062625 A1 3/2008 Batio H05K 7/02 (2006.01) 2008/0082934 A1* 4/2008 Kocienda et al...... 71.5/773 H05K 7/4 (2006.01) 2008/O122796 A1* 5/2008 JobS et al...... 345,173 2008/O168367 A1 7/2008 Chaudhri et al. H05K I3/00 (2006.01) 2009/006 1950 A1 3/2009 Kamachi et al. H05K I3/04 (2006.01) 2009/0094523 A1 4, 2009 Treder et al.

GO6F 3/0488 (2013.01) 3.29s. A $299. Sightleymetal. (52) (e.18 (2009.01) 2009,0187677 A1 7, 2009 Hunt et al. AV e. we 2009, 0235281 A1 9, 2009 Lu et al. CPC ...... G06F 3/04897 (2013.01); G06F 3/1438 2009/0237,420 A1 9, 2009 Lawrenz

(2013.01); G06F 3/1446 (2013.01); G06F 9/44 38885 A. 1939 D.C.uarte et et al al. (2013.01); EC(2013.01); 2009,2.5i. O303205 A1A 12/20092 Seibertin et al. (2013.01); G09G 5/14 (2013.01); G09G 5/34 2009/0320070 Al 12/2009 Inoguchi (2013.01); H04M I/0266 (2013.01); H04N 2009/0322693 A1* 12/2009 Sasakura ...... 345,173 5/44591 (2013.01); H04W 72/06 (2013.01); 2009/0327561 Al 12/2009 Groesch H04W 88/06 (2013.01), H05K5/0017 303. A E SEE E..UT), E..UT), 2010,E.g.: 0037175 A1 339.2/2010 West (2013.01); H05K 13/00 (2013.01); H05K 2010.00457.05 A1 2/2010 Vertegaal et al. 13/046 (2013.01); E05Y 2900/606 (2013.01); 2010.0053089 A1 3/2010 Kwok et al. G06F 1/16 (2013.01); G06F 3/048.s (2013.01): 2993.2010/0081475 A1A * 4/2010299 ChiangSpatricket et al. al...... 455,564 H04W 48/18 (2013.01); Y10T 16/547 2010/0085382 A1 4/2010 Lundqvist et al. (2015.01); Y10T 29/4984 (2015.01); Y10T 2010.0088.634 A1 4/2010 Tsuruta et al. 29/49826 (2015.01) 2010/0095240 A1 4/2010 Shiplacoffetal. 2010, 0138780 A1 6, 2010 Marano et al. 2010/0177047 A1 7/2010 Brenneman et al. (56) References Cited 2010/0182247 A1 7/2010 Petschnigg et al. 2010, 0182248 A1 7, 2010 Chun U.S. PATENT DOCUMENTS 2010.0188321 A1 7, 2010 Yen 2010/0225448 A1 9/2010 Arguin 6,182,231 B1 1/2001 Gilgen 2010/0227642 A1 9, 2010 Kim et al. 6,243,645 B1 $38, M cal 2010/0238788 A1 9, 2010 Boutros et al. 7,070,755 B2 7/2006 Klett et al. 2010.0245209 A1 9, 2010 Miller et al. 7,136,282 B1 1 1/2006 Rebeske 2010.0245369 A1 9, 2010 Yoshino 7,477,205 B1 1399. de Waal et al. 1 2010/0248788 A1* 9, 2010 Yook et al...... 455,566 E33. i. 2.9 Nyaswamy et al. 2010/0257014 A1 10, 2010 Roberts et al. 7765.143 B 7200 get al. 2010/0299436 A1* 11/2010 Khalid et al...... TO9,226 7777.730 B2 & 2010 2011/0047459 A1 2/2011 Van Der Westhuizen 7844.301 B2 1/2010 type 2011/0047503 A1 2/2011 Allen et al...... 715,784 RE43.738 E 9/2011 Williams 2011/0072391 A1 3/2011 Hanggie et al. 8018,398 B1 9/2011 Harris 2011/0074692 A1* 3/2011 Causey et al...... 345/169 8085.216 B2 12/2011 Miller 2011/0095965 A1 4/2011 Yoneoka et al. 8,243,198 B2 8, 2012 Lee et al. 2011/009931.6 A1 4/2011 Tseng et al. 8.411,069 B1 4/2013 Huang 2011/00993.92 A1 4/2011 Conway 8,600,437 B2 12/2013 Kumar 2011/0107272 A1 5/2011 Aguilar 8,633,867 B2 1/2014 Kim et al. 2011/0126141 A1 5/2011 King et al. 8,706.275 B2 4/2014 Peterson et al. 2011/0134097 A1 6/2011 Sugamata US 9.235,374 B2 Page 3

(56) References Cited Official Action for U.S. Appl. No. 13/247,621, mailed Oct. 2, 2013 19 pageS. U.S. PATENT DOCUMENTS Official Action for U.S. Appl. No. 13/247,663, mailed Nov. 1, 2013 28 pages. 2011/O143769 A1 6/2011 Jones et al. 2011/O145758 A1 6/2011 Rosales et al. station for U.S. Appl. No. 13/247,696, mailed Oct. 15, 2013 2011/O1599.29 A1 6, 2011 K tal. 2011 0164042 A1* T/2011 E. ea ...... 345/473 Official Action for U.S. Appl. No. 13/247,71 l, mailed Oct. 18, 2013 2011/O165890 A1 7, 2011 Ho et al. 25 pages. 2011/O187753 A1 8, 2011 Watanabe Official Action for U.S. Appl. No. 13/247,724, mailed Oct. 18, 2013 2011 01993.92 A1 8, 2011 Nishikawa 30 pages. 2011/0202449 A1 8, 2011 West Official Action for U.S. Appl. No. 13/208,828, mailed Oct. 29, 2013 2011/0209058 A1 8, 2011 Hinckley et al. 15 pages. SE. N. 33. NE." "... as Official Action for U.S. Appl. No. 13408,839, mailed Oct. 28, 2013 2011/0219317 A1* 9, 2011 Thatcher et al...... 715/752 28 pages. 2011/0225537 A1* 9/2011 Wang et al...... 71.5/778 Official Action for U.S. Appl. No. 13/627,326, mailed Oct. 1, 201357 2011/0225.553 A1 9/2011 Abramson et al. pageS. 2011/0230.178 A1* 9, 2011 Jones et al...... 455,422.1 Official Action for U.S. Appl. No. 13/247,696, mailed Dec. 19, 2013 2011/0234515 A1 9/2011 Kamijima 26 pages. 2011/0239142 A1* 9, 2011 Steeves et al...... 715,764 & 8 s 2011/0246904 A1 10, 2011 Pinto et al. This sing lation User's Manual Libretto W100. Jun. 2011/0267291 A1 11, 2011 Choi et al. First edition, 119 pages. 2012/0001829 A1 1/2012 Anttila et al. Stern, “Toshiba Libretto W100 preview.” posted Jul. 1, 2010 at www. 2012/0040719 A1 2/2012 Lee et al. engadget.com/2010/07/01/toshiba-libretto-wl 00-preview?, 4 pages. 2012/0042102 A1 2/2012 Chung et al. International Preliminary Report on Patentability for International 2012, 0043937 A1 2/2012 Williams (PCT) Patent Application No. PCT/US2011/054039, mailed Apr. 11, 2012/0054401 A1 3/2012 Cheng 2013 7 pages. 2012,006O109 A1 3/2012 Han et al. International Preliminary Report on Patentability for International 2012/0081292 A1 4/2012 Sirpal et al. (PCT) Patent Application No. PCT/US2011/054046, mailed Apr. 11, 2012,0081303 A1 4/2012 Cassar et al. 2013 10 pages. 2839. A. 13: SE al. International Preliminary Report on Patentability for International 2012,0081312 A1 4/2012 Sirpal et al. (PCT) Patent Application No. PCT/US2011/054042, mailed Apr. 11, 2012/008 1313 A1 4/2012 Sirpal et al. 2013 10 pages. 2012,0081314 A1 4/2012 Sirpal et al. Official Action for U.S. Appl. No. 13/247,621, mailed Apr. 3, 2013, 2012/008 1398 A1 4/2012 Sirpal et al. 16 pages. 2012,0081399 A1 4/2012 Sirpal et al. Official Action for U.S. Appl. No. 13/247,663, mailed May 8, 2013, 2012,0081403 A1 4/2012 Sirpal et al. 22 pages. 2012,0081854 A1 4/2012 Sirpal et al. Official Action for U.S. Appl. No. 13/247,711, mailed Apr. 25, 2013, 2012,0084698 A1 4/2012 Sirpal et al. 21 pages. 2012,0084723 Al 4, 2012 Reeves et al. Official Action for U.S. Appl. No. 13/247,724, mailed May 8, 2013, 2012/008.7078 A1* 4/2012 Medica et al...... 361/679.31 26 pages. 58-386 A 383 SAS a U.S. Appl. No. 13/364, 152, filed Feb. 1, 2012, Sirpal et al. 2012/0131496 A1 5/2012 GoossensImpl et al. et al. U.S. Appl. No. 13/408,828, filed Feb. 29, 2012, Sirpal et al. 2012. O151403 A1 6/2012 Bhogal et al. U.S. Appl. No. 13/408,839, filed Feb. 29, 2012, Sirpal et al. 2012/0176396 A1 7/2012 Harper et al. U.S. Appl. No. 13/627,153, filed Sep. 26, 2012, Sirpal et al. 2012fO235922 A1 9, 2012 Locker et al. U.S. Appl. No. 13/627, 168, filed Sep. 26, 2012, Sirpal et al. 2012fO24.0056 A1 9, 2012 Webber U.S. Appl. No. 13/627,269, filed Sep. 26, 2012, Sirpal et al. 2012fO266098 A1 10, 2012 Webber U.S. Appl. No. 13/627,290, filed Sep. 26, 2012, Sirpal et al. 2012/0274656 Al 1 1/2012 Kang et al. U.S. Appl. No. 13/627,326, filed Sep. 26, 2012, Sirpal et al. 2012/0282914 A1 11/2012 Alexander U.S. Appl. No. 13/627,352, filed Sep. 26, 2012, Sirpal et al. 2013/0021262 A1 1/2013 Chen U.S. Appl. No. 13/627,390, filed Sep. 26, 2012, Sirpal et al. 39.83 A. $38 Resis et al...... 7575 U.S. Appl. No. 13/627,415, filed Sep. 26, 2012, Sirpal et al. 2013/0076637 A1 3, 2013 Telt U.S. Appl. No. 13/627,445, filed Sep. 26, 2012, de Paz et al. 2013,011 1369 A1 5/2013 Pasquero2. et al. U.S. Appl. No. 13/628,432, filed Sep. 27, 2012, Jouin. 2013,0120251 A1 5, 2013 Lee et al. Google Image Result for Fujitsu Dual Screen Phone, published date 2013,0120267 A1 5/2013 Pasquero et al. unknown, retrieved Apr. 18, 2011), 1 page. Retrieved from: www. 2013/O127687 A1 5, 2013 Yu google.com/imgres?imgurl=http://www.computerriver.com/im 2013/0159934 A1 6/2013 Kang ages/dual-screen-phone.jpg. 2013/0305185 Al 11/2013 Nicol, II et al. Google Image Result for LG Dual Touch Screen Concept Phone by 2015,0070824 A1 3/2015 Sirpal et al. Eugene Kim, published date unknown, retrieved Apr. 18, 2011, 1 2015/0097797 A1 4/2015 Sirpal et al. page. Retrieved from: www.google.com/imgres?imgurl=http:// fgadgets.com/wp-content/uploads/2010/08/lg-dual-touch-screen FOREIGN PATENT DOCUMENTS phone-Eugene-Kim-01.jpg. Google Image Result for Fujitsu Dual Screen Phone, published date CN 101477442 T 2009 unknown, retrieved Apr. 18, 2011), 1 page. Retrieved from: www. WO WO 2010/110954 9, 2010 google.com/imgres?imgurl=http://www.gsmdome.com/wp-con OTHER PUBLICATIONS tent/uploads/2010/10/fujitsu-dual-screen-phone w2cP7 54.jpg. Google Image Result for Kyocera Echo, published date unknown, Notice of Allowance for U.S. Appl. No. 13/247,581, mailed Nov. 4. retrieved Apr. 18, 2011, 1 page. Retrieved from: www.google.com/ 2013 12 pages. imgres?imgurl=http://www.hardwaresphere.com/wp-content/up Official Action for U.S. Appl. No. 13/247,606, mailed Sep. 27, 2013 loads/2011/02/kyocera-echo-dual-screen-android-phone-for-sprint 48 pages. network.jpg. Official Action for U.S. Appl. No. 13/247,606, mailed Dec. 9, 2013 Google Image Result for HTC Triple Viper, published date unknown, 36 pages. retrieved Apr. 18, 2011, 1 page. Retrieved from:www.google.com/ US 9.235,374 B2 Page 4

(56) References Cited Notice of Allowance for U.S. Appl. No. 13/408,839, mailed Jul. 9, 2014 8 pages. OTHER PUBLICATIONS Official Action for U.S. Appl. No. 13/627,168, mailed Jun. 20, 2014 18 pages. imgres?imgurl=http://www.santafemods.com/Forum/ Official Action for U.S. Appl. No. 13/627,269, mailed Sep. 11, 2014 AndroidForums/htcTripleViper.png. 15 pages. Google Image Result for Dual-Screen Phone, retrieved Apr. 18. Official Action for U.S. Appl. No. 13/627,290, mailed Jul. 21, 2014 2011, 1 page. Retrieved from: www.google.com/ 10 pages. imgres?imgurl=http://www.netshet.org/wp-content/uploads/2011/ Notice of Allowance for U.S. Appl. No. 13/627.326, mailed Jul. 9, 02/Dual-Scree . . . . 20146 pages. Website entitled “Editor Info' Android Developers, updated Nov. 17. Official Action for U.S. Appl. No. 13/627.352 mailed Aug. 20, 2014 2011, 6 pages, retrieved from (web.archive.org/web/ 11 pages. 2011 1128081741/http://developer.android.com/reference?android? Official Action for U.S. Appl. No. 13/627,415, mailed Aug. 1, 2014 viewfinputmethod/EditorInfo.html). 17 pages. Website entitled, “Kyocera Echo.” Kyocera Communications, Inc., Official Action for U.S. Appl. No. 13/627,445, mailed Jul. 28, 2014 2011, retrieved on Aug. 27, 2012), 6 pages. Retrieved from: www. 12 pages. echobykyocera.com/. Extended European Search Report for European Patent Application Website entitled “LapdockTM for Motorola ATRIX.” Motorola No. 11829920.5, dated Apr. 17, 2014, 7 pages. Mobility, Inc., 2011, retrieved on Apr. 18, 2011), 1 page. Retrieved Official Action for Mexican Patent Application No. MX/a/2013/ from: www.motorola.com/Consumers/US-EN/Consumer-Product 003176, dated Mar. 13, 2014, 2 pages. and-Services/Mobile+Ph. . . . Notice of Allowance for U.S. Appl. No. 13/247,606, mailed Feb. 28, Website entitled “Motorola ATRIX 4G Laptop Dock Review.” 2014 10 pages. phoneArena.com, posted Mar. 2, 2011, retrieved on Apr. 18, 2011). Official Action for U.S. Appl. No. 13/247,621, mailed Mar. 26, 2014 6 pages. Retrieved from: www.phonearena.com/reviews/Motorola 17 pages. ATRIX-4G-Laptop-Dock-Review id2667. Official Action for U.S. Appl. No. 13/247,711, mailed Mar. 21, 2014 Website entitled, “Sony Tablet.” Sony Corporation, 2012, retrieved 26 pages. on Aug. 27, 2012, 3 pages. Retrieved from: www.store.Sony.com/ Official Action for U.S. Appl. No. 13/247,724, mailed Dec. 26, 2013 webapp/wcs/stores/servlet/CategoryDisplay?catalog Id=10551& 30 pages. storeId=10151&langld=-1&categoryld=8198552921644795521. Official Action for U.S. Appl. No. 13/408,828, mailed Dec. 30, 2013 Wikipedia"iOS' last modified Nov. 29, 2011, 7 pages, retrieved from 19 pages. (web.archive.org/web/2011 113.0025243/http://en.wikipedia.org/ Official Action for U.S. Appl. No. 13/408,828, mailed May 13, 2014 wiki/IOS). 19 pages. Burns, C., “Motorola ATRIX 4G Laptop Dock Review.” Android Official Action for U.S. Appl. No. 13/408,839, mailed Dec. 30, 2013 Community, Feb. 20, 2011, retrieved on Apr. 18, 2011, 5 pages. 19 pages. Retrieved from: www.androidcommunity.com/motorola-atrix-4g Official Action for U.S. Appl. No. 13/408,839, mailed May 14, 2014 laptop-dock-review-20110220/. 17 pages. Catacchio, “This Smartphone has two huge screens . . . that rotate.” Official Action for U.S. Appl. No. 13/627,168, mailed Apr. 25, 2014 The Next Web, Inc., Oct. 7, 2010, retrieved on Jul. 21, 2011, 2 23 pages. pages. Retrieved from: www.thenextweb.com/asia/2010/10/07/this Official Action for U.S. Appl. No. 13/627,269, mailed May 15, 2014 Smartphone-has-two-huge-screens-that-rotate?. 11 pages. Posted by Harman03, “Kyocera Echo Dual-screen Android Phone.” Official Action for U.S. Appl. No. 13/627,326, mailed Dec. 31, 2013 posted 4 weeks from Apr. 18, 2011, retrieved on Apr. 18, 2011). 3 48 pages. pages. Retrieved from: www.unp.meff106/kyocera-echo-dual Notice of Allowance for U.S. Appl. No. 13/627.326, mailed May 20, screen-android-phone-143800?. 2014.9 pages. Stein, S., “How does the Motorola Atrix 4G Lapdock compare with a Official Action for U.S. Appl. No. 13/627,352, mailed Mar. 24, 2014 laptop?” Crave CNET, Feb. 9, 2011 retrieved on Apr. 18, 2011), 7 9 pages. pages. Retrieved from: www.news.cnet.com/8301-17938 105 “iPad User Guide for iOS 4.2 Software.” Apple Inc., 2010, 181 pages. 20031251-1.html. Screen shot from: “Samsung Galaxy Tab (T-Mobile) review part 1 International Search Report and Written Opinion for International of 2.” Nov. 20, 2010, retrieved on Jul. 7, 2014), 1 page. Retrieved (PCT) Patent Application No. PCT/US2011/054039, mailed Feb. 21, from: http://www.youtube.com/watch?v=PZAaFbIDoOc. 2012 8 pages. Victor “Capturing Video from Nintendo DS.” Screen Capture News, International Search Report and Written Opinion for International Jul. 7, 2009, 5 pages retrieved from www.screencapturenews.com/ (PCT) Patent Application No. PCT/US2011/054046, mailed Feb. 22, 2009/07/07/capturing-video-from-nintendo-ds. 2012 12 pages. Notice of Allowance for Mexican Patent Application No. MX a? International Search Report and Written Opinion for International 2013/003176, dated May 16, 2014, 1 pages. (PCT) Patent Application No. PCT/US2011/054042, mailed Feb. 21, Official Action for U.S. Appl. No. 13/247,621, mailed Oct. 23, 2014 2012 12 pages. 26 pages. Screen shots from: “Toshiba Libretto W100 demo”. Aug. 19, 2010, Official Action for U.S. Appl. No. 13/247,634, mailed Oct. 22, 2014 retrieved on Jul. 7, 2014), 10 pages. Retrieved from: www.youtube. 11 pages. com/watch?v=WYb6di083oE. Official Action for U.S. Appl. No. 13/247,647, mailed Jun. 10, 2014 Lewis, Screen shots from: “Gotomypc ipad is finally available 41 pages. today.” Feb. 27, 2011 retrieved on Jul. 3, 2014, 5 pages. Retrieved Notice of Allowance for U.S. Appl. No. 13/247.663, mailed Jun. 6. from: www.youtube.com/watch?v=6gmasfDLeU. 2014.9 pages. Official Action for U.S. Appl. No. 13/247,634, mailed Jun. 6, 2014 10 Notice of Allowance for U.S. Appl. No. 13/247,696, mailed Oct. 1, pageS. 2014.9 pages. Official Action for U.S. Appl. No. 13/247,696, mailed Aug. 6, 2014 Notice of Allowance for U.S. Appl. No. 13/247,711, mailed Oct. 1, 12 pages. 2014 10 pages. Official Action for U.S. Appl. No. 13/628,432, mailed Sep. 11, 2014 Official Action for U.S. Appl. No. 13/247,733, mailed Sep. 25, 2014 21 pages. 51 pages. Official Action for U.S. Appl. No. 13/627,153 mailed Jul. 21, 2014 12 Official Action for U.S. Appl. No. 13/627,352, mailed Nov. 13, 2014 pageS. 13 pages. Notice of Allowance for U.S. Appl. No. 13/408,828, mailed Jul. 9, “iPad Status Bar Indicators' My-iGuru.com, May 31, 2011 retrieved 2014 8 pages. on Jan. 10, 2015), 3 pages. Retrieved from: https://web.archive.org/ US 9.235,374 B2 Page 5

(56) References Cited Official Action for U.S. Appl. No. 13/627,390, mailed Jan. 15, 2015 16 pages. OTHER PUBLICATIONS Official Action for U.S. Appl. No. 13/627,415, mailed Dec. 9, 2014 21 pages. web/20110531064820/http://www.my-iguru.com/ipad/ipad-hints Official Action for U.S. Appl. No. 13/627,415, mailed Mar. 26, 2015, tipstipad-status-bar-indicators.php. 23 pages. Official Action for U.S. Appl. No. 13/247,634, mailed Jan. 30, 2015 Official Action for U.S. Appl. No. 13/627,445, mailed Jan. 7, 2015 11 12 pages. pageS. Official Action and Search Report (with English translation) for Notice of Allowance for U.S. Appl. No. 13/247,634, mailed Mar. 19. Chinese Patent Application No. 201180051513.X. dated Mar. 20. 20157 pages. 2015. Official Action for U.S. Appl. No. 13/247,647, mailed Jan. 15, 2015 Extended European Search Report for European Patent Application 49 pages. No. 11829918.9, dated May 4, 2015, 8 pages. Official Action for U.S. Appl. No. 13/628,432, mailed Mar. 12, 2015 English Translation of Official Action for China Patent Application 15 pages. No. 201180051463.5, dated Feb. 27, 2015 12 pages. Notice of Allowance for U.S. Appl. No. 13/627,153 mailed Jan. 22. Official Action for U.S. Appl. No. 13/247,621, mailed Jun. 18, 2015 20157 pages. 19 pages. Official Action for U.S. Appl. No. 13/627, 168, mailed Mar. 27, 2015 Notice of Allowance for U.S. Appl. No. 13/247,724, mailed Jul. 10, 25 pages. 2014 10 pages. Notice of Allowance for U.S. Appl. No. 13/627,290, mailed Jan. 27. Notice of Allowance for U.S. Appl. No. 13/247.733, mailed Apr. 24. 20157 pages. 2015 13 pages. Official Action for U.S. Appl. No. 13/627,352, mailed Feb. 13, 2015 5 pages. * cited by examiner U.S. Patent Jan. 12, 2016 Sheet 1 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 2 of 134 US 9.235,374 B2

100 156 100. FIG. 1F to

144 132 148 152 136

140

FIG. 1E F.G. 1D U.S. Patent Jan. 12, 2016 Sheet 3 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 4 of 134 US 9.235,374 B2

Touch Sensitive Touch Sensitive Display

Configurable Area Configurable Area

26a 208 212 204

Display Data Display Controller Memory Storage ProCeSSOr Controller

224 - 228 Cellular POWer Source and Telephony POWer Control Module Module

Other WireleSS Port integrated 236 GPS Communication interface Audio /O Module

Hardware sic Accelerometer(s) image Audio /O Buttons ensor(s) Capture interface

FIG 2 U.S. Patent Jan. 12, 2016 Sheet 5 of 134 US 9.235,374 B2

FIG. 3A

U.S. Patent Jan. 12, 2016 Sheet 7 of 134 US 9.235,374 B2

432a 428a Tap - S. 420 FIG. 4A 432b 428b Long Press - S. 424

FIG. 4B 432 428b. 4 36 Drag -- S400

FIG. 4C 432 428b. 9. 436 Fick - S. 404

FIG. 4D 432a 432b 428a 436a 436b Pinch -S 408 428b FIG. 4E

432a 432b 136a) 4360 Spread -- S410 428a 428b U.S. Patent Jan. 12, 2016 Sheet 8 of 134 US 9.235,374 B2

t 428b 432b - 400 428a S 2-D- --S 412 436

FIG. 4G

432a 432b 420, 428b. - 400 428a 436

FIG. 4H. U.S. Patent US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 10 of 134 US 9.235,374 B2

524 Q Multi-Display Management Module 568 S. Display Configuration Module

FIG. 5B U.S. Patent Jan. 12, 2016 Sheet 11 of 134 US 9.235,374 B2

t 3 wn reas

3. U.S. Patent Jan. 12, 2016 Sheet 12 of 134 US 9.235,374 B2

CO C CC

s U.S. Patent Jan. 12, 2016 Sheet 13 of 134 US 9.235,374 B2

l O g o l CC

O CN 9 o l

s U.S. Patent Jan. 12, 2016 Sheet 14 of 134 US 9.235,374 B2

s 3

CN wa CO U.S. Patent Jan. 12, 2016 Sheet 15 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 16 of 134 US 9.235,374 B2

O w ver

S. s U.S. Patent Jan. 12, 2016 Sheet 17 of 134 US 9.235,374 B2

s CO O wa

S. U.S. Patent US 9.235,374 B2

U.S. Patent US 9.235,374 B2

------

g 7 U.S. Patent Jan. 12, 2016 Sheet 20 of 134 US 9.235,374 B2

()09/

U.S. Patent Jan. 12, 2016 Sheet 22 of 134 US 9.235,374 B2

()008 U.S. Patent Jan. 12, 2016 Sheet 23 of 134 US 9.235,374 B2

900 S.

904

908 Receive activation of a Window

912 Determine associated display

916 Determine position in stack

920 Generate logic structure

924

FIG. 9 U.S. Patent Jan. 12, 2016 Sheet 24 of 134 US 9.235,374 B2

1004 to

1020 U.S. Patent Jan. 12, 2016 Sheet 25 of 134 US 9.235,374 B2

110 : 1OO

FIG. 11 U.S. Patent Jan. 12, 2016 Sheet 26 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 27 of 134 US 9.235,374 B2

100

Landscape

1010

FIG. 13B

1OO

C Portrait Portrait

1010 U.S. Patent Jan. 12, 2016 Sheet 28 of 134 US 9.235,374 B2

1004 ) 1 OOO

1012b-S 1012C FIG. 14 U.S. Patent Jan. 12, 2016 Sheet 29 of 134 US 9.235,374 B2

FIG. 15 U.S. Patent Jan. 12, 2016 Sheet 30 of 134 US 9.235,374 B2

1004 to

Dual Screen Application - 1020 MaX Mode Portrait U.S. Patent Jan. 12, 2016 Sheet 31 of 134 US 9.235,374 B2

Dual Screen Application – Max Mode Landscape

Window 1 Window 2

FIG. 17 U.S. Patent Jan. 12, 2016 Sheet 32 of 134 US 9.235,374 B2

1004 1000

808

Application Area

F.G. 18 U.S. Patent Jan. 12, 2016 Sheet 33 of 134 US 9.235,374 B2

1004 100

1908

Application Area

Max Mode

------, ------, ------, ------, ------...... 9...9. - 1804

FIG. 19 U.S. Patent Jan. 12, 2016 Sheet 34 of 134 US 9.235,374 B2

Type to Compose 1 || 2 || 3 || 4 || 5 || 6 || 7 || 8 || 9 | O a w E R T Y. U of P

FIG. 20 U.S. Patent Jan. 12, 2016 Sheet 35 of 134 US 9.235,374 B2

FIG. 21 U.S. Patent Jan. 12, 2016 Sheet 36 of 134 US 9.235,374 B2

F.G. 22

FIG. 23 U.S. Patent Jan. 12, 2016 Sheet 37 of 134 US 9.235,374 B2

1004 9. 1 OOO

FIG. 24 U.S. Patent Jan. 12, 2016 Sheet 38 of 134 US 9.235,374 B2

1 OOO

FIG. 25 U.S. Patent Jan. 12, 2016 Sheet 39 Of 134 US 9.235,374 B2

2618too

1020

2604

2608

2616

2612

FIG. 26 U.S. Patent Jan. 12, 2016 Sheet 40 of 134 US 9.235,374 B2

FIG. 27 U.S. Patent Jan. 12, 2016 Sheet 41 of 134 US 9.235,374 B2

too 2618 3 Giff) is A. d/, it

1010

FIG. 28 U.S. Patent Jan. 12, 2016 Sheet 42 of 134 US 9.235,374 B2

2908

2912

2916

FIG. 29 U.S. Patent Jan. 12, 2016 Sheet 43 of 134 US 9.235,374 B2

9. 1000

Application Area

FIG. 30 U.S. Patent Jan. 12, 2016 Sheet 44 of 134 US 9.235,374 B2

1010

3104 3116 31 20

FIG. 31A

FIG. 31B

316

31 O8 U.S. Patent Jan. 12, 2016 Sheet 45 of 134 US 9.235,374 B2

100

3216 Y. 3204 3208 3220 3224.

FIG. 32E 1010 9. 1 OOO

Ys. YY 322 3216 Yo 3224 U.S. Patent Jan. 12, 2016 Sheet 46 of 134 US 9.235,374 B2

3312 - 336

O / GEF) is d/, /t it

Carrier

Otifications

John Smith X Draft File status? 9:34 am

Pat Cooly D-3 Please schedule meeting

Y. 1000 FIG. 33 U.S. Patent Jan. 12, 2016 Sheet 47 of 134 US 9.235,374 B2

Application launcher 9 1 OOO

1010 FIG. 34A

3116

100 FIG. 34B

1010 U.S. Patent Jan. 12, 2016 Sheet 48 of 134 US 9.235,374 B2

3520 3524.

FIG. 35A

F.G. 35B

3520 (-3524 U.S. Patent Jan. 12, 2016 Sheet 49 of 134 US 9.235,374 B2

3604

Application C

3612

3608

3444 Application D

3416 U.S. Patent Jan. 12, 2016 Sheet 50 of 134 US 9.235,374 B2

3704

Y. 1 OOO FIG. 37 U.S. Patent Jan. 12, 2016 Sheet 51 of 134 US 9.235,374 B2

S3802 START

S3804. Detect insertion of Device into SP

S3806 Begin Power Management

Turn off S3808 Displays of Device

S381 O Establish Communication / Connectivity with SP

S3812 Enable Display and other input/Output Functions on SP

Device Software Settings are S3814 Mapped to SP Hardware Settings

S3816 Automatically Align Screen Orientation of Device to SP

S3818 Last Application in Focus Remains in Focus

S382O END FIG. 38 U.S. Patent Jan. 12, 2016 Sheet 52 of 134 US 9.235,374 B2

S3900 START

S3902 Detect SP Orientation

S3904 Detect Application(s) Orientation(s) on Device

Reorient Application(s) As S3906 needed Based on Orientation of SP

FIG. 39 U.S. Patent Jan. 12, 2016 Sheet 53 of 134 US 9.235,374 B2

S4000 START

S4002

S4004 Landscape Orientation?

S4O1 O Display Keyboard in

andscape Mode POrtrait

Display Keyboard in S4012

Orientation Portrait Mode Change?

NO g Orientation Change?

S4O16 F.G. 40 S4018 U.S. Patent Jan. 12, 2016 Sheet 54 of 134 US 9.235,374 B2

S4100 START

S4102

S4104

Detect Gesture Direction

Call Next Application Window into View, Scrolling Currently Viewed Application Window into Stack

Gesture Detected?

S41 10

FIG. 41 U.S. Patent Jan. 12, 2016 Sheet 55 Of 134 US 9.235,374 B2

S4200 START

S42O2 Enable Multiple Application Mode

Detect Selection of S4204 Applications to be Displayed

Split Screen - 1 Portion for S42O6 First Application and 2" Portion for Second Application

Detect S4208 Application in Focus

Highlight in-FOCUS Application

FIG. 42 U.S. Patent Jan. 12, 2016 Sheet 56 of 134 US 9.235,374 B2

S4300 START

Determine if Application can be Maximized

Maximizable?

S4306 Expand Application to Dual Screen Mode or Max Mode

S4308

F.G. 43 U.S. Patent Jan. 12, 2016 Sheet 57 Of 134 US 9.235,374 B2

S4400

S4402 Detect Last Application in Application Window Stack

Detect Gesture Requesting S44.04 Next Window at End of the Stack

FIG. 44 U.S. Patent Jan. 12, 2016 Sheet 58 of 134 US 9.235,374 B2

S4500 START

Display S4502 Desktop on SP

Logically Divide S4504 Desktop on SmartPad

Display 1 Screen of Desktop in S4506 One Logical Portion of SP Display

Display 2nd Screen of Desktop S4508 in Second Logical Portion of SP Display

Allow Carouse S4510 Movement of "Panels" S4512 FIG. 45 U.S. Patent Jan. 12, 2016 Sheet 59 of 134 US 9.235,374 B2

S4600 START

Display S4602 Desktop on SP

logically Divide S4604 Desktop on SmartPad

Display 1 Screen of Desktop in S4608 One Logical Portion of SP Display

Display 2nd Screen of Desktop S461 O Bridging First and Second Logical Portion of SP Display

Display 3rd Screen of Desktop S4612 in Second Logical Portion of SP Display

Allow Carouse S4614 Movement of "Panels' S4616 FIG. 46 END U.S. Patent Jan. 12, 2016 Sheet 60 of 134 US 9.235,374 B2

S4700 START

S4702 Detect ACCeSS Desktop Request

S4704

Display At least One Desktop Panel on Display Partially Display at least SS706 One Other Desktop Panel On Display

Upon Gesture-Based Request, Completely S4708 Display Partially Displayed Desktop Panel on Display FIG. 47 U.S. Patent Jan. 12, 2016 Sheet 61 of 134 US 9.235,374 B2

S4800 START

S4802 Enter Multiple Application Mode

Arrange Stack with One Or More S4804 Applications Being Partially Visible Behind a First Application

Optionally Arrange Stack with One Or S4806 More Applications Being Partially Visible Behind a Second Application

Upon Receipt of Gesture, Allow Carousel-Like Scrolling S4808 Through Stack Until End of Stack, or Allow Continuous "Circular” Scrolling through the Stack in an Alternative Embodiment

END S4810 FIG. 48 U.S. Patent Jan. 12, 2016 Sheet 62 of 134 US 9.235,374 B2

1 O20

1016 Headphone (1212) 4910 Mic

FIG. 49 U.S. Patent Jan. 12, 2016 Sheet 63 of 134 US 9.235,374 B2

1010

FIG. 50B

1000 100

Portrait

100 U.S. Patent Jan. 12, 2016 Sheet 64 of 134 US 9.235,374 B2

Headphone Line U.S. Patent Jan. 12, 2016 Sheet 65 of 134 US 9.235,374 B2

000 1010

1020

F.G. 52 U.S. Patent Jan. 12, 2016 Sheet 66 of 134 US 9.235,374 B2

1020

F.G. 53 U.S. Patent Jan. 12, 2016 Sheet 67 of 134 US 9.235,374 B2

pe?oeaup?oused?,6u?uueae)Á?duueJeeN| U.S. Patent Jan. 12, 2016 Sheet 68 of 134 US 9.235,374 B2

1004 9. OOO

1808

Application Area

Optionally Assigned % TransparenC

FIG. 55 U.S. Patent Jan. 12, 2016 Sheet 69 of 134 US 9.235,374 B2

1004 9. 1OOO

Application Area 1

808

Application Area 2

| 2 || 3 || 4 || 5 6 7 8 9 0. - 5604

F.G. 56 U.S. Patent Jan. 12, 2016 Sheet 70 of 134 US 9.235,374 B2

Application Area

Type to Compose | 1 || 2 || 3 || 4 || 5 || 6 || 7 || 8 || 9 || 0 |

FIG. 57 U.S. Patent Jan. 12, 2016 Sheet 71 of 134 US 9.235,374 B2

FIG. 58 U.S. Patent Jan. 12, 2016 Sheet 72 of 134 US 9.235,374 B2

5908

FIG. 59 U.S. Patent Jan. 12, 2016 Sheet 73 of 134 US 9.235,374 B2

F.G. 60 U.S. Patent Jan. 12, 2016 Sheet 74 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 75 Of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 76 of 134 US 9.235,374 B2

5902

% a w e y u o p

Tab 3. S d f 9 h j Enter SHF z x c v b n m shift 123 e | Bck De

6302

FIG. 63 U.S. Patent Jan. 12, 2016 Sheet 77 of 134 US 9.235,374 B2

64.04

U.S. Patent Jan. 12, 2016 Sheet 78 of 134 US 9.235,374 B2

------(------, ------(------, ------

------. ------, ------K------, ------shift z x civil bin m. shift 123 a Bck De

F.G. 65 U.S. Patent Jan. 12, 2016 Sheet 79 of 134 US 9.235,374 B2

F.G. 66 U.S. Patent Jan. 12, 2016 Sheet 80 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 81 of 134 US 9.235,374 B2

F.G. 69 U.S. Patent Jan. 12, 2016 Sheet 82 of 134 US 9.235,374 B2

Dock to Portrait Tablet

FIG. 70

Dock to Portrait Tablet U.S. Patent Jan. 12, 2016 Sheet 83 of 134 US 9.235,374 B2

100

FIG. 72

U.S. Patent Jan. 12, 2016 Sheet 84 of 134 US 9.235,374 B2

FIG. 73

: U.S. Patent Jan. 12, 2016 Sheet 85 of 134 US 9.235,374 B2

Dock to Portrait Tablet

Dock to Portrait Tablet U.S. Patent Jan. 12, 2016 Sheet 86 of 134 US 9.235,374 B2

Portrait U.S. Patent Jan. 12, 2016 Sheet 87 of 134 US 9.235,374 B2

Dock to Portrait Tablet

Portrait Tablet U.S. Patent Jan. 12, 2016 Sheet 88 of 134 US 9.235,374 B2

FIG. 80

U.S. Patent Jan. 12, 2016 Sheet 89 of 134 US 9.235,374 B2

FIG. 81

U.S. Patent Jan. 12, 2016 Sheet 90 of 134 US 9.235,374 B2

FIG. 82

Max Mode U.S. Patent Jan. 12, 2016 Sheet 91 of 134 US 9.235,374 B2

FIG. 83

Max Mode U.S. Patent Jan. 12, 2016 Sheet 92 of 134 US 9.235,374 B2

U.S. Patent Jan. 12, 2016 Sheet 93 Of 134 US 9.235,374 B2

000; ?

Jan. 12, 2016 Sheet 94 of 134 US 9.235,374 B2

: X: ×

000?.

axssssssssssss

&&& ·ººººººººººººººººººººººººººººººººººººººººººººººººººº U.S. Patent Jan. 12, 2016 Sheet 95 Of 134 US 9.235,374 B2

000£

------

8. 8. 8 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8. 8.