US008836842B2

(12) United States Patent (10) Patent N0.: US 8,836,842 B2 Sirpal et a1. (45) Date of Patent: *Sep. 16, 2014

(54) CAPTURE MODE OUTWARD FACING H05K5/04 (2013.01); G06F1/1641(2013.01); MODES G06F 3/04883 (2013.01); H05K 13/00 Applicant: Z124, George Town (KY) (2013.01); B29D 11/00673 (2013.01); G06F (71) 1/1637(2013.01); G09G 5/34 (2013.01); G09G (72) Inventors: Sanjiv Sirpal, Oakville (CA); 1/00 (2013.01); G06T3/00 (2013.01); G06F Mohammed Selim, Oakville (CA); 3/1423 (2013.01); G06F 3/0412 (2013.01); John S. Visosky, Gormley (CA) G09G 5/00 (2013.01); G06F 3/01 (2013.01); (73) Assignee: Z124, George Town (KY) G06F 3/048 (2013.01) USPC ...... 348/333.06; 348/333.05 ( * ) Notice: Subject to any disclaimer, the term of this Field of Classi?cation Search patent is extended or adjusted under 35 (58) U.S.C. 154(b) by 41 days. CPC ...... H04N 5/23293 USPC ...... 34863301433313 This patent is subject to a terminal dis See application ?le for complete search history. claimer. (56) References Cited App1.No.: 13/627,498 (21) U.S. PATENT DOCUMENTS Filed: Sep. 26, 2012 (22) 5,982,429 A * 11/1999 Kamamoto etal. 348/333.06 6,188,432 B1* 2/2001 Ejima ...... 348/240.99 (65) Prior Publication Data (Continued) US 2013/0076929 A1 Mar. 28, 2013 OTHER PUBLICATIONS Related US. Application Data Image Result for Fujitsu Dual Screen Phone, published date (60) Provisional application No. 61/539,884, ?led on Sep. unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: w. 27, 2011. google.com/imgres7imgurl:http://www.computerriver.com/im ages/dual-screen-ph0nejpg. (51) Int. Cl. H04N 5/222 (2006.01) (Continued) G06F 1/16 (2006.01) Primary Examiner * Tuan Ho (Continued) (74) Attorney, Agent, or Firm * Sheridan Ross RC. (52) US. Cl. CPC ...... G06F 9/44 (2013.01); G06F 1/1616 (57) ABSTRACT (2013.01); H04M1/0266 (2013.01); G06F Methods and devices for selectively presenting a user inter 3/1433 (2013.01); H05K 7/02 (2013.01); H05K face in a dual screen device. More particularly, the method 5/0226 (2013.01); H05K 5/0017 (2013.01); includes a change in the display mode of a multiple screen G06F 3/041 (2013.01); H05K 13/046 device while the device is in an image capture mode. The (2013.01); E05D 3/12 (2013.01); G06F 3/0403 device may present different information on the two screens (2013.01); G023 6/0001 (2013.01); H04N based on several factors including the mode of the image 5/44591 (2013.01); 6an 1/1601 (2013.01); capture feature, the orientation of the device, and/or the state 6an 3/0401 (2013.01); G06F 3/04097 of the device. (2013.01); G06F 3/1446 (2013.01); H05K 7/1452 (2013.01); 6an 9/4443 (2013.01); 20 Claims, 38 Drawing Sheets US 8,836,842 B2 Page 2

(51) Int. Cl. page. Retrieved from: www.google.com/imgres?imgurl:http:// H04M 1/02 (2006.01) fgadgets.com/wp-content/uploads/2010/08/lg-dual-touch-screen H05K 7/02 (2006.01) phone-Eugene-Kim-O 1 .jpg. H05K 5/02 (2006.01) Google Image Result for Fujitsu Dual Screen Phone, published date H05K 5/00 (2006.01) unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: w. G06F 3/041 (2006.01 ) google.com/imgres?imgurl:http://www.gsmdome.com/wp-con H05K 13/04 (2006.01) tent/uploads/2010/ 10/ fuj itsu-dual-screen-phoneiw2cP7i54. jpg. E05D 3/12 (2006.01) Google Image Result for Echo, published date unknown, G06F 3/0483 (2013.01) [retrieved Apr. 18, 201 1], 1 page. Retrieved from: www.google.com/ F21V8/00 (2006.01) imgres?imgurl:http://www.hardwaresphere.com/wp-content/up H04N5/445 (2011.01) loads/201 1/ 02/ kyocera-echo-dual-screen-android-phone-for-sprint G06F 3/0481 (2013.01) network.jpg. G06F 3/0489 (2013.01) Google Image Result for HTC Triple Viper, published date unknown, H05K 7/14 (2006.01) [retrieved Apr. 18, 2011], 1 page. Retrieved from:www.google.com/ G06F 9/44 (2006.01 ) imgres?imgurl:http://www.santafemods.com/Forum/ H05K 5/04 (2006.01) AndroidForums/htcTripleViperpng. G06F 3/0488 (2013.01) Google Image Result for Dual-Screen Phone, [retrieved Apr. 18, H05K 13/00 (2006.01) 201 1], 1 page. Retrieved from: www.google.com/ B29D 11/00 (2006.01) imgres?imgurl:http://www.netshet.org/wp-content/uploads/201 1/ G09G 5/34 (2006.01 ) 02/Dual-Scree . . . . G09G 1/00 (2006.01 ) Website entitled, “Kyocera Echo,” , Inc., G06T 3/00 (2006.01) 2011, [retrieved on Aug. 27, 2012], 6 pages. Retrieved from: w. echobykyocera.com/ . G09G 5/00 (2006.01 ) Website entitled “LapdockTM for AtriX,” Motorola Mobil G06F 3/01 (2006.01 ) ity, Inc, 2011, [retrieved on Apr. 18, 2011], 1 page Retrieved from: G06F 3/048 (2013.01) www.motorola.com/Consumers/US-EN/Consumer-Product-and

G06F 3/14 (2006.01) Services/Mobile+Ph . . . . Website entitled “Motorola AtriX 4G Laptop Dock Review,” (56) References Cited phoneArena.com, posted Mar. 2, 2011, [retrieved on Apr. 18, 2011], 6 pages. Retrieved from: www.phonearena.com/reviews/Motorola U.S. PATENT DOCUMENTS ATRIX-4G-Laptop-Dock-Reviewiid2667.

7,626,630 B2 * 12/2009 Nishino et al...... 348/33306 Website entitled, “Sony Tablet,” Sony Corporation, 2012, [retrieved 2005/0212942 A1 9/2005 Sakai on Aug. 27, 2012], 3 pages. Retrieved from: www.store.sony.com/ 2007/0035616 A1 2/2007 Lee et a1. webapp/wcs/stores/servlet/CategoryDisplay?catalogId: 1055 l 2007/0057866 A1 3/2007 Lee et a1. &storeId:10151&langId:-1&categoryIdI81985 52921644795 521. 2007/0249389 A1 10/2007 Hotta Burns, G, “Motorola AtriX 4G Laptop Dock Review,” Android Com 2008/0059888 A1 3/2008 Dunko munity, Feb. 20, 2011, [retrieved on Apr. 18, 2011], 5 pages. 2008/0231740 A1 9/2008 McIntyre et a1. Retrieved from: www.androidcommunity.com/motorola-atriX-4g 2009/0278806 A1 11/2009 Duarte et al. laptop-dock-review-201 10220/. 2009/0298548 A1 12/2009 Kim et al. Catacchio, “This has two huge screens . . . that rotate,” 2010/0039350 A1 2/2010 Wake?eld et al. The Next Web, Inc., Oct. 7, 2010, [retrieved on Jul. 21, 2011], 2 2010/0060664 A1 3/2010 Kilpatrick, II et al. pages. Retrieved from: www.thenextweb.com/asia/2010/10/07/this 2010/0173678 A1 7/2010 Kim et al. 2010/0227650 A1 9/2010 Kim et al. smartphone -has-two -huge-screens-that-rotate/ . 2011/0003616 A1 1/2011 Gorsica et al. Posted by Harman03, “Kyocera Echo Dual-screen Android Phone,” 2011/0050975 A1 3/2011 Chung et al. posted 4 weeks from Apr. 18, 2011, [retrieved on Apr. 18, 2011], 3 2011/0134269 A1 6/2011 Kim pages. Retrieved from: www.unp.me/fl 06/kyocera-echo-dual 2011/0243397 A1 10/2011 Watkins et al. screen-android-phone- 143 800/. 2011/0267530 A1 11/2011 Chun Stein, S., “How does the Motorola AtriX 4G Lapdock compare with a 2011/0317031 A1 12/2011 Honda laptop?” Crave4CNET, Feb. 9, 2011 [retrieved on Apr. 18, 2011], 7 2012/0242599 A1 9/2012 Seo et a1. pages. Retrieved from: www.news.cnet.com/8301-17938i105 2013/0021262 A1 1/2013 Chen 20031251-1. htrnl. 2013/0076961 A1 3/2013 Sirpal Of?cial Action for US. Appl. No. 13/627,400, mailed Jan. 17,2014, 2013/0076963 A1 3/2013 Sirpal et al. 2013/0076964 A1 3/2013 Sirpal et al. 12 pages. 2013/0086492 A1 4/2013 Sirpal et al. Of?cial Action for U.S.Appl. No. 13/627,453 mailed Feb. 4, 2014, 14 pages. OTHER PUBLICATIONS Of?cial Action for US. Appl. No. 13/627,422 mailed May 9, 2014, 19 pages. Google Image Result for LG Dual Touch Screen Concept Phone by Eugene Kim, published date unknown, [retrieved Apr. 18, 2011], 1 * cited by examiner US. Patent Sep. 16, 2014 Sheet 1 0f 38 US 8,836,842 B2

112

" 110 114 US. Patent Sep. 16, 2014 Sheet 2 0f 38 US 8,836,842 B2

100 1567 S L Q/ “i 1 1002‘ FIG. 1F 5-100

100 E5" 144 158 5 5—132 E r148 6-.9—1728? “3* Uf152 160 5—136

176 128

5— 140

FIG-1E 3-1728] 164? FIG. 10 \t ii: {5-)}

FIG. 1C

5 100 C. E. 7 172a-5 ¢172b

FIG. 1G US. Patent Sep. 16, 2014 Sheet 3 0f 38 US 8,836,842 B2 US. Patent Sep. 16, 2014 Sheet 4 0f 38 US 8,836,842 B2

__§°Ee_"1___ )"120 124“\ ____S_C_r§er_12__ | Gesture Capture l/ l Gesture Capture |

r M m — ~ w *mlj104 108kr — — — — ~ -——-| l | l l | | l | Touch Sensitive Touch Sensitive I I| o ispay| I atI Display | | F1110 1145/ i l | | l l l | l l L?e?e‘ezewe?yjL... u m w m m ...... 112 116 LiEQI?'?éEié'eTéfe'élL... “ ~ U H m ...... M A v jzma j 208 )— 212 ; 204 v i216b Display Data Display commuter Memory Storage Processor Controller A A A A A

l v v v r v e

224 228 V Cellular Power Source and 220 K / Telephony < ‘; Power control Module Module 232 Antenna ; 252 1 1 260 )— 256 \ Other Wireless Port Integrated 236 1 communication 4 > Interface Audio l/O GPS Module A A

4 V V V \ ‘ A ¢ A A A A r

v _ _ v v v v Hardware Spos't'O" Accelerometer(s) Image Audio HO HO Buttons en80r(s) Capture Interface Module / l / / / / 158 £ 172 g 176 g 240$ 244 g 248 i US. Patent Sep. 16, 2014 Sheet 5 0f 38 US 8,836,842 B2

100 O J 0:) ID 336

Landscape

308

328 hnage VMeo Captur

316

Mod?bd Easel

344

Docked 348 Landscape FIG. 3A

US. Patent Sep. 16, 2014 Sheet 7 0f 38 US 8,836,842 B2

4323 428a Tap ‘_~___5-420 FIG. 4A 432b 428b Long Press H“;— 424

FIG. 4B 432 428D 436 Drag *w____5"400

FIG. 4C 432 428b )— 436 Flick M 404

FIG. 4D 4328 432b 4283 4363 436b

Pinch M;- 408 428b FIG. 4E

432a 432b 436a; 43Gb Spread :@ Q 410 4288 428b FIG. 4F US. Patent Sep. 16, 2014 Sheet 8 0f 38 US 8,836,842 B2

420'lxiiiia 428b 432b 400 I

436

FIG. 46

432a 432b 400 420 K 42% ‘2

428a 436

FIG. 4H

US. Patent Sep. 16, 2014 Sheet 10 0f 38 US 8,836,842 B2

524 K Multi-Display Management Module 568 K Display Con?guration Module

572 k Preferences Module

Device State Module

Gesture Module 580 K Requirements Module

Event Module

7\ Binding Module

FIG. 5B US. Patent Sep. 16, 2014 Sheet 11 0f 38 US 8,836,842 B2

)114 \608

Q FIG.6B 1081

)110/ \\604

P1 @ FIG.6A

US. Patent Sep. 16, 2014 Sheet 14 0f 38 US 8,836,842 B2

FIG.6F 1101 US. Patent Sep. 16, 2014 Sheet 15 0f 38 US 8,836,842 B2

6207 114 FIG.6H

108

104 108

104

FIG.66

110 US. Patent Sep. 16, 2014 Sheet 16 0f 38 US 8,836,842 B2

108 V 2110 X <1: 3O FIG.6/ US. Patent Sep. 16, 2014 Sheet 17 0f 38 US 8,836,842 B2 US. Patent Sep. 16, 2014 Sheet 18 0f 38 US 8,836,842 B2

wowJ,

wowM mauvva 263:5m :03on w>Ewcww 5&5: m>som£ @285; £025 m>Ewcmw :5&5 :8&wa 552$ 5$620