US008836842B2
(12) United States Patent (10) Patent N0.: US 8,836,842 B2 Sirpal et a1. (45) Date of Patent: *Sep. 16, 2014
(54) CAPTURE MODE OUTWARD FACING H05K5/04 (2013.01); G06F1/1641(2013.01); MODES G06F 3/04883 (2013.01); H05K 13/00 Applicant: Z124, George Town (KY) (2013.01); B29D 11/00673 (2013.01); G06F (71) 1/1637(2013.01); G09G 5/34 (2013.01); G09G (72) Inventors: Sanjiv Sirpal, Oakville (CA); 1/00 (2013.01); G06T3/00 (2013.01); G06F Mohammed Selim, Oakville (CA); 3/1423 (2013.01); G06F 3/0412 (2013.01); John S. Visosky, Gormley (CA) G09G 5/00 (2013.01); G06F 3/01 (2013.01); (73) Assignee: Z124, George Town (KY) G06F 3/048 (2013.01) USPC ...... 348/333.06; 348/333.05 ( * ) Notice: Subject to any disclaimer, the term of this Field of Classi?cation Search patent is extended or adjusted under 35 (58) U.S.C. 154(b) by 41 days. CPC ...... H04N 5/23293 USPC ...... 34863301433313 This patent is subject to a terminal dis See application ?le for complete search history. claimer. (56) References Cited App1.No.: 13/627,498 (21) U.S. PATENT DOCUMENTS Filed: Sep. 26, 2012 (22) 5,982,429 A * 11/1999 Kamamoto etal. 348/333.06 6,188,432 B1* 2/2001 Ejima ...... 348/240.99 (65) Prior Publication Data (Continued) US 2013/0076929 A1 Mar. 28, 2013 OTHER PUBLICATIONS Related US. Application Data Google Image Result for Fujitsu Dual Screen Phone, published date (60) Provisional application No. 61/539,884, ?led on Sep. unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: w. 27, 2011. google.com/imgres7imgurl:http://www.computerriver.com/im ages/dual-screen-ph0nejpg. (51) Int. Cl. H04N 5/222 (2006.01) (Continued) G06F 1/16 (2006.01) Primary Examiner * Tuan Ho (Continued) (74) Attorney, Agent, or Firm * Sheridan Ross RC. (52) US. Cl. CPC ...... G06F 9/44 (2013.01); G06F 1/1616 (57) ABSTRACT (2013.01); H04M1/0266 (2013.01); G06F Methods and devices for selectively presenting a user inter 3/1433 (2013.01); H05K 7/02 (2013.01); H05K face in a dual screen device. More particularly, the method 5/0226 (2013.01); H05K 5/0017 (2013.01); includes a change in the display mode of a multiple screen G06F 3/041 (2013.01); H05K 13/046 device while the device is in an image capture mode. The (2013.01); E05D 3/12 (2013.01); G06F 3/0403 device may present different information on the two screens (2013.01); G023 6/0001 (2013.01); H04N based on several factors including the mode of the image 5/44591 (2013.01); 6an 1/1601 (2013.01); capture feature, the orientation of the device, and/or the state 6an 3/0401 (2013.01); G06F 3/04097 of the device. (2013.01); G06F 3/1446 (2013.01); H05K 7/1452 (2013.01); 6an 9/4443 (2013.01); 20 Claims, 38 Drawing Sheets US 8,836,842 B2 Page 2
(51) Int. Cl. page. Retrieved from: www.google.com/imgres?imgurl:http:// H04M 1/02 (2006.01) fgadgets.com/wp-content/uploads/2010/08/lg-dual-touch-screen H05K 7/02 (2006.01) phone-Eugene-Kim-O 1 .jpg. H05K 5/02 (2006.01) Google Image Result for Fujitsu Dual Screen Phone, published date H05K 5/00 (2006.01) unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: w. G06F 3/041 (2006.01 ) google.com/imgres?imgurl:http://www.gsmdome.com/wp-con H05K 13/04 (2006.01) tent/uploads/2010/ 10/ fuj itsu-dual-screen-phoneiw2cP7i54. jpg. E05D 3/12 (2006.01) Google Image Result for Kyocera Echo, published date unknown, G06F 3/0483 (2013.01) [retrieved Apr. 18, 201 1], 1 page. Retrieved from: www.google.com/ F21V8/00 (2006.01) imgres?imgurl:http://www.hardwaresphere.com/wp-content/up H04N5/445 (2011.01) loads/201 1/ 02/ kyocera-echo-dual-screen-android-phone-for-sprint G06F 3/0481 (2013.01) network.jpg. G06F 3/0489 (2013.01) Google Image Result for HTC Triple Viper, published date unknown, H05K 7/14 (2006.01) [retrieved Apr. 18, 2011], 1 page. Retrieved from:www.google.com/ G06F 9/44 (2006.01 ) imgres?imgurl:http://www.santafemods.com/Forum/ H05K 5/04 (2006.01) AndroidForums/htcTripleViperpng. G06F 3/0488 (2013.01) Google Image Result for Dual-Screen Phone, [retrieved Apr. 18, H05K 13/00 (2006.01) 201 1], 1 page. Retrieved from: www.google.com/ B29D 11/00 (2006.01) imgres?imgurl:http://www.netshet.org/wp-content/uploads/201 1/ G09G 5/34 (2006.01 ) 02/Dual-Scree . . . . G09G 1/00 (2006.01 ) Website entitled, “Kyocera Echo,” Kyocera Communications, Inc., G06T 3/00 (2006.01) 2011, [retrieved on Aug. 27, 2012], 6 pages. Retrieved from: w. echobykyocera.com/ . G09G 5/00 (2006.01 ) Website entitled “LapdockTM for Motorola AtriX,” Motorola Mobil G06F 3/01 (2006.01 ) ity, Inc, 2011, [retrieved on Apr. 18, 2011], 1 page Retrieved from: G06F 3/048 (2013.01) www.motorola.com/Consumers/US-EN/Consumer-Product-and
G06F 3/14 (2006.01) Services/Mobile+Ph . . . . Website entitled “Motorola AtriX 4G Laptop Dock Review,” (56) References Cited phoneArena.com, posted Mar. 2, 2011, [retrieved on Apr. 18, 2011], 6 pages. Retrieved from: www.phonearena.com/reviews/Motorola U.S. PATENT DOCUMENTS ATRIX-4G-Laptop-Dock-Reviewiid2667.
7,626,630 B2 * 12/2009 Nishino et al...... 348/33306 Website entitled, “Sony Tablet,” Sony Corporation, 2012, [retrieved 2005/0212942 A1 9/2005 Sakai on Aug. 27, 2012], 3 pages. Retrieved from: www.store.sony.com/ 2007/0035616 A1 2/2007 Lee et a1. webapp/wcs/stores/servlet/CategoryDisplay?catalogId: 1055 l 2007/0057866 A1 3/2007 Lee et a1. &storeId:10151&langId:-1&categoryIdI81985 52921644795 521. 2007/0249389 A1 10/2007 Hotta Burns, G, “Motorola AtriX 4G Laptop Dock Review,” Android Com 2008/0059888 A1 3/2008 Dunko munity, Feb. 20, 2011, [retrieved on Apr. 18, 2011], 5 pages. 2008/0231740 A1 9/2008 McIntyre et a1. Retrieved from: www.androidcommunity.com/motorola-atriX-4g 2009/0278806 A1 11/2009 Duarte et al. laptop-dock-review-201 10220/. 2009/0298548 A1 12/2009 Kim et al. Catacchio, “This smartphone has two huge screens . . . that rotate,” 2010/0039350 A1 2/2010 Wake?eld et al. The Next Web, Inc., Oct. 7, 2010, [retrieved on Jul. 21, 2011], 2 2010/0060664 A1 3/2010 Kilpatrick, II et al. pages. Retrieved from: www.thenextweb.com/asia/2010/10/07/this 2010/0173678 A1 7/2010 Kim et al. 2010/0227650 A1 9/2010 Kim et al. smartphone -has-two -huge-screens-that-rotate/ . 2011/0003616 A1 1/2011 Gorsica et al. Posted by Harman03, “Kyocera Echo Dual-screen Android Phone,” 2011/0050975 A1 3/2011 Chung et al. posted 4 weeks from Apr. 18, 2011, [retrieved on Apr. 18, 2011], 3 2011/0134269 A1 6/2011 Kim pages. Retrieved from: www.unp.me/fl 06/kyocera-echo-dual 2011/0243397 A1 10/2011 Watkins et al. screen-android-phone- 143 800/. 2011/0267530 A1 11/2011 Chun Stein, S., “How does the Motorola AtriX 4G Lapdock compare with a 2011/0317031 A1 12/2011 Honda laptop?” Crave4CNET, Feb. 9, 2011 [retrieved on Apr. 18, 2011], 7 2012/0242599 A1 9/2012 Seo et a1. pages. Retrieved from: www.news.cnet.com/8301-17938i105 2013/0021262 A1 1/2013 Chen 20031251-1. htrnl. 2013/0076961 A1 3/2013 Sirpal Of?cial Action for US. Appl. No. 13/627,400, mailed Jan. 17,2014, 2013/0076963 A1 3/2013 Sirpal et al. 2013/0076964 A1 3/2013 Sirpal et al. 12 pages. 2013/0086492 A1 4/2013 Sirpal et al. Of?cial Action for U.S.Appl. No. 13/627,453 mailed Feb. 4, 2014, 14 pages. OTHER PUBLICATIONS Of?cial Action for US. Appl. No. 13/627,422 mailed May 9, 2014, 19 pages. Google Image Result for LG Dual Touch Screen Concept Phone by Eugene Kim, published date unknown, [retrieved Apr. 18, 2011], 1 * cited by examiner US. Patent Sep. 16, 2014 Sheet 1 0f 38 US 8,836,842 B2
112
" 110 114 US. Patent Sep. 16, 2014 Sheet 2 0f 38 US 8,836,842 B2
100 1567 S L Q/ “i 1 1002‘ FIG. 1F 5-100
100 E5" 144 158 5 5—132 E r148 6-.9—1728? “3* Uf152 160 5—136
176 128
5— 140
FIG-1E 3-1728] 164? FIG. 10 \t ii: {5-)}
FIG. 1C
5 100 C. E. 7 172a-5 ¢172b
FIG. 1G US. Patent Sep. 16, 2014 Sheet 3 0f 38 US 8,836,842 B2 US. Patent Sep. 16, 2014 Sheet 4 0f 38 US 8,836,842 B2
__§°Ee_"1___ )"120 124“\ ____S_C_r§er_12__ | Gesture Capture l/ l Gesture Capture |
r M m — ~ w *mlj104 108kr — — — — ~ -——-| l | l l | | l | Touch Sensitive Touch Sensitive I I| o ispay| I atI Display | | F1110 1145/ i l | | l l l | l l L?e?e‘ezewe?yjL... u m w m m ...... 112 116 LiEQI?'?éEié'eTéfe'élL... “ ~ U H m ...... M A v jzma j 208 )— 212 ; 204 v i216b Display Data Display commuter Memory Storage Processor Controller A A A A A
l v v v r v e
224 228 V Cellular Power Source and 220 K / Telephony < ‘; Power control Module Module 232 Antenna ; 252 1 1 260 )— 256 \ Other Wireless Port Integrated 236 1 communication 4 > Interface Audio l/O GPS Module A A
4 V V V \ ‘ A ¢ A A A A r
v _ _ v v v v Hardware Spos't'O" Accelerometer(s) Image Audio HO HO Buttons en80r(s) Capture Interface Module / l / / / / 158 £ 172 g 176 g 240$ 244 g 248 i US. Patent Sep. 16, 2014 Sheet 5 0f 38 US 8,836,842 B2
100 O J 0:) ID 336
Landscape
308
328 hnage VMeo Captur
316
Mod?bd Easel
344
Docked 348 Landscape FIG. 3A
US. Patent Sep. 16, 2014 Sheet 7 0f 38 US 8,836,842 B2
4323 428a Tap ‘_~___5-420 FIG. 4A 432b 428b Long Press H“;— 424
FIG. 4B 432 428D 436 Drag *w____5"400
FIG. 4C 432 428b )— 436 Flick M 404
FIG. 4D 4328 432b 4283 4363 436b
Pinch M;- 408 428b FIG. 4E
432a 432b 436a; 43Gb Spread :@ Q 410 4288 428b FIG. 4F US. Patent Sep. 16, 2014 Sheet 8 0f 38 US 8,836,842 B2
420'lxiiiia 428b 432b 400 I
436
FIG. 46
432a 432b 400 420 K 42% ‘2
428a 436
FIG. 4H
US. Patent Sep. 16, 2014 Sheet 10 0f 38 US 8,836,842 B2
524 K Multi-Display Management Module 568 K Display Con?guration Module
572 k Preferences Module
Device State Module
Gesture Module 580 K Requirements Module
Event Module
7\ Binding Module
FIG. 5B US. Patent Sep. 16, 2014 Sheet 11 0f 38 US 8,836,842 B2
)114 \608
Q FIG.6B 1081
)110/ \\604
P1 @ FIG.6A
US. Patent Sep. 16, 2014 Sheet 14 0f 38 US 8,836,842 B2
FIG.6F 1101 US. Patent Sep. 16, 2014 Sheet 15 0f 38 US 8,836,842 B2
6207 114 FIG.6H
108
104 108
104
FIG.66
110 US. Patent Sep. 16, 2014 Sheet 16 0f 38 US 8,836,842 B2
108 V 2110 X <1: 3O FIG.6/ US. Patent Sep. 16, 2014 Sheet 17 0f 38 US 8,836,842 B2 US. Patent Sep. 16, 2014 Sheet 18 0f 38 US 8,836,842 B2
wowJ,
wowM mauvva 263:5m :03on w>Ewcww 5&5: m>som£ @285; £025 m>Ewcmw :5&5 :8&wa 552$ 5$620