(12) United States Patent (10) Patent No.: US 9.225,799 B1 Dong Et Al
Total Page:16
File Type:pdf, Size:1020Kb
US00922.5799B1 (12) United States Patent (10) Patent No.: US 9.225,799 B1 Dong et al. (45) Date of Patent: Dec. 29, 2015 (54) CLIENT-SIDE RENDERING FOR VIRTUAL 6,433,794 B1 8/2002 Beadle et al. MOBILE INFRASTRUCTURE 6,492.995 B1 12/2002 Atkin et al. 6,647,544 B1 1 1/2003 Ryman et al. 6,757,895 B1 6/2004 Beadle et al. (71) Applicant: Trend Micro Incorporated, Tokyo (JP) 6,799,195 B1 9, 2004 Thibault et al. (72) Inventors: Kan Dong, Nanjing (CN);; YaozhouYaoZhou XXu, 6,842,777 B1 1/2005 Tuli Nanjing (CN); Gen Chen, Nanjing (CN) (Continued) (73) Assignee: Trend Micro Incorporated, Tokyo (JP) FOREIGN PATENT DOCUMENTS EP 1316,873 4/2003 (*) Notice: Subject to any disclaimer, the term of this EP 1377,892 9, 2004 patent is extended or adjusted under 35 (Continued) U.S.C. 154(b) by 391 days. (21) Appl. No.: 13/901,379 OTHER PUBLICATIONS (22) Filed: May 23, 2013 Hadeel Tariq Al-Rayes, “Studying Main Differences between Android & Linux Operating Systems', Oct. 2012, pp. 46-49, vol. 12, Related U.S. Application Data No. 5, International Journal of Electrical & Computer Sciences IJECS-IJENS. (60) Provisional application No. 61/825,829, filed on May 21, 2013. (51) Int. Cl Primary Examiner — Alina N Boutah G06F 12/00 (2006.01) (74) Attorney, Agent, or Firm — Okamoto & Benedicto LLP H04L 29/06 (2006.01) GO6F 3/OO (2006.01) (57) ABSTRACT (52) U.S. Cl. CPC ...................................... H04L 67/42 (2013.01) A virtual mobile infrastructure performs client-siderendering (58) Field of Classification Search by intercepting and redirecting screen data for generating a CPC. G06F 9/443; G06F 9/4445; G06F 9/45533; screen image of a remote mobile operating system from a G06F 9/45537; G06F 9/45558; G06F 13/14: server computer to a mobile client device. The mobile client G06F 3/1423: G06F 3/1454: G06F 17/00; device receives the screen data and generates the final Screen G06F 21/53: H04L 67/1095; H04L 67/42: image of the remote mobile operating system. The screen data HO4L 67/10O2 include drawing application programming interface (API) USPC .................. 709/203, 226, 228:345/418, 522; calls to generate Surfaces for applications running on the 71.5/740 remote mobile operating system and data for compositing the See application file for complete search history. surfaces together. The mobile client device makes the draw ing API calls to generate the Surfaces and generates the final (56) References Cited screen image of the remote mobile operating system by com U.S. PATENT DOCUMENTS Ring the Surfaces in accordance with the compositing 6,091,412 A 7/2000 Simonoff et al. 6,356,931 B2 3, 2002 Ismael et al. 6,401,134 B1 6, 2002 Razavi et al. 11 Claims, 13 Drawing Sheets 35? REMOTE ANDROD LOCAL ANDROID MONITOR BATTERY STATUS BATERY IS CLENT BATTERY OWER THAN RENDERING THRESHOLD 55/ 33.5 SERVER RENDERNG US 9.225,799 B1 Page 2 (56) References Cited 2007/0079244 A1* 4/2007 Brugiolo ....................... 715,740 2007/O199058 A1 8/2007 Baumgartet al. U.S. PATENT DOCUMENTS 2009.0113444 A1* 4/2009 Hackborn et al. ............ T19,312 2011/O167474 A1 7/2011 Sinha 6,842,897 B1 1/2005 Beadle et al. 2012/0084542 A1* 4/2012 Reeves et al. ..................... T13/1 6,941,552 B1 9, 2005 Beadle et al. 2012/0084791 A1* 4/2012 Benedek et al. T19, 313 6,976,059 B1 12/2005 Rogalski et al. 2012/0084792 A1* 4/2012 Benedek et al. T19.313 7,039,691 B1 5/2006 Turnidge 2012,0086,717 A1* 4, 2012 Liu .............. ... 345,564 7,191,211 B2 3, 2007 Tuli 2012/0092335 A1 * 4, 2012 Kim et al. ... 345,419 7,290,129 B2 10/2007 Chebolu et al. 2012/0113091 A1* 5, 2012 Isaacson ...... ... 345,418 7,506,257 B1 3/2009 Chavez et al. 2012fO297341 A1* 11, 2012 Glazer et al. T15,810 7,634,811 B1 12/2009 Kienzle et al. 2012,0315972 A1* 12/2012 Olson et al. .. ... 463, 17 7,735,138 B2 6, 2010 Zhao 2013/0024812 A1 1/2013 Reeves et al. ................. T15,810 7,960,945 B1 6/2011 Onorato et al. ............... 320,132 2013/011 1328 A1* 5, 2013 Khanna et al. ................ T15,234 8,023,974 B1 9, 2011 Diao et al. 2013/0117742 A1* 5, 2013 Newell ......... ... 718.1 8,024,790 B2 9, 2011 Zhao et al. 2013,0167159 A1* 6, 2013 Ricci et al. T19,319 8,239,918 B1 8, 2012 Cohen 2013/0210423 A1* 8/2013 Thing .......... 455,425 8,390,636 B1 3/2013 Agopian ....................... 345,545 2014/0007222 A1* 1/2014 Qureshi et al. ... 726, 16 8.487,918 B1* 7/2013 Nelissen et al. ... 345.204 2014.0068779 A1* 3, 2014 Tan et al. ........ 726/26 8,826,145 B1* 9/2014 Kirkpatricket al. .......... 715,746 2014.0137184 AI 52014 Russello et al. 726.1 2001/0054.062 A1 12/2001 Ismael et al. 2014/0173692 A1* 6/2014 Srinivasan et al. ... 726/4 2002/0129281 A1 9, 2002 Hatfalvi et al. 2015,0193904 A1* 7, 2015. Vermeulen ...... 345,522 2003/0041106 A1 2, 2003 Tuli 2004/014.8608 A1 7/2004 Gendreau et al. FOREIGN PATENT DOCUMENTS 2004/O158830 A1 8/2004 Chung et al. 2004/0230643 A1 11/2004 Thibault et al. WO O2/084459 10, 2002 2005, 0188361 A1 8, 2005 Cai et al. WO O3,O17068 2, 2003 2005, 0198485 A1 9/2005 Nguyen et al. WO 2005066786 7/2005 2005/0246176 A1 11/2005 Creamer et al. 2006/01 12342 A1 5/2006 Bantz et al. * cited by examiner U.S. Patent Dec. 29, 2015 Sheet 1 of 13 US 9.225,799 B1 7(22 Ca 7(7/ f(22 f/26 7(24 PROCESSOR USER INPUT DATA DISPLAY DEVICES STORAGE MONITOR COMMUNICATIONS INTERFACE SOFTWARE MODULES AVG / U.S. Patent Dec. 29, 2015 Sheet 2 of 13 US 9.225,799 B1 CINER}_L OR-HOIVN 2730/Y U.S. Patent Dec. 29, 2015 Sheet 6 of 13 US 9.225,799 B1 REMOTE ANDROID 225 225 2 APP APP APP DALVIK VM ANDROID SYSTEM (X86) 222 27 DRIVERMULTIPLEXERLAYER INPUT SCREEN SENSOR 277 --- 273 L - - - 272 267 26/ 267 262 - 2A3. DALVIK VM)ADNDROID SYSTEM (ARM) sess SERVICE 26/ LINUXKERNEL AVG 6 U.S. Patent Dec. 29, 2015 Sheet 7 of 13 US 9.225,799 B1 207/ 27/ celeber SERVER COMPUTER DEVICE 262 252 22/ REMOTE LOGIN SERVER CLIENT ANDROID AUTHENTICATION 297 LOGN START 235 START 252 + TOKEN KEY TOKEN AUENCATION 25% ACCESS + TOKEN ALLOW AVG 7 U.S. Patent Dec. 29, 2015 Sheet 9 of 13 US 9.225,799 B1 ·LNE|TO 5)NI>HECINE!!! SIARHE||||WE NWHL(JENNOT CITOHSERJHL 6’30// U.S. Patent Dec. 29, 2015 Sheet 10 of 13 US 9.225,799 B1 s U.S. Patent Dec. 29, 2015 Sheet 11 of 13 US 9.225,799 B1 LOCAL REMOTE co 9 a. Gerry Dop edge (62-22-cvbar; 562 U.S. Patent Dec. 29, 2015 Sheet 12 of 13 US 9.225,799 B1 LOCAL REMOTE * EEE a2-cvban O, .Q. A/G //C 36.7 LOCAL REMOTE 362 U.S. Patent Dec. 29, 2015 Sheet 13 of 13 US 9.225,799 B1 LOCAL 364 REMOTE 9Input text G 9input text Google search results: ... Google search results: ... 565. AVG //A REMOTE 9Input text R 9 input text 6 Google Search results: ... Google search results: ... US 9.225,799 B1 1. 2 CLIENTSIDE RENDERING FOR VIRTUAL FIG. 3 shows a schematic diagram of a server computer MOBILE INFRASTRUCTURE running a plurality of ANDROID operating systems in accor dance with an embodiment of the present invention. CROSS-REFERENCE TO RELATED FIG. 4 shows the three layers of an ANDROID operating APPLICATION system. FIG. 5 shows a flow diagram of a method of multiplexing This application claims the benefit of U.S. Provisional a plurality of ANDROID operating systems to a single kernel Patent Application No. 61/825,829, filed on May 21, 2013, driver in accordance with an embodiment of the present which is incorporated herein by reference in its entirety. invention. 10 FIG. 6 shows a schematic diagram illustrating components BACKGROUND OF THE INVENTION of a server computer and a mobile client device in accordance with an embodiment of the present invention. 1. Field of the Invention FIG. 7 shows a flow diagram of a method of logging into The present invention relates generally to mobile devices, one of a plurality of ANDROID operating systems in accor 15 dance with an embodiment of the present invention. and more particularly but not exclusively to computing infra FIG. 8 shows a schematic diagram illustrating client-side structures for Supporting operation of mobile devices. rendering in accordance with an embodiment of the present 2. Description of the Background Art invention. Mobile devices, such as smartphones and tablets, have FIG. 9 shows a flow diagram of a method of switching become commonplace and are now employed as replace between client-side rendering and server-side rendering in ments for portable (e.g., laptops and netbooks) and desktop accordance with an embodiment of the present invention. (e.g., personal computers) computing devices. For example, FIG. 10 shows a schematic diagram of a virtual input Smartphones are now employed not just to make Voice calls method editor (IME) operating with a local IME in accor over traditional mobile telephone networks, but also to dance with an embodiment of the present invention. browse the Internet, watch streamed video, and play online 25 FIG.