University of Bremen Faculty of Mathematics and Computer Science Computer Graphics and Virtual Reality Research Group Master’s Thesis Design and development of hardware and software for wireless Head-Mounted Displays by Dmitry Galkin (matriculation number: 2780410) Supervisors: Dr. Gabriel Zachmann, University of Bremen Dr. Marc Herrlich, University of Bremen Thesis submitted in partial fulfillment of the requirements for the Digital Media Master of Science degree at the University of Bremen. 2014 - 2015 Statutory Declaration I hereby declare that I have written the enclosed Master's thesis by myself, I have not used sources or means without respective declaration in the text. Any thoughts from others or literal quotations are clearly marked and references are given. The Master's thesis was not used in the same or in a similar version to achieve an academic grading or being published anywhere else. ….......................................... ….......................................... date (signature) 2 Abstract With the introduction of a new head-mounted displays models on the consumer market, the interest to those devices raised dramatically in the recent past. Although only a few wireless head-mounted display models are available today, there is a certain range of applications where those are required. Existing solutions that allow using conventional head-mounted displays for such applications wirelessly are either very expensive or too cumbersome. This thesis is aimed to overcome those drawbacks with a new solution of a distributed design. Wired HMD is being connected to a compact, low-cost, portable computer that displays a real-time compressed video stream, transferred from a computer performing the actual rendering of an HMD- enabled application. A prototype according to the proposed design was developed and evaluated. KEYWORDS: WIRELESS HEAD-MOUNTED DISPLAY, HMD, OCULUS RIFT, BANANA PI, VIDEO STREAMING, H.264, USB OVER IP. 3 Table of Contents Abstract....................................................................................................................................3 Table of Contents.....................................................................................................................4 1. Introduction........................................................................................................................6 1.1 Motivation.....................................................................................................................7 1.2 Requirements...............................................................................................................11 2. Related Work....................................................................................................................13 2.1 Wired HMD Interfaces................................................................................................13 2.2 Existing Wireless Video Technologies........................................................................14 2.3 Existing Wireless HMD Solutions..............................................................................16 2.3.1 Intel WiDi and Miracast for HMDs.....................................................................18 2.4 Video Modes and Interfaces........................................................................................19 2.5 Wireless Network Standards.......................................................................................20 2.6 Wireless USB Connections.........................................................................................22 2.7 HMD USB Usage........................................................................................................23 2.8 Related Work Summary..............................................................................................24 3. Solution Proposed............................................................................................................26 3.1 Video Transfer.............................................................................................................26 3.2 Overall System Design................................................................................................27 3.3 H.264/AVC Video Coding Standard...........................................................................29 3.3.1 H.264 Hardware Support.....................................................................................32 3.4 USB Data Transfer......................................................................................................33 3.5 Wireless Network........................................................................................................34 3.6 Proposed Solution Summary.......................................................................................35 3.7 Design Limitations......................................................................................................35 4. Implementation................................................................................................................38 4.1 Prototype Software......................................................................................................38 4.2 Server Software...........................................................................................................39 4.2.1 Screen Capturing.................................................................................................39 4.2.2 Video Encoding...................................................................................................40 4.2.3 Video Streaming..................................................................................................41 4.2.4 Virtual USB Connection......................................................................................43 4.3 Client Software............................................................................................................46 4.3.1 Client Operating System......................................................................................46 4.3.2 Video Playing......................................................................................................47 4.3.3 Client USB Connection.......................................................................................47 4.4 Prototype Hardware.....................................................................................................48 4.4.1 Server Hardware..................................................................................................48 4.4.2 Client Hardware...................................................................................................48 4.5 System Configuration..................................................................................................53 4.5.1 Server Configuration...........................................................................................53 4.5.2 Client Configuration............................................................................................56 4 4.6 Implementation Summary...........................................................................................59 5. Evaluation.........................................................................................................................61 5.1 Evaluation Criteria......................................................................................................61 5.2 Streaming Latency Estimation....................................................................................62 5.3 USB/IP Performance Estimation.................................................................................67 5.4 Other Criteria Estimation............................................................................................69 5.4.1 Refresh rate, frame rate, screen resolution..........................................................69 5.4.2 Battery operation, dimensions, weight, costs......................................................70 5.5 Evaluation Results.......................................................................................................72 6. Summary...........................................................................................................................73 6.1 Limitations..................................................................................................................74 6.2 Future work.................................................................................................................76 6.3 Conclusions.................................................................................................................76 7. Acknowledgements...........................................................................................................79 8. References.........................................................................................................................80 9. Appendix...........................................................................................................................91 9.1 Server Computer.....................................................................................................91 9.2 USB Usage Analysis...............................................................................................91 9.3 USB/IP for Windows..............................................................................................91 9.4 Automation on Server.............................................................................................92 9.5 Automation on