Smart Glasses & Applications
Presenter: Xu Qiu Advisor: C.-C. (Jay) Kuo
11 Outline • Introduction • Comparison of Smart Glasses! • Apps for Google Glass! • Developing Platform! • Development and Implementation of Computer Vision Algorithms
21 Outline • Introduction! ! - Project Introduction - GE90 Jet Engine Repair • Comparison of Smart Glasses! • Apps for Google Glass! • Developing Platform! • Development and Implementation of Computer Vision Algorithms
31 Google Glass From May 2014
http://mcl.usc.edu/news/2014/04/20/media-communications-lab-begins- research-projects-with-google-glass/
4 Google Glass GE90 Jet Engine Repair
https://www.youtube.com/watch?v=ndKqo0pzmqM 5 Google Glass GE90 Jet Engine Repair
•Recognize part
6 Google Glass GE90 Jet Engine Repair
•Recognize part
7 Google Glass GE90 Jet Engine Repair
•Engine status
8 Google Glass GE90 Jet Engine Repair
•Recognize someone
9 Google Glass GE90 Jet Engine Repair
•Pair with Boroscope
10 Google Glass GE90 Jet Engine Repair
•Get directions to…
11 Google Glass GE90 Jet Engine Repair
•Send/get messages
12 Google Glass GE90 Jet Engine Repair
•Task list
13 Outline • Introduction! • Comparison of Smart Glasses! - Google Glass VS Vuzix M100 - Hardware • Apps for Google Glass! • Developing Platform! • Development and Implementation of Computer Vision Algorithms
141 Smart Glasses Comparison
•Google Glass VS Vuzix M100
Google Glass
Vuzix M100 Smart Glasses
15 Vuzix Smart Glasses Display
•An image visually equivalent to a 4 in. smartphone screen held at a typical 14 in. distance.
16 Google Glass Hardware
•Display: Prism projector, 640×360 pixels (equivalent of a 25 in/64 cm screen from 8 ft/2.4 m away) •Sound: Bone conduction transducer •Input: Voice command through microphone, accelerometer, gyroscope, magnetometer, light sensor, proximity sensor •Controller input: Touchpad, MyGlass phone app •Camera: Photos – 5 MP, videos – 720p •Connectivity: Wi-Fi , Bluetooth, micro USB •Weight: 50g •Backward compatibility: - Any Bluetooth-capable phone; - MyGlass app requires Android 4.0.3 + or any iOS 7.0 +
17 Google Glass Hardware
18 Google Glass Hardware
•Touchpad:!
19 Outline • Introduction! • Comparison of Smart Glasses! • Apps for Google Glass! ! - Social network - GDK games - LBS(GPS apps) - Detection/Recognition Apps - Audio apps - Other apps • Developing Platform! • Development and Implementation of Computer Vision Algorithms
201 Google Glass Apps:
•Social network
http://glass-apps.org/google-glass-application-list 21 Google Glass Apps:
•GDK games: Missile Beyond
http://glass-apps.org/google-glass-application-list 22 Google Glass Apps:
•GDK games: GlassCopter
http://glass-apps.org/google-glass-application-list 23 Google Glass Apps:
•LBS(GPS apps): AR Glass for Wikipedia
http://glass-apps.org/google-glass-application-list 24 Google Glass Apps:
•SpeedHud
http://glass-apps.org/google-glass-application-list 25 Google Glass Apps:
•Detection/Recognition Apps
http://glass-apps.org/google-glass-application-list 26 Google Glass Apps: •Audio apps: •Name This Song: “Ok glass, name this song”. •GlassMusic: search by any song name, artist or album. •Other apps: •Speech Helper for Glass: Upload your speech draft and slides. •Check a battery: check a battery for Google Glass.
http://glass-apps.org/google-glass-application-list 27 Outline • Introduction! • Comparison of Smart Glasses! • Apps for Google Glass! • Developing Platform! ! - GDK - Mirror API • Development and Implementation of Computer Vision Algorithms
281 Google Glass Developing platform
•GDK:
Real-time user interaction, offline functionality, access to hardware.
29 Google Glass Developing platform
•Mirror API:
platform independence, common infrastructure, built-in functionality
30 Outline • Introduction! • Comparison of Smart Glasses! • Apps for Google Glass! • Developing Platform! • Development and Implementation of Computer Vision Algorithms! - Relative Viewing Angle Estimation - Future work
311 Google Glass Development and Implementation of Computer Vision Algorithms:
•Visual Saliency Determination •Relative Viewing Angle Estimation •Object Detection
32 Relative Viewing Angle Estimation
•Moving camera instead of smart glasses •Real-time algorithms
(CVPR 2013)Real-time Model-based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues, by Karl Pauwels, Leonardo Rubio, Javier D ́ıaz, 33 Eduardo Ros of University of Granada, Spain. Relative Viewing Angle Estimation Real-time Model-based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues •Estimating and tracking the six-degrees-of-freedom (6DOF) pose of rigid objects of arbitrary shapes in real-time. - highly accurate and robust to noise and occlusions • Problems: -Stereo Camera: -Graphics Processing Units (GPUs)
(CVPR 2013)Real-time Model-based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues, by Karl Pauwels, Leonardo Rubio, Javier D ́ıaz, 34 Eduardo Ros of University of Granada, Spain. Relative Viewing Angle Estimation
•Moving camera instead of smart glasses •Real-time algorithms •No stereo-camera! Multi-target tracking with Single Moving Camera - multi-target tracking - unknown camera motion - did not rely on the use of stereo-camera.
http://www.eecs.umich.edu/vision/mttproject.html
Object detection: Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. In: PAMI. (2009)
35 Relative Viewing Angle Estimation Multi-target tracking with Single Moving Camera
36 Relative Viewing Angle Estimation Future work - Limitation:! Single camera Storage, Computation capacity - Challenges:! Moving camera Unknown background - Possible Solutions:! Cloud Simply the algorithms Use motion to get 3D information
37 Thank you!
Comments or Questions?
38