MEMS Solutions For VR & AR
Sensor Expo 2017
San Jose June 28th 2017 MEMS Sensors & Actuators at ST 2
Motion Environmental Audio Sense Physical change Signal Electro
MEMSMechanical
Mechanical Actuate Electric
Fluidic Piezo actuators Micro Micro-actuators Mirrors ST as MEMS Sensor & Actuators Supplier 3
Market-proven Manufacturing High Volume Expertise in Leading Smart Key Partnerships Technologies Manufacturing multi-sensor Functions in product Front-End / Back-End/ Testing & Calibration Integration Integration development 20 Years of MEMS Sensors & Actuators 4 ST Innovations
Accelerometer Gyroscope Inertial Pressure Microphones UV Humidity GAS & VOC SENSINGmodule sensor sensor sensor 2000 2005 2008 2010 2012 2013 2014 2015 2017
Fluidic Micro Piezo Micro- Mirrors actuators actuators ACTUATING 20 Years of MEMS Sensors & Actuators 5 And Some Iconic Products SENSING 2000 2005 2008 2010 2012 2013 2014 2015 2017 ACTUATING MEMS Sensors & Actuators 6 ST Key Technologies - Enabling Multiple Applications
Sensors Actuators
+
ThELMA* AMR - MAG Piezo-Electric Electrostatic
VENSENS** Cactus Bastille Thermal Electro-magnetic
* Thick Epi-Poly Layer for Microactuators and Accelerometers ** VENice SENSor Changing the MEMS Landscape 7 Strategic Partnerships
Piezo Autofocus Ultrasound Ranging
MEMS Loudspeaker Micro-mirror Projection What is a MEMS μMirror Scanner? 8
• Tiny reflective mechanical device that swings at a given frequency
• Applications spanning Visible to Invisible (IR typically) light Laser Beam Scanning (LBS) 9 Technology Principles and Applications
• Principles: • Light from one/multiple lasers is combined into a single beam
• Beam is relayed onto MEMS scanning mirror(s) Red LD • Mirror(s) scan the beam in a raster pattern • A projected image is created by modulating the
lasers synchronously with the position of the Green LD scanned beam
Blue LD
MEMS scanning mirror Laser Beam Scanning (LBS) 10 Technology Principles and Applications
• Principles: • Light from one/multiple lasers is combined into a single beam • Beam is relayed onto MEMS scanning mirror(s) • Mirror(s) scan the beam in a raster pattern • A projected image is created by modulating the lasers synchronously with the position of the scanned beam
• Applications: • Pico-projection and heads-up display (HUD) • Virtual and Augmented Reality (VR, AR) • 3D Sensing and Advanced Driver Assistance Systems (ADAS) New Applications Driven by Accuracy 11 Accuracy
NAVIGATION AUTONOMOUS VIRTUAL & OPTICAL IMAGE PDR DRIVING AUGMENTED STABILIZATION REALITY
̴ 1 m <20 cm Movement TGT ̴ 10 cm <10 cm Noise Bias Instability Random Walk Stab vs. temp Arm Swing, Max error 1 m Total Distance 250 m Off & Sens tolerance LSM6DSM 12 Google Daydream and Tango Certified
http://www.st.com/content/st_com/en/about/media-center/press-item.html/t3874.html STEVAL-STLKT01V1 SensorTile 13 IoT Design Lab on the tip of a pencil
BLE Microphone
Pressure Sensor Compass MCU IMU STEVAL-STLKT01V1 SensorTile 14 IoT Design Lab on the tip of a pencil
BlueNRG-MS MP34DT04
LPS22HB LSM303AGR STM32L476 LSM6DSM MEMS Solutions For VR & AR Demo Details
Sensor Expo 2017
San Jose June 28th 2017 Virtual Reality (VR) Demo
Provides real sense of presence & immersive experience using SensorTile • Showcases low noise, drift and current consumption of our sensors (6X: LSM6DSL) and ST Sensor Fusion algorithms. • MotionFx and ST MEMS sensors performances allow Low latency ( < 6 ms) to overcome motion sickness. • Demo to be conducted with Google Pixel phone, SensorTile and Google Cardboard. SensorTile is connected to mobile device using USB and motionFX is running on SensorTile.
SensorTile Cardboard solid with Android 7.0 CardBoard Mobile running sends Sensor VR Reality App Fusion data USB
STEVAL-STLKT01V1 mounting: LSM6DSM / A + G LSM30AGR / A+M STM32L476 / MCU VR: Basic Requirements for Sensors
• Minimum recommended sampling frequency for both Gyro and Accelerometer is 200Hz
• Low-Pass Filter cutoff is 30Hz for accelerometer and 50Hz for Gyro. • For headset device, the head movement frequency spectrum must be acquired in entirety. Accelerometer bandwidth can be lower because we are not integrating acceleration.
• In Pixel Phone (Google Daydream) Gyro and Accelerometer can reach 400Hz maximum (Android N implementation).
• System Architecture • Communication between Phone and SensorTile; • Phone Kernel needs to support Serial Communication over USB (ACM_CDC); • Data is sent from SensorTile to Phone over USB; • Algorithm used to achieve for common timebase for timestamping sensor data SensorTile and Phone; • Sensor HAL receives data from USB and sync timestamps.
Cardboard
Android 7.0 SensorTile Mobile running solid with CardBoard USB VR Reality App; sends Sensors and Timestamp sync algo Fusion data HAL Layer Android Device Requirements for VR
• MUST have at least 2 physical cores
• MUST support sustained performance mode.
• MUST provide an exclusive core to the foreground application
• The kernel must support cpusets and corethread affinity
• MUST support the Process.getExclusiveCores API to return the numbers of the cpu cores that are exclusive to the top foreground application. This core MUST not allow any other userspace processes to run on it (except device drivers used by the application), but MAY allow some kernel processes to run as necessary.
• Source: https://source.android.com/devices/tech/power/performance.html Requirements for VR (Sensor Performance)
Device implementations:
• are STRONGLY RECOMMENDED to support android.hardware.sensor.hifi_sensors feature
• MUST meet the gyroscope-, accelerometer-, and magnetometer-related requirements for android.hardware.hifi_sensors. LSM6DSM is Daydream and Tango certified LSM6DSM 21 Android N HiFi sensor specifications cover the Tango specs Best-in-class 6-axis IMU
GYRO Parameter Value LSM6DSM Stationary Bias Stability < 0.0002 °/s *√Hz from 24-hour static dataset
Bias change vs. temp ≤ +/- 0.05 °/ s / °C
Best-Fit line Non-Linearity ≤ 0.2%
Sensitivity change vs. temp ≤ 0.02% / °C
Noise Density ≤ 0.07 °/s/√Hz
XL Parameter Value LSM6DSM Stationary Noise Bias Stability <15 μg * √Hz from 24hr static dataset
Bias change vs. temp ≤ +/- 1mg / °C
Best-Fit line Non-Linearity ≤ 0.5%
Sensitivity change vs. temp ≤ 0.03%/C° Thank you