MEMS Solutions For VR & AR

Sensor Expo 2017

San Jose June 28th 2017 MEMS Sensors & Actuators at ST 2

Motion Environmental Audio Sense Physical change Signal Electro

MEMSMechanical

Mechanical Actuate Electric

Fluidic Piezo actuators Micro Micro-actuators Mirrors ST as MEMS Sensor & Actuators Supplier 3

Market-proven Manufacturing High Volume Expertise in Leading Smart Key Partnerships Technologies Manufacturing multi-sensor Functions in product Front-End / Back-End/ Testing & Calibration Integration Integration development 20 Years of MEMS Sensors & Actuators 4 ST Innovations

Accelerometer Gyroscope Inertial Pressure Microphones UV Humidity GAS & VOC SENSINGmodule sensor sensor sensor 2000 2005 2008 2010 2012 2013 2014 2015 2017

Fluidic Micro Piezo Micro- Mirrors actuators actuators ACTUATING 20 Years of MEMS Sensors & Actuators 5 And Some Iconic Products SENSING 2000 2005 2008 2010 2012 2013 2014 2015 2017 ACTUATING MEMS Sensors & Actuators 6 ST Key Technologies - Enabling Multiple Applications

Sensors Actuators

+

ThELMA* AMR - MAG Piezo-Electric Electrostatic

VENSENS** Cactus Bastille Thermal Electro-magnetic

* Thick Epi- Layer for Microactuators and Accelerometers ** VENice SENSor Changing the MEMS Landscape 7 Strategic Partnerships

Piezo Autofocus Ultrasound Ranging

MEMS Loudspeaker Micro-mirror Projection What is a MEMS μMirror Scanner? 8

• Tiny reflective mechanical device that swings at a given frequency

• Applications spanning Visible to Invisible (IR typically) light Laser Beam Scanning (LBS) 9 Technology Principles and Applications

• Principles: • Light from one/multiple lasers is combined into a single beam

• Beam is relayed onto MEMS scanning mirror(s) Red LD • Mirror(s) scan the beam in a raster pattern • A projected image is created by modulating the

lasers synchronously with the position of the Green LD scanned beam

Blue LD

MEMS scanning mirror Laser Beam Scanning (LBS) 10 Technology Principles and Applications

• Principles: • Light from one/multiple lasers is combined into a single beam • Beam is relayed onto MEMS scanning mirror(s) • Mirror(s) scan the beam in a raster pattern • A projected image is created by modulating the lasers synchronously with the position of the scanned beam

• Applications: • Pico-projection and heads-up display (HUD) • Virtual and (VR, AR) • 3D Sensing and Advanced Driver Assistance Systems (ADAS) New Applications Driven by Accuracy 11 Accuracy

NAVIGATION AUTONOMOUS VIRTUAL & OPTICAL IMAGE PDR DRIVING AUGMENTED STABILIZATION REALITY

̴ 1 m <20 cm Movement TGT ̴ 10 cm <10 cm Noise Bias Instability Random Walk Stab vs. temp Arm Swing, Max error 1 m Total Distance 250 m Off & Sens tolerance LSM6DSM 12 Daydream and Tango Certified

http://www.st.com/content/st_com/en/about/media-center/press-item.html/t3874.html STEVAL-STLKT01V1 SensorTile 13 IoT Design Lab on the tip of a pencil

BLE Microphone

Pressure Sensor Compass MCU IMU STEVAL-STLKT01V1 SensorTile 14 IoT Design Lab on the tip of a pencil

BlueNRG-MS MP34DT04

LPS22HB LSM303AGR STM32L476 LSM6DSM MEMS Solutions For VR & AR Demo Details

Sensor Expo 2017

San Jose June 28th 2017 (VR) Demo

Provides real sense of presence & immersive experience using SensorTile • Showcases low noise, drift and current consumption of our sensors (6X: LSM6DSL) and ST Sensor Fusion algorithms. • MotionFx and ST MEMS sensors performances allow Low latency ( < 6 ms) to overcome motion sickness. • Demo to be conducted with Google phone, SensorTile and . SensorTile is connected to using USB and motionFX is running on SensorTile.

SensorTile Cardboard solid with Android 7.0 CardBoard Mobile running sends Sensor VR Reality App Fusion data USB

STEVAL-STLKT01V1 mounting: LSM6DSM / A + G LSM30AGR / A+M STM32L476 / MCU VR: Basic Requirements for Sensors

• Minimum recommended sampling frequency for both Gyro and Accelerometer is 200Hz

• Low-Pass Filter cutoff is 30Hz for accelerometer and 50Hz for Gyro. • For headset device, the head movement frequency spectrum must be acquired in entirety. Accelerometer bandwidth can be lower because we are not integrating acceleration.

• In Pixel Phone () Gyro and Accelerometer can reach 400Hz maximum (Android N implementation).

• System Architecture • Communication between Phone and SensorTile; • Phone Kernel needs to support Serial Communication over USB (ACM_CDC); • Data is sent from SensorTile to Phone over USB; • Algorithm used to achieve for common timebase for timestamping sensor data SensorTile and Phone; • Sensor HAL receives data from USB and sync timestamps.

Cardboard

Android 7.0 SensorTile Mobile running solid with CardBoard USB VR Reality App; sends Sensors and Timestamp sync algo Fusion data HAL Layer Android Device Requirements for VR

• MUST have at least 2 physical cores

• MUST support sustained performance mode.

• MUST provide an exclusive core to the foreground application

• The kernel must support cpusets and corethread affinity

• MUST support the Process.getExclusiveCores API to return the numbers of the cpu cores that are exclusive to the top foreground application. This core MUST not allow any other userspace processes to run on it (except device drivers used by the application), but MAY allow some kernel processes to run as necessary.

• Source: https://source.android.com/devices/tech/power/performance.html Requirements for VR (Sensor Performance)

Device implementations:

• are STRONGLY RECOMMENDED to support android.hardware.sensor.hifi_sensors feature

• MUST meet the gyroscope-, accelerometer-, and magnetometer-related requirements for android.hardware.hifi_sensors. LSM6DSM is Daydream and Tango certified LSM6DSM 21 Android N HiFi sensor specifications cover the Tango specs Best-in-class 6-axis IMU

GYRO Parameter Value LSM6DSM Stationary Bias Stability < 0.0002 °/s *√Hz from 24-hour static dataset

Bias change vs. temp ≤ +/- 0.05 °/ s / °

Best-Fit line Non-Linearity ≤ 0.2%

Sensitivity change vs. temp ≤ 0.02% / °C

Noise Density ≤ 0.07 °/s/√Hz

XL Parameter Value LSM6DSM Stationary Noise Bias Stability <15 μg * √Hz from 24hr static dataset

Bias change vs. temp ≤ +/- 1mg / °C

Best-Fit line Non-Linearity ≤ 0.5%

Sensitivity change vs. temp ≤ 0.03%/C° Thank you