Decimeter Accurate, Long Range Non-Line-Of-Sight RF Wireless Localization Solution for Public Safety Applications

Decimeter Accurate, Long Range Non-Line-of-Sight RF Wireless Localization Solution for Public Safety Applications

Hun-Seok Kim Assistant Professor EECS, University of Michigan DISCLAIMER

This presentation was produced by guest speaker(s) and presented at the National Institute of Standards and Technology’s 2019 Public Safety Broadband Stakeholder Meeting. The contents of this presentation do not necessarily reflect the views or policies of the National Institute of Standards and Technology or the U.S. Government.

Posted with permission 2 Motivation: Fast and Accurate Indoor Localization § Applications for Indoor Localization § Public safety § Firefighter / first responder rescue operations § Emergency evacuation path planning / guidance § Medicine, equipment, patients, and staffs in hospitals § Customer Applications § Indoor navigation (airport, retail malls, museum, conference center, etc.) § Industrial Applications Image from http://scanonline.com/rtls/ • Intelligent logistics by tracking robots, packages, and workers in warehouses Indoor Localization Technologies ▪ Inertial measurement unit (IMU) based – Accelerometer and gyroscope for 6 degree-of-freedom measurement – Susceptible to error integration ▪ Computer vision based – Simultaneous localization and mapping (SLAM) – Sensitive to light conditions – Computationally demanding ▪ Radio frequency (RF) based – Non-line-of-sight operable – Faster measurement time – Challenges to obtain long range, decimeter accuracy indoors ▪ Sensor fusion – IMU + CV + RF: fusion with adaptive filter Indoor Localization Technologies ▪ Inertial measurement unit (IMU) based – Accelerometer and gyroscope for 6 degree-of-freedom measurement – Susceptible to error integration ▪ Computer vision based – Simultaneous localization and mapping (SLAM) – Sensitive to light conditions – Computationally demanding ▪ Radio frequency (RF) based – Non-line-of-sight operable – Faster measurement time – Challenges to obtain long range, decimeter accuracy indoors ▪ Sensor fusion – IMU + CV + RF: fusion with adaptive filter Research Scope ▪ New Indoor RF Localization Solutions – Year1: RF-Echo with custom ASIC tag – Year2: iLPS for simultaneous communication and localization – Year3: Sound-RF hybrid solution

▪ Global Positioning System (GPS) – Covered Area – Accuracy – Indoor Usage

7 Available RF Localization Solutions and Challenges

8 Available RF Localization Solutions and Challenges

▪ Global Positioning System (GPS) – Covered Area – Accuracy – Indoor Usage ▪ WiFI / Bluetooth (Received signal strength indicator (RSSI)-based) – Covered Area – Accuracy – Indoor Usage ▪ Ultra-wide Band (UWB, IEEE802.15.4a, Decawave) – Covered Area – Accuracy – Indoor Usage

9 Requirements for Public Safety Apps

▪ Easily and quickly deployable infrastructure – First responder rescue missions – Portable and mobile infrastructure desirable Requirements for Public Safety Apps

▪ Easily and quickly deployable infrastructure – First responder rescue missions – Portable and mobile infrastructure desirable ▪ Decimeter-level accuracy in non-line-of-sight indoors – Long range (~100m) operable – Milli-second refresh rate, tens of centimeter accuracy ▪ Ultra-low cost tags – To be ported on numerous IoT devices – Tracking of disposable tags Requirements for Public Safety Apps ▪ Small form factor – Unobtrusive integration into IoT

Small Requirements for Public Safety Apps ▪ Small form factor – Unobtrusive integration into IoT ▪ Low power consumption – Sustainable with a small coin-cell battery – No manual battery management

Small Low-power General RF Localization System

▪ Find the distance between anchors and tags

�1 �2 Anchor Anchor Tag �3

Anchor

15 General RF Localization System

▪ Find the distance between anchors and tags ▪ Distance à Lateration – Time of Arrival (ToA), Time Difference of Arrival (TDoA)

ToA (circle) TDoA (hyperbola) 16 Why RF Indoor Localization is Difficult

▪ Multipath and Non-Line-of-Sight (NLOS)

Direct Path NLOS Reflected Path (much weaker)

time Channel Impulse LOS Response (CIR)

Wall

17 Why RF Indoor Localization is Difficult

▪ Multipath and Non-Line-of-Sight (NLOS)

Direct Path NLOS Reflected Path (much weaker)

time LOS Channel Impulse Direct Path Response (CIR)

time

Wall

18 Time of Arrival (ToA) from Channel Impulse Response (CIR) ▪ Speed of light: �, Time of Arrival (ToA): � ▪ Distance: � = �� à 30cm with ToA of 1ns Time of Arrival (ToA) from Channel Impulse Response (CIR) ▪ Speed of light: �, Time of Arrival (ToA): � ▪ Distance: � = �� à 30cm with ToA of 1ns ▪ Indoor channel is multi-path rich Time of Arrival (ToA) from Channel Impulse Response (CIR) ▪ Speed of light: �, Time of Arrival (ToA): � ▪ Distance: � = �� à 30cm with ToA of 1ns ▪ Indoor channel is multi-path rich ▪ ToA or distance is estimated from channel impulse response (CIR)

@ transmitter Delay spread @ receiver � � =�(�) ToA

Multipath t=0 t=� time t=0 time channel Channel Impulse Response (CIR) TDoA Localization Challenges

▪ CIR estimation in multi-path indoor channels – Direct path can be much weaker than multipaths ▪ Time synchronization between transmitter and receiver – 1ns mismatch à 30cm error ▪ Limited bandwidth (80MHz) for ISM band operation à 3.75m resolution

@ transmitter Delay spread @ receiver � � =�(�) ToA

Multipath t=0 t=� time t=0 time channel Channel Impulse Response (CIR) 22 Proposed System: RF-Echo and iLPS

▪ Decimeter-level (tens of centimeter) ranging accuracy ▪ Large covered area: >100m distance ▪ GPS-like local positioning scheme – Indefinite number of tags localize themselves simultaneously ▪ Reliable wireless communications between anchors and tags – Localization and communication at the same time ▪ Deployable without heavy infrastructure investment

23 Overview: RF-Echo (Year1) ▪ RF-Echo Operating Principle: – Round-trip Time-of-Flight (RToF) – Introduce active reflector tag with frequency conversion • Full-duplex tag: simultaneous TX and RX • Increase ranging distance by active signal amplification at tag • Tag reflection has different frequency from passive reflection • All analog tag design: deterministic echo processing delay

TAG

180nm ASIC 2.4GHz OFDM Mixer + LO PA mm 87 87 . 900MHz OFDM 1 VGA LNA Anchor 2.69 mm USRP x310 Tag RF-Echo ASIC Tag

▪ Tag ASIC in TSMC 180 nm ▪ Low-cost simple tag design 180nm ASIC Mixer + LO PA

– Crystal-less, PLL-free mm 87 87 . ▪ Analog-only design 1 VGA LNA – No digital signal processing circuitry Tag 2.69 mm – Deterministic delay

LNA Mixer + LO VGA PA To VGA

Power RC feedback combi ni ng Double-balanced Mixer Switches From Mixer RFin Broadband input matching RFout 3 : I 1 REF2 On-chi p LC Oscillator RF-Echo System

▪ Round-trip ToF or distance measurement by analyzing channel impulse

response (CIR) @ transmitter Delay spread @ receiver " # =$(#) ToA

Multipath t=0 t=! time t=0 time channel Channel Impulse Response (CIR) 2 .4GHz OFDM

900 180nm ASIC

Anchor MHz OFDM

MHz OFDM MHz 900

GHz OFDM GHz

4 . TagTag 2 Anchor GHz OFDM 2.4

MHz OFDM 900

Anchor 26 RF-Echo System Evaluation

LOS NLOS Testing Tag System Signal Time per Energy System Technology Accuracy Accuracy Dimension Power Bandwidth Type Fix per Fix 31 cm 40 cm LOS: 3-11 m WiTrack FMCW ToF No Tag 1.69 GHz FMCW > 2.5 ms N/A (90%) (90%) NLOS: 6 x 5 m2 31 cm 42 cm LOS/NLOS: Harmonium UWB TDoA 75 mW 3.5 GHz Impulse 52 ms 3900 µJ (90%) (90%) 4.6 x 7.2 x 2.7 m3

SAR + Motion 39 cm 59 cm LOS/NLOS: Ubicarse N/A N/A WiFi 100 ms N/A sensor (median) (median) 15 x 15 m2

12 cm 6 MHz Tagoram RFID SAR N/A LOS: 1 x 2 m2 Passive UHF RFID > 33 ms N/A (median) (UHF)

802.11 WiFi + 14.1 cm 20.7 cm LOS/NLOS: 20 MHz x 35 1.34x 105 Chronos 1.6 W OFDM 84 ms Band-stiching (median) (median) 20 x 20 m2 ch. µJ

ASIC Active reflection + 26 cm 46 cm LOS: 7 x 90 m2 20 µs per 18 µJ RF-Echo 62.8 mW 80 MHz OFDM Neural (90%) (90%) NLOS: 30 x 20 m2 sym. 10 sym. network Research Scope ▪ New Indoor RF Localization Solutions – Year1: RF-Echo with custom ASIC tag – Year2: iLPS for simultaneous communication and localization – Year3: Sound-RF hybrid solution

▪ iLPS: Indoor Local Positioning System ▪ Two kinds of anchors: Main anchor & Reflector anchor – Main anchor broadcasts OFDM signal at 2.4GHz and 5.8GHz

5.8GHz

Main Anchor 2.4GHz Tag

29 Overview: iLPS (Year2)

▪ iLPS: Indoor Local Positioning System ▪ Two kinds of anchors: Main anchor & Reflector anchor – Main anchor broadcasts OFDM signal at 2.4GHz and 5.8GHz – Reflector anchors reflects the signal at 5.8GHz with signal amplification and frequency conversion to 2.4GHz – Estimate TDoA without strict time synchronization between anchors Reflector

2.4GHz

5.8GHz

Main Anchor 2.4GHz TDoA Tag

30 iLPS operation principle

Reflectors can be realized with all- analog processing

31 Inter-anchor Interference Avoidance

▪ Orthogonal (OFDMA) subcarrier allocation for different reflectors

2.4GHz Tag

freq

Main Anchor

freq 5.8GHz 32 Inter-anchor Interference Avoidance

▪ Orthogonal (OFDMA) subcarrier allocation for different reflectors

Idle subcarriers 2.4GHz Tag

freq

Main Anchor

freq 5.8GHz 33 Inter-anchor Interference Avoidance

▪ Orthogonal (OFDMA) subcarrier allocation for different reflectors ▪ Frequency conversion with offset at reflector anchors – Orthogonal subcarrier allocations – Different subcarriers allocated to different reflector anchors 2.4GHz Tag

freq

Main Anchor Shift by 1 subcarrier

freq 2.4GHz 5.8GHz Reflector #1 34 Inter-anchor Interference Avoidance

2.4GHz Shift by 2 subcarriers

Main Anchor

2.4GHz Reflector #2 5.8GHz Reflector #1 35 Inter-anchor Interference Avoidance

2.4GHz Shift by 2 subcarriers

Main Anchor

2.4GHz Reflector #2 5.8GHz Reflector #1 36 iLPS Communication

▪ Packet structure based on IEEE802.11a/g/n WiFi – STF for packet detection, LTF for CIR estimation – Data symbols contain anchor coordinate information and other general data

37 iLPS Communication

▪ Packet structure based on IEEE802.11a/g/n WiFi – STF for packet detection, LTF for CIR estimation – Data symbols contain anchor coordinate information and other general data ▪ Distributed MISO system with frequency diversity – Improves communication reliability Reflector Anchor #1

x � ∗ ℎ(�) � = ��

x � ∗ ℎ(�)

Main Anchor Tag

x � ∗ ℎ(�)

Reflector Anchor #2 38 iLPS TDoA Estimation Accuracy

▪ Ranging accuracy vs Signal bandwidth @ transmitter Delay spread @ receiver 1 " # =$(#) ToA �� ∝ Multipath t=0 t=! time �� t=0 time channel Channel Impulse Response (CIR)

39 iLPS TDoA Estimation Accuracy

▪ Ranging accuracy vs Signal bandwidth @ transmitter Delay spread @ receiver 1 " # =$(#) ToA �� ∝ Multipath t=0 t=! time �� t=0 time channel Channel Impulse Response (CIR) ▪ UWB uses large BW (typically GHz) to achieve high ToA resolution – Transmit power is limited à localization range is limited – Wider bandwidth à more interference

40 iLPS TDoA Estimation Accuracy

41 iLPS TDoA Estimation Accuracy

Machine learning based accuracy enhancement

42 Neural Network Based ToA Estimation

▪ How do we estimate ToA from coarsely measured (BW limited) CIR?

True ToA

43 Neural Network Based ToA Estimation

▪ CIR pattern recognition via neural networks – Train neural network to learn the shape around the first path

Center for ToA pattern matching On-time Early (+)Delay (t=ToA) fc1 fc2 fc3 Late Combine (-)Delay confidence � Off-time Channel Response 4

16 32

44 Neural Network Based ToA Estimation

▪ CIR pattern recognition via neural networks – Train neural network to learn the shape around the first path ▪ Training set generated in Matlab simulation – No need of collecting real-world data before deploying tags

0.15 Synthesized CIR Many synthesized CIR w/ for NLOS (linear) random delay spread

0.1

0.05

Training window 0 0 20 40 60 80 100 120 Time index 45 Baseband Signal Processing at Tags

▪ Tag implementation on USRP Software Defined Radio (SDR) ▪ Similar to WiFi 802.11 a/g/n ▪ Tag only requires a receiver for localization (similar to GPS receiver)

46 iLPS System Evaluation

▪ Experiment environment and testing setup ▪ Main anchor, reflector and tags all implemented using USRP X310 SDR

Hallway

USRP + PC Reflector

Tag

Main Anchor

47 iLPS System Evaluation

▪ 2D localization in University of Michigan EECS building

48 Comparison Table

Simultaneous LOS NLOS Testing System Latency for K System Technology Communication & Accuracy Accuracy Dimension Bandwidth tags Local. Localization FMCW+TDMA 15.9 cm 16.1 cm LOS: 5 x 7 m2 WiTrack 2.0 1.79GHz 10s (K < 5) No (ToA) (median (median) NLOS: 6 x 5 m2

UWB+TDMA 31 cm 42 cm LOS/NLOS: Harmonium 3.5GHz K x 52ms No (TDOA) (90%) (90%) 4.6 x 7.2 x 2.7 m3

OFDM+TDMA 50 cm 50 cm LOS: 10m Yes WASP 125MHz K x 2.5ms (ToA) (85%) (65%) NLOS: 15 x 15 m2 No diversity gain

OFDM+TDMA 14.7 cm 20.7 cm LOS/NLOS: 20MHz x 35 Possible, Chronos (Band-stiching 3 x 84ms (median) (median) 20x 20 m2 overlapped Not demonstrated ToA)