remote sensing Review Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review Shahzad Ahmed 1 , Karam Dad Kallu 2, Sarfaraz Ahmed 1 and Sung Ho Cho 1,* 1 Department of Electronic Engineering, Hanyang University, 222 Wangsimini-ro, Seongdong-gu, Seoul 133-791, Korea; [email protected] (S.A.); [email protected] (S.A.) 2 Robotics and Intelligent Machine Engineering (RIME), School of Mechanical and Manufacturing Engineering (SMME), National University of Science and Technology (NUST), H-12 Islamabad 44000, Pakistan; [email protected] * Correspondence: [email protected]; Tel.: +82-(02)-2220-4883 Abstract: Human–Computer Interfaces (HCI) deals with the study of interface between humans and computers. The use of radar and other RF sensors to develop HCI based on Hand Gesture Recognition (HGR) has gained increasing attention over the past decade. Today, devices have built-in radars for recognizing and categorizing hand movements. In this article, we present the first ever review related to HGR using radar sensors. We review the available techniques for multi-domain hand gestures data representation for different signal processing and deep-learning-based HGR algorithms. We classify the radars used for HGR as pulsed and continuous-wave radars, and both the hardware and the algorithmic details of each category is presented in detail. Quantitative and qualitative analysis of ongoing trends related to radar-based HCI, and available radar hardware and algorithms is also presented. At the end, developed devices and applications based on gesture- recognition through radar are discussed. Limitations, future aspects and research directions related to this field are also discussed. Keywords: hand-gesture recognition; pulsed radar; continuous-wave radars; human–computer Citation: Ahmed, S.; Kallu, K.D.; interfaces; deep-learning for radar signals Ahmed, S.; Cho, S.H. Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review. Remote Sens. 2021, 13, 527. 1. Introduction https://doi.org/10.3390/rs13030527 In recent years, computing technology has become embedded in every aspect of our daily lives and man–machine interaction is becoming inevitable. It is widely believed that Received: 7 January 2021 computer and display technology will keep on progressing further. A gateway which Accepted: 1 February 2021 allows humans to communicate with machines and computers is known as the human– Published: 2 February 2021 computer interface (HCI) [1]. Keyboard and mouse and touch-screen sensors are the traditional HCI approaches. However, these approaches are becoming a bottleneck for Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in developing user friendly interfaces [2]. Contrary to this, human gestures can be a more published maps and institutional affil- natural way of providing an interface between humans and computers. Short-range radars iations. have the ability to detect micro-movements with high precision and accuracy [3]. Radar sensors have shown potential in several research areas such as presence detection [4], vital sign monitoring [5], and Radio Frequency (RF) imaging purpose [6]. Choi et al. [7] used an Ultra-Wideband (UWB) Impulse Radar for indoor people counting. Similar research presented by [8] used milli-metric-wave radar for occupancy detection. In addition to Copyright: © 2021 by the authors. this, radar sensors have shown their footprints in hand-motion sensing and dynamic Licensee MDPI, Basel, Switzerland. This article is an open access article HGR [9–19]. The interest in radar-based gesture recognition has surged in recent years. distributed under the terms and Recently, radar sensors have been deployed in a network-fashion for the detection conditions of the Creative Commons and classification of complex hand gestures to develop applications such as the wireless Attribution (CC BY) license (https:// keyboard [9]. In the aforementioned study [9], in-air movement of the hand was recorded creativecommons.org/licenses/by/ with three UWB radars and a tracking algorithm was used to type 0–9 counting digits 4.0/). in the air. Another study published by same authors presented a continuous alphabet Remote Sens. 2021, 13, 527. https://doi.org/10.3390/rs13030527 https://www.mdpi.com/journal/remotesensing Remote Sens. 2021, 13, x FOR PEER REVIEW 2 of 25 Remote Sens. 2021, 13, 527 2 of 24 the air. Another study published by same authors presented a continuous alphabet writ- ing based on the gestures drawn in front of two radars [20]. A work presented by Ahmed writingand Cho based [12] (2020) on the demonstrated gestures drawn a performa in frontnce of twocomparison radars [ 20of ].different A work Deep presented Convolu- by Ahmedtional Neural and Cho Network [12] (2020) (DCNN) demonstrated-based deep-learning a performance algorith comparisonms using of multiple different radars. Deep ConvolutionalIn addition to that, Neural hand Network gesture (DCNN)-based recognition through deep-learning radar technology algorithms also using found multiple appli- radars.cation in In the addition operating to that, room hand to assist gesture medi recognitioncal staff in through processing radar and technology manipulating also found med- applicationical images in[21]. the operating room to assist medical staff in processing and manipulating medicalBased images on the [21 transmitted]. signal, short-range radar sensors used for HGR can broadly be categorizedBased on the as transmittedpulsed radar signal, and continuous-wave short-range radar (CW) sensors radar. used This for categorization HGR can broadly has bebeen categorized adopted aspreviously pulsed radar in several and continuous-wave radarprelated review (CW) radar.articles This for categorizationapplications other has beenthan adoptedHGR [22]. previously Pulsed radar, in several such radarprelatedas Ultra-Wideband review Impulse-Radar articles for applications (UWB-IR), othertrans- thanmits HGRshort [ 22duration]. Pulsed pulses, radar, whereas such as Ultra-Wideband continuous-wave Impulse-Radar radar, such as (UWB-IR), Frequency transmits Modu- shortlated durationContinuous pulses, Wave whereas (FMCW) continuous-wave radar, transmits radar,and receives such as a FrequencycontinuousModulated wave. Both Continuousthese radars Wave are widely (FMCW) used radar, for HGR transmits purposes. and receives a continuous wave. Both these radars are widely used for HGR purposes. 1.1. Understanding Human Gestures 1.1. UnderstandingPrior to making Human HCI, Gestures an understanding of the term “Gestures” is important. Re- searchersPrior in to [23] making defined HCI, a angesture understanding as a movement of the of termany body “Gestures” part such is important. as arms, hands Re- searchersand face in in order [23] defined to convey a gesture information. as a movement This nonverbal of any communication body part such constitutes as arms, hands up to andtwo facethirds in orderof all tocommunication convey information. among Thisthe humans nonverbal [24]. communication Amongst the constitutesdifferent body up toparts, two hand thirds gestures of all communication are widely used among for constr the humansucting interactive [24]. Amongst HCIs the (Figure different 1) [25]. body As parts,seen in hand Figure gestures 1, 44% are of widely studies used focused forconstructing on developing interactive HCIs using HCIs hand, (Figure multiple1)[ 25 ].hand As seenand infinger Figure movements.1, 44% of studiesHand gestures focused onare developingan important HCIs part using of non-verbal hand, multiple communica- hand andtion finger in our movements. daily life, and Hand we gesturesextensively are anuse important hand gestures part of for non-verbal communication communication purposes insuch our as daily pointing life, and towards we extensively an object useand hand conveying gestures information for communication about shape purposes and space. such asUtilizing pointing hand towards movements an object as and an conveyinginput source information instead of about a keyboard shape andand space.mouse Utilizing can help handpeople movements to communicate as an input with source computers instead in ofmore a keyboard easy and and intuitive mouse way. can help HGR people systems to communicatehave also found with many computers applications in more in environments easy and intuitive which way. demand HGR systemscontactless have interac- also foundtion with many machinery, applications such in as environments hospital surger whichy rooms demand [26,27], contactless to prevent interactionthe spread withof vi- machinery,ruses and germs. such as As hospital a result, surgery contactless rooms HCI [26 can,27 ],be to a preventsafe means the spreadof man–machine of viruses inter- and germs.action in As epidemiological a result, contactless situations HCI such can be as aMERS safe means and the of recent man–machine and ongoing interaction COVID-19 in epidemiologicaloutbreaks. situations such as MERS and the recent and ongoing COVID-19 outbreaks. FigureFigure 1.1.Usage Usage ofof differentdifferent bodybody partsparts toto makemake aa human–computerhuman–computer interface interface (HCI) (HCI) [ 25[25].]. 1.2. Hand-Gesture Based HCI Design 1.2. Hand-Gesture Based HCI Design The system overview of hand-gesture-based HCI development is explained in Figure2 . First,The a neural system spike overview is produced of hand-gesture-based in the brain, which HCI generatesdevelopment a signal is explained that results