DOCSLIB.ORG

3D Skeletal Joints-Based Hand Gesture Spotting and Classification

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
3D Skeletal Joints-Based Hand Gesture Spotting and Classification applied sciences Article 3D Skeletal Joints-Based Hand Gesture Spotting and Classification Ngoc-Hoang Nguyen, Tran-Dac-Thinh Phan , Soo-Hyung Kim, Hyung-Jeong Yang and Guee-Sang Lee * Department of Artificial Intelligence Convergence, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea; [email protected] (N.-H.N.); [email protected] (T.-D.-T.P.); [email protected] (S.-H.K.); [email protected] (H.-J.Y.) * Correspondence: [email protected] Abstract: This paper presents a novel approach to continuous dynamic hand gesture recognition. Our approach contains two main modules: gesture spotting and gesture classification. Firstly, the gesture spotting module pre-segments the video sequence with continuous gestures into isolated gestures. Secondly, the gesture classification module identifies the segmented gestures. In the gesture spotting module, the motion of the hand palm and fingers are fed into the Bidirectional Long Short- Term Memory (Bi-LSTM) network for gesture spotting. In the gesture classification module, three residual 3D Convolution Neural Networks based on ResNet architectures (3D_ResNet) and one Long Short-Term Memory (LSTM) network are combined to efficiently utilize the multiple data channels such as RGB, Optical Flow, Depth, and 3D positions of key joints. The promising performance of our approach is obtained through experiments conducted on three public datasets—Chalearn LAP ConGD dataset, 20BN-Jester, and NVIDIA Dynamic Hand gesture Dataset. Our approach outperforms the state-of-the-art methods on the Chalearn LAP ConGD dataset. Keywords: continuous hand gesture recognition; gesture spotting; gesture classification; multi-modal features; 3D skeletal; CNN Citation: Nguyen, N.-H.; Phan, T.-D.-T.; Kim, S.-H.; Yang, H.-J.; Lee, G.-S. 3D Skeletal Joints-Based Hand Gesture Spotting and Classification. 1. Introduction Appl. Sci. 2021, 11, 4689. https:// Nowadays, the role of dynamic hand gesture recognition has become crucial in vision- doi.org/10.3390/app11104689 based applications for human-computer interaction, telecommunications, and robotics, due to its convenience and genuineness. There are many successful approaches to isolated hand Academic Editor: Hyo-Jong Lee gesture recognition with the recent development of neural networks, but in real-world systems, the continuous dynamic hand gesture recognition remains a challenge due to the Received: 14 April 2021 diversity and complexity of the sequence of gestures. Accepted: 18 May 2021 Initially, most continuous hand gesture recognition approaches were based on tra- Published: 20 May 2021 ditional methods such as Conditional Random Fields (CRF) [1], Hidden Markov Model (HMM), Dynamic Time Warping (DTW), and Bézier curve [2]. Recently, deep learning Publisher’s Note: MDPI stays neutral methods based on convolution neural networks (CNN) and recurrent neural networks with regard to jurisdictional claims in (RNN) [3–7] have gained popularity. published maps and institutional affil- The majority of continuous dynamic hand-gesture recognition methods [3–6] include iations. two separate procedures: gesture spotting and gesture classification. They utilized the spatial and temporal features to improve the performance mainly in gesture classification. However, there are limitations in the performance of gesture spotting due to its inherent variability in the duration of the gesture. In existing methods, gestures are usually Copyright: © 2021 by the authors. spotted by detecting transitional frames between two gestures. Recently, an approach [7] Licensee MDPI, Basel, Switzerland. simultaneously performed the task of gesture spotting and gestures classification, but it This article is an open access article turned out to be suitable only for feebly segmented videos. distributed under the terms and Most of the recent researches [8–11] intently focus on improving the performance of conditions of the Creative Commons the gesture classification phase, while the gesture spotting phase is often neglected on the Attribution (CC BY) license (https:// assumption that the isolated pre-segmented gesture sequences are available for input to creativecommons.org/licenses/by/ the gesture classification. 4.0/). Appl. Sci. 2021, 11, 4689. https://doi.org/10.3390/app11104689 https://www.mdpi.com/journal/applsci Appl. Sci. 2021, 11, x FOR PEER REVIEW 2 of 13 Appl. Sci. 2021, 11, 4689 2 of 13 assumption that the isolated pre-segmented gesture sequences are available for input to the gesture classification. However,However, inin real-worldreal-world systems,systems, spottingspotting ofof thethe gesturegesture segmentationsegmentation playsplays aa crucialcrucial rolerole inin thethe whole whole process process of of gesture gesture recognition, recognition, hence, hence, it greatlyit greatly affects affects the the final final recognition recogni- performance.tion performance. In paper In paper [3], they [3], segmented they segmente the videosd the videos into sets into of imagessets of andimages used and them used to predictthem to the predict fusion the score, fusion which score, means which they mean simultaneouslys they simultaneously did the gesturedid the spottinggesture spot- and gestureting and classification. gesture classification. The authors The in authors [5] utilized in [5] the utilized Connectionist the Connectionist temporal classification temporal clas- to detectsification the to nucleus detect of the the nucleus gesture of and thethe gesture no-gesture and the class no-gesture to assist class the gesture to assist classification the gesture withoutclassification requiring without explicit requiring pre-segmentation. explicit pre-segmentation. In [4,6], the continuous In [4,6], the gestures continuous are often ges- spottedtures are into often isolation spotted based into onisolation the assumption based on thatthe assumption hands will always that hands be put will down always at the be endput down of each at gesturethe end whichof each turnedgesture out which to be turn inconvenient.ed out to be inconvenient. It does not work It does well not for work all situations,well for all suchsituations, as in “zoom such as in”, in “zoom “zoom in”, out” “zoom gestures, out” i.e., gestures, when onlyi.e., when the fingers only the move fin- whilegers move the hand while stands the hand still. stands still. InIn thisthis paper,paper, wewe proposepropose aa spotting-classificationspotting-classification algorithmalgorithm forfor continuouscontinuous dynamicdynamic handhand gesturesgestures which which we we separate separate the the two two tasks tasks like like [4, 6[4]] but and we [6] avoid but we the avoid existing the problems existing ofproblems those methods. of those Inmethods. the spotting In the module, spotting as mo showndule, as in shown Figure1 in, the Figure continuous 1, the continuous gestures fromgestures the unsegmentedfrom the unsegmented and unbounded and unbounde input streamd input are firstly stream segmented are firstly into segmented individually into isolatedindividually gestures isolated based gestures on 3D keybased joints on 3D extracted key joints from extracted each frame from by each 3D humanframe by pose 3D andhuman hand pose pose and extraction hand pose algorithm. extraction Thealgorithm. time series The time of 3D series key of poses 3D key are poses fed into are thefed Bidirectionalinto the Bidirectional Long Short-Term Long Short-Term Memory (Bi-LSTM)Memory (Bi-LSTM) networkwith network connectionist with connectionist temporal classificationtemporal classification (CTC) [12 ](CTC) for gesture [12] for spotting. gesture spotting. FigureFigure 1.1. GestureGesture Spotting-ClassificationSpotting-Classification Module.Module. TheThe isolatedisolated gestures gestures segmented segmented using using the the gesture gesture spotting spotting module module are are classified classified in the in gesturethe gesture classification classification module module with with a multi-modal a multi-modal M-3D M-3D network. network. As indicated As indicated in Figure in Fig-1, inure the 1, gesturein the gesture classification classification module, module, the M-3D the networkM-3D network is built is by built combining by combining multi-modal multi- datamodal inputs data whichinputs comprisewhich comprise RGB, OpticalRGB, Optical Flow, Depth,Flow, Depth, and 3D and pose 3D informationpose information data channels.data channels. Three Three residual residual 3D Convolution 3D Convolutio Neuraln Neural Network Network based based on ResNet on ResNet architectures architec- (3D_ResNet)tures (3D_ResNet) [13] stream [13] stream networks networks of RGB, of OpticalRGB, Optical Flow andFlow Depth and Depth channel channel along along with anwith LSTM an LSTM network network of 3D of pose 3D pose channel channel are effectively are effectively combined combined using using a fusion a fusion layer layer for gesturefor gesture classification. classification. TheThe preliminarypreliminary versionversion of of this this paper paper has has appeared appeared in in [14 [14].]. In In this this paper, paper, depth depth infor- in- mationformation has has been been considered considered together together with with 3D skeleton3D skeleton joints joints information information with with extensive exten- experiments,sive experiments, resulting resulting in upgraded in upgraded performance. performance. TheThe remainderremainder of of this this paper paper is is organized organized as follows.as follows. In SectionIn Section2, we 2, review we review the related the re- works.lated
Load more

Recommended publications
  • The Meanings of the Term Mudra and a Historical Outline of "Hand
    The Meanings of the term Mudra T h e M and a Historical Outline of ae n ni "Hand gestures" g s o f ht e Dale Todaro t re m M u d 梗 概 ar a この 拙 論 は2部 に分 か れ る。 n d 第1部 は"mudra"と い う語 の最 も一 般 的 な 定 義 を 扱 う。仏 教 ・ヒ ン ドゥー 教 a H を 研 究 して い る学 者 や東 洋 の 図像 学 の専 門 家 は、 大 抵、"皿udra"の さ ま ざ まな 意 i torical Outline味 を 知 って い る。 しか し、特 に タ ン トラ にお い て 使 用 され た"mudr翫"の す べ て の 定 義 が、 どん な 参考 文 献 に も見 つ か るわ け で は な い。 従 って、 第1部 は これ ら 種 々の、 一 般 的 な"mudra"の 語 法 を集 め る よ う試 み た。 又、 イ ン ドの舞 踏 や 劇 につ いて 書 いた 人 が、"hasta"と い う語 を 使 用 す べ きで あ るの に、 専 門的 に言 え ば 誤 って"mudra"を 用 いて い る。 それ に つ いて も説 明 を試 み た。 fo " 第1部 よ りも長 い 第2部 で は、"印 契(手 印)"と い う意 味 で使 用 され た"mu- H a dra"の 歴 史 の あ らま しを、 系 統 的 に述 べ た。 印契 の歴 史 上 異 な った 使 用 と意 味 n d g は、 次 の4に お い て 顕著 にみ られ る。 即 ち、1)ヴ ェー ダ の儀 礼、2)規 格 化 され た se ut イ ン ドの舞 踏、3)イ ン ドの彫 刻(仏 教、 ヒ ン ド ゥー 教、 ジ ャイ ナ教)、4)タ ン ト r s"e ラの 成 就 法、 で あ る。 これ ら4の 分 野 は す べ て、 共 通 して、 イ ン ドで 使 用 され た 印 契 の 伝統 か ら由 来 して い る。 そ しで、 い くつか の事 例 に お いて、 イ ン ドか ら 日 本 密 教 の 伝 統 まで に わ た って、 特 定 の"mudra"が 驚 くほ ど継 続 して 使 用 され て い るこ とが、 証 明 で き る。 Introduction The goal of this short essay is twofold.
    [Show full text]
  • This List of Gestures Represents Broad Categories of Emotion: Openness
    This list of gestures represents broad categories of emotion: openness, defensiveness, expectancy, suspicion, readiness, cooperation, frustration, confidence, nervousness, boredom, and acceptance. By visualizing the movement of these gestures, you can raise your awareness of the many emotions the body expresses without words. Openness Aggressiveness Smiling Hand on hips Open hands Sitting on edge of chair Unbuttoning coats Moving in closer Defensiveness Cooperation Arms crossed on chest Sitting on edge of chair Locked ankles & clenched fists Hand on the face gestures Chair back as a shield Unbuttoned coat Crossing legs Head titled Expectancy Frustration Hand rubbing Short breaths Crossed fingers “Tsk!” Tightly clenched hands Evaluation Wringing hands Hand to cheek gestures Fist like gestures Head tilted Pointing index finger Stroking chins Palm to back of neck Gestures with glasses Kicking at ground or an imaginary object Pacing Confidence Suspicion & Secretiveness Steepling Sideways glance Hands joined at back Feet or body pointing towards the door Feet on desk Rubbing nose Elevating oneself Rubbing the eye “Cluck” sound Leaning back with hands supporting head Nervousness Clearing throat Boredom “Whew” sound Drumming on table Whistling Head in hand Fidget in chair Blank stare Tugging at ear Hands over mouth while speaking Acceptance Tugging at pants while sitting Hand to chest Jingling money in pocket Touching Moving in closer Dangerous Body Language Abroad by Matthew Link Posted Jul 26th 2010 01:00 PMUpdated Aug 10th 2010 01:17 PM at http://news.travel.aol.com/2010/07/26/dangerous-body-language-abroad/?ncid=AOLCOMMtravsharartl0001&sms_ss=digg You are in a foreign country, and don't speak the language.
    [Show full text]
  • Hand Gestures
    L2/16-308 More hand gestures To: UTC From: Peter Edberg, Emoji Subcommittee Date: 2016-10-31 Proposed characters Tier 1: Two often-requested signs (ILY, Shaka, ILY), and three to complete the finger-counting sets for 1-3 ​ (North American and European system). None of these are known to have offensive connotations. HAND SIGN SHAKA ● Shaka sign ​ ● ASL sign for letter ‘Y’ ● Can signify “Aloha spirit”, surfing, “hang loose” ● On Emojipedia top requests list, but requests have dropped off ​ ​ ● 90°-rotated version of CALL ME HAND, but EmojiXpress has received requests for SHAKA specifically, noting that CALL ME HAND does not fulfill need HAND SIGN ILY ● ASL sign for “I love you” (combines signs for I, L, Y), has moved into ​ ​ mainstream use ● On Emojipedia top requests list ​ HAND WITH THUMB AND INDEX FINGER EXTENDED ● Finger-counting 2, European style ● ASL sign for letter ‘L’ ● Sign for “loser” ● In Montenegro, sign for the Liberal party ● In Philippines, sign used by supporters of Corazon Aquino ● See Wikipedia entry ​ ​ HAND WITH THUMB AND FIRST TWO FINGERS EXTENDED ● Finger-counting 3, European style ● UAE: Win, victory, love = work ethic, success, love of nation (see separate proposal L2/16-071, which is the source of the information ​ ​ below about this gesture, and also the source of the images at left) ● Representation for Ctrl-Alt-Del on Windows systems ● Serbian “три прста” (tri prsta), symbol of Serbian identity ​ ​ ● Germanic “Schwurhand”, sign for swearing an oath ​ ​ ● Indication in sports of successful 3-point shot (basketball), 3 successive goals (soccer), etc. HAND WITH FIRST THREE FINGERS EXTENDED ● Finger-counting 3, North American style ● ASL sign for letter ‘W’ ● Scout sign (Boy/Girl Scouts) is similar, has fingers together ​ Tier 2: Complete the finger-counting sets for 4-5, plus some less-requested hand signs.
    [Show full text]
  • Pointing Gesture in Young Children Hand Preference and Language Development
    Pointing gesture in young children Hand preference and language development Hélène Cochet and Jacques Vauclair Aix-Marseille University This paper provides an overview of recent studies that have investigated the development of pointing behaviors in infants and toddlers. First, we focus on deictic gestures and their role in language development, taking into account the different hand shapes and the different functions of pointing, and examining the cognitive abilities that may or may not be associated with the production of pointing gestures. Second, we try to demonstrate that when a distinction is made between pointing gestures and manipulative activities, the study of children’s hand preference can help to highlight the development of speech-gesture links. Keywords: toddlers, gestural communication, pointing, handedness, speech- gesture system Emergence of communicative gestures: Focus on pointing gestures Pointing is a specialized gesture for indicating an object, event or location. Chil- dren start using pointing gestures at around 11 months of age (Butterworth & Morissette, 1996; Camaioni, Perucchini, Bellagamba, & Colonnesi, 2004), and this behavior opens the door to the development of intentional communication. One of the prerequisites for the production of pointing gestures is a shared experi- ence between the signaler and the recipient of the gesture, that is, a simultaneous engagement with the same external referent, usually referred to as joint attention (e.g., Carpenter, Nagell, & Tomasello, 1998). While pointing is sometimes regard- ed as a “private gesture” (Delgado, Gómez, & Sarriá, 2009), whose main role is to regulate the infant’s attention rather than to enable the latter to communicate with a recipient, a growing body of research suggests that the onset of pointing gestures reflects a newly acquired ability to actively direct the adult’s attention to outside entities in triadic interactions (e.g., Liszkowski, 2005; Tomasello, Carpenter, & Liszkowski, 2007).
    [Show full text]
  • 16 Gestures by 16 Months
    16 Gestures by 16 Months Children Should Learn at Least 16 Gestures by 16 Months Good communication development starts in the first year of life and goes far beyond learning how to talk. Communication development has its roots in social interaction with parents and other caregivers during everyday activities. Your child’s growth in social communication is important because it helps your child connect with you, learn language and play concepts, and sets the stage for learning to read and future success in school. Good com- munication skills are the best tool to prevent behavior problems and make it easier to work through moments of frustration that all infants and toddlers face. Earlier is Better Catching communication and language difficulties By observing children’s early early can prevent potential problems later with gestures, you can obtain a critical behavior, learning, reading, and social interaction. snapshot of their communication Research on brain development reminds us that “earlier IS better” when teaching young children. development. Even small lags in The most critical period for learning is during the communication milestones can first three years of a child’s life. Pathways in the add up and impact a child’s rate brain develop as infants and young children learn of learning that is difficult to from exploring and interacting with people and objects in their environment. The brain’s architec- change later. Research with young ture is developing the most rapidly during this crit- children indicates that the development of gestures from 9 to 16 ical period and is the most sensitive to experiential months predicts language ability 2 years later, which is significant learning.
    [Show full text]
  • University of Groningen a Cultural History of Gesture Bremmer, JN
    University of Groningen A Cultural History Of Gesture Bremmer, J.N.; Roodenburg, H. IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below. Document Version Publisher's PDF, also known as Version of record Publication date: 1991 Link to publication in University of Groningen/UMCG research database Citation for published version (APA): Bremmer, J. N., & Roodenburg, H. (1991). A Cultural History Of Gesture. s.n. Copyright Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons). The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne- amendment. Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum. Download date: 02-10-2021 The 'hand of friendship': shaking hands and other gestures in the Dutch Republic HERMAN ROODENBURG 'I think I can see the precise and distinguishing marks of national characters more in those nonsensical minutiae than in the most important matters of state'.
    [Show full text]
  • The Use of Gesture in Architecture
    the use of gesture donald kunze • art3idea.psu.edu/boundaries/ In the design of architecture and landscape, the role of dynamics is evident not simply in accommodation of the (presumably) mobile behaviors of the occupants and their perceptions but as an indicator of a ‘signifier-ness’ (a sense of meaningfulness without any particular meaning attached). ‘Gesture’ stands for the ways in which mate- rial form can be imagined to express in ways analogous to human gesturing: pointing, opening, perching, facing, gazing, sneaking, waiting … in short, the full range of dramatic positions that an actor on the stage might take to indicate attitude, intention, emotion, or rôle. Gesture typically involves symmetrical signing. Thickening of one element is accompanied by the thinning of another. A opening directed skyward requires an opening (or closing) at a low point. These pairings can form an ‘idiotic symmetry’ (each element, though balanced by another, equally lacks an assigned function or meaning). Alternately, pairs can refer to a missing or silent third term that works to prevent or foreclose some spatial/temporal relationship. 1. gesture in general Gesture is the ‘language of the body’, a means of silent speech that can be read as intentional or unintentional. Gesture animates and re-configures the body, its parts, and their relationships. Gesture can create divisions, such as an audience and stage division. It can indicate impossibility (biting one’s elbow, an Italian gesture), difficulty (biting one’s hand), intelligence (pulling down the lower eye-lid), discovery (pointing to the head), idiocy (a screwing motion with the index finger to the cheek, an offensive gesture in Germany), and the countless obscene gestures in all cultures that are used for insult or mockery.
    [Show full text]
  • Prepose: Privacy, Security, and Reliability for Gesture-Based Programming
    Prepose: Privacy, Security, and Reliability for Gesture-Based Programming Lucas Silva Figueiredo∗, Benjamin Livshitsy, David Molnary, and Margus Veanesy Federal University of Pernambuco∗ Microsoft Researchy F Abstract|With the rise of sensors such as the Microsoft Kinect, Leap of an OS abstraction called a recognizer which enables Motion, and hand motion sensors in phones (i.e., Samsung Galaxy S6), gesture detection. Yet their approach fails to provide a gesture-based interfaces have become practical. Unfortunately, today, way to extend the system with new recognizers in a safe to recognize such gestures, applications must have access to depth and manner. SurroundWeb [27] demonstrates what a 3D web video of the user, exposing sensitive data about the user and her environ- ment. Besides these privacy concerns, there are also security threats in browser modified with new abstractions for input and sensor-based applications, such as multiple applications registering the output to protect privacy and security would look like. Yet same gesture, leading to a conflict (akin to Clickjacking on the web). it also lacks the capacity for precise automatic reasoning. We address these security and privacy threats with Prepose, a novel We are also inspired by world-drive access control [24], domain-specific language (DSL) for easily building gesture recognizers, which attempts to restrict applications from accessing combined with a system architecture that protects privacy, security, and sensitive objects in the environment. Lastly, Proton [15] reliability with untrusted applications. We run Prepose code in a trusted is an example of defining a higher-level abstraction for core, and only return specific gesture events to applications.
    [Show full text]
  • Module-04-Greetings-Etiquette-And
    Event Planner Module 04 4. Module 04- Greetings etiquette and handshake protocol Table of Contents 4. Module 04- Greetings etiquette and handshake protocol ............................................................................ 1 4.1 Managing Greetings etiquette and handshake protocol ......................................................................... 2 4.2 Handshake, greetings, and kisses ............................................................................................................ 3 4.3 Personal space ......................................................................................................................................... 5 4.4 Hand kisses ............................................................................................................................................... 5 1 | P a g e 4.1 Managing Greetings etiquette and handshake protocol Working as an event manager, you will certainly deal with people from different countries, cultures, and backgrounds. This can be fun, but also can be a little complicated. The customs and rituals involved in greeting often differ from country to country, and unfamiliar customs may be confusing. The situation becomes even more complex when different greeting gestures are required between men and women, women and women, and men and men. Luckily, the ultimate pleasant gesture is used in every country. Although occasionally, in some cultures and under certain circumstances, it may have negative connotations, this ultimate gesture is seldom misunderstood
    [Show full text]
  • Gestures: Your Body Speaks
    GESTURES: YOUR BODY SPEAKS How to Become Skilled WHERE LEADERS in Nonverbal Communication ARE MADE GESTURES: YOUR BODY SPEAKS TOASTMASTERS INTERNATIONAL P.O. Box 9052 • Mission Viejo, CA 92690 USA Phone: 949-858-8255 • Fax: 949-858-1207 www.toastmasters.org/members © 2011 Toastmasters International. All rights reserved. Toastmasters International, the Toastmasters International logo, and all other Toastmasters International trademarks and copyrights are the sole property of Toastmasters International and may be used only with permission. WHERE LEADERS Rev. 6/2011 Item 201 ARE MADE CONTENTS Gestures: Your Body Speaks............................................................................. 3 Actions Speak Louder Than Words...................................................................... 3 The Principle of Empathy ............................................................................ 4 Why Physical Action Helps........................................................................... 4 Five Ways to Make Your Body Speak Effectively........................................................ 5 Your Speaking Posture .................................................................................. 7 Gestures ................................................................................................. 8 Why Gestures? ...................................................................................... 8 Types of Gestures.................................................................................... 9 How to Gesture Effectively..........................................................................
    [Show full text]
  • Emblems & Illustrators Kinesics • Body
    Chapter8, Presentation 2 1 CHAPTER 8 The Nonverbal Code Part 2: Body language - Emblems & Illustrators 2 Kinesics • Body movement, including gestures, hand/arm movement, leg movement, facial expressions, eye gaze, and stance/ posture • Categories…. – Emblems—direct literal verbal translation – Illustrators—accent/complement what is being said • Meta-communicative – Affect displays—facial expressions of emotion • Considered universal – Regulators—behaviors/actions that govern, direct, or manage conversation – Adaptors—actions that satisfy physiological or psychological needs 3 Emblems & Illustrators • Emblems are primarily hand gestures that have a direct literal verbal translation • Illustrators are typically hand and arm movements that accompany speech or function to accent or complement what is being said 4 What does it mean? 5 What does it mean? 6 What does it mean? 7 Functions of emblems & illustrators • Main uses: 1. Greetings & departures 2. To insult or to utter obscenities to others 3. To indicate fight or flight 4. To designate friendly or romantic relationships 1 • Also common: signs for homosexuals • Gestures culturally bound – Sometimes gestures can be regional in nature 8 Emblem gestures: Yes and no 9 Emblem gestures: Yes and no • Regional gestures – “Gesture frontiers” amazingly resilient, resistant to national boundaries – Historical origins – Head toss: originally Greek, used in southern Italy 10 Illustrator Example: Greetings 11 Illustrator Example: Greetings • Handshake: far from universal – “Limp fish” universally disdained?
    [Show full text]
  • Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: a Review
    remote sensing Review Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review Shahzad Ahmed 1 , Karam Dad Kallu 2, Sarfaraz Ahmed 1 and Sung Ho Cho 1,* 1 Department of Electronic Engineering, Hanyang University, 222 Wangsimini-ro, Seongdong-gu, Seoul 133-791, Korea; [email protected] (S.A.); [email protected] (S.A.) 2 Robotics and Intelligent Machine Engineering (RIME), School of Mechanical and Manufacturing Engineering (SMME), National University of Science and Technology (NUST), H-12 Islamabad 44000, Pakistan; [email protected] * Correspondence: [email protected]; Tel.: +82-(02)-2220-4883 Abstract: Human–Computer Interfaces (HCI) deals with the study of interface between humans and computers. The use of radar and other RF sensors to develop HCI based on Hand Gesture Recognition (HGR) has gained increasing attention over the past decade. Today, devices have built-in radars for recognizing and categorizing hand movements. In this article, we present the first ever review related to HGR using radar sensors. We review the available techniques for multi-domain hand gestures data representation for different signal processing and deep-learning-based HGR algorithms. We classify the radars used for HGR as pulsed and continuous-wave radars, and both the hardware and the algorithmic details of each category is presented in detail. Quantitative and qualitative analysis of ongoing trends related to radar-based HCI, and available radar hardware and algorithms is also presented. At the end, developed devices and applications based on gesture- recognition through radar are discussed. Limitations, future aspects and research directions related to this field are also discussed.
    [Show full text]