DOCSLIB.ORG

An Intelligent Baby Monitor with Automatic Sleeping Posture Detection and Notification

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Intelligent Baby Monitor with Automatic Sleeping Posture Detection and Notification Article An Intelligent Baby Monitor with Automatic Sleeping Posture Detection and Notification Tareq Khan School of Engineering, Eastern Michigan University, Ypsilanti, MI 48197, USA; [email protected] Abstract: Artificial intelligence (AI) has brought lots of excitement to our day-to-day lives. Some examples are spam email detection, language translation, etc. Baby monitoring devices are being used to send video data of the baby to the caregiver’s smartphone. However, the automatic understanding of the data was not implemented in most of these devices. In this research, AI and image processing techniques were developed to automatically recognize unwanted situations that the baby was in. The monitoring device automatically detected: (a) whether the baby’s face was covered due to sleeping on the stomach; (b) whether the baby threw off the blanket from the body; (c) whether the baby was moving frequently; (d) whether the baby’s eyes were opened due to awakening. The device sent notifications and generated alerts to the caregiver’s smartphone whenever one or more of these situations occurred. Thus, the caregivers were not required to monitor the baby at regular intervals. They were notified when their attention was required. The device was developed using NVIDIA’s Jetson Nano microcontroller. A night vision camera and Wi-Fi connectivity were interfaced. Deep learning models for pose detection, face and landmark detection were implemented in the microcontroller. A prototype of the monitoring device and the smartphone app were developed and tested successfully for different scenarios. Compared with general baby monitors, the proposed device gives more peace of mind to the caregivers by automatically detecting un-wanted situations. Keywords: human pose detection; face detection; deep learning; cloud messaging; hypertext transfer Citation: Khan, T. An Intelligent protocol (HTTP) server; jetson nano Baby Monitor with Automatic Sleeping Posture Detection and Notification. AI 2021, 2, 290–306. https://doi.org/10.3390/ai2020018 1. Introduction Smart baby monitoring devices are being used to obtain and send video and audio Academic Editor: Giovanni Diraco data of the baby to the caregiver’s smartphone, but most of these devices are unable to recognize or understand the data. In this project, a novel baby monitoring device is Received: 27 April 2021 developed which automatically recognizes the undesired and harmful postures of the baby Accepted: 15 June 2021 Published: 18 June 2021 by image processing and sends an alert to the caregiver’s smartphone—even if the phone is in sleep mode. Deep learning-based object detection algorithms are implemented in the Publisher’s Note: MDPI stays neutral hardware, and a smartphone app is developed. The overall system is shown in Figure1. with regard to jurisdictional claims in The research problem addressed in this paper is to develop methods to automatically published maps and institutional affil- detect (a) whether the baby’s face is covered due to sleeping on the stomach; (b) whether the iations. baby has thrown off the blanket from the body; (c) whether the baby is moving frequently; (d) whether the baby’s eyes are opened due to awakening. One of the challenges of deep learning models is running them in embedded systems where resources such as memory and speed are limited. The methods must work in an embedded system with low latency. The system should also work in both day and night conditions. The detection methods Copyright: © 2021 by the author. Licensee MDPI, Basel, Switzerland. should not be biased and be inclusive to all races of babies. This article is an open access article The objective of this study is to develop a baby monitoring device that will auto- distributed under the terms and matically recognize the harmful postures of the baby by image processing and send an conditions of the Creative Commons alert to the caregiver’s smartphone. The work will be considered successful when the Attribution (CC BY) license (https:// proposed baby monitor can automatically detect the targeted harmful postures and send a creativecommons.org/licenses/by/ notification to the smartphone. Experiments with different postures will be conducted and 4.0/). the latency of the detection algorithms will be measured. AI 2021, 2, 290–306. https://doi.org/10.3390/ai2020018 https://www.mdpi.com/journal/ai AI 2021, 2, FOR PEER REVIEW 2 baby monitor can automatically detect the targeted harmful postures and send a notifica- AI 2021, 2 tion to the smartphone. Experiments with different postures will be conducted291 and the latency of the detection algorithms will be measured. FigureFigure 1. 1.The The overall overall smart smart baby baby monitoring monitoring system: system: (a) baby (a) sleeping;baby sleeping; (b) smart (b) baby smart monitoring baby monitoring devicedevice automatically automatically detects detects harmful harmful postures postures such as such facecovered, as face covered, thrown off thrown the blanket, off the frequently blanket, fre- movingquently or moving awake; (cor) imageawake; data (c) are image sent todata the are smartphone sent to the through smartp thehone Internet through with the the message Internet of with the anymessage harmful of situation; any harmful (d) a real-timesituation; video (d) a of real-time the baby is video shown of in the the baby smartphone, is shown and in notifications the smartphone, andand alerts notifications are generated and alerts whenever are generated the caregiver whenever receives the a message caregiver of receives the harmful a message situation of of the harm- theful baby.situation of the baby. TheThe needs needs and and significances significance ofs the of proposedthe proposed system system are mentioned are mentioned below: below: • About 1300 babies died due to sudden infant death syndrome (SIDS), about 1300 deaths • About 1300 babies died due to sudden infant death syndrome (SIDS), about 1300 were due to unknown causes, and about 800 deaths were caused by accidental suffoca- tiondeaths and strangulationwere due to unknown in bed in 2018 causes, in the and USA ab [1out]. Babies 800 deaths are at higherwere caused risk for SIDSby accidental ifsuffocation they sleep on and their strangulation stomachs as it in causes bed in them 2018 to in breathe the USA less air.[1]. The Babies best are and at only higher risk positionfor SIDS for if a they baby tosleep sleep on is their on the stomachs back—which as it the causes American them Academy to breathe of Pediatrics less air. The best recommendsand only position through for the a baby’s baby firstto sleep year [is2]. on Sleeping the back—which on the back improvesthe American airflow. Academy Toof reduce Pediatrics the risk recommends of SIDS, the baby’s through face the should baby’s be uncovered, first year and[2]. bodySleeping temperature on the back im- shouldproves be airflow. appropriate To reduce [3]. The the proposed risk of baby SIDS, monitor the baby’s will automatically face should detect be uncovered, these and harmfulbody temperature postures of the should baby andbe appropriate notify the caregiver. [3]. The This proposed will help baby to reduce monitor SIDS. will auto- • Babies—especiallymatically detect fourthese months harmful or older—move postures of frequentlythe baby and during notify sleep the and caregiver. can throw This will off the blanket from their body [4]. The proposed system will alert when the baby is help to reduce SIDS. • moving frequently and also whether the blanket is removed. Thus, it helps to keep theBabies—especially baby warm. four months or older—move frequently during sleep and can • Babiesthrow may off wakethe blanket up in the from middle their of body the night [4]. The due proposed to hunger, pain,system or justwill toalert play when the withbaby the is parent. moving There frequently is an increasing and also call whether in the medical the blanket community is removed. to pay attention Thus, it helps to tokeep parents the when baby they warm. say their babies do not sleep [5]. The smart baby monitor detects • whetherBabies themay baby’s wake eyes up arein openthe middle and sends of the an alert. night Thus, due itto helps hunger, the parents pain, or know just to play whenwith the the baby parent. is awake There even is an if increasing he/she is not call crying. in the medical community to pay attention • Whento parents a baby when sleeps they ina say different their babies room, do the no caregiverst sleep [5]. need The to smart check baby the sleeping monitor detects condition of the baby after a regular interval. Parents lose an average of six months’ whether the baby’s eyes are open and sends an alert. Thus, it helps the parents know sleep during the first 24 months of their child’s life. Approximately 10% of parents managewhen the to getbaby only is 2.5awake h of continuouseven if he/she sleep is eachnot crying. night. Over 60% of parents with • babiesWhen aged a baby less thansleeps 24 monthsin a different get no more room, than the 3.25 caregivers h of sleep eachneed night. to check A lack the of sleeping sleepcondition can affect of the qualitybaby after of work a regular and driving; interval. create Parents mental healthlose an problems, average such of six as months’ anxietysleep during disorders the and first depression; 24 months and causeof their physical child’s health life. problems,Approximately such as obesity,10% of parents highmanage blood to pressure, get only diabetes, 2.5 h of and continuous heart disease sleep [6]. each The proposed night. Over smart 60% device of parents will with automaticallybabies aged detectless than the 24 situations months when get no the more caregiver’s than 3.25 attention h of sleep is required each night. and A lack of sleep can affect the quality of work and driving; create mental health problems, AI 2021, 2, FOR PEER REVIEW 3 such as anxiety disorders and depression; and cause physical health problems, such as obesity, high blood pressure, diabetes, and heart disease [6].
Load more

Recommended publications
  • Staff Briefing Package
    Staff Briefing Package Crib Bumpers Petition May 15, 2013 Table of Contents Briefing Memo ............................................................................................................................... iii TAB A: Epidemiology Memorandum .......................................................................................... 11 TAB B: Engineering Memorandum .............................................................................................. 18 TAB C: Economics Memorandum ............................................................................................... 23 TAB D: Compliance Memorandum .............................................................................................. 27 TAB E: Health Sciences Memorandum ........................................................................................ 29 TAB F: Project Manager, Infant Suffocation Project (1992-1995) Memorandum ...................... 78 TAB G: Advice from Other Agencies ........................................................................................ 106 ii Briefing Memo iii UNITED STATES CONSUMER PRODUCT SAFETY COMMISSION 4330 EAST WEST HIGHWAY BETHESDA, MARYLAND 20814 Memorandum Date: May 15, 2013 TO: The Commission Todd A. Stevenson, Secretary THROUGH: Stephanie Tsacoumis, General Counsel Kenneth R. Hinson, Executive Director FROM: DeWane Ray, Assistant Executive Director Office of Hazard Identification and Reduction Jonathan Midgett, PhD, Children’s Hazards Team Leader Office of Hazard Identification and Reduction SUBJECT:
    [Show full text]
  • DESIGN and DEVELOPMENT of WIRELESS BABY MONITORS Eric
    DESIGN AND DEVELOPMENT OF WIRELESS BABY MONITORS By Eric Yi-Kuo Jen B.A.Sc., Simon Fraser University, 2002 A PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ENGINEERING in the School of Engineering Science © Eric Yi-Kuo Jen, 2008 SIMON FRASER UNIVERSITY SUMMER 2008 All rights reserved. This work may not be reproduced in whole or in part, by photocopy or other means, without the permission of the author. SIMON FRASER UNIVERSITY LIBRARY Declaration of Partial Copyright Licence The author, whose copyright is declared on the title page of this work, has granted to Simon Fraser University the right to lend this thesis, project or extended essay to users of the Simon Fraser University Library, and to make partial or single copies only for such users or in response to a request from the library of any other university, or other educational institution, on its own behalf or for one of its users. The author has further granted permission to Simon Fraser University to keep or make a digital copy for use in its circulating collection (currently available to the public at the "Institutional Repository" link of the SFU Library website <www.lib.sfu.ca> at: <http://ir.lib.sfu.ca/handle/1892/112>) and, without changing the content, to translate the thesis/project or extended essays, if technically possible, to any medium or format for the purpose of preservation of the digital work. The author has further agreed that permission for multiple copying of this work for scholarly purposes may be granted by either the author or the Dean of Graduate Studies.
    [Show full text]
  • Installation Guide 1 • Flashes Red Slowly During Video Is Lost
    Go to www.vtechphones.com RM5864HD Overview to register your product for What’s in the box enhanced warranty support and 5-inch Smart Wi-Fi 1080p Camera unit overview Parent unit overview latest VTech product news. Your HD video monitor package contains the following items. Save your sales receipt and Pan and Tilt Monitor original packaging in the event warranty service is necessary. 1 Light sensor 1 9 1 2 Camera lens 2 3 Microphone 3 2 4 7 10 14 4 3 4 Infrared LEDs 5 10 • Allow you to see clearly in a dark surrounding. 6 11 7 11 12 5 LED indicator 5 8 15 • Red is steady on when the camera unit is 13 6 8 9 connecting to the parent unit directly in local mode. 1 LINK LED light 6 Arrow keys • Green is steady on when the camera unit • On when the parent unit is linked to the , , or and parent unit are connecting to your camera unit. • Press to navigate leftward, upward, home Wi-Fi network via the Wi-Fi router. • Flashes when the link to the camera unit rightward or downward, within the 9 main menu and submenus. Installation guide 1 • Flashes red slowly during video is lost. streaming in local mode. • While viewing image from the 2 4 7 2 LED light camera unit, press to pan the camera • Flashes green slowly10 during video • On when the parent unit is connected unit leftward, upward, rightward or Installation guide Quick start guide Important safety 3 downward. instructions streaming via home Wi-Fi network.
    [Show full text]
  • Quick Start Guide
    Go to www.vtechphones.com Important safety instructions What’s in the box VM3254 The applied nameplate is located at the bottom of the 17. Never push objects of any kind into this product 1 Connect the baby monitor (cont’d) 3 Positioning the baby monitor to register your product for enhanced baby unit’s base. through the slots because they may touch dangerous VM3254-2 When using your equipment, basic safety precautions voltage points or create a short circuit. Never spill liquid warranty support and the latest VTech should always be followed to reduce the risk of fire, of any kind on the product. Caution electric shock and injury, including the following: 18. To reduce the risk of electric shock, do not Charge the parent unit battery • Keep the baby unit out of the reach of your baby. Never place or mount the baby unit inside the product news. disassemble this product, but take it to an authorized Video Baby Monitor 1. Follow all warnings and instructions marked on the service facility. Opening or removing parts of the When you have connected and turned on the parent unit, the battery will be charged baby’s crib or playpen. product. product other than specified access doors may 2. Adult setup is required. expose you to dangerous voltages or other risks. automatically. 3. CAUTION: Do not install the baby unit at a height Incorrect reassembling can cause electric shock above 2 meters. when the product is subsequently used. NNotesOTES e t i u e q e r s e c o o c u n p N n a o .
    [Show full text]
  • For Fosbaby P1)
    UUsseerr MMaannuuaall Wireless HD Baby Monitor FosBaby FosBaby P1 V2.1 Table of Contents Security Warning...................................................................................................................................................................1 1 Overview.......................................................................................................................................................................... 1 1.1 Key Features........................................................................................................................................................1 1.2 Read Before Use.................................................................................................................................................2 1.3 Package Contents...............................................................................................................................................2 1.4 Physical Description........................................................................................................................................... 2 1.5 Micro-SD Card..................................................................................................................................................... 4 1.6 Hardware Installation..........................................................................................................................................5 2 Access the Camera.......................................................................................................................................................
    [Show full text]
  • Section 3 Sources of Radiofrequency Electromagnetic Fields
    Section 3 Sources of Radiofrequency Electromagnetic Fields Table of Contents 3.1 Radiofrequency Electromagnetic Waves ........................................................................... 23 What Is a Radiofrequency (RF) Wave? .................................................................... 23 What Is a Continuous RF Wave? ............................................................................ 23 What Is a Pulsed RF Wave? .................................................................................... 24 What is the “microwave hearing effect”? ............................................................... 24 Natural Sources of RF ........................................................................................... 24 Characteristics of Natural RF: ............................................................................... 25 Biological Sources of RF/EMF ................................................................................ 25 3.2 Consumer Products .................................................................................................................... 25 Wireless Phone Evolution ...................................................................................... 25 Wireless Phones ................................................................................................... 26 Mobile Phone Base Stations .................................................................................. 26 Baby Monitors .....................................................................................................
    [Show full text]
  • 2 Before Use 3 Using the Baby Monitor
    VM320 Important safety instructions 1 Connect and charge the battery 2 Before use The applied nameplate is located at the bottom of the baby kind on the product. Video Monitor unit’s base. 19. To reduce the risk of electric shock, do not NOTES Note When using your equipment, basic safety precautions disassemble this product, but take it to an authorized should always be followed to reduce the risk of fire, electric service facility. Opening or removing parts of the • The rechargeable battery is pre-installed in your parent unit. • This baby monitor is intended as an aid. It is not a substitute for proper adult supervision, and should not shock and injury, including the following: product other than specified access doors may expose be used as such. 1. Follow all warnings and instructions marked on the you to dangerous voltages or other risks. Incorrect • Use only the power adapters supplied with this product. product. reassembling can cause electric shock when the • Make sure the baby monitor is not connected to a switch controlled electric outlet. 2. Adult setup is required. product is subsequently used. Test your baby monitor 3. CAUTION: Do not install the baby unit at a height above 20. You should test the sound reception every time you turn • Connect the power adapters in a vertical or floor mount position only. The adapters’ tips are not 2 metres. on the units or move one of the components. designed to hold the weight of baby monitor, so do not connect them to any ceiling, under-the-table, You may test the baby monitor before initial use, and at regular 4.
    [Show full text]
  • Newborn Care Guide
    Newborn Care Guide Congratulations on the birth of your baby! Contacts During Your Stay Caring for a newborn is one of life’s most rewarding and challenging Patient/Family Initiated Rapid Response Our Rapid Response Team is available 24 hours experiences. Your journey will be filled with excitement, joy, and pos- a day to provide additional expertise to evaluate sibly even fear of the unknown. The information in this booklet was your medical condition when necessary. prepared by the healthcare professionals at Woman’s to answer many When To Call of the questions new parents have. Please feel free to call our staff at Call if there is a significant change in your any time while you are in the hospital. condition that you have reported to the primary care nurse/charge nurse, and you Best wishes to you, your baby, and your family! or your family member still have concerns. How To Call • Dial 8499 from any hospital phone. • Tell the operator: º Your name and room number and º A brief description of your condition Other questions may be asked of you in order to direct the appropriate response team. We hope you will never have to call our Rapid Response Team. Patient/Family Initiated Rapid Response is our way of partnering with you to ensure you receive the highest quality of care. Room Service/Food and Refreshments 6:30 AM-8:30 PM To place an order, extension 3663 Questions or concerns, extension 4669 Family members or guests can purchase guest trays. The charge for guests is collected when the meal is delivered rather than being added to your hospital bill.
    [Show full text]
  • Wislow: a Wi-Fi Network Performance Troubleshooting Tool for End Users
    WiSlow: A Wi-Fi Network Performance Troubleshooting Tool for End Users Kyung-Hwa Kim⇤, Hyunwoo Nam† and Henning Schulzrinne⇤ ⇤Department of Computer Science, Columbia University, New York, NY †Department of Electrical Engineering, Columbia University, New York, NY Abstract—Slow Internet connectivity is often caused by poor not depend on a specific network adapter such as the Atheros Wi-Fi performance. The main reasons of such performance degra- chipsets, which were used to achieve similar goals in other dation include channel contention and non-Wi-Fi interference. studies [5], [6]. These features enable WiSlow to run on Although these problem sources can be easily removed in many common end-user machines. cases once they are discovered, it is difficult for end users to identify the sources of such interference. First, we investigate behaviors of 802.11 networks such as retries, frame check sequence (FCS) errors, packet loss, and We investigated the characteristics of different sources that can degrade Wi-Fi performance, and developed WiSlow, a bit rate adaption, which can be observed on ordinary operating software tool that diagnoses the root causes of poor Wi-Fi systems. Our experimental results show that the statistical performance using user-level network probes, and leveraging peer patterns of the above variables vary depending on the problem collaboration to identify the physical location of these causes. sources. For example, with the interference that caused by WiSlow uses two main methods: packet loss analysis and 802.11 non-Wi-Fi devices, we observed a greater number of retried ACK number analysis. The accuracy of WiSlow exceeds 90% packets, fewer FCS errors, and larger variations in the bit rates when the sources are close to Wi-Fi devices.
    [Show full text]
  • Home Safety Checklist Reference Guide
    Home Safety Checklist Reference Guide Revised December 2012 Tobacco additions: November 2015 Created by: Family Home Visiting Unit Maternal and Child Health Section Community and Family Health Division P.O. Box 64882 St Paul, MN 55164-0882 MN Department of Health Family Home Visiting Table of Contents: INTRODUCTION 3 COMMUNICATION SKILLS 4 REFERENCE GUIDE 5 Safe Sleep 5 Bathroom 7 Safe Storage 8 Kitchen 10 Around the House 10 In the car 17 RESOURCES 19 Introduction Why a home safety checklist? Parents want a safe home for their children to play and learn in but sometimes they need a little help thinking about what poses a safety risk in their house. The Home Safety Checklist is intended to be a tool for family home visitors to use during visits with families. The home visitor will need to use professional judgment when deciding how to use the checklist with each family. The MDH Home Safety Checklist was originally created in 1990 and has been updated a few times in the years since. This current revision depicts what current childhood injury trend data show are the leading causes of unintentional injury as well as what family home visitors are seeing in the homes they visit in Minnesota. The revision was accomplished through the efforts of the advisory committee listed below. This group worked to ensure the new documents are accurate and user-friendly. For technical assistance, contact the Maternal and Child Health Section at 651-201-3760, or email the MDH Family Home Visiting program at [email protected]. 2009-2010 Home Safety Checklist Revision Advisory Committee: Local Public Health: Pat Anderson, Pine County Christine Andres, Kanabec County Lona Daley-Getz, Clay County Maren Harris, Dakota Healthy Families Marion Kershner, Ottertail County Laura Larson.
    [Show full text]
  • User's Manual Congratulations
    Go to www.vtechphones.com VM320 1 Connect and charge the battery 2 Before use 3 Using the baby monitor to register your product for enhanced warranty support and VM320-2 NOTES Note Power on or off the baby unit • This baby monitor is intended as an aid. It is not a substitute for proper adult supervision, and should not the latest VTech product news. • The rechargeable battery is pre-installed in your parent unit. • Slide the ON/OFF switch to ON to turn on the baby unit. The POWER LED light turns on. Video Monitor be used as such. • Slide the ON/OFF switch to OFF to turn off the baby unit. The POWER LED light turns off. • Use only the power adapters supplied with this product. • Make sure the baby monitor is not connected to a switch controlled electric outlet. Power on or off the parent unit Test your baby monitor • Slide the ON/OFF switch to ON to turn on the parent unit. The screen and the POWER LED • Connect the power adapters in a vertical or floor mount position only. The adapters’ prongs are not You may test the baby monitor before initial use, and at regular light turns on. designed to hold the weight of baby monitor, so do not connect them to any ceiling, under-the-table, times thereafter. • Slide the ON/OFF switch to OFF to turn off the parent unit. The screen and the POWER LED or cabinet outlets. Otherwise, the adapters may not properly connect to the outlets. light turns off.
    [Show full text]
  • Clearing the RF Smog: Making 802.11 Robust to Cross-Technology Interference
    Clearing the RF Smog: Making 802.11 Robust to Cross-Technology Interference Shyamnath Gollakota † Fadel Adib† Dina Katabi† Srinivasan Seshan⋆ † ⋆ Massachusetts Institute of Technology Carnegie Mellon University {gshyam, fadel, dina}@csail.mit.edu [email protected] ABSTRACT frequency band as wide as 802.11, and all of them emit power that is Recent studies show that high-power cross-technology interfer- comparable or higher than 802.11 devices [17]. Further, the number ence is becoming a major problem in today’s 802.11 networks. De- and diversity of such interferers is likely to increase over time due vices like baby monitors and cordless phones can cause a wire- to the proliferation of new technologies in the ISM band. less LAN to lose connectivity. The existing approach for dealing Traditional solutions that increase resilience to interference by with such high-power interferers makes the 802.11 network switch making 802.11 fall down to a lower bit rate are ineffective against to a different channel; yet the ISM band is becoming increasingly high-power cross-technology interference. As a result, the most crowded with diverse technologies, and hence many 802.11 access common solution today is to hop away to an 802.11 channel that points may not find an interference-free channel. does not suffer from interference [6, 38, 31, 32]. However, the ISM This paper presents TIMO, a MIMO design that enables 802.11n band is becoming increasingly crowded, making it difficult to find to communicate in the presence of high-power cross-technology an interference-free channel.
    [Show full text]