Facial Coding Platform for Measuring Emotions in Farm Animals
Total Page:16
File Type:pdf, Size:1020Kb
bioRxiv preprint doi: https://doi.org/10.1101/2021.04.09.439122; this version posted April 11, 2021. The copyright holder for this preprint (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. 1 Type of the Paper (Article) 2 Happy Cow or Thinking Pig? WUR Wolf – Facial Coding 3 Platform for Measuring Emotions in Farm Animals 4 Suresh Neethirajan1* 5 1 Adaptation Physiology Group, Department of Animal Sciences, Wageningen University & Research, 6700 6 AH Wageningen, The Netherlands. 7 * Correspondence: [email protected] 8 Abstract: Emotions play an indicative and informative role in the investigation of farm animal 9 behaviors. Systems that respond and can measure emotions provide a natural user interface in 10 enabling the digitalization of animal welfare platforms. The faces of farm animals can be one of the 11 richest channels for expressing emotions. We present WUR Wolf (Wageningen University & Re- 12 search: Wolf Mascot)—a real-time facial expression recognition platform that can automatically 13 code the emotions of farm animals. Using Python-based algorithms, we detect and track the facial 14 features of cows and pigs, analyze the appearance, ear postures, and eye white regions, and corre- 15 late with the mental/emotional states of the farm animals. The system is trained on dataset of facial 16 features of images of the farm animals collected in over 6 farms and has been optimized to operate 17 with an average accuracy of 85%. From these, we infer the emotional states of animals in real time. 18 The software detects 13 facial actions and 9 emotional states, including whether the animal is ag- 19 gressive, calm, or neutral. A real-time emotion recognition system based on YoloV3, and Faster 20 YoloV4-based facial detection platform and an ensemble Convolutional Neural Networks (RCNN) 21 is presented. Detecting expressions of farm animals simultaneously in real time makes many new 22 interfaces for automated decision-making tools possible for livestock farmers. Emotions sensing 23 offers a vast amount of potential for improving animal welfare and animal-human interactions. Citation: Lastname, F.; Lastname, F.;24 Keywords: Biometric verification; facial profile images; animal welfare; precision livestock farm- Lastname, F. Title. Sensors 2021, 21, 25 ing; welfare monitoring; YOLOv3; YOLOv4; faster RCNN x. https://doi.org/10.3390/xxxxx 26 Academic Editor: Firstname Lastname 27 1. Introduction Received: date 28 Digital technologies, in particular, precision livestock farming, and artificial intelligence Accepted: date 29 have the potential to shape the transformation in animal welfare [1]. To ensure access to Published: date 30 sustainable and high-quality health attention and welfare in animal husbandry man- Publisher’s Note: MDPI stays neu-31 agement, innovative tools are needed. Unlocking the full potential of automated meas- tral with regard to jurisdictional32 urement of mental and emotional states of farm animals through digitalization such as claims in published maps and insti-33 facial coding systems would help to blur the lines between biological, physical, and dig- tutional affiliations. 34 ital technologies. 35 Animal caretakers, handlers, and farmworkers typically rely on hands-on observations 36 and measurements while investigating methods to monitor animal welfare. To avoid the Copyright: © 2021 by the authors.37 increased handling of animals in the process of taking functional or physiological data, Submitted for possible open access 38 and to reduce the subjectivity associated with manual assessments, automated animal publication under the terms and 39 behavior and physiology measurement systems can complement the current traditional conditions of the Creative Commons 40 welfare assessment tools and processes in enhancing the detection of animals in distress Attribution (CC BY) license 41 or pain in the barn [2]. Automated and continuous monitoring of animal welfare through (http://creativecommons.org/licenses 42 digital alerting is rapidly becoming a reality [3]. /by/4.0/). Sensors 2021, 21, x. https://doi.org/10.3390/xxxxx www.mdpi.com/journal/sensors bioRxiv preprint doi: https://doi.org/10.1101/2021.04.09.439122; this version posted April 11, 2021. The copyright holder for this preprint (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. Sensors 2021, 21, x FOR PEER REVIEW 2 of 13 43 In the human context, facial recognition platforms have long been in use for various ap- 44 plications, such as password systems on smartphones, identification at international 45 border checkpoints, identification of criminals [4, diagnosis of Turner syndrome [5]; de- 46 tection of genetic disorder phenotypes [6]; as a potential diagnostic tool for Parkinson 47 disease [7]; measuring tourist satisfaction through emotional expressions [8]; and quan- 48 tification of customer interest during shopping [9]. 49 Emotions 50 Emotions are believed to be a social and survival mechanism that is present in many 51 species. In humans, emotions are understood as deep and complex psychological expe- 52 riences that influence physical reactions. There is an entire sector of science devoted to 53 understanding the sophisticated inner workings of the human brain, yet many questions 54 related to human emotions remain unanswered. Even less scientific research is focused 55 on understanding the emotional capacity of non-human primates and other animals. The 56 ability to interpret the emotional states of an animal is considerably more difficult than 57 understanding the emotional state of a human [10]. The human face is capable of a wide 58 array of expressions that communicate emotion and social intent to other humans. These 59 expressions are so clear that even some non-human species, like dogs, can identify hu- 60 man emotion through facial expression [11]. Each species has its own unique physiolog- 61 ical composition resulting in special forms of expression. Despite human intellectual ca- 62 pacity, emotional understanding of other species through facial observation has proven 63 difficult. 64 Early studies [12] noted the influence of human bias on interpretations and accidental 65 interference with the natural responses of animals. It is not uncommon for humans to 66 anthropomorphize the expressions of animals. The baring of teeth is an example. Hu- 67 mans commonly consider such an expression to be “smiling” and interpret it as a sign of 68 positive emotions. In other species, such as non-human primates, the baring of teeth is 69 more commonly an expression of a negative emotion associated with aggression [13]. 70 For these reasons and many others, the involvement of technology is critical in main- 71 taining accurate and unbiased assessments of animal emotions and individual animal 72 identification. In recent years, the number of studies concerning technological interven- 73 tion in the field of animal behavior has increased [10]. The ability of customized software 74 to improve research, animal welfare, the production of food animals, legal identification, 75 and medical practices is astounding. 76 Understanding Animal Emotions 77 The human comprehension of animal emotions may seem trivial; however, it is a mutu- 78 ally beneficial skill. The ability of animals to express complex emotions, such as love and 79 joy, is still being debated within the field of behavioral science. Other emotions, such as 80 fear, stress, and pleasure are studied more commonly. These basic emotions have an 81 impact on how animals feel about their environment and interact with it. It also impacts 82 an animal’s interactions with its counter specifics. 83 Non-domesticated species of animals are commonly observed in the wild and main- 84 tained in captivity to understand and conserve their species. Changes in the natural en- 85 vironment, because of human actions, can be stressful for individuals within a species. 86 Captive non-domesticated animals also experience stress created through artificial envi- 87 ronments and artificial mate selection. If even one animal experiences and displays signs 88 of stress or aggression, its companions are likely to understand and attempt to respond to 89 their emotional state [14]. These responses can result in stress, conflict, and the uneven bioRxiv preprint doi: https://doi.org/10.1101/2021.04.09.439122; this version posted April 11, 2021. The copyright holder for this preprint (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. Sensors 2021, 21, x FOR PEER REVIEW 3 of 13 90 distribution of resources [15]. The understanding of emotional expression in captive 91 animals can help caretakers determine the most beneficial forms of care and companion 92 matching for each individual, resulting in a better quality of life for the animals in ques- 93 tion. 94 Companion animals are another category of individuals who can benefit from a deeper 95 understanding of animal emotion. Just like humans, individual animals experience dif- 96 ferent thresholds for coping with pain and discomfort. Since many companion animals 97 must undergo voluntary medical procedures for the well-being of their health and their 98 species, it is important to understand their physical responses. Animals cannot tell hu- 99 mans how much pain they are in, so it is up to their caretakers to interpret the pain level 100 an animal is experiencing and treat it appropriately [16]. This task is most accurately 101 completed when the emotions of an animal are clearly and quickly detectable. 102 The understanding of expressions related to stress and pain is impactful in animal agri- 103 culture. Animals used for food production often produce higher quality products when 104 they do not experience unpleasant emotions [17]. The detection of individual animals 105 experiencing stress allows for the early identification of medical complications as well. A 106 study on sows in parturition showed a uniform pattern of facially expressed discomfort 107 during the birthing cycle [18].