INTERSPEECH 2006 - ICSLP Automatic Acoustic Identification of Insects Inspired by the Speaker Recognition Paradigm Ilyas Potamitis†, Todor Ganchev‡, Nikos Fakotakis‡ † Department of Music Technology and Acoustics, Technological Educational Institute of Crete, Daskalaki-Perivolia, 74100, Rethymno, Crete, Greece [email protected] ‡ Department of Electrical and Computer Engineering, University of Patras, 26500 Rion-Patras, Greece {tganchev, fakotaki}@wcl.ee.upatras.gr Abstract processing and pattern recognition has introduced the possibil- ity of automatically identifying species primarily on the basis of This work reports our research efforts towards developing effi- capturing subtle differences by means of image [1-3] and acous- cient equipment for the automatic acoustic recognition of in- tic signal processing [4-6]. However, the application of pattern sects. In particular, we discuss the characteristics of the acoustic recognition and machine learning techniques to this kind of patterns of a target insect family, namely the cricket family. To problem is still in its infancy. address the recognition problem we apply a feature extraction In brief, acoustic identification of insects is based on their methodology that has been inspired by well documented tactics ability to generate sound either deliberately as a means of com- of speech processing, which were adapted here to the specifics munication or as a by-product of eating, flight or locomotion. of the sound production mechanism of insects, in combination Provided that the bioacoustic signal produced by insects follows with state-of-the-art speaker identification technology. We apply a consistent acoustical pattern that is species-specific, it can be this approach to a large and well documented database of fami- employed for detection and identification purposes. lies and subfamilies of cricket sounds, and we report results that In the present contribution, we address a bioacoustic signal exceed 99% recognition accuracy on the levels of family and classification problem by exploiting technology that was ini- subfamily, and 94% on the level of a specific insect out of 105 tially developed for speech recognition, and afterwards success- species. We deem this equipment will be of practical benefit for fully adapted for the needs of speaker recognition. In particular, non-intrusive acoustic environmental monitoring applications as by adopting the statistical learning techniques inherent to it is directly expandable to other insect species. speaker identification [7], we aim at categorizing acoustic re- Index Terms: bioacoustics, insects, identification cordings to specific families and subfamilies of insects, as well as at identifying the definite species. However, the applicability 1. Introduction of these techniques and especially the feature extraction proce- There are more than 900,000 known insect species and there dure is not straightforward, as the spectral patterns of insect may be as many as ten times that number yet to be identified, sounds differ to a great extent from those of speech, mainly due 10.21437/Interspeech.2006-197 forming the largest ecological group on Earth. Beyond the scien- to the different sound production process (insects do not possess tific interest of investigating the diversity of biological organ- vocal tract and do not use their mouth to produce vocalizations). isms, insects have great economic importance as beneficial or- The selected target groups of insects to be identified are ganisms in agriculture and forestry (insects play significant role families, sub-families and specific species of crickets mainly in the food chain of other species and the fertility of plants). from the singing insects of North America collection (SINA) However, a number of insect species also have negative contri- [8]. This group has been chosen because the SINA project pre- bution to agricultural economy as they constitute a threat to sents a large and representative collection of cricket sounds of plants and crops. North America that are identified and tagged by scientists of Insects are mainly identified by their appearance and sound considerable experience in identifying the taxonomy of insect production that are species-specific. The detection and species species. recognition of insects are usually carried out manually, using The long term objectives of this work are: trapping and observation methods. The detection and identifica- x The development of a pilot automatic detection/ identification tion process is in most cases a highly complex procedure be- equipment capable of detecting/identifying insect species. cause insects are heard more often than seen or trapped (espe- Progressively, this equipment can be extended to other living cially those that live in complex environments or demonstrate organisms that are able to produce consistent acoustic pat- nocturnal activity). Moreover, the development of human exper- terns. This will allow unmanned, non-invasive acoustic sur- tise to capture taxonomic information is costly both in time and veying and environmental monitoring, which will cost- money and requires the construction of expensive reference effectively assess and categorize the biological diversity of collections of fragile insect specimens and comprehensive litera- large regions. ture sources. Non-experts have great difficulty practicing taxon- x The recognition of a wide range of taxa by non-specialists. omy while participating in the construction of biological inven- x Automatic acoustic identification of pests and selective acti- tories, even for routine identifications. vation of repelling mechanisms based on ultrasound emission, Recent progress in computer technology as well as in signal which is tuned to specific insects. 2126 September 17-21, Pittsburgh, Pennsylvania INTERSPEECH 2006 - ICSLP 2. Sound production in insects b) rhythm and duration of pulsations, c) spread of spectral energy around the dominant harmonic, There is a specific number of behavioural modes that have been d) energy of the overtones. observed in connection with sound production in insects. The The most challenging task is to classify cricket species belong- first mode includes those situations in which the insects produce ing to the same family and subfamily as their tonal characteris- sounds to attract the female mates into close proximity (e.g. tics bear resemblance to each other. In Fig. 2 we depict spectro- crickets produce the so called ‘courtship’ or ‘mating’ songs) or grams of members of the same subfamily (top right and top left to cause the female to produce a sound that will help the male to figures, the subfamily Trigonidiinae) and different subfamilies locate her (slant-faced grasshoppers) or cause congregation of (bottom left and right figures, subfamily Oecanthinae and sub- large numbers of males and females (cicadas). The second gen- family Nemobiinae). Another difficulty is that some species eral behavioural mode consists of the situations in which (fol- admit a very sparse representation of their signal in the time- lowing [9]): a) the sound is produced as a reaction to the pres- frequency domain (e.g. subfamily Eneopterinae). Finally, the ence or activities of other organisms. In particular, males, fe- pulsations per unit time are dependent on the environmental males, or immature insects produce acoustic emissions in order settings (e.g., temperature, humidity) while the fundamental to declare their disturbance (when captured and held or because remains fairly unchanged even in different behavioural modes. of the presence of another organism), or warn other insects of Stridulation - Katydid Tymbal Mechanism - Cicada danger, b) a male insect may sing in order to let other males know that an area is his territory, (i.e. mark their territory gener- Hertz 1.5 Hertz 1.5 ally called ‘warning’, ‘intimidation’ or ‘fight’ sounds), c) a fe- 4 4 male produces acoustic emissions in the presence of a male of 1 1 the same species. 0.5 0.5 Besides the sound generation as a means of communication, Frequency 10 Frequency 10 0 0 sound can be produced non-intentionally as a result of eating, 0.5 1 1.5 2 2,5 0.5 1 1.5 2 2.5 flying or locomotion. The sound production mechanism in in- Time Time sects can be summarized as: muscle power contraction leading Vibration - Asian Tiger Mosquito Air Expulsion - short horned grasshopper to mechanical vibration of the sound-producing structure and finally to acoustic loading of this source and sound radiation Hertz 1.5 Hertz 1.5 [10], [11]. 4 4 Sounds are produced by insects in five different ways [9]: 1 1 1. Stridulation: the friction of two body parts; usually heard as 0.5 0.5 chirping, i.e., (crickets, katydids, grasshoppers, bugs, beetles, Frequency 10 Frequency 10 0 0 moths, butterflies, ants, caterpillars, beetle larvae, others) 0.5 1 1.5 2 2.5 0.5 1 1.5 2 2.5 2. Percussion: by striking some body part, such as the feet Time Time (band-winged grasshoppers), the tip of the abdomen (cock- Figure 1 Spectrograms of members of different insect families roaches), or the head (death-watch beetle) against the sub- having different sound production mechanisms strate usually heard as tapping or drumming Top left: Stridulation: Katydid 3. Vibration: the oscillation of body parts such as wings; usu- Top right: Tymbal: Cicada ally heard as humming or rumbling by vibrating some body Bottom left: Vibration: Asian tiger mosquito Bottom right: Air Expulsion: Short-horned grasshopper part, such as the wings, in air (mosquitoes, flies, wasps, bees, others) Anaxipha delicatula Anaxipha n. sp. G TJW 4. Tymbal Mechanism: the quick contraction and release of tymbal muscles (vibrating drum-like membranes); usually Hz Hz heard as a series of clicking sounds (cicadas, leafhoppers, 4 1.5 4 1.5 treehoppers, spittlebugs) 1 1 5. Air Expulsion: the ejection of air or fluid through a body 0.5 0.5 constriction; usually heard as a whistle or hiss (short-horned Frequency 10 Frequency 10 grasshoppers). 0 0 0 5 10 15 0 5 10 15 In Fig. 1 we depict characteristic spectrograms of some of Time (secs) Time (secs) these sound production mechanisms.