Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

Journal of Conventional Weapons Destruction

Volume 16 Issue 3 The Journal of ERW and Mine Action Article 22

October 2012

Amanda Mahoney APOPO

Amy Durgin APOPO

Alan Poling APOPO

Bart Weetjens APOPO

Christophe Cox APOPO

See next page for additional authors

Follow this and additional works at: https://commons.lib.jmu.edu/cisr-journal

Part of the Other Public Affairs, Public Policy and Public Administration Commons, and the Peace and Conflict Studies Commons

Recommended Citation Mahoney, Amanda; Durgin, Amy; Poling, Alan; Weetjens, Bart; Cox, Christophe; Tewelde, Tess; and Gilbert, Tekimiti (2012) "Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy," The Journal of ERW and Mine Action : Vol. 16 : Iss. 3 , Article 22. Available at: https://commons.lib.jmu.edu/cisr-journal/vol16/iss3/22

This Article is brought to you for free and open access by the Center for International Stabilization and Recovery at JMU Scholarly Commons. It has been accepted for inclusion in Journal of Conventional Weapons Destruction by an authorized editor of JMU Scholarly Commons. For more information, please contact [email protected]. Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

Authors Amanda Mahoney, Amy Durgin, Alan Poling, Bart Weetjens, Christophe Cox, Tess Tewelde, and Tekimiti Gilbert

This article is available in Journal of Conventional Weapons Destruction: https://commons.lib.jmu.edu/cisr-journal/ vol16/iss3/22 Mahoney et al.: Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

Pierre Lacroix works as a scientific collaborator at the University of This article describes the performance of Giant African Pouched Rats where reinforcement (reward) or Geneva. He is also pursuing doctorate research on the contribution of extinction (no reward) conditions affected landmine identification. Accuracy deteriorated quickly in the ab- geographical information systems to mine action. Since 2010, he has sence of reinforcement, suggesting that reinforcement is essential. worked with the Geneva International Centre for Humanitarian Demining as a GIS project consultant. by Amanda Mahoney, Amy Durgin, Alan Poling [ Western Michigan University, APOPO ], Bart Weetjens, Christophe Cox, Tess Pierre Lacroix Tewelde, TeKimiti Gilbert [ APOPO ] University of Geneva Route de Drize, 7 CH 1227 Carouge / Switzerland Email: [email protected] The mine detection rats in Mozambique work on training fields and actual mine- fields (operational sites). The training field comprises several 100 sq m, 200 sq m and 400 sq m boxes indicated by ropes along each side. Between zero and four deactivat- ed landmines are buried within each box. The rats are attached to a rope (via a harness) held by two handlers on either side of the box. The rats walk across the box they are searching. When an indication response (pausing and digging) occurs within 1 m of a landmine, the trainer clicks to signal reinforcement and food is delivered. Rocío Escobar works as a When the rats are used operationally, the research fellow at the University location of mines (and other explosive rem- Figure 6. Parameters provided to users at the execution of the model. of Geneva. She holds a master’s Figure courtesy of the authors. nants of war) is unknown prior to clearance degree in environmental science operations. Therefore, knowing whether an and has completed a four-month internship with the Geneva indication response is correct (i.e., within 1 output raster map for each degree of opera- for decision-makers and operators to refine International Centre for Humanitarian m of a mine) or incorrect is impossible. To tional difficulty. For example, the overall sur- their evaluation of the degree of operational Demining on the Operational avoid the possibility of reinforcing incorrect face with a low degree of difficulty is directly difficulty and improve efficiency in their work. Difficulty of Demining project. responses and thereby potentially reducing Illustration of the experimental setup with one trainer and one notetaker. read into the output raster. This kind of in- However, this tool is intended as a guide, and the rat’s subsequent detection accuracy, no re- Rocío Escobar All photos and figures courtesy of the authors. formation may be significant for decision- real world political or economic factors may University of Geneva inforcers are delivered when the rats are used makers and operators, especially in financial lead to or prevent demining activities in a way Route de Drize, 7 s a result of almost 30 years of conditioning in which food reinforces (re- operationally. terms. With further work, in fact, this model that may disagree with the tool. In addition, CH 1227 war, landmines are a devastat- wards) appropriate indication responses (i.e., In technical terms, the rats work under ex- opens the possibility to estimate the financial deminers should be aware that modification of Carouge / Switzerland ing problem in Mozambique. those that occur within 1 m of a mine).5 Incor- tinction (no reinforcement) conditions when Email: [email protected] A implications of their operational choices. one parameter could affect the outputs of the According to a United Nations’ report, an rect indication responses are not reinforced. used operationally and under differential re- model significantly. estimated 20 people step on landmines ev- Training begins in a controlled laboratory inforcement (food reinforcement for correct Conclusion See endnotes page 67 ery month in Mozambique and, due in part setting and proceeds through a series of steps responses, no reinforcement for incorrect re- The 5D model is a first approach for mod- to lack of adequate health care, 60% of those to a large training field. sponses) conditions during training. Extinc- eling an operational difficulty of demining people die.1 Since the mid-1990s, efforts have An early evaluation conducted in 2005 in tion inevitably weakens previously reinforced at a macro level. The model was developed been made to clear Mozambique of land- which seven rats searched 20,234.28 sq m of responses.8,9 For this reason, the rats rotate in ArcGIS® Desktop, which is readily avail- mines, but millions are believed to still con- land in Mozambique indicated that their de- between the training field and the operation- able in most mine-affected countries. Users taminate the country. Anti-Persoonsmijnen tection accuracy exceeded 95%.6 In a more real site. The rationale for this arrangement is interact with the model via an intuitive and Ontmijnende Product Ontwikkeling (Anti- cent evaluation, teams of two rats searched that reinforcement of correct responses on graphical interface by using a set of param- Personnel Landmine Detection Product 93,400 sq m of land in Mozambique, reveal- the training field will sufficiently strength- eters that can be modified each time the pro- Development) started using Giant African ing 41 mines.7 This area was then searched en such behavior to compensate for the re- gram runs, especially the area of study and Pouched Rats (Cricetomys gambianus) for with metal detectors, revealing a 100% de- sponse-weakening effects of extinction at the input factors. Even if the workflow may seem landmine detection in Mozambique in 2007. tection rate by the rats. Such findings suggest operational site. The rats’ performance at the complex, using the model does not require Details on how the rats are trained and used that pouched rats are acceptably accurate in operational site strongly suggests that this is intensive GIS skills. operationally are provided elsewhere.2,3,4 In detecting landmines and, as a result, they are the case, but we have not systematically eval- The resulting map is a good starting point brief, the rats are trained through operant used operationally in Mozambique. uated the extinction effects, though studies

Figure 2. Average percentage of hits per day under reinforcement and extinction conditions.

tion until detection accuracy fell to 0% for at least two consecutive days. may have learned the location of the mines, and at some point they may This sequence was then repeated. not have been operating under blind conditions. However, a second ob- Figure 1. Percentage of hits per day by individual rates in reinforcement and extinction. All rats worked in one box per day, and sessions were conducted up server was present during approximately 20% of the sessions to ensure to five days per week. Sessions were not conducted on weekends, holi- that procedures were followed as written and that there was agreement days or days with heavy rain. Data recorded each day for each rat were in recording. are under way. In an effort to gain information of value to maximize er’s hand and the measuring tape value at the trainer’s feet. After the rat the location of indications, the number of hits (indication responses the effectiveness of APOPO’s MDR team, the present study evaluated walked down the rope in one direction, the trainers took a 0.5 m step When extinction was introduced, accuracy declined for four of the the effects of extinction on the detection accuracy of five rats perform- forward and the rat walked in the opposite direction across the box. In within 1 m of a mine), the number of false alarms (indication responses five rats within three sessions. Enda’s performance did not fall until the ing under controlled conditions that allowed for accurate assessment of all tests, the rats were allowed to traverse the rope only once before they further than 1 m from a mine) and the number of misses (mines with no seventh session but remained at 0% for six of the next seven sessions. their performance. were moved forward. indication response within 1 m). Typically, the rats continued emitting an indication response over the Data were recorded on graph paper that depicted the box measure- Reinforcement Condition. In this condition, when an indication mine on some days during extinction, but failed to indicate on about as Setting, Subjects and Materials ments. Each test box was displayed as a grid comprised of 0.5 m x 0.5 m response occurred within 1 m of a mine, the trainer produced a click many days as they indicated. Upon return to the reinforcement condi- Trials took place in Morogoro, Tanzania on the APOPO training squares. Shaded gray squares corresponded to the mine locations. The sound using a handheld clicker. If the rat began to approach the trainer tion, detection accuracy for Toyota remained variable for six days while field, which contains approximately 1,200 landmines buried in a fenced indication response was scratching the ground for any length of time within 3 seconds of the click, which usually occurred, the trainer deliv- performance for Mar remained at 0% for eight out of nine days before 283,279.95 sq m site. In the portion of the training field used, one mine within 1 m of the landmine. Upon a rat indication, the trainer informed ered food. If the rat did not approach the trainer within 3 seconds of the improving to the initial reinforcement-condition level. Performance for was buried in a marked 100 sq m box. Some of the boxes in APOPO's the notetaker, who recorded the location of the response and whether or click, the trainer did not present food. If a rat walked over a mine with- Nijad and Bila recovered to 100% accuracy in two days, and Enda’s per- training field have markings to indicate landmine locations and some not the trainer should sound a click and deliver food to the rat. In the out indicating, the rat continued clearing the rest of the box. Each rat formance improved to this level after three days. Upon return to extinc- do not. The boxes without markings were used in the present study to reinforcement condition, the trainer was instructed to sound a click and searched each area of the box only once. tion, responding fell within two to four days for all rats. Performance provide blind testing conditions, under which the trainers were unaware deliver food (i.e., provide a reinforcer or reward) each time an indication Extinction Condition. Extinction sessions were the same as rein- again took several days to recover to prior reinforcement levels for Enda of mine locations. The tests used six boxes, each containing just the one response within 1 m of a mine was emitted. Reinforcers were never pro- forcement sessions, with the exception that neither a click nor food was and Mar, although the performance of Bila, Toyota and Nijad recovered mine. Each test took an average of 17.8 minutes with a range in time of vided in the extinction condition. presented following either correct or incorrect identification responses. in zero to two days. 8 to 25 minutes. Second Reinforcement and Second Extinction Conditions. The sec- Figure 2 summarizes findings across the five rats. This figure clearly Five rats participated in this test. Each rat had recently passed a blind Experimental Design ond reinforcement condition, which was identical to the first reinforce- shows that overall the rats’ accuracy in detecting landmines was high test in which it located each of eight unmarked mines in a 400 sq m area A multiple baseline with reversal design evaluated detection ac- ment condition, occurred after the first extinction condition. The second during the first reinforcement condition and quickly declined when ex- with no more than one false alarm. The rats were distributed between curacy under reinforcement and extinction conditions.10 In a multiple extinction condition was the same as the first one and was the last condi- tinction was arranged. Accuracy remained inconsistent and relative- two trainer teams; each team comprised two trainers and one notetaker. baseline design, different subjects are initially exposed to the condition arranged for each rat. Figure 1 shows the number of days that each ly low after reinforcement was again arranged but eventually reached The notetakers were APOPO minefield supervisors. APOPO certified all tions of interest on different days. This design demonstrates that the rat was exposed to each experimental condition. a high level. The rats’ accuracy again declined even more rapidly when trainers and selected them because they demonstrated good adherence changes observed when conditions change are the result of the change Independent-observer Agreement. A second observer independent- extinction was introduced a second time. For this reason, these rats will to standard operating procedures. Materials included clickers to signal in conditions and not the result of some other factor (e.g., weather ly collected data during 21.3% of sessions. The second observer agreed not be used in actual future detection operations. availability of the food rewards, data sheets, a banana (the food reinforc- conditions, day of the week, time of exposure to a condition). A rever- with the primary data collector on 98.1% of rat indications. Few false alarms (incorrect identification responses) occurred under er) and mine detection training box materials. sal design calls for returning to a prior condition, which in this case any condition, and the number of false alarms per session did not con- Training box materials consisted of measuring tape stretched along was the reinforcement condition. Thus, all of the rats were exposed Results sistently differ under reinforcement and extinction conditions. None of one side of the box and a rope that stretched across the box between to a reinforcement condition, then extinction, reinforcement and fi- Figure 1 (page 62) shows the percentage of hits (correct identification the rats emitted more than three false alarms on any given day, and an the two trainers and guided the rat as it walked in the box. The rats nally extinction. responses) per day by individual rats during reinforcement and extinc- individual rat typically emitted zero or one false alarm each day. were attached to the rope via a harness and lead cord and could walk When performance remained at 100% accuracy under the rein- tion conditions. Because each box had one mine, accuracy was either back and forth along the rope. The trainers held two measuring tapes forcement condition over at least four consecutive days, the extinction 0% or 100%. During the initial reinforcement condition, the rats identi- Discussion between them. One end of each tape was attached to the rat’s harness condition began. Since there was only one mine per box, if the rat found fied all mines except for a single mine missed by Nijad in the third ses- This study evaluated the performance of APOPO’s MDRs under rein- at zero. Thus, the exact location of the rat’s indications could be deter- it, the detection accuracy was 100%; if it did not indicate a mine, the sion. In general, because accuracy was 100% on the first day, the rats forcement and extinction conditions and found that, in general, the rats mined through the coordinates of the measuring tape value in the train- detection accuracy was 0%. The rat worked under the extinction condi- did not appear to learn from the use of the same six boxes. The trainers demonstrated high accuracy and stable performance after sufficient expo- https://commons.lib.jmu.edu/cisr-journal/vol16/iss3/22 2 62 research and development | the journal of ERW and mine action | fall 2012 | 16.3 16.3 | fall 2012 | the journal of ERW and mine action | research and development 63 Mahoney et al.: Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

sure to the reinforcement condition and vari- tion to APOPO management. Prior research Endnotes able but substantially lower accuracy during conducted under operational conditions indi- the extinction condition. There was high carry- cates that APOPO’s rats are accurate in detect- The Need for Collaboration Between Ordnance Manufacturers and UXO over from the reinforcement condition in that ing landmines under the conditions arranged Clean-up Personnel, Imber [ from page 8 ] performance remained variable after the rein- in Mozambique.6,7 APOPO draws upon sever- 1. The Vietnamese refer to the Vietnam War as the American War. “Viet- nam War: History.” BBC News. http://tinyurl.com/bsbswsw. Accessed 30 forcement condition was reinstated. Sometimes al means of reinforcement delivery in opera- August 2012. several reinforcement sessions were necessary tional conditions: frequent quality control 2. “Vietnam.” Landmine and Cluster Munition Monitor. http://bit.ly/Qgqa- for performance to recover to 100% accuracy. checks, data collected regularly on individ- Amy Durgin is a behavioral researcher Christophe Cox is APOPO’s ro. Accessed 17 August 2012. In APOPO’s operational setting, the rat ual rat performance and ample opportuni- at APOPO and a doctoral candidate at CEO and developed much of the 3. “MAG’s history.” MAG. http://bit.ly/S0aKqm. Accessed 17 September does not receive reinforcers, because it is un- ty for reinforcement on the nearby training Western Michigan University (U.S.). She tuberculosis detection application. 2012. known where mines lay and, consequently, field. How the rats would perform under other assists APOPO with conducting research Cox is a product engineer and has 4. One example of this categorization in the United States can be seen in whether the rat’s indications are correct (i.e., conditions, for example, if they worked for designed to improve and maintain an worked frequently in Africa. the Occupational Safety and Health Association’s regulations for UXO efficient and effective rat training program. technicians: OSHA requires American-employed UXO technicians to within 1 m of a mine) while searching. The longer periods each day or in areas with differ- Christophe Cox complete a 40-hour Hazardous Waste course and subsequent eight-hour study’s aim was to determine how many days ent landmine concentrations, is speculative. Amy Durgin SUA-APOPO an MDR can work on a minefield, without re- The present data strongly suggest, however, SUA-APOPO Sokoine University of Agriculture refresher course to work at UXO clearance sites. 5. HERO standards are designed to limit the potential for electromag- ward, before performance degrades. Under that their accuracy would decline significantly Sokoine University of Agriculture PO Box 3078 PO Box 3078 netic radiation to adversely affect munitions and electro-explosive the conditions of the present study, this period if they worked for periods during which sever- Morogoro / Tanzania Morogoro / Tanzania Email: [email protected] devices. High electromagnetic environments can be produced by mod- was conclusively determined to be quite short. al indication responses occurred and were Email: [email protected] ern communication transmitters and can cause premature actuation The rats’ accuracy in detecting mines fell, on not reinforced. This study and previous ones of the ordnance into detonation or dudding. “Munitions Glossary.” average, after 3.1 days of exposure to extinc- provide a research base that informs APOPO’s GlobalSecurity.org. http://bit.ly/Oq3u6k. Accessed 8 August 2012. tion, although their false alarm rates did not operating procedures in a way that contin- 6. “Parari 2011: 10th Australian Explosive Ordnance Symposium.” http:// change systematically. Furthermore, recovery ually optimizes operating procedures and bit.ly/PLuXiP. Accessed 8 August 2012. of the asymptotic accuracy level following ex- ensures the rats’ performance is maintained at tinction took up to nine days. high levels under operational settings. Darfur: Baseline KAPB Survey, Winkler and Ragab [ from page 11 ] To maximize experimental control, the APOPO’s primary goal is using pouched 1. “Estimates for the Total Number of IDPs for all of Sudan.” International Displacement Monitoring Centre. http://bit.ly/9RNVzR. Accessed 10 Oc- present study only used 100 sq m boxes con- rats effectively and efficiently for humanitarian tober 2012. taining a single mine. In operational demining purposes, not conducting scent-detection re- 2. “Sudan.” Landmine and Cluster Munition Monitor. in Mozambique, the overall density of land- search. Such research is, however, the best Alan Poling is a psychology professor at Tess Tewelde is the Program Manager http://bit.ly/TZwR5N. Accessed 10 October 2012. mines is substantially lower. For example, in means to that end and for that reason is given Western Michigan University (U.S.). He for APOPO’s Mozambique Mine Action 3. The Ordnance Disposal Office began its operations in El Fasher in 2005 one study the rats located 41 landmines in a high priority by the organization. Conducting has played an integral role in research and Program. Tewelde has 11 years of and became an integral component of UNAMID in 2008. It works in di- 93,400 sq m area, which yields an average of research uses personnel, time and financial re- development at APOPO since 2009. experience in mine action in Africa in rect support of UNAMID priorities to create a safe environment for the humanitarian and commercial sectors. 0.04 mines per 100 sq m box, although in some sources that could go directly toward mine civilian population in Darfur. More information can be found at http:// cases a rat may pass over two or more mines in clearance or land release. Therefore, we attempt Alan Poling darfurodo.org. Western Michigan University Tess Tewelde a small area. The effects of extinction on the to choose research topics carefully and to design Mozambique Program Manager 4. Sudan’s National Mine Action Authority (NMAA) was established Department of Psychology through Presidential Decree No. 299 in December 2005 and adopted the performance of MDRs under such conditions, studies in a way that minimizes cost. Small-N 3700 Wood Hall SUA-APOPO Sudan Mine Action Bill in 2010. In 2012, NMAC and UNAMID ODO where target density is highly variable but low research strategies characteristic of behavior Kalamazoo, MI 49008-5439 / USA PO Box 649 Maputo / Mozambique signed a letter of cooperation, and NMAC started establishing offices in overall, remain to be determined. Of course, analysis have proven especially valuable in this Tel: +001 269 387 4500 Fax: +001 269 387 4550 Tel: +258 827 273 378 Darfur. More information can be found at http://su-mac.org. performance in extinction depends on a num- regard, and we recommend them to the human- Email: [email protected] Email: [email protected] 5. “International Mine Action Standards Mine Risk Education Best Prac- 10,13 ber of environmental variables. These vari- itarian demining community. tice Guidebooks 1–12.” Electronic Mine Information Network. http://bit. ables seemingly would include the number of See endnotes page 67 ly/OPYlXv. Accessed 10 October 2011. responses emitted without reinforcement and 6. National Mine Action Centre. http://bit.ly/TjkmCw. Accessed 10 Octo- the manner in which reinforcement was ar- ber 2011. ranged prior to extinction. 7. “Key Facts and Figures for Sudan with a Focus on Darfur.” Relief Web. Future research in this area might inves- http://bit.ly/QOizjy. Accessed 15 January 2012. tigate the effects of training with intermit- 8. “Information Management Overview.” Geneva International Centre for Humanitarian Demining. http://bit.ly/OdhhTy. Accessed 10 October tent reinforcement, which is well-known to 2012. prolong accurate performance under extinc- 9. Traditional community leaders 11 tion. Though APOPO has not yet evaluated 10. When discovering the use of UXO as school bells out of economic neces- this methodology, it has used intermittent re- sity, UNAMID ODO responded by including school bells as part of the inforcement, with trainers rewarding 85% of risk education materials. indications. APOPO plans to study intermit- Amanda Mahoney is a behavioral Bart Weetjens is a product TeKimiti Gilbert joined APOPO as the 11. For the full final report, read “Knowledge, Attitudes, Practices, Beliefs researcher at APOPO and a doctoral engineer and APOPO’s founder. He Head of Mine Action in March 2012. tent reinforcement and evaluate optimal pa- (KAPB) Survey: Baseline Study 2012 Final Report.” http://bit.ly/OSLdkz. candidate at Western Michigan University conceived of using scent-detecting Prior to that time, he worked extensively Accessed 10 October 2012. rameters and effectiveness. (U.S.). She completed this work in partial rats for humanitarian purposes. with the United Nations and has worked APOPO is currently investigating the util- fulfillment of her doctoral dissertation. in land clearance around the world. ity of exposure to reinforcement conditions, Bart Weetjens Grenade Blast Kills 25 in Turkey, Reitman [ from page 15 ] SUA-APOPO 1. Turkey: 25 Killed in Ammunition Depot Blast.” NPR, 6 September 2012. prior to or following the extinction condi- Amanda Mahoney TeKimiti Gilbert SUA-APOPO Sokoine University of Agriculture Head of Mine Action http://tinyurl.com/8hlx4of. Accessed 13 September 2012. tion.12 The success of this procedure depends Sokoine University of Agriculture PO Box 3078 SUA-APOPO 2. “Turkey grenade depot blast kills 25.” BBC News. http://tinyurl.com/ largely upon how well the rats discriminate PO Box 3078 Morogoro / Tanzania Tel: +33 4504 10889 cxzffk9. Accessed 13 September 2012. between training (reinforcement) and opera- Morogoro / Tanzania Email: [email protected] Email: [email protected] 3. “Blast in Military Ammunition Depot Kills 25 Soldiers.” Sabah. http:// tional (extinction) conditions. Tel: +001 586 292 0644 tinyurl.com/9okj9fp. Accessed 13 September 2012. Email: [email protected] These tests were conducted for experi-

mental purposes to provide relevant informa-