NEWSFOCUS on March 15, 2011 www.sciencemag.org Counting the Dead in Afghanistan A military data set of civilian casualties, provided exclusively to promenade, licking ice cream cones and Science watching a volleyball game. Downloaded from , indicates that the war has become more lethal to the Afghan The spell is broken as sirens blare and population, largely because of indiscriminate insurgent attacks everyone dives for the ground. Once per day on average, insurgents manage to send small KANDAHAR, AFGHANISTAN—By day, you Inside its outer maze of concrete walls rockets fl ying over the walls. The Canadians don’t see the war. The wind throws up a and machine gun nests is a world of surreal in the hockey rink famously play through red haze that obscures everything. Bearded contrasts. Thousands of ISAF soldiers clog these attacks, each team refusing to fl inch. Afghan men in colorful vests whip their the streets, some hurrying to dinner at the But newcomers on the base can’t help but donkeys, lugging vegetables and fi rewood. end of a grueling workday, others just start- fl inch constantly—every jet overhead sounds Unless you spot an armored convoy picking ing their night shifts. The conversations are like an incoming rocket. It turns out to be a its way across the rugged landscape or hear in English, French, German, Dutch, and other drill this time. But an hour later, a dozen mis- a fi ghter jet scream overhead, you couldn’t languages from the 48 nations that have con- siles erupt from the base’s launchers, break- even guess the century. But once night falls, tributed troops to the ISAF coalition. There is ing the speed of sound in seconds and fading a brilliant cluster of lights appears, as if a no safety beyond the walls, so all recreation over the horizon like cigarette embers. Some- new city has sprouted in the valley several must happen here. The Americans brought in where to the west people are dying. kilometers south of the provincial capital. a string of restaurants, including T.G.I. Fri- What began as an invasion to capture This is Kandahar Airfi eld. What used to be day’s. The Canadians built a full-sized, open- Osama bin Laden in 2001 has become a full- a derelict airport littered with debris from air hockey rink—the fl oor is concrete rather blown occupation and counterinsurgency. the Soviet era has grown into a massive base than ice—and they play in full gear under the As the war in Afghanistan grinds toward the for the International Security Assistance glare of stadium lighting. Tonight, men and decade mark—the longest in U.S. history—

Force (ISAF). women in uniform amble along a wooden public support is waning. When he testifi es AFP/GETTY IMAGES CREDIT:

1256 11 MARCH 2011 VOL 331 SCIENCE www.sciencemag.org Published by AAAS THE WAR IN AFGHANISTAN | NEWSFOCUS

Dangerous protectors. As the confl ict in Afghani- up to 20% more civilians were killed in 2010 South Vietnamese allies. “We were told that stan drags on, civilian casualties are increasing. compared with the year before—ISAF has we were over there only as advisers,” he says. become a safer fi ghting force. The majority of Even the offi cial number of troops deployed before the U.S. Congress next week, ISAF deaths, and nearly all of the recent increase, to Vietnam looked fishy. “We were being Commander Gen. David Petraeus will face are attributed to indiscriminate attacks by manipulated by the government through the some diffi cult questions. Most contentious is insurgents rather than ISAF soldiers. In spite media,” says Sutherland, now a statistician at the issue of civilian casualties. Media reports of a troop surge and the launch of new oper- the University of Massachusetts, Amherst. of surging violence give the impression of a ations against the Taliban last year, the data Forty years later, with wars raging in Iraq country slipping out of control, but ISAF has provided by the UN show a 26% drop in civil- and Afghanistan, the military has become far reported steady progress. “There is so much ian deaths caused by military forces. And more cautious with casualty statistics. The rhetoric fl ying around and none of it can be both the UN and ISAF data sets show a drop new approach was described by U.S. Army tested,” says Neil Johnson, a physicist at the in deaths due to air strikes last year, by 50% Gen. Tommy Franks. “We don’t do body University of Miami in Florida who studies and 10%, respectively. counts,” he told reporters in Afghanistan in the dynamics of warfare. “What we need is All of these data, as well as other infor- 2002. Although the deaths of coalition sol- hard, reliable data.” mation never before released, are now online diers in Iraq and Afghanistan are available— A deluge of data arrived last year from at http://scim.ag/afghandata. Taken together, for example, at www.icasualties.org—an WikiLeaks, the organization that has made they provide the clearest picture yet of the “information vacuum” has surrounded civil- public thousands of classifi ed documents. It human cost of the war. ian casualties, says Michael Spagat, an econ- includes the raw observation of casualties by omist at Royal Holloway, University of Lon- soldiers on the ground in Afghanistan, but the Counting bodies don. So researchers trying to measure the many unknowns surrounding those reports Millions of people have died in modern wars, human cost of the wars have had to turn to have left researchers puzzled about how to but the exact number is anyone’s guess. His- other data sources. interpret them. A few independent organi- torians estimate that at least 10 million peo- One source is the media. The Iraq Body zations, including the United Nations, have ple were killed in World War I and at least Count (IBC) Web site has tallied Iraqi civil- published their own reports on civilian casu- 50 million in World War II. Although records ian casualties—over 100,000 and counting— on March 15, 2011 alties in Afghanistan, but only for illustrat- exist for the number of soldiers who went to from media reports since the U.S.-led inva- ing broad trends. The data underlying their war and never came home, most sion in March 2003. However, reports have never been released. civilian deaths went uncounted. this method can provide only a For the fi rst time, those data are now pub- The military did not system- Online lower limit to the true number. licly available. In January, ISAF provided atically track casualties beyond sciencemag.org Another source is the civil- Science with a database of civilian casualties their own troops. Offi cial data ian population itself. Several called CIVCAS. It is the military’s internal That changed in the late sets, resources research teams have used house- for exploring the infor- record of the death and injury of Afghan civil- 1960s, when Americans opened hold surveys to estimate casual- mation, and podcast www.sciencemag.org ians, broken down by month, region, weap- their newspapers to fi nd an offi - interview with John ties in Iraq. A 2006 survey pub- onry, and perpetrator. By its reckoning, 2537 cial count of the people dying in Bohannon at http://scim. lished in The Lancet claiming a civilians were killed and 5594 were wounded the Vietnam War. Each week, the ag/afghanspecial. civilian death toll of 600,000 has over the past 2 years, with 12% of those casu- U.S. military released those num- been widely criticized (Science, 6 alties attributed to ISAF forces and the rest bers to the media, dividing the weekly casu- March 2009, p. 1278). A larger survey in Iraq to insurgents. The death toll is 93% identical alties between U.S. soldiers, U.S. allies, and led by the World Health Organization came to to that in the WikiLeaks data, revealing those “Communists killed.” One of the readers was a fi gure close to 150,000 for the same period. raw fi eld observations to be far more reliable Michael Sutherland, then a statistics Ph.D. But in Afghanistan, these methods may Downloaded from than researchers had suspected. student at Harvard University. “The military be impossible. The country’s size and popu- In February, after learning that the mili- had a goal of achieving a 10-to-1 kill ratio,” lation are close to those of Iraq—both have tary was releasing these data, both the UN he says. “The idea was that if we were kill- 30 million people in an area comparable to the and an Afghan human rights organization ing 10 of their guys for every one of ours, we size of France—but that is where the similari- agreed to release versions of their own civil- were winning.” Sutherland, who was eligible ties end. Whereas Iraq is a fl at country with a ian casualty data to Science. They show twice for the draft and had friends already serving well-educated and mostly urban population, as many civilians killed over the same period, in Vietnam, started collecting and analyzing Afghanistan is a nightmare for fi eld research. including 393 deaths by air strikes that were those weekly numbers. Most Afghans live in small villages nestled not counted in the military database. ISAF The kill ratio was indeed approaching within rugged and poorly connected river offi cials acknowledge the gap. “The civilian 10-to-1, but he discovered other patterns. valleys. Their inaccessibility thwarts sur- casualties reported by the UN have always It was clear that at least some of the data veys, and a lack of journalists on the ground been higher than those reported by ISAF,” were fabricated. The frequency of the num- makes rigorous media-based casualty counts says U.S. Navy Rear Adm. Gregory Smith, bers’ last digits was skewed, with far too few “extremely diffi cult,” says John Sloboda, the the director of communications for NATO zeros and fi ves compared with chance. The director of IBC based in London. based in Kabul. “But the trends have been reason? “If you’re making up numbers, you These diffi culties have not stopped some very consistent.” never say that 150 Communists were killed,” organizations from publishing estimates of Science assembled a team of experts to Sutherland says. “Instead, you use 147 or the death toll in Afghanistan. The Red Cross analyze the released data sets. They con- 152.” By comparing the trends in deaths, he has monitored the flow of casualties into clude that while the war has grown deadlier also found that U.S. soldiers were clearly hospitals, for example, and the UN has col- for Afghan civilians over the past 2 years— the ones fi ghting the battles rather than the lected statistics through its regional offi ces.

www.sciencemag.org SCIENCE VOL 331 11 MARCH 2011 1257 Published by AAAS NEWSFOCUS

Although these organizations have pub- (see p. 1261). What I was not allowed to do a document describing the standard operating lished estimates periodically, none has given was take the data with me. ISAF offi cials were procedures of ISAF’s civilian casualty–track- researchers access to their data to make an concerned that sensitive information associ- ing system. “This is crucial,” says Johnson. independent assessment. ated with civilian casualty data—such as the “It’s not enough to see the numbers; you need Of course, the organization in the best tactics and movements of troops—could be to know how they are collected.” position to directly record civilian casual- revealed. But after 3 months of negotiation, ties is the military itself, with nearly 150,000 ISAF agreed to give the entire CIVCAS data- From deaths to data observers on the ground witnessing the vio- base to Science for public release. Nestled deep inside Kandahar Airfield is lence every day. But it seemed that the mili- “Our database is 100% transparent,” says an inner bastion, like the nucleus of a cell. tary kept no record of those observations— Smith. “Ultimately, this is a war being fought Stacked white trailers have been converted that is, until last year when WikiLeaks here in Afghanistan on behalf of the Afghan into a village of temporary offi ces. At the showed otherwise. people.” Along with the death rate of sol- center is the Combined Joint Operations diers, he says, this is “the most signifi cant Center. Dozens of offi cers sit in an audito- From WikiLeaks to CIVCAS release data set in identifying whether or not you’re rium facing a wall of giant screens. Some Starting in July 2010, the largest leak of secret making progress.” show maps of the combat space, and oth- military information in history went public. In a series of confi dential meetings over ers show live video feeds from unmanned WikiLeaks gave media outlets 92,000 inter- the past year, ISAF has provided the UN and drones. Since operations were launched last nal military documents related to the war in year to push the Taliban out of their original Afghanistan and 400,000 from Iraq, allegedly “Ultimately, this is a war homeland around Kandahar, this has become provided by a low-ranking U.S. soldier who the nerve center of the war. is now in custody. The documents include the being fought … on behalf “We have a casualty,” says an Australian raw operational reports from troops on the of the Afghan people.” offi cer sitting in the top row. (She requested ground between January 2004 and Decem- that her name not be used because she is ber 2009. In Iraq over that period, soldiers The information compiled not an authorized ISAF spokesperson.) On reported a total of 79,000 civilian deaths, by ISAF is “the most her computer is a window of spooling text on March 15, 2011 15,000 of which the media missed (Science, called JChat—it looks like an Internet chat 29 October 2010, p. 575). Afghanistan has signifi cant data set in session—providing a real-time view of the been spared Iraq’s sectarian violence. Over identifying whether or events of the war. One line of text is red, indi- the same 6-year period, the leaked documents not you’re making cating a request for medical evacuation. It is note 4024 Afghan civilian deaths. one of many in this region, with about a dozen The information vacuum was breached, progress.” evacuations requested per day. but researchers have been wary of using the —REAR ADM. GREGORY SMITH, “It’s an Afghan girl wounded by shrapnel,” data. The military’s raw operational reporting NATO she says, deciphering the string of acronyms www.sciencemag.org was not intended for research, so any errors, in JChat. The offi cer is not concerned about biases, and inconsistencies they might con- some human rights organizations full access who caused the injury. Her job is to save the tain are unknown. Did investigations confi rm to CIVCAS. “The other organizations will girl’s life by identifying the most efficient those casualties? How many bodies identifi ed share their compiled data but not their raw path to a doctor. She plots the grid references as “combatants” later turned out to be civil- data,” says Smith. “That’s really to protect on a map, and a series of overlapping circles ians, and how many of the wounded subse- their own access, freedom, and indepen- appear around the nearest helicopters. The quently died from their injuries? Without dence, which is certainly something we rec- path is computed and the orders are given. those answers, the data do little to dispel the ognize and respect.” Help is on the way. Downloaded from fog of uncertainty. After learning that ISAF was releasing Meanwhile, a parallel stream of data has Late last summer, a confidential source CIVCAS, some of those organizations pro- already started to fl ow. “This is the start of a within ISAF informed me that the military vided Science with data as well. The United process that we call Consequence Manage- was curating a database of civilian casualties. Nations Assistance Mission in Afghanistan ment,” says Col. Martin Bricknell, a senior He described a dedicated military team that (UNAMA) provided 3 years of their monthly U.K. military doctor and the medical direc- investigates civilian casualties and analyzes casualty statistics, including a detailed tor for southern Afghanistan. The soldiers on trends in the fi nal tally to help ISAF reduce accounting of deaths by air strikes since Jan- the ground with the injured girl have already the number. In a series of e-mail exchanges uary 2009. The Kabul-based organization radioed in a First Impression Report, describ- with Science, ISAF offi cials confi rmed that Afghanistan Rights Monitor (ARM) provided ing the deaths and injuries that occurred. such a tracking system does exist and that its Science with the highest-resolution data of Once they return to base, they will give their output is an internal database of civilian casu- all, describing individual incidents during the commander a more detailed account. Within alties called CIVCAS. fi rst half of 2010. “Providing these data is a 9 days, ISAF Headquarters is expecting a In October 2010, ISAF hosted me in Kabul great public service,” says Sutherland, who is CIVCAS assessment report. Besides review- and Kandahar as an embedded reporter. I was undertaking the fi rst statistical analysis. ing the known facts of the case, the purpose is given access to military personnel at every Science is making all these available at to improve future operations, Bricknell says. level of the civilian casualty–tracking system, http://scim.ag/afghandata. ISAF has released “Is there anything that we can learn from that from the collection and quality-checking of additional sets of information to help research- so we can reduce casualties next time?” CIVCAS data to the analysis that leads to new ers analyze the casualty data, including pre- At that point, the ISAF leadership will combat directives. I was also able to tour med- cise monthly troop deployment numbers over decide whether to investigate further. “We ical facilities and interview medical personnel the past 3 years. Perhaps the most important is are absolutely determined to [track] the con-

1258 11 MARCH 2011 VOL 331 SCIENCE www.sciencemag.org Published by AAAS THE WAR IN AFGHANISTAN | NEWSFOCUS Civilian Casualties in Afghanistan 2009–10

Regional Command Regional Regional Regional (North) Command Command Regional Command (West) (East) Regional Command (Capital) Command (South) DEC (Southwest)

NOV Caused by Military

OCT Jets Helicopters SEP Escalation of Force Direct Fire AUG Indirect Fire Road Traffic Accidents

JUL Unknown

JUN Caused by Insurgents

MAY Direct Fire Indirect Fire APR Improvised Explosive Device

Complex on March 15, 2011 MAR Unknown

FEB Killed 2010 JAN Wounded

DEC

Casualties in time and space. www.sciencemag.org NOV The seasonal rhythms and shifting battlefi elds of the war OCT emerge in this view of the 8131 Afghan civilians killed or SEP injured over the past 2 years, recorded in a military data- base called CIVCAS. (No data AUG Downloaded from were available for the fi rst 5 months of 2010 in the South- JUL west region.)

JUN

MAY

APR

MAR

FEB

2009 JAN

Kabul

CREDIT: GEORGE MICHAEL MICHAEL BROWER GEORGE CREDIT: Kandahar

www.sciencemag.org SCIENCE VOL 331 11 MARCH 2011 1259 Published by AAAS NEWSFOCUS

sequences of the confl ict to the civilian pop- do overflights of cemeteries to determine many as in 2009. CIVCAS, which separately ulation,” Bricknell says. “Therefore we take any fresh grave sites and confi rm or deny the defi nes casualties caused by jets and helicop- every allegation of a civilian casualty very numbers involved.” He adds that other orga- ters, shows 11% fewer deaths. There are also seriously.” If local government offi cials give a nizations trying to track civilian casualties subtler signs of progress in the data. different account of the event, for example, an face the same challenges. In April last year, ISAF units received incident assessment team will be assembled Smith does not question the accuracy of a new directive concerning “escalation of and sent into the fi eld. For the most serious UNAMA’s body count. “The UN has a much force”: the shooting of civilians due to com- incidents, offi cials from the UN or other orga- broader mandate” to track civilian casual- munication breakdowns. Most of these inci- nizations join the team. ties, he says, “and the resources to do that.” dents occur at the hundreds of checkpoints The CIVCAS database tracks all of these Rather than creating a defi nitive record for that ISAF operates across the country. In the deaths and injuries. Between January 2009 history, the purpose of CIVCAS, he says, is course of analyzing their casualty data, ISAF and December 2010, it logged a total of 2537 “marking progress.” commanders noticed a trend. “What became civilians killed and 5594 wounded. About very clear to me is that all the [civilian] fatali- 80% of the deaths and injuries are attributed A safer force ties occurred between the 100-meter point and to insurgents. (The CIVCAS data go back to Over the past month, Sutherland, Spagat, the 0-meter point” approaching a checkpoint, January 2008, but insurgent-caused casual- Johnson, and other experts have analyzed Wilson says. Beyond that range, a fl are usu- ties were not tracked until 2009.) CIVCAS and the other civilian casualty ally suffi ces to warn drivers to slow down. But Throughout the war, critics have accused data sets at Science’s request. We have also if a vehicle has not slowed down yet, Wilson ISAF of undercounting civilian casualties, built a timeline of the past 2 years of the war says, “this is where as a soldier you suddenly particularly those caused by their own soldiers. that compares those data with the casual- think, ‘I’m about to die because this vehicle Just last month, a battle in Kunar province on ties reported in the media, available at http:// is going to drive in here and detonate.’ ” The Afghanistan’s eastern border with Pakistan scim.ag/afghandata. problem, he says, is that soldiers in that situa- generated confl icting accounts. According to tion had no nonlethal options. villagers, ISAF killed 65 civilians, including “Counting seems like such a The new directive gave soldiers more 50 women and children. According to ISAF, options for warning drivers at a distance, on March 15, 2011 only insurgents were killed. simple thing, but it is the Wilson says. These include laser dazzlers, The data provided by UNAMA do show only way to see the actual paint ball guns, and even chalk bullets. “If you far more casualties than those from ISAF. For fi re them at a vehicle,” he says, “they will ping 2009 and 2010, its data include 5191 civilian effect of the war.” off and make such a loud noise that, if they’re deaths, over 70% of them caused by “antigov- —MICHAEL SUTHERLAND, a genuinely innocent person, they’ll get the ernment elements,” 20% by “pro-government UNIVERSITY OF MASSACHUSETTS, AMHERST message.” The CIVCAS data put numbers forces,” and the rest undetermined. Compared to that narrative. Deaths due to escalation of with CIVCAS, they attribute nearly three By all accounts, the war has grown dead- force dropped by 50% in the 8 months after www.sciencemag.org times the number of civilian deaths caused by lier for Afghan civilians. The CIVCAS data the April 2010 directive went out, compared military forces, only a small portion of which show a 19% increase in the total number of with the same 8 months in 2009. are Afghan national rather than ISAF forces. civilians killed in 2010 compared with the While they applaud the release of these One of the most significant discrepancies year before, and the UN data show a 15% data sets, researchers are grappling with their comes from the 529 civilians that UNAMA jump. But at the same time, there are signs that limitations. “One problem is that organiza- claims were killed by “air attacks” in 2009 ISAF has become a safer fi ghting force, tread- tions are all using different defi nitions in their and 2010. CIVCAS shows only 136 civilians ing more lightly on local populations. data,” Johnson says. For example, whereas the killed by jets and helicopters over that period. Although the overall death toll in Afghani- UN data have separate categories for casual- Downloaded from Asked whether casualty reporting by sol- stan has risen, the increase was not wrought ties caused by “mortar and rocket fi re” and diers might be biased, British Army Lt. Col. by soldiers. Over 90% of last year’s spike in “shooting,” ISAF pools all ground-based bat- George Wilson, who oversees ISAF Conse- CIVCAS, and the entirety of that in the UN tle casualties into “direct fi re” and “indirect quence Management, demurs. “No, I think data, is attributed to insurgents. IED explo- fi re.” The different organizations also divide the converse,” he says. “I genuinely think sions have continued to cause the majority of the country along slightly different lines we get honest reporting from the ground.” civilian deaths. By contrast, 2010 saw a 26% for coding the location of casualties. “They According to Wilson and other ISAF drop in the number of civilians killed by sol- should be trading notes,” Johnson says, “at sources, the cause of the disparity in body diers in the UN data. In their own data, ISAF least so their data can be easily compared.” counts is methodology. admitted to killing 12% more civilians com- Such limitations would be solved if all “Raw numbers will never be the same, pared with the year before, while wounding data for civilian casualties were released at the and there’s a good reason for that,” Smith 20% fewer. This happened in the context of level of individual events rather than aggre- says. “We do not have a presence in all 34 the largest military offensive in years and a gated monthly. “Ultimately, that is the only provinces,” he says, and therefore CIVCAS surge that doubled the number of troops in way you can verify them,” says IBC’s Slo- does not track all casualties. “We only count Afghanistan to 140,000 last year. The lack of a boda. The data provided to Science by ARM that which we see.” In some cases, such as corresponding spike in military-caused casu- come closest to this level of resolution, but so alleged casualties from air strikes, “we can do alties is surprising. far, they only cover the fi rst half of last year. a tremendous amount of forensics, … [but] This is especially true of air strikes, by “Counting seems like such a simple seldom do we see the actual bodies. Some- far the most dangerous military activity for thing,” Sutherland says. “But it is the only times we have access to someone who was civilians. In the data provided by the UN, air way to see the actual effect of the war.” wounded, but not always,” he says. “You can strikes killed 171 civilians in 2010, half as –JOHN BOHANNON

1260 11 MARCH 2011 VOL 331 SCIENCE www.sciencemag.org Published by AAAS