Slippery Polls Uses and Abuses of Opinion Surveys from Arab States
Total Page:16
File Type:pdf, Size:1020Kb
Slippery Polls Uses and Abuses of Opinion Surveys from Arab States David Pollock Policy Focus #82 | April 2008 All rights reserved. Printed in the United States of America. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any infor- mation storage and retrieval system, without permission in writing from the publisher. © 2008 by the Washington Institute for Near East Policy Published in 2008 in the United States of America by the Washington Institute for Near East Policy, 1828 L Street NW, Suite 1050, Washington, DC 20036. Design by Daniel Kohan, Sensical Design and Communication Front cover: Palestinians line up to vote in parliamentary elections at a polling station in the West Bank village of Dura, January 25, 2006. Copyright AP Wide World Photos/Kevin Frayer. About the Author David Pollock, a visiting fellow at The Washington Institute, was formerly a senior advisor for the Broader Middle East and a member of the State Department’s Policy Planning Staff. He has also served as chief of Near East/South Asia/Africa research at the U.S. Information Agency, where he was the top authority on public opinion in those regions. He testified to Congress on the subject in 2007 and has written widely on it as well, including the pioneer- ing 1993 Institute Policy Paper The “Arab Street”? Public Opinion in the Arab World. n n n The opinions expressed in this Policy Focus are those of the author and not necessarily those of the Washington Institute for Near East Policy, its Board of Trustees, or its Board of Advisors. Table of Contents Acknowledgments. v Executive Summary . vii Part I: The Limits of Opinion Surveys in Arab States Introduction: Arab Polls in Perspective . 3 Problems of Practical Fieldwork in Arab Societies. 7 Problems of Technical Fixes . 11 Problems with Poll Design. 17 Problems in Analyzing Polls. 23 Part II: What Polls of Arab Publics Really Show Introduction: Selecting the Best Surveys. 33 Two Remarkable Regional Trends . 34 Beyond Terror: Different Views or Different Values?. 40 Arab Polling: Examples of Best Practices . 45 Policy Implications and Recommendations. 55 Acknowledgments THE RESEARCH AND PUBLICATION of this study was funded through a special grant from Washington Institute trustees Jane and Barry Haimes, reflecting their commitment to the importance of sound scholarship in promoting tolerance, understanding, and peace throughout the Middle East. This Policy Focus is a publication of The Washington Institute’s Project Fikra, a multifaceted initiative generat- ing ideas to help Americans identify, nurture, and support mainstream Muslims in the contest against radicalism and extremism. Project Fikra was established through the generous support of the Keston Family Foundation. In addition, the author would like to thank Institute research assistants Albar Sheikh, Jasmine El-Gamal, and Rana Shabb for their superb and devoted help with various stages of this project. The Washington Institute for Near East Policy v Executive Summary A CLOSE EXAMINATION suggests major problems remain of possible bias in developing and presenting exist with the pervasive overreliance on Arab public such findings. To be sure, a few pollsters strive to avoid opinion polls. Although a few of these polls are fairly such bias; but given the highly politicized nature of credible, most are methodologically suspect, so their Middle East issues, they rarely succeed. More to the numbers are unreliable—even apart from the numer- point, consumers of their polling product rarely have ous instances of loaded questions, selective presenta- the time or the training to pick apart all the more sub- tions, and biased analyses. tle biases that regularly seep into the process of collect- Almost always, these poll numbers sound alarm- ing, analyzing, and presenting Arab survey data. ingly bad—even if their relation to reality is open to The first step in fixing these problems is to insist on serious question. Their primary focus is on how low the full disclosure of all methodological and other details; American image has sunk. Even to the extent that any without them, any poll is automatically suspect. If that of these polls are true, however, this focus is at most information is provided, the reader can make further only half the truth. The better and much-less-remarked judgments about the adequacy or inadequacy of the half is a dramatic rise in the level of popular opposi- product. Sampling methods, questionnaire design, tion to anti-American terrorism in every Arab society integrity of reporting, objectivity of interpretation, polled on this question over the past four years. Nev- and any other relevant factors—especially the sponsor- ertheless, almost no one points out what this clear new ship of every poll and the nature of Arab government trend must mean: attitudes toward the United States supervision or interference, real or perceived—must all and attitudes toward terrorism in the societies where it be subjected to serious scrutiny. supposedly incubates are hardly related to each other Second, even if the numbers do not lie, they tell at all. us almost nothing about actual popular behavior, let At the most down-to-earth methodological level, alone about the interactions (either at home or abroad) unusually severe demographic and, especially, social among key segments of Arab societies: the business or and political constraints restrict polling in Arab coun- educational community, the clergy, the military, the tries. In most cases, Arab governments must approve media, the village or neighborhood power brokers, or any polling, which raises serious questions about pos- the technical elite—or, for that matter, the violent ele- sible censorship (or self-censorship), surveillance, ments on the margins. intimidation, or outright falsification, particularly on Third, these numbers tell us almost nothing about even mildly controversial questions. the policies or the longer-term political prospects of Although polling is somewhat more common in the the governments in any of these countries, which are region today than it was until just a few years ago, that without exception autocratic and only dimly or indi- could actually increase rather than decrease the level rectly attuned to public opinion. of official concern and interference with survey activi- In short, even with the best of intentions, many ties and results. Experience and expert personal obser- “Arab world” surveys suffer from severe and mutually vation suggest that is indeed the case today in several reinforcing problems of sample design and execution, major Arab states. Social controls are also quite oner- social controls, government surveillance, dearth of ous in many places, leading an unknown, but prob- credibility checks, and most of all, absence of any clear ably relatively high, proportion of respondents to offer links to events on the ground. At best, these polls are socially acceptable but disingenuous responses. just imperfect snapshots of what people are willing Even if these problems could somehow be solved, to say to strangers, quite possibly with only a tenuous or at least finessed, the equally serious problem would connection to actual behavior at either the popular or The Washington Institute for Near East Policy vii David Pollock Slippery Polls the policy level. At worst, they are so fatally flawed as For example, majorities of both Egyptians and Jor- to verge on the fraudulent. danians still deny that Arabs carried out the attacks of In between those two poles lie the “average” Arab September 11, 2001. Most Arabs polled say they want polls, which are usually merely misleading as a tool for democracy—but also sharia (Islamic) law and cleri- analyzing, predicting, or influencing interactions with cal influence on government. Surprisingly, those who the United States. The only findings worth taking seri- value democracy more seem to be slightly more, not ously (and even then with a grain of salt) are those con- less, inclined to support anti-American terrorism. And firmed over and over again by different credible poll- most Arabs polled, contrary to common misconcep- sters, in response to relatively unbiased questions, with tion, have negative views both of the United Nations a consistent trend over several years. and of the role of religion in the West. The most important such finding, examined in Third, on the basis of criteria previously defined for detail in part II of this essay, is precisely the one that best polling practices, part II examines country case is least often acknowledged: Popular sympathy for studies. Again, many surprises, including a few pleas- terrorism, including terrorism against Americans, has ant ones, are in store for those who dissect the data declined drastically in every Arab society polled—even carefully. though popular attitudes toward the United States Among the Palestinians, conventional wisdom have improved only slightly and unevenly in those notwithstanding, polls did predict the overall 2006 countries since their sharp decline right after the occu- vote fairly accurately. But they missed the Hamas pation of Iraq in 2003. victory, because they neglected to factor in a hybrid Since 2004, the most striking new trend in regional electoral system that greatly magnified the Hamas opinion is the steady surge toward greater popular oppo- plurality. Two years later, however, Hamas missteps sition to terrorism. This trend specifically includes grow- in power have produced clear erosion in its popular ing opposition to any attacks on American civilians any- support. where among all Arab publics polled on such questions In Iraq, despite the violence, polling is also among by different pollsters, many times over. These findings the best in the region, in great part because more are all the more remarkable because the same polls show people now feel free to speak their minds. The latest little improvement in popular views of the United States results suggest some convergence of views among the or of American policies during that period.