Measurement Instruments for Opinions in Europe
Total Page:16
File Type:pdf, Size:1020Kb
(XUREDURPHWHU0HDVXUHPHQW ,QVWUXPHQWVIRU2SLQLRQVLQ(XURSH :LOOHP(6DULV 0D[.DDVH (GV ZUMA, Mannheim 1997 Saris/Kaase (Eds.): Eurobarometer. Measurement Instruments for Opinions in Europe =HQWUXPIU8PIUDJHQ0HWKRGHQXQG$QDO\VHQ =80$ ZUMA is a member of the "Gesellschaft Sozialwissenschaftlicher Infrastruktureinrichtungen e.V." (GESIS). GESIS is a member of the "Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz" (WGL). 9RUVLW]HQGHUGHV7UlJHUYHUHLQV=80$H9: Prof. Dr. Max Kaase 'LUHNWRU: Prof. Dr. Peter Ph. Mohler $QVFKULIW B2,1 PO Box 12 21 55 D-68 072 Mannheim 7HOHIRQ 0621/1246-100 (PDLO [email protected] ,QWHUQHW http://www.social-science-gesis.de/ *(6,6 http://www.zuma-mannheim.de/ =80$ ,6%1 =80$ =80$1DFKULFKWHQ6SH]LDO%DQG 7DEOHRIFRQWHQWV Preface ..........................................................................................................................................4 Chapter 1 TheEurobarometer - A tool for comparative survey research ............................5 %\0D[.DDVHDQG:LOOHP(6DULV Chapter 2 Sample design and consequences .......................................................................24 %\3HWHU6FKXEHUWDQG$QJHOLND*UHLO Chapter 3 Deviations from the population and optimal weights.........................................32 %\6DELQH+lGHUDQG6LHJIULHG*DEOHU Chapter 4 Telephone ownership - a cause of sampling bias in Europe? .............................45 %\-UJHQ/DVV Chapter 5 Sizes of the different effects: Coverage, mode and nonresponse........................64 %\-UJHQ/DVV:LOOHP(6DULVDQG0D[.DDVH Chapter 6 Mode effects in the standard Eurobarometer questions......................................75 %\:LOOHP(6DULVDQG-DFTXHV$+DJHQDDUV Chapter 7 Mode effects on open-ended agenda questions...................................................89 %\+HUPDQQ6FKPLWW3HWHU6FKURWWDQG0LFKDHOD7KRPD Chapter 8 The left-right self-placement question in face to face and telephone surveys................................................................................................100 %\+DQV'LHWHU.OLQJHPDQQ Chapter 9 Comparability across mode and country.............................................................111 %\:LOOHP(6DULV Chapter 10 Adjustment for differences between face to face and telephone interviews.......126 E\:LOOHP(6DULV Chapter 11 Summary and discussion ....................................................................................142 %\:LOOHP(6DULVDQG0D[.DDVH References.....................................................................................................................................155 Biographical notes on contributors...............................................................................................161 Saris/Kaase (Eds.): Eurobarometer. Measurement Instruments for Opinions in Europe 3UHIDFH For more than two decades the biannual representative sample survey studies conducted on behalf of the Directorate General X of the European Commission - the so-called Eurobarometer - have been a major data source for comparative research in the social sciences, especially political science, although they are mainly conducted for policy counseling reasons (for an assessment of these studies see Reif and Inglehart, 1991; the analytic potential of the Eurobarometers is documented in Kaase and Newton, 1995). In order to speed up the policy information process, Karlheinz Reif, at that time responsible for the Eurobarometer operation at the European Commission, in 1993/1994 had proposed the normal Eurobarometers to be supplemented by a regular tracking study on European attitudes on a monthly basis. Since this objective could not be reached by face to face personal interviews for reasons of cost and speed of data gathering, the option had to be validated to gather the tracking material through personal telephone inerviews. An important question to be answered there was whether, given the uneven distribution of private telephone ownership in the member countries of the European Union, it was nevertheless possible to produce valid, reliable and representative data based on telephone ownership household samples. Fortunately, this question could be answered, through the ingenuity and entrepreneurship of Karlheinz Reif, on experimental basis. For one, Reif assembled a group of methodologically proficient social scientists to advise him on this problem. Second, through an investment by the Berlin-based FORSA institute and its director Manfred Güllner, as well as additional resources from the European Commission, an experimental setting could be constructed (for details see chapters 1 and 2 of this book) which was able to confront data from face to face and telephone interviews covering representative samples of the respective populations, and to add a telephone panel component in three countries (Belgium, France and Spain) to the regular April 1994 Eurobarometer (EB 41.0), which permitted tests of possible mode effects comparing face to face and telephone interviews and tests of the functional equivalence of questions across countries. The following chapters present analyses of such problems and draw conclusions for future research. Given the fact that all over Europe especially in commercial market research the number of telephone interviews is drastically increasing and that of personal face to face interviews is decreasing, the findings from the experimental research described in this book reach far beyond the Eurobarometers. The editors thus very much hope that this study will contribute to improving the quality of social research in the commercial as well as in the academic realm. As was mentioned before, this work owes a lot to Karlheinz Reif and to Manfred Güllner. Next to the editors, Roger Jowell, Hans-Dieter Klingemann, and Hermann Schmitt were the principal investigators in the advisory group. The editors also appreciate the decision by ZUMA to publish the manuscript in its special series on methodological problems in social research, and in particular the great effort by Suzanne Kabel of the University of Amsterdam and of Jolantha Müllner of ZUMA in producing this manuscript. Willem E. Saris, Max Kaase Amsterdam and Berlin, November 1997 =80$1DFKULFKWHQ6SH]LDO%DQG &+$37(5 7+((852%$520(7(5 $722/)25&203$5$7,9(6859(<5(6($5&+ 0$;.$$6($1':,//(0(6$5,6 'HYHORSPHQWRIFRPSDUDWLYHVXUYH\UHVHDUFKLQSROLWLFDOVFLHQFH The fourties in the United States witnessed the development and implementation of techniques which were capable of reliably assessing social as well as political attitudes, beliefs and behaviours of large-scale populations through small samples of respondents. The application of probability theory in the development of sampling procedures in connection with the emergence of the standardised personal interview laid the ground for what has since long become a normal tool of social research: surveys with a limited number of respondents, say 2000, producing information that can be reliably generalised to the population from which the sample was drawn, almost independent of the size of that population. Soon after the end of World War II this methodology spread from the United States to Western Europe where it quickly became the most frequently used instrument in market, social and political research. However, the expertise required to apply this tool intelligently, reasons of cost and not the least a certain conservatism in the academic world for quite a while made surveys a rather scarce phenomenon in sociology and political science. Only as the field of comparative government slowly transformed into comparative politics with its more pronounced interest in political processes and, as one central element in it, citizen orientations, survey data began to assume the kind of important place they now possess in this field (for electoral research as one example see the overview in Thomassen 1994). More than anything else it was the 'Civic Culture' comparative study of political orientations in the United States, the United Kingdom, (West) Germany, Italy and Mexico by Almond and Verba (1963) which revealed the analytical potential of this new methodology for political science. There are many reasons why this variant of empirical research did not spread much more quickly after the initial success of the 'Civic Culture'. Some of the problems are spelled out succinctly in a book by Szalai and Petrella (1977) discussing five major East-West comparative survey studies which had been conducted in the 60s, partly under the auspices of the Vienna- based European Co-ordination Centre for Research and Documentation in the Social Sciences (Vienna Centre). The experiences from those studies, as summarised in the Szalai and Petrella book, give a good flavour of the theoretical and methodological challenges as well as the practical difficulties this kind of comparative survey research has had to confront (see also Verba, 1971). It is little wonder then that the Political Action study (Barnes et al., 1979) Saris/Kaase (Eds.): Eurobarometer. Measurement Instruments for Opinions in Europe conducted in seven European nations and in the United States in the 70s (with a replication in three of those countries around 1980; see Jennings et al., 1990) for a long time stood alone in university-based comparative survey research in political science. Since then, the 1981 and 1990 European Values Studies (Harding et al., 1986; Ester et al., 1993) as well as the Beliefs in Government project (Kaase and Newton, 1995) are further hallmarks of comparative survey research and its uses in political science.