1

Presented jointly by the Steering Committee and the Methodology Committee for an ESS

The European Social (ESS) – a research instrument for the social sciences in Europe

Summary

Summary version of the Report prepared for the Standing Committee for the Social Sciences (SCSS) of the European Science Foundation (ESF)

The full version of this report is also available from ESF 2 The European Social Survey (ESS) – a research 3 instrument for the social sciences in Europe

Contents page

The Case for a European Survey 5

Designing the European Social Survey 6 Population to be surveyed 6 7 Timing 8 Participating countries 8

Themes for the European Social Survey 9 Examples of questions for the core modules 10 Examples of possible module topics 10

Methodological Research 11

Data Management, Archiving and Distribution 11

Organisational Structure 11

Costs 12

Funding 13

Appendices Expert Group on a European Social Survey - List of members 14 Steering Committee - List of members 14 Methodology Committee - List of members 16 4 5

The Case for a they do not deal with the whole European Social range of individual orientations (attitudes, beliefs and behaviours) Survey that are central in understanding modern societies, and that can be The social sciences, in common assessed by survey research. with the natural sciences, aim for generalisations across time and Very many individual surveys space. They must also, however, have been and will be conducted pay close attention to the social in particular European countries. and institutional arrangements Inevitably, however, these are that structure human interaction. tailored to individual projects and Europe’s cultural diversity thus interests. The social sciences, if makes it a natural laboratory for they are to make progress, require the social sciences, which can regular cross-national surveys that analyse differences in institutions, are conceptually well anchored, structures, behaviours and beliefs conducted according to rigorous across European states and relate methodological standards and are these to explanations of human available at little cost to the entire interaction. research and policy community. Such studies must be designed for The social sciences in Europe have use by a broad variety of people a long tradition in empirical for a broad variety of purposes. No analysis. However, most of the such database currently exists in empirical research that has been Europe, and this is the essential done is not of a truly comparative rationale for a regular European nature. The result is that essential Social Survey (ESS). This will comparative data are either provide a core research missing altogether, or are available infrastructure for a broad range of in such different forms in social science disciplines: political different countries that the basis science, sociology, social for comparison becomes psychology, mass communication, extremely fragile. economic sciences, modern social history and social anthropology. Of course, there is a wealth of The data will be of value to data on individuals and scholars, politicians, policy- households that are regularly makers and the public alike. As collected by statistical offices and survey builds upon survey, users other administrative agencies. will be able to construct a long- These data, however, are term account of change and frequently not comparable across development in the social world nations and are often not of modern Europe. accessible to researchers for reasons of administrative self- interest or data protection. Absolutely crucially, however, 6 Designing the European Social Survey

Designing the researchers and area specialists. European Social The equivalence problem also implies the need for extensive Survey pretesting and very thorough methodological work, as well as Since the ESS combines its core context-sensitive documentation. concerns for international comparison and the study of mid- An innovative high quality survey to long-term change, its design such as the ESS not only opens up will involve interviewing a unique opportunity for independent cross-sectional methodological research but samples of people in each wave of requires, in order to maintain the the survey, conducted once every highest standards, systematic two years, with a common core of methodological scrutiny before questions being asked in each the data are made available. In wave. Each wave will also include addition technology-induced up to three research modules that developments in survey will be repeated over much longer methodology, such as computer- time intervals. The ESS will thus assisted interviewing, must be steadily unfold its full potential as anticipated and evaluated. more and more waves of the survey accumulate. The expected high cost of the ESS has given rise to debates whether A major problem in comparative the data should be embargoed in research is the translation of some way and made available to terms and concepts. Even interested researchers from non- seemingly simple translations of participating countries only at a single words give problems that substantial price that would help arise from different cultural recover costs. It is not proposed, meanings. Back-translation however, to charge any but the procedures can be used to aim for most marginal cost for ESS linguistic equivalence. Yet even datasets. The core philosophy excellent translations provide no behind the ESS is to enhance the guarantee that terms are entire social research functionally equivalent in infrastructure in Europe and different societies. The search for anything that undermines this comparisons that are valid over goal must be avoided. time and between cultures strongly implies a need for theory- Population to be surveyed driven research and a clear Increasingly permeable European specification of the meaning of borders make definition of the core concepts. This implies that population from which to sample translation and back-translation more complex than it would have procedures should be conducted been in the past. Since the ESS has on the basis of extensive a time perspective that looks deep consultations with both expert 7

into the 21st century, the key The most important element of definition of the the population definition population must be made very proposed for the ESS is the carefully. It is also vital that inclusion of non-national equivalent sampling procedures residents. The biggest difficulty are used in all participating that will be encountered in this countries. respect is a language problem. Obviously, when a non-national Taking these factors into resident speaks and understands consideration, the survey will cover the language of the country of people 15 years and older, with no residence, no problem entails. upper age limit, who are resident in Otherwise, in countries where a the country, regardless of minority language is spoken as a nationality, citizenship or legal first language by 5 % or more of status. This definition is the total population, the important for a variety of reasons. will be translated First, in starting at the age of 15, into that language and suitable easy sampling from voter registers interviewers will be trained. Since will not be possible. Nonetheless a substantial number of tested there are good reasons that derive language versions of the from socialization theory that questionnaire will be available, imply a lower age limit. Second, furthermore, there may well be “resident populations” include practical solutions to the language people living in institutions such problem in many cases where the as old age homes, university or 5 % threshold is not reached. school housing, prisons, hospitals and the military as well as groups Sample such as the homeless. These are The sample will be selected by typically excluded from major strict random probability methods national surveys. The substantial at every stage. The relative ageing of many European selection probabilities of every populations, however, means that sample member will be known and serious consideration must be recorded on the data set. Quota given to including people living in sampling will not be used at any old age homes. The same applies stage. While random sampling is to people who spend extended not equally common in all time living in educational countries, it can be implemented institutions. On the other hand, without major problem and people in inaccessible institutions provides a solid basis of such as prisons and military comparability across countries. barracks can be excluded from the population for practical reasons, Even though many commercial although some flexibility will be surveys are now conducted via the needed in this matter. telephone, telephone coverage of private households is nowhere 8 Designing the European Social Survey

near complete in many European preparation. This timing will of countries. This means that the course depend on the speed with interviewing mode for the ESS which the responsible bodies take will be face-to-face personal their decisions on funding the interviews. ESS.

A very important decision with a Participating countries big impact on costs concerns the It is not yet clear which countries number of interviews to be will participate in the first wave conducted in each country. A of the ESS. There was very great high-quality survey such as the interest on the part of ESF ESS will need a relatively large member countries in the design sample size to allow for the of the blueprint, but this may not effective statistical analysis of indicate an automatic willingness data relating to relatively small to bear the costs of joining the groups. Since, on the other hand, ESS programme. The more or less survey cost is such an important full participation of all ESF factor, some flexibility on sample member countries cannot size may be needed. The therefore be a sine qua non for the recommended sample size is thus ESS to proceed. On the other 2500 and the minimum is set at hand, every effort should be made 2000 not considering design to ensure as wide a participation as effects. Therefore, the effective possible. Every additional country minimum sampling size must be that joins increases the analytical 1500 interviews. The target power and value of the project. response rate is high, at a For every non-participating minimum of 75% of eligible country, the decision not to sample members. Response rates participate will be very costly in cannot be legislated, but they can terms of not being involved in the be heavily influenced by insisting detailed design work and, most on fieldwork procedures that importantly, in not having access maximise the chances of finding to an important dataset that links elusive sample members. it to all participating countries.

Timing While it would have been attractive for many reasons to conduct the ESS every year, practical arguments of costs and feasibility imply that it be conducted every second year, at least in the preliminary waves. The first wave is planned for the year 2001 in order to allow for high-quality design and 9

Themes for the modules are the basis of the European Social continuous aspect of the ESS and will provide the opportunity to Survey test and develop dynamic approaches analysing the social, The focus of the ESS will be the political, and cultural beliefs and systematic study of European behaviours of Europeans. citizens’ attitudes, attributes, and behaviour relating to a core set of The selection of each project economically, socially and module will be based on three politically relevant domains. It important criteria. First, each will study distributions, element of the ESS will be differences and changes across designed and developed on the time and space in the social, basis of sound theoretical political and cultural beliefs and arguments. Second, instruments behaviours of Europeans. The used in the ESS will in general ESS will not be just another have proven their usefulness in survey concerned empirical research (although not with specific current or necessarily in an international fashionable themes. Rather, it will comparative setting). To avoid a be a systematic instrument conservative bias, opportunities designed to enable and stimulate will be provided for innovative innovative research on the basis topics and questions, but these of existing knowledge, but at the will require meticulous pretesting same time flexible enough to and methodological scrutiny. cover new ground. Third, each instrument used in ESS will be relevant for analysing Each wave of the ESS will consist the dynamics of the social, of three parts: a core module cultural, and political beliefs and designed to tap change and behaviours of Europeans. persistence in attitudes; a core Researchers will be invited to module dealing with social and submit proposals for modules to demographic attributes; modules be included, and an international for specific research projects. competition will be held to decide About 15 minutes of which project modules are time will be devoted to each of actually put into the field. In this the core modules, and about 10-12 way, the ESS will be a facility that minutes to each of the two is wide open to the scientific project modules. This gives a total community, demonstrating best- interview length of about 55 practice standards and stimulating minutes. To increase the power new developments in the social and efficiency of the ESS, a third sciences. module will be administered in the form of a self-completion questionnaire. The two core 10 Themes for the European Social Survey

Examples of questions for the voluntary associations, interest core modules groups, and church related Obviously, a listing of questions organisations; informal networks, for the core modules can only be professional contacts, etc. very provisional at this stage, but some indication of the types of . Social context and environment question that might be used may Social and political embeddedness, be helpful. economic development, organizational structure (civil . Individual attitudes and society), etc. attributes – issues and problems (such as Examples of possible module crime, inequality, unemployment, topics public spending, etc.); In order to convey at least a – orientations towards democracy flavour of what might be covered, (satisfaction, political trust, and without prejudice, the confidence in institutions, left- following topics are offered from right self-placement, etc.); the three realms of the social – media usage/exposure and structure, the political structure communication (reading and the cultural sphere. They are newspapers and magazines, no more than examples. watching tv, listening to the radio, internet and multimedia usage); . Social structure – political involvement (political – families, primary groups, and interest, political participation, social networks; voting behaviour, etc.); – mobility, immigration and – social and political orientations multi-culturalism; (individualisation, victimisation, – social inclusion and inequality; postmaterialism, etc.); – socio-political identity . Political structure (national identity, ethnocentrism, – civil society and trust; xenophobia, patriotism, etc.); – democracy and political – leisure activities, life style involvement; issues, etc. – interest groups and political – Sex/gender, nationality, birth, parties; family situation, housing/ residence, social class, education, . Cultural sphere employment/unemployment, – life styles and life course religion, occupation, household, including life-long education; personal income, household – media and the knowledge income, number of people in society; household, etc. – subnational, national and transnational identities. . Social position and networks Family structure; involvement in 11

Methodological distribution of the international Research ESS data set. Distribution will be via modern media, currently CD A large and innovative venture ROM or Internet, subsets of data such as the ESS requires extensive may be distributed via diskette. methodological and quality Given rapid developments in data control. It is important that the distribution and retrieval ESS routinely monitor and assess technologies, internet services for the implementation and effect of easy retrieval and access to ESS the questionnaire, sampling and data will be design and of non-responses. This developed. will aid interpretation of the data and ensure that high standards ESS supports a policy of free and and comparability are maintained. easy access to its integrated In addition to this routine dataset. Data will be made assessment, the opportunity available to the scientific should be grasped to carry out community at no more than original methodological research handling charges. Other uses will designed to extend knowledge be subject to agreement with ESS (and ultimately to improve best and the Archive. practice) concerning the effects of non-response and the role of the interview in this process: Organisational Structure

Data Management, An enterprise of the size and the Archiving and continuity of the ESS cannot be mustered without a stable support Distribution structure. On the other hand, since the survey will serve the A decision will be taken about social science community as a which of the existing European whole, significant “bottom-up” data archives become(s) the ESS elements must integrated into the archive. Rigorous standards will organizational structure. This be defined to ensure the indicates the following principles: compatibility of ESS national data sets for integration into a The ESS will have both common core for international . Steering and Methodology comparison. The national data sets Committees for overall will be checked, cleaned and substantive and methodological documented by the survey direction. organization after the completion The practical conduct of the of fieldwork and then be sent to . survey requires a permanent the ESS archive for integration, Methodological Team to supervise documentation, archiving and the survey, act as a link between 12 Costs

Principal Investigators and Data will be responsible for guiding the Archive(s) and be in regular work of the Methodological Team contact with the Methodology and in particular for the design of Committee on all ESS-related the ongoing methodological topics, including methodological research. research. The Methodological . The most important bottom- Team will be linked to an up elements in the ESS are the experienced and resourceful survey teams of researchers who compete organization. The Methodological to design the topical modules. Team will comprise: These groups should be small in – a full-time senior coordinator size (3-6), but should preferably who will stimulate, organise and have a multi-national composition. supervise all aspects of the ESS from design to successful delivery; – four half-time positions for Costs experienced researchers for (1) pretesting, questionnaire What follows are educated construction and translation; estimates of expected costs. These (2) development are divided in two parts: (1) the and implementation; total cost of the field work and (3) data quality check and data data preparation in (we assume for multi-level analyses; 16) participating countries, and (4) data analysis and indicator (2) the fixed cost of the ESS that validation/construction; will occur more or less – two half-time positions for independently of the number of junior researchers; participating countries (the only – a full-time secretary. major exception here is the cost of half-time Principal Investigator for . The Steering Committee, two years in each of 16 countries, consisting of one senior social for which the gross amount will, of scientist from each participating course, vary with the number of country, will be responsible for countries involved). selecting themes for the ESS modules by way of an Cost of surveys international competition. Extensive consultation has taken Steering Committee members will place on fieldwork costs in likely also provide the links between the participating countries. The mean ESS and their national estimate for fieldwork in a single communities. Finally, the Steering country was 262,000 EURO, VAT Committee will be the major link excluded; the median was 255.000 to funding organizations and take EURO. Assuming that those 16 all general budgetary decisions. countries which provided the . The Methodology Committee, Methodology Committee with consisting of 6 to 8 senior social cost estimates will participate, scientists with expertise in social then the overall field work cost science methodological research, would be 4,200,000 EURO. 13

Fixed costs In order to obtain an overall cost Since the rationale for the various figure for the first two-year wave cost elements has already been laid of the ESS, all individual items out in previous sections, these are are costed for a two-year period. simply itemized and costed below. These costs are (in 1000 EURO):

Costs for first wave of survey – two years costs (in 1000 EURO) . one project director, full-time ...... 200 . four senior researchers, half-time ...... 320 . two junior researchers, half-time...... 130 . one secretary, full-time ...... 70 . auditing and financial control ...... 50 . 16 principal investigators, half-time...... 800 . overhead 20 % of 1.570 ...... 314 1.884 Methodological research . pretest work (600 interviews each in three countries) ...... 225 . methodological experiments ...... 60 285 Other cost . archival work...... 150 . travel expenses committees (25 members, 2 meetings per year, 1000 EURO each) ...... 100 . travel expenses topic specialists (5 members, 3 meetings, 1000 EURO each) ...... 15 . travel expenses principal investigators (15 members, 3 meetings, 1000 EURO each) ...... 48 . consultancy fees ...... 50 363

Total fixed cost 2.532 Contingency fund 10 % ...... 253 2.785 Total cost of European Social Survey (for 2 years) . field work ...... 4.200 . fixed costs ...... 2.785 Grand total ...... 6.985

Funding individual National Science Foundations to fund their own The working assumption at the survey costs every two years, time of the writing of this report averaging at an amortised is that the central (fixed) costs of annualised cost of around 130,000 around 2.8 m EURO per survey EURO for each participating (1.4 m EURO per year) would be nation (more in the Scandinavian centrally-funded (ideally via a countries and Germany, less in mechanism such as the EU’s Fifth many other countries and much Research Framework less in some). Programme). This would leave the 14 Appendix 1.1 Expert Group on a European Social Survey – List of members

Professor Max Kaase Professor Juan Diez Professor Roger Jowell Professor Antonio (Chairman) Nicolas Director Schizzerotto Wissenschaftszentrum (Professor of Sociology, Social and Community Facoltà di Sociologia Berlin Universidad Complutense Planning Research Università degli Studi di für Sozialforschung de Madrid) (SCPR) Trento FS Sozialer Wandel, ASEP 35 Northampton Square Via Verdi 26 Institutionen und Paseo de la Castellana 173, London EC1V 0AX 38100 Trento Vermittlungsprozesse 5º I United Kingdom Italy Reichpietschufer 50 28046 Madrid 10785 Berlin Spain Professor Leif Dr. L. Henk Germany Nordberg Stronkhorst Professor Fredrik Åbo Academy University Head, Scientific Mr. Bruno Cautrès Engelstad Department of Statistics Statistical Agency CIDSP/IEP de Grenoble Institute for Social Fänriksgatan 3B NWO-WSA Domaine Universitaire Research (ISF) 20500 Turku Postbus 93120 B.P. 45 Munthesgate 31 Finland 2509 AC The Hague 38402 Saint Martin 0260 Oslo The Netherlands d’Hères cedex Norway France Dr. John H. Smith (Secretary) European Science Foundation 1 quai Lezay-Marnésia 67080 Strasbourg cedex France Appendix 1.2 Steering Committee – List of members Professor Max Kaase Professor Jacques Mr. Bruno Cautrès Professor Henryk (Chairman) Billiet CIDSP/BDSP Domanski Wissenschaftszentrum Department of Sociology Institut d’Etudes Institute of Philosophy Berlin für Catholic University Politiques de Grenoble and Sociology Sozialforschung Leuven Domaine Universitaire - Polish Academy of FS Sozialer Wandel, Van Evenstraat 2B B.P. 48 Sciences Institutionen und 3000 Leuven 38040 Grenoble Cedex 9 Nowy Swiat 72 Vermittlungsprozesse Belgium France Palac Staszica Reichpietschufer 50 Tel: +32 16 32 31 57 Tel: +33 4 76 82 60 94 00-330 Warsaw 10785 Berlin Fax: +32 16 32 33 65 Fax: +33 4 76 82 60 50 Poland Germany E-mail: E-mail: [email protected] Tel: +48 22 826 96 17 Tel: +49 30 254 91 293 [email protected] grenoble.fr Fax: +48 22 826 99 48/ Fax: +49 30 254 91 318 +48 22 826 78 23 E-mail: [email protected] Professor Antonio Professor Nikiforos E-mail: berlin.de Brandao Moniz Diamandouros [email protected] Faculdade de Ciencias e National Centre for Social Professor Rune Åberg Tecnologia Research Professor Yilmaz University of Umeå Universidade Nova de 14-18 Messoghion Av. Esmer Department of Sociology Lisboa 11527 Athens Faculty of Economic and 901 87 Umeå Quinta da Torre Greece Administrative Sweden 2825 Monte de Caparica Tel: +30 1 74 89 123 Sciences, Bogazici Tel: +46 90 786 59 60 Portugal Fax: +30 1 74 89 127 University Fax: +46 90 786 66 94 Tel: +351 1 295 4464 E-mail: [email protected] P.K. 2 Bebek E-mail: Fax: +351 1 295 4461 Istanbul [email protected] E-mail: [email protected] Turkey Tel: +90 212 263 15 50 Fax: +90 212 265 63 57 E-mail: [email protected] 15

Dr. Peter Farago Professor José Ramón Substituted by: Substituted by: Landert Farago Montero Mr. Torben Fridberg Mr. Pierre Desmarez Davatz & Partner Fundación Juan March Socialforskningsinstituttet TEF – CP 124 Grossmünsterplatz 6 Centro de Estudios Herluf Trolles Gade 11 Avenue Jeanne 44 8001 Zurich Avanzados en 1052 Copenhagen K 1050 Bruxelles Switzerland Ciencias Sociales Denmark Belgium Tel: +411 265 39 90 Castello, 77 Tel: +45 33 48 08 47 Tel: +32 2 650 34 30 Fax: +411 265 39 99 28006 Madrid Fax: +45 33 48 08 33 Fax: +32 2 650 33 35 E-mail: [email protected] Spain E-mail: [email protected] E-mail: [email protected] Tel: +34 1 435 42 40 Professor Roger Fax: +34 1 576 34 20 Professor Shalom Professor Niko Tosˇ Jowell E-mail: Schwartz Faculty for Social Sciences Social and Community [email protected] Department of University of Ljubljana Planning Psychology Kardelieva Pz. 4 Research (SCPR) Dr. Karl H. Müller Hebrew University of 1000 Ljubljana 35 Northampton Square Institute for Advanced Jerusalem Slovenia London EC1V 0AX Studies United Kingdom Mount Scopus Tel: +386 61 349 994 Department of Sociology Jerusalem 91905 Fax: +386 61 1686 054 Tel: +44 171 250 1866 Stumpergasse 56 Israel E-mail: [email protected] Fax: +44 171 250 1524 1060 Vienna E-mail: [email protected] Tel: +972 2 588 30 24 Austria Fax: +972 2 588 11 59 Professor Michael Tel: +43 1 59 991 212 E-mail: Warren Professor Stein 30 Cole Park Road Kuhnle Fax: +43 1 59 991 191 [email protected] Department of E-mail: Twickenham Comparative Politics [email protected] Mrs. Ineke Stoop TW1 1HS Middlesex University of Bergen Department of United Kingdom Christiesgate 15 Professor Leif Information and Tel: +44 181 891 3130 5007 Bergen Nordberg Automation Fax: +44 181 891 1970 Norway Åbo Academy University Sociaal en Cultureel E-mail: Tel: 47 55 58 21 79 Department of Statistics Planbureau [email protected] Fax: 47 55 58 94 25 Fänriksgatan 3B Parnassusplein 5 E-mail: 20500 Turku P.O. Box 16164 Dr. John H. Smith [email protected] Finland 2500 BD The Hague (ESF Scientific Secretary Tel: +358 2 215 4163 The Netherlands for the Social Sciences) Ms. Geneviève Professor Michael Fax: +358 2 215 4677 Tel: +31 70 340 70 00 Laver E-mail: Schauinger Fax: +31 70 340 70 44 (Administrative Assistant) Department of Political [email protected] E-mail: [email protected] Sciences Mrs. Beatriz Garcia The Policy Institute Dr. Niels Ploug Diaz-Ambrona Professor Françoise (Administrative Secretary) Trinity College National Institute for Thys-Clement Dublin 2 Social Research Pro-Recteur European Science Ireland Herluf Trolles Gade, 11 Université Libre de Foundation Tel: +353 1 608 20 36 1052 Copenhague K Bruxelles 1, Quai Lezay-Marnésia Fax: +353 1 677 05 46 Denmark Avenue F.D. Roosevelt 50 67080 Strasbourg Cedex E-mail: [email protected] Tel: +45 33 48 08 40 Case Postale 140 France Fax: +45 33 48 08 33 1050 Bruxelles Tel: +33 (0)3 88 76 71 31 Professor Guido E-mail: [email protected] Belgium Fax: +33 (0)3 88 37 05 32 Martinotti E-mail: [email protected] Via del Caravaggio, 5 Tel: +32 2 650 41 73 20144 Milano Fax: +32 2 650 44 95 Italy E-mail: [email protected] Tel: +39 2 861 089 Fax: +39 2 861 891 E-mail: [email protected] 16 Appendix 1.3 Methodology Committee - List of members

Professor Roger Dr. Ekkehard Professor Antonio Dr. John H. Smith Jowell (Chairman) Mochmann Schizzeroto (ESF Scientific Secretary for Social and Community Zentralarchiv fur Dipartimento di the Social Sciences) Planning emprische Sociologia e Ricerca Ms. Geneviève Research (SCPR) Sozialforschung Sociale Schauinger 35 Northampton Square Universität zu Köln Università di Trento (Administrative Assistant) London EC1V 0AX Bachermer Str. 40 Via Verdi 26 Mrs. Beatriz Garcia United Kingdom 50931 Köln 41 38100 Trento Diaz-Ambrona Tel: +44 171 250 1866 Germany Italy (Administrative Secretary) Fax: +44 171 250 1524 Tel: +49 221 394 737 Tel: +39 0461 881 315 European Science E-mail: [email protected] Fax: +49 221 476 9477 Fax: +39 0461 881 348 Foundation E-mail: [email protected] E-mail: 1, Quai Lezay-Marnésia Professor Jacques koeln.de [email protected] 67080 Strasbourg Cedex Billiet France Department of Sociology Professor José Professor Dr. Jan van Tel: +33 (0)3 88 76 71 31 Catholic University Ramón Montero Deth Fax: +33 (0)3 88 37 05 32 Leuven Fundación Juan March Lehrstuhl für Politische E-mail: [email protected] Van Evenstraat 2B Centro de Estudios Wissenschaft und 3000 Leuven Avanzados en International Belgium Ciencias Sociales Vergleichende Tel: +32 16 32 31 57 Castello, 77 Sozialforscung Fax: +32 16 32 33 65 28006 Madrid Universität Mannheim E-mail: Spain Seminargebaude A 5 [email protected] Tel: +34 1 435 42 40 68131 Mannheim Fax: +34 1 576 34 20 Germany Dr. Peter Lynn E-mail: Tel: 49 621 292 5316 Survey Methods Centre [email protected] Fax: 49 621 292 5315 Social & Community E-mail: [email protected] Planning Research Professor Willem mannheim.de 35 Northampton Square Saris London EC1V 0AX Department of Statistics Dr. Joachim Vogel United Kingdom and Methodology Statistics Sweden Tel: +44 171 250 18 66 Faculty of Political, Box 24300 Fax: +44 171 250 15 24 Social and Cultural 104 51 Stockholm E-mail: [email protected] Sciences Sweden University of Tel: +46 8 783 4930 Dr. Nonna Mayer Amsterdam Fax: +46 8 783 4772 Centre d’Etude de la Vie O.Z. Achterburgwal 237 E-mail: Politique Française 1012 DL Amsterdam [email protected] 10, rue de la Chaise The Netherlands 75007 Paris Tel: +31 20 525 2091 Professor Max Kaase France Fax: +31 20 525 2086 (Chairman Steering Tel: +33 1 49 54 21 49 E-mail: [email protected] Committee) Fax: +33 1 49 54 20 25 Wissenschaftszentrum E-mail: [email protected] Berlin für Sozialforschung Reichpietschufer 50 10785 Berlin Germany Tel: +49 30 254 91 293 Fax: +49 30 254 91 318 E-mail: [email protected] berlin.de 1

Presented jointly by the Steering Committee and the Methodology Committee for an ESS

The European Social Survey (ESS) – a research instrument for the social sciences in Europe

Report prepared for the Standing Committee for the Social Sciences (SCSS) of the European Science Foundation (ESF)

A summary version of this report is also available from ESF 2 The European Social Survey (ESS) – a research 3 instrument for the social sciences in Europe

Contents page

1. Preface 5

2. The Case for a European Social Survey 7 2.1 European diversity and the social sciences 7 2.2 Helping to build a European identity in social research 8 2.3 The data problem 9

3. Designing the European Social Survey: General Principles and Concrete Considerations 10 3.1 General principles 10 3.1.1 No duplication of efforts is acceptable 10 3.1.2. Doing this research is not just a one-time affair 12 3.1.3 Comparability must be based on equivalence 13 3.1.4 Multi-level analyses must ensue 15 3.1.5 Continuous methodological research is essential 16 3.1.6 Flexibility and stability must be amalgamated 16 3.1.7 Data access must be easy and inexpensive 17 3.2 Population and sample 17 3.2.1 Population defined 18 3.2.2 Sample defined 20 3.3 Timing 22 3.4 Participating countries 22

4. Selection of Themes for the European Social Survey 23 4.1 Research focus 23 4.2 Core sets and module topics 23 4.2.1 Core sets 25 4.2.2 Module topics 25 4.3 Choice of topics 26 4.3.1 Procedural rules and criteria 26 4.3.2 Examples of questions for the two core sets 27 4.3.3 Selection and examples of module topics 27

5. Methodological Research 28 5.1 Questions and question wording 28 5.2 Sampling and non-response on the ESS 29

6. Data Management, Archiving and Distribution 31 6.1 Integration, documentation, validation and archiving of the database 32 4

6.2 Refinement of data base 32 6.3 Data distribution 33 6.4 Event data 33 6.5 Other archival concerns 34

7. Organisational Structure 35

8. Costs 38 8.1 General remarks 38 8.1.1 Cost of survey 38 8.1.2 Fixed costs 38

9. Funding 39

Appendices 41 Expert Group on a European Social Survey – List of members 41 Steering Committee – List of members 42 Methodology Committee - List of members 44 Regional variables in cross-national surveys 45 Letter to national Steering Committee members regarding sampling 46 Sample Design for the ESS - a draft specification 47 The necessity of ongoing methodological research in the ESS context 51 Survey Specification of a Proposed European Social Survey 58 1. Preface 5

In 1988, the Standing Committee International Social Survey for the Social Sciences (SCSS) of Programme (ISSP), conducted the European Science Foundation yearly by 1998 in almost 30 (ESF) had decided to sponsor, in countries around the globe and its usual à la carte mode, a research based on a loose cooperation of programme aimed at a stocktaking national research teams. As a of the political orientations of consequence, the SCSS decided to citizens in the democratic promote an effort to design an countries of Europe as they had academically driven representative developed from the 50s through survey involving, if possible, all the 80s – the Beliefs in member countries of the ESF, but Government project. The core in principle being open also for idea behind the programme was to involvement of countries beyond base this study on existing data ESF membership, especially bases because only in this way the considering countries of Central analysis of stability and change in and Eastern Europe (in addition, these orientations over a forty- at the time of writing this report, year period was feasible. the United States have expressed an interest in a linkage between The project, according to peer the US and assessments in professional the ESS). journals of the five books originating from Beliefs in In order to prepare the ground for Government (see e.g. Comparing such an ESF-tutored project, the European Politics, by Abramson SCSS, in 1995, set up a small and Inglehart, American Political expert group led by Max Kaase, Science Review, 92, March 1998, one of the two co-directors of the pp. 185-190) can be regarded as a Beliefs in Government project and success. At the same time, in regular member of the SCSS, with doing the concrete project work, it the purpose of developing some had soon become apparent that criteria for an eventual ESS,(for a there was little in terms of list of the expert group members national survey evidence which see Appendix 1.1). This expert could meet the minimal criterion group proposed in a written report of at least functional equivalence submitted to the SCSS at its April across countries, plus 1996 meeting in Paris that the longitudinality, which would have plan for an ESS should be enhanced the scope of analyses vigorously pursued and, in due beyond known data holdings like time, should be transformed into a the Eurobarometers detailed proposal to the SCSS (representative biannual surveys specifying the conceptual, conducted since 1974 in the methodological, organisational European Union member and financial details necessary to countries on behalf of the enable the SCSS to take a decision European Commission) and the in favour or against the ESS. The 6 1. Preface

SCSS accepted the expert group Committee meetings. proposal unanimously and also Furthermore, Profs. Jacques recommended that ESF approach Billiet and José Ramón Montero its member organisations for served as regular members in both participation in an ‘à la carte’ committees. funding drive for the conduct of the preparatory work for the ESS. The tasks of the Steering It can certainly be interpreted as a Committee were basically sign of the broad interest among twofold: (a) to offer general member organisations that of the guidance to the work of the 21 ESF member countries 17, plus Methodology Committee, (b) to the observer country Israel, have communicate the ESS idea to the financially supported this phase. social science community in their This is intended to prepare the respective countries and be ground for the SCSS to take a available as liaison and contact decision on the ESS in 1999. nodes between the ESS committees and their national The expert group had suggested clientele. The main task of the that designing the ESS should be Methodology Committee was to guided by two groups: (1) a develop the concrete framework Steering Committee (for a list of for all aspects needed to be members see Appendix 1.2) in considered in designing such a which all of the ESF member complex and novel survey. countries were to be represented by a senior social researcher The Steering Committee met four proposed by the pertinent national times: ESF-member, and (2) a . June 6 and 7, 1997, in Methodology Committee (for a Strasbourg; list of members see Appendix 1.3). . December 6, 1997, in Berlin; The Steering Committee was . June 8, 1998, in Istanbul; chaired by Max Kaase, . December 14, 1998, in Lisbon. Wissenschaftszentrum Berlin für Sozialforschung (WZB), Berlin, The Methodology Committee also and the Methodology Committee met four times: by Roger Jowell, Social and . October 30 and 31, 1997, in Community Planning Research Paris; (SCPR), London. In order to . March 19-20, 1998, in Paris; establish direct communication . June 14, 1998, in Berne; between the two committees, the . October 8, 1998, in Paris. chairman of the Methodology Committee regularly participated The support given by ESF as a guest in the Steering members, by the organisations Committee meetings, and the hosting the meetings, and – last chairman of the Steering but certainly not least – by the Committee in the Methodology members of both committees 7

who have enhanced the concept One of the great questions of our of the ESS through their times is what in the light of intellectual contributions, their continuing differentiation, large- time and their enthusiasm is scale migration and highly appreciated. All this internationalisation holds certainly is an encouraging contemporary nation states example for those who believe in together. Not the least, it is the the scholarly, methodological, social sciences to which politicians, practical and emotional benefits the public and individual citizens of comparative survey research in alike look for guidance in Europe. A word of sincere thanks understanding these developments is also due to the chairman Robert and for help in solving problems Erikson and members of the SCSS originating from them. Obviously, for their continuous support and this task cannot even be begun to encouragement of the ESS, and to be approached without systematic both John Smith and Geneviève continuous empirical study of the Schauinger of ESF headquarters phenomena at hand. who have been a never-ceasing source of help for all associated As with the natural sciences, also with the ESS effort. the social sciences aim for generalisations across time and space. However, as disciplines 2. The Case for a dealing with people and with the European Social interactions and interdependencies between them, Survey they must pay careful attention to the social and institutional 2.1 European diversity arrangements framing these and the social sciences interactions and interdependencies. Therefore, the Modern democratic societies have social sciences can particularly differentiated and developed into thrive on the enormous richness highly complex structures of of constitutional, institutional constitutional arrangements, and cultural variations across and, political and social institutions, to a certain extent, also within the legal regulations, mediating European nation states. These mechanisms, networks of groups variations have emerged over time and individuals. As these entities and, since they often display a interact across time and space, distinctive, concrete shape, may in they create those political, comparison at best be regarded as economic, social and cultural functional equivalents. Such structures and processes which are variability across units like nation the essence of modern western states opens up unique avenues society and, in particular, the for the kind of research, for nation state. example, which seeks to 8 2. The Case for a European Social Survey

investigate the extent to which 2.2 Helping to build a individual behaviour is shaped by European identity in cultural heritage and is social research constrained by particular social or institutional arrangements. This One of the great achievements of historically derived cultural policy makers in Europe after the diversity makes Europe a natural second world war was paving the laboratory for the social sciences way for a process of integration in the sense that, while they do while at the same time not command the important maintaining key features of the method of large scale political, social and cultural experimentation available in the identity of the European nation natural sciences, they can at least states. With the demise of analyse differences in institutions, communist rule in Central and intermediary structures and Eastern Europe, this process is individual beliefs across European now reaching out beyond the states and relate them, to take one countries of Western Europe, at example, to patterns of solutions the same time creating new for common problems found in challenges, but also new options these countries. This purpose for Europe. would be well served by all measures contributing to a more Starting in the late 60s, the thrust integrated European social science of a Europe slowly growing into community with shared goals and more than just a loosely high quality standards. connected set of nation states has also reached into the science Beyond the challenge in community, and over the years comparing European nations, new organisations of a general there is another important kind - like the European Science dimension to be considered: Foundation - or reflecting the europeanisation. As Europe moves needs of special research more and more toward some communities - like the European integrated political entity, research Consortium for Political Research on the preconditions, conduct and which links more than 200 consequences of this process will political science institutes across require increasing attention by Europe - have come into being. the social sciences and by political However, regarding empirical actors. For instance, in political comparative research the situation science the internationalisation of is less satisfactory. Other than the governance has already become a European Commission and bi- major topic of research for some lateral arrangements there are still time now, with particular too few mechanisms which emphasis on legitimacy beliefs permit a coordinated approach of and on contrasting concepts of research funding agencies to deal personal identity construction. with multinational research 9

proposals. Also, many scholars are 2.3 The data problem only slowly alerted to the benefits of studying other than their own The social sciences in Europe have country, and the growing interest a long tradition in empirical in comparative research is analysis. Not the least because of unfortunately hampered by the the exodus of many outstanding fact that the necessary data are so social scientists from Europe to difficult to come by. the US motivated by the rise of fascism in Germany and Italy in the This background is important in 30s, empirical research especially as order to understand why the idea survey research began to blossom in of setting up a European Social the US already in the 40s (Paul F. Survey has met with so much Lazarsfeld, the Austrian sociologist enthusiasm by those who over the who became a co-founder of the last four to five years have been Columbia school of social research, actively involved in getting this is a telling case in point). Since after project off the ground. Some the end of the war European social researchers have drawn an analogy science began to catch up in this with the space shuttle which respect from the 50s onwards, at permits a wide range of present a general scarcity of data experiments from different as such can no longer be disciplines for many different diagnosed. However, the element research groups. Maybe it is this of European diversity emphasised particular feature of the ESS - its above as a major asset, at this point potential to answer a broad variety turns into a problem because most of new research questions - which of the empirical research has already attracted the interest undertaken in the pertinent and support of scholars for the disciplines is not of an ESS from disciplines outside the internationally comparative social sciences. nature. This is in part true because many, probably most, researchers In particular, the bottom up are not cross-nationally oriented elements in the design of the ESS to start with. Often, however, (see sections 4 and 7 for details) internationally comparative are hoped to activate research research either does not even get networks across Europe and, in off the ground or remains weak the long run, stimulate the because it is so cumbersome and interest of young researchers in costly to organize a comparative the substance and methodology research project, and because the of comparative survey research. necessary comparative data are not readily available at all or are available in such a divergent fashion across countries that the basis for comparison becomes extremely fragile. 10 3. Designing the European Social Survey: general principles and concrete considerations

Of course, there is a wealth of studies need to be designed in a public micro data (data on small way that they can be used by a units, usually individuals, but also broad range of researchers for a higher-level units like variety of research purposes households) which are regularly originating from demands from collected by statistical offices or the science system as well as from other administrative actors. These political actors. At present, in data, however, are frequently not Europe no data base exists which comparable across nations because meets all the above requirements, of the lack of functionally and this is why the plan for a equivalent measurement systems. regular European Social Survey is In addition, they are often not put forward. It aims at a clientele freely accessible (e.g. anonymized in a broad scope of social science public use files, as in the US) to disciplines: political science, researchers for reasons of sociology, social psychology, mass administrative self-interest or communication, economic because of data protection sciences, modern social history regulations. Also, while such data and social anthropology, and its are invaluable for documenting findings will be of interest to objective developments and can scholars, politicians and the public serve – as in the case of the ESS – alike. as important contextual indicators framing individual behaviour, they do not cover the whole 3. Designing the wealth of individual orientations European Social (attitudes, beliefs and behaviours) Survey: general which are so important in understanding modern societies principles and concrete and which can be assessed by considerations survey research. 3.1 General principles As far as surveys are concerned, in many instances such information 3.1.1 No duplication of efforts is assembled as a normal part of is acceptable the research process, but tailored Since it was clear from the to individual projects and beginning that an ESS would interests. However, the social involve a large financial sciences also require surveys investment, a major task for those which are conceptually well- involved in the planning was to anchored, are conducted according assess the extent to which the ESS to rigorous methodological might simply more or less standards, take place on a regular duplicate research efforts which basis and are made available at were already underway little cost to the social science comparatively or, in a relevant community. Such general purpose fashion, in (groups of) individual 11

European countries. This collections which must be assessment was, among other mentioned here are the European things, also based upon information Household Panel Survey and the gathered from member European Labour Force Survey: organisations conducted by the these surveys with their focus on ESF Strasbourg office about socio-economic, behavioural and current projects of the ESS type. factual data do not overlap with The result of this inquiry was that the ESS. there is a lot of high-quality country-specific survey studies – There is a much closer proximity some of them on a regular basis in approach of the ESS to two like the General Social Surveys in other regular survey data Britain and Germany, the Welfare collections. The first of these, the Surveys in Sweden and Germany, biannual - sample the various national election studies of the population 15 years studies or the studies of the Social and older in the member countries and Cultural Planning Office in of the European Union (since the Netherlands and the many 1990, such surveys have also been data collections by the national conducted in varying countries of statistical offices. However, these Central and Eastern Europe) - are studies reflect specific national run on behalf of the European information needs and can hardly, Commission in Brussels. They are if at all, be used for a very useful tool for comparative internationally comparative, research in the social sciences longitudinal research. The because, after some embargo time, question, therefore, is whether they are made available for general there already exists other research use through the network of which better satisfies the criteria European data archives. However, of cross-national comparability since they are designed to meet and longitudinality. the information needs of the European Commission, they are Two types of studies can be not framed with academic mentioned here as candidates: the research in mind. This situation, first pertaining to factual socio- in fact, was a major force in demographic and socio-structural developing the blueprint for an information, and the second academically-driven ESS. assessing social, political and cultural attitudes, beliefs and The second comparative survey orientations. Examples of the to be mentioned first type are the comparative here is the International Social studies coordinated by the Survey Programme (ISSP), a self- Statistical Office of the European funded academic project presently Union (EUROSTAT) and (by the end of 1998) reaching out conducted by the national into 30 countries on all statistical offices. The two data continents. This major difference 12 3. Designing the European Social Survey: general principles and concrete considerations

in geographical scope between the were rare and precious, and it was ISSP and the ESS, which will be not regarded as a major confined to Europe, already makes shortcoming that they could say the ESS distinct. But there are other little about medium- to long- important differences as well. range processes of social, political Notably while the ESS will be a and cultural change. Instead, the one hour stand-alone survey, the emphasis was on structure and at ISSP in most of its member best on the study of short-term countries is a 15 minute change in individual attitudes and supplement to various national behaviours through the panel General Social Surveys and is method – a series of interviews conducted in the participating stretched along the time countries in a variety of data dimension with the same collection modes from a self- respondents. Here, for instance completion questionnaire after the innovative six-wave panel the “normal” survey to study of the 1940 US presidential independent mail surveys. election by Lazarsfeld, Berelson and Gaudet is still counted as a There is a further periodic major breakthrough. comparative survey (the European Values Survey) but it takes place However, the more widespread only about every ten years and and frequent survey research concentrates on religious and became and its methodology and moral values. practices better understood, the more the social science Thus, in sum, it can be stated that community began to discover the at present there is no comparative, potential of repeated surveys also long-range survey in existence in for the study of macroscopic Europe, which in its conceptual change. Both the growing foundation, methodological rigour numbers of surveys devoted to as well as innovation and user the idea of exactly replicating potential approaches the planned previous studies, and the ESS. establishment of data archives founded for the purpose of This said, it should also be pointed collecting, documenting and out that the ESS usually will not disseminating existing data be able to replace other already holdings and thereby opening up existing surveys which have been the field for secondary analysis, developed in different, often have paved the way for the study national, contexts and serve of long-term change based on different purposes and clienteles. surveys. The Beliefs in Government project funded by 3.1.2. Doing this research is not just a one-time affair the ESF between 1989 and 1994 epitomizes the analytical potential In the 40s and 50s, the early days of this approach. of survey research, such studies 13

For the time being, the panel This rationale, it was felt, holds its method and the method of ground at least until some waves accumulating random samples of of the ESS have been carried out independent cross sections of the in the independent cross section same population over time are still mode and enough expertise has the two most commonly used been assembled to assess whether a methods for studying change; change to a panel mode is feasible both can be regarded as variants of and justified in terms of the longitudinal research. However, additional resources necessary and the two approaches differ in the in relation to its larger analytic methodology, in the kind of potential. With the particular research questions to be answered emphasis of the ESS on the study and in the resources necessary to of mid- to long-term change, and implement them. The in terms of parsimony and cost Methodology Committee has effectiveness, the method of discussed at length the option of interviewing independent cross running the ESS as a ( household ) sections across time is the one best panel study, i.e. a study where one suited for the ESS, at least in its or all members of a selected initial phase. This does not, of household of a certain starting course, exclude the selective use age are continually interviewed of panel elements especially for over time, like in the British methodological purposes. It must Household Panel Study which be emphasized that the logic of began in 1991, or the German this approach implies by necessity Socio-economic Panel which that a decision for an ESS should started in 1984. While this kind assume the principal of a panel approach is clearly more understanding that the ESS will powerful in the options it offers realise its full potential as more for analysis, it is also extremely and more waves of this survey complex and requires, as the panel will accumulate. progresses over the years, large and increasing resources for 3.1.3 Comparability must be administration, field work and based on equivalence particularly data handling. The A major problem to be Methodology Committee felt, encountered in cross-national therefore, and the Steering comparative research (both cross- Committee agreed, that the sectional and longitudinal) is the complexity of the ESS as a translation of terms and concepts. comparative survey in terms of Even seemingly straightforward methodological sophistication, the number of countries and need translations of single words for consistent administration provide complications due to across countries is already so high different cultural meanings of that a decision in favour of a these words. For identifying panel approach would have semantic errors in translations jeopardized the whole project. several variants of back- 14 3. Designing the European Social Survey: general principles and concrete considerations

translation procedures can be used (1970: 108): “An instrument is aiming at lexical equivalence or equivalent across systems to the linguistic equivalence. Yet even extent that the results provided by excellent translations would only the instrument reliably describe imply literal equivalence and with (nearly) the same validity a provide no guarantee that the particular phenomenon in terms used are equivalent in different social systems.” different languages. Excellent translations and back-translations In order to assess the degree are the starting point – not the comparability the idea of solution for developing functional equivalence is crucial: comparative measures. similar concepts should be related to other concepts in different If the same indicators are used for settings in more or less the same identical theoretical concepts, way. Functional equivalence is special efforts are due to control based on the notion that for different types of bias: comparability is not an attribute construct bias, method bias, and of single terms or concepts, but an item bias. With culture- or attribute of the relationships nation-specific indicators of among these terms or concepts. concepts there are problems Therefore, comparability can only tracing cross-cultural or cross- be obtained if auxiliary national differences. If – on the information is available from the other hand – identical very beginning of the instruments are constructed for development of specific measures. various settings, it is unlikely that one obtains an appropriate For the ESS the requirement to observation of national or establish cross-national, cross- cultural aspects and differences. cultural, and longitudinal For the establishment of equivalence results in an emphasis equivalence a differentiation on theory driven research (see between the stimuli actually Section 4.2) and a clear applied and the concepts behind specification of the meaning of these measures is required. the core concepts and the use of Because comparisons should be context-specific stimuli for based on similar concepts in different contexts. This implies – different settings, the measures in turn – that translation and should be equivalent and not back-translation procedures identical. Most of the time this should be expanded with implies that the stimuli are extensive consultation of both different in specific national or expert researchers and area cultural settings. In the words of specialists. Differentiation of the Przeworski and Teune, in their actual questionnaires in different seminal work “The Logic of countries and attuning questions Comparative Social Inquiry” to specific contexts (that is, using 15

different stimuli in different theory building in the social contexts if necessary) is the only sciences. One pertinent example is way to maximise comparability of the most similar systems design results obtained with the ESS. (a concept developed by Needless to say that the Przeworski and Teune). The basic equivalence problem implies the logic of this design is that in need for extensive pretesting and reasonably similar societies – like very thorough methodological is the case in Europe – it is work (see Section 5 and Appendix possible to study the impact of 4) as well as context-sensitive variations in the institutional documentation. (macro)structure of societies on individual behaviour in the 3.1.4 Multi-level analyses must countries which are part of the ensue . Thus, individual Just looking at a set of marginals orientations must no longer only and discovering that countries be explained by other variables substantially differ in their mean also measured on the level of position on – to take an example – individuals. Rather, the satisfaction with the way explanatory scope in multi-level democracy works in the respective analysis is extended to the social countries immediately raises the and institutional environment, in question about where these which individuals are embedded, differences come from. There may thereby considerably extending be a whole wealth of factors the analytical scope of survey influencing an individual’s research. position on such a measure within a country (like whether the party As a consequence, right from the he or she identifies with is in or beginning the ESS must take all out of government), but this can necessary steps to ascertain that as hardly account for differences in many as conceptually meaningful the mean level of democratic institutional and contextual satisfaction between countries. In variables are added to the this situation the search for individual data and that links to institutional, macro properties of data holdings of e.g. EUROSTAT the systems under scrutiny comes are included which permit the into the fore, or – in social science integration of individual and terminology – the need for multi- contextual information (for level analysis for the explanation details see Appendix 2). Similarly, of country differences (the same political and economic macro argument pertains, of course, indicators on the level of nations when it comes to social instead of must be included early on in the political attitudes). micro data files.

Multi-level analyses are an important instrument of complex 16 3. Designing the European Social Survey: general principles and concrete considerations

3.1.5 Continuous interviews (different question methodological research is format and wordings, stop and essential think manipulations, source There was agreement in both ESS manipulations, counter-argument committees that an innovative, techniques, etc.). high quality survey like the ESS not only opens up a unique In addition, the ESS will become a opportunity for methodological new, indispensable instrument in research, but requires, in order to the training of social researchers remain on a high level, systematic across Europe not only in the accompanying methodological substance but also in the scrutiny before the data are made methodology of comparative available. research. It should thus help to improve the methodological The scope of this part of the ESS sophistication of social scientists is wide: it involves detailed studies within their national contexts. of sampling frames and their effects on , 3.1.6 Flexibility and stability cognitive tests of the core must be amalgamated concepts and their Given the long-range perspective, operationalisation, tests of the the overall magnitude and the quality of measurement need for success for the ESS, there instruments within and across is a temptation to think too countries, construction of scales quickly about a big, stable and indices which are equivalent infrastructure for the ESS. Both across countries, test-retest and ESS committees therefore from Multi-Trait-Multi-Method the beginning were aware of the (MTMM) studies assessing the need to find a good balance validity and reliability of between stable and flexible questions, translation studies, and elements in the organisation of the construction of functionally the ESS. The two committees equivalent indicators for believe that the concrete proposal comparison where face for setting up the survey reflects equivalence is not available (like in a good way the perceived with regard to educational tension and benefits between the systems) (for details see Section two principles. 5). In addition, and in anticipation of technology-induced new Firstly, it is proposed that – developments in survey research assuming the survey will be (see also Section 3.2.2) conducted every second year as is methodological innovations must suggested – two waves of data be anticipated, taking into collection are initially decided account recent experimentations upon and that once the second in public opinion research made wave has been completed, ESF possible by computer-assisted (SCSS) or any other appropriate 17

body conducts an assessment any measure counteracting this review of ESS as a basis for a goal should be avoided; (2) under decision on its continuation or conditions of almost (at least in termination. academia) ever present access to electronic networks, distribution Secondly, a flexible organisation is controls would be close to envisaged, where the committee impossible to implement. structure which has proved useful in preparing this document is 3.2 Population and maintained and a bottom-up sample philosophy is to be implemented in designing and conducting the Europe within and beyond the individual national surveys. context of the European Union is Elements of stability will be a becoming increasingly integrated small permanent staff, a economically, politically and – to a temporarily fixed attachment of certain degree – even culturally, the permanent staff to a and because of its wealth and resourceful academic research political freedom it appears more institution and the selection of a and more attractive to people well-established data archive for from less advantaged parts of the all tasks involving the production, world. As one example of documentation and dissemination integration, citizens from those of the comparative data set (for EU countries which are part of details of the proposed the Schengen agreement can now organisation see Section 7). cross national borders without even being questioned about their 3.1.7 Data access must be easy national identity. As a and inexpensive consequence of this increasing The expected high cost of the ESS permeability and also of more and has given rise to debates whether more people from non-EU the data should be embargoed in countries living there without some way and should be made official acknowledgement, the available, for example to interested question of the definition of the researchers from non- population from which to sample participating countries, only at a for the ESS is getting much more substantial price to recover some complex than it would have been of the ESS cost. There were two in the past. In addition, since the main reasons why both ESS ESS has – if successful – a time committees advise against perspective which reaches quite a charging any but the most bit into the 21st century, the core marginal cost (for producing and definition of the population from sending a CD-ROM data set, if which to sample has to be such applicable): (1) if the philosophy that it can carry that prospective behind the ESS is to enhance weight. social research in Europe, then 18 3. Designing the European Social Survey: general principles and concrete considerations

Related to this, but with a the population to be sampled and substantial quality of its own is the acceptance of what would the question of the sampling constitute equivalent sampling procedure to be implemented frames. across the ESS-participating countries. Here, the need to 3.2.1 Population defined implement an equivalent Definition: The survey will cover sampling frame is evident. persons 15 years and older, with no upper age limit, who are resident in In some countries, registers of the country regardless of persons, CD ROM telephone nationality, citizenship or legal directories or voter registers are status. available from which national samples are routinely and It must be understood that this is effectively drawn. Key questions, a rather consequential definition however, will be whether such for a variety of reasons. First, one registers will satisfy the criteria needs to consider the age limit of for the defined population 15 years. In most countries, for completely, or at least reasonably instance, the age limit for voting well, even under conditions of is still 18 years (although in some growing migration and a lowering of this limit is under immigration and what acceptable debate). In starting at the age of alternative sampling frames are 15, easy sampling from voter available. These are difficult registers will not be possible. Still, problems which have taken up a the committees agreed that under large part of the debates in the socialisation theory perspective it two ESS committees. Regarding is necessary to set the lower age the definition of the population limit. as well as the appropriate sampling design, the conclusion was that it Second, the concept of resident is very important to set inclusive populations ALSO includes persons high quality criteria, that will living in institutions like old age make the ESS distinct from other homes, university or school surveys. However, it is recognised housing, prisons, hospitals and the that under resource constraints military as well as the so-called some minor compromises may be “fluid” parts of the population necessary. This is tolerable for the like the homeless. These are purpose of the ESS as a general groups which are often not social survey provided that the included in current national compromises are regulated, sampling designs although in documented and evaluated. For principle they belong there. On example, if the survey is to be a the other hand, some of these practical proposition, there are groups of persons are generally certain to remain some small not available to survey research for ambiguities in the definition of various reasons, and it was felt 19

that setting up an “ideal” The most important element of population concept would in the the population definition end make the conduct of the ESS proposed for the ESS is the unmanageable. For instance, inclusion of non-national people in inaccessible institutions residents. There was absolute like prisons and the military are to agreement in the two committees be excluded from the population that the kind of emerging Europe definition. (Some flexibility in briefly addressed above (Section this matter, though, may have to 3.2) requires to include from the be built in the scheme to allow for start non-nationals who reside in country-specific problems which an address in the country in cannot be anticipated at this question. The biggest difficulty point; Israel may here be a special which the ESS will encounter in case because of the number of the this respect is a language problem, people in military service.) and this in two ways: (1) the language of the interviewee, and Regarding such inaccessible parts (2) the language of the of the population, one has to keep interviewer. Here, again, in mind that in most of the parsimony is a guiding light. countries the above mentioned Obviously, in case a non-national groups constitute only a small resident speaks and understands part of the population and would the language of the country of therefore, even if they could be residence, no problem ensues. sampled, be represented in the Otherwise, in countries where a surveys in numbers so small as not minority language is spoken as a to be fit for separate quantitative first language by 5 % or more of analysis. There was therefore the total population, the agreement in the committees that questionnaire will be translated such groups need not be part of into that language too, and the statistically defined ESS suitable interviewers will be population from which the trained to administer it. sample will be drawn. However, if one considers specifically that Given the fact that – should the quite a few countries in the expected number of countries coming decades will experience a participate in the ESS – a substantial ageing of their substantial number of tested populations, then the inclusion language versions will be available into the sampling frame of people (hopefully in the Computer living in old age homes will from Assisted Personal Interview the beginning have to be given (CAPI)-mode), even in cases serious consideration. The same where the 5 % minority language applies to people who spend threshold is not reached, one can extended time in educational still expect that in quite a few institutions and live there. instances such individuals can nevertheless be included in the 20 3. Designing the European Social Survey: general principles and concrete considerations

survey; more can be said about this full coverage mark, the major problem only after the first round consideration in favour of face- of the ESS is concluded. to-face personal interviews as the preferred mode for the ESS 3.2.2 Sample defined was that the ESS, in order to reach Definition: The sample is to be a good balance between cost and selected by strict random substantive scope, has to include probability methods at every stage. at least two topical modules, in The relative selection probabilities addition to the two core standard of every sample member must be demographic and attitudinal known and recorded on the data set. questions, and preferable one Quota sampling must not be used at additional module to be covered in any stage. the self-completion mode (for details see Section 4). The Methodology Committee has prepared this recommendation Here, the respondent himself/ based on a survey in all countries herself either in the presence of represented in the ESS steering the interviewer or at a later point Committee (see Appendix 3.1). of his/her liking - in the form of The result was that random a drop off questionnaire - answers sampling, while not equally the questions in the third module present in all countries as a and the questions posed for routine procedure, was a sampling methodological purposes. Such an method which can be approach is not possible in regular implemented without major telephone interviews which as a problems in all countries. At the standard research routine usually same time, it has become clear may not last for much more than that sampling modes across 30 minutes before the respondent countries will at best be quits. functionally equivalent in the sense that the criteria in the above Obviously, new developments in definition are met. electronic networking such as interviewing through the Internet One point of major consideration may in the future require a was the mode of interviewing, change in the present decision in given the fact that in many favour of face-to-face personal European countries almost half interviews, but this appears of the surveys (with an emphasis, premature in the case of the ESS however, on commercial surveys) for about the next decade. are by now already conducted through the telephone. In A very important and cost- addition to the fact that not in all relevant decision refers to the European countries alike number of interviews per country telephone coverage of private in the ESS. In general social households has come close to the surveys, it has been argued that 21

their multi-purpose character as the design actually used). This makes it advisable to have a larger effect is commonly caused by the number of respondents than in use of different selection single-purpose surveys because probabilities in household samples research interests are more and by geographical clustering divergent and statistical analysis (for details see Appendix 3.2). of groups less prevalent in the population becomes possible (this Obviously, this has to be taken is why the German ALLBUS into account when determining between 1980 and 1990 has had an the size of the national samples. average of about 3000 Since, on the other hand, survey respondents). While such a large cost is such an important factor, number may be desirable, this the Methodology Committee felt philosophy is not shared that some flexibility for the everywhere, and therefore there individual countries regarding was no agreement on a specific sample size is in order. It therefore number in the two ESS proposes a recommended sample committees regarding the optimal size of 2500 and a minimum of sampling size (for those who are 2000 not considering design not statistical experts it may be effects. However, it strongly worthwhile to point to the fact endorses the requirement that the that in terms of standard errors effective sample size must not and confidence limits there is fall below 1500 interviews. practically no relationship between the absolute size of the In order to properly calculate the population and the size of the effective sample size and the sample necessary for a specific set necessary statistical estimated of desired confidence limits). (standard errors), each step of the selection process will have to be The most important point documented in detail by the originating from the discussion in national surveying organisation the Methodology Committee and the supervising team of the regarding sampling and sample principal investigator. This is also size was the observation - for non- necessary for the computation of sampling experts somewhat the completion rate, that is the surprising - that different share of successfully conducted sampling designs have a interviews in relation to the initial considerable impact on standard sample of valid addresses. Our errors, independently of sample target response rate must size. In other words, the effective therefore be high, and the sample size can be considerably Methodology Committee has set smaller than the actual sample size it at a minimum of 75 % of (effective sample size is the size of eligible sample members. We a which realise that this target level might produces the same standard errors not in the end be attained in all 22 3. Designing the European Social Survey: general principles and concrete considerations

countries, but believe it is 3.4 Participating appropriate to aim as high as countries possible in order to lift the lowest response rates higher and not to At this point it is not yet clear depress the highest ones. Response which countries will decide to rates cannot of course be participate in the first wave of the legislated for, but they can be ESS should the SCSS recommend heavily influenced by insisting – its start. While the interest of ESF as the specification does (see member countries to get involved Appendix 5) – on certain in the design of the blue print has fieldwork procedures that been very high, this cannot and maximise the chances of finding does not indicate that all will elusive sample members. automatically join the ESS. Therefore, it must be clear from 3.3 Timing the beginning that a more or less full participation of all ESF It would have been attractive for member countries cannot be a many reasons, in particular for the conditio sine qua non for the ESS chance to obtain a wider topical to begin. On the other hand, all coverage, to propose doing the efforts should be made to ESS every year. Since the purpose ascertain as wide a participation as of the ESS is mainly to study possible. The ESS deals, in essence, mid- to long range change, the with the social and political health practical arguments of cost and of the nations of Europe. Every major increase in necessary additional country which joins research infrastructure for a one- will increase the theoretical and year rhythm resulted in the practical input into the project unanimous recommendation by and will enhance its analytical both ESS committees to plan the power and usefulness, not to speak ESS, at least in the beginning, of the potential for such that it will be conducted methodological innovation for every second year. Its start is the national research envisaged for the year 2001 in communities and for student order to design and prepare the training. By contrast, for each study in the necessary precision non-joining country a decision and quality. This timing will, of not to participate will be costly in course, depend on the speed by terms of not being involved in the which the responsible bodies will detailed planning of the work, in take their decision on the ESS. missing out on the opportunities for the study of change and in not being able to locate its position in relation to the other, participating countries. 23

4. Selection of Themes knowledge, but at the same time for the European flexible enough to cover new theoretical and practical ground. Social Survey Besides, the ESS will have to meet the highest methodological 4.1 Research focus standards of the profession. On the basis of these considerations From the very beginning of the ESS structure as well as the planning the ESS, it was clear that main criteria for the selection of its research focus should further topics can be derived. the development of European comparative social research. 4.2 Core sets and Neither specific national interests module topics nor current topics or issues can be the main concern of ESS. Instead, Each wave of the ESS should the focus has to be the systematic basically consist of three parts: (1) study of European citizens’ a first core set of questions for attitudes, attributes, and the observation of change and behaviour relating to a core set of persistence in attitudes, (2) a economically, socially and similar second core set for social politically relevant societal and demographic attributes, and domains. The general theme of (3) modules for specific topics ESS, then, is the study of (including space for distributions, differences and methodological testing). For the changes in the social, political, first two parts about 15 minutes and cultural beliefs and of interview time should be behaviours of Europeans across available. For each of the two time and countries as well as substantive, topical modules 10- the explanation of these 12 minutes of interview time will differences and changes. be set aside, amounting to a total interview length of about 55 Structure and selection of minutes. In order to increase the substantive topics for the ESS power and efficiency of the ESS, should therefore reflect this the option of a third module emphasis on the need for should be pursued in the form of European comparative and a self-completion questionnaire; longitudinal research in the social here the necessary methodological sciences. It will not be just research could find its main place. another public opinion survey concerned with specific current The two core sets establish the — or even fashionable — themes, continuous part of the ESS and but a systematically designed and should provide the opportunity developed instrument to enable for testing and developing and to stimulate innovative dynamic approaches to the study research on the basis of existing of the social, political, and 24 4. Selection of Themes for the European Social Survey

cultural beliefs and behaviours of instruments. They will require Europeans. Obviously, the actual meticulous pretesting and questionnaire does not have to methodological scrutiny. show the envisaged divisions — Broadly speaking, four main types only for the two core sets a fixed of empirical evidence can be design of the question order can discerned: be considered. – evidence obtained from instruments which have been The selection of topics for each of applied in comparative research the three parts should be based on already (for instance in the the following procedural rules or Eurobarometer, World Values criteria. It should be Survey, ISSP, or Political Action); . theory driven. Each – evidence obtained in one or instrument used in the ESS more representative surveys in should be designed and developed specific regions or countries (for on the basis of sound theoretical instance in national general social arguments. This implies the use surveys or national election and/or modification of existing studies); approaches as well as a rejection of – evidence obtained on the basis instruments applied for of systematic pre-testing of new explorative research or inductive instruments or experiments; reasoning. The ESS should be – evidence to be obtained for directed at the needs of those innovative new approaches to be members of the scientific used in ESS for the first time. For community searching for these innovations the ESS should explanations of (changes in) offer the opportunity to develop social and political beliefs of new instruments and facilities; Europeans from a comparative this must be closely connected to perspective. the methodological testing of . based on empirical evidence. regular ESS-instruments. A Instruments used in the ESS procedure to develop and test new should in general have proven instruments similar to the their usefulness in empirical practices used for the American research. This implies that National Election Study might be instruments which already have designed for these innovations. been applied in empirical research . relevant for studying change. (although not necessarily in an Each instrument used in ESS international comparative setting) should be clearly relevant for and have shown substantive monitoring and analysing the results are especially promising dynamics of social, cultural, and candidates for inclusion. To avoid political beliefs and behaviours of a conservative bias, opportunities Europeans. For the core set this must be provided for innovative requirement is self-evident; for topics and questions in addition to the module it implies that the the use of established selection of themes should not be 25

directed at current issues or topics, economic and demographic but instead focus on specific nature (like age, education, social phenomena which do not require class, residence, income). continuous monitoring. . social position and networks. Obviously, this does not imply These areas include all properties that at the operational level which indicate the position each references to current issues or European has in his or her social fashionable topics should be (or environment (like family structure, can be) avoided. social participation, mobility). . social context and environment. 4.2.1 Core sets These areas include all properties Along the principal lines of the of the societies in which European three procedural rules or criteria citizens live (like the degree of mentioned above, the main areas social inequality, welfare state to be covered by the two core sets provisions, public spending, crime of questions have to be specified. rates, or party competition). The usual distinction between ‘attitudes’ on the one hand and Secondly, from a different ‘background variables’ on the perspective, when the core set of other does not seem to be a very questions is determined, the promising starting point here. dimensions which need to be Besides, any a priori selection of a considered systematically are the specific theoretical approach location of the individual would immediately threaten the respondents in the social requirement of designing the ESS structure, the political structure as an infra-structural ‘tool’ and in the cultural sphere of the providing information for a wide society in question. variety of scholars. From these considerations follows that the 4.2.2 Module topics selection of topics for the core set The selection of module topics in should start with a rough the ESS should take place on the identification of the main areas to basis of the expectation that be covered. Firstly, these can be including these topics in a summarised in the following comparative setting will bring way: evident gains for substantive research. Therefore, these topics . individual attitudes and should not reflect actual issues, attributes. These areas include all but rather cover those topics properties which can be attributed which are relevant for to each European citizen, both of understanding the social, political, an attitudinal and behavioural and cultural beliefs and nature (like trust, satisfaction, behaviours of Europeans without expectations, alienation, cultural the need to collect this preferences, political information in each wave of the participation) and of a socio- ESS. Researchers should be 26 4. Selection of Themes for the European Social Survey

invited to submit proposals for 4.3 Choice of topics topics to be included according to the procedural rules mentioned 4.3.1 Procedural rules and below (see also Section 7). In this criteria way, the ESS can be a facility open Even a first selection of topics for to the scientific community, inclusion in the core set and as demonstrate best practice additional modules presents standards and stimulate new substantial problems if the above developments in the social mentioned general criteria are sciences. used. For each candidate set of questions, one must collect In addition to the inclusion of information and evaluate the these substantive topics, the empirical evidence based on a modules should be used for matrix of the three procedural methodological testing of the rules or criteria and the main areas indicators included in the core set to be covered. This results in the (like the use of different variants following check-list for the or variations in question wording) decision which questions to as well as offer the opportunity for include in the ESS. testing new instruments with clearly innovative aspects.

Main Areas to be Covered: Procedural Rules or Criteria: 1. Theory Driven 2. Empirical Evidence 3. Relevance for Change

Core Set: 1. Individual Attitudes and Attributes - . Theme (a) xx x . Theme (..) xx x

2. Social Position and Networks . Theme (k) xx x . Theme (..) xx x

3. Social Context and Environment . Theme (v) xx x . Theme (..) xx x

Module Topics: . Topic (A) xx x . Topic (B) xx x . Topic (C) xx x 27

4.3.2 Examples of questions . Social position and networks for the two core sets Family structure; involvement in Obviously, a listing of the voluntary associations, interest questions for the two core sets and groups, and church related also for the module topics will organisations; informal networks, have to be left to the next work professional contacts, etc. stages in preparing the ESS. For . Social context and environment the core, typical topics may Social and political embeddedness, include: economic development, organisational structure (civil . Individual attitudes and society), etc. attributes – issues and problems (national 4.3.3 Selection and examples problems like crime, inequality, of module topics unemployment, public spending, Both ESS committees strongly etc.); believe that the most important – orientations towards democracy function of the ESS is to promote (satisfaction, political trust, comparative social research in confidence in institutions, left- Europe and to create a widespread right self-placement, etc.); sense of community and – media usage/exposure and involvement among European communication (reading social researchers. Therefore, the newspapers and magazines, most important asset in the ESS watching tv, listening to the radio, context is its potential to activate internet and multimedia usage); this engagement through the – political involvement (political opportunity to play a major role in interest, political participation, the selection and development of voting behaviour, etc.); the module topics. This must be – social and political orientations achieved by an open competition (individualisation, victimisation, for the best proposals. postmaterialism, etc.); – socio-political identity Further on in the development of (national identity, ethnocentrism, the ESS, the Steering Committee xenophobia, patriotism, etc.); will design the procedures most – leisure activities, life style appropriate for safeguarding that issues, etc. from such a competition the – Sex/gender, nationality, birth, theoretically most interesting and family situation, housing/ methodologically best designed residence, social class, education, proposals will be selected (see also employment/unemployment, Section 7). Therefore, the Steering religion, occupation, household, Committee decided not to develop personal income, household further in detail the paradigmatic income, number of people in proposal for module topics by the household, etc. Methodology Committee. However, for the first ESS wave, 28 5. Methodological Research

in the light of the large amount 5. Methodological of organisational and scientific Research work to be done in the beginning phase of the ESS, the Steering A large and innovative venture Committee has expressed its like the ESS requires a lot of willingness, if so required, to take methodological and practical on the responsibility for selecting quality control. This refers first to both the 2-3 topic modules and the design of the questionnaires the cross-national groups of and the necessary work with scholars to entrust with the job of respect to translation and back- conceptualizing them. translation, and second to all aspects of sampling in order to However, in order to convey at guarantee the quality of the least a flavour of what might be national and comparative studies. covered, and without prejudice, the Steering Committee offers the 5.1 Questions and following topics from the three question wording realms of the social structure, the political structure and the cultural The Methodology Committee has sphere. They are no more than suggested that in three countries examples. large scale pilot studies will be done which allow serious . Social structure statistical analysis. Furthermore, – families, primary groups, and it was suggested that in all social networks; countries in the main study drop – mobility, immigration and off questionnaires will be used for multi-culturalism; methodological research. This – social inclusion and inequality; allows for the following studies (for details see Appendix 4): . Political structure – civil society and trust; . before the pilot studies – democracy and political cognitive studies of the most involvement; important concepts in the – interest groups and political questionnaires should be done so parties; that one can be sure that the questions are understood and . Cultural sphere answered as they were expected to – life styles and life course, be answered. Such studies should including life-long education; concentrate on basic terms like – media and the knowledge social security , the welfare state society; or democracy . It is probably best – subnational, national and to perform such checks in a place transnational identities. where different language forms can be tested and have been tested 29

before so that comparisons can be measures to determine the made. The Dutch Statistical strength of the opinions of Office (CBS) performs such tests people. Differences in for European studies; sophistication can cause . during such a cognitive study considerable differences between at least 30 interviews should be countries with respect to taped for an interaction analysis in correlations. Such differences order to see if there are serious might not exist between people problems with the data collection; with equal knowledge or . predictions have to be made sophistication with respect to the which questions of the issue. One possibility is to ask reformulated questionnaires will extra questions concerning the cause most serious problems with knowledge about the issue at respect to reliability and validity. hand. An alternative would be to These predictions can be done on provide alternative framings or the basis of previous studies. For counter-arguments for different those questions which are most opinions. In doing so one can problematic, alternative determine the strength of the formulations will be tested in the opinions and compare people with quantitative pilot studies by a equal strength of opinions; combination of split ballot . after the data have been experiments and MTMM (multi- collected, the most relevant trait-multi-method) studies in variables have to be screened order to detect strong method looking for unexpected results effects and large random errors; with respect to different . even though these tests will background variables and across improve the quality of the countries. The data should only be measures, method effects, made publicly available after this measurement errors and response screening is done and the most differences between the different serious problems have been solved countries are unavoidable. The or documented. drop off questionnaires in the different countries allow the 5.2 Sampling and non- estimation of response response on the ESS probabilities and reliability of the most important questions which It is important that the ESS might vary from country to should routinely monitor and country and in this way allow for assess the implementation and correction of these errors. How effect of the sampling design and this can be done should be the effect of non-response. This provided in the documentation of will aid interpretation of the data the data when they are made and will ensure that high available for the users; standards and comparability are . the pilot and/or drop off maintained. In addition to this studies should also contain routine assessment, the 30 5. Methodological Research

opportunity should be grasped to address, identification number of carry out original methodological selected person, etc); research designed to extend . non-response quality control: knowledge (and, ultimately, to For each country, an analysis of improve practice) about the effects both the response outcomes and of non-response and the role of the response effort (number of the interview in this process: visits made, by days of weeks and times of day, etc) for all sample . Sample design quality control: cases will be carried out and The form of sampling required published because this is essential for the ESS is likely to represent for assessing the quality of the an increase in quality above usual sample obtained. This analysis will practice for many countries. In also incorporate covariates such as some countries, some components area characteristics and of the sampling method may be interviewer-observed items. To completely novel. The enable this to be achieved, the introduction of new and survey specification will stipulate improved methods should be seen that the data set must include as a major potential achievement detailed contact and outcome of the survey. However, it is information for all sample cases essential to assess the success of (respondents and non- the methods employed. First, respondents); analysis of the quality of . analysis of the effectiveness of implementation of the sample response elicitation effort: This is design must be undertaken. currently a very important and Second, analysis of the impact of much-debated topic amongst the sample design on analysis survey organisations. In recent must be undertaken in a years, the effort required to systematic way across countries. maintain response rates on social Specifically, design effects and surveys appears to have increased their components (due to dramatically in many countries. stratification, clustering, The measures employed to differential selection probabilities maximise response vary across etc.) should be estimated and countries and contexts, and published for a range of variables relatively little is known about the and estimates. To enable this to be effectiveness of response achieved, the survey specification elicitation efforts. The ESS will will stipulate that the data set provide the opportunity for recent must include indicators of PSU work carried out in the UK and (primary sampling unit) and USA to be extended and applied to stratum membership for every a cross-national comparative sample case, selection probability, context. The work would consist and records of any interviewer- of relating interviewer calling administered selection procedure patterns and effort to non- (i.e. number of persons resident at response bias in order to help 31

identify which parts of the demographic and experience response elicitation process are information), using the same most (or least) effective at anonymised number. minimising survey errors. The chance to systematically identify cross-national differences in these 6. Data Management, relationships adds further to the Archiving and value of this research. To enable this to be achieved, the survey Distribution specification will stipulate that In due time, through open tender the data set must include, for each or other procedures, a decision will call on each sample case have to be taken which of the (respondents and non- existing data archives in Europe respondents), an outcome code, shall become the ESS archive. and the date and time of the call; This choice will have to be made . analysis of the role of according the specifications in interviewer characteristics: Some this section of the report. At this important research into the role point, it can remain an open of interviewers in obtaining question whether one archive or a survey response, and the ways in set of cooperating archives will be which interviewers affect survey the best solution. In the following error, has been carried out in text only the singular is used as a recent years. The ESS offers the matter of practicality. opportunity to extend and enhance this work in some Rigorous standards must be important areas. For example, defined to ascertain compatibility there is much that could be of the ESS national data sets for learned from analysis of the integration into a common core relationships between interviewer for international comparison. The characteristics and survey errors national data sets should be (particularly unit and item non- checked, cleaned and documented response bias and interviewer- by the survey organisation after correlated variance), which could the completion of fieldwork and be carried out and compared across then be sent to the ESS archive for countries and survey integration, documentation, organisations for the first time. To archiving and distribution of the make this analysis possible, the international ESS data set. This data set will include an requires well-structured anonymised interviewer procedures involving the Principal identification number (which is Investigators, the ESS necessary anyway to aid the data Methodological Team, the quality control procedures) as well Methodology Committee, and the as the responses to a very short ESS Archive (for the proposed ESS form to be administered to all organisation and its main bodies interviewers (collecting basic see Section 7). 32 6. Data Management, Archiving and Distribution

6.1 Integration, standardisation must be based on documentation, culture specific knowledge and validation and archiving therefore must be achieved in of the database cooperation with the Principal Investigators; This part of the process involves . The integrated ESS module the following procedures: with common core data set and codebook in printed or electronic . national data sets from form will be delivered to the participating countries are sent Methodological Team for testing within three months after and verification. This is scheduled completion of field work to the to happen no later than six month archive. The national data sets after arrival of the last data set in must be checked for plausibility, the Archive. After completion of cleaned and documented by the tests based on intensive standard team of the national principal analyses by the Methodological investigator and must be Team, the data set is returned to documented according to the Archive for final processing international standards to be and documentation. provided by the Archive; . the Archive prepares backup 6.2 Refinement of data copies of the originals of national base datasets and stores them according to archival standards; The individual microdata from . the Archive systematically the surveys then could be merged checks for completeness of data with geographically referenced and related materials (field contextual data from aggregate reports, specimen copy of statistics agreed upon by ESS and questionnaires, study to be provided by the descriptions) before integrating Methodological Team. Another the national data sets into the option may be to just supply the common core data set. This users with references in the data includes further checking, set permitting the linkage to other cleaning and the construction of existing data bases (see Section derived variables in cooperation 3.1.4 of the report and Appendix with Principal Investigators, the 2). Furthermore, the integrated Methodological Team and the data are stored into a relational Methodology Committee. data base according to Integration also requires international archival standards. standardisation of national This is to establish time series and variables for the international to support the extraction of comparative common core data variables for comparative analyses, set beyond the level of including documentation of standardisation to be followed in trends. In addition, the data base national data sets already. This management should support scale 33

and index construction which has easy retrieval and access to ESS to be based on cooperation with questionnaires and data will be the Principal Investigators and the developed on the variable and Methodological Team. Various study item level. developments are under way like ILSES (Integrated Library- and Since there are substantial Survey Data Extraction Service), differences in data protection NESSTAR (Networked European regulations across Europe, it may Social Science Tools and be necessary – like in the case of Resources) or ACCESS the German Socio-economic panel (Microsoft-based Information study (SOEP) – to make data System) which cover most of the access dependent on a contractual required functionalities. Of agreement to be signed by the data course, the integrated data set and archive and the data recipient. documentation including all related materials will be 6.4 Event data permanently stored according to archival standards. It is well known from earlier comparative survey research that 6.3 Data distribution in some fields, such as electoral analysis, individual reactions to Data distribution is organised by certain questions will be the Archive in agreement with influenced by contextual factors ESS via the Council of European and by significant events. For Social Science Data Archives example, a question about the (CESSDA) and the International subjective interest in politics of a Federation of Data Organisations respondent may well be answered for the Social Sciences (IFDO). differently at the height of a The ESS Archive implements the national campaign for a general administration of embargoes (if election compared to a time when imposed for a time period of not no election is imminent. The more than two years by principal contextual impact on individual investigators) and standard data response behaviour will not create protection measures. major difficulties for the ESS as long as the contexts and events Distribution will be offered via vary individually in an modern media, currently CD idiosyncratic fashion. The impact, ROM or via Internet, subsets of however , of a contextual factor or data may be distributed via an event must be considered and, diskette. Distribution rights rest whenever possible, controlled as with the ESS Archive and are soon as whole societies are thus executed in agreement with ESS. influenced in a way which is not Given rapid developments in data uniform across the countries in distribution and retrieval the ESS. technologies, internet services for 34 6. Data Management, Archiving and Distribution

In addition, it has to be Other surveys central to topics remembered that the ESS will in covered by ESS should be acquired the long run also become an by the ESS Archive and should be important asset for historical made available for preparation of micro analysis. As a consequence, new ESS modules and for from the beginning an comparisons within the range of information tool which for the resources the Archive can make lack of a better term may be called available for this purpose an event data inventory will have specifically. Also, list server and to be designed. This inventory discussion group functions shall must offer to the researchers a be implemented, and data flaws brief, pertinent synopsis of major detected in the process of further political, social and other data analysis by ESS groups or the potentially relevant events in the social science community at large ESS countries; this is particularly will be documented by the important for the ESS since its Archive. Finally, literature based modular approach will in the long on ESS data shall be compiled by run cover a wide area of the Methodological Team and substantive concerns. documented by the Archive.

Preferably, this inventory should begin with the starting year of the survey and must be regularly updated. A data archive is the optimal place for this because it could use the support of other data archives in the participating countries in standardizing and providing the relevant information. Details and cost will have to be specified in due time with the archive(s) involved in the ESS.

6.5 Other archival concerns

ESS supports a policy of free and easy access to its integrated dataset. Data should be made available to the scientific community basically at handling charges. Other uses are subject to agreement with ESS and the Archive. 35

7. Organisational members of the team will be Structure selected by members of the Steering Committee; It was mentioned in the . the ESS Methodological Team must be linked to an experienced beginning that in the organisation and resourceful hosting survey of the ESS a good marriage of organisation; flexible and stable elements must the Steering Committee be achieved. Undoubtedly, as was . interacts regularly with the observed in the ESS Steering Methodology Committee in all Committee deliberations from the ESS matters; it is responsible for start, an enterprise of the size and all general dealings with the the continuity of the ESS cannot Principal Investigators and the be mustered without a stable Assembly of Principal support structure. On the other Investigators. In particular, it will hand, since the survey is be responsible for selecting the conceptualized as one for the themes for the ESS topical social science community in modules and for organizing the Europe at large (and maybe competition among European beyond), a lot of bottom-up social scientists for designing the elements must be mobilized. modules. In addition, the Steering Committee members will have The Methodology Committee has the obligation to link the ESS to spent a substantial amount of its their national communities and to time in considering the way that act as communication nodes. the demands of such a big survey Finally, the Steering Committee can be organisationally met. In will be the major link to the this, it has started from the funding organisations of the ESS following assumptions: and will have to take all general . both the Steering Committee budgetary decisions. and the Methodology Committee . the Methodology Committee will have to become Standing will be responsible for guiding the Committees in the guidance of work of the ESS Methodological the survey; Team and in particular for the . the concrete conduct of the design of the ongoing survey requires a permanent methodological research. support group (ESS Organisational/Methodological The following page contains a Team) to supervise the survey, act graphic demonstration of the core as a link between the Principal ESS organisational set up. While Investigators and the Data not all elements in that structure Archive (see Section 6) and be in will interact on the same level of regular contact with the intensity (indicated by full or Methodology Committee in all dotted arrows), it is clear that the ESS-related topics, including ESS represents and requires a methodological research; the complex network approach. 36

ESF/ESS

METHODOLOGY STEERING COMMITTEE COMMITTEE

SUBJECT SPECIALISTS

...... ARCHIVES ASSEMBLY OF COUNTRY- ESS SPECIFIC PRINCIPAL ORGANISATIONAL/ INVESTIGATORS NATIONAL SURVEY ..... ORGANISATIONS METHODOLOGICAL INSTITUTIONS

TEAM ...... COUNTRY COUNTRY 1 2

SOCIAL SCIENCE COUNTRY RESEARCH INSTITUTES 3

It appears somewhat premature to expertise in social science specify the ESS structure very methodological research; much beyond what was just said. . both ESS committees for However, a few general reasons of continuity and proven considerations or suggestions success should continue in their should be mentioned: present composition at least for the first ESS wave, should the ESS . the Steering Committee be funded; should consist of senior social . the Data Archive and the scientists from participating hosting research institute to countries and should be which the ESS Methodological nominated by their national ESF Team will be attached, should be member organisations (provided selected on the basis of an open the ESF – as envisaged and competition, its criteria being set deemed desirable – continues to jointly by the Steering Committee play a major tutoring role vis à vis and the Methodology Committee; the ESS); . the appropriate national . the Methodology Committee funding bodies would reach a should consist of 6 to 8 senior decision, after consultation with social scientists with active the Steering Committee, on the 37

selection of the Principal detailed discussion in the ESS Investigator in that country; the Methodology Committee has Principal Investigator cannot at resulted in the following the same time be a member of the recommendations: Steering Committee; . the selection of the national . a full-time position for a senior organisation conducting the coordinator who would stimulate, fieldwork of the survey will be organise and supervise all aspects organized by the Principal of the ESS from its design to its Investigator; it will be based on successful delivery; the acceptance of the standards . four half-time positions for set by the Methodology experienced researchers for Committee (see Appendix 5); (1) pretesting, questionnaire . the most important bottom-up construction and translation, plus element in the ESS is the teams of related methodological research; researchers who compete in (2) sampling frame development designing the topical modules and implementation, plus related (referred to as ‘subject specialists’ methodological research; in the table, opposite). These (3) data quality check, liaison groups should be small in size (3- with archive(s) data for multi- 6), but should preferably have a level analyses, plus related multi-national composition. It methodological research; will be one of the most important (4) data analysis and indicator tasks for the Steering Committee validation/construction, plus to elicit a vital response from the related methodological research; European social science community to participate in this . two half-time positions for competition. Obviously, the junior researchers; selection of topics by the Steering . a full-time secretary Committee will be decisive here . personnel resources for because it defines the extent to auditing and financial control. which researchers from different social science disciplines will become involved in the ESS.

One of the most pressing problems to be solved is that research councils and other funding agencies agree that a minimal permanent staff (permanent in terms of positions, not necessarily in terms of individuals) is absolutely mandatory for running the ESS. A 38 8. Costs

8. Costs major exception here is the calculated cost of half-time 8.1 General remarks position for Principal Investigators for two years in 16 Since cost is a particularly touchy countries where the gross amount subject, a major note of caution will, of course, vary with the has to be introduced in dealing number of countries involved). with this part of the proposal. As is evidenced by the previous 8.1.1 Cost of survey sections of this document, The chairman of the substantial thought especially by Methodology Committee sent a the Methodology Committee has document (reprinted as Appendix been invested in proposing what 5) to all members of the Steering can be considered to be a sound Committee asking them to structure for the ESS and the way present him with estimates by an various groups of participants will experienced survey organisation have to interact in order to make for the fieldwork cost in the given the ESS work. Based on the country based on his detailed organisational concept laid out in specification. Section 7, in the following paragraphs an effort is made to at Sixteen of the countries presented least somewhat reliably estimate in the Steering Committee have the overall cost of the survey. It responded so that a broad must be clear from the start that information base is available. If these are educated estimates and one takes the mean and the median that the calculation covers a average estimates per country in period of two years, in EURO, VAT excluded, then accordance with the two-year . the mean cost per survey is time frame for the first wave of 262.000; the ESS. In addition, any changes . the median cost per survey is in the concept for the ESS which 255.000. emerge during the deliberation process leading to a definite Assuming that those 16 countries recommendation by the SCSS which have provided the may affect these estimates. They Methodology Committee with are divided in two parts: (1) the cost estimates will participate, total cost of the field work and then the overall field work cost is data preparation derived from the 4.200.000 EURO. information by the 16 countries from which cost estimates were 8.1.2 Fixed costs obtained, and (2) the fixed cost of Since the rationale for the various the ESS in the sense that these cost items has already been laid will occur more or less out in previous sections of the independently of the number of report, in the following only the participating countries (the only 39

individual cost positions are two-year period. According to itemized and presented with a information from Steering price tag. In order to obtain an Committee members about their overall cost figure for the first national practices, a 20 percent two-year wave of the ESS, all overhead on the fixed costs is individual items are costed for a included.

Costs for first wave of survey – two years costs (in 1000 EURO) . one project director, full-time ...... 200 . four senior researchers, half-time ...... 320 . two junior researchers, half-time...... 130 . one secretary, full-time ...... 70 . auditing and financial control ...... 50 . 16 principal investigators, half-time...... 800 . overhead 20 % of 1.570 ...... 314 1.884 Methodological research . pretest work (600 interviews each in three countries) ...... 225 . methodological experiments ...... 60 285 Other cost . archival work...... 150 . travel expenses committees (25 members, 2 meetings per year, 1000 EURO each) ...... 100 . travel expenses topic specialists (5 members, 3 meetings, 1000 EURO each) ...... 15 . travel expenses principal investigators (15 members, 3 meetings, 1000 EURO each) ...... 48 . consultancy fees ...... 50 363 Total fixed cost 2.532 Contingency fund 10 % ...... 253 2.785 Total cost of European Social Survey (for 2 years) . field work ...... 4.200 . fixed costs ...... 2.785 Grand total ...... 6.985

9. Funding individual National Science Foundations to fund their own The working assumption at the survey costs every two years, time of the writing of this report averaging at an amortised is that the central (fixed) costs of annualised cost of around 130.000 around 2,8 m EURO per survey EURO for each participating (1,4 m EURO per year) would be nation (more in the Scandinavian centrally-funded (ideally via a countries and Germany, less in mechanism such as the EU’s Fifth many other countries, and much Research Framework less in some). Programme). This would leave the 40 Appendix 1.1 41 Expert Group on a European Social Survey – List of members

Professor Max Kaase (Chairman) Professor Antonio Schizzerotto Wissenschaftszentrum Berlin Facoltà di Sociologia für Sozialforschung Università degli Studi di Trento FS Sozialer Wandel, Via Verdi 26 Institutionen und Vermittlungsprozesse 38100 Trento Reichpietschufer 50 Italy 10785 Berlin Germany Dr. L. Henk Stronkhorst Head, Scientific Statistical Agency Mr. Bruno Cautrès NWO-WSA CIDSP/IEP de Grenoble Postbus 93120 Domaine Universitaire 2509 AC The Hague B.P. 45 The Netherlands 38402 Saint Martin d’Hères cedex France Dr. John H. Smith (Secretary) European Science Foundation Professor Juan Diez Nicolas 1 quai Lezay-Marnésia (Professor of Sociology, 67080 Strasbourg cedex Universidad Complutense de Madrid) France ASEP Paseo de la Castellana 173, 5º I 28046 Madrid Spain

Professor Fredrik Engelstad Institute for Social Research (ISF) Munthesgate 31 0260 Oslo Norway

Professor Roger Jowell Director Social and Community Planning Research (SCPR) 35 Northampton Square London EC1V 0AX United Kingdom

Professor Leif Nordberg Åbo Academy University Department of Statistics Fänriksgatan 3B 20500 Turku Finland 42 Appendix 1.2 Steering Committee – List of members

Professor Max Kaase Professor Nikiforos Professor Stein Kuhnle (Chairman) Diamandouros Department of Comparative Politics Wissenschaftszentrum Berlin für National Centre for Social Research University of Bergen Sozialforschung 14-18 Messoghion Av. Christiesgate 15 FS Sozialer Wandel, 11527 Athens 5007 Bergen Institutionen und Greece Norway Vermittlungsprozesse Tel: +30 1 74 89 123 Tel: 47 55 58 21 79 Reichpietschufer 50 Fax: +30 1 74 89 127 Fax: 47 55 58 94 25 10785 Berlin E-mail: [email protected] E-mail: [email protected] Germany Tel: +49 30 254 91 293 Professor Henryk Domanski Professor Michael Laver Fax: +49 30 254 91 318 Institute of Philosophy and Department of Political Sciences E-mail: [email protected] Sociology The Policy Institute Polish Academy of Sciences Trinity College Professor Rune Åberg Nowy Swiat 72 Dublin 2 University of Umeå Palac Staszica Ireland Department of Sociology 00-330 Warsaw Tel: +353 1 608 20 36 901 87 Umeå Poland Fax: +353 1 677 05 46 Sweden Tel: +48 22 826 96 17 E-mail: [email protected] Tel: +46 90 786 59 60 Fax: +48 22 826 99 48/+48 22 Fax: +46 90 786 66 94 826 78 23 Professor Guido Martinotti E-mail: [email protected] E-mail: [email protected] Via del Caravaggio, 5 20144 Milano Professor Jacques Billiet Professor Yilmaz Esmer Italy Department of Sociology Faculty of Economic and Tel: +39 2 861 089 Catholic University Leuven Administrative Fax: +39 2 861 891 Van Evenstraat 2B Sciences, Bogazici University E-mail: [email protected] 3000 Leuven P.K. 2 Bebek Belgium Istanbul Professor José Ramón Montero Tel: +32 16 32 31 57 Turkey Fundación Juan March Fax: +32 16 32 33 65 Tel: +90 212 263 15 50 Centro de Estudios Avanzados en E-mail: [email protected] Fax: +90 212 265 63 57 Ciencias Sociales E-mail: [email protected] Castello, 77 Professor Antonio Brandao 28006 Madrid Moniz Dr. Peter Farago Spain Faculdade de Ciencias e Tecnologia Landert Farago Tel: +34 1 435 42 40 Universidade Nova de Lisboa Davatz & Partner Fax: +34 1 576 34 20 Quinta da Torre Grossmünsterplatz 6 E-mail: [email protected] 2825 Monte de Caparica 8001 Zurich Portugal Switzerland Dr. Karl H. Müller Tel: +351 1 295 4464 Tel: +411 265 39 90 Institute for Advanced Studies Fax: +351 1 295 4461 Fax: +411 265 39 99 Department of Sociology E-mail: [email protected] E-mail: [email protected] Stumpergasse 56 1060 Vienna Mr. Bruno Cautrès Professor Roger Jowell Austria CIDSP/BDSP Social and Community Planning Tel: +43 1 59 991 212 Institut d’Etudes Politiques de Research (SCPR) Fax: +43 1 59 991 191 Grenoble 35 Northampton Square E-mail: [email protected] Domaine Universitaire - B.P. 48 London EC1V 0AX 38040 Grenoble Cedex 9 United Kingdom France Tel: +44 171 250 1866 Tel: +33 4 76 82 60 94 Fax: +44 171 250 1524 Fax: +33 4 76 82 60 50 E-mail: [email protected] E-mail: [email protected] 43

Professor Leif Nordberg Professor Françoise Thys- Dr. John H. Smith Åbo Academy University Clement (ESF Scientific Secretary for the Department of Statistics Pro-Recteur Social Sciences) Fänriksgatan 3B Université Libre de Bruxelles Ms. Geneviève Schauinger 20500 Turku Avenue F.D. Roosevelt 50 (Administrative Assistant) Finland Case Postale 140 Mrs. Beatriz Garcia Diaz- Tel: +358 2 215 4163 1050 Bruxelles Ambrona Fax: +358 2 215 4677 Belgium (Administrative Secretary) E-mail: [email protected] Tel: +32 2 650 41 73 European Science Foundation Fax: +32 2 650 44 95 1, Quai Lezay-Marnésia Dr. Niels Ploug E-mail: [email protected] 67080 Strasbourg Cedex National Institute for Social France Research Substituted by: Tel: +33 (0)3 88 76 71 31 Herluf Trolles Gade, 11 Mr. Pierre Desmarez Fax: +33 (0)3 88 37 05 32 1052 Copenhague K TEF – CP 124 E-mail: [email protected] Denmark Avenue Jeanne 44 Tel: +45 33 48 08 40 1050 Bruxelles Fax: +45 33 48 08 33 Belgium E-mail: [email protected] Tel: +32 2 650 34 30 Fax: +32 2 650 33 35 Substituted by: E-mail: [email protected] Mr. Torben Fridberg Socialforskningsinstituttet Professor Niko Tosˇ Herluf Trolles Gade 11 Faculty for Social Sciences 1052 Copenhagen K University of Ljubljana Denmark Kardelieva Pz. 4 Tel: +45 33 48 08 47 1000 Ljubljana Fax: +45 33 48 08 33 Slovenia E-mail: [email protected] Tel: +386 61 349 994 Fax: +386 61 1686 054 Professor Shalom Schwartz E-mail: [email protected] Department of Psychology Hebrew University of Jerusalem Professor Michael Warren Mount Scopus 30 Cole Park Road Jerusalem 91905 Twickenham Israel TW1 1HS Middlesex Tel: +972 2 588 30 24 United Kingdom Fax: +972 2 588 11 59 Tel: +44 181 891 3130 E-mail: [email protected] Fax: +44 181 891 1970 E-mail: [email protected] Mrs. Ineke Stoop Department of Information and Automation Sociaal en Cultureel Planbureau Parnassusplein 5 P.O. Box 16164 2500 BD The Hague The Netherlands Tel: +31 70 340 70 00 Fax: +31 70 340 70 44 E-mail: [email protected] 44 Appendix 1.3 Methodology Committee - List of members

Professor Roger Jowell Professor José Ramón Dr. Joachim Vogel (Chairman) Montero Statistics Sweden Social and Community Planning Fundación Juan March Box 24300 Research (SCPR) Centro de Estudios Avanzados en 104 51 Stockholm 35 Northampton Square Ciencias Sociales Sweden London EC1V 0AX Castello, 77 Tel: +46 8 783 4930 United Kingdom 28006 Madrid Fax: +46 8 783 4772 Tel: +44 171 250 1866 Spain E-mail: [email protected] Fax: +44 171 250 1524 Tel: +34 1 435 42 40 E-mail: [email protected] Fax: +34 1 576 34 20 Professor Max Kaase E-mail: [email protected] (Chairman Steering Committee) Professor Jacques Billiet Wissenschaftszentrum Berlin für Department of Sociology Professor Willem Saris Sozialforschung Catholic University Leuven Department of Statistics and Reichpietschufer 50 Van Evenstraat 2B Methodology 10785 Berlin 3000 Leuven Faculty of Political, Social and Germany Belgium Cultural Sciences Tel: +49 30 254 91 293 Tel: +32 16 32 31 57 University of Amsterdam Fax: +49 30 254 91 318 Fax: +32 16 32 33 65 O.Z. Achterburgwal 237 E-mail: [email protected] E-mail: [email protected] 1012 DL Amsterdam The Netherlands Dr. John H. Smith Dr. Peter Lynn Tel: +31 20 525 2091 (ESF Scientific Secretary for the Survey Methods Centre Fax: +31 20 525 2086 Social Sciences) Social & Community Planning E-mail: [email protected] Ms. Geneviève Schauinger Research (Administrative Assistant) 35 Northampton Square Professor Antonio Schizzeroto Mrs. Beatriz Garcia Diaz- London EC1V 0AX Dipartimento di Sociologia e Ambrona United Kingdom Ricerca Sociale (Administrative Secretary) Tel: +44 171 250 18 66 Università di Trento European Science Foundation Fax: +44 171 250 15 24 Via Verdi 26 1, Quai Lezay-Marnésia E-mail: [email protected] 38100 Trento 67080 Strasbourg Cedex Italy France Dr. Nonna Mayer Tel: +39 0461 881 315 Tel: +33 (0)3 88 76 71 31 Centre d’Etude de la Vie Politique Fax: +39 0461 881 348 Fax: +33 (0)3 88 37 05 32 Française E-mail: [email protected] E-mail: [email protected] 10, rue de la Chaise 75007 Paris Professor Dr. Jan van Deth France Lehrstuhl für Politische Tel: +33 1 49 54 21 49 Wissenschaft und Fax: +33 1 49 54 20 25 International Vergleichende E-mail: [email protected] Sozialforscung Universität Mannheim Dr. Ekkehard Mochmann Seminargebaude A 5 Zentralarchiv fur emprische 68131 Mannheim Sozialforschung Germany Universität zu Köln Tel: 49 621 292 5316 Bachermer Str. 40 Fax: 49 621 292 5315 50931 Köln 41 E-mail: [email protected] Germany mannheim.de Tel: +49 221 394 737 Fax: +49 221 476 9477 E-mail: [email protected] Appendix 2 45 Regional variables in cross-national surveys

Regional disaggregation is usually a standard procedure in national surveys. These classifications normally follow (larger) administrative and territorial divisions (counties, larger territorial units). Sometimes smaller geographical areas (municipalities, parishes) are clustered by demographic/ social/economic characteristics. Regional analysis in comparative surveys is problematic since the measurement of regional variation is affected by national traditions of administrative and territorial classifications. Hence, there are few examples of regional disaggregation in comparative studies. For the ESS the NUTS-system of EUROSTAT (Nomenclature of Territorial Units for Statistics), applied in official regional statistics for EU-member states, offers a new option. NUTS is a five- level hierarchical classification (NUTS I-5) and is used for the production of detailed regional statistics (see Regions: Statistical Yearbook). The NUTS system is also primarily based on existing administrative classification, as well as traditional territorial subdivisions. The principles of regional desaggregation may vary much between countries. EUROSTAT surveys normally carry the NUTS-code (ECHP, LFS; also the demographic database and most economics statistics). There is a large variation of regional indicators available in EUROSTATs regional database REGIO and reported in the Statistical yearbook, which can be linked to cross-national surveys which include the NUTS code, such as: . population structure, population change, density, fertility, migration . labour force: employment and unemployment; by age, sex, branch . GDP per capita, selected economic indicators · education . health services, mortality, causes of death . housing stock and housing amenities . consumption: electricity, cars. Part of this information may give a contextual background relating to general living conditions, social and economic change, and regional marginalisation. However, the comparative value of this information is related to the comparability of the principles of the NUTS-classification itself which have been applied in the member states. The NUTS code (e.g. the third level) can be used for further territorial aggregation of clusters of similarity, for instance distinguishing between centre and periphery, growing and declining regions, by employment levels, by living conditions, and educational resources. This can be done by using REGIO. Matching the planned ESS micro data base and REGIO by the NUTS code seems to be a simple procedure. The value of NUTS data has to be judged from the included topics. For comparative analysis it may be of less value, as compared to national research for which it no doubt will be an important standard disaggregation. Recommendation: Include the NUTS-code in the ESS, but leave it to the individual researcher to match their data with REGIO. 46 Appendix 3.1 Letter to national Steering Committee members regarding sampling

Dear < > I am writing to you as your country’s representative on the Steering Committee for the ESF’s project on preparing a blueprint for the European Social Survey (ESS). On behalf of the methodology committee, I would like to obtain some basic information about sampling procedures in < >. Below is a list of questions. I am asking the same questions of all the countries, in order to provide the methodology committee with some systematic information that will help us to devise an appropriate sampling strategy. If you are not able to answer the questions, please feel free to pass the questions on to an appropriate colleague in your country. If you do this, would you please let me know, preferably including the name and email address of the colleague who will be responding. In order to inform the methodology committee’s report to the next steering committee meeting, I need to have your response to the questions by 20 April. The questions should only take a few minutes to answer - I am not expecting lengthy essays! Please let me have your responses by email if possible, otherwise by fax. If you have any queries or if my questions are unclear, please do not hesitate to email or phone me. Thank you for your help.

1. Is strict probability sampling commonly used for household/individual surveys in your country? 2. What sorts of survey organisations use strict probability sampling, and in what circumstances/ for what sorts of surveys? 3. When probability sampling is used, for national surveys, are sample designs usually multi-stage (for example, localities sampled as first stage, individuals within localities as second stage)? If yes, how many stages, and what are the sampled units at each stage? 4. Could population registers or population lists of some sort be used for sampling? If yes, in what form and at what geographical level do the lists exist (for example, a single national computerised list, or a list for each town or administrative area)? And are they accessible to all survey agencies, or only government agencies? 5. Is substitution of non-responding households/individuals a common practice in your country? If yes, are preselected lists (random samples) provided as substitutes, or are interviewers just given certain rules to follow? 6. Is weighting of survey data for non-response a common practice in your country? 7. Is quota sampling used for household or individual surveys in your country? 8. What sorts of survey organisations use quota sampling, and in what circumstances/ for what sorts of surveys? 9. If it were to be proposed that the ESS should use strict probability sampling methods, with no quota controls and no substitution procedures allowed, would you anticipate any particular problems in your country? If yes, what? 10. Are there any other features of general population sampling in your country that you think we should take into account? 11. Can you provide a copy of, or a reference to, a publication which documents (in some detail) a typical sample design in your country (for example, a survey technical appendix or other paper)? Many Thanks!

(Note that in many cases the initial responses to these questions prompted follow-up questions and a dialogue often ensued.) Appendix 3.2 47 Sample Design for the ESS - a draft specification

Survey Population The survey population consists of all persons aged 15 and over (no upper age limit) resident in private households in the country, regardless of nationality, citizenship or legal status *. It is recommended that those persons living in institutions (other than student accommodation and accommodation related to the military and security forces, such as prisons and army barracks) should also be included if possible.

Sampling Method The sample is to be selected by strict probability methods at every stage. This means that the relative selection probabilities of every sample member will be known, and will be recorded on the data set. No form of quota sampling can be used at any stage.

Substitution Substitution of either non-responding sample members or vacant addresses is not permitted. The size of the sample to be selected initially will be determined so as to result in approximately the desired number of achieved interviews under realistic assumptions about response rate and ineligibility rate (see below). Once the sample is selected, no selected addresses/ persons may be substituted.

Sample Size Interviews should be achieved with a minimum of 2,000 persons. It is recommended that the sample size should be at least 2,500 in each country. In addition, it must be ensured that the effective sample size is at least 1,500. The meaning of effective sample size is explained below. The exception to this is for small countries with a total population of less than 3 million. In those countries, the minimum requirement is 1,000 interviews and an effective sample size of 800. Countries will be expected to determine the appropriate size of initial sample to select, based upon realistic estimates of the response rate that will be achieved and likely design effects. Each of the key components of design effect are discussed below, followed by a worked example of how to determine the initial sample size.

Response Rate To ensure the quality and reputation of the ESS, it is vital that the survey achieves the highest possible response rate in each country. The target should be 75%, though it is anticipated that many countries should be able to achieve response rates in excess of this. The survey organisation must take all reasonable measures to maximise response. In particular, no sample member must be classified as a “non-contact” until contact has been attempted on at least four occasions, including at least one evening and one weekend. Interviewers should be trained in response- maximisation techniques and doorstep interactions.

Over-Sampling It is perfectly acceptable to over-sample particular subgroups - in other words to use differing selection probabilities - provided that the total sample still complies with the effective sample size criterion. There are two reasons why we anticipate that some countries might wish to over-sample some groups. First, it may be possible to anticipate low-response strata. For example, if response rate is predicted to be 65% in large cities but 80% elsewhere, it would be efficient to over-sample the

* Later on in the deliberations of the two ESS committees, the definition of the population was changed to aim also at the inclusion of significant parts of the institutional population, like people living in old age homes. 48 Appendix 3.2 Sample Design for the ESS - a draft specification

large cities by a factor of 1.23 (80/65). Such use of over-sampling is encouraged. (The resultant non-response weight should largely cancel out with the selection probability weight, thus minimising any precision loss due to weighting.) Second, it may be desirable to over-sample certain minority groups, to permit separate analysis of them. But it is important to assess the effect of such over-sampling on the design effect due to differing selection probabilities (see below).

Effective Sample Size The effective sample size (neff) is the size of a simple random sample which would produce the same precision (standard errors) as the design actually used. Typically, neff is less than the actual number of achieved interviews, m, as certain aspects of survey design - for example, clustering, the use of differing selection probabilities - tend to reduce the precision of estimates. The reduction of precision is known as the design effect (DEFF): DEFF = Actual sampling variance / Sampling variance with srs of same size; DEFF = m / neff, so neff= m / DEFF. We therefore need to be able to predict the value of DEFF for a proposed sample design, in order to determine how many interviews should be achieved in order to produce a particular value of neff. We suggest that two components of DEFF should be taken in to account at the design stage - the design effect due to differing selection probabilities (DEFFp) and the design effect due to clustering (DEFFc).

Then, DEFF = DEFFp x DEFFc. We then also need to predict the survey response rate (and the proportion of ineligibles on the sampling frame, if relevant) in order to determine the size of initial sample, n, to select in order to achieve approximately m interviews.

Design Effects due to Differing Selection Probabilities In some countries which have accessible population registers, it will be possible to select an equal- probability sample from the survey population. In other countries, it will be necessary to select the sample in stages, with the penultimate stage being residential addresses. In this case, each person’s selection probability will depend on their household size. Another reason for having differing selection probabilities would be if important minority groups were to be over-sampled. If differing selection probabilities are to be used - for whatever reason - the associated design effect should be predicted. This can be done very simply, using the following formula

where there are respondents in the selection probability class, each receiving a weight of

(this formula assumes that the population variance of survey variables will not vary over selection probability classes - a reasonable assumption in most situations) 49

Design Effects Due to Clustering It is anticipated that in most countries it will be efficient to select a multi-stage, clustered, sample. In such situations there will also be a design effect due to clustering,

where b is the mean number of respondents per cluster and ρ is the intra-cluster correlation (or “rate of homogeneity”) - a measure of the extent to which persons within a clustering unit are more homogeneous than persons within the population as a whole (see Kish, , pp. 161-164). This design effect can be estimated, at least crudely, from knowledge of other surveys and/or the nature of the clustering units. In practice, all elements of the overall design effect, including that due to differing selection probabilities and that due to clustering, will take different values for different survey estimates. For sample design purposes, an average value should be used.

Example: How to determine the size of sample to select We have prescribed neff > 1500. To determine m, we must first estimate

DEFF = DEFFp x DEFFc. 1. Suppose the proposed clustering units are administrative areas of around 5,000 households on average and that based on data from other surveys, we expect that for these areas, ρ will take values of around 0.02 for many variables. Then, if we are proposing a design with a mean of 15 interviews per cluster:

DEFFc = 1 + (15-1) x 0.02 = 1.28. [Note: If there is no available empirical evidence at all upon which to base an estimate of ρ , then we suggest that a value of 0.02 should be used.]

2. Suppose that the only available sampling frame is a list of addresses and that these must be selected with equal probabilities. The proposed design is then randomly to select one person to interview at each address. This is the only aspect of the proposed design that involves differing selection probabilities. Then, we can use population statistics on the distribution of household size to estimate the number of respondents in each selection probability class, thus:

No. of persons Proportion of No. of Relative aged 18+ households achieved weight in household in population interviews 2 iHi/H mi wi miwi miwi

1 0.35 0.35m 1 0.35m 0.35m 2 0.45 0.45m 2 0.90m 1.80m 3 0.12 0.12m 3 0.36m 1.08m 4 0.06 0.06m 4 0.24m 0.96m 5 0.02 0.02m 5 0.10m 0.50m 1.95m 4.69m 50 Appendix 3.2 Sample Design for the ESS - a draft specification

The population distribution of household size appears in the first two columns. From this, we can predict that the sample distribution will be as shown in the third column. We can thus predict

DEFFP: 2 DEFFp = m x 4.69m/(1.95m) = 4.69 /1.952 = 1.23. 3. Thus, we predict DEFF = 1.28 x 1.23 = 1.57. Consequently, to achieve neff > 1,500 with this design, we would need m > 1,500 x 1.57 = 2,355. 4. The final stage is to calculate the sample size to select initially in order to be likely to achieve around 2,355 interviews. Suppose we anticipate a response rate of 80% and that 5% of the sampling frame units will be ineligible (e.g. addresses which do not contain a resident household), then: n = (m / 0.80) / 0.95 = 3,098 So we would select a sample of at least 3,100 addresses.

Non-response Weighting It is anticipated that it may be necessary to weight the survey data to correct for non-response bias. To aid this process, a small number of data items should be recorded by interviewers for every selected case (respondents and non-respondents). These items are likely to include characteristics of the dwelling and of the surrounding area, which can be easily observed by interviewers. Each country should subsequently supply a data file containing a record for each selected sample case, with an indicator of response outcome, plus these interviewer-recorded items and the appropriate geographical identifier to permit linkage to the geographic data described elsewhere. The weighting will then be carried out centrally.

Documentation The sampling procedures used, and response obtained, should be fully documented in a technical report of the survey. As a minimum, the report of sampling should include a definition and description of the sampling units used at each stage, a description of any stratification of the sampling frame and a description of ways in which selection probabilities could have varied at each stage. The survey data file should include a variable(s) indicating the relative selection probabilities of cases, to enable appropriate weighting to be carried out, and indicators of selection stratum and PSU, to enable standard errors and design effects to be estimated and reported. As a minimum, the report of response should include the number of selected cases falling into each of the following categories (which are intended to be mutually exclusive and comprehensive): not eligible; no contact after 4+ visits; personal refusal; proxy refusal; achieved interview. Appendix 4 51 The necessity of ongoing methodological research in the ESS context

In order to make the ESS different from normal commercial research, various requirements have to be satisfied. The most important are the following: . the variables should be related to existing theories or theories which will be tested (see Section 4); . variables should be included which allow the connection with data of other units like regions (see Appendix 2); . the results should be maximally comparable across countries (see Section 3); . the quality of the procedures should be controlled and reported; . research should be done continuously on the quality of the data and the improvement of the procedures without disturbing the trends in the data. Given that the first three points are already discussed in other sections of the report, this document will concentrate on the last two points.

1. Routine control of data quality The key issue in statistical research is standardisation of measurement. This applies to all steps in the production process, including questionnaire design, sampling procedures, data collection methods, data editing, estimation and presentation. Quality controls are designed to check the process of standardisation step by step. In ordinary cross-sectional national surveys normally a standard question program is run. In these cases the same general procedures (a standardized stimulus situation) are applied to all respondents. However, comparative research adds another dimension to the management of data collection. A large number of national institutes (private business, public as well as research institutes), will be involved. These organisations display a large variation in survey tradition, practical procedures, ambitions, technical experience, and resources. There will be language and cultural barriers which have to be bridged, and national practices have to be sacrificed to support comparability. In addition to the normal quality controls in national surveys cross-national comparability needs to be ascertained. Hence, the general survey design as well as comparative quality control has to be centralized. A central group is needed to take full responsibility for the design of the ESS as well as for comparative quality control. Optimal comparability is not arrived at by the principle of subsidiary. This has been the experience of most comparative surveys, including the European Community Household Panel and ISSP. Furthermore, the mere number of participating countries demands centralisation of responsibility. This means that one will have to take quality control further than in other comparative surveys only conducted in a few countries. The final data set has to be well cleaned in order to minimize later contacts back to the national level, and to deliver an entry to the subject area in the form of already tested and centrally approved basic tables. For this purpose the ESS team and the Methodology Committee will have to closely interact. The quality program must include three steps: . clearly defined quality goals approved by each participating institute, granting maximum comparability and quality. These goals should take the form of a written contract; 52 Appendix 4 The necessity of ongoing methodological research in the ESS context

. a detailed quality control programme for the various steps in the survey, defining action at national as well central level; . developing a standardized set of (cross-national) quality indicators derived from the first two points, a set of cross-national basic tables, and a technical report. The following list covers the major steps in a general quality control program: . questionnaire design: English master questionnaire; two-way translation; functional equivalent items have to be negotiated at central level; a standardized questionnaire with additional national options; including basic information on the interview context; . population and sampling; same universe; strictly probability samples; representativeness allowing for variant sampling procedures; standardized post-stratification using available demographic sources; . field work: same method (personal interview); response rate improvement program; national commitment to high response rates; agreed upon registration of variables for non-response analysis, comparative analysis of the structure of non-response; defined rules concerning item non- response treatment; . measurement: standardized pretests; fixed common questionnaire (allowing for additional national modules), limited test-retest studies in all countries to estimate reliability (see below); consistency with other comparative statistics (demographics, etc); . editing: standard program developed and executed at central level (this program should also define the initial editing procedures at national level); controlling/editing the cross-national structure of data by using standard tabulation (joint activity of the subject matter and methodology groups); feed back to the national level and correction; developing basic derived variables; . timeliness: a centralized quality control program requires strict conformity of the participating countries to a common contracted timing of data collection/editing and delivery of the national data sets; . activities related to the dissemination of the data set and archiving: a general technical report including cross-national comparison of applied methods, quality indicators, and quality control activities/corrections; . production of a comparative basic table compendium in standard format including a) population/percent estimates for all variables by country, b) three-way tables (indicators by country and basic background variables).

2. Continuous methodological research There are three major problems which have to be taken into account in the design of the ESS. The first is the problem of method effects which always occur; the second is the problem of comparability of data across countries, and the third is the problem of lack of a crystallized opinion. 2.1 Method effects Starting with the first issue it is well known that any method has a specific effect on the data which 53

are collected. Sudman, Bradburn and Schwarz (1996), Schuman and Presser (1981), Billiet et al. (1986), and Saris and Schwarz (1996) and others have illustrated this problem with many examples. Thus, it is possible to get differences of 20% in response distributions depending on the use of the word “allow” or “forbid” in the question (Schuman and Presser 1981). It would also be very unfortunate to collect data where 50% of the correlations found is due to method effects. This is not impossible as MTMM (multi-trait-multi-method) studies have shown. This being so, one has to be concerned to choose a method which has the least artificial effects. Thus, continuous methodological research is needed. There are four well-developed procedures for this research which can be applied. The first is the testing of questionnaires in cognitive laboratories in order to determine how different words are interpreted by the respondents. Sudman, Bradburn and Schwarz (1996) discuss several different methods to evaluate the process by which the respondent comes to an answer. For the basic concepts of the questionnaires such tests are very important to make sure that people from different countries interpret the questions in the same way and derive their answer in approximately the same way. Such a test can be done in an early stage of the design of the questionnaires before the pilot studies. The second approach is the interaction analysis of the communication between the interviewer and the respondent. Careful analysis of this communication can show what goes wrong in the data collection. This approach is recommended for the pilot studies in three countries in order to avoid serious problems. A third possibility is the use of split ballot experiments where different subgroups of the sample get different questionnaire forms in order to see if large differences in responses are obtained with the different forms. The fourth option is MTMM research which allows to study the effects of the methods on the correlations between the variables. All four methods should be used to evaluate the core questions in order to avoid serious problems with these questions. The cognitive studies and the interaction analysis should be done for only a limited number of interviews. The interaction analysis can be applied on the whole interview, screening also the questions for the specific issues. The cognitive analysis should only be done for the most crucial parts. The other two procedures have to be planned in detail and should be concentrated on the most important variables in the studies. 2.2 Cross-cultural comparability Comparative research is not yet so established that it is well-known which questions can be compared across countries and which not or how the formulations have to be adapted. Many differences in cross-cultural research are due to the lack of equivalence of procedures used in the different countries. Several problems and approaches mentioned for research in general also apply , of course, to comparative research. But the problems of cross-cultural studies have not been discussed so widely. Therefore, in this paper they will get some special attention, starting with the differences between response distributions. 2.2.1 Differences between response distributions If the probability for a specific response given the opinion one has, is different in one country from that in another country (for example because of a different sensitivity to social desirability), then 54 Appendix 4 The necessity of ongoing methodological research in the ESS context

the response distributions will not be the same even though the distributions of the opinions in the different countries might be the same. That this is not unthinkable has been shown in a study related to questions of the Eurobarometer (chapter 6 and 9 in Saris and Kaase 1997) . As an example, the question on “satisfaction with the way democracy works” was discussed. The formulation in English is: On the whole , are you very satisfied, fairly satisfied, not very satisfied / not at all satisfied with the way democracy works in (our country)? Would you say you are (1) very satisfied, (2) fairly satisfied, (3) not very satisfied / (4) not at all satisfied (5) DK/no answer? Latent class analysis showed that people with opinion (3) “not very satisfied” answered differently

in the three different countries. If pij stands for the probability to give answer i if one has opinion j, below only the probabilities of the answers 1 to 4 for the group with opinion 3 are given because the probabilities for all other categories were the same. The results were:

p13 p23 p33 p43 France .00 .000 .669 .330 Belgium .000 .072 .422 .507 Spain .000 .067 .656 .277

This table shows that the probabilities in Belgium are significantly different from those in the other two countries. It will be clear that the distribution of the observed variable will also be different if the response probabilities are so different. The same has been found for all satisfaction variables in the Eurobarometer experiment and for questions concerning interest in the EU and knowledge about the EU. In these cases the explanation seems to be the translation. In Spain, the English labels have been used. In Dutch questionnaires in Belgium the labels used are: very satisfied, rather satisfied, not so satisfied and not at all satisfied. and in French it is: très satisfait (= very satisfied), plutôt satisfait (= rather satisfied), plutôt pas satisfait(= rather not satisfied) and pas du tout satisfait (= not at all satisfied). Although these labels were seen as functionally equivalent, the results were quite different. Several other examples of the same kind can be found in chapter 9 of Saris and Kaase (1997). These differences in response probabilities will cause differences between countries, which are normally interpreted as substantial differences. Before one can draw this conclusion one has, however, to first correct for these differences in response probabilities. This can be done but it requires repeated observations from the same people and afterwards a transformation based on the obtained response probabilities. How these corrections can be done has been discussed in chapter 11 of Saris and Kaase (1997). The conclusion from all this is that research on the response probabilities is needed in order to say if one can compare the data from different countries directly without correction or not. 55

2.2.2 Difference in correlations As a consequence of different response probabilities in different countries not only the distributions will be different, but also the relationships between variables. In a different study, using correlations to describe the relationships between several variables, the results presented in Table 1 have been found (Scherpenzeel 1995). This table shows that the method can have strong effects on the results. Comparing the correlations between the two countries on a 10 point scale one sees that the correlation between satisfaction with life in general (GLS) with all three other variables is smaller in the Netherlands than in Hungary, but that is not true anymore for two out of the three correlations looking at the table for the 5- point-scale. This cannot be explained by sampling fluctuations because in both countries the same people gave answers on both scales. The explanation for the differences is a specific combination of random errors and method effects. Differences in response probabilities, method effects and random measurement errors can be determined using repeated observations. On life satisfaction and many other variables studies have been done on the effects of these errors in the USA and by the International Research Group on Methodological and Comparative Survey Research (IRMCS) in 13 different countries of Europe. The methodology has been discussed in Saris and Munnich (1995), and an application on cross cultural life satisfaction research has been reported in Saris et al. (1996).

Table 1: Correlations between four satisfaction variables measured with four different methods obtained from the same respondents

Netherlands GLS SH SF SC GLS SH SF SC 10 points scale (polychoric corr) 5 points scale (polychoric corr)

GLS 1.00 1.00 SH .458 1.00 .381 1.00 SF .456 .434 1.00 .445 .349 1.00 SC .491 .325 .333 1.00 .462 .232 .270 1.00

Hungary GLS SH SF SC GLS SH SF SC 10 points scale (polychoric corr) 5 points scale (polychoric corr)

GLS 1.00 1.00 SH .490 1.00 .341 1.00 SF .637 .468 1.00 .664 .380 1.00 SC .519 .254 .308 1.00 .296 .182 .247 1.00

This is the kind of research which is absolutely necessary for the most important variables in comparative research. Without such studies, correlations across countries cannot be compared. Therefore, for all variables used in the ESS information should be available about what corrections are needed in order to make the results comparable across countries. 56 Appendix 4 The necessity of ongoing methodological research in the ESS context

2.3 The strength of the opinions In the USA there has been a debate for a long time about the question whether people have sufficient information to participate in political decisions. Converse (1964) started this discussion which has led to a number of methodological papers suggesting that the uncertainty in the opinions might be due to random measurement error (Achen 1975; Judd, Krosnick and Milburn 1981) but recently this issue is again taken up by Zaller (1992) and Sniderman et al.(1991). European scholars hardly took part in this debate; it seems that European social scientists did not see the lack of opinion as such a serious problem. However, especially in European issues it can be shown that the opinions of the people can be changed quite easily. In an experiment (Saris 1997), two extra questions were sufficient to change the opinion of the Dutch population from pro veto rights to a position against veto rights, as is shown in table 2.

Table 2: The effect of information of the right of veto question on the response distributions in the first subsample

Subsample 1 before Pro -European Subsample 1 after Pro-European information information

The countries within the European Union have On the basis of these experiences one can a right of veto, which means that a country is ask the question of whether the right of veto able to block a decision of the European should remain or whether it should be Union. adjusted to majority decision-making.

Do you think that The Netherlands should Do you think that the current form should maintain its right of veto with respect to remain or do you think that a form of important decisions, even if that it is done at decision-making based on a majority should the expense of decision-making in the be introduced (for example 2/3 of the European Union? countries have to agree). %% don’t know 25.3 don’t know 13.8 maintain the right of veto 55.5 maintain the right of veto 26.7 give up the right of veto 19.2 decisions by majority rule 59.5 total (n=833) 100.0 (n=833) 100.0

The information about the strength of an opinion is very important. A weakly developed opinion can easily be changed as was shown above. Billiet et al. (1986) suggest that also for the prediction of behaviour the information about the strength of an opinion is very important. Given that all three problems lead to the same conclusion that repeated observations are necessary, continuous methodological research connected with the development and routine performance of the ESS using repeated observations would be very desirable. 57

References (1995) : The responsive voter: evaluating survey questions by campaign information and Multitrait Multimethod . Achen C.H. (1975) Mass the dynamics of candidate studies. Amsterdam political attitudes and the evaluation, American Political Unpublished Ph.D. dissertation survey response, American Science Review 89: 309-326. of the University of Amsterdam. Political Science Review 69: . Luskin R.C. (1987) 1218-1231. Measuring political . Schuman and Presser (1981) . Battista J.M. and W.E. Saris sophistication, American Questions and answers in (1997) Tests of stability in Journal of Political Science 31: attitude surveys: experiments on attitude research. Quality and 856-899. question form, wording and Quantity (Forthcoming). context. New York: Academic . Piazza T. and P.M. Press. . Billiet J., G. Loosveldt and Sniderman (1996) L. Waterplas (1986) Het survey Incorporating experiments . Sniderman P.M., R.A. Brody - interview onderzocht. Leuven: into computer-assisted and P.E. Tetlock (1991) Universiteit van Leuven. surveys. Paper presented at Reasoning and Choice: the Intercasic conference in San explorations in political . Converse P.E. (1964) The Antonio, Texas, December 11- psychology. Cambridge: nature of belief systems in 14. Cambridge University Press. mass publics, in: D. Apter (ed.) Ideology and Discontent, New . Saris W.E. (1997) The . Sniderman P.M., T. Piazza York: Free Press, 206-261. public opinion about the EU and A. Kendrick (1991) can easily be swayed in Ideology and issue persuability: . Converse P.E. (1970) different directions, Acta dynamics of racial policy Attitudes and Non-attitudes : Politica 32: 406-436. attitudes, in: P.M. Sniderman, continuation of a dialogue, in: R.A. Brody and P.E. Tetlock, E.R. Tufte (ed.), The . Saris W.E. and Munnich A. Reasoning and Choice: quatitative analysis of social (1995) The Multitrait- explorations in political problems. Reading, Mass: Multimethod approach to psychology. Cambridge: Addison-Wesley. evaluate measurement Cambridge University Press, pp. instruments. Budapest: Eötvös 223-243. . Judd C.M. and M. Milburn University Press. (1980) The structure of . Sudman S., N.M. Bradburn attitude systems in the general . Saris W.E., R. Veenhoven, and N. Schwarz (1996) Thinking public: comparisons of a A. Scherpenzeel and B. about answers. San Francisco: structural equation model, Bunting (eds.) (1996) Life Jossey Bass Publisher. American Sociological Review satisfaction in West and 89,2: 309-325. Eastern Europe. Budapest: . Vogel J. (1995) Options for Eötvös University Press. Integrated and comparative . Judd C.M., J.A. Krosnick statistics on living conditions. Saris W. E. and M. Kaase and M.A. Milburn (1981) . Paper presented at the Eurostat Political involvement and (1997). Eurobarometer: Conference on Social Indicators attitude structure in the measurement instrument for and social reporting, general public, American opinions in Europe. ZUMA- Luxembourg, February 1995. Sociological Review 46: 660- Nachrichten Spezial No. 2. 669. Mannheim: ZUMA. . Zaller J.R. (1992) The nature and origins of mass opinion. Scherpenzeel A.A. . Lodge M., M.R. . Cambridge: Cambridge Question of Quality; Steenbergen and S. Brau University Press. 58 Appendix 5 Survey Specification of a Proposed European Social Survey

Introduction The proposed European Social Survey would be carried out to the highest academic standards in all participating countries, to an agreed common specification. It would be supervised by a small full-time central methodological team who would in turn report to a Steering Committee comprising members from all participating nations. Each country will also appoint a Principal Investigator to oversee the conduct of its proposed survey. The PI may or may not be associated with the survey agency which is selected to carry out the survey work. Given the nature of the proposed project, every participating country will doubtless have to adapt or change several of its routine procedures, standards and methods in order to achieve cross- national comparability.

The sample The survey will cover persons aged 15 and over (no upper age limit) resident within private households in each, regardless of nationality, citizenship or legal status. The sample is to be selected by strict random probability methods at every stage. The relative selection probabilities of every sample member should be known and recorded on the data set. Quota sampling will not be used at any stage. Substitution of either non-responding sample members or vacant addresses is not permitted at any stage. The minimum number of interviews to be achieved is 2,000, but the recommended number is 2,500. In countries whose total population is less than 2 million, the minimum number is 1,000 interviews. In any event, the minimum number of ‘effective’ interviews (after discounting for design effects) will be 1,500. Each country will determine the appropriate size of initial sample to select, based upon realistic estimates of the response rate that will be achieved and the eligibility rate (if appropriate). The sampling procedures used, and response obtained, will be fully documented in a technical report of the survey. The report of sampling should include a definition and description of the sampling units used at each stage, a description of any stratification of the sampling frame and a description of ways in which selection probabilities could have varied at each stage. Reports of response will include the number of selected cases falling into each of the following categories (which are intended to be mutually exclusive and comprehensive): not eligible and why; no contact after 4+ visits; personal refusal; proxy refusal; achieved interview (partial); achieved interview (full). The survey data file will include one or more variables to indicate the relative selection probabilities of cases, so that appropriate weighting may subsequently be carried out.

The questionnaire There will be a face-to-face interview questionnaire and a self-completion questionnaire, both administered by the interviewer within the respondent’s household (in some cases the self- completion questionnaire may be left behind for subsequent return by post). It will be designed in English by the PIs collectively, in consultation with cross-national groups of subject specialists, the central methodological team and the survey agencies in each country. The questionnaire may 59

be administered either via paper and pencil or computer-assisted interviewing, preferably by the latter in those countries where it is already cost-effective to do so. The face-to-face questionnaire will consist of roughly 240 items (a one- hour interview, including an extensive socio-demographics section). The self-completion questionnaire will consist of around 40-50 items (10 - 12 minutes). Each country will translate the questions into its main language or dialect. If any other minority language is spoken as a first language by 5% or more of the population of a country, it should be translated into that language too. If, however, that other minority language happens to be the main language of another participating country, then that translation nay be adopted, where appropriate. Translations will be done iteratively (back and forth between English and the other language by different independent translators) until a satisfactory accommodation is reached, but may be carried out by proficient rather than professional translators if available, such as from within the survey team. In addition to three large pilot tests (600 cases each) in at least three participating countries, plus the use of cognitive laboratories to test the main concepts to be used, pre-tests of 50 interviews will take place in all countries. They will be conducted in at least five different areas of each country by at least five different interviewers. All pilot interviewers will be de-briefed in the presence of the PI for that country.

Fieldwork Detailed standard interviewer instructions will be prepared (and translated). Interviewers’ assignment sizes will be set to ensure that the average assignment size per interviewer does not exceed 20 interviews, and that no interviewer carries out more than two assignments per survey. Field outcomes for each call at each address (or other primary unit), will be documented and ultimately keyed from a standardised set of summary codes. Non-response will also be defined and calculated according to a standardised set of categories. Of these, ‘non-contacts’ must be kept to a maximum of 3% of all sampled units, and the target overall response rate in every country will be 75% of eligible units. To achieve these targets, a minimum of four personal visits to each unit will be made, including at least one in the evening and one at the weekend, spread over at least two different weeks. Since it may later be necessary to weight the survey data to correct for non-response bias, a small number of data items will be recorded by interviewers for every selected case (respondents and non-respondents). These items will include characteristics of the dwelling and of the surrounding area observable by interviewers. To allow for difficult-to-contact people, the overall duration of fieldwork will be a minimum of 30 days and a maximum of 90 days. Quality control back-checks (by telephone or in person) must be carried out and documented in a standardised form on 10% of refusals and 10% of non-contacts. 60 Appendix 5 Survey Specification of a Proposed European Social Survey

Coding and editing The centrally-produced questionnaire will be accompanied by a code-book specifying the codes that must be used by each country. All questions will be pre-coded, except occupation and education, which will be coded either by interviewers or in the office. Occupation and education will be coded for both respondent and spouse, the first to ISCO, the second to an agreed cross- national schema. Each country team will implement a set of range and logic checks that will be specified centrally. Some of these may be implemented via CAPI where applicable. Each country’s ‘clean’ data-file will conform both to the full code-book specification and have ‘passed’ the specified range and logic tests. The data-file will also contain a record for each selected sample case, indicating response outcome plus all interviewer-recorded items referred to above, as well as an appropriate geographical identifier.

Overall timetable The survey period will split into four periods, each of six months’ duration, as follows: Months 1-6: Preparation work, inc. final design, translation and planning Months 7-12: Main field work (intended to be same three months everywhere), followed by supply of clean data to central archive Months 13-18: Archive to prepare a combined, fully-documented data-set Months 19-24: Data-set to be proved and approved by the central methods team prior to its general release.