Dana Centre Front-End Evaluation Report

Naked Science Evaluation of 18 months of Contemporary science dialogue events Science Museum Visitor Research Group May 2004 A project run by And supported by the Contents 1. Introduction 3 2. Background 3 3. Methodologies 4 4. Physical events: Key findings 6 4.1 Audience profile 4.2 Qualities of content that will engender successful dialogue 4.3 Formats for events that will engender successful dialogue 4.4 At an event: Barriers and opportunities to the audience’s engagement 5. Online events: Key findings 12 5.1 Online audience profile 5.2 Barriers and opportunities to the audience’s engagement 6. Appendices 14 a. Statistics from Naked Science events b. Summary of event evaluation conducted c. Developing effective dialogue based Museum events for adults d. Indicators of dialogue – events e. Indicators of dialogue – online f. What is a controversy in science and how can we do controversial events by design? g. Focus group reports h. Event evaluation reports i. E-dialogue: front end research j. Discussion board evaluation k. Dana website evaluation reports l. Evaluation of Dana event training course 2 1. Introduction This report details information and experience gathered from running a pilot series of science dialogue events known as Naked Science . The events took place over a period of 18 months between August 2002 and February 2004. Naked Science has been financed by the Wellcome Trust and run by the Science Museum with the goal of bringing forward this knowledge into the Science Museum’s new events space, the Dana Centre (opened November 2003). A unique opportunity to explore adults-only science dialogue events, the Naked Science series has maintained a commitment to the use of innovative techniques for both evaluation and delivery of its events. Naked Science and the Dana Centre have a target audience of independent adults aged 18 – 45. It is this audience who have been the subject of extensive research through focus groups, interviews and questionnaires looking at the audience’s attitudes, preconceptions, motivations and barriers to engagement in contemporary science dialogue. In addition, all 16 events have been individually evaluated as part of an iterative process that has enabled the series to develop through an increasing understanding of the audience and events over the 18-month period. 2. Background The Dana Centre target audience of independent adults aged 18 – 45 can be further broken down into different elements according to age, gender, professional background and so on. Whilst there are some differences between the attitudes and opinions of these various groups, there is also a significant volume of information that remains consistent across the categories. By conducting questionnaires, interviews, and focus groups with the target audience both before and after attending an event, we have been able to develop a reliable understanding of the needs, wants, and expectations of the Dana Centre audience, its sub groups and entirety. 2.1 Attitudes to science Consultation with the audience has shown that the ambition of engaging audiences in a dialogue on contemporary science issues will inevitably have to tackle the preconceptions and prejudices of this target audience. Most notably, the target audience have overwhelming expressed a cynical attitude towards contemporary science and scientists. Whether from an arts or science background the audience has felt this distrust about both the development of contemporary science and how this is articulated to the public. Coupled with this comes a more personal feeling of powerlessness: ‘contemporary science is progressing independently and nothing I can say or do will change that ’(independent adult in consultation). Clearly these emotional barriers will affect if and how the audience engages with contemporary science issues. But emotional barriers are not the only force at play at this stage. Of less, but nonetheless significant danger to our audience’s engagement is the question of intellectual accessibility. This can be understood as a set of intellectual barriers that prevent a person’s motivation, understanding and subsequent engagement in an issue. Perhaps surprisingly, the problem of inaccessibility is seen in both those with a science or non-science background. In both instances, the audience felt put off from discussing a topic if it was not related to their exact field of study. 3 2.2 Engagement in contemporary issues Knowing the potential difficulty of engaging the Dana Centre target audience in a dialogue about contemporary science issues, it becomes interesting to explore what positive factors about an event might encourage their engagement. Whilst the audience may fear the prospect of complicated content that excludes their involvement, we have discovered that our audience feel more able to approach a topic that is outside their field of expertise when the content is relevant (relevant to the audience members) and concrete (rather than abstract). For example, speakers should have direct experience of the issues concerned and a genuine interest in the particular topic. Outside the framework of events, it is apparent that our audience as a whole do not often engage in contemporary issues. When engagement occurs, it is always in an area strongly relevant to the individuals concerned. For example, religious groups meeting to discuss their religion in connection with particular issues, or an individual taking part in a politically motivated rally because of its relevance to themselves and their society. 3. Methodologies The process of evaluating dialogue events poses a number of problems to the researcher. How do you measure something that lasts for a matter of hours only, and is not numerically quantifiable? What is a measure of dialogue, and how do you know if an event has truly been successful? In answer to these questions, Naked Science events have been assessed using a number of innovative methodologies, enabling the evaluator to assess an event and make recommendations. Using a combination of techniques at each event has ensured that the evaluation is robust and can reflect on several different aspects at one event. For example, establishing the profile of the audience may give statistical information, whilst conducting a detailed observation allows for investigation of the quantity and quality of dialogue that is taking place. The different theoretical models and evaluation methodologies used to research Naked Science events are detailed below. 3.1 Model of an effective event The development of a model that describes what an effective event should look like has proved invaluable in describing the successes and failures of each event. The model was developed as an adaptation of Abraham Maslow’s work on the self-actualisation of an individual in society, and describes 4 stages in a hierarchy that must be attained for an event to be entirely successful (See Appendix f). If all of the stages are reached (at least in part) then there will be genuine dialogue with longer lasting implications for the participants at that event. At each level, a number of measurable factors determine the event’s success. These factors are then used as the basis for evaluation of an event, resulting in a qualitative judgement about specific attributes that should be changed for future events. 3.2 Indicators of dialogue Establishing whether dialogue has occurred at a particular event, and to what extent, cannot rely solely on quantitative data. Whilst this will tell us how many comments have been made, or even, how many comments were made by women and so on, it cannot be used as a sole measure of true dialogue. Dialogue at events has a complex definition, and accordingly the measure of dialogue 4 requires a network of reference points and guides. For this purpose, indicators of dialogue have been developed to provide the evaluator with a qualitative guide to measuring dialogue at an event (See Appendix g). 3.3 Detailed Observation The introduction of detailed observation has allowed us to track dialogue and take note of any barriers to the audience’s engagement whilst the events are taking place. 3.4 Questionnaires Questionnaires provide qualitative information about an event, and have been conducted with the audience and speakers to give the broadest picture. They enable us to uncover a broad spectrum of information such as what participants wanted to change about an event, as well as more subtle information such as what prevented the participants from engaging. Whilst staff working on the night of an event will have some idea of its success, there is nonetheless the potential for discrepancy between their views and the experience of an audience member or speaker. Questionnaires will close this gap and have been conducted using email, the telephone and face- to-face techniques as appropriate to the situation. The use of email questionnaires is a new technique that has proved highly successful over the Naked Science series. Email questionnaires have a high return rate and enable the audience to state their opinions honestly and discreetly. 3.5 Focus groups Focus groups have allowed us to develop an understanding of the motivations, concerns and pre-conceptions of the target audience. A total of nine focus groups have ensured that this information fairly represents the target audience, and covers a range of their views. In total, 5 differing sub-groups of the audience were investigated over the 9 sessions: local art students, members of the black community, local science students, designers, and a Christian group. Conducted before an event, it is possible to establish the barriers and opportunities that are already in place for engaging this audience, whilst meetings after an event have investigated reactions to the events themselves. 3.6 Accompanied surfs In addition to the events themselves, online material has been evaluated using accompanied surfs with the target audience. Participants are asked to visit particular pages of the website and are observed as they navigate, or browse for particular information.

Load more