The medium and the message: Comparing the effectiveness of six methods of

misinformation delivery in an eyewitness memory paradigm

Ciara M. Greene1*, Richard Bradshaw1, Charlotte Huston2 & Gillian Murphy2

1 School of Psychology, University College Dublin, Ireland

2 School of Applied Psychology, University College Cork, Ireland

*Corresponding author. Please send correspondence to [email protected]

Data availability: The data and materials associated with this paper may be found at https://osf.io/vgqw4/.

This paper is currently in press at the Journal of Experimental Psychology: Applied

1 Abstract

Studies of eyewitness memory commonly employ variations on a standard misinformation paradigm. Participants are 1) exposed to an event (e.g. a simulated crime), 2) misled about certain details of the event and 3) questioned about their memory of the original event. Misinformation may be provided in the second step via a range of methods. Here, we directly compared the effectiveness of six misinformation delivery methods – leading questions, elaborate leading questions, doctored photographs, simple narratives, scrambled narratives, and missing word narratives. We presented 1182 participants with a video of a simulated robbery and randomly assigned them to receive misinformation about two out of four critical details via one of these methods. In line with the levels of processing account of memory, we report that methods that encourage deeper processing of misinformation result in more memory distortions. Contrary to previous reports, doctored photographs were not a successful method of implanting misinformation.

The six delivery methods resulted in minimal differences in confidence and metamemory estimates, but participants were more likely to notice the presence of misinformation in the simple narrative condition. We conclude with suggestions for the selection of an appropriate method of misinformation delivery in future studies.

Public Significance Statement

Decades of research has shown that providing an eyewitness with inaccurate information about a crime that they witnessed can distort their memory and future testimony. This study compares the effect of six different methods of delivering this misinformation to an eyewitness. The results of this study will benefit researchers in this field, and provide important evidence about how exposure to different kinds of information after witnessing a crime can influence an eyewitness in the real world.

Keywords: Misinformation; eyewitness memory; levels of processing; leading questions; doctored photographs

2 3 INTRODUCTION

Decades of research have demonstrated that memory for experienced events can be distorted by the provision of post-event misinformation (Frenda et al., 2011). Evidence for this misinformation effect has been demonstrated in a range of experimental paradigms, however a typical study follows a three-part procedure. First, participants are exposed to an initial stimulus – for example, a video or slideshow depicting a crime. Next, misinformation is presented, often in the form of leading questions or a misleading narrative description of the initial event. Finally, the participant is asked to report details of the original event, usually via a questionnaire or interview. Studies employing this paradigm frequently find that participants report elements of the misinformation during the final memory test (Loftus, 2005).

The source monitoring framework (Johnson et al., 1993; Mitchell & Johnson, 2000) provides one account of how misinformation influences memory. According to this model, individuals evaluate the perceptual and semantic characteristics of mental experiences in order to infer the source of the experience. For example, if someone has merely imagined an event, there will often be limited temporal, spatial and semantic information associated with the memory, while the opposite is true of lived experiences. In the context of false memories like those created in misinformation studies, this source monitoring process fails; individuals attribute the semantic, perceptual or emotional characteristics arising from one source to another, incorrect source (Lindsay & Johnson, 2000;

Zaragoza & Lane, 1994). One important determinant of the likelihood of source monitoring errors is the similarity between different sources. Where there is considerable overlap between sources – for example, between a witnessed event and the post-event description of it – source misattributions may be more likely (Mitchell & Johnson, 2000). If the degree to which individuals experience source confusion depends in part on the nature of the post-event source, it becomes important to consider the means by which misinformation is delivered to participants.

4 Misinformation delivery methods

Early studies of the misinformation effect typically used leading questions to deliver post-event information (e.g. Dodd & Bradshaw, 1980; Loftus, 1975; Loftus et al., 1978; Loftus & Palmer, 1974). A notable example of this can be found in a study by Loftus et al. (1978) in which participants viewed a slide showing a car stopped at a Yield sign. Having been exposed to the misleading question, “Did another car pass the red Datsun while it was stopped at the Stop sign?”, many participants later affirmed that the initial slide showed a Stop sign. The aim of these studies was often to demonstrate that hints and assumptions embedded in an investigator’s question could influence the responses of eyewitnesses to real-world crimes; see Loftus (2005) for an overview of much of this research. In other studies, misinformation is delivered via a narrative account of the event that contains misleading descriptions of the initial stimulus (e.g. McCloskey & Zaragoza, 1985). Typically, participants in these studies are instructed to respond to subsequent questions with reference to their memory of the initial stimulus, rather than the post-event narrative, and are sometimes explicitly informed that the narrative may be misleading. Participants who subsequently report memories of having experienced stimuli that were in fact only suggested to them can therefore be considered to have experienced source confusion (Zaragoza & Lane, 1994). Narratives are sometimes presented in the form of newspaper accounts of a crime (e.g. Paterson & Kemp, 2006), or are introduced as the testimony of another witness to the event (Gabbert et al., 2004; Meade &

Roediger, 2002). In some cases, participants are simply required to read or listen to this misleading narrative (Eakin et al., 2003; Echterhoff et al., 2007; Greene et al., 2020; Stark et al., 2010; Zhu, Chen,

Loftus, Lin, He, Chen, Li, Xue, et al., 2010); in others, they are required to engage more deeply with the material, for example by rearranging a scrambled narrative into chronological order (Jaschinski &

Wentura, 2002; Luna & Martín‐ Luengo, 2012; Zaragoza & Lane, 1994, Experiment 2).

5 Altered photographs of a witnessed event or scene are another commonly-used form of misinformation. Photographs provide a visual image of the misinformation, and may therefore have considerable perceptual overlap with the memory of the original event. This source similarity might be expected to produce a greater rate of source misattributions (Johnson et al., 1993; Mitchell &

Johnson, 2000), and indeed, evidence suggests that the closer misinformation becomes to the qualities of a real memory, the more fluently it will be recalled and accepted (Garry & Gerrie, 2005;

Nash et al., 2009; Whittlesea, 1993). Viewing photographs of a scene makes people believe they have been there before when they visit for the first time (Brown & Marsh, 2008), and presenting photographs of an action that participants had only imagined completing resulted in an increase in participants ‘remembering’ having actually completed the action (Henkel & Carbuto, 2008; Nash et al., 2009). Similarly, participants who watched a video of an event and then viewed photographs of the scene that included items that were not present in the original video were more likely to falsely remember having seen those items in the video (Koutstaal et al., 1999; Schacter et al., 1997).

Research suggests misinformation is more likely to be accepted if it comes from a credible source

(Dodd & Bradshaw, 1980; French et al., 2011; Scoboria et al., 2012). Photos are unique compared to other media in their apparent provision of ‘proof’ that the depicted event actually occurred; despite individuals’ awareness of how easily photographs can be altered, they are often viewed as a particularly credible form of evidence (Nash, 2018). An example of this is seen in the so-called

‘truthiness’ effect, whereby the inclusion of a non-probative photograph increases belief in inaccurate information (Fenn et al., 2013; Newman et al., 2012). Recent advances in digital photographic editing technology have allowed researchers to present participants with doctored photographs from their childhood, showing, for example, the participant taking a ride in a hot-air balloon with a relative. These methods have been found to result in a high rate of false memories for the fabricated event (Strange et al., 2008; Strange et al., 2006; Wade et al., 2002). Similar methods

6 have demonstrated the power of doctored images to produce false memories of public events

(Frenda et al., 2013; Nash, 2018; Sacchi et al., 2007).

Perhaps surprisingly then, research has suggested that misleading narratives may produce more false memories than doctored photographs. Garry and Wade (2005) exposed participants to either a doctored photograph or a misleading narrative about a childhood hot-air balloon ride. Results indicated that 50% of those in the doctored photograph condition formed false memories, compared with 82% of those in the narrative condition. The authors of this work have suggested that photographs may constrain imagination and reduce mental elaboration of details of the event, whereas narratives leave elements of the event unspoken and allow room for the participant to generate their own mental image of the event (Garry & Gerrie, 2005; Garry & Wade, 2005). These studies aimed to implant memories of personal childhood experiences. It is less clear how narrative and photographic evidence might influence memory for non-autobiographical events. In one study,

Braun and Loftus (1998) found that participants’ recall of the colour of a chocolate bar wrapper was influenced to an equivalent degree by a misleading verbal description or a photograph depicting the wrapper as blue (rather than green). There is surprisingly little evidence comparing the effect of visual and verbal misinformation on eyewitness memory. There is however evidence that making details easier to visualise (e.g. by providing additional descriptive detail in a question) can increase participants’ tendency to endorse the misinformation. For example, Sharman and Powell (2012) found that participants were more likely to answer in the affirmative when asked, ‘Did the robber hold up the bank with a shotgun that had a black barrel and a dark brown stock?’ than those who were asked the simpler question, ‘Did the robber hold up the bank with a shotgun?’

Levels of processing

In any medium, the depth to which misinformation is processed may influence the subsequent formation of false memories. The ‘levels of processing’ theory (Craik, 2002; Craik & Lockhart, 1972)

7 was developed to account for shortcomings in existing hierarchal memory store models by placing greater emphasis on memory strength as the primary concern of memory retention. This account argues that the strength of a memory trace and its likely retention is determined by the nature of stimulus processing. According to this model, shallow (or structural) processing is concerned only with surface characteristics of stimuli, while deep (or semantic) processing is a more meaningful process whereby a person invests time in elaboration and analysis of the stimulus. This form of processing uses existing knowledge from episodic memory to incorporate meaning into what is being experienced and leaves a longer-lasting and more resilient memory trace. By the same token, misinformation which is processed more deeply might also be expected to result in a stronger false memory.

Research using the Deese-Roediger-McDermott (DRM) paradigm has suggested that deep processing of word lists can increase false recognition of semantically related lure words that were not in fact presented (Rhodes & Anastasi, 2000; Thapar & McDermott, 2001; Toglia, 1999). The DRM paradigm is often used as an experimental measure of false memory generation, however it is important to note that performance on this task is only very weakly related to susceptibility to the misinformation effect (Calvillo & Parong, 2016; Ost et al., 2013; Zhu et al., 2013). There is little direct evidence of the effect of depth of processing on misinformation acceptance, though research does suggest that requiring participants to engage more deeply with the misinformation - for example by forming a mental image of a scene, generating a description of a false item or imagining performing in a fictional action - can increase reports of misinformation and confidence in the false memory (Garry et al., 1996; Goff & Roediger, 1998; Lane & Zaragoza, 2007; Sharman et al., 2004). Similarly, participants who are required to reorder a scrambled narrative make more source misattribution errors than those who simply read an unscrambled narrative (e.g. Zaragoza & Lane, 1994), and several previous studies have used this approach to implant misinformation (e.g. Jaschinski &

Wentura, 2002; Luna & Martín‐ Luengo, 2012). Another method of ensuring deeper semantic

8 processing is to require participants to consider whether a word could fit into a sentence; for example, Craik and Tulving (1975) presented word stimuli (e.g. FRIEND) and asked if the word would fit in the sentence, "He met a _____ in the street". They reported that this technique resulted in deeper processing than structural tasks such as identifying the case in which a word was printed or determining whether it rhymed with another word. Einstein et al. (1984) reported a similar experiment, in which participants who were required to fill in missing letters in a text recalled more of the text than those who simply read the passage for comprehension. Combined with evidence that semantic processing produces illusory memories in the DRM (Gallo, 2010), this suggests that engaging in such a task while reading a misleading post-event narrative might be expected to result in deeper processing of the misinformation, and therefore a higher rate of false memories. To our knowledge this technique has not yet been evaluated in a misinformation paradigm. In the present paper, we therefore include this novel task as another means of evaluating the effect of deep processing. The comparison of each of these methods against a simple, shallowly processed post- event narrative may provide further evidence for the roles of levels of processing in misinformation acceptance.

Comparison of misinformation delivery methods

It is not clear which methods of misinformation delivery are most effective at producing memory distortions. This question is of importance for psychological scientists working in the field of eyewitness memory, who may wish to know which procedures will induce the required effect. It also has applied implications for eyewitness testimony, as it may be important to know which kinds of post-event misinformation are most likely to produce memory distortions in an eyewitness. Several studies have previously compared the effects of misinformation provided by an experimenter (e.g. via leading questions or a written narrative) with misinformation delivered in a social setting, for example via conversation with another witness (Blank et al., 2013; Bodner et al., 2009; Gabbert et al., 2004; Meade & Roediger, 2002; Paterson & Kemp, 2006; Shaw III et al., 1997). Paterson and

9 Kemp also included a condition in which misinformation was delivered via a newspaper account, but found no differences between this method and the leading questions. Notably, these studies all had relatively small sample sizes, rendering their between-subjects comparisons somewhat underpowered. To our knowledge, no studies have directly compared the effects of post-event misinformation delivered via leading questions, written narratives and doctored photographs.

In addition to eyewitness accuracy, another important focus of eyewitness memory research has been eyewitness confidence (Wixted & Wells, 2017). Some studies have suggested that more confident eyewitnesses may be more convincing, so this issue has clear theoretical and applied implications (Cash & Lane, 2020; Leippe et al., 1992). According to the cue-belief model of eyewitness confidence and accuracy judgements (Leippe et al., 2009), eyewitnesses complete two judgements when assessing confidence in their own memory. These include an evaluation of the familiarity of the event (based on features of the memory such as how readily it comes to mind and how detailed it is) and a subjective assessment of the likelihood that the memory is accurate, drawing on factors such as the circumstances under which the event was observed and the witness’s beliefs about their own memory ability (i.e. metamemory). Leippe and colleagues (2009) suggest that memory retrieval that feels faster, easier or richer will be associated with higher confidence.

The ‘levels of processing’ account would suggest that misinformation delivery methods that involve deeper processing of information might increase the strength of misinformation memories (Craik,

2002; Craik & Lockhart, 1972), thus biasing this confidence estimate. Confident witnesses are usually found to be more credible by jurors, even after controlling for accuracy (Brewer & Burke, 2002;

Whitley Jr & Greenberg, 1986). Thus, researchers may have an interest in knowing how misinformation delivery methods are likely to affect participants’ confidence in and beliefs about their own memory.

The present study

10 In the present study, participants were exposed to a video of a simulated robbery. They then received misinformation or control information about four critical details via one of six delivery methods. Finally, participants completed a questionnaire assessing memory for the critical details.

The six delivery methods comprised three written narrative conditions (a simple narrative, a scrambled narrative and a narrative with missing words), two leading questions conditions (simple leading questions and elaborate/detailed leading questions) and a doctored photographs condition.

We aimed to identify the delivery method that produced the largest misinformation effect immediately and after a one week delay. We also investigated the effect of misinformation delivery methods on eyewitnesses’ confidence in their responses to a memory questionnaire, and general beliefs about their own memory abilities. In order to control for participants’ existing beliefs about their memory, these metamemory questions were presented before and after eyewitnesses were exposed to misleading information in different formats. The aim of this analysis is establish whether particular methods of misinformation delivery are more likely to shake a witness’s belief in the accuracy of their own memory, with potential consequences for the witness’s confidence and credibility under later questioning.

Based on the literature outlined above, we predicted that participants would report more misinformation after being presented with elaborate leading questions relative to simple leading questions. We also predicted higher rates of false memory after reading a narrative that requires participants to place the sections of a narrative in chronological order or fill in missing words, relative to simply reading a narrative. Engaging with the narrative in this way may encourage deeper processing of the misleading material and thus might be expected to enhance the incorporation of misinformation into memory. Finally, we expected that forms of delivery in which the misinformation was easier to visualise – i.e. the elaborate leading questions and doctored photos conditions – would result in more false memories. We did not preregister any hypotheses regarding eyewitness confidence and metamemory, however the cue-belief model (Leippe et al., 2009)

11 outlined above might lead us to predict that delivery methods that encourage deeper processing might result in higher confidence ratings and metamemory estimates. Thus, a smaller reduction in memory estimates following exposure to misinformation may be expected in the missing-items and scrambled narrative conditions.

Open-ended questions, in which participants are invited to actively recall details of the event (e.g.

“describe what the thief was wearing”), tend to result in fewer reports of misinformation than closed questions in which participants are encouraged to choose from a limited number of options

(e.g. “was the thief’s jacket black or brown?”). It has been suggested that this is because open questions are more likely to elicit ‘don’t know’ responses (Brubacher et al., 2020; Sharman & Powell,

2012; Waterman et al., 2001). We therefore counterbalanced the provision of closed and open questions across experimental conditions.

Individual differences in a range of factors, including age, working memory capacity, autobiographical memory specificity and personality, have been associated with variation in susceptibility to misinformation (Farina & Greene, 2020; Gerrie & Garry, 2007; Greene & Murphy,

2020; Jaschinski & Wentura, 2002; Roediger III & Geraci, 2007; Watson et al., 2005; Zhu, Chen,

Loftus, Lin, He, Chen, Li, Moyzis, et al., 2010). Of particular relevance to the present study, research has suggested that people with higher levels of intelligence or cognitive ability may be less susceptible to misinformation (Greene et al., 2020; Murphy et al., 2019; Zhu, Chen, Loftus, Lin, He,

Chen, Li, Xue, et al., 2010). We therefore controlled for cognitive ability in a secondary analysis by including scores on the Wordsum, a subtest of the Wechsler Adult Intelligence Scale which is highly correlated with full-scale IQ (Huang & Hauser, 1996).

Researchers and those working in applied contexts may also be interested in knowing whether the effects of misinformation are short-lived or persist over time. To that end, we included a follow-up assessment to compare the effect of each of the misinformation delivery methods after a one-week

12 delay. We did not preregister any specific hypotheses regarding this one-week follow-up, but previous literature suggests that the misinformation effect is likely to persist, or may even increase – a phenomenon known as the “sleeper effect” (Frost, 2000; Frost et al., 2002; Underwood & Pezdek,

1998). As with the initial assessment, we will investigate whether deeper processing is associated with a larger misinformation effect.

Finally, researchers may also wish to know whether the purpose of their experiment is transparent to participants, and whether the misinformation delivery method used affects the probability that participants will detect the presence of misinformation. To investigate this, we included an open- ended question at the end of the study in which participants were asked to indicate what they believed the purpose of the experiment to be. In an exploratory analysis, we compared the proportion of participants who were alerted to the presence of misinformation across the six delivery methods.

13 METHODS

Participants

The study was completed by 1182 participants before our preregistered stopping date of 30 th April

2020, very close to our target of 1200. Nineteen participants reported being under the age of 18 and were therefore excluded in line with ethical restrictions. The final sample (N = 1165) had a mean age of 29.80 years (SD = 12.78, range = 18-74) and included 856 females (73.5%), 297 males (25.5%) and five participants (0.4%) who reported their gender as ‘other’. Participants were recruited through social media, email requests, survey sharing websites and snowball sampling, and were not compensated for their participation.

Preregistration and design

The research design and analysis was preregistered at https://aspredicted.org/tm8rt.pdf and the study protocol was approved by the Human Research Ethics Committee at University College Dublin.

This study employed a 6 (misinformation delivery method) x 2 (misleading/control) mixed between- within-subjects design. Participants were randomly assigned to one of six misinformation delivery methods (simple leading questions, elaborate leading questions, simple narrative, scrambled narrative, missing-words narrative and doctored photos,). Post-event information about four critical details was presented to each participant; participants received misinformation about two items and neutral information about the other two in a counterbalanced manner (see materials for further details). Thus, the effect of misinformation was assessed within subjects, and each critical item acted as its own control.

Power analysis using G*Power 3.1 (Faul et al., 2007) indicates that the final sample of 1165 provides

95% power to detect a small effect (f = .09) in the within/between-subjects interaction.

14 Materials

Video

The video was just over 2 minutes long, and was presented without sound. The video begins with a young woman pushing a small child in a pram down the footpath of a suburban street. She is stopped by an older lady holding a map, who appears to ask for directions. The young woman takes out her phone to help the older woman, and then continues on her way. She approaches a pedestrian crossing, and starts to cross the road against the lights; the green traffic light is clearly visible at the top of the frame for approximately seven seconds. A cyclist approaching the crossing stops abruptly and just avoids hitting her, and an argument ensues. While they argue, an elderly man crosses the road at the pedestrian crossing and walks behind the arguing pair. Shortly after, a middle-aged man enters the frame from behind the camera. He walks behind the woman for a short distance, before turning back, taking her handbag from the top of her pram and walking away toward the camera. The video cuts to a close-up of the thief looking through the bag and stealing a wallet and phone, which he puts in his pocket. The woman then realizes that her bag has been stolen and gives chase. The thief runs past another woman leaving a nearby park, and discards the stolen bag. The victim recovers her bag and converses with witnesses. The full video may be viewed at https://osf.io/vgqw4.

Post-event misinformation

Information about four critical items was manipulated across six methods of misinformation delivery. The critical items were 1) the colour of the traffic light; 2) the nature of the thief’s interaction with a witness; 3) the thief’s presence on the pedestrian crossing and 4) the items stolen by the thief. Two versions of each misinformation delivery method were created, each of which was designed to implant misinformation about two out of the four critical items. Version A suggested that the traffic light was red as the woman began to cross the road (when it was in fact green) and that the thief bumped into a woman leaving the park as he ran away (in fact he had no interaction

15 with her). Version B suggested that the thief crossed the road at the pedestrian crossing before stealing the bag (when in fact he entered the frame from behind the camera after another man crossed the road) and that the thief stole car keys as well as the wallet and phone. Thus, each version contained one piece of misinformation about a critical detail from the first half of the video and one from the second half. Neutral information was provided about the two control items in each version so that the correct answer could not be deduced from the post-event information. Three of the pieces of misinformation (the traffic light, the interaction with the witness and thief’s presence on the pedestrian crossing) directly contradicted the video, while the fourth (the stolen car key) added an extra item to the actual stolen items (wallet and phone).

Six methods of post-event misinformation delivery were developed, as follows:

1. Simple leading questions. An 8-item questionnaire which incorporated the four critical items

(see Table 1) and four filler items. Misinformation about two of the critical items was

embedded in the questions, e.g. “What speed do you estimate the cyclist was moving at

when he approached the red light?”. Filler items asked about the colour of the young

woman’s coat, what the older woman was holding when she approached the young woman,

what the young woman did when she realized her bag had been stolen and whether the

participant saw the thief throw the bag away. No instructions were provided prior to

administration of the leading questions.

2. Elaborate leading questions . An 8-item questionnaire, including four critical and four filler

items. The content of the questions varied from the simple questions only in that more

descriptive detail was provided in each question, for both the critical and filler items; e.g.

“What speed do you estimate the cyclist was moving at when he approached the red light

beside the pedestrian crossing and braked hard to avoid hitting the buggy?” No instructions

were provided prior to administration of the leading questions.

16 3. Simple narrative . A 15-sentence description of the video written in the present tense and

presented in a single paragraph. Two critical items were misleadingly described in each

version of the narrative, e.g. “A cyclist ignores a red light and just avoids bumping into her

buggy”. Participants were simply instructed to read the narrative.

4. Scrambled narrative . The same narrative as in the simple narrative condition, with the 15

sentences presented in list format in randomised order. Participants were instructed to drag

and drop the sentences to rearrange them into a sequence that best reflected the events of

the video.

5. Missing words narrative . The 15-sentence narrative was presented in a single paragraph,

with eight words replaced by a blank space and the numbers 1-8. Participants were

instructed to type the missing word into a corresponding numbered box. Words were

omitted from sentences describing the four critical details and four additional details. The

missing words could be easily inferred from the context and did not require detailed

memory of the scene, however the words were selected such that correct responses

required engaging with the misinformation; e.g. “A cyclist ignores a red traffic ( 4 ) and just

avoids bumping into her buggy”.

6. Doctored photographs . A series of ten photographs depicting the events of the video.

Photographs of the critical details were altered in Adobe Photoshop to change the colour of

the traffic light, place a car key in the thief’s hand, overlay the thief’s face over that of the

elderly man on the pedestrian crossing, and place the fleeing thief in direct contact with the

woman exiting the park. Participants viewed altered images for two of the four critical

details. Participants were instructed to take a minute to look at the slides below to remind

them of the events of the video. Photographs were presented singly and participants were

allowed to move through the slideshow at their own pace.

17 Screenshots of the four critical items as seen in the original video are presented in Figure 1. Details of the misinformation and control information provided to participants in each written form of misinformation delivery are provided in Table 1, and the altered and control images from the doctored photos condition may be seen in Figure 2. All study materials may be viewed online at https://osf.io/vgqw4.

Figure 1. Screenshots from the video, showing all four critical details: a) the woman approaches the pedestrian crossing as the traffic light is green; b) the thief enters the frame from behind the camera before turning back to steal the handbag; c) the thief steals a wallet and phone from the bag; d) the thief runs away, past a woman exiting the park.

18 Figure 2. Altered and control images presented in the ‘doctored photos’ condition. Altered images depict a) a red traffic light, b) the thief stealing a car key from the bag, c) the thief crossing the road at the pedestrian crossing and d) the thief bumping into a woman as he flees the scene.

19 Table 1. Misleading and control descriptions of the critical details in each of the written forms of misinformation delivery. Misinformation relating to the traffic light and interaction with the witness was presented in Version A; misinformation relating to the pedestrian crossing and the stolen items was presented in Version B.

Critical detail Misleading Control Simple leading questions Traffic light What speed do you estimate the cyclist was moving at What speed do you estimate the cyclist was moving at when he approached the red light? a when he approached the traffic light? b Interaction with witness As the thief ran away, he bumped into a woman As the thief ran away a woman was walking out of the walking out of the park; what colour was her jacket? a park; what colour was her jacket? b Pedestrian crossing Do you think the woman saw the thief cross the road Do you think the woman saw the thief before he stole before he turned back to steal her bag? b her bag? a Stolen items After the thief took the items from the handbag, did he After the thief took the items from the handbag, did he put the stolen car key in his pocket or keep it in his put them in his pocket or keep them in his hand? a hand? b Elaborate leading questions Traffic light What speed do you estimate the cyclist was moving at What speed do you estimate the cyclist was moving at when he approached the red light beside the when he approached the traffic light beside the pedestrian crossing and braked hard to avoid hitting pedestrian crossing and braked hard to avoid hitting the the buggy? a buggy? b Interaction with witness As the man ran past the entrance to a park, there were As the man ran past the entrance to a park, there were a a number of people standing around. The thief bumped number of people standing around. One female witness into one woman as she walked out of the park gates. spoke to the woman as she retrieved her bag. What What colour was her jacket? a colour was her jacket? b Pedestrian crossing The woman was distracted by her argument with the The woman was distracted by her argument with the

20 cyclist as the thief crossed the road at the pedestrian cyclist as the thief walked by and stole her bag. Do you crossing before turning back to steal her bag. Do you think she saw him walk past before he took it? a think she saw him before he took it? b Stolen items The thief rummaged in the stolen shoulder-bag and The thief rummaged in the stolen shoulder-bag and removed a number of items, one at a time. Did he put removed a number of items, one at a time. Did he put the stolen car key in the front pocket of his black these items in the front pocket of his black jacket, or jacket, or keep it in his hand? b keep them in his hand? a Simple and scrambled narratives Traffic light A cyclist ignores a red light and just avoids bumping A cyclist is approaching the pedestrian crossing and just into her buggy. a avoids bumping into her buggy. b Interaction with witness After bumping into a woman leaving a nearby park, he He runs past a woman leaving a nearby park and discards discards the stolen bag. a the stolen bag. b Pedestrian crossing As they argue, a man crosses the road at the As they argue, a man walks past before changing pedestrian crossing and walks past, before changing direction and walking back towards the arguing pair. direction and walking back towards the arguing pair. Noticing that the woman is distracted he steals her bag. a Noticing that the woman is distracted he steals her bag.

b Stolen items We see him take some items from the bag and put the We see him take some items from the bag. a stolen car key in his pocket b Missing words narrative Traffic light A cyclist is approaching the pedestrian (____) and just A cyclist ignores a red traffic (___) and just avoids avoids bumping into her buggy. b bumping into her buggy. a

Interaction with witness He runs past a (___) leaving a nearby park and discards After bumping into a (___) leaving a nearby park, he the stolen bag. b

21 discards the stolen bag. a

Pedestrian crossing As they argue, a man crosses the road at the As they argue a man walks past before changing pedestrian crossing and walks past, before changing (___) and walking back towards the arguing pair. a (___) and walking back towards the arguing pair. b Stolen items We see him take a wallet, (___) and keys from the bag. a b We see him take some items from the (___). a Presented in Version A b Presented in Version B

Note: Misinformation is highlighted in bold for illustrative purposes. Bold font was not used in the information presented to participants.

22 Memory questionnaire

The memory questionnaire consisted of four items assessing memory for the critical details (to evaluate effects of misinformation) and four items assessing memory for non-critical items (to evaluate memory accuracy), as well as an open-ended question inviting a description of the thief and an assessment of the participant’s confidence in their responses (see Table 2). The first eight questions were presented in randomized order, followed by the description of the thief and confidence question. Half of the items were presented as closed questions offering a binary choice

(e.g. “was the traffic light red or green?”) while the other half were presented as open/unrestricted questions (e.g. “what did the thief take from the bag?”). Responses to all questions were provided via a text box.

Table 2. Memory questionnaire presented at both immediate and follow-up test

Questionnaire item Correct answer Critical items When the woman started to cross the road, was the traffic light Green facing the cyclist red or green? Did the thief have any interaction with the woman in the brown None; he ran past her jacket who was leaving the park? If so, please describe the interaction. Did the thief cross the road at the pedestrian crossing before No stealing the bag? What did the thief take from the bag? Please list everything. Wallet and phone

Non-critical items What colour scarf was the woman who asked for directions Yellow wearing? 1 Did the cyclist bump into the buggy when he stopped at the No pedestrian crossing? 1 What kind of trousers was the mother wearing? 1,2 Black leather Was there anyone in the parked red car that the thief ran past No after stealing the bag? 1,2 What colour coat was the woman who asked for directions Black wearing? 2

23 What did the thief do with the bag after removing items from it? Threw it in a bush

2

Other questions Please describe the thief’s appearance in as much detail as you - can remember How confident are you in your responses to this questionnaire? - (1 = Not at all confident, completely guessing; 9 = extremely confident) 1 Presented at immediate test

2 Presented at follow-up test

Metamemory questions

Two metamemory questions were presented before the provision of misinformation, and again after the memory questionnaire. The first asked “If 100 people watched this video and we assessed how accurate their memory was, how well do you think you would fair?”. Response options were “Better than 95% of people”, “Better than 75% of people”, Better than 50% of people”, “Better than 25% of people” and “Among the worst eyewitnesses”. The second question asked participants to consider the entire video and report what percentage of details they believed they could remember, on a sliding scale from 0 to 100%.

Wordsum

The Wordsum (Thorndike and Gallup, 1944) is a 10-item vocabulary test which forms part of the

Wechsler Adult Intelligence Scale and is often used as a proxy for general intelligence (e.g. in the US

General Social Survey; Meisenberg, 2015). Wordsum score is highly correlated with full-scale IQ

(Huang & Hauser, 1996). Participants are presented with a target word and asked to choose the word closest in meaning to the target from a list of 5 other words. For example, given the target word CLOISTERED, participants must choose the closest match from the words ‘miniature’,

‘bunched’, ‘arched’, ‘malady’ and ‘secluded’. They may also select the option ‘don’t know’.

24 Procedure

The experiment was conducted online via the survey platform Qualtrics. Participants were informed that the purpose of the study was to establish how accurately individuals could remember a short video. After providing consent, participants were instructed to watch the video once before moving onto the next page. Participants then provided demographic information and answered the first set of metamemory questions. Post-event misinformation was then presented; participants were randomly assigned to view version A or B of one of the six misinformation delivery methods described above.

Participants then completed the Wordsum test; participants were simply informed that we wanted to know how people go about guessing words they do not know, but were not informed that this test would be used as a measure of cognitive ability. Participants were then instructed to recall the video they saw the start of this experiment, and were told, “You will now be asked some questions about the content of that video. Please answer as accurately as you can”. The memory questionnaire was then presented. The two metamemory questions were presented once more, before participants were asked to indicate via an open text box what they believed the purpose of the experiment to be. Finally, participants were asked whether they were willing to be contacted to complete a brief follow-up survey; those who consented were asked to provide their email address, while those who did not were fully debriefed as to the true purpose of the experiment. Participation took approximately 15 minutes in total.

Follow-up surveys were sent via email seven days after completion of the original experiment. A 10- item questionnaire was presented, including the four original critical items and four filler items. Two of the filler items were identical to those presented at immediate test, and two were novel (see

25 Table 2). These eight questions were presented in random order, followed by a request for a description of the thief and an estimate of confidence, as in the immediate test.

26 RESULTS

Data coding and scoring

Responses to the critical items were coded for the presence or absence of misinformation and responses to the non-critical items were coded as correct or incorrect. The proportion of participants reporting misinformation was significantly higher for all four critical items in the misled condition

(mean = 31.67%) than the control condition (mean = 14.45%); see supplemental materials for an item-by-item analysis.

Each participant received a misinformation score of between 0 and 2 for their responses to the critical items about which they were misled (narrative A: the traffic light and the interaction with the witness; narrative B: the pedestrian crossing and the stolen items) and an score of between 0 and 2 for the other two critical items about which they were not misled. Participants also received a true memory score between 0 and 4 for their responses to the four non-critical items. Coding was performed by two independent raters, and disagreements were resolved by consensus. Initial inter- rater reliability was very high (kappa = 1 for all critical items and between .91 and .99 for non-critical items). Further analysis of the non-critical items may be found in supplemental materials.

Participants’ responses to the final question about the purpose of the experiment were also coded for any indication that participants had detected the presence of misinformation. Any mention of incorrect details provided in the questions/narratives/photographs, or allusions to efforts to influence or alter eyewitness memory were coded as awareness of the misinformation. Responses that simply referred to memory (e.g. “testing recall”, “seeing how many details people can remember”) were not coded as indicating awareness. Coding was performed by two independent raters, with initial inter-rater reliability of .77. Disagreements were resolved by a third coder. The full dataset may be viewed online at https://osf.io/vgqw4.

27 Misinformation effect: Immediate recall

Misinformation rates across conditions can be seen in Figure 3. A 6 x 2 mixed ANOVA was conducted to investigate the number of pieces of misinformation reported as a function of misinformation delivery method and condition (misled/control). A main effect of condition was observed (F(1,1158)

2 = 219.32, p < .001,  p = 0.16), such that more misinformation was reported in the misled condition

(M = 0.61 pieces of misinformation, SE = .05) than in the control condition (M = 0.27, SE = 0.01). A

2 main effect of misinformation delivery method was also observed, (F(5,1158) = 2.88, p = .01,  p =

2 0.01), along with an interaction effect (F(5,1158) = 8.02, p < .001,  p = 0.03). Pairwise comparisons demonstrate a significant misinformation effect (p < .001) in all delivery methods except the doctored photographs (p = .39). See Figure 3 for an illustration, and supplemental materials for full test statistics. Tukey post hoc tests revealed that the quantity of misinformation reported about misled items in the doctored photos condition (M = 0.39, SD = 0.55) was significantly less than that reported in all other conditions (p < .001), except the simple narrative condition (M = 0.56, SD =

0.69; p = .15). The highest rate of misinformation was reported in the missing items narrative (M =

0.76, SD = 0.71); this score was significantly different to that observed in the simple narrative condition (p = .03) and doctored photos condition (p < .001), but did not differ from the other conditions (see supplemental materials for details). The magnitude of the difference between the misinformation and control condition was largest in the missing items narrative (d = 0.60) and smallest in the doctored photos condition (d = 0.05), followed by the simple narrative condition (d =

0.35). As outlined in our preregistration, this analysis was repeated with the inclusion of Wordsum

2 score as a covariate. No main effect of Wordsum was observed (F(1,1157) = 0.15, p = .70,  p = 0.00), and there was no interaction between Wordsum score and any other variable. The pattern of results was very similar to that observed in the initial ANOVA; the full analysis may be seen in supplemental materials.

28 Figure 3. Mean number of pieces of misinformation reported in the misled and control conditions at immediate recall for each misinformation delivery method. Effect sizes (Cohen’s d) are provided for the difference between the misled and control condition within each delivery method. Error bars represent 95% confidence intervals. *p < .05

Misinformation effect: Follow-up questionnaire

In total, 556 participants (46.9%) consented to be contacted for follow-up one week after the original study. Of these, 346 participants (29.69% of the original sample) completed the follow-up questionnaire. Returning participants were equally distributed across the six misinformation delivery methods; N in each condition ranged from 53 to 63. On average, participants reported more misinformation at follow-up after one week (M = 0.73 pieces, SD = 0.72) than during the immediate recall test (M = 0.61 pieces, SD = 0.67; t (345) = 3.22, p = .001, d = 0.17). Additional analysis revealed that this effect only reached significance for the simple narrative and missing items narrative conditions (see supplemental materials for details).

29 A 2 (misinformation condition: misled/control) x 6 (delivery method) ANOVA revealed a main effect

2 of misinformation condition (F(1,340 = 54.85, p < .001,  p = 0.14), such that, after one week, participants in the misled condition reported more pieces of misinformation (M = 0.73, SE = 0.04) than participants in the control condition (M = 0.38, SE = 0.03). Item-by-item analysis of the four critical items may be found in supplemental materials; a significant misinformation effect was observed for the items relating to the interaction with the witness, the pedestrian crossing, and the stolen items, but not for the item about the colour of the traffic light. A main effect of delivery

2 condition was also observed, F(5,340) = 5.44, p < .001,  p = 0.07. Post hoc tests indicated that both the simple narrative (M = 0.69, SE = 0.06) and missing-items narrative (M = 0.71, SE = 0.06) were associated with a higher rate of misinformation than either the simple leading questions condition

(M = 0.39, SE = 0.06) or doctored photos condition (M = 0.41, SE = 0.06). See supplemental materials for full details of post hoc analyses. No significant interaction effect was observed, F(5,340) = 1.71, p

2 = .13,  p = 0.025, but note that, as a result of the high rate of attrition, this analysis was relatively underpowered compared to the analysis of immediate responses.

Figure 4. Mean number of pieces of misinformation reported in the misled and control conditions at one-week follow-up for each misinformation delivery method. Effect sizes (Cohen’s d) are provided

30 for the difference between the misled and control condition within each delivery method. Error bars represent 95% confidence intervals. *p < .05

A significant misinformation effect was observed for all delivery methods except the simple leading questions and the doctored photo condition (see supplemental materials). The magnitude of the misinformation effect in each delivery condition can be seen in Figure 4. As at immediate test, the largest effects were observed in the scrambled narrative and missing-items narrative conditions (d =

0.54).

Eyewitness confidence and metamemory

A one-way ANOVA was conducted to examine the effect of misinformation delivery method on participants’ confidence in their responses to the initial questionnaire; see Figure 5A. A significant

2 main effect was observed (F(5,1116) = 2.85, p = .01,  p = .013). Post hoc Tukey tests revealed higher confidence in the scrambled narrative condition (M = 5.67, SD = 1.55) compared with the simple narrative (M = 5.13, SD = 1.65) and doctored photos conditions (M = 5.11, SD = 1.70). See supplemental materials for full details of post hoc tests. Confidence was not significantly correlated with the quantity of misinformation reported in either the misled (r = -.02, p = .46) or control conditions (r = -.05, p = .12).

A 2 (pre/post assessment) x 6 (delivery method) ANOVA compared participants’ estimate of the percentage of details of the scene they could remember; estimates declined from 60.67% (SD = 0.57) before the presentation of misinformation to 49.03% (SD = 0.61) afterwards (F(1,1112) = 534.30, p

2 2 <.001,  p = .33). No main effect of delivery method was observed (F(5,1112) = 1.98, p = .08,  p

2 = .01), but there was a significant interaction effect (F(5,1112) = 2.53, p = .03,  p = .01). Examination of Figure 5B reveals an ordinal interaction whereby the effect of time was observed with all delivery methods, but a larger effect is observed for some (e.g. elaborate leading questions and doctored photos) than for others (e.g. scrambled narrative). Post hoc paired samples t-tests comparing the

31 pre- and post-assessment metamemory estimates in each misinformation delivery condition may be found in supplemental materials. Participants also reported the percentage of participants they believed they would outperform on a 5-point scale. A main effect of time was observed, F(1,1117) =

2 303.04, p <.001,  p = .21; participants’ belief in their own performance declined after completing the experiment (before test: M = 3.2 out of 5, SD = 0.02; after test: M = 2.79, SD = 0.03). There was

2 no main effect of misinformation delivery method (F(5,1117) = 1.20, p = .31,  p = .005) and no

2 interaction effect (F(5,1117) = 1.36, p = .24,  p = .01); thus, self-reported relative accuracy was reduced following completion of the experiment at a roughly equal rate in every condition (see

Figure 5C). Taken together, these results suggest that participants confidence in their ability to remember the details of the event under investigation was differentially affected by the misinformation delivery method, but their broader beliefs in their own memory abilities were not.

32 Figure 5. Mean responses to confidence and metamemory questions. (A) Mean confidence in responses to the questionnaire; (B) Mean percentage of details from the video that participants believed they could remember, before and after completion of the memory questionnaire; (C)

Participant’s estimate of their accuracy relative to other people, before and after completion of the memory questionnaire. Effect sizes for the difference between pre- and post-assessment estimates are provided in Cohen’s d. Error bars represent 95% confidence intervals.

33 Awareness of misinformation

Of the 1066 participants who provided a response to the question about the purpose of the experiment, only 71 (6.7%) indicated any awareness that misinformation was presented as part of the experiment. Averaging across all six conditions, participants who detected the presence of misinformation reported fewer pieces of misinformation in the memory questionnaire (M = 0.46, SD

= 0.58) than those who did not (M = 0.64, SD = 0.69; t (1064) = 2.15, p = .03, d = 0.28). There was no difference in the number of non-critical items correctly reported by those who did (M = 0.25, SD =

0.50) or did not report the presence of misinformation (M = 0.28, SD = 0.48; t(1064 = 0.49, p = .63, d

= 0.06), indicating that the effect of awareness was specific to memory for misinformation. The percentage of participants who detected the misinformation ranged from 2.2% in the doctored photos condition to 11.6% in the simple narrative condition, and a chi square analysis indicated that the rate of awareness differed significantly between conditions (2 (5) = 14.14, p = .01, V = 0.11).

Post hoc z-tests compared the proportion of participants reporting awareness in each condition with every other condition. See Table 3 for details.

Table 3. Number and percentage of participants reporting awareness of the misinformation across the six delivery methods

Delivery Method N responses N (%) reporting awareness

Simple leading questions a,b 171 9 (5.3%)

Elaborate leading questions b,c 184 14 (7.6%)

Simple narrative c 181 21 (11.6%)

Scrambled narrative a,b,c 171 10 (5.8%)

Missing-items narrative b,c 176 13 (7.4%)

Doctored photographs a 183 4 (2.2%) Note: Each subscript letter (a, b, c) denotes a subset of delivery methods for whom the percentage reporting awareness does not differ significantly from one another at the .05 level.

34 DISCUSSION

Research on the topic of misinformation has proliferated in recent decades, with a wide range of methodologies regularly employed. As a result, researchers may not be sure which method of post- event misinformation delivery is best suited to their needs. Here, we present a direct comparison of six different methods. In general, our results are in line with a levels of processing account of memory, and partially support the hypothesis that misinformation that is easier to visualise will be more easily recalled.

Levels of processing

As expected, we found that methods that encourage deeper semantic processing of the misinformation produce higher rates of misinformation reporting at a subsequent test. Participants who were asked to place the elements of a scrambled narrative in chronological order, or who had to use context clues from the narrative to fill in missing words, reported more misinformation than participants who simply read the same narrative. This supports previous research suggesting that depth of processing is a critical factor in misinformation acceptance (Garry et al., 1996; Goff &

Roediger, 1998; Lane & Zaragoza, 2007; Sharman et al., 2004; Zaragoza & Lane, 1994). Importantly, these effects were observed without a corresponding decrease in accuracy in the control condition, suggesting that the effect is specific to misinformation. Of the written methods employed, the simple narrative was the least effective at immediate test, despite being one of the most commonly used approaches in experimental research (e.g. Eakin et al., 2003; Echterhoff et al., 2007; Greene et al., 2020; Stark et al., 2010; Zhu, Chen, Loftus, Lin, He, Chen, Li, Xue, et al., 2010). A clear “sleeper effect” (cf. Underwood & Pezdek, 1998) was however observed after one week, whereby participants in the simple narrative condition reported a higher rate of misinformation following a delay. This effect is thought to come about as the details of the original event fade over time, and participants become more prone making source-monitoring errors by confusing the post-event

35 misinformation with the original event (Frost et al., 2002). A similar effect was observed in the scrambled narrative and missing-items narrative conditions (though this did not quite reach significance in the missing-items condition). These conditions continued to show the strongest misinformation effect after one week, providing additional support for the levels of processing account outlined above.

Simple vs. elaborate leading questions

Previous research has suggested that including additional detail in a leading question may enhance the misinformation effect (Sharman & Powell, 2012). In the present study, both simple leading questions and elaborate leading questions resulted in significant misinformation effects, however contrary to our expectations, the effect size in both conditions was very similar at immediate test, and the quantity of misinformation reported in the two conditions did not differ significantly. After a one week delay however, the misinformation effect was no longer significant in the simple leading questions condition, and the effect size in the elaborate leading questions condition was more than double that in the simple condition. It is important to note that this follow-up analysis was relatively underpowered, and that the simple leading questions did have a moderate continued effect on memory. Nevertheless, the more detailed questions – which may aid visualisation of the misinformation – do appear to have elicited a longer-lasting effect.

Doctored photographs

In contrast with previous research on the topic, (e.g. Braun & Loftus, 1998; Henkel & Carbuto, 2008;

Koutstaal et al., 1999; Nash et al., 2009; Schacter et al., 1997), the doctored photographs employed in the present study did not elicit a higher rate of misinformation relative to viewing control

(unaltered) photographs of the scene. In order to ensure a match with other conditions in the experiment, participants were allowed to move through the slideshow at their own pace and were not required to engage in any depth with the photographs. This null effect may therefore be partially

36 explained by a levels of processing account (Craik & Lockhart, 1972), however other self-paced conditions (such as the simple narrative) did produce significant misinformation effects. It is also possible that, in viewing the doctored images, participants did not focus on the altered elements of the image but instead attended to other features of the scene; for example, they may not have noticed the red traffic light at the top of the image if they were focussed on the arguing couple below. Finally, participants may have consciously discounted the misinformation presented in the doctored images if they detected the alterations and realised that they were being provided with inaccurate details. However, Nash (2018) showed that even very badly doctored photographs can produce memory distortions (e.g. an image of a crowd scene with a Photoshopped police horse whose feet don’t touch the ground). Moreover, when asked what they believed the experiment to be about, only 2.2% of participants in the doctored photographs condition suggested that the experiment may have included misinformation. Of these, only one participant explicitly reported anything relating to image alteration: ”I think it had something to do with false memories, as one of the images shown showed the robber bumping into the woman exiting the park, whilst I distinctively remember the woman who was robbed hitting her”. No other participants in this condition indicated any awareness that the images had been doctored, and overall this condition was associated with the lowest rate of misinformation detection in the experiment. These results suggest that doctored photos may not be a useful method of implanting misinformation. Our findings stand in contrast to the predictions of the source monitoring framework, which holds that significant overlap between sources – e.g. in the case of a video and screenshots from that same video – should result in increased source monitoring errors and therefore higher false memory rates (Mitchell & Johnson,

2000). However, given that doctored photographs have been repeatedly shown to influence memory in other studies (e.g. Frenda et al., 2013; Nash, 2018; Sacchi et al., 2007; Strange et al.,

2008; Strange et al., 2006; Wade et al., 2002), further evidence is required to examine the utility of doctored photographs under a range of different conditions.

37 Effects of cognitive ability

We found no main effect of verbal cognitive ability, in contrast with literature suggesting that individuals with higher levels of intelligence or cognitive ability may be less susceptible to misinformation (Greene et al., 2020; Murphy et al., 2019; Zhu, Chen, Loftus, Lin, He, Chen, Li, Xue, et al., 2010). The misinformation effects described above were preserved after cognitive ability was controlled for and there was no interaction between verbal cognitive ability and any of the misinformation conditions. This suggests that these misinformation delivery methods are appropriate for use with diverse samples and susceptibility is not predicted by cognitive ability.

Confidence and metamemory

Participants did not appear to have any particular insight into their own performance; confidence was not significantly correlated with misinformation acceptance, and the metamemory questions revealed a reduction in perceived accuracy post-test, across all delivery methods. Though the delivery methods used in this study placed different levels of demand on participants (e.g. looking at some images vs. constructing a chronological narrative), only very small differences in confidence were observed. In line with the cue-belief model (Leippe et al., 2009), the scrambled narrative – which was designed to encourage deeper processing of the misinformation – did result in a slightly higher level of confidence and slightly more positive post-test metamemory estimates. However, no such effect was seen for the missing items narrative which also induced deeper processing, reflected in a higher rate of misinformation reporting. In general, the effects of delivery method on confidence and metamemory were small, suggesting that the medium by which misinformation is imparted to participants has only small effects on participants impression of their memory for the specific task, and minimal impact on their perceptions of their memory ability in general. Of note however, we evaluated confidence with reference to the whole 10-item questionnaire and not for individual

38 items. Thus, it is possible that a more sensitive measure, perhaps focussed on individual critical details, might reveal stronger effects.

Choosing a method of misinformation delivery: guidance for memory researchers

Researchers choose methods of misinformation delivery for a variety of reasons. In some cases, the intention is to replicate real-world interrogation conditions as closely as possible. In others, the aim is to investigate situational or individual factors that predict susceptibility to misinformation. In the latter case, researchers may be eager to ensure that a sufficient quantity of misinformation is reported to allow comparisons between individuals or experimental conditions. In cases such as these, we recommend the use of scrambled narratives or narratives with missing words to ensure sufficiently deep processing of the material. The simple narrative format may be sufficient when the intention is to implant misinformation over the longer term, though these findings are tentative as the follow-up sample was comparatively small and we only assessed memory at a single follow-up session after one week. Researchers who are more concerned with ecological validity may wish to rely on a leading questions format, as this approach mimics a situation commonly encountered by eyewitnesses. Our data provide additional evidence that elaborate or detailed leading questions may be more effective than simple leading questions, especially when memory is to be tested after a delay. On the basis of the results described here, we cannot recommend the use of doctored photographs to implant misinformation in eyewitness memory paradigms.

Awareness of misinformation

By design, misinformation studies entail deceiving participants as to the true purpose of the experiment. Specifically, it is usually essential to the integrity of the study that participants should not be aware that they are being misled as to the true nature of the experimental stimuli.

Researchers may therefore be eager to ensure that their ruse is maintained and the deception at the heart of their study is not revealed to participants prematurely. Of the six delivery methods

39 employed in this study, the highest rate of misinformation awareness was observed in the simple narrative condition, where nearly 12% of participants correctly guessed the purpose ofthe experiment. We suggest that this higher rate of detection may have come about because participants in this condition were simply asked to read a narrative, with no associated task.

Participants may therefore have been more likely to conclude that the narrative was a treatment rather than a test, and become more suspicious as to our motives.

An alternative explanation for these findings could be that greater awareness of misinformation indicates deeper processing, if we assume that participants must carefully process both the original information and the post-event information in order to detect the discrepancy. By this account, the simple narrative would be deemed to have been processed most deeply, which would run counter to previous research regarding the requirements for deep processing (Craik, 2002; Craik & Lockhart,

1972; Zaragoza & Lane, 1994). In the absence of evidence regarding the depth of processing of the original event, this must remain an open question for now; future research may wish to manipulate levels of processing of both the original event and the post-event misinformation in order to properly test this hypothesis. On balance, we feel it is more likely that the high rate of awareness in the simple narrative condition was simply due to participants’ attention being drawn to the oddity of being asked to read a description of an video they just watched, with no associated task. In experiments where maintaining ignorance of the study goals is key, researchers may therefore wish to consider using a method that purports to test participants’ knowledge rather than transparently conveying information.

Eyewitness memory in the real world

The primary aim of this study was to investigate the effectiveness of methods of misinformation delivery that may be used in experimental contexts. It is however also important to consider the potential implications of these results for real-world eyewitnesses. Some of the methods employed

40 here are more likely to be encountered in a criminal justice context than others; for example, a witness in a trial may be exposed to leading questions, which may be phrased simply or elaborately, depending on the style and intention of the interrogator. The evidence reported above suggests that this difference in style is likely to have minimal effects at initial questioning, though its impact may increase after a delay – possibly because the witness has had time to reflect on the details provided in the question and to construct a memory that contains the misinformation. Given that witnesses are frequently questioned about their recollections after a considerable delay, the results from the follow-up questionnaire may be particularly important to note. Similarly, a witness may read an account of a crime in a newspaper, or may come across another witness’s written description of the event, but they are unlikely to encounter a narrative presented in scrambled form or with missing words. These misinformation delivery methods are perhaps of most relevance to experimental contexts, however the general message regarding levels of processing has wider implications. In general, these methods suggest that engaging more deeply with an inaccurate account of the witnessed events can result in a higher rate of recall of misinformation. Thus, witnesses who are particularly motivated to engage with any material related to the witnessed event may be more susceptible to being misled.

Limitations

In order to ensure that participants’ exposure to information about the critical items was tightly controlled, surface differences in the design of the delivery methods were permitted. Specifically, despite apparent differences in their structure, the 8 questions in the leading question conditions,

15 sentences in the narrative conditions and 10 photographs in the doctored photos condition all provided an equivalent amount of information about the four critical and four non-critical items addressed in the memory questionnaire. Every effort was made to control exposure to other aspects of the event, however the nature of the methods themselves ensured some differences. For example, the elaborate leading questions contained more extraneous detail than the simple leading

41 questions by definition, while the filler photographs in the doctored photos condition contained more visual detail than could be contained in a brief written description. These features might have served as a reminder of irrelevant details such as the colour of the victim’s hair, or the number of cars in the background, with potential consequences for memory of the video as a whole. The potential for effects relating to irrelevant details of the delivery method cannot therefore be ruled out, but as such features are essential components of the delivery methods their effects cannot be separated from the methods themselves.

Another concern relates to the manipulation of depth of processing. Participants in the scrambled narrative condition were required to reorder sentences that contained the critical details; thus, the levels of processing manipulation occurred at the level of the narrative, rather than at the level of individual details. It could therefore be argued that this manipulation would draw attention to the overall sequential order of the narrative, rather than to the critical details themselves. In contrast, the missing-items narrative explicitly drew attention to the specific critical items, as the missing word always related to the critical item (e.g. “a cyclist ignores a red traffic ___”). Notably, the effect sizes for the scrambled narrative and missing-items narrative were very similar, indicating that this difference in manipulation had little impact on processing of the misinformation.

The nature of the details about which participants are misinformed can also influence the degree of memory distortion. Evidence suggests that additive misinformation (where details that were not included in the original event are added to the post-event description) results in more suggestibility than contradictory misinformation (where details of the original event are changed in the post-event description), and may be less amenable to warnings about the presence of misinformation (Huff &

Umanath, 2018; Nemeth & Belli, 2006). The present study contained one piece of additive misinformation (the stolen car key, mentioned in addition to the wallet and phone that were actually stolen) and three pieces of contradictory misinformation (the red vs. green traffic light, the thief

42 bumping into the witness rather than avoiding her, and the thief crossing the road at the pedestrian crossing instead of emerging from behind the camera). The item-by-item analysis in supplemental materials shows that the additive item did indeed lead to a greater misinformation effect, as in previous literature. Importantly, because our primary analysis used total misinformation score

(summed across all items) as the dependent variable, and the assignment of items to misinformation or control condition was counterbalanced across narrative versions, item type is not expected to have had any effect on these results. We do however suggest that future research may wish to directly examine the effect of misinformation delivery methods on contradictory and additive forms of misinformation.

Conclusion

The research described above is clearly of importance for psychological scientists working in this field, as it provides a foundation for selecting the most appropriate method for use in future studies.

These findings also have implications for individuals in law enforcement and justice, who wish to know how exposure to different kinds of information might affect a witness’s subsequent memory for an event. Though there are a great many studies that have employed the misinformation paradigm, there is a surprising lack of research comparing misinformation delivery methods. From these results, it appears that casual exposure to altered photographs of an event is unlikely to result in memory distortions, however given that previous literature has demonstrated clear effects of doctored photographs on memory, additional research will be required before firm conclusions can be drawn. In contrast, all other misinformation delivery methods employed in this tightly controlled, preregistered study resulted in large and significant misinformation effects, though the magnitude of these effects varied widely. Our findings suggest that encountering a simple inaccurate description of an event – for example, in a newspaper – is likely to have a relatively small impact on a witness at immediate test, but a larger effect after a delay. The more deeply the witness is asked to engage with the material, the more likely it is that their testimony will be unreliable. Thus, we argue that,

43 when considering the impact of misinformation on eyewitness memory, the medium may be as important as the message.

44 Acknowledgements: We would like to thank Agnieszka Biedrycka for providing the video stimulus used in this experiment and Aaron Bolger for his assistance in doctoring the photographs used in the study. We would also like to thank Jade Duffy for assistance with data coding.

45 REFERENCES

Blank, H., Ost, J., Davies, J., Jones, G., Lambert, K., & Salmon, K. (2013). Comparing the influence of directly vs. indirectly encountered post-event misinformation on eyewitness remembering. Acta Psychologica, 144(3), 635-641.

Bodner, G. E., Musch, E., & Azad, T. (2009). Reevaluating the potency of the memory conformity effect. Memory & Cognition, 37(8), 1069-1076.

Braun, K. A., & Loftus, E. F. (1998). Advertising's misinformation effect. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 12(6), 569- 591.

Brewer, N., & Burke, A. (2002). Effects of testimonial inconsistencies and eyewitness confidence on mock-juror judgments. Law and Human Behavior, 26(3), 353-364.

Brown, A. S., & Marsh, E. J. (2008, 2008/02/01). Evoking false beliefs about autobiographical experience. Psychonomic Bulletin & Review, 15(1), 186-190. https://doi.org/10.3758/PBR.15.1.186

Brubacher, S. P., Sharman, S. J., Scoboria, A., & Powell, M. B. (2020). The effect of question type on resistance to misinformation about present and absent details. Applied Cognitive Psychology.

Calvillo, D. P., & Parong, J. A. (2016). The misinformation effect is unrelated to the DRM effect with and without a DRM warning. Memory, 24(3), 324-333.

Cash, D. K., & Lane, S. M. (2020). Judging memory: strong verbal confidence and lineup context influence inferences about eyewitnesses. Psychology, Crime & Law, 1-21. https://doi.org/10.1080/1068316X.2020.1774588

Craik, F. I. (2002, Sep-Nov). Levels of processing: past, present. and future? Memory, 10(5-6), 305- 318. https://doi.org/10.1080/09658210244000135

Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of verbal learning and verbal behavior, 11(6), 671-684.

Craik, F. I., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104(3), 268.

Dodd, D. H., & Bradshaw, J. M. (1980). Leading questions and memory: Pragmatic constraints. Journal of verbal learning and verbal behavior, 19(6), 695-704.

Eakin, D. K., Schreiber, T. A., & Sergent-Marshall, S. (2003). Misinformation effects in eyewitness memory: The presence and absence of memory impairment as a function of warning and misinformation accessibility. Journal of experimental psychology: Learning, Memory, and Cognition, 29(5), 813.

Echterhoff, G., Groll, S., & Hirst, W. (2007). Tainted truth: Overcorrection for misinformation influence on eyewitness memory. Social cognition, 25(3), 367-409.

46 Einstein, G. O., McDaniel, M. A., Bowers, C. A., & Stevens, D. T. (1984). Memory for prose: The influence of relational and proposition-specific processing. Journal of experimental psychology: Learning, Memory, and Cognition, 10(1), 133.

Farina, F., & Greene, C. M. (2020). Examining the effects of memory specificity and perceptual load on susceptibility to misleading information. Applied Cognitive Psychology. https://doi.org/https://doi.org/10.1002/acp.3669

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

Fenn, E., Newman, E. J., Pezdek, K., & Garry, M. (2013). The effect of nonprobative photographs on truthiness persists over time. Acta Psychologica, 144(1), 207-211.

French, L., Garry, M., & Mori, K. (2011). Relative–not absolute–judgments of credibility affect susceptibility to misinformation conveyed during discussion. Acta Psychologica, 136(1), 119- 128.

Frenda, S. J., Knowles, E. D., Saletan, W., & Loftus, E. F. (2013). False memories of fabricated political events. Journal of Experimental Social Psychology, 49(2), 280-286.

Frenda, S. J., Nichols, R. M., & Loftus, E. F. (2011). Current issues and advances in misinformation research. Current Directions in Psychological Science, 20(1), 20-23.

Frost, P. (2000). The quality of false memory over time: Is memory for misinformation “remembered” or “known”? Psychonomic Bulletin & Review, 7(3), 531-536.

Frost, P., Ingraham, M., & Wilson, B. (2002). Why misinformation is more likely to be recognised over time: A source monitoring account. Memory, 10(3), 179-185.

Gabbert, F., Memon, A., Allan, K., & Wright, D. B. (2004). Say it to my face: Examining the effects of socially encountered misinformation. Legal and Criminological Psychology, 9(2), 215-227.

Gallo, D. A. (2010, 2010/10/01). False memories and fantastic beliefs: 15 years of the DRM illusion. Memory & Cognition, 38(7), 833-848. https://doi.org/10.3758/MC.38.7.833

Garry, M., & Gerrie, M. P. (2005). When photographs create false memories. Current Directions in Psychological Science, 14(6), 321-325.

Garry, M., Manning, C. G., Loftus, E. F., & Sherman, S. J. (1996). Imagination inflation: Imagining a childhood event inflates confidence that it occurred. Psychonomic Bulletin & Review, 3(2), 208-214.

Garry, M., & Wade, K. A. (2005, 2005/04/01). Actually, a picture is worth less than 45 words: Narratives produce more false memories than photographs do. Psychonomic Bulletin & Review, 12(2), 359-366. https://doi.org/10.3758/BF03196385

Gerrie, M. P., & Garry, M. (2007). Individual differences in working memory capacity affect false memories for missing aspects of events. Memory, 15(5), 561-571.

47 Goff, L. M., & Roediger, H. L. (1998). Imagination inflation for action events: Repeated imaginings lead to illusory recollections. Memory & Cognition, 26(1), 20-33.

Greene, C. M., Maloney-Derham, R., & Mulligan, K. (2020). Effects of perceptual load on eyewitness memory are moderated by individual differences in cognitive ability. Memory, 28(4), 450- 460. https://doi.org/https://doi.org/10.1080/09658211.2020.1729811

Greene, C. M., & Murphy, G. (2020, 2020/12/04). Individual differences in susceptibility to false memories for COVID-19 fake news. Cognitive Research: Principles and Implications, 5(1), 63. https://doi.org/10.1186/s41235-020-00262-1

Henkel, L. A., & Carbuto, M. (2008). Remembering what we did: How source misattributions arise from verbalization, mental imagery, and pictures. Applied memory, 213-234.

Huang, M.-H., & Hauser, R. M. (1996). Trends in black-white test score differentials: II. The WORDSUM vocabulary test. Center for Demography and Ecology, University of Wisconsin-- Madison.

Huff, M. J., & Umanath, S. (2018). Evaluating suggestibility to additive and contradictory misinformation following explicit error detection in younger and older adults. Journal of Experimental Psychology: Applied, 24(2), 180.

Jaschinski, U., & Wentura, D. (2002). Misleading postevent information and working memory capacity: An individual differences approach to eyewitness memory. Applied Cognitive Psychology, 16(2), 223-231.

Johnson, M. K., Hashtroudi, S., & Lindsay, D. S. (1993, Jul). Source monitoring. Psychological Bulletin, 114(1), 3-28.

Koutstaal, W., Schacter, D. L., Johnson, M. K., & Galluccio, L. (1999). Facilitation and impairment of event memory produced by photograph review. Memory & Cognition, 27(3), 478-493.

Lane, S. M., & Zaragoza, M. S. (2007). A little elaboration goes a long way: The role of generation in eyewitness suggestibility. Memory & Cognition, 35(6), 1255-1266.

Leippe, M. R., Eisenstadt, D., & Rauch, S. M. (2009). Cueing confidence in eyewitness identifications: Influence of biased lineup instructions and pre-identification memory feedback under varying lineup conditions. Law and Human Behavior, 33(3), 194-212.

Leippe, M. R., Manion, A. P., & Romanczyk, A. (1992). Eyewitness persuasion: How and how well do fact finders judge the accuracy of adults' and children's memory reports? Journal of personality and social psychology, 63(2), 181-197. https://doi.org/10.1037/0022- 3514.63.2.181

Lindsay, D. S., & Johnson, M. K. (2000). False memories and the source monitoring framework: Reply to Reyna and Lloyd (1997). Learning and Individual Differences, 12(2), 145-161.

Loftus, E. F. (1975). Leading questions and the eyewitness report. Cognitive psychology, 7(4), 560- 572.

48 Loftus, E. F. (2005). Planting misinformation in the human mind: A 30-year investigation of the malleability of memory. Learning & Memory, 12(4), 361-366. https://doi.org/10.1101/lm.94705

Loftus, E. F., Miller, D. G., & Burns, H. J. (1978). Semantic integration of verbal information into a visual memory. Journal of experimental psychology: Human learning and memory, 4(1), 19.

Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of verbal learning and verbal behavior, 13(5), 585-589.

Luna, K., & Martín‐ Luengo, B. (2012). Improving the accuracy of eyewitnesses in the presence of misinformation with the plurality option. Applied Cognitive Psychology, 26(5), 687-693.

McCloskey, M., & Zaragoza, M. (1985). Misleading postevent information and memory for events: Arguments and evidence against memory impairment hypotheses. Journal of Experimental Psychology: General, 114(1), 1.

Meade, M. L., & Roediger, H. L. (2002). Explorations in the social contagion of memory. Memory & Cognition, 30(7), 995-1009.

Meisenberg, G. (2015). Verbal ability as a predictor of political preferences in the United States, 1974–2012. Intelligence, 50, 135-143.

Mitchell, K. J., & Johnson, M. K. (2000). Source monitoring: Attributing mental experiences. The Oxford handbook of memory, 179-195.

Murphy, G., Loftus, E. F., Grady, R. H., Levine, L. J., & Greene, C. M. (2019). False Memories for Fake News During Ireland’s Abortion Referendum. Psychological science, 30(10), 1449-1459.

Nash, R. A. (2018). Changing beliefs about past public events with believable and unbelievable doctored photographs. Memory, 26(4), 439-450.

Nash, R. A., Wade, K. A., & Lindsay, D. S. (2009). Digitally manipulating memory: Effects of doctored videos and imagination in distorting beliefs and memories. Memory & Cognition, 37(4), 414- 424.

Nemeth, R. J., & Belli, R. F. (2006). The influence of schematic knowledge on contradictory versus additive misinformation: False memory for typical and atypical items. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 20(5), 563-573.

Newman, E. J., Garry, M., Bernstein, D. M., Kantner, J., & Lindsay, D. S. (2012). Nonprobative photographs (or words) inflate truthiness. Psychonomic Bulletin & Review, 19(5), 969-974.

Ost, J., Blank, H., Davies, J., Jones, G., Lambert, K., & Salmon, K. (2013). False memory not equal false memory: DRM errors are unrelated to the misinformation effect. PLoS One, 8(4), e57939. https://doi.org/10.1371/journal.pone.0057939

49 Paterson, H. M., & Kemp, R. I. (2006). Comparing methods of encountering post‐ event information: The power of co‐ witness suggestion. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 20(8), 1083-1099.

Rhodes, M. G., & Anastasi, J. S. (2000). The effects of a levels-of-processing manipulation on false recall. Psychonomic Bulletin & Review, 7(1), 158-162.

Roediger III, H. L., & Geraci, L. (2007). Aging and the misinformation effect: A neuropsychological analysis. Journal of experimental psychology: Learning, Memory, and Cognition, 33(2), 321.

Sacchi, D. L., Agnoli, F., & Loftus, E. F. (2007). Changing history: Doctored photographs affect memory for past public events. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 21(8), 1005-1022.

Schacter, D. L., Koutstaal, W., Johnson, M. K., Gross, M. S., & Angell, K. E. (1997). False recollection induced by photographs: a comparison of older and younger adults. Psychology and aging, 12(2), 203.

Scoboria, A., Wysman, L., & Otgaar, H. (2012). Credible suggestions affect false autobiographical beliefs. Memory, 20(5), 429-442.

Sharman, S. J., Garry, M., & Beuke, C. J. (2004). Imagination or exposure causes imagination inflation. The American Journal of Psychology, 157-168.

Sharman, S. J., & Powell, M. B. (2012). A comparison of adult witnesses' suggestibility across various types of leading questions. Applied Cognitive Psychology, 26(1), 48-53.

Shaw III, J. S., Garven, S., & Wood, J. M. (1997). Co-witness information can have immediate effects on eyewitness memory reports. Law and Human Behavior, 21(5), 503-523.

Stark, C. E., Okado, Y., & Loftus, E. F. (2010). Imaging the reconstruction of true and false memories using sensory reactivation and the misinformation paradigms. Learning & Memory, 17(10), 485-488.

Strange, D., Hayne, H., & Garry, M. (2008). A photo, a suggestion, a false memory. Applied Cognitive Psychology.

Strange, D., Sutherland, R., & Garry, M. (2006). Event plausibility does not determine children's false memories. Memory, 14(8), 937-951.

Thapar, A., & McDermott, K. B. (2001). False recall and false recognition induced by presentation of associated words: Effects of retention interval and level of processing. Memory & Cognition, 29(3), 424-432.

Toglia, M. P. (1999). Recall accuracy and illusory memories: When more is less. Memory, 7(2), 233- 256.

Underwood, J., & Pezdek, K. (1998). Memory suggestibility as an example of the sleeper effect. Psychonomic Bulletin & Review, 5(3), 449-453.

50 Wade, K. A., Garry, M., Read, J. D., & Lindsay, D. S. (2002). A picture is worth a thousand lies: Using false photographs to create false childhood memories. Psychonomic Bulletin & Review, 9(3), 597-603.

Waterman, A. H., Blades, M., & Spencer, C. (2001). Interviewing children and adults: The effect of question format on the tendency to speculate. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 15(5), 521-531.

Watson, J. M., Bunting, M. F., Poole, B. J., & Conway, A. R. (2005). Individual differences in susceptibility to false memory in the Deese-Roediger-McDermott paradigm. Journal of experimental psychology: Learning, Memory, and Cognition, 31(1), 76.

Whitley Jr, B. E., & Greenberg, M. S. (1986). The Role of Eyewitness Confidence in Juror Perceptions of Credibility 1. Journal of Applied Social Psychology, 16(5), 387-409.

Whittlesea, B. W. (1993). Illusions of Familiarity. Learning, Memory, 19(6), 1235-1253.

Wixted, J. T., & Wells, G. L. (2017). The relationship between eyewitness confidence and identification accuracy: A new synthesis. Psychological Science in the Public Interest, 18(1), 10-65.

Zaragoza, M. S., & Lane, S. M. (1994). Source misattributions and the suggestibility of eyewitness memory. Journal of experimental psychology: Learning, Memory, and Cognition, 20(4), 934.

Zhu, B., Chen, C., Loftus, E. F., Lin, C., & Dong, Q. (2013). The relationship between DRM and misinformation false memories. Memory & Cognition, 41(6), 832-838.

Zhu, B., Chen, C., Loftus, E. F., Lin, C., He, Q., Chen, C., Li, H., Moyzis, R. K., Lessard, J., & Dong, Q. (2010). Individual differences in false memory from misinformation: Personality characteristics and their interactions with cognitive abilities. Personality and Individual Differences, 48(8), 889-894.

Zhu, B., Chen, C., Loftus, E. F., Lin, C., He, Q., Chen, C., Li, H., Xue, G., Lu, Z., & Dong, Q. (2010). Individual differences in false memory from misinformation: Cognitive factors. Memory, 18(5), 543-555.

51