How Memory of the Past, a Predictable Present and Expectations of the Future Underpin Adaptation to the Sound Environment

Studies in the research profile Built Environment Doctoral thesis no. 1

How memory of the past, a predictable present and expectations of the future underpin adaptation to the sound environment

Anatole Nöstl

Faculty of Engineering and Sustainable Development Department of Building, Energy and Environmental Engineering

Studies in the research profile Built Environment Doctoral thesis no. 1

How memory of the past, a predictable present and expectations of the future underpin adaptation to the sound environment

Anatole Nöstl © Anatole Nöstl 2015

Gävle University Press ISBN 978-91-88145-00-0 ISBN 978-91-88145-01-7 (pdf) urn:nbn:se:hig:diva-20082

Distribution: University of Gävle Faculty of Engineering and Sustainable Development SE-801 76 Gävle, Sweden +46 26 64 85 00 www.hig.se

Print: Ineko AB, Kållered 2015 Abstract

By using auditory distraction as a tool, the main focus of the present thesis is to investigate the role of memory systems in human adaptation processes towards changes in the built environment. Report I and Report II focus on the question of whether memory for regularities in the auditory environment is used to form predictions and expectations of future sound events, and if violations of these expectations capture attention. Collectively the results indicate that once a stable neural model of the sound environment is created, violations of the formed expectations can capture attention. Furthermore, the magnitude of attentional capture is a function of the pitch difference between the expected tone and the presented tone. The second part of the thesis is concerned with, (a) the nature (i.e. the specificity) of the neural model formed in an auditory environment and, (b) whether complex cognition in terms of working memory capacity modulates habituation rate. The results in Report III show that the disruptive effect of the deviation effect diminishes with the number of exposures over time, and also as a function of working memory capacity. The aim of Report IV was to investigate the nature (and specificity) of the neural model formed in an auditory environment. If the neural model is fashioned around a specific stimulus then an observable increase of response latency should occur in conjunction with the deviant change. The results in Experiment 1 in Report IV, however, show that the habituation rate remained the same throughout the experiment. To further test the specificity of the neural model the modality of the deviant event was switched (from auditory to visual and vice versa) in Experiment 3 in Report IV. The collective findings indicate that the formed neural model may be of a more general nature than previously suggested. The aim of Experiment 2 in Report IV was to investigate what properties of the sound environment underpin habituation rate, more specifically if predictability of a deviant trial facilitates the habituation process. The finding that the habituation rate was similar whether there was a fixed temporal interval between the deviant trials or a random interval suggests that the amount of occurrences (i.e. number of deviant trials) determines habituation rate, not the predictability of a deviant trial.

Keywords: Memory, Expectations, Adaptation, Habituation, Predictions, Attentional capture, Auditory distraction

Sammanfattning

Denna avhandling undersöker vilken roll minnessystem har i anpassningen till förändringar i den byggda miljön. Delrapport I och Delrapport II fokuserar på frågan om regelbundenheter i den auditiva miljön används för att skapa förväntningar och prediktioner gällande framtida händelser, och vidare, om avvikelser från dessa förväntningar fångar uppmärksamheten. Sammantaget tyder resultaten på att uppmärksamheten fångas om nämnda förväntningar inte infrias. Vidare visar resultaten att magnituden av den fångade uppmärk- samheten är en funktion av skillnaden mellan den förväntade tonen och den presenterade tonen. Den andra delen av avhandlingen undersöker (a) karaktären (dvs. speci- ficiteten) av den neurala modellen och (b) om komplex kognition i termer av arbetsminneskapacitet påverkar habituation. Resultaten i Delrapport III visar att den störande effekten av den avvikande tonen minskar dels med antalet exponeringar och dels som en funktion av arbetsminneskapacitet. Syftet med Delrapport IV var att undersöka hur specifik den skapade neurala modellen är. Resultaten i Experiment 1 i Delrapport IV visar att habituationstakten för- blev densamma under hela experimentet även om den avvikande tonen byttes ut under experimentets gång. Detta tillsammans med resultaten i Experiment 3 i Delrapport IV, där habituation kunde påvisas även om modaliteten av den avvikande händelsen byttes från auditiv till visuell och vice versa, indikerar att den neurala modellen är av en mer allmän karaktär än vad man tidigare trott. Syftet med Experiment 2 i Delrapport IV var att undersöka vilka egen- skaper i ljudmiljön som påverkar habituationstakten. Upptäckten att takten var likvärdig oavsett om det fanns ett fast eller ett slumpmässigt intervall mellan de avvikande tonerna tyder på att det är mängden förekomster (dvs. antalet avvikande toner), snarare än predicerbarhet, som avgör habituationstakten.

Nyckelord: Minne, förväntningar, anpassning, habituation, prediktioner, fångad uppmärksamhet, auditiv distraktion

Acknowledgements

I would like to express my gratitude to the people who have supported me in order to complete this work. More specifically I would like to thank; Patrik Sörqvist, supervisor, whose energy, creativity and will to explore new ideas without hesitation is unmatched. Mikael Johansson, assistant supervisor, the main reason I chose to study psychology in the first place. Robert Ljung, colleague and travelling partner, the “the-glass-is-half-full” kind of guy you need when things are looking grim. The “old” crew, Staffan Hygge, Anders Kjellberg and Hans Allan Löfberg who collectively possess so much knowledge one would wish you could work forever. John Marsh, British humor at its best mixed with the ability to produce papers at an uncanny rate, every PhD student should keep one of these close by. The rest of my colleagues, appearing in no special order: Marijke Keus van de Poll, Andreas Haga, Helena Jancke, Mattias Holmgren, Niklas Hallin, Anders Hurtig and Elina Pekkola. My parents, siblings and grandparents, thank you for always supporting me no matter what. Finally I would like to thank Bella, Maggie and Tilly, the three musketeers.

List of papers

This thesis is based on the following papers, which are referred to in the text by Roman numerals:

Paper I Nöstl, A., Marsh, J. E., & Sörqvist, P. (2012). Expectations Modulate the Magnitude of Attentional Capture by Auditory Events. PLoS ONE 7(11): e48569. doi: 10.1371/journal.pone.0048569

Paper II Nöstl, A., Marsh, J. E., & Sörqvist, P. (2014). What We Expect Is Not Always What We Get: Evidence for Both the Direction-of-Change and the Specific- Stimulus Hypotheses of Auditory Attentional Capture. PLoS ONE 9(11): e111997. doi: 10.1371/journal.pone.0111997

Paper III Sörqvist, P., Nöstl, A., & Halin, N. (2012). Working memory capacity modulates habituation rate: Evidence from a cross-modal auditory distraction paradigm. Psychonomic bulletin & review, 19(2), 245-250. doi: 10.3758/ s13423-011-0203-9

Paper IV Nöstl, A. (submitted). Habituation to distraction from deviant sound: Does predictability matter?

Reprints were made with permission from the publishers.

Table of Contents

1 Background 1 The built environment and the psychology of the person in the environment 1 Noise in the built environment and how people adapt to it 1 The retroactive role of memory 3 Attention and orienting responses 3 Habituation 4 The role of complex cognition (working memory) in habituation (basic learning) 5 The proactive role of memory 6 Expectations 7 Violations of expectations 7 Theories of auditory distraction 8 A comparison between the three theories of attention capture: novelty, perceptual change, and expectations 9 Summary and Purpose 10 2 Summary of reports 11 Research questions and aims 11 General method 11 Report I 12 Report II 14 Report III 17 Report IV 18 3 General discussion 21 Implications for theories of attentional capture 21 Implications for habituation 21 Memory systems as a tool for adaptation to the environment (Conclusion) 22 Remarks for future research 22 References 25

1 Background

The built environment and the psychology of the person in the environment The world around us is constantly evolving. In order for an organism to survive and thrive in an ever-changing environment, an ability to adapt and adjust to its surrounding environment is crucial. Furthermore, the value of adaptation increases as a function of the pace in which the changes occur (e.g. the pace at which the world evolves). Picture yourself sitting at an airport trying to read a book. The present goal at hand is to comprehend what is being read, and while reading comprehension would be facilitated by ignoring all sound sources in the vicinity (e.g. other passengers’ conversations or irrelevant speaker announcements) it is important not to miss any announcements containing relevant information regarding your own flight (e.g. the call for boarding or a personal message). The aforementioned example illustrates how people must find an appropriate balance between focusability and distractibility for operating at a satisfactory level in the environment. This thesis is about the role of memory in these processes: how memory of the past makes us able to adapt to a static environ ment (habituation), how memory of regularities learned from the past can be used to predict future events in the environment (expectations), and how complex cognition (working memory) interacts with these more basic learning processes. Specifically, the aim of the current thesis is twofold: First, to study how the interplay between memory/experience and predictions about future events in the environment influence our responses toward changes in the surrounding environment. The second part of the thesis is concerned with habituation towards sound. Here, habituation rate is studied through the contributions from environmental properties (or characteristics), such as predictability and physical characteristics of a sound stream, and mental abilities such as working memory capacity. To this end, sound (or noise: undesired sound) acts as the primary experimental tool for investigating the role of memory in human adaption to the built environment.

Noise in the built environment and how people adapt to it Noise burden from road traffic and other types of irrelevant sound is a major problem within modern cities. There are several examples of studies from the field of built environment research whose results reveal the negative effects of chronic environmental noise (e.g. traffic noise) on general well- being (Evans, 2003; Evans, Bullinger & Hygge, 1998) and a wide range

1 of cognitive abilities (Hygge, Evans & Bullinger, 2002; Lercher, Evans & Meis, 2003; Maxwell & Evans, 2000; Ouis, 2001; Stansfeld et al., 2005). For example, a comprehensive longitudinal study conducted at the Munich’s international airport, gave unique insight to how environmental noise can affect cognitive abilities (Evans, Hygge & Bullinger, 1995). What makes the research setting special is that the study was performed during the time when the city of Munich decided to relocate the airport. This provided an oppor tunity to measure before- and after effects related to the distance between the inhabitants and the airport. The results revealed several interesting findings including that unavoidable environmental noise can disrupt cognitive abilities (e.g. reading, long term memory and attention) even though it has no semantic content. Irrelevant sound can be disturbing even in an enclosed built environment – such as a school classroom or an office space – which can be deemed as safe. Indeed, there is a cornucopia of studies carried out in the past decades showing that irrelevant sound impairs performance of a wide range of cogni tive tasks, often carried out in these environments, such as reading comprehension (Boyle, 1996; Oswald, Tremblay & Jones, 2000), prose memory (Bell, Buchner & Mund, 2008; Sörqvist, Ljungberg & Ljung, 2010), serial recall (Beaman & Jones, 1998; Hughes, Vachon & Jones, 2005; Jones, Hughes & Macken 2010), proof reading (Halin, Marsh, Haga, Holmgren & Sörqvist, 2014) and writing (Keus van de Poll, Ljung, Odelius & Sörqvist, 2014; Sörqvist, Nöstl & Halin 2012). Although the collective results from these studies agree on the point that environmental sound can indeed have negative impact on performance, there is an ongoing debate whether the human brain is able to adapt to the surrounding sound environment over time. Whereas Tremblay and Jones (1998; see also Röer, Bell, Dentale & Buchner, 2011) failed to find evidence in support of habituation (i.e. decreased responsiveness towards auditory distraction over time), other studies (Banbury & Berry, 1997; Bell, Röer, Dentale & Buchner, 2012; Morris & Jones, 1990; Röer, Bell & Buchner, 2014) have reported evidence of habituation. For example, results from a study by Banbury and Berry (1997) in which a prose task was utilized, revealed that participants are less susceptible to distraction by irrelevant sound if the participants are exposed to background speech (Experiment 1) or office noise (Experi- ment 2) during a period of 20 minutes prior to the prose task. The robustness of the observed habituation process is however ambiguous, at least regarding habituation towards background speech: Whereas habituation towards office noise without speech was unaffected by a 5-minute quiet period between the pre-test exposure and the actual prose test, habituation of office noise with speech was disrupted (i.e. a dishabituation process started) when a brief quiet period before the test was included. These findings suggest that a habituation process is initiated as long as the sound environment is somewhat static but the process may be sensitive to the complexity (e.g. characteristics) of the auditory environment.

2 The retroactive role of memory In this part of the thesis, I go through basic concepts in cognitive psychology needed for the study of the interaction between basic mental abilities and how humans adapt and behave in a static and changing sound environment.

Attention and orienting responses Certain novel events in the environment possess the ability to capture our attention. This may seem inconvenient and experienced as a nuisance in modern society. From an evolutionary point of view, however, the shift of attention towards a novel was crucial to survival. To our ancestors it was of uttermost importance to respond to a brisk sound in a nearby bush as this could imply that a predator or a pray was approaching the campfire. Although the risk of being attacked by a wild animal in everyday life has since then drastically decreased, the need for a mechanism to filter out irrelevant sound (e.g. traffic noise, telephones), while being distractible by sudden changes in the environment which could provide cues for imminent danger (i.e. a fire alarm or the sudden approach of a car) persists. Attention, the ability to focus on stimuli of interest in the surrounding environment, has been a hot topic in psychological research for more than a century. A classic experimental setting to address questions regarding the balance between focusability and distractibility within the field of auditory research is the ‘dichotic listening’ task (Cherry, 1953). In this task participants listen to two different sound streams using headphones and each sound stream is presented in a different ear. Instructions to attend to only one ear and shadow that specific stream (e.g. shadow only what is heard through the left headphone) are given. Studies (Conway, Cowan & Bunting, 2001; Moray, 1959; Wood & Cowan, 1995) have shown that certain characteristics of the unattended sound stream are more captivating than others. For example, hearing your own name (compared to ‘dog’ for example) in the unattended sound stream increases the probability for attention to be diverted away from what is presently attended to. As our senses are constantly bombarded with input from our surroundings and as it is impossible to process all of the incoming information, orientation processes play an important role in attention. The orientation reflex (or response; OR) as defined in modern literature was coined by Pavlov (1927) and he described the OR as an intermediate (and involuntary) response to changes in the environment. Generally, Pavlov referred to this phenomenon as the “What is it?” response. Sokolov, another prominent Russian researcher, proposed the idea that repeated exposure to a stimulus results in an internal memory representation, or neural model, containing different features (or properties) of that specific stimulus (Sokolov, 1963). Moreover, he suggested that in case a stimulus is encountered, it will automatically be compared to the relevant neural model and in case the stimulus does fit (i.e., the stimulus is not novel), the OR is inhibited, whereas if the stimulus does not fit with the neural model (i.e., the stimulus is novel), the OR is executed. Although

3 the concept of a neural model was widely accepted, Sokolov left a few important questions unanswered: a) How many exposures of a stimulus are needed to form a reliable neural model? and b) How sensitive is the neural model (i.e. what magnitude of perceived change is needed to elicit an OR)? Since Sokolov’s seminal work, a plethora of studies (e.g. Haenschel et al., 2005; Molholm et al., 2005; see Näätänen, Jacobsen & Winkler, 2005 for a review) have focused on examining the nature of the proposed neural model by using the mismatch negativity paradigm (MMN; Näätänen, 1990). In the MMN paradigm, the deviation effect is quantified by the difference in the magnitude of event related potentials (ERP) between standard trials and deviant trials and thus the MMN is considered a reliable tool to investigate if a novel stimulus fits with the neural model or if it deviates from it. Using this technique, Cowan et al. (1993) conducted a study in which participants were exposed to short sound bursts (sine wave tones) of varying frequency while event related potentials were measured. Each stimulus-train consisted of 9 tones (8 standard tones and 1 deviant tone) and in order to investigate the amount of standard tones required to build a neural model the deviant tone could occur at 5 different positions; Position 1, 2, 4, 6 or 8. To address the question regarding the sensitivity of the neural model, the standard tone and deviant tone frequencies could either appear in a fixed frequency (i.e. the frequency was constant during the entire condition) or a roving frequency (i.e. the frequency of the standard tone and the deviant tone shifted between each stimulus train). Main findings include the following: The results from the roving deviant condition indicate that the neural model is built even though the sound environment changes over time, it does however, take longer time to build a reliable neural model in a changing environment compared to a static one. Furthermore, the finding that the neural model goes into a dormant state in between stimulus trains and is re-activated on demand, demonstrates a likely interplay between short-term memory and long-term- memory. Collectively the findings suggest that the main purpose of memory is not its function as a storage unit per se, it can rather be seen as a prerequisite in the process of adapting to the environment.

Habituation Considering the great variability in the environment, an organism that responds to the slightest change in its surroundings is not functional. Evolution has shaped a functional adaptation to this circumstance. When the stimulus, that elicits the orienting-response, is presented repeatedly, a habituation process takes place (i.e., the OR response attenuates; Ben-Shakhar & Lieblich, 1982; Thomson & Spencer, 1966). Habituation of the orienting response is, essentially, a necessity for selective attention (Cowan, 1995). Habituation can be described as a form of adjustment to exogenous changes in the environment, and during this process an organism becomes less responsive to a stimulus as stimulus-familiarity increases. A prominent view of the habituation process, which is relevant to the literature on auditory distraction, suggests that the brain constructs a neural

4 model of the surrounding sound environment which tends to include the deviant sound after a certain number of exposures. Once the deviant sound is attached to the neural model it will no longer elicit an orienting response. However, if a change in the physical characteristics of the deviant sound occurs the established neural model will no longer fit and it will require an update including the new sound. According to Sokolov’s (1960) model, the buildup time of the neural model (and also its accuracy) should, to a large extent, depend on the deviant characteristics. Furthermore, considering previous research, the characteristics of the deviant stimulus should modulate the magnitude by which that deviant captures attention. The neural model account suggests that any change of the deviant should render the neural model built inadequate and therefore in need of updating. The buildup of the neural model can also be described as basic (Bayesian) learning: Each unfamiliar stimulus is assigned a weight/prior (i.e., the probability that the stimulus carries vital information) and immediately, after each new exposure, these weights are updated (and the stimulus becomes no longer unfamiliar). And in case the stimulus is deemed to carry no relevant new information (e.g., poses no danger), the weight is decreased with each exposure to the unfamiliar stimulus. The constant updating of beliefs results in a decreased orientation response (i.e., a switch of the locus-of-attention towards the novel stimulus), and after a certain amount of encounters, the stimulus may be classified as irrelevant and can thereby be ignored without any potential penalties for the organism. Both the neural model perspective and the learning perspective rest on the assumption that memory systems are a vital part of the ability to adjust to a changing environment and thereby for habituation in general to occur.

The role of complex cognition (working memory) in habituation (basic learning) A prerequisite for habituation to occur is to learn the regularities in the environment, which requires an interaction between top-down and bottom- up processes on one hand and memory systems on the other. This inter action could therefore depend on the efficiency of higher-order (or complex) cognitive abilities such as working memory capacity. Especially since individual differences in working memory capacity, as operationalized by complex-span tasks such as reading span (Daneman & Carpenter, 1980), operation span (OSPAN; Engle & Turner, 1989) or the size comparison span (Sörqvist et al., 2010), are often regarded as a reliable precursor to executive control, even more so when some form of distraction is added to the task. Executive control is associated with what is called top-down control of attention. A common approach in research regarding attention is to dicho- tomize attentional processes into top-down control and bottom-up input. In this case, top-down control refers to the ability to select goal relevant stimuli (e.g. process sought after targets in a search task or shifting attention to, and process, relevant information at an airport) for further processing whereas bottom-up input refers to stimuli which have entered attentional

5 processes due to their salience (e.g. stimuli that pop out from the current environment). Recent research on this topic suggests that instead of treating the top-down and the bottom-up processes as separate entities one should focus more on the interaction between the processes (Awh, Belopolsky & Theeuwes, 2012). The interplay between top-down and bottom-up processes becomes particularly important in the process of adjusting to the environment since bottom-up input facilitates the extraction of feature regularities in the surrounding environment and these extracted patterns may be used to learn from the environment and thereby being able to adapt to any unfore- seen changes within it. This in turn has consequences for attentional control as bottom-up input can be incorporated into the goal-directed behavior and thereby may enhance the performance for the task at hand. Evidence in line with the idea that working memory capacity is related to top-down control comes from several studies on distraction and interference: For example, high-WMC individuals perform better at the Stroop task (Hutchison, 2011), Flanker task (Shipstead, Harrison, & Engle, 2012) and have a decreased likelihood of hearing their own name in the to-be-ignored channel in a dichotic listening task (Conway, Cowan & Bunting, 2001). The reason behind this relationship is argued to be that individuals with high WMC excel at keeping focus on task related material (Heitz & Engle, 2007), are better at maintaining instructions in the face of distraction (Unsworth & Engle, 2007) and are also able to inhibit responses that arise in conjunc tion with distraction (Friedman & Miyake, 2004; Hasher, Lustig, & Zacks, 2007; Lustig, May, & Hasher, 2001). The notion that high-WMC individuals have superior attention related capabilities and thereby are less susceptible to external distraction, is however only one part of the story as to why WMC may be related to basic learning. For example, the finding that a neural model enters a dormant state in case it is not presently used (Cowan et al., 1993) suggests that the ability to effectively retrieve task relevant information from LTM (e.g. re-activation of the correct neural model) once it has been dis- placed from STM should be equally important in order to learn from, and interact with, the environment. There are several studies (e.g. Kane & Engle, 2003; Rosen & Engle, 1997) which have investigated the interplay between WMC and LTM. Collectively the results from these studies provide evidence in line with the idea that high-WMC individuals possess superior retrieval capabilities from LTM, mainly due to the effective use of cues which facilitate the search of relevant information stored in LTM (see Unsworth & Engle, 2007, for a review). Considering that habituation is closely connected to LTM, WMC should also modulate the habituation process in general.

The proactive role of memory Up to this point, I have been discussing how memory, retrospectively, picks up regularities in the sound environment to adapt efficiently and how trait capacity for these cognitive control processes could potentially contribute to these retrospective processes have also been discussed. In the current part of

6 the thesis, I will discuss how people can use memory of the past proactively to adapt to events that are still yet to come.

Expectations Expectations and predictions are a large part of life. Weather reports, the future value of a stock price, gambling, even choosing a partner is based on some sort of prediction about a future outcome. In financial decision making and in weather forecasts advanced statistical models are used to filter out relevant information from an abundance of noise. The human brain is also equipped with statistical tools and uses prior knowledge about the world (i.e. memory) in order to facilitate its interaction (e.g. perceptual processes, judgments) with the surrounding environment (Gigerenzer, 2008; Tversky & Kahneman, 1974). Due to this, we are barely ever in the present, we are rather in the twilight between memories of the past and an imaginary future world consisting of predictions based on former knowledge and experiences (Bar, 2007, 2009). This becomes rather evident in the field of visual perception (Bar, 2003) whereby the context in which an object is presented is used to constrain the number of possible objects by using probability theory (i.e. predictions are based on the probability that the object exists in the present context). A blurred out image of a hairdryer might lead to different associations depending on what context it is presented in; the image may be classified as a power drill if the context consists of a tool shed, or it may be classified as a pistol if the context consists of a crime scene. Even though the use of predictions is an effective way to speed up perceptual processes chance still exists that an object is wrongly classified as something else simply because it is encountered in a novel (or unusual) context.

Violations of expectations The idea that the aforementioned predictive networks also are used in auditory context has been supported by several studies in the recent years. Evidence in line with the idea that the brain actively creates online predictions in auditory environments, stems from a study by Bendixen, Schröger & Winkler (2009) in which the predictability of a tone sequence in an active listening task was manipulated. In this study participants were actively listening to tones, either fashioned as a short predictable melody or appearing in random order and occasionally a tone was omitted during the sequence. By using ERP, Bendixen et al. revealed that the brain elicited similar activity whether the predictable tone was heard or omitted. This suggests that the neural model for the expected input is pre-activated in preparation of future events. Further support for the role of expectations comes from studies which have included explicit warnings preceding each deviant in a cross-modal oddball task (Hughes et al., 2013; Parmentier, & Hebrero, 2013; Shelton, Elliot, Eaves & Exner, 2009; Sussman, Winkler & Schröger, 2003). The studies indicate that the deviation effect is eliminated if information about upcoming deviant trials is given (at least if the accuracy of the warnings is 100%).

7 Whereas the aforementioned studies use explicit warnings about upcoming deviants (and thereby tapping into the area of cognitive control), other studies indicate that rules governing the sound may also be learned incidentally over time and that a deviation effect is observed in case these rules are violated (Parmentier et al., 2011; Vachon, Hughes & Jones, 2012). For example, Par- mentier et al (2011) conducted a study using a cross-modal oddball design in which the participants were instructed to classify visual digits and each digit was preceded by either a standard tone or a novel tone. To tease apart the base rate and expectation accounts, the majority of novel trials (8 out of 9 cases) were presented in pairs, and on rare occasions a standard tone was presented after the first novel. The results revealed that the second deviant trial in a pair of two consecutive deviant trials elicits no deviation effect compared to standard trials, and paradoxically, a standard following a deviant in this case was more captivating than another deviant. Further evidence in line with the expectation account comes from a study in which the role of expectation was investigated in a serial recall context (Vachon et al., 2012). In this study, the to-be-remembered items were presented visually and the auditory to-be-ignored items consisted of a sequence of letters spoken by either a female voice or a male voice, switching between the two voice types every 6th trial. Interestingly, no deviation effect was observed when the voice switches occurred (Experiment 1) even though the 1st trial after a switch can be considered novel. However, if an unpredictable switch occurred (e.g. a switch from male to female on the 2nd trial of a male quintet) a noticeable deviation effect was observed. These studies lend support towards the notion that expectation is a key factor in a wide range of experimental settings and, moreover, that standard stimuli can capture attention if they are presented in an unexpected context.

Theories of auditory distraction In the field of auditory distraction, the collective results regarding the role of complex cognition in auditory distraction are somewhat ambiguous as top-down control does not always provide a reduction of auditory distraction: Whereas several studies have established a link between the attentional control and deviation effect, the changing-state effect seems unaffected by top-down control. Evidence opposing the idea that complex cognition modulates susceptibility to auditory distraction in general stems from research carried out in the context of short-term memory, especially in a setting that involves serial recall (Jones & Macken, 1993). A common way to investigate the impact of irrelevant sound on short term memory is by using a cross-modal task in which the to-be-remembered items are presented visually and the to-be-ignored items are presented auditory. Interestingly, the magnitude of the impact of the auditory presented to-be-ignored items is dependent on the physical properties of the sound. For example, the sequence ‘s r k t w c’ disrupts the ability to recall items in serial order (known as the changing-state effect; Jones & Macken,

8 1993) to a larger degree than the sequence ‘k k k k k k’. However, including an ‘m’ in the aforementioned sequence (e.g. ‘k k k m k k’) may also result in a noticeable drop in performance (referred to as the deviation effect). There are two different competing accounts trying to explain the phenomenon: The unitary account (Bell, Dentale, Buchner, & Mayr, 2010; Bell, Röer, Dentale, & Buchner, 2012; Chein & Fiez, 2010) and the duplex-mechanism account (Hughes, Vachon & Jones, 2007). Whereas the unitary account suggests that every item in an ever changing sound stream can be considered to be deviating from the preceding item and thereby captures attention with every exposure, the duplex-mechanism account differentiates between attentional capture and interference-by-process. The interference-by-process account suggests that the changing-state effect occurs because the changing sound sequence interferes with a) an automated order process and b) the constantly changing sequence encumbers rehearsal of presented items. That is, even though both of the different sound sequences impair performance they do so by influencing different underlying mechanisms. To shed further light on the matter, Sörqvist et al (2014) conducted a comprehensive meta study which revealed a positive link between working memory capacity and susceptibility to the deviation effect (i.e. individuals with high WMC are less likely to be distracted) but no such link was found between WMC and the changing state effect. This finding is rather troublesome for the unitary account which states that the changing-state effect occurs because each change in the sequence of auditory items is seen as a deviant and should in turn be modulated by top-down cognitive control. To reiterate; there is a general consensus that WMC seems to modulate susceptibility towards distraction however the exact mechanisms behind this are still under debate. The current thesis is solely concerned with the other part of the duplex-mechanism account, namely attentional capture.

A comparison between the three theories of attention capture: novelty, perceptual change, and expectations Certain novel events in our environment possess the ability to capture our attention. A novel encounter automatically activates the novelty-detection response which in turn elicits an involuntary orientation response towards the novel event, resulting in a re-orientation process in order to redirect attention back to the focal task at hand. This chain of responses can be observed by using electrophysiological equipment and the brain wave pattern that emerges in connection with a novel encounter is known as mismatch negativity (MMN; Näätänen, 1995; see Näätänen, Paavilainen, Rinne, & Alho, 2007 for a review). There are several well established accounts trying to explain the reason behind why some events may capture our attention. A prominent and original explanation is the perceived local change account. This account states that the orientation response emerges because the deviating tone clashes with the neural model in memory built from the recent standard sequence (Jaramillo, Paavilainen & Näätänen, 2000). Another widely accepted explanation to the deviation effect is the low-base rate probabi-

9 lity account. Here, the low-base rate probability (i.e. rarity) of the deviant is thought to cause the deviation effect (Näätänen, 1990; Schröger, 1997). Although both of the aforementioned accounts offer an explanation to why the letter ‘c’ captures attention in the sequence ‘m m m m c’ they yet fail to explain why the number ‘5’ in the sequence ‘1 2 3 5’ captures attention. This phenomenon has given birth to two very similar explanations: a) the violated expectations account and b) violation of a general rule governing the expectations in the environment. Both of these accounts imply that past experience of the environment is used to form predictions about future events. Further- more, the magnitude of discrepancy between the expected and the actually encountered event required to elicit attentional capture should co-vary with the amount of knowledge acquired of the present environment: That is, increased knowledge of the environment results in stronger predictions of future events, and hence minimizes the required discrepancy between the encountered event and the expected event to capture attention.

Summary and Purpose Certain sounds in the environment can capture our attention. There have been several accounts – such as the perceived local change account and the low base rate account – attempting to explain the phenomenon. Recent research has shown that expectations, more specifically the violations of expectations, play an important role in auditory distraction. The specificity of these formed expectations is however unclear. The strict neural model account suggests that any change in the environment and thereby a violated expectation would not fit into the established model. Although this might hold for large changes in the environment, an organism that responds to the slightest change would simply not be functional. Therefore, stimuli should have to exceed a certain threshold in order to violate expectations and thereby undergo evaluation in terms of the stimuli being compared to the neural model. The main concern of studies up to present has however been to explore the role of expectations in a more general sense and the specificity of formed expectations has been largely neglected. The generality of formed expectations and of the neural model also has impact on research regarding habituation. Although it has been established that we can habituate towards various sounds in the environment, the results are still quite ambiguous regarding what kind of sounds, and especially, how the complexity of the sound environment influences our ability to habituate and thereby enable us to adjust to the surrounding environment.

10 2 Summary of reports

Research questions and aims Report I: The research questions in Report I were: a) does the magnitude of attentional capture depend on the discrepancy between the expected tone and the presented tone, and b) does the effect increase with knowledge of the sound environment?

Report II: Based on the results in Report I, the purpose of Report II was to further investigate the violations-of-expectations account by contrasting the direction-of-change hypothesis (i.e. general rule oriented predictions) and the specific-stimulus hypothesis (i.e. specific stimulus bound predictions).

Report III: The purpose of Report III was to investigate the role of working memory capacity in habituation of the deviation effect. Specifically, the research question was: Is the habituation rate modulated by individual differences in working memory capacity?

Report IV: The research questions in Report IV were: a) how specific is the neural model (in term of the type of stimulus it encloses), and b) does predictability (in terms of temporal distance between each occurrence) of the deviant tone influence the buildup time of the neural model?

General method Cross-modal oddball task All experiments reported in the thesis used some variation of the cross- modal oddball task. The oddball task was modeled after Parmentier (2008). The participants were requested to categorize arrows as pointing either to the left (<<<) or to the right (>>>) by pressing the corresponding arrow key on the computer keyboard. They were told to use their dominant hand when pressing the button, to emphasize both speed and accuracy, and to ignore all sounds. At the beginning of each trial, a 200-ms sound was presented. In the most basic form of the cross-modal oddball task, most of the trials are

11 preceded by a standard sound (standard trial) and occasionally the standard sound is exchanged with a deviant sound (deviant trial). The deviation effect is quantified by the difference in response time between deviant trials and standard trials. The sounds were normalized and were presented binaurally through headphones (Sennheiser HD 202) at approximately 65 dB (A). An arrow was presented at the offset of the sound. The arrow was visible for 600 ms before it was replaced by a 250-ms visual mask (###). The computer recorded the response latency between the onset of the arrow and when the participant pressed a button. A key press later than 600 ms from the onset of the arrow was recorded as an error response. When the visual mask disappeared, the next trial was initiated.

Figure 1. The figure shows an illustration of the cross-modal oddball task.

Operation span A computerized version of the operation span (OSPAN) task (Turner & Engle, 1989) was adopted to measure WMC in Report III. Mathematical operations [e.g., “Is (5 + 3) × 3 = 24?”] were presented on a computer screen. The participants were told to respond “yes” or “no” to the operation, as quickly as possible, by pressing a button on the keyboard. When a response was recorded, the screen went blank for 500 ms, and then a one-syllable noun (e.g., dog), which the participants were told to remember for later recall, was presented for 800 ms. Each word was presented only once during the task. When the to-be-remembered word disappeared, a new mathematical operation was presented or the list ended, depending on the length of the list. The list length varied from two to six words. A total of 10 lists were used (2 of each list length), and the length increased across the task. When a list ended, the participants were asked to recall the words in the order of presentation by typing on the computer keyboard.

Statistical analyses All reported effects are statistically significant at alpha = .05. See the individual papers at the end of the dissertation for detailed analyses.

Report I Method In Experiment 1 the tones in the cross-modal oddball task consisted of three different standard tones, 440 Hz, 660 Hz and 880 Hz. The tones were arranged in a repetitive sequence across trials (i.e. 660-440-660-880-660-440-660-

12 880-660-…). Occasionally either the 440 Hz tone or the 880 Hz tone was replaced by a 220 Hz deviant tone or an 1100 Hz deviant tone. In Experiment 2 the standard tone sequence consisted of …660-220-660-440-660-880-660- 1100-660-220-660-… and so on. In contrast to Experiment 1, Experiment 2 did not utilize any novel tones per se. Instead, standard tones could occur in novel context (i.e. the 440 Hz tone and the 880 Hz tone was occasionally replaced by either the 220 Hz tone or the 1100 Hz tone).

Figure 2. The figure shows an illustration of the four experimental conditions in Experiments 1 (panels 1A and 1B) and 2 (panels 1C and 1D), respectively. Panels 1A and 1C depict the conditions where a high-pitch deviant replaces either a high or low pitch standard. Panels 1B and 1D depict the conditions where a low-pitch deviant replaces either a high or low pitch standard. Dark grey circles represent standard tones, light grey circles represent replaced (arguably expected) tones and black circles represent replacing deviant tones.

Results and discussion As can be seen in Figure 3 and Figure 4 (Panel B) the high-pitch deviant was more captivating when it replaced a low-pitch standard, and the low- pitch deviant was more captivating when it replaced a high-pitch standard. Given the increased magnitude of attentional capture as a function of pitch difference between the presented and replaced tones, even though the discrepancy between the preceding tone and the presented tone remained the same throughout the experiments, lends further support for the expectation account of the deviation effect. Furthermore, considering that the pattern- deviants in Experiment 2 consisted of standard tones in unusual context, novelty per se is neither sufficient nor a necessity for attentional capture to occur. The finding that this only occurs in the latter half of Experiment 2 suggests that learning and thereby the fashioning of a reliable neural model requires an increased amount of trials as a function of the complexity of the auditory environment.

13 Figure 3. The figure shows the results from Experiment 1. Each visual target was preceded by one of three standard tones that together formed a repetitive cross-trial sequence (660 Hz –880 Hz –660 Hz –440 Hz –660 Hz –880 Hz –660 Hz –440 Hz –660 Hz etc.). Occasionally, either the 440 Hz tone or the 880 Hz tone was replaced by a 220 Hz deviant tone or an 1100 Hz deviant tone. The figure shows the mean response time to visual targets following a deviant tone for each of the four types of replacements respectively (trials with correct responses are included only). Error bars are standard error of means.

Figure 4. The figure shows the results from Experiment 2 divided into the first half (panel A) and the second half (panel B) of the experiment. Each visual target was preceded by one of several standard tones that together formed a repetitive cross-trial sequence (660 Hz – 220 Hz –660 Hz –440 Hz –660 Hz –880 Hz –660 Hz –1100 Hz –660 Hz –220 Hz –660 Hz –440 Hz 660 Hz –880 Hz- 660 Hz –110 Hz –660 Hz etc.). Occasionally, either the 440 Hz tone or the 880 Hz tone was replaced by a 220 Hz pattern deviant or a 1100 Hz pattern deviant. The figure shows the mean response time to visual targets following a tone replacement for each of the four types of replacements respectively (trials with correct responses are included only). Error bars are standard error of means.

Report II Method In Experiment 1 the standard tones were arranged to resemble a well- known pattern, ‘Twinkle, twinkle, little star’. The sequence was presented in C-Major. Occasionally the sequence was interrupted by replacing the note

14 ‘E5’ with one of four different pattern deviants: ‘E3’, ‘E4’, ‘F5’ or ‘F#5’. Replacement took place after two consecutive presentations of ‘F5’.

Figure 5. The left panel is an illustration of ‘Twinkle, twinkle, little star’. Black arrows indicate sound sequence positions where tone replacements were occationally inserted during Experiment 1. The right panel is an illustration of the design of Experiment 1 and the possible tone replacement at the sequence position wherein the tone E5 would be presented in the original ‘Twinkle, twinkle, little star’ sound sequence. The light grey circle indicates the replaced tone (E5) and the black circles depict replacing/deviant tones. The set of replacing/deviant tones included F#5 (i.e., a small deviation from the replaced tone and an unexpected sequence-change direction), F5 (i.e., a perceptual change from the previous sound stimulus is held constant, but a small deviation from the replaced tone and an unexpected change direction), E4 (i.e., an intermediate deviation from the replaced tone, but a change in the expected direction) and E3 (i.e., a large deviation from the replaced tone, but a change in the expected direction).

In Experiment 2, ‘Twinkle, twinkle, little star’ could appear in either low- scale (C-major) or high-scale (F-major). Occasionally, the sequence of standard tones was interrupted by replacing the first ‘A5’ note with a pattern- deviant, ‘E6’. In Experiment 3 the tones were arranged in the same fashion as in Experiment 2. However, instead of using only one pattern-deviant, the ‘A5’ note was replaced by either an ‘E6’ tone or a ‘B6’ tone.

Figure 6. Left panel shows an illustration of the design in Experiment 2. The A5 note was ocassionally replaced by a deviant E6 tone (light grey note). Panel A depicts the condition in which the pitch of the deviant was in the expected direction of pitch transition, and Panel B displays the condition in which the pitch of the deviant was in the opposite (unexpected) direction. Right hand panel depicts the design in Experiment 3. The A5 note was either replaced by a E6 tone (light grey note) or a B6 tone (dark grey note). Panel A depicts the condition in which the pitch of the deviant was in the expected direction of pitch transition, and Panel B displays the condition in which the pitch of the deviant was in the opposite (unexpected) direction.

15 Results and discussion As can be seen in Figure 7, the results from Experiment 1 lend further support for the specific-stimulus hypothesis: The tone with the largest discrepancy between the presented and the expected tone (‘E3’) elicited a larger amount of attentional capture than the intermediate one (‘E4’). Most importantly the ‘E3’ tone was more captivating than the ‘F#5’ tone even though the ‘E3’ tone violated the expected direction of pitch and the ‘F#5’ did not.

Figure 7. Mean response time data for standard trials and the four types of pattern deviants in Experiment 1. The deviants were F#5 (i.e., a small deviation from the replaced tone and an unexpected sequence-change direction), F5 (i.e., a perceptual change from the previous sound stimulus is held constant, but a small deviation from the replaced tone and an unexpected change direction), E4 (i.e., an intermediate deviation from the replaced tone, but a change in the expected direction) and E3 (i.e., a large deviation from the replaced tone, but a change in the expected direction). Error bars represent standard error of means.

In Experiment 2 the direction-of-change hypothesis was explicitly tested. Whereas the results in Experiment 1 support the specific-stimulus hypothesis, the finding that the deviant pattern captured attention to a larger extent when the expected direction of pitch was violated supports the direction-of-change hypothesis (see Figure 8).

Figure 8. Mean response times in Experiment 2. The pattern deviant E6 was more disruptive when presented in a tone scale context that made it violate the expected direction-of-pitch change. Error bars represent standard error of means.

16 Interestingly, as can be seen in Figure 9, the results from Experiment 3 support neither the specific-stimulus hypothesis nor the direction-of-change hypothesis. The findings suggest that the more complex sound environment in Experiment 3 hinders the buildup of a fine-tuned and sensitive neural model and the change detection mechanism goes from being a dynamic mechanism to a static one.

Figure 9. Mean response times in Experiment 3. ‘Expected direction’ is the condition where the strophe was played in C-major, whereas ‘Opposite direction’ is the condition where the strophe was played in F-major. Error bars represent standard error of means.

Report III Method Working memory capacity was measured using a computerized version of the operation span task (Turner & Engle, 1989). In the cross-modal oddball task, the standard tone consisted of a 440 Hz sinewave tone and a burst of white noise was used as a deviant tone. The occurrence of a deviant trial was pseudo-randomized and was presented approximately every tenth trial. A total of 612 trials were presented divided across six blocks, each block consisting of 102 trials.

Results and discussion The results show that the deviation effect diminishes over time and more so as a function of working memory capacity. This may be due to WMC being related to superior inhibition capability, more stable mapping between stimulus-no-consequence responses in memory, or it may be due to high- WMC individuals’ ability to acquire regularities in the sound environment and thereby predict the upcoming occurrence of a deviant

17 Figure 10. How quickly individuals with high and low working memory capacity (WMC) responded to visual targets that were preceded by a frequently presented standard sound or by a rarely presented deviant sound across six consecutive blocks of trials (response time for correct responses are included only). Error bars are standard errors of means.

Report IV Method In Experiment 1, a 440 Hz sinewave tone was used as a standard tone and either a burst of white noise or a ‘serrated’ tone was used a deviant tone. The deviant trial occurred about every tenth trial. There were a total of 816 trials divided across eight blocks, 102 trials per block. After four blocks a switch between the deviant sounds was conducted. The order of the deviant tones was counter-balanced. Experiment 2 had two conditions. Condition 1 was identical to the design in Report III. In condition 2, however, the predictability of deviant trials was manipulated by incorporating pseudo- randomized interval lengths between deviant trials. The design of Experi- ment 3 was similar to Experiment 1, however, instead of switching between two different deviant tones, the modality in which the deviant was presented shifted from auditory to visual and vice versa. A burst of white noise was used in the auditory condition and a red ‘blob’ covering approximately 80% of the computer screen was used in the visual condition.

Results and discussion The results in Experiment 1 (see Figure 11) show that the overall habituation process remains even though the deviant sound is switched in the middle of the experiment and thereby violates any specific tone-bound expectations. This suggests that the nature of the habituation process is underpinned by a more general mechanism. This assumption received further support in Expe- riment 3 (see Figure 12) which revealed that habituation persists even though a switch of the modality (from auditory to visual and vice versa) in which the deviant is presented occurred.

18 Figure 11. Mean response times in Experiment 1 over eight consecutive blocks of trials. The deviant switch occurred between block 4 and block 5. Error bars represent standard error of means.

Figure 12. Mean response times in Experiment 3 over eight consecutive blocks of trials. The deviant switch between visual and auditory occurred between block 4 and block 5. Error bars represent standard error of means.

The findings in Experiment 2 (see Figure 13) provide further evidence in line with this notion as habituation seems unaffected by temporal predictability of the deviant trials. Collectively the results suggest that the habituation process is rather insensitive to prediction and changes in the environment as long as the general rule is not violated.

19 Figure 13. Mean response times in Experiment 2. The upper hand panel represents mean response times when the deviant tone occurred at a predictable interval and the lower hand panel show the mean response times when the occurrence of a deviant tone was pseudo-randomized. Error bars represent standard error of means.

20 3 General discussion

Implications for theories of attentional capture The aim of Report I and Report II was to investigate whether people use memory of past regularities in the sound environment to form expectations of future sound events, and whether violations of these expectations capture attention. To this end, a cross-modal oddball design in which the sounds preceding each visual target formed a repetitive sequence was used. Occa- sionally the sequence was interrupted by a pattern-deviant, which could have one of four different properties: A large/small pitch difference between the pattern-deviant and the standard tone it replaced and it could have the same/ opposite change of direction as the standard tone it replaced. The collective results revealed further support for the general idea that expectations underpin the deviation effect. Importantly, the results undermine the perceived local change account. If local change was the determinant for the deviation effect, then the pattern-deviants should have elicited the same amount of attentional capture regardless of which tone they replaced. This is not the case; the observed deviation effect is rather a function of the discrepancy between the presented and replaced tone. The exact nature of the formed expectations is, however, unclear. Whereas the results in Report I and in Experiment 1 of Report II indicate that expectations are fashioned around a specific stimulus (i.e. the specific stimulus hypothesis), the results in Experi- ment 2 of Report II show that violation of more general expectations regarding direction of pitch change are enough to elicit attentional capture. The somewhat surprising results in Experiment 3 of Report II suggest that both the local and global mechanisms are sensitive to, and may be influenced by, the complexity of the auditory environment. Another interesting aspect of attentional capture is that although expectations (and not perceived local change) seem to modulate attentional capture, base rate cannot be excluded as explanation for the phenomenon that sound can capture attention. Although the experiments here report that standard tones can capture attention (Experiment 2, Report I) and are by definition therefore not novel (or have low base rate) the context in which they appear has low base rate.

Implications for habituation By using a cross-modal oddball task the results in Report III revealed that the magnitude of the deviation effect decreases with the number of exposures over time and also as a function of working memory capacity. This finding

21 suggests that the orienting response elicited by a deviating event can be modulated by top-down control and in turn, habituation is not only a purely incidental and stimulus-driven process, it can be facilitated by cognitive control. Most importantly, the results indicate that individuals with high WMC seem superior at adapting to their environment. The aim of Report IV was to investigate the nature (and specificity) of the neural model formed in an auditory environment. According to Sokolov, the neural model should be sensitive to changes in the sound environment, and be in need of updating if changes occur. To address this question, a cross- modal oddball task in which an inter-experimental switch of the deviant sound occurred. If the neural model is fashioned around a specific stimulus then an observable increase of response latency should occur in conjunction with the deviant change. The results in Experiment 1, however, show that the habituation rate remained the same throughout the experiment. This finding together with the results from Experiment 3, indicate that the formed neural model may be of a more general nature than previously suggested. Whereas Experiment 1 was concerned with the specificity of the built neural model, the aim of Experiment 2 was to investigate what properties of the sound environment underpins habituation rate, more specifically if predictability of a deviant trial facilitates the habituation process. The finding that the habituation rate was similar whether there was a fixed temporal interval between the deviant trials or a random interval suggests that the amount of occurrences (i.e. number of deviant trials) determine habituation rate, not predictability. Collectively, the results indicate that once a certain complexity threshold is crossed, the habituation process depends on a rather general memory representation of the sound environment.

Memory systems as a tool for adaptation to the environment (Conclusion) The aim of the current thesis was to investigate the role of memory systems in human adaptation to the built environment. According to the findings in the reports presented in this thesis, the process of adaption can roughly be reduced into the following steps: (a) Learning (i.e. identifying regularities in the environment), (b) the formation of expectations and predictions based on the extracted regularities, (c) identifying irregularities (i.e. violations of expectations), (d) evaluating found irregularities (i.e. deciding an appropriate orientation response), (e) in case the irregularity holds no value, incorporate it into the neural model and thereby complete the habituation process. To reiterate: The ability to adapt to changes in the built environment is underpinned by a dynamic interplay between memory systems and distraction (i.e. irregularities in the environment).

Remarks for future research Although the work presented in this thesis sheds some new light on how memory systems are used to adapt to the environment, some topics explored

22 need further research. An especially interesting area is how the adaptation process works in more complex environments. As can be seen in Report II, the expectations change as a function of complexity (i.e. go from specific to general), and extended work is needed to find the exact threshold in which the expectations transgress from being bound to a specific stimulus to be of a more general nature. The same pattern is seen in the habituation process, it seems to be more general than previously suggested. Not only does it adapt to changes within the same modality, the finding that habituation can transgress between modalities suggests that habituation is more reliant on abstract (and general) rules. The studies reported here only consider behavioral data and in order to gain further insight regarding the habituation process replica- tion by using the MMN paradigm may be advisable. Further research regarding the role of complex cognition in more complex environments is also needed.

23 24 References

Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in cognitive sciences, 16(8), 437-443. Banbury, S., & Berry, D. C. (1997). Habituation and dishabituation to speech and office noise. Journal of Experimental Psychology: Applied, 3(3), 181. Bar, M. (2003). A cortical mechanism for triggering top-down facilitation in visual object recognition. Journal of cognitive neuroscience, 15(4), 600-609. Bar, M. (2007). The proactive brain: using analogies and associations to generate predictions. Trends in cognitive sciences, 11(7), 280-289. Bar, M. (2009). The proactive brain: memory for predictions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1521), 1235-1243. Beaman, C. P., & Jones, D. M. (1998). Irrelevant sound disrupts order information in free recall as in serial recall. The Quarterly Journal of Experimental Psychology: Section A, 51(3), 615-636. Bell, R., Buchner, A., & Mund, I. (2008). Age-related differences in irrelevant-speech effects. Psychology and aging, 23(2), 377. Bell, R., Dentale, S., Buchner, A., & Mayr, S. (2010). ERP correlates of the irrelevant sound effect. Psychophysiology, 47(6), 1182-1191. Bell, R., Röer, J. P., Dentale, S., & Buchner, A. (2012). Habituation of the irrelevant sound effect: Evidence for an attentional theory of short-term memory disruption. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(6), 1542. Bendixen, A., Schröger, E., & Winkler, I. (2009). I heard that coming: event-related potential evidence for stimulus-driven prediction in the auditory system. The Journal of Neuroscience, 29(26), 8447-8451. Boyle, R. (1996). Effects of irrelevant sounds on phonological coding in reading comprehension and short term memory. The Quarterly Journal of Experimental Psychology: Section A, 49(2), 398-416. Chein, J. M., & Fiez, J. A. (2010). Evaluating models of working memory through the effects of concurrent irrelevant information. Journal of Experimental Psychology: General, 139(1), 117. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. The Journal of the acoustical society of America, 25(5), 975-979. Conway, A. R., Cowan, N., & Bunting, M. F. (2001). The cocktail party phenomenon revisited: The importance of working memory capacity. Psychonomic bulletin & review, 8(2), 331- 335. Cowan, N., Winkler, I., Teder, W., & Näätänen, R. (1993). Memory prerequisites of mismatch negativity in the auditory event-related potential (ERP). Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(4), 909. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of verbal learning and verbal behavior, 19(4), 450-466. Evans, G. W. (2003). The built environment and mental health. Journal of Urban Health, 80(4), 536-555. Evans, G. W., Bullinger, M., & Hygge, S. (1998). Chronic noise exposure and physiological response: A prospective study of children living under environmental stress. Psychological science, 9(1), 75-77. Evans, G. W., Hygge, S., & Bullinger, M. (1995). Chronic noise and psychological stress. Psychological Science, 333-338.

25 Friedman, N. P., & Miyake, A. (2004). The relations among inhibition and interference control functions: a latent-variable analysis. Journal of experimental psychology: General, 133(1), 101. Gigerenzer, G. (2008). Why heuristics work. Perspectives on psychological science, 3(1), 20-29. Haenschel, C., Vernon, D. J., Dwivedi, P., Gruzelier, J. H., & Baldeweg, T. (2005). Event-related brain potential correlates of human auditory sensory memory-trace formation. The Journal of Neuroscience, 25(45), 10494-10501. Halin, N., Marsh, J. E., Haga, A., Holmgren, M., & Sörqvist, P. (2014). Effects of speech on proofreading: Can task-engagement manipulations shield against distraction?. Journal of Experimental Psychology: Applied, 20(1), 69. Hasher, L., Lustig, C., & Zacks, R. T. (2007). Inhibitory mechanisms and the control of attention. Variation in working memory, 227-249. Heitz, R. P., & Engle, R. W. (2007). Focusing the spotlight: individual differences in visual attention control. Journal of Experimental Psychology: General, 136(2), 217. Hughes, R. W., Hurlstone, M. J., Marsh, J. E., Vachon, F., & Jones, D. M. (2013). Cognitive control of auditory distraction: impact of task difficulty, foreknowledge, and working memory capacity supports duplex-mechanism account. Journal of Experimental Psychology: Human Perception and Performance, 39(2), 539. Hughes, R. W., Vachon, F., & Jones, D. M. (2005). Auditory attentional capture during serial recall: Violations at encoding of an algorithm-based neural model?. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(4), 736. Hughes, R.W., Vachon, F., & Jones, D. M. (2007). Disruption of shortterm memory by changing and deviant sounds: Support for a duplex-mechanism account of auditory distraction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 1050-1061. Hutchison, K. A. (2011). The interactive effects of listwide control, item-based control, and working memory capacity on Stroop performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(4), 851. Hygge, S., Evans, G. W., & Bullinger, M. (2002). A prospective study of some effects of aircraft noise on cognitive performance in schoolchildren. Psychological Science, 13(5), 469-474. Jaramillo, M., Paavilainen P., & Näätänen, R. (2000). Mismatch negativity and behavioural discrimination in humans as a function of the magnitude of change in sound duration. Neuroscience Letters, 290(2): 101-104. Jones, D. M., & Macken, W. J. (1993). Irrelevant tones produce an irrelevant speech effect: Implications for phonological coding in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 369-381. Jones, D. M., Hughes, R. W., & Macken, W. J. (2010). Auditory distraction and serial memory: The avoidable and the ineluctable. Noise and Health, 12, 201–209. Keus van de Poll, M. K., Ljung, R., Odelius, J., & Sörqvist, P. (2014). Disruption of writing by background speech: The role of speech transmission index. Applied Acoustics, 81, 15-18. Lercher, P., Evans, G. W., & Meis, M. (2003). Ambient noise and cognitive processes among primary schoolchildren. Environment and Behavior, 35(6), 725-735. Lustig, C., May, C. P., & Hasher, L. (2001). Working memory span and the role of proactive interference. Journal of Experimental Psychology: General, 130(2), 199. Maxwell, L. E., & Evans, G. W. (2000). The effects of noise on pre-school children’s pre-reading skills. Journal of environmental Psychology, 20(1), 91-97. Molholm, S., Martinez, A., Ritter, W., Javitt, D. C., & Foxe, J. J. (2005). The neural circuitry of pre-attentive auditory change-detection: an fMRI study of pitch and duration mismatch negativity generators. Cerebral Cortex, 15(5), 545-551.

26 Moray, N. (1959). Attention in dichotic listening: Affective cues and the influence of instructions. Quarterly journal of experimental psychology, 11(1), 56-60. Morris, N., & Jones, D. M. (1990). Habituation to irrelevant speech: Effects on a visual short- term memory task. Perception & Psychophysics, 47(3), 291-297. Näätänen, R. (1990). The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behavioral and Brain Sciences, 13, 201-233. Näätänen, R. (1995). The mismatch negativity: a powerful tool for cognitive neuroscience. Ear and hearing, 16(1), 6-18. Näätänen, R., Jacobsen, T., & Winkler, I. (2005). Memory-based or afferent processes in mismatch negativity (MMN): A review of the evidence. Psychophysiology, 42(1), 25-32. Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology, 118(12), 2544-2590. Oswald, C. J., Tremblay, S., & Jones, D. M. (2000). Disruption of comprehension by the mean- ing of irrelevant sound. Memory, 8(5), 345-350. Ouis, D. (2001). Annoyance from road traffic noise: a review. Journal of environmental psychology, 21(1), 101-120. Parmentier, F. B. (2008). Towards a cognitive model of distraction by auditory novelty: The role of involuntary attention capture and semantic processing. Cognition, 109(3), 345-362. Parmentier, F. B., & Hebrero, M. (2013). Cognitive control of involuntary distraction by deviant sounds. Journal of experimental psychology: learning, memory, and cognition, 39(5), 1635. Parmentier, F. B., Elsley, J. V., Andrés, P., & Barceló, F. (2011). Why are auditory novels distracting? Contrasting the roles of novelty, violation of expectation and stimulus change. Cognition, 119(3), 374-380. Röer, J. P., Bell, R., & Buchner, A. (2014). Evidence for habituation of the irrelevant-sound effect on serial recall. Memory & cognition, 42(4), 609-621. Röer, J. P., Bell, R., Dentale, S., & Buchner, A. (2011). The role of habituation and attentional orienting in the disruption of short-term memory performance. Memory & cognition, 39(5), 839-850. Schröger, E. (1997). On the detection of auditory deviations: A pre-attentive activation model. Psychophysiology, 34, 245-257. Shelton, J. T., Elliott, E. M., Eaves, S. D., & Exner, A. L. (2009). The distracting effects of a ringing cell phone: An investigation of the laboratory and the classroom setting. Journal of Environmental Psychology, 29(4), 513-521. Shipstead, Z., Harrison, T. L., & Engle, R. W. (2012). Working memory capacity and visual attention: Top-down and bottom-up guidance. The Quarterly Journal of Experimental Psychology, 65(3), 401-407. Sokolov, E. N. (1960). Neuronal models and the orienting reflex. The central nervous system and behavior, 187-276. Stansfeld, S. A., Berglund, B., Clark, C., Lopez-Barrio, I., Fischer, P., Öhrström, E., M M Haines, M. M, Head, J., Hygge, S., van Kamp, I., Berry, B. F., & RANCH Study Team. (2005). Aircraft and road traffic noise and children’s cognition and health: a cross-national study. The Lancet, 365(9475), 1942-1949. Stansfeld, S. A., Haines, M. M., Burr, M., Berry, B., & Lercher, P. (2000). A review of environmental noise and mental health. Noise and Health, 2(8), 1. Sussman, E., Winkler, I., & Schröger, E. (2003). Top-down control over involuntary attention switching in the auditory modality. Psychonomic Bulletin & Review, 10(3), 630-637.

27 Sörqvist, P., Ljungberg, J. K., & Ljung, R. (2010). A sub-process view of working memory capacity: evidence from effects of speech on prose memory. Memory, 18(3), 310-326. Sörqvist, P., Nöstl, A., & Halin, N. (2012). Disruption of writing processes by the semanticity of background speech. Scandinavian journal of psychology, 53(2), 97-102. Sörqvist, P., Stenfelt, S., & Rönnberg, J. (2012). Working memory capacity and visual – verbal cognitive load modulate auditory – sensory gating in the brainstem: toward a unified view of attention. Journal of cognitive neuroscience, 24(11), 2147-2154. Thompson, R. F., & Spencer, W. A. (1966). Habituation: a model phenomenon for the study of neuronal substrates of behavior. Psychological review, 73(1), 16. Tremblay, S., & Jones, D. M. (1998). Role of habituation in the irrelevant sound effect: Evidence from the effects of token set size and rate of transition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(3), 659. Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent?. Journal of memory and language, 28(2), 127-154. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124-1131. Unsworth, N., & Engle, R. W. (2007). The nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory. Psychological review, 114(1), 104. Vachon, F., Hughes, R. W., & Jones, D. M. (2012). Broken expectations: violation of expectancies, not novelty, captures auditory attention. Journal of experimental psychology: learning, memory, and cognition, 38(1), 164. Wood, N., & Cowan, N. (1995). The cocktail party phenomenon revisited: how frequent are attention shifts to one’s name in an irrelevant auditory channel?. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(1), 255.

28 Papers Associated papers have been removed in the electronic version of this thesis.

For more details about the papers see: http://urn.kb.se/resolve?urn=urn:nbn:se:hig:diva-20082 How memory of the past, a predictable present and expectations of the future underpin adaptation to the sound environment

Gävle University Press show that the disruptive effect of the deviation effect ISBN 978-91-88145-00-0 diminishes with the number of exposures over time, ISBN 978-91-88145-01-7 (pdf) and also as a function of working memory capacity. Furthermore, the finding that habituation occurs even though the nature of the deviant changed over time University of Gävle suggests that the formed neural model may be of Faculty of Engineering a more general nature than previously suggested. and Sustainable Development Considering that the habituation rate was similar SE-801 76 Gävle, Sweden whether there was a fixed temporal interval between +46 26 64 85 00 the deviant trials or a random interval suggests www.hig.se that the amount of occurrences (i.e. number of deviant trials) determines habituation rate, not the predictability of a deviant trial.

Anatole Nöstl