Sensory Memory Is Allocated Exclusively to the Current Event-Segment
Total Page:16
File Type:pdf, Size:1020Kb
Sensory memory is allocated exclusively to the current event-segment Item Type Article Authors Tripathy, Srimant P.; Ögmen, H. Citation Tripathy SP and Ögmen H (2018) Sensory memory is allocated exclusively to the current event-segment. Frontiers in Psychology. 9: 1435. Rights (c) 2018 The Authors. This is an Open Access article distributed under the Creative Commons CC-BY license (http:// creativecommons.org/licenses/by/4.0/) Download date 28/09/2021 02:42:01 Link to Item http://hdl.handle.net/10454/16722 ORIGINAL RESEARCH published: 07 September 2018 doi: 10.3389/fpsyg.2018.01435 Sensory Memory Is Allocated Exclusively to the Current Event-Segment Srimant P. Tripathy 1* and Haluk Ögmenˇ 2 1 School of Optometry and Vision Science, University of Bradford, Bradford, United Kingdom, 2 Department of Electrical and Computer Engineering, University of Denver, Denver, CO, United States The Atkinson-Shiffrin modal model forms the foundation of our understanding of human memory. It consists of three stores (Sensory Memory (SM), also called iconic memory, Short-Term Memory (STM), and Long-Term Memory (LTM)), each tuned to a different time-scale. Since its inception, the STM and LTM components of the modal model have undergone significant modifications, while SM has remained largely unchanged, Edited by: representing a large capacity system funneling information into STM. In the laboratory, Qasim Zaidi, visual memory is usually tested by presenting a brief static stimulus and, after a delay, University at Buffalo, United States asking observers to report some aspect of the stimulus. However, under ecological Reviewed by: Ronald A. Rensink, viewing conditions, our visual system receives a continuous stream of inputs, which University of British Columbia, Canada is segmented into distinct spatio-temporal segments, called events. Events are further Stephen Emrich, segmented into event-segments. Here we show that SM is not an unspecific general Brock University, Canada funnel to STM but is allocated exclusively to the current event-segment. We used a *Correspondence: Srimant P. Tripathy Multiple-Object Tracking (MOT) paradigm in which observers were presented with disks [email protected] moving in different directions, along bi-linear trajectories, i.e., linear trajectories, with a single deviation in direction at the mid-point of each trajectory. The synchronized deviation Specialty section: This article was submitted to of all of the trajectories produced an event stimulus consisting of two event-segments. Perception Science, Observers reported the pre-deviation or the post-deviation directions of the trajectories. a section of the journal Frontiers in Psychology By analyzing observers’ responses in partial- and full-report conditions, we investigated the involvement of SM for the two event-segments. The hallmarks of SM hold only for the Received: 14 October 2017 Accepted: 23 July 2018 current event segment. As the large capacity SM stores only items involved in the current Published: 07 September 2018 event-segment, the need for event-tagging in SM is eliminated, speeding up processing Citation: in active vision. By characterizing how memory systems are interfaced with ecological Tripathy SP and Ögmenˇ H (2018) Sensory Memory Is Allocated events, this new model extends the Atkinson-Shiffrin model by specifying how events Exclusively to the Current are stored in the first stage of multi-store memory systems. Event-Segment. Front. Psychol. 9:1435. Keywords: memory, sensory memory, iconic memory, short-term memory, modal model of memory, event doi: 10.3389/fpsyg.2018.01435 segmentation, tracking, multiple-object tracking Frontiers in Psychology | www.frontiersin.org 1 September 2018 | Volume 9 | Article 1435 Tripathy and Ögmenˇ Sensory Memory Allocation to Event-Segments INTRODUCTION Herzog, 2016). According to this model, SM has two parallel components, one based on a retinotopic reference-frame (rSM) The classical multi-store (or modal) model of human memory and a second one based on motion-grouping based (non- (Atkinson and Shiffrin, 1968, 1971) posits that information is retinotopic) reference frames (nrSM). encoded and stored in three memory systems (Figure 1A): First, Another recent modification to the multi-store functional a large-capacity but rapidly decaying SM stores information for architecture of memory systems has been the introduction of a few 100 ms. A subset of the contents of SM is transferred into a new visual memory component, intermediate between SM a more durable STM, which can store items for a few seconds. and STM (Sligte et al., 2008; van Moorselaar et al., 2015). However, STM’s capacity is severely limited. Finally, some items This component is proposed to have large capacity and a in STM are transferred into LTM whose contents can last as long retention time in the order of several seconds. It was suggested as a lifetime. that this memory component lacks robustness with respect Since its inception, the STM and LTM components of to changing inputs and hence was termed fragile visual STM. the modal model have undergone significant modifications Evidence suggests that the erasure of the contents of the (Baddeley and Hitch, 1974; Klatzky, 1980; Cowan, 2001; Alvarez fragile STM depends on both location-specific and object-specific and Cavanagh, 2004; Baddeley, 2007; Bays and Husain, 2008; matches between the contents and the new input, leading to Zhang and Luck, 2008; Van den Berg et al., 2012), while the the proposal that this memory component holds higher object- classical view of SM as a large capacity system funneling a static level representations but that these representations are anchored snapshot of visual inputs into STM has persisted for several to specific locations (Pinto et al., 2013). However, other studies decades (Sperling, 1960; Averbach and Coriell, 1961; Coltheart, have questioned the existence of fragile VSTM (Matsukura and 1983; Haber, 1983; Breitmeyer and Ögmen,ˇ 2006). Recent models Hollingworth, 2011; Makovski, 2012). extended SM in terms of its architecture and function. In the laboratory, visual memory is usually tested by In terms of how SM is implemented in the brain, recently presenting a brief static stimulus (e.g., shapes with different a multi-layer approach reflecting the hierarchy of the visual colors) and, after a delay, asking observers to report some cortex has been proposed (Rensink, 2014). It was found that aspect of the stimulus. However, under normal ecological viewing the retention-duration in SM depends on the nature (but not conditions, due to the movements of the observer and those on the difficulty) of the task that SM is called to subserve of objects in the environment, our visual system receives a (Rensink, 2014). For change detection, target-identity report, continuous stream of dynamic inputs. Perceptually, the visual and static detection tasks, the corresponding SM retention- system segments the continuous stream of visual information durations were 120, 190, and 240ms, respectively. This finding into distinct spatio-temporal segments, called events (Johansson was interpreted to reflect a multi-layer cortical implementation et al., 1980; Warren and Shaw, 1985). For example, flipping a of SM, according to which SM can store information and make it coin can be considered as an event, which in turn can have sub- available both within and across the cortical layers by relying on segments (Zacks and Swallow, 2007), such as the launching of horizontal and vertical feedback, respectively. Hence, according the coin, following the movement of the coin in the air, and to this model, the architecture of SM is distributed across the catching the coin, all occurring within a second. An important cortex following the hierarchy of visual cortical processes. This question is then how events and their segments are stored in distributed implementation suggests that information flow from SM. Previous studies examined how information embedded in SM to STM and to other visual processes occurs within and across events is stored in STM and LTM. It has been reported that cortical layers because, architecturally, SM is an integral part of objects that were present around event boundaries were better various visual processing networks. recalled than other objects in the stimulus (Swallow et al., Another aspect of cortical organization is retinotopy: Via 2009). This finding has been interpreted as evidence that event the optics of the eyes, neighboring points in the environment boundaries structure the contents of STM and LTM (Swallow are mapped to neighboring points on the retina and these et al., 2009). Given this background, our goal was to analyse the neighborhood relations are preserved in early visual cortical relationship between event segments and memory mechanisms, areas. Under ecological viewing conditions, the observer (eyes, in particular for relatively short events that span the first two head, body) and many objects in the environment are in stages of the modal model, namely SM and STM. We focused motion. While a retinotopically encoded SM can store and on these two memory stores because, due to their time-scales, process information that is stable (i.e., static) with respect they are inherently involved in real-time ecological aspects of to its retinotopic reference-frame, the storage and processing perception and cognition and because they can be distinguished of dynamic information is problematic (Haber, 1983): Any from each other by well-established experimental techniques relative