ACT-R and fMRI 1 Using the ACT-R Cognitive Architecture in combination with fMRI Data Jelmer P. Borst ([email protected]) & John R. Anderson Carnegie Mellon University Please cite as: Borst, J.P, Taatgen & Anderson, J.R. (in press). Using the ACT-R Cognitive Architecture in combination with fMRI data. In B. U. Forstmann, & E.-J. Wagenmakers (Eds.), An Introduction to Model-Based Cognitive Neuroscience. Springer: New York. ACT-R and fMRI 2 Abstract In this chapter we discuss how the ACT-R cognitive architecture can be used in combination with fMRI data. ACT-R is a cognitive architecture that can provide a description of the processes from perception through to action for a wide range of cognitive tasks. It has a computational implementation that can be used to create models of specific tasks, which yield exact predictions in the form of response times and accuracy measures. In the last decade, researchers have extended the predictive capabilities of ACT-R to fMRI data. Since ACT-R provides a model of all the components in task performance it can address brain-wide activation patterns. fMRI data can now be used to inform and constrain the architecture, and, on the other hand, the architecture can be used to interpret fMRI data in a principled manner. In the following sections we first introduce cognitive architectures, and ACT-R in particular. Then, on the basis of an example dataset, we explain how ACT-R can be used to create fMRI predictions. In the third and fourth section of this chapter we discuss two ways in which these predictions can be used: region-of- interest and model-based fMRI analysis, and how the results can be used to inform the architecture and to interpret fMRI data. Keywords: ACT-R; Cognitive Architecture; fMRI; model-based fMRI; ROI analysis. ACT-R and fMRI 3 Introduction In 1973, Newell wrote a commentary in which he caricatured the current psychological practice as “playing a game of 20 questions [with nature]” [1]. While Newell considered the individual experiments and theories presented at the symposium to be “exceptionally fine” (p. 291), he was worried that the results would never be integrated into an overarching theory of the mind. As a solution, Newell proposed the idea of cognitive architectures (the actual term is not in his 1973 paper but was well in use at CMU when Anderson arrived in 1978; see for instance [2]). A cognitive architecture is first and foremost a psychological theory: it explains for instance how our memory system works. Instead of being limited to a single psychological construct, however, architectures typically account for complete tasks, from perception to response execution. In addition – and unlike most classical psychological theories – a cognitive architecture is implemented as a computer simulation, which can be used to create cognitive models of specific tasks (e.g., the Stroop task, associative recognition, driving a car). This approach has multiple advantages. First, the models yield precise predictions, for instance reaction times and accuracy measures. Particularly when complete tasks are modeled – often models even interact with the same interface as human subjects – a direct comparison with human data is possible. Second, the underlying psychological components (e.g., memory, vision) are shared by the different tasks, and have to be truly general. If a simulated memory system only works for a single task it probably contains too many task-specific constructs. A cognitive architecture forces one to keep the components general enough to work for many different tasks. Third, because complete tasks are modeled, interactions between perception and central cognition (and between cognitive components themselves) arise naturally from the architecture, which can have a large impact on experimental results [3,4]. For decades, models developed in cognitive architectures were validated using response times, accuracy measures, and sometimes eye movements [e.g., 5]. However, behavioral data ACT-R and fMRI 4 does not always provide enough constraints to distinguish between different models [6; Chapter 13]. For example, the time leading up to a response typically consists of multiple cognitive steps, which can be arranged in different ways. Researchers turned to neuroimaging data for additional constraints and guidance in developing architectures [e.g., 6,7]. Cognitive architectures are well- matched to fMRI data: One cannot ignore any of the perceptual, cognitive, or motor components of a task when designing or interpreting fMRI experiments (because they all show up in brain activity) and a cognitive architecture requires that the modeler address all of these components (to get a running model). In this chapter we describe how the cognitive architecture ACT-R can be used in combination with fMRI data. We will first explain ACT-R in some detail. Then, based on an example task, we will demonstrate the different steps of generating fMRI predictions from an ACT-R model. Subsequently, we discuss two different ways of using these predictions: a region- of-interest analysis and a model-based fMRI analysis. We conclude with a short section on how the two methods complement each other. ACT-R Currently, several cognitive architectures are in use, for example SOAR [2], ACT-R [6], EPIC [8], and 4CAPS [7]. In this chapter we will focus on ACT-R1, because it has an explicit mapping between components of the architecture and brain regions. However, most ideas in this chapter are also applicable to other architectures. 1 For the range of tasks (and associated publications) that have been modeled with ACT-R, see http://act-r.psy.cmu.edu/. ACT-R can also be downloaded from this website. ACT-R and fMRI 5 a) b) ACT-R Control State Problem Declarative State Memory +6060 +50 +40 Visual Procedural Manual Perception Module Control +30 +20 +10 Aural Vocal Perception Control +0 External World +0 -10 -20 Figure 1. The main modules of ACT-R (a) and associated brain regions (b). Numbers indicate the z-coordinate of each slice (MNI coordinates); the colors of the regions correspond to the colors in (a). ACT-R consists of a set of independent modules that function around a central procedural module (Figure 1a). There are modules for perception (visual and aural) and action (manual and vocal), and several central cognitive modules (for details on the individual modules, see Anderson, 2007, or Anderson, 2005). The modules interact with the procedural module through buffers of limited size. The procedural module consists of rules that specify what cognitive action to take given the contents of the buffers. For instance, a rule might request the retrieval of the meaning of word encoded in the visual buffer. An ACT-R model consists of such rules and of knowledge in declarative memory (e.g., the meaning of the word ‘chair’). Thus, ACT-R itself can be seen as the fixed hardware – the architecture – of the mind, while the models function as software that runs on this hardware. The modules of ACT-R have been mapped onto small regions in the brain, which are shown in Figure 1b. These regions are assumed to be active when the corresponding module is active (see the section on region-of-interest analysis). ACT-R and fMRI 6 a) Paired Trial 2.0 s 6.0 s 6.0 s 6.0 s 2.0 s 6.0 s b) Generated Trial 2.0 s 6.0 s 6.0 s 6.0 s 2.0 s 6.0 s Figure 2. Experimental procedure. Adapted from Figure 1 in Anderson, Byrne, et al. (2008) by permission of the publisher. Copyright 2008 of the original by Oxford University Press. Using ACT-R to predict fMRI data In this section we will describe how ACT-R can be used to predict fMRI data. First, we describe the task that we will use as an example throughout this chapter. We will then introduce the model, followed by how it can be used to generate fMRI predictions. The Lisp code for the model and Matlab code to generate the predictions can be downloaded from http://act- r.psy.cmu.edu/, under the title of this chapter. The Example Task: Associative Fan To illustrate the analysis we will use a previously published experiment with an associated ACT-R model (Experiment 2, [9]). This experiment was designed to test the assumption that declarative memory activity is reflected by a region in the prefrontal cortex (see Figure 1b, the pink regions), while representational activity of the problem state module (roughly comparable to a capacity-limited working memory store, e.g., [10]) is reflected by a region in the ACT-R and fMRI 7 posterior parietal cortex (Figure 1b, dark blue regions). To this end, memory and representational requirements were independently manipulated in an associative recognition task. Figure 2a shows the basic procedure. A trial started with a 2 second fixation screen, followed by a 6 second study presentation of a paired-associate. Subjects were asked to memorize the paired-associate that was presented, in this case ‘band – 2’. The study probe was followed by a 6-second fixation screen, after which a test probe was shown for a maximum of 6 seconds or until the response was given. The test probe consisted of a word (i.e., ‘band’); subjects had to respond with the associated number (i.e., ‘2’). Memory requirements were manipulated within-subject by varying the delay between study and test items. The trial in Figure 2a is an example of having a study and test item in the same trial, but they could be as far as 7 trials apart. There were three levels: no delay, short delay (1-2 trials), and long delay (6-7 trials).