DOCSLIB.ORG

Reinforcement Learning: the Good, the Bad and the Ugly Peter Dayana and Yael Nivb

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Reinforcement Learning: the Good, the Bad and the Ugly Peter Dayana and Yael Nivb CONEUR-580; NO OF PAGES 12 Available online at www.sciencedirect.com Reinforcement learning: The Good, The Bad and The Ugly Peter Dayana and Yael Nivb Reinforcement learning provides both qualitative and huge range of paradigms and systems. The literature in quantitative frameworks for understanding and modeling this area is by now extensive, and has been the topic of adaptive decision-making in the face of rewards and many recent reviews (including [5–9]). This is in addition punishments. Here we review the latest dispatches from the to rapidly accumulating literature on the partly related forefront of this field, and map out some of the territories where questions of optimal decision-making in situations invol- lie monsters. ving slowly amounting information, or social factors such Addresses as games [10–12]. Thus here, after providing a brief a UCL, United Kingdom sketch of the overall RL scheme for control (for a more b Psychology Department and Princeton Neuroscience Institute, extensive review, see [13]), we focus only on some of the Princeton University, United States many latest results relevant to RL and its neural instan- Corresponding authors: Dayan, Peter ([email protected]) and tiation. We categorize these recent findings into those Niv, Yael ([email protected]) that fit comfortably with, or flesh out, accepted notions (playfully, ‘The Good’), some new findings that are not as snugly accommodated, but suggest the need for exten- Current Opinion in Neurobiology 2008, 18:1–12 sions or modifications (‘The Bad’), and finally some key This review comes from a themed issue on areas whose relative neglect by the field is threatening to Cognitive neuroscience impede its further progress (‘The Ugly’). Edited by Read Montague and John Assad The reinforcement learning framework 0959-4388/$ – see front matter Decision-making environments are characterized by a # 2008 Elsevier Ltd. All rights reserved. few key concepts: a state space (states are such things as locations in a maze, the existence or absence of DOI 10.1016/j.conb.2008.08.003 different stimuli in an operant box or board positions in a game), a set of actions (directions of travel, presses on different levers, and moves on a board), and affectively Introduction important outcomes (finding cheese, obtaining water, and Reinforcement learning (RL) [1] studies the way that winning). Actions can move the decision-maker from one natural and artificial systems can learn to predict the con- state to another (i.e. induce state transitions) and they can sequences of and optimize their behavior in environments produce outcomes. The outcomes are assumed to have in which actions lead them from one state or situation to the numerical (positive or negative) utilities, which can next, and can also lead to rewards and punishments. Such change according to the motivational state of the environments arise in a wide range of fields, including decision-maker (e.g. food is less valuable to a satiated ethology, economics, psychology, and control theory. animal) or direct experimental manipulation (e.g. poison- Animals, from the most humble to the most immodest, ing). Typically, the decision-maker starts off not knowing face a range of such optimization problems [2],and,toan the rules of the environment (the transitions and out- apparently impressive extent, solve them effectively. RL, comes engendered by the actions), and has to learn or originally born out of mathematical psychology and oper- sample these from experience. ations research, provides qualitative and quantitative com- putational-level models of these solutions. In instrumental conditioning, animals learn to choose actions to obtain rewards and avoid punishments, or, However, the reason for this review is the increasing more generally to achieve goals. Various goals are realization that RL may offer more than just a compu- possible, such as optimizing the average rate of acqui- tational, ‘approximate ideal learner’ theory for affective sition of net rewards (i.e. rewards minus punishments), or decision-making. RL algorithms, such as the temporal some proxy for this such as the expected sum of future difference (TD) learning rule [3], appear to be directly rewards, where outcomes received in the far future are instantiated in neural mechanisms, such as the phasic discounted compared with outcomes received more activity of dopamine neurons [4]. That RL appears to be immediately. It is the long-term nature of these goals so transparently embedded has made it possible to use it that necessitates consideration not only of immediate in a much more immediate way to make hypotheses outcomes but also of state transitions, and makes choice about, and retrodictive and predictive interpretations interesting and difficult. In terms of RL, instrumental of, a wealth of behavioral and neural data collected in a conditioning concerns optimal choice, that is determining www.sciencedirect.com Current Opinion in Neurobiology 2008, 18:1–12 Please cite this article in press as: Dayan P, Niv Y, Reinforcement learning: The Good, The Bad and The Ugly, Curr Opin Neurobiol (2008), doi:10.1016/j.conb.2008.08.003 CONEUR-580; NO OF PAGES 12 2 Cognitive neuroscience an assignment of actions to states, also known as a policy have high values themselves). Model-free learning rules that optimizes the subject’s goals. such as the temporal difference (TD) rule [3] define any momentary inconsistency as a prediction error, and use it to By contrast with instrumental conditioning, Pavlovian (or specify rules for plasticity that allow learning of more classical) conditioning traditionally treats how subjects accurate values and decreased inconsistencies. Given learn to predict their fate in those cases in which they correct values, it is possible to improve the policy by cannot actually influence it. Indeed, although RL is preferring those actions that lead to higher utility out- primarily concerned with situations in which action selec- comes and higher valued states. Direct model-free tion is germane, such predictions play a major role in methods for improving policies without even acquiring assessing the effects of different actions, and thereby in values are also known [21]. optimizing policies. In Pavlovian conditioning, the pre- dictions also lead to responses. However, unlike the Model-free methods are statistically less efficient than flexible policies that are learned via instrumental con- model-based methods, because information from the ditioning, Pavlovian responses are hard-wired to the environment is combined with previous, and possibly nature and emotional valence of the outcomes. erroneous, estimates or beliefs about state values, rather than being used directly. The information is also stored RL methods can be divided into two broad classes, model- in scalar quantities from which specific knowledge about based and model-free, which perform optimization in very rewards or transitions cannot later be disentangled. As different ways (Box 1, [14]). Model-based RL uses experi- a result, these methods cannot adapt appropriately ence to construct an internal model of the transitions and quickly to changes in contingency and outcome utilities. immediate outcomes in the environment. Appropriate Based on the latter characteristic, model-free RL has actions are then chosen by searching or planning in this been suggested as a model of habitual actions [14,15],in world model. This is a statistically efficient way to use which areas such as the dorsolateral striatum and the experience, as each morsel of information from the amygdala are believed to play a key role [17,18].How- environment can be stored in a statistically faithful and ever, a far more direct link between model-free RL and computationally manipulable way. Provided that constant the workings of affective decision-making is apparent in replanning is possible, this allows action selection to be the findings that the phasic activity of dopamine readily adaptive to changes in the transition contingencies neurons during appetitive conditioning (and indeed and the utilities of the outcomes. This flexibility makes the fMRI BOLD signal in the ventral striatum of model-based RL suitable for supporting goal-directed humans, a key target of dopamine projections) has many actions, in the terms of Dickinson and Balleine [15]. of the quantitative characteristics of the TD prediction For instance, in model-based RL, performance of actions error that is key to learning model-free values [4,22–24]. leading to rewards whose utilities have decreased is It is these latter results that underpin the bulk of neural immediately diminished. Via this identification and other RL. findings, the behavioral neuroscience of such goal- directed actions suggests a key role in model-based RL ‘The Good’: new findings in neural RL (or at least in its components such as outcome evaluation) Daw et al. [5] sketched a framework very similar to this, for the dorsomedial striatum (or its primate homologue, and reviewed the then current literature which pertained the caudate nucleus), prelimbic prefrontal cortex, the to it. Our first goal is to update this analysis of the orbitofrontal cortex, the medial prefrontal cortex,1 and literature. In particular, courtesy of a wealth of exper- parts of the amygdala [9,17–20]. iments, just two years later we now know much more about the functional organization of RL systems in the Model-free RL, on the other hand, uses experience to brain, the pathways influencing the computation of pre- learn directly one or both of two simpler quantities (state/ diction errors, and time-discounting. A substantial frac- action values or policies) which can achieve the same tion of this work involves mapping the extensive findings optimal behavior but without estimation or use of a world from rodents and primates onto the human brain, largely model. Given a policy, a state has a value, defined in terms using innovative experimental designs while measuring of the future utility that is expected to accrue starting the fMRI BOLD signal.
Load more

Recommended publications
  • Machine Learning for Neuroscience Geoffrey E Hinton
    Hinton Neural Systems & Circuits 2011, 1:12 http://www.neuralsystemsandcircuits.com/content/1/1/12 QUESTIONS&ANSWERS Open Access Machine learning for neuroscience Geoffrey E Hinton What is machine learning? Could you provide a brief description of the Machine learning is a type of statistics that places parti- methods of machine learning? cular emphasis on the use of advanced computational Machine learning can be divided into three parts: 1) in algorithms. As computers become more powerful, and supervised learning, the aim is to predict a class label or modern experimental methods in areas such as imaging a real value from an input (classifying objects in images generate vast bodies of data, machine learning is becom- or predicting the future value of a stock are examples of ing ever more important for extracting reliable and this type of learning); 2) in unsupervised learning, the meaningful relationships and for making accurate pre- aim is to discover good features for representing the dictions. Key strands of modern machine learning grew input data; and 3) in reinforcement learning, the aim is out of attempts to understand how large numbers of to discover what action should be performed next in interconnected, more or less neuron-like elements could order to maximize the eventual payoff. learn to achieve behaviourally meaningful computations and to extract useful features from images or sound What does machine learning have to do with waves. neuroscience? By the 1990s, key approaches had converged on an Machine learning has two very different relationships to elegant framework called ‘graphical models’, explained neuroscience. As with any other science, modern in Koller and Friedman, in which the nodes of a graph machine-learning methods can be very helpful in analys- represent variables such as edges and corners in an ing the data.
    [Show full text]
  • Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
    Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning Felipe Petroski Such Vashisht Madhavan Edoardo Conti Joel Lehman Kenneth O. Stanley Jeff Clune Uber AI Labs ffelipe.such, [email protected] Abstract 1. Introduction Deep artificial neural networks (DNNs) are typ- A recent trend in machine learning and AI research is that ically trained via gradient-based learning al- old algorithms work remarkably well when combined with gorithms, namely backpropagation. Evolution sufficient computing resources and data. That has been strategies (ES) can rival backprop-based algo- the story for (1) backpropagation applied to deep neu- rithms such as Q-learning and policy gradients ral networks in supervised learning tasks such as com- on challenging deep reinforcement learning (RL) puter vision (Krizhevsky et al., 2012) and voice recog- problems. However, ES can be considered a nition (Seide et al., 2011), (2) backpropagation for deep gradient-based algorithm because it performs neural networks combined with traditional reinforcement stochastic gradient descent via an operation sim- learning algorithms, such as Q-learning (Watkins & Dayan, ilar to a finite-difference approximation of the 1992; Mnih et al., 2015) or policy gradient (PG) methods gradient. That raises the question of whether (Sehnke et al., 2010; Mnih et al., 2016), and (3) evolution non-gradient-based evolutionary algorithms can strategies (ES) applied to reinforcement learning bench- work at DNN scales. Here we demonstrate they marks (Salimans et al., 2017). One common theme is can: we evolve the weights of a DNN with a sim- that all of these methods are gradient-based, including ES, ple, gradient-free, population-based genetic al- which involves a gradient approximation similar to finite gorithm (GA) and it performs well on hard deep differences (Williams, 1992; Wierstra et al., 2008; Sali- RL problems, including Atari and humanoid lo- mans et al., 2017).
    [Show full text]
  • Acetylcholine in Cortical Inference
    Neural Networks 15 (2002) 719–730 www.elsevier.com/locate/neunet 2002 Special issue Acetylcholine in cortical inference Angela J. Yu*, Peter Dayan Gatsby Computational Neuroscience Unit, University College London, 17 Queen Square, London WC1N 3AR, UK Received 5 October 2001; accepted 2 April 2002 Abstract Acetylcholine (ACh) plays an important role in a wide variety of cognitive tasks, such as perception, selective attention, associative learning, and memory. Extensive experimental and theoretical work in tasks involving learning and memory has suggested that ACh reports on unfamiliarity and controls plasticity and effective network connectivity. Based on these computational and implementational insights, we develop a theory of cholinergic modulation in perceptual inference. We propose that ACh levels reflect the uncertainty associated with top- down information, and have the effect of modulating the interaction between top-down and bottom-up processing in determining the appropriate neural representations for inputs. We illustrate our proposal by means of an hierarchical hidden Markov model, showing that cholinergic modulation of contextual information leads to appropriate perceptual inference. q 2002 Elsevier Science Ltd. All rights reserved. Keywords: Acetylcholine; Perception; Neuromodulation; Representational inference; Hidden Markov model; Attention 1. Introduction medial septum (MS), diagonal band of Broca (DBB), and nucleus basalis (NBM). Physiological studies on ACh Neuromodulators such as acetylcholine (ACh), sero- indicate that its neuromodulatory effects at the cellular tonine, dopamine, norepinephrine, and histamine play two level are diverse, causing synaptic facilitation and suppres- characteristic roles. One, most studied in vertebrate systems, sion as well as direct hyperpolarization and depolarization, concerns the control of plasticity. The other, most studied in all within the same cortical area (Kimura, Fukuda, & invertebrate systems, concerns the control of network Tsumoto, 1999).
    [Show full text]
  • Model-Free Episodic Control
    Model-Free Episodic Control Charles Blundell Benigno Uria Alexander Pritzel Google DeepMind Google DeepMind Google DeepMind [email protected] [email protected] [email protected] Yazhe Li Avraham Ruderman Joel Z Leibo Jack Rae Google DeepMind Google DeepMind Google DeepMind Google DeepMind [email protected] [email protected] [email protected] [email protected] Daan Wierstra Demis Hassabis Google DeepMind Google DeepMind [email protected] [email protected] Abstract State of the art deep reinforcement learning algorithms take many millions of inter- actions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision- making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains. 1 Introduction Deep reinforcement learning has recently achieved notable successes in a variety of domains [23, 32]. However, it is very data inefficient. For example, in the domain of Atari games [2], deep Reinforcement Learning (RL) systems typically require tens of millions of interactions with the game emulator, amounting to hundreds of hours of game play, to achieve human-level performance. As pointed out by [13], humans learn to play these games much faster. This paper addresses the question of how to emulate such fast learning abilities in a machine—without any domain-specific prior knowledge.
    [Show full text]
  • When Planning to Survive Goes Wrong: Predicting the Future and Replaying the Past in Anxiety and PTSD
    Available online at www.sciencedirect.com ScienceDirect When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD 1 3 1,2 Christopher Gagne , Peter Dayan and Sonia J Bishop We increase our probability of survival and wellbeing by lay out this computational problem as a form of approxi- minimizing our exposure to rare, extremely negative events. In mate Bayesian decision theory (BDT) [1] and consider this article, we examine the computations used to predict and how miscalibrated attempts to solve it might contribute to avoid such events and to update our models of the world and anxiety and stress disorders. action policies after their occurrence. We also consider how these computations might go wrong in anxiety disorders and According to BDT, we should combine a probability Post Traumatic Stress Disorder (PTSD). We review evidence distribution over all relevant states of the world with that anxiety is linked to increased simulations of the future estimates of the benefits or costs of outcomes associated occurrence of high cost negative events and to elevated with each state. We must then calculate the course of estimates of the probability of occurrence of such events. We action that delivers the largest long-run expected value. also review psychological theories of PTSD in the light of newer, Individuals can only possibly approximately solve this computational models of updating through replay and problem. To do so, they bring to bear different sources of simulation. We consider whether pathological levels of re- information (e.g. priors, evidence, models of the world) experiencing symptomatology might reflect problems and apply different methods to calculate the expected reconciling the traumatic outcome with overly optimistic priors long-run values of alternate courses of action.
    [Show full text]
  • Pnas11052ackreviewers 5098..5136
    Acknowledgment of Reviewers, 2013 The PNAS editors would like to thank all the individuals who dedicated their considerable time and expertise to the journal by serving as reviewers in 2013. Their generous contribution is deeply appreciated. A Harald Ade Takaaki Akaike Heather Allen Ariel Amir Scott Aaronson Karen Adelman Katerina Akassoglou Icarus Allen Ido Amit Stuart Aaronson Zach Adelman Arne Akbar John Allen Angelika Amon Adam Abate Pia Adelroth Erol Akcay Karen Allen Hubert Amrein Abul Abbas David Adelson Mark Akeson Lisa Allen Serge Amselem Tarek Abbas Alan Aderem Anna Akhmanova Nicola Allen Derk Amsen Jonathan Abbatt Neil Adger Shizuo Akira Paul Allen Esther Amstad Shahal Abbo Noam Adir Ramesh Akkina Philip Allen I. Jonathan Amster Patrick Abbot Jess Adkins Klaus Aktories Toby Allen Ronald Amundson Albert Abbott Elizabeth Adkins-Regan Muhammad Alam James Allison Katrin Amunts Geoff Abbott Roee Admon Eric Alani Mead Allison Myron Amusia Larry Abbott Walter Adriani Pietro Alano Isabel Allona Gynheung An Nicholas Abbott Ruedi Aebersold Cedric Alaux Robin Allshire Zhiqiang An Rasha Abdel Rahman Ueli Aebi Maher Alayyoubi Abigail Allwood Ranjit Anand Zalfa Abdel-Malek Martin Aeschlimann Richard Alba Julian Allwood Beau Ances Minori Abe Ruslan Afasizhev Salim Al-Babili Eric Alm David Andelman Kathryn Abel Markus Affolter Salvatore Albani Benjamin Alman John Anderies Asa Abeliovich Dritan Agalliu Silas Alben Steven Almo Gregor Anderluh John Aber David Agard Mark Alber Douglas Almond Bogi Andersen Geoff Abers Aneel Aggarwal Reka Albert Genevieve Almouzni George Andersen Rohan Abeyaratne Anurag Agrawal R. Craig Albertson Noga Alon Gregers Andersen Susan Abmayr Arun Agrawal Roy Alcalay Uri Alon Ken Andersen Ehab Abouheif Paul Agris Antonio Alcami Claudio Alonso Olaf Andersen Soman Abraham H.
    [Show full text]
  • Technical Note Q-Learning
    Machine Learning, 8, 279-292 (1992) © 1992 Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. Technical Note Q-Learning CHRISTOPHER J.C.H. WATKINS 25b Framfield Road, Highbury, London N5 1UU, England PETER DAYAN Centre for Cognitive Science, University of Edinburgh, 2 Buccleuch Place, Edinburgh EH8 9EH, Scotland Abstract. ~-learning (Watkins, 1989) is a simpleway for agentsto learn how to act optimallyin controlledMarkovian domains. It amounts to an incremental method for dynamic programming which imposes limited computational demands. It works by successivelyimproving its evaluationsof the quality of particular actions at particular states. This paper presents and proves in detail a convergencetheorem for ~-learning based on that outlined in Watkins (1989). We show that 0~-learningconverges to the optimum action-valueswith probability 1 so long as all actions are repeatedly sampled in all states and the action-values are represented discretely. We also sketch extensions to the cases of non-discounted, but absorbing, Markov environments, and where many O~values can be changed each iteration, rather than just one. Keywords. 0~-learning, reinforcement learning, temporal differences, asynchronous dynamic programming 1. Introduction O~-learning (Watkins, 1989) is a form of model-free reinforcement learning. It can also be viewed as a method of asynchronous dynamic programming (DP). It provides agents with the capability of learning to act optimally in Markovian domains by experiencing the con- sequences of actions, without requiring them to build maps of the domains. Learning proceeds similarly to Sutton's (1984; 1988) method of temporal differences (TD): an agent tries an action at a particular state, and evaluates its consequences in terms of the immediate reward or penalty it receives and its estimate of the value of the state to which it is taken.
    [Show full text]
  • Explore Cape May County's Past
    Volume 34, Number 1 Listed on the NJ State Register of Historic Places on September 27, 2016 2019 Explore Cape May County’s Past Celebrating 46 Years of History Education and Preservation WELCOME Historic Cold Spring Village opened to the What were the arduous household tasks the public in May 1981 with a three-fold mission women of the family needed to perform? Many of historic preservation, history education and of them were the very same jobs needed to heritage tourism. Its primary goal is to inform maintain a current home; only they were done visitors about how our ancestors lived and without the advantages of modern technology. worked two centuries ago. Among the many One household chore common to both our lovingly restored buildings where visitors can ancestors and ourselves was washing the fam- learn about the museum from our skilled and ily’s clothes, a task made easier today thanks friendly interpreters are a one-room school, to washing machines and dryers. However, INSIDE a tavern-hotel, a blacksmith shop, a basket- washing clothes in Early America was a daunt- Village Map ........................................................................................................... Page 2 making shop, a woodworking shop and a tin- ing task. Typically, two large, heavy iron caul- "The Cold Spring" ................................................................................................. Page 1 smithing shop. These artisans demonstrate the drons had to be set up outside, suspended from Historic Cold Spring Village Membership ............................................................ Page 3 physically demanding tasks and trades of an iron tripods. The children would take yokes, Building Key .......................................................................................................... Page 3 age long gone. designed for carrying buckets of water filled Calendar of Events 2019 ......................................................................................
    [Show full text]
  • Relieving Pain in America: a Blueprint for Transforming Prevention, Care, Education, and Research
    This PDF is available from The National Academies Press at http://www.nap.edu/catalog.php?record_id=13172 Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research ISBN Committee on Advancing Pain Research, Care, and Education; Institute of 978-0-309-25627-8 Medicine 382 pages 6 x 9 PAPERBACK (2011) Visit the National Academies Press online and register for... Instant access to free PDF downloads of titles from the NATIONAL ACADEMY OF SCIENCES NATIONAL ACADEMY OF ENGINEERING INSTITUTE OF MEDICINE NATIONAL RESEARCH COUNCIL 10% off print titles Custom notifcation of new releases in your feld of interest Special offers and discounts Distribution, posting, or copying of this PDF is strictly prohibited without written permission of the National Academies Press. Unless otherwise indicated, all materials in this PDF are copyrighted by the National Academy of Sciences. Request reprint permission for this book Copyright © National Academy of Sciences. All rights reserved. Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research Committee on Advancing Pain Research, Care, and Education Board on Health Sciences Policy Copyright © National Academy of Sciences. All rights reserved. Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
    [Show full text]
  • ED431868.Pdf
    DOCUMENT RESUME ED 431 868 CE 077 566 TITLE Focus on Basics, 1998. INSTITUTION National Center for the Study of Adult Learning and Literacy, Boston, MA. SPONS AGENCY Office of Educational Research and Improvement (ED), Washington, DC. PUB DATE 1998-00-00 NOTE 118p. CONTRACT R309B60002 AVAILABLE FROM Focus on Basics, 44 Farnsworth St., Boston, MA 02210-1211; e-mail: [email protected] PUB TYPE Collected Works - Serials (022) JOURNAL CIT Focus on Basics; v2 n1-4 1998 EDRS PRICE MF01/PC05 Plus Postage. DESCRIPTORS Academic Persistence; Adult Basic Education; *Adult Literacy; Educational Change; *Educational Practices; *Educational Research; *Literacy Education; *Outcomes of Education; Program Effectiveness; Student Motivation; *Teaching Methods IDENTIFIERS *General Educational Development Tests ABSTRACT This volume contains the four 1998 quarterly issues of this newsletter that present best practices, current research on adult learning and literacy, and information on how research is used by adult basic education teachers, counselors, program administrators, and policy makers. The following are among the major articles included: "Power, Literacy, and Motivation" (Greg Hart); "The First Three Weeks: A Critical Time for Motivation" (B. Allan Quigley); "Build Motivation by Building Learner Participation" (Barbara Garner); "Staying in a Literacy Program" (Archie Willard); "Stopping Out, Not Dropping Out" (Alisa Belzer); "Where Attendance Is Not a Problem" (Moira Lucey); "Getting into Groups" (Michael Pritza); "The GED [General Educational Development Test]: Whom Does It Help?" (John H. Tyler); "Project-Based Learning and the GED" (Anson M. Green); "Describing Program Practice: A Typology across Two Dimensions" (Barbara Garner); "Retention and the GED" (Jamie D. Barron Jones); "The Spanish GED" (Anastasia K.
    [Show full text]
  • Human Functional Brain Imaging 1990–2009
    Portfolio Review Human Functional Brain Imaging 1990–2009 September 2011 Acknowledgements The Wellcome Trust would like to thank the many people who generously gave up their time to participate in this review. The project was led by Claire Vaughan and Liz Allen. Key input and support was provided by Lynsey Bilsland, Richard Morris, John Williams, Shewly Choudhury, Kathryn Adcock, David Lynn, Kevin Dolby, Beth Thompson, Anna Wade, Suzi Morris, Annie Sanderson, and Jo Scott; and Lois Reynolds and Tilli Tansey (Wellcome Trust Expert Group). The views expressed in this report are those of the Wellcome Trust project team, drawing on the evidence compiled during the review. We are indebted to the independent Expert Group and our industry experts, who were pivotal in providing the assessments of the Trust’s role in supporting human functional brain imaging and have informed ‘our’ speculations for the future. Finally, we would like to thank Professor Randy Buckner, Professor Ray Dolan and Dr Anne-Marie Engel, who provided valuable input to the development of the timelines and report. The2 | Portfolio Wellcome Review: Trust Human is a Functional charity registeredBrain Imaging in England and Wales, no. 210183. Contents Acknowledgements 2 Key abbreviations used in the report 4 Overview and key findings 4 Landmarks in human functional brain imaging 10 1. Introduction and background 12 2 Human functional brain imaging today: the global research landscape 14 2.1 The global scene 14 2.2 The UK 15 2.3 Europe 17 2.4 Industry 17 2.5 Human brain imaging
    [Show full text]
  • Six Reasons Why Feral House Mouse Populations Might Have Low Recapture Rates
    Wdl. Res., 1994, 21, 559-67 Six Reasons Why Feral House Mouse Populations Might Have Low Recapture Rates Charles J. Krebs, Grant R. Singleton and Alice J. Kenney CSIRO Division of Wildlife and Ecology, PO Box 84, Lyneham, ACT 2602, Australia. Abstract Many feral house mouse populations have low recapture rates (0-20%) in live-trapping studies carried out at 2-4-week intervals. We consider six hypotheses to explain low recapture rates. We radio-collared 155 house mice between September 1992 and May 1993 in agricultural fields on the Darling Downs of south-eastern Queensland during a phase of population increase. Low recapture rates during the breeding season were due to low trappability and during the non-breeding period to nomadic movements. During the breeding season radio-collared mice of both sexes survived well and moved mostly small distances (<ll m). Low trappability has consequences for the precision of population indices that rely on catch per unit effort. Capture-recapture models robust to heterogeneity of trap responses should be used to census feral Mus populations. Introduction One of the most characteristic features of mark-recapture studies of feral house mice (Mus domesticus) is that there is a very low recapture rate of marked individuals. While similar studies of voles and mice have recapture rates of 60-100% when samples are taken every few weeks (Krebs and Boonstra 1984), for feral house mouse populations recapture rates of 0-20% are the norm. Singleton (1987) reported recapture rates of 11-20% for Longworth live-traps at monthly intervals. In southern Queensland Willliams and Wilson (1980) found only 19% of 2253 tagged mice were caught one month later.
    [Show full text]