Hedonic Quality Or Reward? a Study of Basic Pleasure in Homeostasis And

Special issue on Grounding Emotions in Robots Adaptive Behavior 1–25 Ó The Author(s) 2016 Hedonic quality or reward? A study of Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1059712316666331 basic pleasure in homeostasis and adb.sagepub.com decision making of a motivated autonomous robot Matthew Lewis and Lola Canãmero Abstract We present a robot architecture and experiments to investigate some of the roles that pleasure plays in the decision making (action selection) process of an autonomous robot that must survive in its environment. We have conducted three sets of experiments to assess the effect of different types of pleasure—related versus unrelated to the satisfaction of physiological needs—under different environmental circumstances. Our results indicate that pleasure, including pleasure unrelated to need satisfaction, has value for homeostatic management in terms of improved viability and increased flexibility in adaptive behavior. Keywords Action selection, embodied autonomous robots, homeostasis, hormonal modulation, motivation and emotion, pleasure 1 Introduction ‘liking’ can take place: as linked to the satisfaction of survival-related physiological needs, and as a purely We present a biologically-inspired robot architecture hedonic quality not directly linked to need satisfaction. and experiments to investigate some of the roles that In both cases, we have focused our study on the influ- pleasure plays in the decision making or action selection ence that pleasure has on the perception of external sti- (AS) process of a motivationally autonomous robot muli, and the underlying mechanism we have adopted that must survive or remain ‘viable’ (Ashby, 1960) in its to model pleasure is a simulation of hormonal modula- environment. This study is framed by our long-term tion of perception. Unlike related work in robotics interest in the interactions between emotion, motivation (Krichmar, 2008, 2012, 2013; Sporns & Alexander, and cognition from an embodied perspective, and is a 2002), our simulated hormone constitutes an abstract first step towards a more systematic study of the roles model aimed to capture the (gross) dynamics of modu- of pleasure in such interactions. lation rather than modeling the behavior of specific Many definitions of pleasure have been provided in chemicals underlying pleasure and more generally the literature, reflecting the fact of its multi-faceted affective phenomena. Hormonal modulation has been nature, its multiple meanings, and its multiple underly- used in robots for other purposes, such as behavior ing biological mechanisms, to the extent that some control (Moioli, Vargas, & Husbands, 2009), learning authors talk about ‘pleasures’ rather than ‘pleasure’ as based on value and reward systems (e.g. the 2013 spe- a single notion (Frijda, 2010; Kringelbach & Berridge, cial issue of Frontiers in Neurorobotics on this topic; see 2010), e.g. sensory pleasure, non-sensory linkings, pleasures of achievement, pleasures of gain and relief, social pleasure, activity pleasures, esthetic pleasures. Embodied Emotion, Cognition and (Inter-)Action Lab, School of Underlying the different views, however, we can iden- Computer Science, University of Hertfordshire, UK tify a common element of ‘positive affect’, of enjoy- Corresponding authors: ment, which is also found in lay, non-technical Matthew Lewis, School of Computer Science, University of definitions (e.g. in dictionaries such as the Merriam– Hertfordshire, College Lane, Hatfield, Hertfordshire, AL10 9AB, UK. Webster or the Oxford). In this paper, we take such a Email: [email protected] Lola Canãmero, School of Computer Science, University of broad, common-sense definition of pleasure as ‘liking’, Hertfordshire, College Lane, Hatfield, Hertfordshire, AL10 9AB, UK. focusing on two different contexts in which such Email: [email protected] Downloaded from adb.sagepub.com at CORNELL UNIV on October 23, 2016 2 Adaptive Behavior (Krichmar and Ro¨hrbein, 2013) for a review of the ‘liking’ something to ‘wanting’ it? This link is, once papers), to model intrinsic motivation (Kaplan & more, often conceptualized in terms of ‘usefulness’ and Oudeyer, 2007; Luciw, Kompella, Kazerounian, & ‘reward’ and, in the latter case, in the context of learn- Schmidhuber, 2013), energetic autonomy (Lowe et al., ing. Again, in this paper we depart from this view, in 2010; Sauze´& Neal, 2010), and evolutionary and mod- two respects. First, we think that hedonic quality (just ular robotics (Hamann, Stradner, Schmickl, & ‘liking’, ‘pure pleasure’ unrelated to need satisfaction) Crailsheim, 2010). might also have an important role in preference beha- Although pleasure is intimately related to affect, and vior and motivation (Frijda, 1986, 2010; Young, 1961). more specifically to emotion, the nature of this relation- Second, given the involvement of pleasure in basic life ship remains elusive. Some consider pleasure as an emo- regulation in line with its early evolutionary origins tion—e.g. one of the earliest forms of emotion that (Damasio, 1999; Panksepp, 1998), we think that, to evolved (Panksepp, 1998)—others think that pleasure, better understand pleasure, we need to consider its even though at the origin of emotions, is not an emo- roles in the context of ‘simpler’ cognitive functions, and tion itself but ‘a constituent quality of certain emotions notably in the context of perception, since pleasure as well as a trigger for certain emotions’ (Damasio, involves closely related hedonic and sensory aspects. 1999, p. 76). Firmly grounded in the body’s biological We have thus investigated the effect of pleasure on the ‘machinery’ at different levels, the term ‘pleasure’ is nor- perception of external stimuli and, more specifically, of mallyusedtodenoteasubjective quality: a positive the stimuli relevant to the satisfaction of homeostati- hedonic quality, an affective sensation or feeling of plea- cally-controlled internal needs. As we will see, in our santness. To some, pleasure has both sensory and affec- model, the modulation of motivation-related percep- tive elements (Nafe, 1924), whereas for most, pleasure is tion through pleasure provides a link between ‘liking’ the affective component (Cabanac, 2010; Kringelbach, and ‘wanting’, as it changes the ‘attentional effort’ 2010), a ‘gloss’ (Frijda, 2010) of sensory processing. (Sarter, Gehring, & Kozak, 2006) or ‘incentive salience’ Pleasure is often associated with positive valence. (Berridge & Robinson, 1998) of the stimuli. As Pessoa However, the interactions between hedonic feelings, (2013, p. 190) puts it: ‘At the perceptual level, items positive affect, and approach behavior can be very with affective/motivational content act as if they had complex (Leknes & Tracy, 2010), and their underlying increased salience, which improves performance if they mechanisms are not clear. This link is often explained are task relevant but impairs it if they are task in terms of the biological ‘utility’ (e.g. adaptive value, irrelevant.’ evolutionary usefulness) of pleasure, which, from this In previous work (Canãmero & Avila-Garcıá, 2007), perspective, would signal that stimuli are beneficial we had used hormonal modulation of the perceptual (Cabanac, 1971, 1979; Panksepp, 1998) and foster the element of motivation as a function of increased inter- acceptance of such stimuli (Frijda, 1986). This view is nal deficits and the presence of threats in the environ- particularly compatible with models of the body in ment, i.e. hormone was released signaling that things terms of homeostasis (Leknes & Tracy, 2010), as well were not functioning well. In this paper, a simulated as with models of learning inspired by classic and oper- modulatory pleasure hormone is released as a function ant conditioning (Rolls, 2014), and is reflected by their of need satisfaction, signaling an improvement in the focus on the ‘reward’ aspect of pleasure to the neglect interaction with the environment and fosters ‘open- of other aspects. This is also the predominant interest ness’—increasing the interaction with the environment of robotic models that take into account pleasure, or continuing an interaction that is going well. We thus mostly in the context of reinforcement learning (e.g. follow the principle that pleasure signals well function- Cos, Canãmero, Hayes, & Gillies, 2013; Gadanho & ing (Panksepp, 1998). Hallam, 2001; Hiolle & Canãmero, 2009; Kitano, To facilitate systematic analysis of experimental 1995). In previous work, we have investigated the role results, we have tested our model of pleasure using what of pleasure as reward in the maintenance of homeosta- in the AS literature is known as a ‘two-resource prob- sis in a reinforcement learning model (Cos et al., 2013). lem’ (2RP) (Spier & McFarland, 1997), implementing In this paper, we depart from the idea that pleasure is the simplest decision-making scenario. As its name sug- necessarily linked with reward—in the same way as gests, in this scenario, an agent (animal or robot) must value is not necessarily linked with reward (Krichmar autonomously decide which of the two resources avail- &Ro¨hrbein, 2013)—or with signaling biological useful- able in the environment it should consume in a timely ness, opening the door to the investigation of the role fashion in order to satisfy its two survival-related needs of other types of pleasure not directly related with the successfully. The experiments reported here aim to com- satisfaction of needs (Frijda, 2010),

Load more