A generic framework for task selection driven by synthetic emotions Claudius Gros Institute for theoretical physics Goethe University Frankfurt Frankfurt a.M., Germany Email: http://itp.uni-frankfurt.de/∼gros Abstract—Given a certain complexity level, humanized agents • A set of criteria characterizing tasks that have been may select from a wide range of possible tasks, with each executed, the experience of the agent. activity corresponding to a transient goal. In general there • A set of rules for task switching that are based fully or will be no overarching credit assignment scheme allowing to compare available options with respect to expected utilities. in part on experiences. For this situation we propose a task selection framework that Implementations may distinguish further between dedicated is based on time allocation via emotional stationarity (TAES). and humanized settings. For a specific dedicated application Emotions are argued to correspond to abstract criteria, such appropriate hand-picked sets of evaluation criteria and switch- as satisfaction, challenge and boredom, along which activities ing rules can be selected. Within this approach the resulting that have been carried out can be evaluated. The resulting timeline of experienced emotions is then compared with the overall behavior can be predicted and controlled to a fair ‘character’ of the agent, which is defined in terms of a preferred extend. Being hand-crafted, the disadvantage is that extensions distribution of emotional states. The long-term goal of the agent, and transfer to other domains demand in general extensive to align experience with character, is achieved by optimizing the reworks. frequency for selecting the individual tasks. Upon optimization, Consider an agent with two initial abilities, to play Go the statistics of emotion experience becomes stationary. and chess via internet servers. The respective winning rates could be taken in this case as appropriate evaluation criteria, I. INTRODUCTION other may be a challenge (close games) and boredom (games lasting forever). As an extension, the agent is provided with A wide range of specific tasks can be handled efficiently by a connection to a chat room, where the task is to answer present day robots and machine learning algorithm. Playing a questions. Humans would sent in chess board positions and game of Go is a typical example [1], with CT-scan classifica- the program provide in return the appropriate analysis, e.g. tion [2] and micro-helicopter piloting [3] being two other tasks in terms of possible moves and winning probabilities. The that are accomplished with comparative modest computational program has then three possibilities, as shown in Fig.1, to play resources. As a next step one may consider agents that have not Go or chess, and to connect to the chat room, with the third just a one, but a range of capabilities. like being able to play task differing qualitatively from the first two. Time allocation several distinct board games. Classically, as in most today’s frameworks designed specifically for the first two options, to applications, a human operator interacting with the program play Go or chess, would most probably cease to work when decides which of the agents functionality he or she wants the chat room is added, f.i. because winning ratios are not to access. Humanized agents may however be envisioned to suitable for characterizing a chat session. Here we argue that function without direct supervision, also with respect to task a characteristic trait of humanized computing is universality, arXiv:1909.11700v1 [cs.AI] 25 Sep 2019 selection [4]. Autonomous agents that are endorsed with the which translates in the context of time allocation frameworks capability to connect on their own to either a Go or a chess to the demand that extensions to new domains should be a server could be matched f.i. with either a human opponent, minor effort. upon entering the queuing system, or with another board- Of particular interest to humanized time allocation frame- playing program. The problem is then how to allocate time works is emotional control, which is known to guide human [5], namely how to decide which type of game to play. decision making. Starting with an overview, we will discuss Deciding what to do is a cornerstone of human activities, first the computational and neurobiological role of mammalian which implies that frameworks for multi-task situations de- emotions, stressing that algorithmic implementations need to serve attention. A possible route is to define and to maxi- reproduce functionalities and not qualia like fear and joy. A mize an overarching objective function, which could be, for concrete implementation based on the stationarity principle is example, to play alternatively Go or chess in order to improve then presented in a second step. Synthetic emotions correspond the respective levels of expertise, as measured, f.i., by the in this framework to a combination of abstract evaluation respective win rates. This example suggest that time-allocation criteria and motivational drives that are derived from the frameworks need two components, see Fig.1: objective to achieve a predefined time-averaged distribution [20]. Alternative choices are analyzed using logical reasoning, evaluate do with the outcome being encoded affectively [21]. Here we use ‘evaluation criteria’ as a generic term for the associated emo- tional values. Risk weighting has similarly both cognitive and emotional components [22], where the latter are of particular importance for long-term, viz strategic decision taking [23]. One feels reassured if a specific outlook is both positive and certain, and uncomfortable otherwise. Go chess chat The picture emerging from affective neuroscience studies is that the brain uses deductive reasoning for the analysis of behavioral options and emotional states for the respective Fig. 1. Illustration of a general time-allocation framework. The different weighting. A larger number of distinct types of emotional options, here to play Go, to play chess and to chat, are evaluated once selected, states [24], like anger, pride, fear, trust, etc, is consequently with the evaluation results feeding back into the decision what to do next. needed when the space of accessible behavioral options in- creases [25]. of emotional activities, the ‘character’ of the agent. III. COGNO-EMOTIONAL ARCHITECTURES An alternative to the here explored route to multi-task problems is multi-objective optimization [6], a setting in A minimal precondition for application scenaria incorporat- which distinct objectives dispose of individual utility function ing a basic cogno-emotional feedback loop is the option for that need to be optimized while respecting overall resource the program to switch between tasks [26]. An example is a limitation, like the availability of time. We focus here on multi-gaming environment for which the program decides on emotional control schemes, noting that emotional control and its own, as detailed out further below, which game to play multi-objective optimization are not mutually exclusive. next. II. COMPUTATIONAL ROLE OF EMOTIONS A. Multi-gaming environments Emotions have emerged in the last decades as indispensable We consider an architecture able to play several games, preconditions for higher cognition [7], [8], with the reason such as Go, chess, Starcraft or console games like Atari. The being that the core task of emotional response is not direct opponents may be either human players that are drawn from causation of the type “fleeing when afraid”, akin to a reflex, a standard internet-based matchmaking systems, standalone but the induction of anticipation, reflection and cognitive competing algorithms or agents participating in a multi-agent feedback [9]. In general, being afraid will not result in a challenge setup [27]. Of minor relevance is the expertise direct behavioral response, but in the allocation of cognitive level of the architecture and whether game-specific algorithms resources to the danger at hand. are used. A single generic algorithm [1], such as standard The interrelation between emotion and cognition is two- Monte Carlo tree search supplemented by a value and policy faced. Emotions prime cognitive processes [10], being con- generating deep network [28], would do the job. For our trolled in return by cognition [11]. The latter capability, to purpose, a key issue is the question whether the process regulate emotions [12], f.i. when restraining one’s desire for determining which game to play is universal, in the sense that unhealthy food, is so pronounced that it can be regarded to it can be easily adapted when the palette of tasks is enlarged, be a defining characteristics our species [13]. With regard to f.i. when the option to connect to a chat room is added. synthetic emotions, it is important to note that the cogno- For a complete cogno-emotional feedback loop an agent emotional feedback loop present in our brain implies that able to reason logically on an at least rudimentary level would emotional imprints are induced whenever cognitive capabilities be needed. This does not hold for the application scenario are used to pursue a given goal, such as playing and winning considered here. As a consequence, one may incorporate the a game of Go [14]. feedback of the actions of the agent onto its emotional states On a neuronal level one may argue [15], that the classical and the emotional priming of the decision process, but not a distinction between affective and cognitive brain regions is full-fledged cognitive control of emotions. misleading [16]. Behavior should be viewed instead as a B. Emotional evaluation criteria complex cogno-emotional process that is based on dynamic coalitions of brain areas [17], and not on the activation of a In a first step one has to define the qualia of the emotional specific structure, such as the amygdala [18]. This statement states and how they are evaluated, viz the relation of distinct holds for the neural representations of the cognitive activity emotions to experiences.