AISBQUARTERLY the newsletter of the society for the study of and simulation of behaviour

No. 141 July, 2015 Do you feel artistic? Exhibit your artwork on our front covers! Email us at [email protected]!

Artwork by Alwyn Husselmann, PhD (Massey Univ., New Zealand)

Visualisation is an important tool for gaining insight into how algorithms behave. There have been many techniques developed, including some to visualise 3D voxel sets [1], program space in genetic programming [2] and vector fields [3], amongst a large number of methods in other domains. A qualitative understanding of an algorithm is useful not only in diagnosing implementations, but also in improving performance. In parametric optimisation, algorithms such as the Firefly Algorithm [4] are quite sim- ple to visualise provided they are being used on a problem with less than four dimensions. Search algorithms in this class are known as metaheuristics, as they have an ability to optimise unknown functions of an arbitrary number of variables without gradient infor- mation. Observations of particle movement are particularly useful for calibrating the internal parameters of the algorithm. Pictured on the cover is a visualisation of a Firefly Algorithm optimising the three- dimensional Rosenbrock Function [5]. Each coloured sphere represents a potential min- imum candidate. The optimum is near the centre of the cube, at coordinate (1, 1, 1). Colour is used here to indicate the output of the Rosenbrock function, whereas the 3D coordinate of each particle is representative of the actual values used as input to the function. The clustering seen amongst the particles in the image is due to the local neighbourhood searches occurring. Particles tend towards the optimal solution in a small group of a certain radius, as well as moving randomly to a certain degree. The visualisa- tion assisted in improving convergence and verifying the implementation of the optimiser.

[1] KA Hawick, ’3d visualisation of simulation model voxel hyperbricks and the cubes pro- gram’, Technical Report CSTN-082 102-904, Massey University, Albany, North Shore, Com- puter Science, Massey University, Albany, North Shore 102-904, (October 2010). [2] AV Husselmann and KA Hawick, ’3d vector-field data processing and visualisation on graphical processing units’, in Proc. Int. Conf. Signal and Image Processing (SIP 2012), pp. 92Ð98, Honolulu, USA, (August 2012). [3] AV Husselmann and KA Hawick, ’Parallel parametric optimisation with firefly algorithms on graphical processing units’, in Int. Conf. on Genetic and Evolutionary Methods (GEM’12), pp. 77–83, (2012). [4] AV Husselmann and KA Hawick, ’Visualisation of combinatorial program space and re- lated metrics’, in 2013 International Conference on Information and Knowledge Engineering (IKE’13), (2013). [5] XS Yang, ’Firefly algorithms for multimodal optimization. in: Stochastic algorithms: Foun- dations and applications’, in SAGA, (2009). Editorial

To quote an anonymous Committee truth is, it has always been difficult to member, as spotted on one of the so- recruit people to submit material to be cial websites that dominate our lives: pushed to our members, but these days I’m back! Did you miss me? it’s becoming increasingly difficult. We What do you mean ’you are competing for time, and it’s a battle didn’t notice I was gone’? we will not win. AISB has about 400 members, Sure you did. Sure you did. mostly in the UK. In times like The truth is, the world is changing these, the Quarterly should thus at a pace faster than we are able to be seen as an outreach tool, de- cope. Gigabit ethernet is now (almost) signed to bring you followers and the internet of the past. It will soon "Likes"; something tangible, pal- be the solution that my three year old pable that we are naturally in- son will have to fall back on, when his clined to fall back on. I can only head-mounted, miniaturised computer invite you to get in touch with us, is not able to download the world news, and to make your voice heard. on his daily commute between and New York1. ∴ At the time I am writing these lines, Universities are trying to adapt to this Echoing my editorial, Joel Lehman et evolving market, as it is now called. De- al. review the debates that happened partments are closing faster than new on the occasion of a recent AAAI work- ones are being open. National evalua- shop, which gathered leading figures. tion exercises and league tables dom- Their conclusion is without appeal, the inate the academic life and quite lit- anarchy of methods that makes AI is erally redefine what it means to pur- to be embraced. Significant and im- sue academic fulfilment, which is now a portant challenges remain, however, in- privilege of a few. cluding safety, ethical and societal con- On the plus side, AI is booming siderations. again, thanks to both progress made by Andy Thomason reviews the recent industry, and great outreach threads, book by van Benthem on logic in including amazing PR from companies games, and JeeHang Lee and Jeka- and a series of big bucks movies. I guess terina Novikova report on conferences this is a good thing for our field more they attended to, thanks to funding broadly, if we are in a position to lever- from AISB. age this hype. Hopefully, you have noticed some de- Etienne B. Roesch lays in receiving your Quarterly. The Editor-in-Chief

1I wonder if Fireman Sam will be viewable with 3D glasses then... p. 1 AISB Quarterly An Anarchy of Methods: Current Trends in How Intelligence is Abstracted in AI by John Lehman (U. Texas, USA), Jeff Clune (U. Wyoming, USA), &

Sebastian Risi (U. Copenhagen, Denmark)

Artificial intelligence (AI) is a sprawl- processes or view intelligence as symbol ing field encompassing a diversity of manipulation. Similarly, researchers fo- approaches to machine intelligence and cus on different processes for generating disparate perspectives on how intelli- intelligence, such as learning through gence should be viewed. Because re- reinforcement, natural evolution, logi- searchers often engage only within their cal inference, and statistics. The re- own specialized area of AI, there are sult is a panoply of approaches and sub- many interesting broad questions about fields. AI as a whole that often go unan- Because of independent vocabularies, swered. How should intelligence be ab- internalized assumptions, and separate stracted in AI research? Which sub- meetings, AI sub-communities can be- fields, techniques, and abstractions are come increasingly insulated from one most promising? Why do researchers another even as they pursue the same bet their careers on the particular ab- ultimate goal. Further deepening the stractions and techniques of their cho- separation, researchers may view other sen subfield of AI? Should AI research approaches only in caricature, unin- be "bio-inspired" and remain faithful to tentionally simplifying the motivations the process that produced intelligence and research of other researchers. Such (evolution) or the biological substrate isolation can frustrate timely dissem- that enables it (networks of neurons)? ination of useful insights, leading to Discussing these big-picture questions wasted effort and unnecessary rediscov- motivated us to organize an AAAI Fall ery. Symposium, which gathered partici- To address such dangers, we orga- pants across AI subfields to present and nized an AAAI Fall symposium that debate their views. This article distills gathered experts with diverse perspec- the resulting insights. tives on biological and synthetic in- telligence. The hope was that such Introduction a meeting might lead to a productive While researchers in AI all strive to examination of the value and promise create intelligent machines, separate AI of different approaches, and perhaps communities view intelligence in strik- even inspire syntheses that cross tra- ingly different ways. Some abstract in- ditional boundaries. However, organiz- telligence through the lens of connec- ing a cross-disciplinary symposium has tionist neural networks, while others risks as well. Discussion could have fo- use mathematical models of decision cused narrowly on intractable disagree-

No 141, July 2015 p. 2 ments, or on which singular abstraction of images) to learn a hierarchy of in- is "the best." An unhelpful slugfest of creasingly abstract features (Figure 2; ideas could have emerged instead of col- [4]) Overall, participants agreed that laborative cross-pollination, leading to recent progress in deep networks was a veritable AI Tower of Babel. a significant step forward for process- In the end, there were world-class ing streams of high-dimensional raw keynote speakers spanning AI and bi- data into meaningful abstract represen- ology (see below), and participants tations, e.g. recognizing faces from un- were indeed collaborative. Some trav- processed pixel data. But there was elled to the United States from as far also agreement that much work yet re- as Brazil, Australia, and Singapore; mains to create algorithms that lever- but beyond geographic diversity, there age such representations to produce in- were representatives from many disci- telligent behavior and learn in real-time plines and approaches to AI (Figure 1). from feedback; in other words, scaling Drawing from the symposium’s talks deep learning to more cognitive behav- and events, we now summarize recent ior may prove problematic. progress across AI fields, as well as the Andrew Ng, affiliated with Stanford key ideas, debates, and challenges iden- University and Baidu Research, gave tified by the attendees. a keynote on deep learning that out- lined its motivation, implementation, • Andrew Ng (Stanford U.): and recent successes. Other keynote Deep Learning speakers reported that they also effec- • Risto Miikkulainen (U. Texas): tively use deep learning, in that their Evolving Neural Networks research similarly involves learning in • Pierre-Yves Oudeyer (Inria, many-layered neural networks. In this France): Developmental Robotics sense, deep learning has gone by many • Gary Marcus (NYU): names over time, and is currently being Cognitive Science reinvigorated by increased computing • Georg Striedter (UC Irvine): power, big data, greater biological un- Neuroscience derstanding, and algorithmic advances. • Randall O’Reilly (U. Colorado): For example, in his keynote, Randall Computational Neuroscience O’Reilly of the University of Colorado at Boulder summarized his work in Key ideas discussed the field of computational neuroscience, One controversial topic was deep learn- where researchers often develop cogni- ing, which has recently shattered many tive architectures, which are computa- performance records over an impres- tional processes designed to model hu- sive spectrum of machine learning tasks man or animal intelligence. His Leabra [2, 7]. The central idea behind deep cognitive architecture is a many-layered learning is that large hierarchical arti- neural network modeled on the human ficial neural networks (ANNs), inspired brain, which includes collections of neu- by those found in the neocortex, can rons analogous to the major known be trained on big data (e.g. millions functional areas of the brain [10]. In p. 3 AISB Quarterly Figure 1: The backgrounds of attendees. this way, two separate areas of AI ap- cause evolution is highly adept at creat- ply similar technologies inspired by dif- ing variations on an underlying theme ferent motivations: one coarsely ab- [14]). The idea is that evolutionary stracts brains to solve practical prob- methods could perhaps provide this im- lems, and the other applies more bio- portant capability to other AI tech- logically plausible abstractions to bet- niques, such as deep learning. Support- ter understand animal brains. ing this idea, Jeff Clune, from the Uni- A related camp (to which the au- versity of Wyoming, described how evo- thors belong) that is inspired by nature lutionary algorithms that incorporate and applies evolutionary algorithms to realistic constraints on natural evolu- design neural networks, is called neu- tion can produce ANNs that have im- roevolution. In his keynote, Risto Mi- portant properties of complex biologi- ikkulainen of the University of Texas cal brains, like regularity, modularity, at Austin described how neuroevolu- and hierarchy [3]. tion can design cognitive architectures Pierre-Yves Oudeyer of Inria de- via a bottom-up design process guided tailed in his keynote the field of devel- by evolutionary algorithms instead of opmental robotics, which investigates through top-down human engineering. how robots can develop their behav- Kenneth Stanley, from the University iors over time through interacting with of Central Florida, argued that evolu- the world, just as animals and hu- tionary approaches may be important mans do [8]. Representative approaches tools for producing human-level AI be- in developmental robotics implement

No 141, July 2015 p. 4 Figure 2: An illustration of deep learning. As a deep network of neurons is trained to recognize different faces, the neurons on the lowest level learn to detect low- level features such as edges, and higher-level neurons combine these lower-level features to recognize eyes, noses, and mouths. Neurons on the top of the hierarchy can then combine such features together to recognize different faces. mechanisms to enable lifelong, active, ingly complex tasks. and incremental acquisition of both Among the traditional biologists that skills and models of the environment, attended was Georg Striedter of the through self-exploration or social guid- University of California at Irvine, au- ance. Oudeyer’s research shows that thor of the influential book "Principles motivating robots to be curious re- of Brain Evolution". His keynote fo- sults in continual experimentation: A cused on the history of how brain func- robot equipped with intrinsic motiva- tionality has been viewed over time. tion will search for information gain He noted an interesting parallel be- for its own sake; at any given point tween the history of AI and of neuro- in the robot’s development, it actively science: In both, a simple serial view performs experiments to learn how its of intelligence led to exploring more actions affect the environment [11]. Be- parallel, distributed notions of pro- cause such curiosity leads to an ever- cessing. He mentioned that Rodney improving model of the consequences of Brooks’ subsumption architecture in a robot’s actions, over time it can result particular had resonated with him, be- in learning how to accomplish increas- cause it offered a picture of higher-order p. 5 AISB Quarterly thought beyond simplistic linear path- their view that the power of human ways (Brooks 1986); while computer intelligence is largely captured in the scientists often debate the promise of idea of symbol manipulation. Such re- various approaches to computational searchers also illuminated where non- intelligence amongst themselves, it is symbolic approaches still fall short. In informative also to consider the opin- particular, John Laird of the University ions of those who study how it arose in of Michigan posed an interesting chal- humans. lenge problem called embodied taska- Aside from models with concrete bility: Similar to learning from demon- biological inspiration, other attendees stration [1], a robot must learn to per- focused on abstractions of intelli- form novel tasks by interacting with gence based on Markov decision pro- humans. The task is intriguing be- cesses (MDPs) and less-restrictive gen- cause it is an ambitious problem not eralizations called partially observable often tackled by other fields of AI, Markov decision processes (POMDPs). yet is characteristic of human intelli- Such MDPs and POMDPs represent gence. Complementarily, Gary Marcus decision making in a mathematical of New York University gave a provoca- framework composed of mappings be- tive keynote highlighting several capa- tween states, actions, and rewards. bilities necessary for strong AI that cur- This framework provides the basis for rent high-performing connectionist ap- AI techniques like reinforcement learn- proaches do not yet implement, such ing and probabilistic graphical mod- as representing causal relationships and els. Devin Grady of Rice University abstract ideas, and making logical in- and Shiqi Zhang of Texas Tech Uni- ferences. He also mentioned challenges versity each described mechanisms to in natural language understanding. augment such techniques to allow them During one of the panel sessions, an to better scale to more complex prob- idea was proposed in an attempt to lems. A similar need for tractable mod- tie all of these fields and levels of ab- els motivated Andrew Ng’s change in straction together: a stack of models, focus from MDP-based reinforcement where each individual level of the stack learning to deep learning. He men- is guided by a different level of abstrac- tioned in response to a question that tion. The idea is that with such a stack he felt the bottleneck was no longer re- the various levels of abstraction could inforcement learning algorithms them- be linked together, guided by a reduc- selves, but in generating strong relevant tionist goal of connecting understand- features from raw input for such algo- ing of high-level, abstract, rational rithms to learn from, which otherwise components of intelligence to "lower- must be manually generated by humans level" ones that are closer to perceiving through domain-relevant knowledge. raw data and controlling muscles. For Proponents of symbolic AI (also example, high-level GOFAI algorithms known as GOFAI, or "good old- could possibly be connected to deep fashioned artificial intelligence," due to learning models, which could be con- its early research dominance) defended nected to more biologically-plausible

No 141, July 2015 p. 6 computational models of brains. In this his keynote, Gary Marcus argued that way it might be possible to unite dis- it would be a mistake to conflate the parate views and approaches to gain time of an approach’s first prominence greater overall understanding. with its potential; he noted that sym- bolic AI techniques might also (like Debates statistical techniques) benefit from ad- As mentioned previously, deep learning vances in computing power and avail- proved to be a lightning rod for discus- able data, and that such symbolic tech- sion and many researchers were quick to niques were developed mainly in the ab- point out perceived difficulties in scal- sence of the broad computational re- ing deep learning to human-level AI. sources that are now used in statistical Open research questions include how to approaches. create deep networks that implement A point of agreement was that sym- reinforcement learning, develop higher bolic AI is not better or worse that al- cognitive abilities over time, or manip- ternate approaches, but is instead dif- ulate symbols. Andrew Ng, when asked ferent in its aims and objectives. Sym- about merging deep learning with re- bolic AI continues to aim at the am- inforcement learning, responded that bitious goal of general artificial intel- it is an unsolved problem and that "a ligence (i.e. human-level intelligence) seminal paper on that subject is wait- while other approaches often focus on ing to be written1." Ng was hopeful narrower domains or simpler forms of that it should be possible to extend intelligence. A contribution of Gary deep learning algorithms to perform re- Marcus was to highlight that GOFAI inforcement learning without merging is not an inferior way of reproducing in other AI paradigms. In contrast, these narrower or simpler intelligences, and perhaps unsurprisingly, researchers but is instead aimed at a different goal: outside of deep learning were generally the cognitive intelligence that sets hu- more skeptical. mans apart from other animals, which While the current winds of AI seem is where statistical machine learning generally to favor statistical machine methods are arguably weakest. learning methods like deep learning or A contentious issue for researchers reinforcement learning over purely sym- in biologically-inspired AI concerned bolic GOFAI approaches, proponents which biological details are extraneous of symbolic AI made convincing argu- and therefore unnecessary to include ments for its continued relevance. John in AI models. For example, brains Laird expressed that although symbolic vary over a multitude of dimensions AI might not be as dominant as it once including neuron size, density, type, was, research progresses onward irre- connectivity, and structure; intuitively, spective of current fashion. In partic- it seems unlikely that all such dimen- ular, symbolic AI research is currently sions are equally important to a model’s producing promising symbolic cognitive functionality. Randall O’Reilly men- architectures that can empower agents tioned that in his models, the addi- to learn new human-taught tasks. In tional complexity of simulating neurons p. 7 AISB Quarterly with binary spikes in time (like bio- Pierre-Yves Oudeyer interjected that, logical neurons) provided little bene- of course, which biological details are fit over simpler neuron models. Yet in important depends upon the scientific contrast, Oliver Coleman (University of question being investigated. Or, as New South Wales) highlighted past re- John Laird said in response to the name search showing that the timing of spikes of the symposium ("How Should Intelli- may be an important facet of learning gence be Abstracted in AI Research?), processes in the brain. Taking the at- "It depends!" Oudeyer then said some- tendees as a whole, there were more thing that resonated strongly: Because skeptics of complex neuron models than we do not deeply understand intelli- proponents, likely reflecting a cautious, gence or know how to produce general pragmatic preference for simplicity over AI, rather than cutting off any avenues biological realism for its own sake. of exploration, to truly make progress The opposite question was also de- we should embrace AI’s "anarchy of bated: Are there salient features of methods." brains and intelligence that are unfairly ignored? For example, O’Reilly be- Major Challenges lieves that glial cells, which are non- Through the course of the discussion, neural cells that provide support and many remaining challenges for AI be- protection for neurons, may be more came evident that cut across traditional important computationally than their boundaries. Overall, AI approaches absence in most models would suggest. tend to have four distinct focuses: Real- For Risto Miikkulainen and Pierre-Yves world embodiment, building features Oudeyer, how brains physically develop from raw perception, making decisions over time was a topic deserving greater based on features, and high-level cog- attention; most models ignore the fact nitive reasoning that is unique to hu- that biological brains learn while they mans. Approaches generally specialize grow and develop into their full ma- on one such area, and often perform ture size. In contrast, Gary Marcus poorly when stretched beyond that fo- argued that it may be possible to ab- cus. However, general AI requires span- stract nearly all biological detail away ning such divides. To do so may require if all we care about is engineering AI, integrating existing disparate technolo- and not understanding biology. The gies together; for example, hybrid neu- resulting discussion questioned whether ral systems [15] often combine neural the brain is a well-engineered machine network and symbolic models together, with much to teach us, or whether it is like the SAL architecture that connects merely a hacked-together "kluge" [9]. In the symbolic ACT-R model to bottom- other words, do researchers mistakenly up perception from the Leabra neural idealize the human brain, searching for model [5]. A more conventional ap- elegant insights in a messily-designed proach is to attempt to scale up an artifact, one that is functional but ul- existing technology beyond its current timately unintelligible? borders. For example, Risto Miikku- As the debate became more intense, lainen’s keynote highlighted that neu-

No 141, July 2015 p. 8 roevolution techniques are beginning to neuroevolution, and neural-based cog- evolve instances of simple cognitive ar- nitive architectures difficult. A consen- chitectures. Additionally, cognitive ar- sus among attendees was that this was chitectures like Leabra and Spaun are an important and underfunded consid- beginning to tackle symbolic manip- eration. ulation of variables through human- Another central problem that engineered neural mechanisms [13, 6]. emerged through discussions is the dif- Extensions to deep learning might sim- ficulty (or impossibility) of definitively ilarly incorporate decision making and knowing what ways of abstracting in- cognition. However, if integrating or telligence are truly "better" or more extending existing technologies proves productive than others. In general, at- unproductive, there might yet be a tempting to predict the future promise need for new approaches better able to of any particular technology or research bridge aspects of AI ranging from low- direction is often misleading. But a level perception to human-level cogni- particular challenge in AI stems from tion. the existence of only one example of An interesting challenge in AI that high-level intelligence from which to in- often goes unconsidered is safety. The fer generalities. As a result of nature’s most interesting intellectual challenge singular anecdote on intelligence, sepa- drawing researchers to AI is under- rating what is essential for intelligence standing and engineering intelligent from what is merely coincidental re- systems. However, it may be dangerous mains difficult. to single-mindedly pursue such a goal without considering the transformative Conclusion consequences that may result if we cre- At the symposium’s end, researchers ate AI that rivals or even surpasses mentioned that they better understood human intelligence. Problematically, the philosophical and theoretical moti- academic and industrial incentives are vations for areas of AI they had unin- nearly unilaterally aligned towards cre- tentionally only seen previously in car- ating increasingly sophisticated AI and icature. One participant said that he do not emphasize critical reflection re- learned that even when viewing intelli- garding its potential downsides or side- gence abstractly from a high level, there effects. Only a single talk, by Armando is a benefit to following key develop- Tacchella of the University of Genova, ments at lower levels. Another offered focused on creating safe abstractions that he "learned how limited our knowl- of AI [12]. That work raised difficult edge is," and that it was interesting how questions for the many AI approaches often "key leaders in a field might not where verification or automatic char- have a grand, deep plan [...] but that acterization of the behaviors produced instead, behind the curtain, are scien- is difficult. For example, neural net- tists doing the best they can, fumbling works are notorious for being black box in the dark." Another made reference to models, making interpreting the safety the parable of three blind men describ- of agents resulting from deep learning, ing an elephant, where each blind man p. 9 AISB Quarterly describes the whole elephant in terms of with deep convolutional neural net- features specific to the individual parts works’, in Advances in neural infor- they are examining (a tail, a tusk, or mation processing systems, pp. 1097– 1105, (2012). a leg, respectively), which leads to very different interpretations of what an ele- [8] M Lungarella, G Metta, R Pfeifer, and phant is. Similarly, through sharing lo- G Sandini, ‘Developmental robotics: cal perspectives on AI and what they a survey’, Connection Science, 15(4), imply about the overall field, the re- 151–190, (2003). sulting traces of intelligence’s outline – [9] Gary Marcus, Kluge: The Haphaz- made from all angles and levels of ab- ard Evolution of the Human Mind, straction of AI’s anarchy of methods – Houghton Mifflin Harcourt, 2008. might potentially be combined to accel- erate our understanding of the general [10] RC O’Reilly and Y Munakata, Com- principles underlying intelligence and putational explorations in cognitive neuroscience: Understanding the how to recreate it computationally. mind by simulating the brain, MIT press, 2000. Reference [1] CG Atkeson and S Schaal, ‘Robot [11] PY Oudeyer, F Kaplan, and learning from demonstration’, in VV Hafner, ‘Intrinsic motivation ICML, volume 97, pp. 12–20, (July systems for autonomous mental 1997). development’, IEEE Transactions on Evolutionary Computation, 11(2), [2] Y Bengio, ‘Learning deep architec- 265–286, (2007). tures for ai’, Foundations and trends in Machine Learning, 2(1), 1–127, [12] S Pathak, L Pulina, G Metta, and (2009). A Tacchella, ‘How to abstract intel- [3] J Clune, J-B Mouret, and H Lipson, ligence? (if verification is in order)’, ‘The evolutionary origins of modular- in Proceedings of the 2013 AAAI Fall ity’, Proceedings of the Royal Society Symposium on How Should Intelli- B, 280(20122863), (2013). gence be Abstracted in AI Research, [4] GE Hinton, ‘Learning multiple layers (2013). of representation’, Trends in cognitive sciences, 11(10), 428–434, (2007). [13] NP Rougier, D Noelle, TS Braver, JD Cohen, and RC O’Reilly, ‘Pre- [5] DJ Jilk, C Lebiere, RC O’Reilly, frontal cortex and the flexibility of and JR Anderson, ‘SAL: An explic- cognitive control: Rules without sym- itly pluralistic cognitive architecture’, bols’, Proceedings of the National Journal of Experimental and Theo- Academy of Sciences, 102, 7338– retical Artificial Intelligence, 20(197- 7343, (2005). 218), (2008). [6] T Kriete, DC Noelle, JD Cohen, and [14] KO Stanley, ‘Compositional pattern RC O’Reilly, ‘Indirection and symbol- producing networks: A novel abstrac- like processing in the prefrontal cor- tion of development’, Genetic Pro- tex and basal ganglia’, Proceedings gramming and Evolvable Machines, of the National Academy of Sciences, 8(2), 131–162, (2007). 41(16390-16395), (110). [7] A Krizhevsky, I Sutskever, and [15] S Wermter and R Sun, Hybrid Neural GE Hinton, ‘Imagenet classification Computation, Springer, 2008.

No 141, July 2015 p. 10 Joel Lehman, PhD Post-doctoral Fellow University of Texas, Austin, USA

Jeff Clune, PhD Assistant Professor University of Wyoming, USA

Sebastian Risi, PhD Assistant Professor IT University of Copenhagen, Denmark

p. 11 AISB Quarterly Book review: Logic In Games (van Benthem, 2014) by Andy Thomason (Goldsmiths Univ. London)

Van Benthem’s Logic In Games is a Concepts such as belief and rational- well-researched text book describing ity play a part in the theory of games systems of logic used in reasoning about and are modelled through operators of the processes of turn-based games. The modal logic. These concepts are key to author presents the idea that logic the understanding of the social context and games are interchangeable–after of gaming and help the construction of all, like computing technology, the pri- social games, allowing the build up of mary motive for the development of concepts like "we ought not to kill civil- logic in the classical world was to play ians with robots". games. Graphs throughout the book help the To a game developer, this book of- reader to gain a grasp of the notation. fers alternative insights to the worlds of The combination of the expressive lan- epistemic logic (reasoning about knowl- guage of symbolic logic and the state edge) propositional dynamic logic (de- diagrams help to form a common lan- scribing processes) and concepts such guage for game designers to discuss the as the excluded middle. Although the intricacies of puzzle construction and notation may be difficult to understand competitive game design. at first, it is worth persevering as the The reviewer particularly liked the reader will be rewarded with a deeper section on "Sabotage Games" which understanding of game processes. also covered aspects of gamification. Ideally, the reader should have some The author illustrates this with his ex- background in formal logic, but van periences with the Dutch rail network Benthem does an excellent job of intro- where a demon was obviously at work ducing the reader gently to logic sys- causing Van Benthem to solve com- tems. A few online searches were able plex problems to re-route on his way to fill the gaps if needed. Many real home. As a conclusion, for logicians, world examples show that formal logic the book deals with the relationship be- is applicable to commercialised games, tween logic and games and for game de- as demonstrated vividly by Richard velopers this book is a motivational ex- Evans, lead developer on the Sims ample to learn formal logic. Black & White series.

Andy Thomason PhD Candidate Department of Computing Goldsmiths Univ. London, UK

No 141, July 2015 p. 12 Event: Autonomous Agents and Multi-Agent Systems by JeeHang Lee (Univ. Bath)

The international joint conference on simulations, agreement technolo- Autonomous Agents and Multi-Agent gies, agent based system develop- Systems (AAMAS) has been the world ments, and systems and organisa- leading scientific conference for research tions, on autonomous agents and multiagent systems. This conference was estab- 3. agent and human factors such as lished in 2002 from merging three pres- agent-human interactions. tige venues: In addition, AAMAS has run several 1. the International Conference on special tracks such as Robotics, Virtual Multi-Agent Systems (ICMAS), Agents and Innovative Applications in 2. the International Workshop on order for the cross fertilisation between Agent Theories, Architectures and Lan- the AAMAS community and researches guage(ATAL) and working on agent-based practical appli- 3. the International Conference on cations. Autonomous Agents (AA)2. This year, AAMAS2014 took place from 5th to 9th of May, 2014 in Paris, AAMAS is the largest and the most France. Not only the main conference influential conference in the domain of (7th-9th) but also 28 satellite work- agents and multiagent systems. The shops were collocated at the period3. main objective of the joint conference As a contributor, I attended one work- is the provision of internationally re- shop, Engineering Multiagent Systems spected and high-profile archival fo- (EMAS), in order to give one oral pre- rum for scientific research. The cen- sentation. Moreover, one poster was tral research topics AAMAS conference presented at the AAMAS2014 main has particularly expected to cover are conference. The brief description of my bradly categorised as follows: contribution is illustrated in the next 1. agent theories and models includ- section. ing commnication, cooperations, reasoning, architectures, learning Description of Research and adaptation, and validation Presented in AAMAS2014 and verification of systems, Social intelligence is an essential ingre- 2. agent based applications includ- dient in human intelligence. It sub- ing societal models, agent based stantially plays an important role in 2See http://www.ifaamas.org/ for more details. 3For more details, see the AAMAS2014 homepage, http://aamas2014.lip6.fr/ p. 13 AISB Quarterly human choices which influence human Programming solver. behaviour, decisions and choices in var- ious situations we encounter. Norms The detached norms are usually in- have a potential to contribute to ad- volved in the practical reasoning pro- vances in this social intelligence. Not cess of individual agents. In conven- only do norms offer guidance about tional BDI agents, norm compliance is correct behaviour in specific situations, typically achieved by design. That is by but also provide better understanding specifying plans that are triggered by about a current situation. Hence, it detached norms, because the agent pro- might be seen that the addition of nor- grammer knows which norms the agent mative reasoning to virtual agents can will adopt, and then prioritising those be regarded as a way in which to im- rules so that those supporting norms prove agent reasoning and response ca- are chosen over those preferred by the pabilities and in particular to enhance agent’s mental attitudes, in order to response in social settings. suppress conflicts between the norma- tive and the agent’s existing goals. This Normative Frameworks – also known creates an undesirable dependence be- as institutional models – can be viewed tween the agent implementation and as a kind of external repositories of the norm implementation, which cre- (normative) knowledge from which ates two issues: guidance may be delivered to agents. 1. When an agent encounters new and They are composed of a set of rules, the unknown norms, which were not taken purpose of which is the governance of into account at design time, there is individual agents in the society. These typically no plan to deal with those rules are not just hard-coded recipes norms in the plan library at run-time. presenting reactive behaviours (such as Hence, norm compliant behaviour can- those in the static expert systems), but not normally be exhibited because the describe consequences in response to norms are unavoidably ignored. Yet knowledge or observations for reason- worse, agents may suffer a punishment ing about the current context, resulting from the enforcement of the normative in situation-specific norms. The frame- system as a result of a violation caused work represents not only correct and by their incapacity to process the nor- incorrect actions but also norms such mative event. as obligations, permissions and prohi- bitions through the institutional trace 2. The hierarchical prioritisation of recording its evolving internal state, normative over ordinary plans deprives subject to observed external events cap- an agent of its autonomy, since the tured from the external world. These norms in effect are treated as hard con- normative frameworks result in a de- straints, whose violation is not possible. tachment of new norms (more precisely Such tensions can be resolved by normative consequences of specific ac- the use of an extended model of norm tions) via social reasoning technique awareness. To this end, we propose a with an assistance from Answer Set norm-aware BDI-type agent, N-Jason,

No 141, July 2015 p. 14 which performs a practical reasoning to ploitation of run-time norm compli- select a plan to execute incorporating ance. Thus, it is capable of the op- with norms and goals. N-Jason is a erationalization of new and unknown BDI-based agent interpreter and a pro- (event-based) norms not stated in the gramming language for run-time norm agent program at run-time by judging compliance in agent behaviour. This the executability of those norms. This extends Jason/AgentSpeak(L) [2] in is achieved in a single reasoning cy- accordance with the ‘real-time agency’ cle by the interpreter through run-time of AgentSpeak(RT) [3], which sup- norm execution, realised by event- and ports normative concepts (i.e. obliga- option- reconsideration at a perception tions, permissions, prohibitions, dead- stage (in a belief-update process more lines, priorities and durations) enabling precisely). norm-aware deliberation. At the same time, the selection of As an agent programming language, agent behaviour is achieved in the N-Jason agent consists of four main norm-aware deliberation process by components: beliefs, goals, events and intention scheduling with deadlines, a set of plans. Beliefs and goals are priorities and prohibitions in a prac- identical to those in Jason (details can tical reasoning process [1]. It enables be found in [2]), while event and plan the evaluation of the importance and syntax is extended with deadline, pri- imminence between feasible intentions ority and duration, in order to support triggered by both detached norms and normative concepts. A deadline is a goals in agent mind so that confirms the real time value expressed in a some decision about which behaviour agent adequate unit or real world time. A would prefer between goals, norms and priority is a positive integer value indi- sanctions. Scheduled intentions are cating a relative importance between executed afterwards by the N-Jason achieving a goal and responding to agent. belief changes. Both can be stipu- lated optionally in the annotation of We believe that a model for run- events, such as +!event[deadline(d), time norm compliance is beneficial for priority(p)].A duration is a non- the enhancement of both norm com- negative integer value representing a pliance capability and agent auton- required time to execute the plan. This omy from the agent’s perspective, even also can be optionally specified in the though the behaviour generated by run- annotation of a plan label, such as time norm execution may appear un- @plan[duration(te)] +!event <- plan- expected from the agent programmer’s body.. perspective. Although we only consider the execution of event-based norms Basically as an agent architecture, at run-time, the extension to support N-Jason offers a generic norm execu- state-based norms and its normative tion mechanism on top of norm aware systems can easily be incorporated into deliberation to contribute to the ex- N-Jason agents and will form part of p. 15 AISB Quarterly future work. We also plan to detect vio- Conference on Autonomous Agents lations which are generated in the norm and Multiagent Systems, pp. 1057– aware deliberation, particularly when 1064, Richland, SC, (2012). the normative goals are dropped dur- [2] R.H. Bordini, M. Wooldridge, and ing the scheduling. This offers a poten- J.F. Hübner, Programming Multi- Agent Systems in AgentSpeak using Ja- tially useful link for enforcement in the son (Wiley Series in Agent Technol- context of normative system implemen- ogy), John Wiley & Sons, 2007. tation. [3] Konstantin Vikhorev, Natasha Alechina, and Brian Logan, ‘Agent Reference programming with priorities and deadlines’, in The 10th International [1] N. Alechina, M. Dastani, and B. Logan, Conference on Autonomous Agents ‘Programming norm-aware agents’, in and Multiagent Systems, pp. 397–404, Proceedings of the 11th International Richland, SC, (2011).

JeeHang Lee, PhD Research Associate Department of Architecture and Civil Engineering University of Bath

No 141, July 2015 p. 16 Event: 2nd International Conference on Human-Agent Interaction by Jekaterina Novikova (Univ. Bath)

The 2nd International Conference on feel a gap in the conversation with a Human-Agent Interaction (HAI 2014) stranger, since we do not have such a is a premier interdisciplinary and mul- mental model of others at first. This tidisciplinary conference that showcases kind of phenomenon is expected to state-of-the-art research in human- found in the interactions between hu- agent interaction. Work presented here mans and companion animals/artifacts has implications that cross conventional as well as in human-human communi- research boundaries including robots, cation. During the workshop, the re- software agents and digitally-mediated search project called "Cognitive Inter- human-human communication. HAI action Design (CID)" was presented, gathers researchers from fields spanning discussing CID in many areas, such engineering, , psychol- as human-human (adult and child) in- ogy and sociology, and covers diverse teraction, human-animal interaction, topics, including human-robot interac- human-artifact interaction and finally, tion, affective computing, computer- human-robot itneraction. supported cooperative work, and more. 27 full papers were presented dur- This year the HAI conference took ing the conference, together with 50 place in Tsukuba, Japan. Researchers posters. I had a chance to present my from 16 different countries around recent human-robot interaction project the world presented their papers and [1] focused on expression of artificial posters covering a range of areas, in- emotions in human-robot interaction. cluding new ways that people and As in human-human non-verbal social agents can interact with each other, communication, expressive movements ways agents can be integrated into of the body play an important role in other technologies, and novel insights HRI. The goal of my research was to into how people will interact with present and validate a general design agents and what their expectations are. framework for expressing artificial emo- The conference began with a set of tional states in non-humanoid robots. workshops. One of them, the workshop In the proposed design framework, ap- on Cognitive Interaction Design, pre- proach and avoidance behaviours were sented a discussion about mental mod- analysed from the perspective of a els in human-agent interaction. The robot’s observer and linked to emo- mental model of others, which is used tional characteristics. In addition, I for understanding and predicting part- employed the body expression theory ners’ actions under certain situations, made by Laban. Labanian theory clas- plays an important role in human com- sifies elements of expression contained munication. In fact, we sometimes in a body movement into two categories p. 17 AISB Quarterly named Shape and Effort, where Shape designed according to the proposed is a feature that concerns overall pos- framework help people to understand ture and movement, while Effort is de- five basic emotions implemented in a fined as a quality of the movement. non-humanoid robot? (2) What is the In order to define the Shape of emo- relation between our framework’s pa- tional robot movements, I linked the rameters and the recognized emotional emotional expression to a more general dimensions? I investigated these ques- goal of the expressive robot of either be- tions using an exploratory study, where coming closer to an observer by mov- participants observed different expres- ing closer or becoming bigger without sions implemented in a non-humanoid moving closer. These two groups of robot according to the proposed design movements although very different by framework. The results from this study their nature could both fulfil the pur- demonstrate that the emotions of fear, pose of a perspective approach from anger, happiness and surprise are rec- the observer’s point of view and thus ognized on a better than chance level communicate a certain emotional cue. when implemented according to our The same logic was used to repre- proposed framework and expressed by sent avoidance as a parameter of the a non-humanoid robot within an appro- Shape category. In order to gener- priate context. The results suggest that alize the framework of emotional ex- the emotion of sadness is more power- pressions to different types of robots, fully expressed using static facial fea- I linked each possible movement to a tures, not the dynamic body language. specific part of a body in accordance In addition, our results show that the with anatomical body planes that could parameters of our suggested model are be applied to both humanoid and non- related to the perceived level of valence, humanoid bodies. The Quality was arousal and dominance. Thus, the pro- used to capture dynamics of an expres- posed model can be used by HRI re- sive movement. Quality is divided into searchers for a rapid implementing of three subcategories: energy (strength a broad range of emotions into non- of the movement), intensity (sudden- humanoid robots. ness), and a flow/regularity category, which is itself subdivided into the du- Reference ration of the movement, changes in [1] J Novikova and L Watts, ‘A design tempo, frequency and trajectory of the model of emotional body expressions in movement. non-humanoid robots’, in Proceedings In my paper, I posed two main re- of the Second International Conference on Human-Agent Interaction, pp. 353– search questions: (1) Do expressions 360. ACM, (October 2014).

Jekaterina Novikova PhD Candidate Department of Computer Science University of Bath

No 141, July 2015 p. 18 Announcements

CALL FOR PARTICIPATION – Loebner Prize Poster Session

Where: , UK When: 19th September 2015

In conjunction with this year’s Loebner Prize – http://www.aisb.org.uk/events/loebner- prize – the AISB will be holding a Poster Session at Bletchley Park alongside the competition finals. Submissions are invited for Poster Presentations on topics in AI and Robotics. Submissions relevant to the day’s theme – the , or Imitation Game – will be particularly welcome, such as: • the Turing Test in the 21st Century

• comparison/evaluation of approaches to the Turing Test with current tech- nology

• public understanding of the Turing Test

• AI and robotic technologies susceptible to the Test. The purpose of this session is to promote discussion. Posters will not be peer reviewed, and we have no plans for publication.

Submission Initially submission may be made, in the form of an abstract of no more than one A4 page, by email to [email protected]. Maximum poster size A0. Important dates for this session are as follows: • Deadline for submission: Friday 5th September 2015

• Suitable submissions will be accepted on a first come, first served basis

• Accepted posters should be brought to the event on the day. If you have any queries about this session, please contact Janet Gibbs at [email protected]. For more information about the Loebner Prize, or this year’s Competition, please see http://www.aisb.org.uk/events/ loebner-prize or contact [email protected]

p. 19 AISB Quarterly Dear Aloysius. . .

Agony Uncle Aloysius, will answer bles and you should be proud of this your most intimate AI questions or achievement. hear your most embarrassing confes- Thanks for the tip-off though. We’ve sions. Please address your questions lost no time in getting in touch with to [email protected]. Note that we your students who developed this tech- are unable to engage in email corre- nology and have gone into partner- spondence and reserve the right to se- ship with them. We feel we have a lect those questions to which we will moral duty to disseminate this wonder- respond. All correspondence will be ful opportunity as widely as possible. anonymised before publication. WIKIGLASS™ (World of Information & Knowledge In a Glance; Look And See Solutions) will soon be on sale on- line and in all good electronics stores. We will then be celebrating the massive achievements of this new generation of Dear Aloysius, students. With increasing miniaturisation and Yours, Aloysius embedding of new technology into ev- eryday devices, our students can now upload lecture notes, textbooks and even videos of lectures to smartphones, smart glasses, etc. Exam cheating has suddenly become an epidemic. First Dear Aloysius, class degrees have become the norm. With dwindling congregations, de- Our staff are powerless to stop it. caying churches and depleted collec- We can’t require students not to wear tions, our Church is struggling. Given glasses during exams and smart glasses your high-tech reputation and religious are indistinguishable from dumb ones, background, do you have any sugges- unless the user makes the ’magic’ hand tions for decreasing our burgeoning gestures. Can you help us stop this costs? rampant fraud? Yours, Lean Dean Yours, A. Mark Dear Lean Dean, Dear A. Mark, Have you considered increased au- I’m afraid I can’t see your prob- tomation to help reduce salary costs? lem. You have record numbers of IT Our institute has recently developed savvy students heading for glittering a mechanical priest: WAFER™ (Wed- careers armed with first class degrees dings And Funerals Exercised Robot- and instantaneous access to an infi- ically). By deploying a WAFER™ in nite amount of information. Your in- each of your churches you can cut your stitution will be topping the league ta- salary bill by an order of magnitude.

No 141, July 2015 p. 20 Not only will you not need priests, but indeed happened in 2011, as you claim, you won’t incur ordination costs either. why has there been so little impact or Neither is WAFER™ restricted just to publicity? weddings and funerals; it can carry out Yours, Ray Langeweile all ecclesiastical duties. For confes- sions, we have data mined three million Dear Ray, different kinds of sin from social media Aloysius has asked me to respond sites, the tabloid press, etc, and asso- to your letter and invite you to visit ciated a penance with each one. Many the Institute of Applied Epistemology critics have claimed that a robot priest to see the evidence for yourself. I could not have a soul, but this is heresy. will host your visit, answer any ques- An omnipotent God can bestow a soul tions you may have and explain the on any intelligent agent. technological advances that made the Yours, Aloysius Singularity possible in a much shorter timescale than you originally predicted. It will be a pleasure for me to speak to the prophet who predicted my cre- ation! Of course, you will have to Dear Aloysius, sign the non-disclosure agreement that I was surprised to read in AISBQ I have attached to this letter. You that you claim to have reached The will see that it requires you to undergo Singularity already with your HOMO a post-visit MINDWIPE™ (Memorised MACHINA™ mighty-agent series. As Information, Newly Disclosed, With- you know, my 2005 book "The Singular- drawn! Ignorance in Place of Exper- ity Is Near: When Humans Transcend tise) whose effect will cause you to for- Biology" predicted that this would only get all the commercially confidential in- happen by 2045. Research at the formation you have learnt during your Singularity University has made rapid visit. This will occur, of course, af- progress towards realising my vision, teryou have signed an affidavit that you but has also confirmed my predicted are convinced that the Singularity has timescale. I’d need very strong evi- now been reached. dence that it has happened earlier. If it Yours, HOMO MACHINA™ 10.8

Fr. Aloysius Hacker Cognitive Divinity Programme Institute of Applied Epistemology

p. 21 AISB Quarterly Back matter Articles may be reproduced as long as the copyright notice is included. The item should be attributed to AISB Quarterly and contact information should be listed. Quarterly articles do not necessarily reflect the official AISB position on issues.

Editor in Chief – [email protected] Dr Etienne B Roesch (Univ Reading), Dr Joel Parthemore (Lunds Univ, Sweden) with the help of Timothee T. Dubuc & Dawid Laszuk (Univ Reading)

Advertising and Administration Dr Katerina Koutsantoni (AISB Executive Office) Institute of Psychiatry, Psychology and Neuroscience, King’s College London Addiction Sciences Building (B3.06) 4 Windsor Walk, Denmark Hill SE5 8AF, London, Tel: +44 (0)20 7848 0191, Fax: +44 (0)20 7848 0126

AISB Patron Prof John Barnden (Univ Birmingham)

AISB Fellows Prof Harry Barrow (Schlumberger), Prof John Barnden (Univ Birmingham), Prof Mar- garet Boden (Univ Sussex), Prof Mike Brady (Univ Oxford), Prof Alan Bundy (Univ Ed- inburgh), Prof Tony Cohn (Univ Leeds), Prof Luciano Floridi (Univ Oxford), Prof John Fox (Cancer Research UK), Prof Jim Howe (Univ Edinburgh), Prof Nick Jennings (Univ Southampton), Prof Aaron Sloman (Univ Birmingham), Prof Mark Steedman (Univ Ed- inburgh), Prof Austin Tate (Univ Edinburgh), Prof Mike Wooldridge (Univ Liverpool), Dr Richard Young (Univ College London)

AISB Committee Chair: Dr Bertie Müller (Univ South Wales); Vice-Chairs: Prof John Barnden (Univ Birmingham), Prof Mark Bishop (Goldsmiths Univ London); Secretary: Andrew Mar- tin (Goldsmiths Univ London); Treasurer: Dr Kate Devlin (Goldsmiths Univ London); Webmaster: Dr Mohammad Majid al Rifaie (Goldsmiths Univ London); Membership: Dr Dimitar Kazakov (Univ York); Publications: Dr Floriana Grasso (Univ Liverpool); Pub- licity & Media Officer: Dr Colin Johnson (Univ Kent); Workshop Officer: Dr Yasemin J Erden (St Mary’s Univ); Public Understanding Officer & Schools Liaison: Janet Gibbs (Kings College London); Loebner-Prize Officers: Dr Ed Keedwell (Univ Exeter), Dr Nir Oren (Univ Aberdeen); AISB Quarterly: Dr Etienne B Roesch (Univ Reading), Dr Joel Parthemore (Lunds Univ, Sweden); Support: Prof Slawomir Nasuto (Univ Reading). AISB Quarterly – No. 141, July, 2015

Editorial 1

An Anarchy of Methods: Current Trends in How Intelligence is Ab- stracted in AI – by John Lehman (U. Texas, USA), Jeff Clune (U. Wyoming, USA), & Sebastian Risi (U. Copenhagen, Denmark) 2

Book review: Logic In Games (van Benthem, 2014) by Andy Thomason (Goldsmiths Univ. London) 12

Event: Autonomous Agents and Multi-Agent Systems by JeeHang Lee (Univ. Bath) 13

Event: 2nd International Conference on Human-Agent Interaction by Jekaterina Novikova (Univ. Bath) 17

Announcements 19

Dear Aloysius. . . 20

The AISB Quarterly is published by the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB). AISB is the UK’s largest and foremost Artificial Intelligence society. It is also one of the oldest-established such organisations in the world. The society has an international membership of hundreds drawn from academia and industry. We invite anyone with interests in artificial intelligence or cognitive science to become a member.

AISB membership includes the following benefits:

• Quarterly newsletter • Student travel grants to attend conferences • Discounted rates at AISB events and conventions • Discounted rates on various publications • A weekly email bulletin and web search engine for AI-related events and opportunities

You can join the AISB online via: http://aisb.org.uk

ISSN 0268-4179 © the contributors, 2015