DOCSLIB.ORG

Learning Human Behavior from Observation for Gaming Applications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Learning Human Behavior from Observation for Gaming Applications University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2007 Learning Human Behavior From Observation For Gaming Applications Christopher Moriarty University of Central Florida Part of the Computer Engineering Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Masters Thesis (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Moriarty, Christopher, "Learning Human Behavior From Observation For Gaming Applications" (2007). Electronic Theses and Dissertations, 2004-2019. 3269. https://stars.library.ucf.edu/etd/3269 LEARNING HUMAN BEHAVIOR FROM OBSERVATION FOR GAMING APPLICATIONS by CHRIS MORIARTY B.S. University of Central Florida, 2005 A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in the School of Electrical Engineering and Computer Science in the College of Engineering and Computer Science at the University of Central Florida Orlando, Florida Summer Term 2007 © 2007 Christopher Moriarty ii ABSTRACT The gaming industry has reached a point where improving graphics has only a small effect on how much a player will enjoy a game. One focus has turned to adding more humanlike characteristics into computer game agents. Machine learning techniques are being used scarcely in games, although they do offer powerful means for creating humanlike behaviors in agents. The first person shooter (FPS), Quake 2, is an open source game that offers a multi-agent environment to create game agents (bots) in. This work attempts to combine neural networks with a modeling paradigm known as context based reasoning (CxBR) to create a contextual game observation (CONGO) system that produces Quake 2 agents that behave as a human player trains them to act. A default level of intelligence is instilled into the bots through contextual scripts to prevent the bot from being trained to be completely useless. The results show that the humanness and entertainment value as compared to a traditional scripted bot have improved, although, CONGO bots usually ranked only slightly above a novice skill level. Overall, CONGO is a technique that offers the gaming community a mode of game play that has promising entertainment value. iii Dedicated to my Mom for letting my Dad buy the Apple IIGS instead of a car. iv ACKNOWLEDGMENTS I’d like to first acknowledge my friends at the UCF Intelligent Systems Lab. They are some of the smartest and most helpful folks I’ve ever met. In particular, Brian Becker and Gary Stein, whose programming wizardry far exceeds that of which any one person should possess. I can’t thank my family enough for the much needed moral support they always gave me. Last but definitely not least, I also thank my advisor, Dr. Gonzalez, for all his knowledge and guidance. v TABLE OF CONTENTS LIST OF FIGURES ......................................................................................................... X LIST OF TABLES .......................................................................................................... XI CHAPTER 1: INTRODUCTION AND BACKGROUND ............................................ 1 1.1 History of Video Games ........................................................................................... 1 1.1.2 Consoles vs PC .................................................................................................. 2 1.1.3 Graphics Don’t Mean Everything Anymore ...................................................... 3 1.2 Background of Research ........................................................................................... 5 1.2.1 Next-Generation Game Consoles Adding More Than Graphics ....................... 5 1.3 Game Engine as a Test-Bed for Research ................................................................. 6 1.4 Game AI Behind the Times ...................................................................................... 8 1.5 Machine Learning for Gaming .................................................................................. 9 1.6 Combining Machine Learning and Knowledge Engineering ................................. 11 1.7 CxBR and IASKNOT Comparison ......................................................................... 14 1.8 Default Level of Intelligence .................................................................................. 16 CHAPTER 2: LITERATURE REVIEW ..................................................................... 17 2.1 First Person Shooter Engines for Research ............................................................. 17 2.1.1 Systems That Simplify the Use of a Game Environment ................................ 17 2.2 Complex AI Employed Through Game Engines .................................................... 19 2.2.1 SOAR Rule-Based Architecture ...................................................................... 19 2.2.2 “D’Artagnan Cognitive Architecture” ............................................................. 21 2.2.3 Machine Learning Study .................................................................................. 23 2.2.4 Neural Networks applied to a FPS ................................................................... 25 2.3 Genetic Algorithms for Games ............................................................................... 27 2.3.1 Neuro-Evolution .............................................................................................. 28 2.4 Learning from Observation ..................................................................................... 29 2.4.1 “Learning Robotic Primitives” ........................................................................ 29 2.4.2 Complex Learning from Observation Techniques ........................................... 30 2.5 Summary ................................................................................................................. 31 CHAPTER 3: PROBLEM DEFINITION AND HYPOTHESIS................................ 34 3.1 General Problem Statements ................................................................................... 34 3.2 Specific Problem Statements .................................................................................. 35 3.2.1 Humanness in Games ....................................................................................... 35 3.2.2 Validating Humanness and Entertainment Value ............................................ 37 3.3 Hypothesis............................................................................................................... 37 3.4 Contributions........................................................................................................... 37 CHAPTER 4: APPROACH ........................................................................................... 38 vi 4.1 Introduction to Contextual Game Observation (CONGO) ..................................... 38 4.1.1 Learning from Observation using CxBR and NN............................................ 39 4.1.2 Synergistic Combination .................................................................................. 39 4.2 Overview of the System .......................................................................................... 40 4.2.1 Contextual Observation Module ...................................................................... 40 4.2.1.1 Context Inheritance ................................................................................... 41 4.2.1.2 Modified CxBR Engine Algorithm ........................................................... 42 4.2.1.3 Pseudo Code for Modified CxBR Algorithm ........................................... 42 4.2.2 Training module ............................................................................................... 44 4.2.2.1 RPROP – Neural Network Training Algorithm........................................ 44 4.2.2.2 Time Delay Neural Networks ................................................................... 45 4.2.3 Gaming Performance Module .......................................................................... 46 4.2.3.1 Performance CxBR engine ....................................................................... 46 4.3 Context Breakdown ................................................................................................ 48 4.3.1 Reaction Contexts ............................................................................................ 50 4.4 Summary ................................................................................................................. 50 CHAPTER 5: IMPLEMENTATION ........................................................................... 52 5.1 Quake 2 Introduction .............................................................................................. 52 5.1.1 Game Play ........................................................................................................ 52 5.1.2 ACEbot ............................................................................................................ 53 5.2 Quake 2 Source Code .............................................................................................. 54 5.2.1 Quake Structure ............................................................................................... 55 5.2.1.1 Sever Executable Modifications
Load more

Recommended publications
  • The Resurrection of Permadeath: an Analysis of the Sustainability of Permadeath Use in Video Games
    The Resurrection of Permadeath: An analysis of the sustainability of Permadeath use in Video Games. Hugh Ruddy A research paper submitted to the University of Dublin, in partial fulfilment of the requirements for the degree of Master of Science Interactive Digital Media 2014 Declaration I declare that the work described in this research paper is, except where otherwise stated, entirely my own work and has not been submitted as an exercise for a degree at this or any other university. Signed: ___________________ Hugh Ruddy 28th February 2014 Permission to lend and/or copy I agree that Trinity College Library may lend or copy this research Paper upon request. Signed: ___________________ Hugh Ruddy 28th February 2014 Abstract The purpose of this research paper is to study the the past, present and future use of Permadeath in video games. The emergence of Permadeath games in recent months has exposed the mainstream gaming population to the concept of the permanent death of the game avatar, a notion that has been vehemently avoided by game developers in the past. The paper discusses the many incarnations of Permadeath that have been implemented since the dawn of video games, and uses examples to illustrate how gamers are crying out for games to challenge them in a unique way. The aims of this are to highlight the potential that Permadeath has in the gaming world to become a genre by itself, as well as to give insights into the ways in which gamers play Permadeath games at the present. To carry out this research, the paper examines the motivation players have to play games from a theoretical standpoint, and investigates how the possibilty of failure in video games should not be something gamers stay away from.
    [Show full text]
  • Shading in Valve's Source Engine
    Advanced Real-Time Rendering in 3D Graphics and Games Course – SIGGRAPH 2006 Chapter 8 Shading in Valve’s Source Engine Jason Mitchell9, Gary McTaggart10 and Chris Green11 8.1 Introduction Starting with the release of Half-Life 2 in November 2004, Valve has been shipping games based upon its Source game engine. Other Valve titles using this engine include Counter-Strike: Source, Lost Coast, Day of Defeat: Source and the recent Half-Life 2: Episode 1. At the time that Half-Life 2 shipped, the key innovation of the Source engine’s rendering system was a novel world lighting system called Radiosity Normal Mapping. This technique uses a novel basis to economically combine the soft realistic 9 [email protected] 10 [email protected] 11 [email protected] 8 - 1 © 2007 Valve Corporation. All Rights Reserved Chapter 8: Shading in Valve’s Source Engine lighting of radiosity with the reusable high frequency detail provided by normal mapping. In order for our characters to integrate naturally with our radiosity normal mapped scenes, we used an irradiance volume to provide directional ambient illumination in addition to a small number of local lights for our characters. With Valve’s recent shift to episodic content development, we have focused on incremental technology updates to the Source engine. For example, in the fall of 2005, we shipped an additional free Half- Life 2 game level called Lost Coast and the multiplayer game Day of Defeat: Source. Both of these titles featured real-time High Dynamic Range (HDR) rendering and the latter also showcased the addition of real-time color correction to the engine.
    [Show full text]
  • Using the MMORPG 'Runescape' to Engage Korean
    Using the MMORPG ‘RuneScape’ to Engage Korean EFL (English as a Foreign Language) Young Learners in Learning Vocabulary and Reading Skills Kwengnam Kim Submitted in accordance with the requirements for the degree of Doctor of Philosophy The University of Leeds School of Education October 2015 -I- INTELLECTUAL PROPERTY The candidate confirms that the work submitted is her own and that appropriate credit has been given where reference has been made to the work of others. This copy has been supplied on the understanding that it is copyright material and that no quotation from the thesis may be published without proper acknowledgement. © 2015 The University of Leeds and Kwengnam Kim The right of Kwengnam Kim to be identified as Author of this work has been asserted by her in accordance with the Copyright, Designs and Patents Act 1988. -II- DECLARATION OF AUTHORSHIP The work conducted during the development of this PhD thesis has led to a number of presentations and a guest talk. Papers and extended abstracts from the presentations and a guest talk have been generated and a paper has been published in the BAAL conference' proceedings. A list of the papers arising from this study is presented below. Kim, K. (2012) ‘MMORPG RuneScape and Korean Children’s Vocabulary and Reading Skills’. Paper as Guest Talk is presented at CRELL Seminar in University of Roehampton, London, UK, 31st, October 2012. Kim, K. (2012) ‘Online role-playing game and Korean children’s English vocabulary and reading skills’. Paper is presented in AsiaCALL 2012 (11th International Conference of Computer Assisted Language Learning), in Ho Chi Minh City, Vietnam, 16th-18th, November 2012.
    [Show full text]
  • Cyber-Synchronicity: the Concurrence of the Virtual
    Cyber-Synchronicity: The Concurrence of the Virtual and the Material via Text-Based Virtual Reality A dissertation presented to the faculty of the Scripps College of Communication of Ohio University In partial fulfillment of the requirements for the degree Doctor of Philosophy Jeffrey S. Smith March 2010 © 2010 Jeffrey S. Smith. All Rights Reserved. This dissertation titled Cyber-Synchronicity: The Concurrence of the Virtual and the Material Via Text-Based Virtual Reality by JEFFREY S. SMITH has been approved for the School of Media Arts and Studies and the Scripps College of Communication by Joseph W. Slade III Professor of Media Arts and Studies Gregory J. Shepherd Dean, Scripps College of Communication ii ABSTRACT SMITH, JEFFREY S., Ph.D., March 2010, Mass Communication Cyber-Synchronicity: The Concurrence of the Virtual and the Material Via Text-Based Virtual Reality (384 pp.) Director of Dissertation: Joseph W. Slade III This dissertation investigates the experiences of participants in a text-based virtual reality known as a Multi-User Domain, or MUD. Through in-depth electronic interviews, staff members and players of Aurealan Realms MUD were queried regarding the impact of their participation in the MUD on their perceived sense of self, community, and culture. Second, the interviews were subjected to a qualitative thematic analysis through which the nature of the participant’s phenomenological lived experience is explored with a specific eye toward any significant over or interconnection between each participant’s virtual and material experiences. An extended analysis of the experiences of respondents, combined with supporting material from other academic investigators, provides a map with which to chart the synchronous and synonymous relationship between a participant’s perceived sense of material identity, community, and culture, and her perceived sense of virtual identity, community, and culture.
    [Show full text]
  • Preparation of Papers for R-ICT 2007
    Use of Graph Theory in Video Game Artificial Intelligence for Promoting Replay Value Case Study: “AI Director” of Valve’s Left 4 Dead Alvin Natwiguna - 13512030 Program Studi Teknik Informatika Sekolah Teknik Elektro dan Informatika Institut Teknologi Bandung, Jl. Ganesha 10 Bandung 40132, Indonesia [email protected] Abstract—One of the main factors that makes a video game The Infected is split to two types, the Normal Infected – as a product – has a long life cycle is its replay value. Replay and the Special Infected. The Normal Infected are zombies value describes the ‘depth’ of an entertainment medium; in that attack at close range. Each has a low pool of health, this case, a video game. Besides the content of the game, one but they sometimes attack in large mobs. The Special of the main factors that influences a video game’s replay value is the system on which the video game is built upon. This Infected are zombies that have special features which paper will discuss the use of graph theory in explaining makes them quite deadly to the survivors. For example, the Valve’s Left 4 Dead artificial intelligence system – the “AI Tank has a monstrous, large body capable of taking large Director” – that promotes its replay value. amount of damage, punching cars and throwing a large chunk of rock to the survivors. It also has a deadly punch Index Terms—Artificial Intelligence, Graph, Left 4 Dead, that can incapacitate a survivor in one hit. Special Infected Replay Value. is further divided to normal and boss infected, based on how frequently they spawn, i.e.
    [Show full text]
  • What's an Agent, Anyway? a Sociological Case Study Leonard N
    What's An Agent, Anyway? A Sociological Case Study Leonard N. Foner Abstract. The term "agent" has been picked up, widely appropriated, and in many cases misappropriated, by technical publications, lay publi- cations, and many researchers in computer science. I examine some of the appropriations and misappropriations, talk about their reasons and implications, and delve into a case study of an agent that is used to il- lustratively define what I consider an agent to be, and why many of the current uses of the term are misleading. The agent I consider is Julia, a TinyMUD robot of the Maas-Neotek family, all of whom have been in almost constant use at several sites on the Internet for the last two or three years. I also speak at length about the sociology of such agents, what we might expect to find in the future, and why sociology is such an important aspect to consider when investigating agent-oriented pro- 1 gramming. Agents and appropriation There has been a flurry of references to "agents" in both the lay and technical press in recent months. While the idea of computational agents that do one’s bid- 2 ding goes back decades, the explosion of recent press has been astonishing. Pattie 3 Maes and her group’s work on learning interface agents has been cited in at least two dozen publications in the last year or so, many of them lay press (e.g., The Los Angeles Times, Mass High Tech or semi-technical press (e.g., MacWeek, Mac- World, etc). A symposium at MIT on agents and their programming, held in Octo- ber of 1992, drew at least a thousand participants.
    [Show full text]
  • 8 Guilt in Dayz Marcus Carter and Fraser Allison Guilt in Dayz
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Sydney eScholarship FOR REPOSITORY USE ONLY DO NOT DISTRIBUTE 8 Guilt in DayZ Marcus Carter and Fraser Allison Guilt in DayZ Marcus Carter and Fraser Allison © Massachusetts Institute of Technology All Rights Reserved I get a sick feeling in my stomach when I kill someone. —Player #1431’s response to the question “Do you ever feel bad killing another player in DayZ ?” Death in most games is simply a metaphor for failure (Bartle 2010). Killing another player in a first-person shooter (FPS) game such as Call of Duty (Infinity Ward 2003) is generally considered to be as transgressive as taking an opponent’s pawn in chess. In an early exploratory study of players’ experiences and processing of violence in digital videogames, Christoph Klimmt and his colleagues concluded that “moral man- agement does not apply to multiplayer combat games” (2006, 325). In other words, player killing is not a violation of moral codes or a source of moral concern for players. Subsequent studies of player experiences of guilt and moral concern in violent video- games (Hartmann, Toz, and Brandon 2010; Hartmann and Vorderer 2010; Gollwitzer and Melzer 2012) have consequently focused on the moral experiences associated with single-player games and the engagement with transgressive fictional, virtual narrative content. This is not the case, however, for DayZ (Bohemia Interactive 2017), a zombie- themed FPS survival game in which players experience levels of moral concern and anguish that might be considered extreme for a multiplayer digital game.
    [Show full text]
  • Game Review | the Legend of Zelda file:///Users/Denadebry/Desktop/Hpsresearch/Papersfromgreen
    Game Review | The Legend of Zelda file:///Users/denadebry/Desktop/HPSresearch/PapersFromGreen... Matt Waddell STS 145: Game Review Table of Contents: Da Game Story Line & Game-play Technical Stuff Game Design Success Endnotes In 1984 President Hiroshi Yamauchi asked apprentice game designer Sigeru Miyamoto to oversee R&D4, a new research and development team for Nintendo Co., Ltd. Miyamoto's group, Joho Kaihatsu, had one assignment: "to come up with the most imaginative video games ever." 1 On February 21, 1986, Nintendo Co., Ltd. published The Legend of Zelda (hereafter abbreviated LoZ) for the Famicom in Japan. It was Miyamoto's first autonomous attempt at game design. In July 1987, Nintendo of America published LoZ for the Nintendo Entertainment System in the United States, this time with a shiny gold cartridge. Origins: where did LoZ come from? Miyamoto admits that LoZ is partly based on Ridley Scott's movie Legend. Indeed, Miyamoto's video game shares more than just a title with Scott's 1985 production. But Miyamoto's fundamental inspiration for LoZ remains his childhood home, with its maze of rooms, sliding shoji screens, and "hallways, from which there seemed to be a medieval castle's supply of hidden rooms." 2 Story Line: a brief synopsis. In the land of Hyrule, the legend of the "Triforce" was being passed down from generation to generation; golden triangles possessing mystical powers. One day, an evil army attacked Hyrule and stole the Triforce of Power. This army was led by Gannon, the powerful Prince of Darkness. Fearing his wicked rule, Zelda, the princess of Hyrule, split the remaining Triforce of Wisdom into eight fragments and hid them throughout the land.
    [Show full text]
  • Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning Frank G
    180 IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES, VOL. 7, NO. 2, JUNE 2015 Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning Frank G. Glavin and Michael G. Madden Abstract—In current state-of-the-art commercial first person late the most kills. Objective based games also exist where the shooter games, computer controlled bots, also known as non- emphasis is no longer on kills and deaths but on specific tasks player characters, can often be easily distinguishable from those in the game which, when successfully completed, result in ac- controlled by humans. Tell-tale signs such as failed navigation, “sixth sense” knowledge of human players' whereabouts and quiring points for your team. Two examples of such games are deterministic, scripted behaviors are some of the causes of this. We ‘Capture the Flag’ and ‘Domination’. The former involves re- propose, however, that one of the biggest indicators of nonhuman- trieving a flag from the enemies' base and returning it to your like behavior in these games can be found in the weapon shooting base without dying. The latter involves keeping control of pre- capability of the bot. Consistently perfect accuracy and “locking defined areas on the map for as long as possible. All of these on” to opponents in their visual field from any distance are in- dicative capabilities of bots that are not found in human players. game types require, first and foremost, that the player is profi- Traditionally, the bot is handicapped in some way with either a cient when it comes to combat.
    [Show full text]
  • University of California Santa Cruz Learning from Games
    UNIVERSITY OF CALIFORNIA SANTA CRUZ LEARNING FROM GAMES FOR GENERATIVE PURPOSES A dissertation submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in COMPUTER SCIENCE by Adam J. Summerville June 2018 The Dissertation of Adam J. Summerville is approved: Professor Michael Mateas, Chair Professor Noah Wardrip-Fruin Professor Santiago Ontañón Dean Tyrus Miller Vice Provost and Dean of Graduate Studies Copyright © by Adam J. Summerville 2018 Table of Contents List of Figures vi List of Tables xiv Abstract xvi Dedication xvii Acknowledgments xviii 1 Introduction 1 1.1 Research Contributions ........................... 5 I Learning From Game 7 2 How Humans Understand Games 8 2.1 Human Annotation Tasks .......................... 17 2.2 Entity Persistence .............................. 17 2.3 Camera Motion ................................ 18 2.4 Room Detection ............................... 21 2.4.1 Teleportation ............................. 23 2.4.2 Traversal ............................... 25 2.5 Animation Cycles ............................... 32 2.6 Mode Dynamics ................................ 34 2.7 Conclusion .................................. 36 3 Mappy – A System for Map Extraction and Annotation via Observa- tion 37 3.1 NES Architecture and Emulation ...................... 41 3.2 Operationalizing Entity Persistence ..................... 43 3.3 Inferring Camera Motion .......................... 49 iii 3.4 Determining Room Transitions ....................... 59 3.4.1 Automatic Mapping ........................
    [Show full text]
  • PROCEDURAL CONTENT GENERATION for GAME DESIGNERS a Dissertation
    UNIVERSITY OF CALIFORNIA SANTA CRUZ EXPRESSIVE DESIGN TOOLS: PROCEDURAL CONTENT GENERATION FOR GAME DESIGNERS A dissertation submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in COMPUTER SCIENCE by Gillian Margaret Smith June 2012 The Dissertation of Gillian Margaret Smith is approved: ________________________________ Professor Jim Whitehead, Chair ________________________________ Associate Professor Michael Mateas ________________________________ Associate Professor Noah Wardrip-Fruin ________________________________ Professor R. Michael Young ________________________________ Tyrus Miller Vice Provost and Dean of Graduate Studies Copyright © by Gillian Margaret Smith 2012 TABLE OF CONTENTS List of Figures .................................................................................................................. ix List of Tables ................................................................................................................ xvii Abstract ...................................................................................................................... xviii Acknowledgments ......................................................................................................... xx Chapter 1: Introduction ....................................................................................................1 1 Procedural Content Generation ................................................................................. 6 1.1 Game Design...................................................................................................
    [Show full text]
  • Cheating by Video Game Participants
    Cheating by Video Game Participants J.R. Parker University of Calgary [email protected] Abstract In this paper, I explore the concept of cheating in the medium of the video game. Why do people do it? How does online cheating differ from offline? The existence of cheat codes seems to imply that cheating is rampant and even encouraged - or perhaps it’s just that the codes are poorly named. I also look at criminal activity in games, and at some ways to reduce cheating activities in online games. Introduction The word ‘cheat’ carries with it ancient societal passions. There is deception associated with the word, a lie, but more than that. There is a deliberation about cheating that makes it a worse lie than most, and there is an implication of doing someone harm in order to benefit oneself. Biblical cheats have died for their sin, and card cheats in the old West suffered a similar fate. Technology has changed the manner in which people can cheat, and has created many new ways to do it. So-called computer crime, identity theft, phishing, and other modern ways of cheating are enabled by the prevalent use of digital technology in many areas of society. Of course, the activities listed above are actually crimes; yet non-criminal cheating takes place too. Plagiarism is now very easy because of search engines and the Internet (easier to detect, too!). People misrepresent their identity on the Web with greater frequency, sometimes in self-defense. Cheating at games has traditionally been held in low regard because people that you play with are often friends.
    [Show full text]