<<

Changing the Game: Measuring and Influencing Player Emotions Through Meta AI

Yuta Mizuno Daiki Satoi AI Technical Game Designer AI Researcher SQUARE ENIX CO., LTD. SQUARE ENIX CO., LTD. [email protected] [email protected] Meta AI example

Meta AI (AI Director) controls the whole game…

Meta AI

Difficulty Story Generation Weather

Ally NPC(s) Player Enemy Melee: Character NPC(s) Ranged:

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Meta AI example

Meta AI (AI Director) controls the whole game... e.g. weather, enemies, difficulty, story generation...

Left 4 Dead – “From COUNTER-STRIKE to LEFT 4 DEAD: Creating Replayable Cooperative Experiences” (Booth, GDC2009)

TOTAL WAR - “Battle AI in TOTAL WAR: SHOGUN 2 and Lessons Learned” (Gudmundsson, PGAI 2011)

WARFRAME - “Space Ninjas with Machineguns!” (Brewer, GDC2013)

Far Cry 4 - “Far Cry’s AI: A manifesto for systemic and emergent game-play and open world ” (Varnier, GAIC2014)

Assassin’s Creed Origins - “Virtual Insanity: Meta AI on 'Assassin's Creed: Origins'” (Lefebvre, GDC2018)

Final Fantasy XV - “Eos is Alive: The AI Systems of 'Final Fantasy XV‘” (Miyake, GDC2018)

-> Meta AI is becoming more necessary

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why? Let’s start by asking “Why?”

Why do you make games?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why? Why do we need to try?

Why do you try to make your games the best?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why? Why do we need to make the game best?

For me, the reason why: Games are NOT necessary

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games are not necessary, So… What will happen if games are not necessary?

GAME = NOT necessary

Not needed for living… food/sleep/reproduction

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games are not necessary, So… What will happen if games are not necessary?

GAME = NOT necessary

Not needed for living… food/sleep/reproduction Easy to avoid to buy

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games are not necessary, So… What will happen if games are not necessary?

GAME = NOT necessary

Not needed for living… food/sleep/reproduction Easy to avoid to buy Easy to quit

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games are not necessary, So… What will happen if games are not necessary?

GAME = NOT necessary

Not needed for living… food/sleep/reproduction Easy to avoid to buy Easy to quit Why are we working so hard?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games are not necessary, So… What will happen if games are not necessary?

GAME = NOT necessary

Not needed for living… food/sleep/reproduction Easy to avoid to buy Easy to quit But it makes me excited!

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What should we do? If it’s not necessary, people judge very severely. Games are NOT necessary So, we must compel people to play

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What is needed? If it’s not necessary, people judge very severely. Games are NOT necessary Good game is NOT enough Better game is still NOT enough BEST game is the only one way

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why, How, What about game The way to make the best game is…

Game

Why

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why, How, What about game The way to make the best game is…

Game

Why Make the Best games

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why, How, What about game The way to make the best game is…

Game

Why Make the Best games Adapt to different play-styles How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why, How, What about game The way to make the best game is…

Game

Why Make the Best games Adapt to different play-styles How play-skills What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why, How, What about game The way to make the best game is…

Game

Why Make the Best games Adapt to different play-styles How play-skills What Make games with Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Introduction

• Yuta Mizuno – AI technical Game Designer – Meta AI research, Advocating and Consulting Meta AI to Products – The Meta AI General design • Daiki Satoi, Ph.D. – AI researcher – R&D on Meta AI, Game Play Analysis, Character AI – The 2D Emotion-based Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI overview – Why Meta AI – What is Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI General Design – How to make Meta AI – What can Game Designer do for Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI Practical Model – Emotion-based Meta AI – Visualization for Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Usefulness of Meta AI General Design • Using Ru-I-Di-ism with fellow game designers • Measuring and influencing player’s emotion • Benefits of 2D Emotion-based Meta AI • Importance of Visualizing gameplay and emotion

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI overview – Why Meta AI?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game

Game

Why

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game Adapting complex game world reality Game

Why

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game Adapting complex game world reality Game Adapting different skills of players

Why

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game Adapting complex game world reality Game Adapting different skills of players Adapting Generated level Why

How

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game Adapting complex game world reality Game Adapting different skills of players Adapting Generated level Why

How Changing games dynamically for players

What

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why Meta AI? The way to make the best game is…

NEEDS for great game Adapting complex game world reality Game Adapting different skills of players Adapting Generated level Why

How Changing games dynamically for players

What Making games with Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI overview

– What is Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What is Meta AI?

Meta AI is one type of Game AIs

Meta AI

Game AI

Character AI Navigation AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What is Meta AI? Meta AI is … • Senses the whole Game World • Dynamically controls the whole game

Meta AI

Ally NPC(s)

Player Enemy Character NPC(s) Melee: Ranged:

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What is Meta AI?

• Senses the whole Game World • Dynamically controls the whole game

Meta AI → How to implement

Ally Meta AI? NPC(s)

Player Enemy Character NPC(s) Melee: Ranged:

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI General Design

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI General Design – How to make Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI This is a general design of Meta AI.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision making Recog- Motion nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision making Recog- Motion nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision making Recog- Motion nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision making Recog- Motion nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision making Recog- Motion nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Start From An Agent Architecture At the beginning, consider an agent architecture, well known in the robotics field.

Character AI Decision Information making Recog- Motion Flow nition making

Sensor:Body Effector:Body Environment

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Then, compare the general design of Meta AI to the agent architecture.

Meta AI Game Information World Maker Operation Analyzer Generator Flow

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Comparison With Character AI We can see that Meta AI is similar to character AI.

Character AI Decision making Recog- Motion nition making

Meta AI Game World Maker Operation Analyzer Generator

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Comparison With Character AI We can see that Meta AI is similar to character AI.

Character AI Enemy is near Decision I have a sword making Recog- Motion nition making

Meta AI Game World Maker Operation Analyzer Generator

Player is bored

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Comparison With Character AI We can see that Meta AI is similarAttack to thecharacter enemy AI. with the sword Character AI Decision making Recog- Motion nition making

Meta AI Game World Maker Operation Analyzer Generator

Raiding by the Attack groups

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Comparison With Character AI We can see that Meta AI is similar to character AI. GO! Character AI (SHOUT) Decision making Recog- Motion nition making

Meta AI Game World Maker Operation Analyzer Generator

GO! (Order to teams)

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Main Part Of Meta AI The explanation of the Meta AI main part is finished. Let’s move to the game world part.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game World For Meta AI What information is usable by the Meta AI in the game world?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game World For Meta AI What information is usable by the Meta AI in the game world?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World Nav Mesh Knowledge Heat Map Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game World For Meta AI What information is usable by the Meta AI in the game world?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World Nav Mesh Knowledge Heat Map Representation

Controller input Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Sensor of Meta AI Sensors gather information and send it to the world analyzer.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Sensor of Meta AI Sensors gather information and send it to the world analyzer.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. World Analyzer What does world analyzer do?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. World Analyzer The world analyzer makes usable information for the game maker.

Meta AI Game WorldWorld MakerBattle situationOperation AnalyzerAnalyzer Path expectationGenerator Sensor Player Emotion Effector Sensor(t=1) Game World Sensor(t=2) Knowledge Representation Interaction Sensor(t=n) Space Knowledge Player Representation

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. World Analyzer The world analyzer makes usable information for the game maker. Attack? Defense? Escape?

Target candidates Constraints of Actions

Ally NPC(s)

Enemy NPC(s)

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI What does the game maker do?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game Maker Makes plans and selects the best one World Analyzer Game Maker

Locations of Plan 1 BEST Enemy/Friend/ ... Raiding by Attack group Terrain KR Plan 2 Tactical position Hunting down to the narrow bridge Log of past plans Plan 3 weak enemy Player emotions and healing items

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game Maker

Makes plans and selects the best one BEST Attack? Defense? Escape?

Target candidates Constraints of Actions

BEST Ally NPC(s)

Enemy NPC(s) Player Character

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Operation Generator What does the operation generator do?

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Operation Generator The operation generator breaks down the plans into operations Game Maker Operation Generator Raiding by Attack group Plan 1 BEST Raiding by Attack group Rapidly Break down Type:Attack Plan 2 the plan Hunting down to Power: 25000 the narrow bridge Order it Plan 3 Squad manager Popping weak enemy 4 enemies, 2teams and healing items

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI The operation generator sends parameters through effectors.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI The operation generator sends parameters through effectors.

Attack Meta AI Rapidly Game POW:2500 World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Representation

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI Effectors send them to the interaction space.

Attack Meta AI Rapidly Game POW:2500 World Maker Operation Analyzer Generator Effector(t=1) Effector(t=2) Sensor Game World Effector(t=n) Knowledge Interaction Representation Space

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI The interaction space controls the game with the operations.

Attack Meta AI Rapidly Game POW:2500 World Maker Operation Analyzer Generator Effector(t=1) Effector(t=2) Sensor Game World Effector(t=n) Knowledge Interaction Representation Space Enemy manager Spawn Manager Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Operation Generator The operation generator sends operations to the enemy manager.

Operation Generator Enemy Manager Interaction Space

Ally NPC(s)

Enemy NPC(s) Player Character

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Main Part Of Meta AI This is the main part of Meta AI.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Interaction Representation Space

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. General Design Of Meta AI This is a general design of Meta AI.

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Interaction Representation Space

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI General Design – How to make Meta AI – How can game designers improve Meta AI?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Game Designer for Meta AI Elements controlled by Meta AI are deeply related to .

Meta AI Game World Maker Operation Analyzer Generator

Sensor Effector Game World

Knowledge Interaction Representation Space

Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Shake Player’s Emotion With Meta AI When emotions are shaken, we are moved.

Game Designer Tense - Good at thinking “How to shake Stressed Excited emotions?” Sad Happy

Bored Relaxed ->Can pick up elements Fatigued For Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Too Flexible, Too Complex How to identify which element is the best to be controlled by Meta AI?

Meta AI: Too Flexible Games: Too Complex

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Too Flexible, Too Complex How to identify which element is the best to be controlled by Meta AI?

Meta AI: Too Flexible Games: Too Complex

Break down into smaller elements

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Too Flexible, Too Complex How to identify which element is the best to be controlled by Meta AI?

Meta AI: Too Flexible Games: Too Complex

Break down into smaller elements with fellow game designers

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Too Flexible, Too Complex Break down your game into 3 elements “Ru-I-Di”

Rule Fun game should have

”Ru-I-Di –ism”

Interaction Dilemma [Yonemitsu, 2006] (Similar to Luigi )

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Rule defines and controls Interaction and Dilemma of your game

The rule defines what should be Interaction Rule and Dilemma, controls the difficulty and frequency of Interaction and Dilemma.

Interaction Dilemma e.g.) Key input -> What will happen

Condition of Win / Spawning Physics

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule (1) Player’s Action Interaction Dilemma

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule Player’s Action Interaction Dilemma (2) Game’s Reaction

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule (3) Player’s Reaction Interaction Dilemma

Game’s Reaction

Progress of game with this loop → Fun of the game

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Rule for Interaction and Dilemma

Rule Interaction is the loop of action and reaction.

Interaction Dilemma - Player’s action and reaction - Game system’s action and reaction - …

For example …

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule (1) Player’s Action Moving Interaction Dilemma

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule Player’s Action Interaction Dilemma (2) Game’s Spawning Reaction Enemy

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Interaction makes the game loop. Interaction = the reaction loop Rule (3) Shooting Player’s Reaction Interaction Dilemma

Game’s Reaction Spawning

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di -ism Dilemma to make interaction continuously

Rule Causes the player to make interactions continuously

Interaction Dilemma Players are forced to select by dilemma

Makes the player believe to be able to control the success or failure

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Ru-I-Di –ism (e.g. Dilemma) Meta AI sets two treasures to different places There is energy shared between the bomb attack and normal shots Dilemma Many enemies = seems a good time to bomb There is a rare enemy that can only be defeated with the bomb -> Do you use the bomb now? Or keep it for the rare enemy? -> Meta AI controls spawning of rare enemy

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Games Are Too Complex To Adjust Break down the game into three ingredients

Rule Fun game should have

”Ru-I-Di –ism” Interaction Dilemma

Ru-I-Di –ism helps game designers figure out which elements are the best for Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Make Your Own Meta AI What can we do to make the BEST games?

Rule

Interaction Dilemma

Use Ru-I-Di -ism with fellow Game Designers to make your own Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Summary

• Meta AI Practical Model – Emotion-based Meta AI – Visualization for Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. In usual combat scenes…

Player may have Each NPC probably have many parameters many pieces of equipment

Ally NPC(s) Polishing the The game may have battle is quite many levels Enemy NPC(s) difficult! Player Character

The game may have several difficulty settings Blah blah blah……

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Effectors of Meta AI

How should the Meta AI control the elements to make the game more fun?

Ally NPC should heal Enemy NPC should the player’s HP? dodge more? Ally NPC(s) Enemy NPC should get Enemy NPC(s) out of cover? All of them? Player Character Enemy NPC should use power attack more?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Our goal

Tense We want to know Stressed Excited the player’s emotion

Sad Happy

Bored Relaxed And try to move it toward a specific direction Fatigued

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Our goal

How should the Meta AI control the elements to make the game more fun?

??? Tense Enemy NPC should dodge more? Stressed Excited ??? Ally NPC should heal the player’s HP? Sad Happy Ally NPC(s) ??? Enemy NPC Bored Relaxed should get out of cover? Enemy NPC(s) Fatigued Player Character ??? Enemy NPC should use power attack more?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Our goal

How should the Meta AI control the elements to make the game more fun?

??? Tense Enemy NPC should dodge more? Stressed Excited ??? Ally NPC should heal the player’s HP? Sad HappyWhich emotion transition is appropriate Ally NPC(s) for which??? gameEnemy-changing? NPC Bored Relaxed should get out of cover? Enemy NPC(s) Fatigued Player Character ??? Enemy NPC should use power attack more?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Related works: intensity-based Meta AI

E.g., Left 4 Dead and Warframe • Controlling the game by introducing periodic variation on the player's emotional intensity. (i.e. relaxed/stressed emotion) • In general, Meta AI controls the intensity by spawning enemy NPC and adjusting spawning position.

Meta AI (AI Director) on Left 4 Dead [Booth 2009]

[Booth 2009] Booth, M.: Replayable Cooperative Game Design: Left 4 Dead, GDC 2009 (2009)

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Problems of intensity-based Meta AI

1. In many cases, impossible to control the number of NPCs • Games with fixed number of characters or objects are common even now. • Even when non-fixed, game design or level may prevent the Meta AI from spawning NPC at specific times or desired positions.

Shooter game Fighting game Card game

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Problems of intensity-based Meta AI

2. Difficult to deal with the player’s various types of emotions

??? Intensity Tense ??? Stressed Excited Stressed ???

Sad Happy ??? ??? Bored Relaxed Relaxed ??? Fatigued

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Key idea: 2D Emotion Map

• Represent the player’s emotion with a combination of “Hope of Winning” and “Fear of Losing.”

• Inspired by the psychological emotion model. [Watson and Tellegen 1985]

F (Fear of Losing) F (Fear of Losing) 1.0 1.0

Nervous Tense Excited Emotion Point Stressed Alert Elated (EP) H Upset Happy H (Hope of (Hope of -1.0 0 1.0 -1.0 Sad 0 Content 1.0 Winning) Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0 -1.0 [Watson and Tellegen 1985] Watson, D. and Tellegen, A.: Toward a consensual structure of mood, Psychological Bulletin, Vol.98, pp.219-235 (1985)

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H (Hope of Player Character Enemy NPC -1.0 Sad 0 Content 1.0 Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

[Case 1/4] F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H (Hope of Player Enemy NPC -1.0 Sad 0 Content 1.0 Character Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

[Case 2/4] F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H (Hope of Player Enemy NPC -1.0 Sad 0 Content 1.0 Character Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

[Case 3/4] F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H (Hope of Player Enemy NPC -1.0 Sad 0 Content 1.0 Character Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

[Case 4/4] F (Fear of Losing) 1.0

…… …… Nervous Tense Excited Stressed Alert Elated

Upset Happy H (Hope of Player Character Enemy NPC -1.0 Sad 0 Content 1.0 Winning)

Depressed Fatigued Serene Bored Calm Relaxed

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

Various emotions/situations can be expressed by combination of Hope/Fear values

F (Fear of Losing) Intensity 1.0

Nervous Tense Excited Stressed = Lose Stressed Alert Elated

Upset Happy H (Hope of -1.0 Sad 0 Content 1.0 Winning)

Relaxed = Win Depressed Fatigued Serene Bored Calm Relaxed …… ……

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Our goal

How should the Meta AI control the elements to make the game more fun?

??? Tense Enemy NPC should dodge more? Stressed Excited ??? Ally NPC should heal the player’s HP? Sad HappyWhich emotion transition is appropriate Ally NPC(s) for which??? gameEnemy-changing? NPC Bored Relaxed should get out of cover? Enemy NPC(s) Fatigued Player Character ??? Enemy NPC should use power attack more?

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Why 2D Emotion Map?

It can map the Meta AI’s operations to the player’s emotions clearly

Decrease the Fear Increase the Fear Enemy NPC should dodge more? F (Fear of Losing) 1.0 Ally NPC should heal Nervous Tense Excited Stressed Alert Elated the player’s HP? Decrease the Hope Ally NPC(s) Upset Happy H Enemy NPC (Hope of -1.0 Sad 0 Content 1.0 should get Winning) out of cover? Enemy NPC(s) Depressed Fatigued Serene Increase the Hope Player Character Bored Calm Relaxed Enemy NPC should use power attack more? -1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Method overview

F F F

Goal EP Operation World Analyzer H GameH Maker H Next EP Generator Current EP

1. Estimate Current EP 2. Plan Goal EP 3. Update Next EP

Sensors 2D Emotion-based Meta AI Effectors

Gameplay Data Operation to the game world • Action log • NPCs’ behavior/parameters • Status of characters • Level generator • Status of the game world • Object spawner

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Method overview

F F F

Goal EP Operation H H H Next EP Generator Current EP

1. Estimate Current EP 2. Plan Goal EP 3. Update Next EP World Analyzer Game Maker Sensors 2D Emotion-based Meta AI Effectors

Gameplay Data Operation to the game world • Action log • NPCs’ behavior/parameters • Status of characters • Level generator • Status of the game world • Object spawner

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Method overview

F F F

Goal EP Operation H H H Next EP Generator Current EP

1. Estimate Current EP 2. Plan Goal EP 3. Update Next EP World Analyzer Game Maker Sensors 2D Emotion-based Meta AI Effectors

Gameplay Data Operation to the game world • Action log • NPCs’ behavior/parameters • Status of characters • Level generator • Status of the game world • Object spawner

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. What is Current EP?

How much does the player

F (Fear of Losing) feel likely to lose? 1.0

How much does the player Current EP H -1.0 0 1.0 (Hope of feel likely to win? Winning)

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Estimate Current EP in a fighting game

Hit rate: 50% Hit rate: 50%

F (Fear of Losing) 1.0

HP HP

Current EP H -1.0 0 1.0 (Hope of Winning) Player Character Enemy NPC

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Estimate Current EP in a fighting game

Hit rate: 50%60% Hit rate: 50%

F (Fear of Losing) 1.0

HP HP

Current EP H -1.0 0 1.0 (Hope of Winning) Player Enemy NPC Character

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Method overview

F F F

Goal EP Operation H H H Next EP Generator Current EP

1. Estimate Current EP 2. Plan Goal EP 3. Update Next EP World Analyzer Game Maker Sensors 2D Emotion-based Meta AI Effectors

Gameplay Data Operation to the game world • Action log • NPCs’ behavior/parameters • Status of characters • Level generator • Status of the game world • Object spawner

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Plan Goal EP

Highly depends on the game design • In one game project, every 5 seconds the Goal EP is put on the opposite side of the Current E P. • In other words, Meta AI always tries to shake the player’s emotion.

F F F

Goal EP Current EP Current EP

H H H

Current EP Goal EP Goal EP

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Plan Goal EP

Bias the Goal EP • In one game project, we restricted movable area of the Goal EP by the battle progression. • This bias the player's emotion to be more excited or happy at the end of battle. F (Fear of Losing) 1.0 F F F Nervous Tense Excited Current EP Current EP Current EP Stressed Alert Elated

H H H Upset Happy H -1.0 Sad 0 Content 1.0 (Hope of Goal EP Goal EP Goal EP Winning) Depressed Fatigued Serene Bored Calm Relaxed Beginning Climax - 1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Method overview

F F F

Goal EP Operation H H H Next EP Generator Current EP

1. Estimate Current EP 2. Plan Goal EP 3. Update Next EP World Analyzer Game Maker Sensors 2D Emotion-based Meta AI Effectors

Gameplay Data Operation to the game world • Action log • NPCs’ behavior/parameters • Status of characters • Level generator • Status of the game world • Object spawner

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Control a fighting game using Next EP

How easily does the enemy attack the player? How easily does the player attack the enemy? • Frequency of dangerous attacks • Frequency of alert movements • Distance to start attack • Selection rate of slow actions F (Fear of Losing) • Reaction speed 1.0

H -1.0 0 1.0 (Hope of Winning) Player Character Enemy NPC Next EP

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Control a fighting game using Next EP

How easily does the enemy attack the player? How easily does the player attack the enemy? • Frequency of dangerous attacks • Frequency of alert movements • Distance to start attack • Selection rate of slow actions F (Fear of Losing) • Reaction speed 1.0

H -1.0 0 1.0 (Hope of Winning) Player Character Enemy NPC Next EP Become difficult to beat the enemy

-1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Meta AI – Game World Interaction

Element examples of the game world Your game’s Affect to Current EP Controlled by Next EP genre Hope Fear Hope Fear • Hit rate of the player’s attack • Hit rate of the enemy’s attack • Frequency of alert movements • Frequency of dangerous HP of the enemy HP of the player Selection rate of actions with attacks Fighting game • • • large gap • Distance to start attack • Reaction speed

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Meta AI – Game World Interaction

Element examples of the game world Your game’s Affect to Current EP Controlled by Next EP genre Hope Fear Hope Fear • Hit rate of the player’s attack • Hit rate of the enemy’s attack • Frequency of alert movements • Frequency of dangerous HP of the enemy HP of the player Selection rate of actions with attacks Fighting game • • • large gap • Distance to start attack • Reaction speed

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Meta AI – Game World Interaction

Element examples of the game world Your game’s Affect to Current EP Controlled by Next EP genre Hope Fear Hope Fear • Hit rate of the player’s attack • Hit rate of the enemy’s attack • Frequency of alert movements • Frequency of dangerous HP of the enemy HP of the player Selection rate of actions with attacks Fighting game • • • large gap • Distance to start attack • Reaction speed

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Meta AI – Game World Interaction

Element examples of the game world Your game’s Affect to Current EP Controlled by Next EP genre Hope Fear Hope Fear • Hit rate of the player’s attack • Hit rate of the enemy’s attack • Frequency of alert movements • Frequency of dangerous HP of the enemy HP of the player Selection rate of actions with attacks Fighting game • • • large gap • Distance to start attack • Reaction speed • Same as fighting game • Same as fighting game • Same as fighting game • Same as fighting game • How many actions the player • How many actions the enemy • Spawning frequency of • Spawning frequency of trap or Shooter game have to attack the enemy (e.g., have to attack the player (e.g., recovery , weapon or enemy attack type, possessed item, attack type, possessed item, ammo moving route) moving route) • Evaluated score of the player • Evaluated score of the enemy • Drawing probability of cards • Drawing probability of cards Card game • Player’s decision is quick and • Player’s decision is slow and that provide chance to player that provide chance to enemy precise uncertain • Search depth of enemy

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Simple 1 on 1 combat example

Fear Goal EP Enemy NPC Next EP Player Character Hope

Current EP

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Case 1: Next Fear is high – Enemy uses rapid attack

Fear

Hope

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Case 2: Next Fear is low – Enemy uses slow attack

Fear

Hope

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Gameplay analysis tool

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Emotion strength (similarity)

F Actually this EP is not pure “Tense/Alert”

Nervous Tense Excited Stressed Alert Elated

Upset Happy H Sad 0 Content

Depressed Fatigued Serene Bored Calm Relaxed

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Emotion strength (similarity)

F Actually this EP is not pure “Tense/Alert” F Close to 3 types of emotion

Nervous Tense Excited Nervous Tense Excited Stressed Alert Elated Stressed Alert Elated (0.0) (0.8) (0.4)

Upset Happy H H Sad 0 Content 0 Upset Happy Sad Content Depressed Fatigued Serene (0.0) (0.2) Bored Calm Relaxed

Depressed Fatigued Serene Bored Calm Relaxed (0.0) (0.0) (0.0)

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Charts of emotion strength

Excited/Elated Happy/Content Serene/Relaxed Fatigued/Calm Depressed/Bored Upset/Sad Nervous/Stressed Tense/Alert

Time

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Trajectories of EP on 2D Emotion Map

F

Current EP Start End

Follows a little later (Meta AI wants it to do so) H Next EP Start End Goal EP Start End

Follows

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Charts of character status / battle events

We can see the line charts of character status such as HP or MP and discrete game events such as “Hit Attack by Player” through the icons on the bottom of timeline.

Miss Attack by Enemy Hit Attack by Player

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Feedbacks from a real game

Testers (10 people) • Beginners and experts felt the battles overall a bit difficult but fun. The battle concept was achieved! :) • Almost all of them finally clear the battles in 1~3 tries. Game designer • After the parameter tuning of Meta AI, The tool made his work EP and his emotion (voice recorded) became mostly consistent. 100 times better! :) • Gameplay analysis tool: – Good to find reasons behind strong variations of the player’s emotion. – Still difficult to understand accumulative effect of game events on the player’s emotion.

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Two-dimensional future works

Within the 2D Emotion Map, we will be able to deal with the player’s and NPC’s emotion more effectively using well-known Game AI techniques. Point query system (similar to EQS of Unreal Engine 4) for Goal EP Cost map using heat map, rules, F Goal EP or handcrafted data Cost high

Pathfinding for Next EP H Next EP 0

Estimate Current EP using key input patterns / facial expression low Current EP

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

Rule

Interaction Dilemma

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H -1.0 Sad 0 Content 1.0 (Hope of Winning)

Depressed Fatigued Serene Bored Calm Relaxed

1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

F (Fear of Losing) 1.0

Nervous Tense Excited Stressed Alert Elated

Upset Happy H -1.0 Sad 0 Content 1.0 (Hope of Winning)

Depressed Fatigued Serene Bored Calm Relaxed

1.0

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Takeaways

• Meta AI General Design helps changing your game dynamically • Use Ru-I-Di-ism with fellow game designers to make your own Meta AI • Measure and influence the player’s emotion • 2D Emotion-based Meta AI is beneficial for various games • Visualize emotions with gameplay data to polish the Meta AI

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved. Thank you for listening!

Contact • Yuta Mizuno: [email protected] • Daiki Satoi: [email protected] Trademarks • Left 4 Dead is a trademark or registered trademark of Valve Corporation. • Total War: SHOGUN 2 is a trademark or registered trademark of The Creative Assembly Limited. • Warframe is a trademark or registered trademark of Digital Extremes Ltd. • Assassin’s Creed: Origins is a trademark or registered trademark of Ubisoft Entertainment. • Unreal Engine 4 is a trademark or registered trademark of Epic Games, Inc. • All other trademarks are the property of their respective owners.

© 2019 SQUARE ENIX CO., LTD. All Rights Reserved.