Computation Arts Julieta Gueorguieva Concordia University id: 6378420 CART434: Advanced 3D Studio

Secluded Final Project Documentation & Class Presentation

Abstract

Secluded is a Game Project in the Horror/Suspense genre that uses biometrics as his primary way of navigation and input. It works with the Microsoft Sensor, attempting to eliminate handheld controllers and the Pulse Sensor Amped in conjunction with the Arduino microcontroller, to track Heart Rate and use it to alter the game environment and conditions. The goal is to use these input systems to enhance the game experience and immerse the player further into the world of the game, in addition to personalizing the experience through biometric data. The story of the game is about a character that is trapped in a room that seems to loop back in itself and constantly changing, scarring and confusing him. After playing the game both with and without the biofeedback system, I can say that playing with the system makes it feel more real and exciting and also adds a level of difficulty as the player has to get used to the controls.

1 Index

● Abstract p.1 ● Project Description p.3 ● Context p.3 ○ Nevermind p.3 ○ Silent Hill p.3 ○ Rise of Nightmares p.4 ● Pre­Production Summary p.4 ○ Kinect and Arduino in Unity p.4 ○ Programming controls in Unity p.5 ● Production Steps p.6 ○ Week 1 ■ The Game Concept p.6 ■ Level Design p.7 ■ Concept Art p.9 ■ Finalize Controls and Mechanics p.10 ○ Week 2 ■ Modelling Proxies p.11 ■ Level Setup p.12 ○ Week 3 ■ Make Cutscenes p.13 ■ Hud p.13 ○ Week 4 ■ Playtest p.14 ■ Polish Graphics p.14 ■ Lights p.16 ■ Sound p.16 ● Conclusion p.17 ● References p.18

2

Project description This project is about using biometric data to enhance user experience during game play. The goal is to get a character control set up using Kinect and Heart rate data input to influence the state of the Console. The Project targets gaming and more precisely the horror game genre. In the future, I see this project developing into a full and polished horror game where the player needs to control his emotional state to play the game properly and have a full user experience.

The Result is a Game where the player wakes up in a bed in a small room not knowing who he is, where he is and how he got there. The room has two doors but neither lead to an exit, but rather bring him back to the same room. Only things keep changing, appearing and disappearing and the room itself transforms into a nightmare, wrecked and covered in blood as if in some other parallel dimension. As the player tries to escape from this irrational place, he gets bits and pieces of flashbacks that will make sense of who he is, where he is and why at the end.

Context

The gaming industry has been changed a lot by the current technological advancements. Graphics, processing power, control innovations and lately Biometrics. Games seek to be more immersive and more personalized to each player seeking to create a unique experience. Based on the success of the Microsoft Kinect Sensor I believe that the future holds near reality simulations where the boundary between game and real emotions and experience is blurred. This belief motivates me to research the subject of biometrics applied to games and contribute to the direction that I believe the future of games is going in. Here are some inspiration examples of other games related to my Project:

Nevermind “Nevermind is – at its core – an adventure game (in the spirit of classic games like Myst), where you must explore strange worlds and solve puzzles to unlock the terrifying mystery that lurks within each patient’s inner psyche”1 The context of the game is quite interesting and in addition to that it uses biofeedback to detect the level of fear of the player and make the game harder as the player gets more scared and thus making it a goal for the player to keep his calm, a sort of a journey in which he need to learn to be in control of his emotions. This is my main inspiration as i believe that in the future biofeedback will become part of the everyday gaming experience. in fact, Kinect uses biometrics to detect player movement so in a way it has already been present in the gaming industry.

3 Silent Hill 4: The Room “Silent Hill 4: The Room is the fourth installment in the Silent Hill psychological horror series, published by Konami and developed byTeam Silent, a production group within Konami Computer Entertainment Tokyo [...] Unlike the previous installments, which were set primarily in the town of Silent Hill, this game is set in the fictional town of South Ashfield, and follows Henry Townshend as he attempts to escape from his locked­down apartment.” 2 Since my childhood I have always been interested in horror games and movies. Silent Hill is one of my favorite game series. The Room was an inspiration for this project as it helped me envision a game that could be made in a short period of time due to the fact that it is constrained in a small room. A room could be the safest place in the world and yet the scariest if it is compromised and there is no escape.

Rise of Nightmares “Rise Of Nightmares is a developed and published by for the . The game was revealed at the 2010 Tokyo Game Show, and is designed specifically for Kinect. It is the first M­rated Kinect game released.” 3 This is a game that I have not had the chance to play, but it is the first example that I have seen of a horror game designed for the kinect. It made me wonder of what great immersive experience it would be if it is paired with the arduino microcontroller. It gave me an idea how to approach my controls and navigation in the game.

Summarize the pre­production Research & Development stage

Can Kinect and Arduino function properly with Unity simultaneously?

First I had to get the Kinect Working and after extensive reading and research i found the Center for Simulations and Virtual Environment Research 4 article that walked me through connecting my Sensor to Unity3D. Then, I wanted to make sure that Arduino and Kinect can be running simultaneously on Unity, as I was afraid that there might be some serial communication interference. Using the Uniduino Library 5 I had a blinking light and a moving

4 skeleton driven by the kinect working at the same time. Later, I decided to omit the Uniduino Plug­in and write a script to do the serial communication, as it was more suitable for working with the Pulse Sensor. I read some forums 6, watched some videos 7 and read the sensor’s code walkthrough 8 to be able to write a code that will suit my purposes. That worked well and i was able to get my heart rate.

2. Will I be able to program the kinect so that it controls the game the way i want it? Forward and Backward

Walking I used a trigger placed in front of the skeleton provided with the sample kinect files from the kinect wrapper for unity provided in the Kinect tutorial mentioned above. I placed colliders in the feet that moved when the player moved. When there was a right_leg collision, that meant that the right foot was extended forward and I made my character go forward. When the same foot was retracted towards the back, that made the left_foot collider go into the trigger as the torso of the skeleton moves forward to be in the center of the skeleton object at all times.

5 Detail the Production steps of the final project.

Week 1 (2­8)

The Game Concept

I had a general idea of the feel of my game but not a clear idea of what it would be like. I wanted a small environment that would require less assets, so that I could concentrate on the Kinect, the Pulse Sensor, the programming in unity and the quality of the assets and my final product. Inspired by Silent Hill series and especially the 4th one “The Room“ I decided that having everything happening in the same room, would require few assets, be scary as it will give the player the feeling of being trapped and would require some smart level design.To integrate the pulse data, I decided that after a certain time, just like in the Silent Hill MOVIES (not the games) the room will turn all crazy on the player, and the calmer the player is the more time he will have before that occures.

6 Level design map In my case, level design meant making the puzzle and the story of the game. On a piece of paper, I drafted the levels and their solutions and what that meant to the story. After i did my proxies, and basic concept art, I rendered a series of images, illustrating the state of the room and made a storyboard that Illustrated the story, the puzzles and their solutions, the logic of the navigation and the objects that i needed to model.

level 1: establishing the logic of the rooms, which door leads where.

7 During that process I also did some nCloth exploration and testing for my real assets. I needed to get make a realistic bedsheet and a broken mirror, and be able to import them into unity.

8 Concept art I designed how the room should look. Two doors one in front of each other to confuse the player about which end of the room he is at. The bed is there as a reference to be able to situate yourself, are you to the left, to the right of it. Weather it has a sheet or not. It is like a landmark to be able to orient yourself.

9 Finalize controls and mechanics

This part was never fully done up until the very end of the project as while I was programming the game there were issues that i had to keep resolving and perfecting my navigation system.

Rotation I added a rotation_control object (just a sphere with its z direction pointing as if it is an eyesight) in the head of the skeleton which rotated left and right when shoulders were rotated and up and down when head was tilted. At first I was disappointed that head left and right rotation was not detected and I had to use the shoulders, but then it worked well as the player does not need to struggle between rotating and looking away from the screen while playing, but rather rotating his shoulders and keeping his gaze towards the screen. Then i added four objects outside the skeleton and added a script to the rotation_control object that calculated the angle between the direction it was facing and the place of the four objects. When head is rotated, it detects if it is rotating up or down and the shoulders right or left and send the information to a camera object which in turn increments its rotation in the proper direction.

Interaction It is the same principle as the walking. There is a object with a collider in the hand in the skeleton. There is a trigger object in front of the skeleton (the green spherical object). when there is a collision, a message is sent that triggers interactions.

10 Week 2 (9­15)

Modeling / Proxies

I modelled all the proxies for the objects i needed in the game so that I can start programming and worry about polishing it later.

1. Room ○ doors ○ walls ○ ceiling ○ floor

2. Bed 3. Sheet 4. Mirror 5. Shattered mirror 6. Knife 7. Painting 8. Destroyed painting 9. Doors 10. Music sheet 11. Piano

11

Scene/ level set up I used the proxies to set the whole thing in unity. i created trigger areas, spon points and all object prefabs so that I can change the graphics, but the dimensions and placement and scripts attached will remain.

12 Week 3 (16­22)

Make cutscenes I made renders in maya that i used as cutscenes explaining what should happen in the cutscene as if it were a movie.

I used Photoshop to get the text, because It was easier than figuring out and scripting the guiText in unity.

Include cut scenes I added the cutscenes to the game as separrate levels. made them sprites and pointed the camera towards them.

Hud The hud I made it simple, just a heart and a bar. I had to learn how to work with gui objects such as guiText, guiBox, guiContent. I used guiGroups to organize the interface the way i wanted and to make the bar decrease and increase.

13 Week 4(23­29)

Playtest During playtesting I realized that navigation is hard. I was used to it, because i knew how it worked and how should I stand and how to step forwards or backwards. however, other people who tried were very clumsy at it as they merely did what was intuitive (ex: just putting one foot in front instead of leaning forward as well.) This means that the Kinect system needs improvments. Also, depending on the height of the person, the head tilting was different. For example, I am 1m 54 cm tall and if I set the angles in a way that it works for me, someone that is taller, finds in harder to look down. Also, the pulse sensor required a glove to be more stable. Some people were very confused, others caught the logic quickly.

Polish / Better Graphics Even if my game was functional and playable, I was not happy with the look of it and the mood that it was setting. I spent the last week working on proper models to replace the proxies with. I did all the modelling in Maya and did all the materials using the mental ray material_x material to get the highest quality possible with minimal render time for all my cutscenes. However, I had an issue exporting it to fbx. I then created other materials that could work in Unity. So i ended up having high quality renders and a bit lower quality for real time graphics. I used creative commons images for my textures.

Here are some of the Cutscenes. the rest of them are in Building assets/ actual assets/new cutscenes

14 15

Lights

Because my game lacked any special effects, I decided to go the old fashioned way and use light to make it scary. Having low light, limited vision and surrounded by darkness in the unknown helped me make my game a bit more scary and mysterious. Here is a screenshot of my lights in one of the levels. the downfall of this is that It is hard to see all the work I put on making those assets look good. However, they still show in my cutscenes, so not all work is wasted.

Sound

I wanted to have sound, a constant soundtrack, some piano clips as well. I wanted some glass shattering sound effects, door opening, textile ripping and so on. The sound made the game look less static. I sourced my sounds from the creative commons on www.soundCloud.com and from www.freesound.org.

16 Conclusions

Overall Project I find that overall my project was successful as it achieved the goals that it had set. It proved that Kinect and Arduino can work simultaneously in unity, created an environment that this can be applied to and made my game more exciting to play. Also, I managed to achieve my goal of making a functional and playable game and even got to make proper assets, cutscenes and sound effects. I also learned a few new things about unity, lights, nClothes, file compatibilities, rendering, materials and lots of programming.

What did not work There are two issues about the game.

The first one is that it is hard to control your movement with the kinect. Due to the existence of professional games that do not have that problem, i think it is a matter of improvement. I managed to learn the skills to prove my concept, but i did not have enough programming skills to make it work smoothly. I programmed the game and after playing it i was good at it. But the sensor did not respond in the same way if someone was taller than me for example. Also, the manner in which I programmed the controls, required precise movement and was dysfunctional if the player will do something similar but not quite what it had to be. This is not a dancing game so precision is not required and liberty of movement has to be accommodated. I believe that this can be fixed with more research and improvements and also working in collaboration with someone who knows more about the kinect sensor than i do.

The second thing is the Pulse sensor. Without all the graphics it worked fine, but once all was in place, it was making the game glitch. Perhaps, I must look for another way to treat my serial communication and also need a more powerful processor to run the game on. Also, the sensor itself might not be the most accurate one. Perhaps it could be replaced with something that works better, or even the arduino microcontroller could be replaced with better microcontroller.

Other than that, the game has minor bugs and unfinished business. The textures and uvs are not optimised to reduce texture size, the cutscenes, into, conclusion are not finished. I imagined them more as movie sequences rather than still images. Also the sound could be a lot better, better and more sound effects to make the game look less static. Some special effects and animations would be nice as well. Most importantly, the story of the game does not communicate very well through the game and cutscenes. Also I am not sure how pertinent the story is in itself.

17 I think that due to the technology, the genre and the concept of the game it has future into further polishing and development. I would see it in an installation environment with surround sound systems, 3d projections and other special effects to make it feel more real.

Reference and bibliography

1 ­ http://www.nevermindgame.com/about/ 2 ­ http://en.wikipedia.org/wiki/Silent_Hill_4:_The_Room 3 ­ http://riseofnightmares.wikia.com/wiki/Rise_of_Nightmares 4 ­ http://www.uniduino.com/ 5 ­ http://ucsim.uc.edu/blog/2014/02/how­to­use­microsoft­kinect­with­unity/ 6 ­ http://forum.arduino.cc/index.php/topic,40001.0.html 7 ­ http://forum.arduino.cc/index.php/topic,40001.0.html 8 ­ http://pulsesensor.myshopify.com/pages/pulse­sensor­amped­arduino­v1dot1 sound effects www.freesound.org Final piano ­ http://www.freesound.org/people/Endersniper123/sounds/215767/ The rest of the sounds are also taken from this website.

soundcloud.com Other piano ­ https://soundcloud.com/search?q=Chilleminor%20Piano

18