Mälardalen University Press Licentiate Theses No. 181 Mälardalen University Press Licentiate Theses No. 181 TOWARDS IMMERSIVE MOTION CAPTURE ACTING DESIGN, EXPLORATION AND DEVELOPMENT OF AN AUGMENTED SYSTEM SOLUTION TOWARDS IMMERSIVE MOTION CAPTURE ACTING DESIGN, EXPLORATION AND DEVELOPMENT OF AN AUGMENTED SYSTEM SOLUTION Daniel Kade Daniel2014 Kade 2014 School of Innovation, Design and Engineering School of Innovation, Design and Engineering Abstract Current and future animations seek for realistic motions to create a perception of authentic and human-like animations. A technology widely used for such purposes is motion capture. Therefore, to create such human-like animations, motion capture actors enrich the movements of digital avatars with realistic and believable motions and emotions. Acting for motion capture, as it is performed today, does not provide a natural acting environment. This is mostly because motion capture actors do not see and feel the virtual environment they act for, while acting. In many cases this can result in unnatural motions such as stiff looking and emotionless movements. To investigate ways to solve this, we first identify the challenges actors are facing as well as concepts to support a motion capture actor. Furthermore, we discussed, how the task of supporting motion capture actors was approached and which factors were discovered to provide support when designing and im- plementing a solution. Initial prototypes have been created to address the men- tioned issues and to find suitable solutions to support and immerse motion cap- ture actors during their performance. For this thesis, one goal was to conduct research by focusing on the question: What are the experiential qualities of immersion in an interactive system to create an immersive acting environment that supports motion capture actors. The developed application provides a flexibility to set up and modify digital assets and scenes quickly and with an easy to use interface. Furthermore, the Copyright © Daniel Kade, 2014 prototype helps to provide an understanding on which hardware and software ISBN 978-91-7485-161-8 prototypes can be designed and used to build an immersive motion capture ISSN 1651-9256 environment. The built prototype allows to investigate user experiences, user Printed by Arkitektkopia, Västerås, Sweden tests and the satisfaction of users and their effects on motion capture acting. i Abstract Current and future animations seek for realistic motions to create a perception of authentic and human-like animations. A technology widely used for such purposes is motion capture. Therefore, to create such human-like animations, motion capture actors enrich the movements of digital avatars with realistic and believable motions and emotions. Acting for motion capture, as it is performed today, does not provide a natural acting environment. This is mostly because motion capture actors do not see and feel the virtual environment they act for, while acting. In many cases this can result in unnatural motions such as stiff looking and emotionless movements. To investigate ways to solve this, we first identify the challenges actors are facing as well as concepts to support a motion capture actor. Furthermore, we discussed, how the task of supporting motion capture actors was approached and which factors were discovered to provide support when designing and im- plementing a solution. Initial prototypes have been created to address the men- tioned issues and to find suitable solutions to support and immerse motion cap- ture actors during their performance. For this thesis, one goal was to conduct research by focusing on the question: What are the experiential qualities of immersion in an interactive system to create an immersive acting environment that supports motion capture actors. The developed application provides a flexibility to set up and modify digital assets and scenes quickly and with an easy to use interface. Furthermore, the prototype helps to provide an understanding on which hardware and software prototypes can be designed and used to build an immersive motion capture environment. The built prototype allows to investigate user experiences, user tests and the satisfaction of users and their effects on motion capture acting. i Swedish Summary / Sammanfattning Inom dagens och framtidens animation efterstravas¨ realistiska rorelser¨ for¨ att ge betraktaren en uppfattning om autentiska och manniskoliknande¨ animeringar. Motion capture anvands¨ i stor utstrackning¨ fr detta andam¨ al.˚ Motion capture- skadespelare˚ ger digitala avatarer realistiska och trovardiga¨ rorelser¨ och kanslor.¨ Dessvarre¨ erbjuder inte nuvarande motion capture-teknik en naturlig skade-˚ spelarmiljo.¨ Det ar¨ framst¨ pa˚ grund av att skadespelare˚ inte ser eller pa˚ annat satt¨ uppfattar den virtuella miljon¨ de agerar i. Det har¨ leder i manga˚ fall till onaturliga rorelser.¨ For¨ att undersoka¨ losningar¨ till detta problem, identifierar vi forst¨ de ut- maningar som skadespelarna˚ stalls¨ infor¨ samt vilka stod¨ som finns for¨ motion capture-skadespelare.˚ Vidare diskuteras hur det ar¨ mojligt¨ att stodja¨ motion capture-skadespelare˚ och vilka faktorer som maste˚ beaktas nar¨ en andam¨ alsenlig˚ losning¨ designas och implementeras. Efter detta skapar vi de forsta¨ prototype- rna for¨ att ta itu med utmaningarna runt motion capture och for¨ att finna lmpliga losningar¨ for¨ att stodja¨ och skapa immersion for¨ motion capture-skadespelare.˚ Ett av avhandlingens mal˚ har varit att fokusera fragan:˚ Vilka upplevelsek- valiteter runt immersion for¨ ett interaktivt system skapar en immersiv miljo¨ som stodjer¨ motion capture skadespelare?˚ Den interaktiva forskningsprototypen ger ett flexibilitet och lattanv¨ ant¨ anvandargr¨ anssnitt¨ for¨ att andra¨ i scener och digitala objekt. Forskningspro- totypen bidrar ocksa˚ till frstaelse˚ for¨ hur hardvara˚ och mjukvara kan utfor- mas och anvandas¨ for¨ att bygga en immersiv motion capture-miljo.¨ Prototypen mojligg¨ or¨ experiment, anvandartester¨ och undersokningar¨ av anvandarupplevelse,¨ anvandartillfredst¨ allelse¨ och dess effekter pa˚ motion capture-skadespeleri.˚ iii Swedish Summary / Sammanfattning Inom dagens och framtidens animation efterstravas¨ realistiska rorelser¨ for¨ att ge betraktaren en uppfattning om autentiska och manniskoliknande¨ animeringar. Motion capture anvands¨ i stor utstrackning¨ fr detta andam¨ al.˚ Motion capture- skadespelare˚ ger digitala avatarer realistiska och trovardiga¨ rorelser¨ och kanslor.¨ Dessvarre¨ erbjuder inte nuvarande motion capture-teknik en naturlig skade-˚ spelarmiljo.¨ Det ar¨ framst¨ pa˚ grund av att skadespelare˚ inte ser eller pa˚ annat satt¨ uppfattar den virtuella miljon¨ de agerar i. Det har¨ leder i manga˚ fall till onaturliga rorelser.¨ For¨ att undersoka¨ losningar¨ till detta problem, identifierar vi forst¨ de ut- maningar som skadespelarna˚ stalls¨ infor¨ samt vilka stod¨ som finns for¨ motion capture-skadespelare.˚ Vidare diskuteras hur det ar¨ mojligt¨ att stodja¨ motion capture-skadespelare˚ och vilka faktorer som maste˚ beaktas nar¨ en andam¨ alsenlig˚ losning¨ designas och implementeras. Efter detta skapar vi de forsta¨ prototype- rna for¨ att ta itu med utmaningarna runt motion capture och for¨ att finna lmpliga losningar¨ for¨ att stodja¨ och skapa immersion for¨ motion capture-skadespelare.˚ Ett av avhandlingens mal˚ har varit att fokusera fragan:˚ Vilka upplevelsek- valiteter runt immersion for¨ ett interaktivt system skapar en immersiv miljo¨ som stodjer¨ motion capture skadespelare?˚ Den interaktiva forskningsprototypen ger ett flexibilitet och lattanv¨ ant¨ anvandargr¨ anssnitt¨ for¨ att andra¨ i scener och digitala objekt. Forskningspro- totypen bidrar ocksa˚ till frstaelse˚ for¨ hur hardvara˚ och mjukvara kan utfor- mas och anvandas¨ for¨ att bygga en immersiv motion capture-miljo.¨ Prototypen mojligg¨ or¨ experiment, anvandartester¨ och undersokningar¨ av anvandarupplevelse,¨ anvandartillfredst¨ allelse¨ och dess effekter pa˚ motion capture-skadespeleri.˚ iii German Summary / Zusammenfassung Aktuelle und zukunftige¨ Animationen streben nach realistischen Bewegungen um eine Wahrnehmung von authentischen und menschenahnliche¨ Animatio- nen zu erstellen. Eine Technologie, die haufig¨ fur¨ solche Zwecke verwendet wird, ist Motion-Capture. Deswegen bereichern Motion Capture Schauspieler die Bewegungen der digitalen Avatare mit realistischen und glaubwurdigen¨ Bewegungen und Emotionen um solche menschenahnlichen¨ Animationen zu erstellen. Schauspiel fur¨ Motion Capture, wie es heute durchgefuhrt¨ wird, bietet keine naturlich¨ wirkende Schauspielumgebung. Dies beruht hauptsachlich¨ da- rauf, dass Motion Capture Schauspieler die virtuelle Umgebung in der sie spie- len nicht wahrend¨ des Schauspiels sehen und fuhlen¨ konnen.¨ In vielen Fallen¨ kann dies dann zu unnaturlichen¨ Bewegungen fuhren.¨ Um Losungswege¨ zu finden, haben wir zunachst¨ die Herausforderungen denen Schauspieler konfrontiert sind, sowie die Bedurfnisse¨ und Anforderun- gen an einen Motion Capture Schauspieler identifiziert. Dann argumentieren wir, wie die Unterstutzung¨ von Motion Capture Schauspielern moglich¨ ist und welche Faktoren bei der Konzeption und Implementierung einer Losung¨ berucksichtigt¨ werden mussen.¨ Danach wurden erste Prototypen erstellt, um die genannten Probleme zu addressieren und geeignete Losungen¨ zu finden um Motion Capture Schauspieler auch wahrend¨ des Schauspiels zu unterstutzen¨ und diese in die Umgebungen des Aktes zu vertiefen. Ein Ziel dieser Licentiate-