44 INTERACTIONS SEPTEMBER–OCTOBER 2018INTERACTIONS INTERACTIONS.ACM.ORG

c r e d i t t k Valentin Schwind, University of Stuttgart Katrin Wolf, Hamburg University of Applied Sciences Niels Henze, University of Regensburg

Avoiding the Valley in Virtual Character Design

Insights have made lost about $150 million. Similarly, →→The uncanny valley is dramatic progress in recent years. computer games such as L.A. Noire a serious challenge They deliver almost perfect or Mass Effect: Andromeda were for designers when renderings that can barely be highly criticized for their human creating realistic virtual distinguished from reality. But characters that don’t look quite right. characters. while improving environmental While the film industry aims to →→Characters and avatars realism significantly increases user create characters for acceptance or must be designed using immersion, increasing the realism of , the domains of games and consistent levels of virtual characters seems to hinder (VR) aim to design realism. their acceptance. Highly realistic characters to foster immersion in the →→Guidelines to avoid computer-generated characters virtual world. Observing unpleasant the uncanny valley in movies, computer games, and virtual characters is likely to disturb are needed to design immersive applications often evoke immersion, as is the off-putting successful products. negative feelings—even disgust. appearance of one’s own virtual body Consequently, such characters can in VR. cause immense financial losses, such Negative reactions to highly

IMAGE BY DESIGN36 / SHUTTERSTOCK.COM/ DESIGN36 BY IMAGE as Disney’s , which realistic characters had already been INTERACTIONS.ACM.ORG C SEPTEMBER–OCTOBER 2018 INTERACTIONS 45 observed in the 1970s by Masahiro inanimate objects [3]. film Alita: Battle Angel, where the Mori, a roboticist who noticed The uncanny valley poses serious character Alita has very realistic humans’ negative response to challenges for human-computer features but unnaturally large eyes or prosthetics that are not quite interaction (HCI) and human- (Figure 2). human-like [1]. Mori developed a interaction (HRI). Improved Avoid “dead eyes.” A virtual graph that describes the non-linear technologies per se are not able to character’s eyes are especially relationship between human likeness address these challenges when artists important. Using eye tracking, and affinity (Figure 1). It suggests have an unlimited number of design we found that users fixate on the that with increasing human likeness, choices but no design guidelines eyes before they consider other an entity becomes more and more to help them avoid the uncanny features in assessing a character accepted by humans, up to a point valley. On the contrary, technologies as real or not real. This supports where it looks almost real. When the that can produce highly realistic findings showing that people rate entity looks almost real, it falls into characters further aggravate the the realism and inconsistencies of what Mori called the uncanny valley; problem, as even subtle designs by human characters mainly based on only when human likeness increases artists can significantly increase the the realism of the eyes. This also further does the entity become uncanny valley’s effect. Therefore, explains why skin makeup does not accepted. we need to think about how it can detract from animacy, in contrast Current research explains the be avoided. What are some ways with atypical eyes or the eyes of a uncanny valley as resulting from out of the valley? And what about dead person. This symptom of “dead conflicting cues in a character’s the acceptance of avatars, when eyes” is responsible for making appearance, causing a perceptual someone’s own body in virtual reality artificial characters feel eerie and mismatch. Humans readily accept is perceived as uncanny? Recent work strange. Subtle features in the eyes unrealistic characters when they aims to provide guidelines to avoid communicate intentions, behavior, are consistently unrealistic, as the eerie effects of virtual characters and well-being. These factors are frequently seen in cartoons. and make them acceptable as real essential in the assessment of and Conflicting cues arise when a humans. In the following, we provide affinity for the depiction. character displays multiple levels an overview of some design guidelines Use stylization and childish of realism at the same time, such as developed to prevent the uncanny features for stylization. Research unnaturally large eyes in a realistic valley. in evolutionary finds face. Debaleena Chattopadhyay Steer clear of atypicalities at that humans respond positively to and Karl F. MacDorman noted that high levels of realism. One of the any aesthetic stimuli experienced computer-generated models of virtual main findings of our work is that as being conducive to survival and characters are inherently inconsistent atypicalities (strong deviations from reproduction. That is why many in their realism. This inconsistency the human norm) for high levels of designs, even if they are not human- is caused by the creation process of realism cause negative sensations like, can be aesthetically pleasing, as a virtual character, as some features in humans and animals [3]. The they may include features that suggest are more difficult to sculpt, texture, negative effects of atypical features, fitness. Participants in our studies and render than others. The outcome such as unnaturally large eyes or and trained artists also often use is unequal levels of realism, which human emotions in realistic animals, childish features to increase affinity make it difficult to assign a category are larger than for characters with in their character designs. Children to the entity, increasing feelings reduced realism. Consistently and infants elicit protective instincts of uncertainty in the observer. rendered realism reduces the negative and feelings of caretaking, which are This uncertainty elicits unpleasant effects of atypicality and increases the origin of social communities and feelings such as eeriness or disgust affinity, as indicated by the first peak mutual understanding. Humans have [2]. Research indicates that the in the uncanny valley. This means these feelings even toward beings not uncanny valley triggers these feelings that realistic renderings and detailed of their kind, which is potentially one in humans to mark a potential threat textures of skin or eyes should not reason why young animals, toys, and or the risk of being infected with a be combined with features that are teddy bears are not in the uncanny transmissible disease. Interestingly, not human-like. Atypical features valley. However, childish features it has been shown that the uncanny at high levels of realism can cause should not be applied to a realistic valley occurs only when looking the uncanny valley. You can see this adult, as they would be atypical. at humans and animals [3,4], not effect in the trailer to the upcoming Features that evoke childhood in a positive way, such as snub noses and round head shapes, can also be deliberately used for stylization and to prevent the uncanny valley. Humans readily accept Use aesthetics and appealing features. Designers can increase a unrealistic characters when they character’s physical attractiveness are consistently unrealistic, to avoid uncanny effects. While creating faces using our faceMaker as frequently seen in cartoons. avatar generator (Figure 3), research

46 INTERACTIONS SEPTEMBER–OCTOBER 2018 INTERACTIONS.ACM.ORG participants chose appealing features for human faces: Symmetry and + smooth skin have been found to be the moving healthy uncanny{ valley most important factors in physical still person attractiveness. Additional desirable humanoid robot bunraku features for virtual faces include puppet healthy and natural skin tones, natural and realistic proportions, clear gender cues, and volumetric and healthy-colored lips. Interestingly, stuffed animal participants did not seek perfection. industrial robot Most of the participants reduced details on the skin, but not

completely. This finding is in line human likeness 50% 100% with previous work showing that familiarity corpse prosthetic hand slight imperfections are still more appealing than perfectly smooth skin or facial symmetries. Regarding zombie skin realism in face design, there is a narrow window in modulating the Figure 1. Mori’s graph of the uncanny valley [1], translated and simplified by MacDorman [7]. desired affinity; on the one hand, The graph shows the uncanny valley and the non-linear relationship between human likeness perfection is not as appealing as slight and perceived affinity to an entity. Mori also assumes that movement, as indicated by the dashed imperfection; on the other hand, too line, increases the effect. many imperfections cause uncanny effects. Balancing the character details can be a difficult challenge for designers aiming to increase the likeability of their characters. However, we can recommend an average face as a foundation for creating faces. It is the ideal base from which to make faces appealing and has the least distance to all human faces, which significantly simplifies customization and individualization for the user or artist. Lean on user creativity. For our research, the avatar generator . faceMaker helps us to determine the preferred characteristics of virtual faces and avoid the unpleasant corporation

effects of the uncanny valley. For Figure 2. Atypical features at high levels of realism can cause the uncanny valley, as shown here f i l m interactive applications in HCI in a still from the upcoming film Alita: Battle Angel. f o x such as games, an avatar creator allows individual customizations c e n t u r y for every user. Our system offers 37 parameters, including the most important features such as gender, t w e n t i e t h skin details, hair color, lip volume,

© 2017 and skin color. Interestingly, if they .

f o x have a choice, users are more likely

to create human-like rather than cartoon-like faces and to bridge c e n t u r y the valley using attractive and individual features. Furthermore, t w e n t i e t h

users put a lot of effort into expressing themselves through the customization of an avatar. Thus, we c o u r t e s y learned that avatar customization i m a g e itself can help users create the 2

avatar they like and identify with, f i g u r e which decreases the uncanny Figure 3. The avatar generator faceMaker.

INTERACTIONS.ACM.ORG SEPTEMBER–OCTOBER 2018 INTERACTIONS 47 AACCMM Confeerreennccee PPrrooceediinnggss NNooww AAvailaabbllee v viiaa PPrriinntt--oon-Deemmaanndd!!

Did you know that you can now order many popular ACM conference proceedings via print-on-demand?

Institutions, libraries and Figure 4. Female participants are not immersed using male hands due to a perceived mismatch individuals can choose between VR hair and musculature and their appearance in real life. from more than 100 titles valley. A positive side effect of AVOIDING THE UNCANNY on a continually updated avatar customization is that it can VALLEY IN VIRTUAL REALITY list through Amazon, Barnes enhance social presence in online In VR, the effects of the uncanny & Noble, Baker & Taylor, environments and increase the valley are much greater than in other replayability of role-playing games. modalities. The fusion of vision Ingram and NACSCORP: Use the uncanny valley effect. and tactile sensations dramatically CHI, KDD, Multimedia, Designers of virtual characters can increases the experience of being use the uncanny valley to deliberately someone or somewhere else. If the SIGIR, SIGCOMM, SIGCSE, elicit eerie effects for villains and user is to feel immersed with their SIGMOD/PODS, ambiguous characters. One example own avatar in VR, multiple aspects and many more. of such a villain is Gollum from Lord should be considered to avoid the of the Rings, whose transformation negative effects of the uncanny from his former human appearance valley: For available titles and was even part of the movie’s plot. IConsider the user’s diversity. His atypical appearance is accepted, Users are very familiar with the ordering info, visit: as it underlines the nature of the characteristics of their own body. If librarians.acm.org/pod character. Villains such as Peter the user’s virtual body is rendered Cushing’s CGI double of Tarkin with a high level of realism, even in or ’ small deviations can cause strong young double in 2 are more discomfort, for example, when controversial, as their virtual clones women use a male body and see a try to resemble the real actor. lot of hair on their arms and hands However, Rogue One was mainly [5] (Figure 4). To enable immersion, criticized not for showing the CGI not only the gender but also skin double of a villain but rather for the color and unique features should short appearance of ’s be considered. Alternatively, less young double of Princess Leia at the realistic, androgynous VR bodies very end of the film. without specific gender cues or

If the user’s virtual body is rendered with a high level of realism, even small deviations can cause strong discomfort.

48 INTERACTIONS SEPTEMBER–OCTOBER 2018 INTERACTIONS.ACM.ORG virtual clothes can help to avoid VR, we found that users with different psychological phenomena. Scientfic unpleasant effects. A camera typing proficiencies benefit from more Reports 2 (2012), 864; https://doi. mounted on a VR headset can also realistic avatars. Inexperienced typists org/10.1038/srep00864 3. Chattopadhyay, D. and MacDorman, K.F. help to render the user’s real body look at their avatar hands to coordinate Familiar faces rendered strange: Why in VR. However, this can reduce their movements above the keyboard. inconsistent realism drives characters immersion due to inconsistent We assume that inexperienced into the uncanny valley. Journal of styles and lighting between the typists need more depth cues than Vision 16, 11 (2016), 7; https://doi. real and virtual worlds. From our experienced typists, who do not refer org/10.1167/16.11.7 experience with avatar generators, to depth cues to localize their hands. 4. Schwind, V., Leicht, K., Jäger, S., Wolf, K., and Henze, N. Is there an uncanny valley customizations of the virtual self Additional surfaces, structures, of virtual animals? A Quantitative and also help to prevent unpleasant shadings, shadows, and lights support qualitative investigation. International effects in VR. The ability of the the binocular disparity and provide Journal of Human-Computer Studies 111C user to customize their appearance more control for localizing their own (2017), 49–61; https://doi.org/10.1016/j. according to their preferences could body movement. Thus, designers and ijhcs.2017.11.003 help them to feel more immersed. developers should consider whether 5. Schwind, V., Knierim, P., Tasci, C., Avoid altered body structures at users can smoothly perform such Franczak, P., Haas, N., and Henze, N. “These are not my hands!”: Effect of Designers of tasks using the virtual hand within high levels of realism. gender on the perception of avatar hands stylized characters often change the their application. Depth cues can in virtual reality. Proc. of the 2017 CHI physical structure of their characters. affect the task performance and must Conference on Human Factors in Computing Cartoonists, for example, simplify be considered when, for example, Systems. ACM, New York, 2017; https:// their drawings and reduce the number designers choose a flat or cartoon style. doi.org/10.1145/3025453.3025602 of fingers. This kind of stylization 6. Schwind, V., Knierim, P., Chuang, was adopted and preserved by CONCLUSION L., and Henze, N. “Where’s Pinky?”: The effects of a reduced number of many video games. We found that a Increased realism and the uncanny fingers in virtual reality. Proc. of the reduced number of fingers does not valley pose serious challenges for 2017 CHI Conference on Computer- affect the feeling of presence with an designers. Mori originally suggested Human Interaction in Play; h t t p s ://d o i . abstract avatar [6]. Users consider that designers should avoid high org/10.1145/3116595.3116596 their avatar to be a useful tool or body levels of human likeness to achieve 7. MacDorman, K.F. Androids as an extension; thus, they accept altered high levels of affinity. Our guidelines experimental apparatus: Why is there an uncanny valley and can we exploit it. Proc. body structures using abstract styles. specify additional methods that of CogSci-2005 Workshop: Toward Social However, designers should not reduce tackle the problems with the uncanny Mechanisms of Science. 2005, the number of fingers at high levels of valley. However, more research is 106–118; https://www.lri.fr/~sebag/ realism in VR, as it causes unpleasant needed in quantifying perception and Slides/uncanny.pdf feelings and has a negative effect on interaction with virtual characters immersion. Some of our participants and avatars. In future research, we Valentin Schwind has a background in I computer science with a focus on interactive felt phantom pain or a strong also should go beyond the uncanny disconnection to their own body and valley: Are there design principles for media. Before starting his Ph.D., he worked as an art and technical director for CGI/VFX were confused by the incongruence virtual characters and avatars that and video games. In July 2018 he of the vision and haptics. Thus, high not only avoid the uncanny valley finalized his Ph.D. thesis, which focuses on levels of human likeness imply a but even increase the effectiveness of virtual characters, the uncanny valley, virtual matching body structure for immersive working or collaborating with other applications, and the exploration of new applications using virtual avatars. avatars in the virtual world? Can design spaces in virtual reality. Users get used to their avatar. avatars and virtual characters help →→[email protected] Users can quickly get used to their us to decrease mental workload and Katrin Wolf received her Ph.D. from TU avatar and the VR experience. In our increase social presence? And we Berlin, where she focused on understanding studies, we observed that participants should ask more generic questions novel interaction techniques for mobile devices. She is currently professor of media stopped complaining about their about how to integrate virtual informatics at the Hamburg University of appearance after only a few minutes. characters as virtual assistants Applied Sciences. Besides tangible and Some users mentioned that after they into our daily life. More research is mobile interaction, she is also interested in got used to a virtual appearance, needed to enable smooth interactions improving the user experience of immersive they felt disturbed when their body between humans and artificial environments. changed again. Thus, we assume that characters. →→[email protected] habituation effects may occur; users Niels Henze did his Ph.D. on mobile learn to interact with their avatar Endnotes interaction with physical objects. He is even if deviations and inconsistencies 1. Mori, M., MacDorman, K.F., and Kageki, currently professor of human-computer interaction at the University of Regensburg. between the avatar and the real N. The uncanny valley. IEEE and Automation Magazine 19, 2 (2012), His research focuses on understanding appearance exist. 98–100; https://doi.org/10.1109/ users, novel interactive systems, and using Consider depth cues. The task MRA.2012.2192811 models to improve interaction with computing performance of users can be affected 2. Moore, R.K. A Bayesian explanation of systems. by their avatar. Investigating typing in the “uncanny valley” effect and related →→[email protected]

DOI: 10.1145/3236673 © 2018 ACM 1072-5520/18/09 $15.00

INTERACTIONS.ACM.ORG SEPTEMBER–OCTOBER 2018 INTERACTIONS 49