Undergraduate Thesis Prospectus Non-photorealistic Ray Tracing with Paint and Toon Styles (technical research project in Computer Science) Accessibility for the Marginalized: Unleashing the Amateur Potential of Video Games (sociotechnical research project) by Nicholas Moon May 9, 2020 technical project collaborators: Megan Reddy On my honor as a University student, I have neither given nor received unauthorized aid on this assignment as defined by the Honor Guidelines for Thesis-Related Assignments. Nicholas Moon Technical advisor: Luther Tychonievich, Department of Computer Science STS advisor: Peter Norton, Department of Engineering and Society General Research Problem How have the evolution of computer graphics and development tools provided new opportunities for artistic expression through digital media? Since the late 20th century, computers have offered artists both new ways to use existing artistic media, and entirely new media. Digital art is easily shareable through social media, digital download, and streaming services; it is also more accessible for proper or improper reuse and manipulation. Through affordable and user-friendly software, such media have become more accessible to more artists. Games such as Dsy4ia (Anthropy, 2012) and Depression Quest (Quinn, 2013), and films such as I Lost My Body (Xilam Animation, 2019), serve as new forms of artistic venue. Non-photorealistic Ray Tracing with Paint and Toon Styles How can NPR rendering methods be integrated into a path-tracing 3D renderer to produce a style like Studio Ghibli’s? Path-tracing is a rendering architecture for generating images of 3D scenes that enables realistic light phenomena. Often, physically based rendering (PBR) is the goal of path-tracing, and is used most in architectural visualization, visual effects, and animation. Non-photorealistic rendering is a technique of eschewing physical accuracy to produce imagery emulating styles and techniques present in physical art. Most current NPR implementations don’t incorporate physical light simulation in order to reduce computation, but this comes at the cost of reflection, refraction, and other light properties (Du & Akleman, 2016). This research project focuses on working within the path-tracing architecture to render in a 2D art-style, specifically one that mimics Studio Ghibli movies (fig. 1). The goal is to enable background and foreground 1 rendering with distinctive stylistic qualities, incorporating realistic light transport. My advisor is Luther Tychonievich in the Computer Science department. This is a capstone research project, and my partner is Megan Reddy. Figure 1. Frame from Howl’s Moving Castle, a Studio Ghibli film. (Studio Ghibli, 2004). Traditionally, NPR has seen usage in real-time or post-processing areas, such as cel- shading in The Legend of Zelda: The Wind Waker (fig. 2) or cross hatching (fig. 3). However, these methods don’t allow for more complex light transport phenomena, and don’t provide the artist with the powerful and customizable toolkits seen in modern rendering material sets (Du and Akleman, 2016). Figure 3. Real-Time Cel-Shading in The Legend of Zelda: The Wind Waker (Nintendo, 2004). 2 Figure 3. Post-Processing Cross-Hatching applied to render of skull (Du and Akleman, 2017). Alternatively, there’s recently been an incorporation of NPR elements into traditional physically based renderers, usually using a combination of material shaders and filtering algorithms. For example, Arnold, a renderer packaged with the popular Autodesk Maya 3D modelling and animation software, added this functionality in 2018 (Autodesk, 2018). While this has allowed for ray-traced interactions, these implementations are often the least supported, the most uncooperative with other functions of the renderer, and the most inaccessible for artists to use because of both additional required programming and less out-of-the-box functionality or parameterization. In order to accomplish the goal of NPR path-tracing, a prototyping-first method is being taken to conduct research and implementation. Specifically, student developed rasterizing and ray-tracing rendering engines are being developed and extended as a playground for adapting previous NPR work and generating new techniques. The layered nature of traditional 2D animation has also enabled parallelism in algorithm development, with the cel-shaded foreground rendering being researched concurrently with the painterly backgrounds. 3 The final deliverable has not been fully decided. Currently, the plan is to have the final implementation of the NPR environment be incorporated into the open-source bidirectional path- tracer LuxCoreRender (2018), but building off the basic path-tracer being created as a final project for CS 4810 is also an option because of its cleaner and malleable nature. This path-tracer will generate images from 3D scenes that blend the different art styles present in the foreground and background of Studio Ghibli movies. Ideally, this renderer will also be adaptable to different art-styles as well, affording freedom in the domain of NPR to 3D artists and filmmakers. Accessibility for the Marginalized: Unleashing the Amateur Potential of Video Games How have amateur and minority videographers taken advantage of the growing accessibility of video game and animation tools to introduce innovative creative fiction that challenges the historically white, male, heterosexual, and cisgender perspective? Computer art tools are more accessible than ever. Applications once exclusive to computers at Pixar can now be freely downloaded, and game development tools once programmed in assembly or bought as middleware are now useable by artists with no special technical expertise. About 244 million Americans play digital games, ballooning with adoption of smart phones (NPD, 2020). However, among employees of U.S. video game companies in 2019, only one quarter were women; people of color are also underrepresented (IGDA, 2019). Nevertheless, women artists and artists of color have used digital tools to produce innovative media for video games, much of which challenges the predominant conventions in games. Researchers have studied similar innovations among amateur artists who use phone cameras. Schleser, Wilson, and Keep (2013) found that mobile moviemaking has widened the diversity of voices and methods in film, particularly in Korea. Anyone with a touchscreen device 4 now has access to high-definition recording. In South Korea, mobile film festivals have promoted segyehwa, or internationalization. Such events are recognitions of the cultural influence of media, a recognition also evident among LGBT, black, and women game creators who use video games to normalize their experiences. According to Stöckel and Pettersson (2016), accessible applications have enabled indie Steam games. Developers of games that represent marginalized perspectives have used accessible digital tools to evade the homogeneity that the commercial game market has favored. Sens (2015) observes that with artist-friendly tools such as Twine, independent developers could host game jams in queer spaces. To Harvey (2014), Twine is a harbinger of a democratization in game development that can accommodate creators such as Anna Anthropy, and the revolution she espouses. The LGBTQIA+ animation and game community is represented, for example, by GaymerX, an advocacy that “provides an opportunity for marginalized individuals to attend GDC who could not otherwise afford the experience.” It finds funding to support queer developers (GaymerX, 2020). Representatives of black game developers include the Black In Gaming SIG, which offers “game design techniques, education forums and interactive workshops” to black amateur developers (Black In Gaming, 2020). Women in Games “seeks a games industry, culture and community free of gender discrimination, where full equality of opportunity, treatment and conditions empowers all women to achieve their full potential.” It offers mentorship, hosts game jams, and advises students and amateur developers (Women In Games, 2020). Free and open-source software companies are engaged too. Unity contends “the world is a better place with more creators in it,” and that “creativity can and should come from anyone, 5 anywhere” (2020). Game vectors include for-profit storefronts and free content hosting platforms. The platform Interactive Fiction Database is “a collaborative, community project – it’s a little like a Wiki for IF.” It hosts games made in Twine and similar tools (IFDB, 2020). A loose collection of internet activists associated with the GamerGate movement resist the democratization of game development. Organizing in online forums such as 4Chan or r/KotakuInAction, these gamers have generally been hostile to women and minorities in game development, especially when the games recognize marginalized groups’ experiences, such as gender transition, mental illness, and immigration (Parkin, 2014). References Anthropy, A. (2012) Dys4ia [Video game]. Glenside, PA: Newgrounds. Autodesk, Inc. (2018, April 4). Introducing Arnold 5.1 [Press release]. https://www.arnoldrenderer.com/news/arnold-5-1/ Du, Y. & Akleman, E. (2017). Designing Look-And-Feel Using Generalized Crosshatching. Proceedings of SIGGRAPH ’17 Talks. ACM Digital Library. Du, Y. & Akleman, E. (2016). Charcoal rendering and shading with reflections. In ACM SIGGRAPH 2016 Posters (SIGGRAPH '16), 32, 1–2. ACM Digital Library. GaymerX. (n.d.) GaymerX GDC Scholarship. https://gaymerx.org/gaymerx-gdc-scholarship