DOCSLIB.ORG

Developing a Cinematic Language for Virtual Reality Filmmaking

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Developing a Cinematic Language for Virtual Reality Filmmaking Developing a Cinematic Language for Virtual Reality Filmmaking Author Zhang, Yu Published 2020-11-16 Thesis Type Thesis (PhD Doctorate) School Griffith Film School DOI https://doi.org/10.25904/1912/4021 Copyright Statement The author owns the copyright in this thesis, unless stated otherwise. Downloaded from http://hdl.handle.net/10072/399975 Griffith Research Online https://research-repository.griffith.edu.au Developing a Cinematic Language for Virtual Reality Filmmaking Yu Zhang BA, Art Design BE, Computer Science and Technology MA, Drama and Film & TV Science Griffith Film School Arts, Education and Law Griffith University Submitted in fulfillment of the requirements of the degree of Doctor of Philosophy March 2020 ABSTRACT Many international film festivals, such as those held in Venice, Cannes and Melbourne, have recently established award categories for Cinematic Virtual Reality (CVR) films. However, short videos and animation works still dominate the global CVR film industry and full-length CVR feature films remain largely unsupported. This study aims to contribute to the promotion of the CVR feature film as a viable narrative format. As the focus of this investigation, CVR film represents not only an exciting nexus between computer technology and filmic creativity but also a transformation in the way films are produced and consumed. Filmmakers aspiring to engage with this medium are faced with its significant innovative potential and consequently, with negotiating a new CVR language, production methods and techniques. The main purpose of this research is to contribute to the development of a cinematic language that can be applied to CVR filmmaking. The present study has made much progress to that end, focusing on three aspects of CVR filmmaking: CVR cinematography, cutting and editing and composition and spatial narrative. The outcomes of this study can be applied in the form of a toolbox to assist would-be CVR film directors in mastering the shooting and associated production techniques specific to CVR film production. Employing a Practice-Based Research (PBR) methodology, I have documented the progress of this research from conception to completion via a reflective journal. The main components of the PBR include a study of traditional film techniques, from which a series of CVR previsualisation (previs) experiments were conducted utilising key scenes from selected well-known films for CVR simulation. The results of these experiments were in turn used as the foundation for the second component of this research – the production of an original experimental CVR feature-length film that I directed, titled Calling. The final component of this research is a suite of new creative methods garnered from the previs experiments, culminating in a glossary of 23 new CVR-specific terms and production techniques. i As research on CVR film language and techniques is still emerging, traditional filmmakers are especially motivated to better understand the potential of this novel genre and production mode. Unlike conventional filmmaking cameras, CVR cameras have multiple lenses, which demand different shooting and production methods. Additionally, the CVR space that this form of filmmaking captures breaks the flat picture frame of conventional film. Therefore, traditional filmmakers must transcend their reliance on long- standing principles of “flat film” if they are to leverage the new and innovative storytelling potentials of the CVR medium. One of the major findings of this study is that although traditional film language is able to be partially applied to the production of CVR film, traditional filmmakers need to be strategically mindful of the differences, both large and small, between “flat film” and CVR film. In particular, CVR film directors must be willing to reconceptualise their own roles in the overall filmmaking process as the CVR film is much less director-centered than traditional film and thus more audience-centered. From this reconceptualisation, CVR filmmakers need to shift their narrative-centric mode of thinking to a spatial thinking mode if they are to embrace the full creative possibilities of this emerging filmic medium. ii STATEMENT OF ORIGINALITY This work has not previously been submitted for a degree or diploma in any university. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made in the thesis itself. __ Yu Zhang March 2020 iii TABLE OF CONTENTS ABSTRACT ................................................................................................................ i STATEMENT OF ORIGINALITY .............................................................................. iii LIST OF FIGURES ................................................................................................... vi LIST OF TABLES ................................................................................................... viii ACKNOWLEDGEMENTS......................................................................................... ix CHAPTER 1: INTRODUCTION & BACKGROUND ................................................... 1 Statement of the Problem ............................................................................................. 1 Motivations for Research .............................................................................................. 1 Scope of the Study........................................................................................................ 2 Defining Cinematic Virtual Reality ................................................................................. 3 Cinematic Virtual Reality as a New Audio-visual Medium.............................................. 4 Cinematic Virtual Reality (CVR) as an Emerging Cinematic Language ......................... 5 Evolution of Virtual Reality in Brief ................................................................................ 6 Differences between Conventional 2D and CVR Film Production ................................. 8 Screen differences ........................................................................................................ 8 Significance of the Study............................................................................................. 13 Research Outcomes ................................................................................................... 16 Conclusion .................................................................................................................. 16 CHAPTER 2: CONTEXTUAL REVIEW ................................................................... 17 Introduction ................................................................................................................. 17 Section One: Review of Virtual Reality Films .............................................................. 17 Section Two: Creating the Cinematic Shot .................................................................. 30 Section Three: Editing ................................................................................................. 38 Section Four: Composition and Narrative .................................................................... 44 Conclusion .................................................................................................................. 52 CHAPTER 3: METHODOLOGY - PRACTICE-BASED RESEARCH ....................... 53 Introduction ................................................................................................................. 53 PBR as an Emerging Research Paradigm in Art and Design ...................................... 53 The Research Framework........................................................................................... 56 A Research Methodology Embedded in Cinema Theory and Practice ........................ 57 iv Practical and Analytical Tools ..................................................................................... 59 The Creative Artefact – Making of Calling, Research Questions, Evaluations & Reflections .................................................................................................................. 64 Conclusion .................................................................................................................. 66 CHAPTER 4: FOUNDATION WORK – 3D CVR PREVIsualisation ........................ 67 Introduction ................................................................................................................. 67 Findings & Discussion of CVR Previsualisation Experiments ...................................... 67 Conclusion .................................................................................................................. 96 CHAPTER 5: CALLING, A CASE STUDY – THE MAKING OF A CVR FEATURE- LENGTH FILM ......................................................................................................... 99 Introduction ................................................................................................................. 99 Applied Production Technique 1: CVR Cinematography ............................................. 99 Applied Production Technique 2: Editing in Calling ................................................... 110 Applied Production Technique 3: CVR Long Take .................................................... 118 Applied Production Technique 4: Spatial Composition .............................................. 121 Applied Production Technique
Load more

Recommended publications
  • Advances in Artificial Reality and Tele-Existence
    R. Liang, Z. Pan, A. Cheok, M. Haller, R.W.H. Lau, H. Saito (Eds.) Advances in Artificial Reality and Tele-Existence 16th International Conference on Artificial Reality and Telexistence, ICAT 2006, Hangzhou, China, November 28 - December 1, 2006, Proceedings Series: Information Systems and Applications, incl. Internet/Web, and HCI, Vol. 4282 This book constitutes the refereed proceedings of the 16th International Conference on Artificial Reality and Telexistence, ICAT 2006, held in Hangzhou, China in November/ December 2006. The 138 revised full papers presented were carefully reviewed and selected from a total of 523 submissions. The papers are organized in topical sections on anthropomorphic intelligent robotics, artificial life, augmented reality, mixed reality, distributed and collaborative VR system, haptics, human factors of VR, innovative applications of VR, motion tracking, real time computer simulation, tools and technique for modeling VR systems, ubiquitous/wearable computing, virtual heritage, virtual medicine and health science, virtual reality, as well as VR interaction and navigation techniques. 2006, XLVI, 1350 p. In 2 volumes, not available separately. Printed book Softcover ▶ 159,99 € | £144.00 | $219.00 ▶ *171,19 € (D) | 175,99 € (A) | CHF 213.21 eBook Available from your bookstore or ▶ springer.com/shop MyCopy Printed eBook for just ▶ € | $ 24.99 ▶ springer.com/mycopy Order online at springer.com ▶ or for the Americas call (toll free) 1-800-SPRINGER ▶ or email us at: [email protected]. ▶ For outside the Americas call +49 (0) 6221-345-4301 ▶ or email us at: [email protected]. The first € price and the £ and $ price are net prices, subject to local VAT. Prices indicated with * include VAT for books; the €(D) includes 7% for Germany, the €(A) includes 10% for Austria.
    [Show full text]
  • Evaluation of Virtual Reality Tracking Systems Underwater
    International Conference on Artificial Reality and Telexistence Eurographics Symposium on Virtual Environments (2019) Y. Kakehi and A. Hiyama (Editors) Evaluation of Virtual Reality Tracking Systems Underwater Raphael Costa1,Rongkai Guo2,and John Quarles1 1University of Texas at San Antonio, USA 2Kennesaw State University, USA Abstract The objective of this research is to compare the effectiveness of various virtual reality tracking systems underwater. There have been few works in aquatic virtual reality (VR) - i.e., VR systems that can be used in a real underwater environment. Moreover, the works that have been done have noted limitations on tracking accuracy. Our initial test results suggest that inertial measurement units work well underwater for orientation tracking but a different approach is needed for position tracking. Towards this goal, we have waterproofed and evaluated several consumer tracking systems intended for gaming to determine the most effective approaches. First, we informally tested infrared systems and fiducial marker based systems, which demonstrated significant limitations of optical approaches. Next, we quantitatively compared inertial measurement units (IMU) and a magnetic tracking system both above water (as a baseline) and underwater. By comparing the devices’ rotation data, we have discovered that the magnetic tracking system implemented by the Razer Hydra is approximately as accurate underwater as compared to a phone-based IMU. This suggests that magnetic tracking systems should be further explored as a possibility
    [Show full text]
  • Hiperencuadre/Hiperrelato: Apuntes Para Una Narratologia Del Film Postclásico1
    Hiperencuadre/Hiperrelato: Apuntes para una narratologia del film postclásico1 Jose Antonio Palao Errando Universitat Jaume I de Castelló [email protected] Resumen: El discurso cinematográfico no ha dejado de luchar por adaptarse y competir con las pantallas rivales que le han surgido con la revolución digital, lo cual ha transformado su propia textura. Podemos clasificar estas transformaciones en dos grandes órdenes. Por un lado, la puesta en escena y la propia concepción del raccord han dado lugar a lo que denominaremos hiperencuadre, en el que la pantalla fílmica se ofrece como receptáculo de todas las pantallas con las que rivaliza (ordenador, vídeos de vigilancia, imágines de satélite, etc.). Por otro lado, el cine posclásico opta por la deconstrucción de la linealidad narrativa y el despliegue de distintas tramas y niveles de acción y representación que nos llevan a la noción de hiperrelato. Palabras clave: Hiperencuadre, Hiperrelato, Pantalla, Hipernúcleo, Narratología. Abstract: The film discourse has constantly fought to adapt and struggle with the competitor screens that have arisen with the digital revolution, which has transformed its own texture. We can sort these transformations in two large kinds. On the one hand, the staging and conception of the raccord have given rise to what we'll denominate hyperframe, in which the cinematic screen is being offered as enclosure of all the screens that contends with (computer, video surveillance, satellite images, etc. ). On the other hand, the cinema postclassic chooses the deconstruction of the narrative linearity and the deployment of different plots and levels of action and representation that lead us to the notion of hipernarration.
    [Show full text]
  • Virtual and Augmented Reality Applications in Medicine And
    OPEN ACCESS Freely available online iology: C ys ur h re P n t & R y e Anatomy & Physiology: s m e o a t r a c n h A ISSN: 2161-0940 Current Research Reie Article Virtual and Augmented Reality Applications in Medicine and Surgery- The Fantastic Voyage is here Wayne L Monsky1*, Ryan James2, Stephen S Seslar3 1Division of Interventional Radiology, Department of Radiology, University of Washington Medical Center, Washington, USA; 2Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, Washington, USA; 3Division of Pediatric Cardiology, Department of Pediatrics, Seattle Children's Hospital, University of Washington, Seattle, WA, USA ABSTRACT Virtual Reality (VR), Augmented Reality (AR) and Head Mounted Displays (HMDs) are revolutionizing the way we view and interact with the world, affecting nearly every industry. These technologies are allowing 3D immersive display and understanding of anatomy never before possible. Medical applications are wide-reaching and affect every facet of medical care from learning gross anatomy and surgical technique to patient-specific pre-procedural planning and intra-operative guidance, as well as diagnostic and therapeutic approaches in rehabilitation, pain management, and psychology. The FDA is beginning to approve these approaches for clinical use. In this review article, we summarize the application of VR and AR in clinical medicine. The history, current utility and future applications of these technologies are described. Keywords: Virtual reality; Augmented reality; Surgery; Medicine; Anatomy; Therapy INTRODUCTION DEFINITIONS AND BRIEF HISTORY Recalling the 1966 science fiction film “Fantastic Voyage”, watching The human visual system in amazement as the doctors traveled inside the patient’s body to repair a brain injury.
    [Show full text]
  • Virtual Reality and Virtual Reality System Components Oluleke
    Proceedings of the 2nd International Conference On Systems Engineering and Modeling (ICSEM-13) Virtual Reality and Virtual Reality System Components Oluleke Bamodu1,2, a and Xuming Ye1, b 1 College of Mechanical Engineering, Shenyang University, Shenyang, China 2 Faculty of Computing, Engineering and Technology, Staffordshire University, United Kingdom [email protected], [email protected] Keywords: virtual reality, VR, virtual reality system, hardware, software, VR engine. Abstract. This paper is a synoptic review of virtual reality, its features, types, and virtual reality systems, as well as the elements of the VR system hardware and software, which are essential components of virtual reality systems. Introduction Applications of Virtual Reality (VR) have and continue to increase over the last couple of years. This is partly due to its usefulness in many fields and as a result of the attention given to it by the media. This trend is expected to continue in the future with the advancement of technology in areas like computer graphics, computer vision, controls, image processing and other technology-affiliated components. This paper aims to dissect the nature, the role, the component, the application and applicability of the virtual reality system, as well as, explore briefly the characteristics and use of VR system hardware and software. Definition of Virtual Reality Defining Virtual Reality can prove to be a difficult task because there is no standard definition for it. It is said to be an oxymoron, as it is referred to by some school of thought as “reality that does not exist” [1]. Many different fields have different meaning associated to it, while some people have even misused the term in many cases.
    [Show full text]
  • New Realities Risks in the Virtual World 2
    Emerging Risk Report 2018 Technology New realities Risks in the virtual world 2 Lloyd’s disclaimer About the author This report has been co-produced by Lloyd's and Amelia Kallman is a leading London futurist, speaker, Amelia Kallman for general information purposes only. and author. As an innovation and technology While care has been taken in gathering the data and communicator, Amelia regularly writes, consults, and preparing the report Lloyd's does not make any speaks on the impact of new technologies on the future representations or warranties as to its accuracy or of business and our lives. She is an expert on the completeness and expressly excludes to the maximum emerging risks of The New Realities (VR-AR-MR), and extent permitted by law all those that might otherwise also specialises in the future of retail. be implied. Coming from a theatrical background, Amelia started Lloyd's accepts no responsibility or liability for any loss her tech career by chance in 2013 at a creative or damage of any nature occasioned to any person as a technology agency where she worked her way up to result of acting or refraining from acting as a result of, or become their Global Head of Innovation. She opened, in reliance on, any statement, fact, figure or expression operated and curated innovation lounges in both of opinion or belief contained in this report. This report London and Dubai, working with start-ups and corporate does not constitute advice of any kind. clients to develop connections and future-proof strategies. Today she continues to discover and bring © Lloyd’s 2018 attention to cutting-edge start-ups, regularly curating All rights reserved events for WIRED UK.
    [Show full text]
  • Adam NOWAK Czcionka Times New Roman (TNR) 13
    SILESIAN UNIVERSITY OF TECHNOLOGY PUBLISHING HOUSE SCIENTIFIC PAPERS OF THE SILESIAN UNIVERSITY OF TECHNOLOGY 2019 ORGANISATION AND MANAGEMENT SERIES NO. 134 1 POPULAR STRATEGIES AND METHODS FOR USING AUGMENTED 2 REALITY 3 Dawid PIECHACZEK1*, Ireneusz JÓŹWIAK2 4 1 University of Science and Technology, Wroclaw; [email protected], 5 ORCID: 0000-0001-6670-7568 6 2 University of Science and Technology, Wroclaw; [email protected], 7 ORCID: 0000-0002-2160-7077 8 * Correspondence author 9 Abstract: Augmented reality (AR) is a modern technology which integrates 3D virtual objects 10 into the real environment in real time. It can be used for many purposes, which should improve 11 different processes in daily life. The paper will analyze the areas in which this technology is 12 currently used. First, the history of the development of augmented reality will be recalled. 13 Then, this technology will be compared to virtual reality because these terms are often 14 incorrectly used interchangeably. This paper describes the tools and popular platform solutions 15 related to augmented reality. The most common problems related to the use of this technology 16 will be discussed, including popular approaches concerning optical and video combining 17 methods. The existing applications and their potential in solving everyday problems will be 18 analyzed. Finally, the perspectives for the development of augmented reality and its possibilities 19 in the future will be discussed. This paper provides a starting point for using and learning about 20 augmented reality for everyone. 21 Keywords: augmented reality, graphical elements, image processing, image recognition. 22 1.
    [Show full text]
  • COIMBATORE-641046 DEPARTMENT of COMPUTER APPLICATIONS (Effective from the Academic Year 2019-2020)
    Certificate course in Augmented Reality -2019-20-ud Annexure No: 55F Page 1 of 4 SCAA Dated: 09.05.2019 Annexure – V BHARATHIAR UNIVERSITY:: COIMBATORE-641046 DEPARTMENT OF COMPUTER APPLICATIONS (Effective from the academic Year 2019-2020) Certificate Course in Augmented Reality Eligibility for admission to the course A pass in Higher Secondary Examination (+2) conducted by the Government of Tamil Nadu or an examination accepted as equivalent there to by the syndicate. Duration of the course The candidates can undergo this course both in full-time (3 months) and part-time (6 months). The certificate programme consists of theory and practical courses. Regulations The general Regulations of the Bharathiar University Choice Based Credit System are applicable to this certificate programme. The Medium of Instruction and Examinations The medium of instruction and Examinations shall be in English. Submission of Record Notebooks for Practical Examinations Candidates taking the Practical Examinations should submit bonafide Record Note Books prescribed for the Examinations. Otherwise the candidates will not be permitted to take the Practical Examinations. Revision of Regulations and Curriculum The above Regulation and Scheme of Examinations will be in vogue without any change for a minimum period of three years from the date of approval of the Regulations. The University may revise /amend/ change the Regulations and Scheme of Examinations, if found necessary. Scheme of Examinations Max Course Code Subject and Paper L P Credits Marks Paper I Augmented Reality 4 2 6 150 Certificate course in Augmented Reality -2019-20-ud Annexure No: 55F Page 2 of 4 SCAA Dated: 09.05.2019 Course Title :AUGMENTED REALITY No.
    [Show full text]
  • Resource Materials on the Learning and Teaching of Film This Set of Materials Aims to Develop Senior Secondary Students' Film
    Resource Materials on the Learning and Teaching of Film This set of materials aims to develop senior secondary students’ film analysis skills and provide guidelines on how to approach a film and develop critical responses to it. It covers the fundamentals of film study and is intended for use by Literature in English teachers to introduce film as a new literary genre to beginners. The materials can be used as a learning task in class to introduce basic film concepts and viewing skills to students before engaging them in close textual analysis of the set films. They can also be used as supplementary materials to extend students’ learning beyond the classroom and promote self-directed learning. The materials consist of two parts, each with the Student’s Copy and Teacher’s Notes. The Student’s Copy includes handouts and worksheets for students, while the Teacher’s Notes provides teaching steps and ideas, as well as suggested answers for teachers’ reference. Part 1 provides an overview of film study and introduces students to the fundamentals of film analysis. It includes the following sections: A. Key Aspects of Film Analysis B. Guiding Questions for Film Study C. Learning Activity – Writing a Short Review Part 2 provides opportunities for students to enrich their knowledge of different aspects of film analysis and to apply it in the study of a short film. The short film “My Shoes” has been chosen to illustrate and highlight different areas of cinematography (e.g. the use of music, camera shots, angles and movements, editing techniques). Explanatory notes and viewing activities are provided to improve students’ viewing skills and deepen their understanding of the cinematic techniques.
    [Show full text]
  • Teaching Visual Storytelling for Virtual Production Pipelines Incorporating Motion Capture and Visual Effects
    Teaching Visual Storytelling for virtual production pipelines incorporating Motion Capture and Visual Effects Gregory Bennett∗ Jan Krusey Auckland University of Technology Auckland University of Technology Figure 1: Performance Capture for Visual Storytelling at AUT. Abstract solid theoretical foundation, and could traditionally only be ex- plored through theory and examples in a lecture/lab style context. Film, television and media production are subject to consistent Particularly programs that aim to deliver content in a studio-based change due to ever-evolving technological and economic environ- environment suffer from the complexity and cost-time-constraints ments. Accordingly, tertiary teaching of subject areas such as cin- inherently part of practical inquiry into storytelling through short ema, animation and visual effects require frequent adjustments re- film production or visual effects animation. Further, due to the garding curriculum structure and pedagogy. This paper discusses a structure and length of Film, Visual Effects and Digital Design de- multifaceted, cross-disciplinary approach to teaching Visual Narra- grees, there is normally only time for a single facet of visual nar- tives as part of a Digital Design program. Specifically, pedagogical rative to be addressed, for example a practical camera shoot, or challenges in teaching Visual Storytelling through Motion Capture alternatively a visual effects or animation project. This means that and Visual Effects are addressed, and a new pedagogical frame- comparative exploratory learning is usually out of the question, and work using three different modes of moving image storytelling is students might only take a singular view on technical and creative applied and cited as case studies. Further, ongoing changes in film story development throughout their undergraduate years.
    [Show full text]
  • A Review of Extended Reality (XR) Technologies for Manufacturing Training
    technologies Article A Review of Extended Reality (XR) Technologies for Manufacturing Training Sanika Doolani * , Callen Wessels, Varun Kanal, Christos Sevastopoulos, Ashish Jaiswal and Harish Nambiappan and Fillia Makedon * Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA; [email protected] (C.W.); [email protected] (V.K.); [email protected] (C.S.); [email protected] (A.J.); [email protected] (H.N.) * Correspondence: [email protected] (S.D.); [email protected] (F.M.) Received: 30 October 2020; Accepted: 5 December 2020; Published: 10 December 2020 Abstract: Recently, the use of extended reality (XR) systems has been on the rise, to tackle various domains such as training, education, safety, etc. With the recent advances in augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies and ease of availability of high-end, commercially available hardware, the manufacturing industry has seen a rise in the use of advanced XR technologies to train its workforce. While several research publications exist on applications of XR in manufacturing training, a comprehensive review of recent works and applications is lacking to present a clear progress in using such advance technologies. To this end, we present a review of the current state-of-the-art of use of XR technologies in training personnel in the field of manufacturing. First, we put forth the need of XR in manufacturing. We then present several key application domains where XR is being currently applied, notably in maintenance training and in performing assembly task.
    [Show full text]
  • Video Rewrite: Driving Visual Speech with Audio Christoph Bregler, Michele Covell, Malcolm Slaney Interval Research Corporation
    ACM SIGGRAPH 97 Video Rewrite: Driving Visual Speech with Audio Christoph Bregler, Michele Covell, Malcolm Slaney Interval Research Corporation ABSTRACT Video Rewrite automatically pieces together from old footage a new video that shows an actor mouthing a new utterance. The Video Rewrite uses existing footage to create automatically new results are similar to labor-intensive special effects in Forest video of a person mouthing words that she did not speak in the Gump. These effects are successful because they start from actual original footage. This technique is useful in movie dubbing, for film footage and modify it to match the new speech. Modifying example, where the movie sequence can be modified to sync the and reassembling such footage in a smart way and synchronizing it actors’ lip motions to the new soundtrack. to the new sound track leads to final footage of realistic quality. Video Rewrite automatically labels the phonemes in the train- Video Rewrite uses a similar approach but does not require labor- ing data and in the new audio track. Video Rewrite reorders the intensive interaction. mouth images in the training footage to match the phoneme Our approach allows Video Rewrite to learn from example sequence of the new audio track. When particular phonemes are footage how a person’s face changes during speech. We learn what unavailable in the training footage, Video Rewrite selects the clos- a person’s mouth looks like from a video of that person speaking est approximations. The resulting sequence of mouth images is normally. We capture the dynamics and idiosyncrasies of her artic- stitched into the background footage.
    [Show full text]