A Review of Film Editing Techniques for Digital Games Rémi Ronfard
Total Page：16
File Type：pdf, Size：1020Kb
A Review of Film Editing Techniques for Digital Games Rémi Ronfard To cite this version: Rémi Ronfard. A Review of Film Editing Techniques for Digital Games. Workshop on Intelligent Cinematography and Editing, May 2012, Raleigh, United States. hal-00694444 HAL Id: hal-00694444 https://hal.inria.fr/hal-00694444 Submitted on 4 May 2012 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A Review of Film Editing Techniques for Digital Games Remi Ronfard INRIA, LJK, Université de Grenoble, France [email protected] 1. INTRODUCTION the editor is left with fewer or no options. As a result, the Automated film editing involves the generation of the po- scene may have to be shot again from another angle. This is sition, orientation, motion and selection of virtual cameras usually not a problem because it is easy (and cheap) to do in interactive 3D graphics applications. There is a pressing so. When implementing automated systems, it is important demand for techniques to assist and automate the control of to take the rules of editing into account in the early stages virtual cameras in the computer games industry where the of planning and controlling the camera. Otherwise, a lot of rapid development of personal computers and high perfor- effort will be wasted on attempting to edit shots that "do mance consoles has led to substantial improvements in the not cut together". This will be examined in depth in Section visual fidelity of games. The goal of this survey is to charac- 3. terize the spectrum of applications that require automated film editing, present a summary of state-of-the-art models In traditional cinematography, cutting can be taken into ac- and techniques, and identify both promising avenues and count by following one of several working practises. We men- hot topics for future research tion three of them. 2. CINEMATOGRAPHY AND EDITING 1. Cutting in the head means that the director has al- One fundamental part of cinematography, as outlined in ready decided very precisely every single shot, usually Maschielli's 5C's of cinematography [14] is to provide shots in the form of a storyboard. In that case, it suffices to that can easily be edited together. In the early days of cin- shoot each action or beat in the screenplay from a sin- ema, the interplay between cinematography and editing was gle viewpoint. Textbooks in film-making warn against a matter of trial and error. As noted by Barry Salt [20], it the dangers of the method because it cannot recover took several years before cinematographers and editors un- easily from errors in planning. derstood the "exit left enter right" editing rule. Before that, the rule was usually obeyed because it appeared to work bet- This approach is very suitable for real-time applica- ter in most cases. But the "wrong" solution was still used tions. It consists in planning the editing first, resulting from time to time. When it finally became clear what the in a list of shots that can then be rendered exactly as "right" solution was, cinematographers stopped shooting the planned following the timeline of the final movie. alternate solution because they knew it was useless. After One drawback of that approach is that the animation more than a century of cinema, good professional cinematog- itself cannot always be predicted in all its actual de- raphers have thus "internalized" the rules of editing in such tails. As a result, it may be difficult to plan exactly a way that they can avoid shots that will not cut together. when to cut from shot to shot. In games, we are probably still at an earlier stage because it 2. Three-take technique A common variant of "cutting is not yet quite clear how the rules of cinematography should in the head" consists in shooting a little more of the translate for an interactive game, which is a very different action from each planned camera position. As a result, situation from a movie. each action is shot from three camera positions - one according to the shot list, one from the immediately In computer graphics, the camera is controlled by animators. previous viewpoint and one from the next viewpoint. A good professional animator should have a similar sense of This has the advantage that the exact cutting point which shots will cut together. When this is not the case, can be resolved at a later stage. 3. Master-shot technique Another common practice consists in planning all the camera works for shooting the scene in one continuous take - the "master shot" - and then adding shots of various sizes to show the details of the action in various sizes (close-ups and medium shots). Editing can then more carefully pre- pared by insuring that all those shots will cut nicely with the master shot, resulting in a typical sequence of "Master-Closeup-Master-Closeup", etc. Note that those techniques are very useful in practice be- preventing editing errors of all orders. But that is of course cause they are more general than "film idioms" where the not the entire story because there are still infinitely many camera positions are prescribed once and for all. "correct" camera pairs that can be cut together at any given time. A second part of automated editing is therefore to 3. AUTOMATIC CAMERA EDITING evaluate when to cut to which shot. This section covers the approaches that draw on a theory of film editing for planning and performing camera place- The classical Hollywood concept of editing [14] recommends ment and composition. Here scenes are described in terms that successive shots should minimize perceptually disrup- of actions and communicative goals that must be translated tive transitions. The modern viewpoint [9] stresses the con- into successive shots. Cutting between cameras adds con- sistency of the narrative structure which overrule disturbing siderable freedom in the focalization and order of presenta- transitions, as attention will primarily be directed to grasp- tion of the visual material. Cutting between cameras also ing the succession of significant events in the story. A good introduces constraints. We review the most important con- computational theory of film editing should probably stand straints and corresponding rules (180 degree rule, 60 degree in the middleground between those two viewpoints. On the rule) and explain how they can be expressed and solved al- one hand, it is difficult to get a good model of "perceptu- gorithmically. Then, we review the principles that can be ally disruptive transitions". At best, a computational model used to evaluate the quality of a shot sequences and the al- may be expected to avoid the most obvious mistakes, still gorithmic strategies that can be used to solve for the best leaving a large number of possibilities. On the other hand, sequence. Finally, we review the strengths and limitations the narrative structure of an animated scene may not always for some of the existing systems proposed for real-time, live- be easily uncovered, again leaving multiple choices. editing [He96,Funge98] as well as offline, post-production editing [Elson07] and sketch promising future directions for Few editors have written about their art with more depth research in this area. than Walter Murch [16]. In his book, he introduces a Rule of Six with six layers of increasing complexity and importance Automatic film editing has a long history, dating back at in the choice of how and when to cut between shots: least to Gilles Bloch's PhD thesis in 1986 [1]. In this sec- tion, we present both procedural and declarative approaches. Three-dimensional space of action. Respect of 3-D con- A procedural approach to movie editing builds an explicit tinuity in the real world: where people are in the room solution. A good example of that is the Virtual Cinematog- and their relations to each other (accounts for only 4 rapher system (VC) where each idiom is implemented as % of what makes a good cut) finite state machine. A reactive approach is essentially a procedural approach where multiple courses of events can Two-dimensional space of screen. Respect of 2D conti- be taken into account. A declarative approach states the nuity. Where people appear on the screen. Where the constraints and rules and lets a separate solver find a so- lines of action, look, movement project on the screen. lution that meets all the constraints, and/or maximizes a (5 %) measure of quality. Eye-trace. Respect of the audience's focus of interest be- 3.1 Editing rules and constraints fore and after the cut. (7 %) It is important to understand the motivation between the Rhythm. Cut at a moment which is both right and inter- so-called "rules of editing". Most of them are in fact con- esting. (10 %) straints. What that means is that it may not be possible to cut from any two arbitrary cameras. Why not ? Because Story. Cut in a way that advances the story. (23 %) some transitions may provoke false inferences. For a cut be- tween two shots to work, it is fundamental that it does not Emotion. Cut in a way that is true to the emotion of the break the logic of human perception.