A Cognitive Approach to the Perception of Digital

THE NEGOTIATION OF REALITY:

A COGNITIVE APPROACH TO THE PERCEPTION

OF DIGITAL HOLLYWOOD CINEMA

By

Georgios K. Dimitriadis

A dissertation submitted to the Department of Translation and Intercultural Studies,

School of English Language and Literature,

Faculty of Philosophy – Aristotle University of Thessaloniki, Greece

In fulfillment of the requirements for the degree of Doctor of Philosophy

2014 ii

TABLE OF CONTENTS

TABLE OF CONTENTS ...... II ACKNOWLEDGMENTS ...... V ABSTRACT ...... VII NOTES ON TERMINOLOGY ...... IX

CHAPTER 1 INTRODUCTION – A NEW THEORY FOR AN OLD ART? ...... 1

1.1. THE CHALLENGES TOWARDS A THEORY OF DIGITAL CINEMATIC REALITIES ...... 3 1.2. THE RISE OF COGNITIVISM ...... 7 1.3. PROBLEMS AND SOLUTIONS: AN OUTLINE TO THE PRESENT STUDY ...... 15 1.3.1. Theoretical Framework ...... 15 1.3.2. Content and Structure of Chapters ...... 19

CHAPTER 2 TOWARDS DIGITAL TECHNOLOGY & VISIONS OF THE WORLD IN CINEMA ...... 26

2.1. BRINGING ONGOING FEUDS TO AN END ...... 26 2.2. CINEMA: PHOTOGRAPHY, PAINTING AND THE NEW MEDIUM ...... 29 2.3. TECHNOLOGIES OF REALITY ...... 38 2.3.1. Technology in Aid of Realism ...... 38 2.3.2. Collage and Form ...... 45 2.4. THE DIGITAL AVANT-GARDE ...... 52 2.5. LOOKING THROUGH WORLDS BOUNDED BY TECHNOLOGY ...... 57 2.5.1. On Windows and Frames ...... 57 2.5.2. Re-routing the Role of Visual Perception in Cinema Theory: Beyond the Window/Frame Polarity ...... 65

CHAPTER 3 RE-COMPOSING REALITY...... 71

3.1. THE WORLD IN FRAGMENTS: FROM CONTINUOUS TO CONTIGUOUS ...... 71 3.2. ADDRESSING CINEMA EDITING, NOW AND THEN ...... 78 3.3. APPROACHING THEORIES OF VISUAL FRAGMENTATION ...... 85 3.3.1. Helmholtz’s theory of Perception ...... 87 3.3.2. Gestalt theory of Perception ...... 90 3.3.3. Gibson’s Ecological theory of Perception ...... 94 3.3.4. Lakoff and Johnson’s Cognitive Semantics ...... 99 3.4. APPROACHING MENTAL SCHEMATA: JULIAN HOCHBERG ON PERCEPTION ...... 104 3.5. DIGITAL EDITING AND PERCEPTUAL RE-CONSTRUCTION: APPLYING HOCHBERG’S MODEL ...... 113 3.5.1. Then and Now: Changes to Consider ...... 113 3.5.2. Addressing the First Differentiating Feature: the Cognitively Demanding Environment of Digital Worlds ...... 116 3.5.3. Addressing the Second Differentiating Feature: Rapid Editing ...... 126

CHAPTER 4 MOTION: BRINGING IMAGES TO LIFE ...... 135

4.1. MOTION AS LIFE: FROM AUTOMATA TO CYBERNETICS ...... 135 iii

4.2. THE DEBATE ON THE REALITY OF MOTION ...... 142 4.3. WHAT CINEMATIC MOTION IS NOT: AN ONGOING HISTORY ...... 145 4.4. ADDRESSING THE VARIATIONS OF CINEMATIC MOTION ...... 151 4.5. GESTURAL MOVEMENT ...... 153 4.5.1. Biological Motion and Motion Capture Technology ...... 153 4.5.2. Associating Biological Motion with Hochberg’s Model of Perception ...... 158 4.6. PROXEMIC MOVEMENT ...... 165 4.6.1. Placing Moving Objects in Space ...... 165 4.6.3. Moving in Cinematic Space Across Edited Shots ...... 175

CHAPTER 5 THE POWER OF STORYTELLING ...... 185

5.1. MOTION WITH A PURPOSE: FROM ‘MOVING AROUND’ TO ‘GOING SOMEWHERE’ ...... 185 5.2. FROM OLD, TO NEW, TO DIGITAL ...... 189 5.2.1. The Old and the New…or the Same? ...... 189 5.2.2. CGI in the Narrative Vs. Spectacle Debate ...... 195 5.3. COHERENCE AS THE PREREQUISITE FOR A UNIFIED THEORY OF NARRATIVE COMPREHENSION ...... 200 5.3.1. Accommodating Oneself in a Fantasy Movie World ...... 200 5.3.2. The Importance of Narrative Familiarity and Coherence in Synthetic Worldmaking ...... 203 5.4. COGNITIVE COMPREHENSION OF CONTINUITY NARRATIVE SEQUENCES IN MOVIES ...... 209 5.4.1. Dos and Don’ts in Applying Cognitive Psychology in Cinema Narratives ...... 209 5.4.2. A Cognitive Notation Model for Visual Narratives: Preserving Only What Matters ...... 217 5.4.3. To Cut or Not to Cut? The Importance of Breakpoints for Narrative Comprehension in Action Sequences ...... 224 5.5. THE MIND WE INHABIT: THE FUNCTION OF HUMAN MEMORY AS NARRATIVE ...... 230 5.5.1. Perceiving the World As Narratives and Vice Versa ...... 230 5.5.2. Applying Theory of Memory to Comprehending Movie Structure...... 235 5.5.3. Memory in the Context of Story vs. Form ...... 242 5.5.4. Weaving Worlds: the Role of Memory in Narrative Worldmaking ...... 246

CHAPTER 6 CLOSING REMARKS ...... 252

6.1. EXPANDING THE PRESENT STUDY ...... 252 6.2. ON UMBRELLAS AND RAINY DAYS ...... 256

WORKS CITED ...... 260

A. BOOKS, ARTICLES AND MISCELLANEOUS ...... 260 B. MOVIES ...... 273 C. AUDIOVISUAL SUPPLEMENTS ...... 278 Ci. Movie Clips ...... 278 Cii. Images in Appendix II retrieved from ...... 280

APPENDIX I - GLOSSARY ...... 282 APPENDIX II - ILLUSTRATIONS ...... 291

iv

To my parents and my sisters,

and my Evangelia

Thank you for enduring with me throughout these years v

ACKNOWLEDGMENTS

It is said that a dissertation is a lonely path that one has to walk alone, and yet, when one finds himself at the finish line, it does not seem so. Throughout the time of preparation and writing, a number of people contributed in their own way to this work, and for this I am grateful to all of them.

First and foremost, I would like to thank my mentor, Dr. Michalis Kokonis, who has been my main supporter, colleague, associate and friend all this time. He has not only helped me with the theoretical background and problem-solving insight required for engaging in this study, but also with the optimism and confidence necessary for completing it. His genuine interest and research in varied fields within the vast desert called Cinema Studies has proven invaluable in our conversations.

Many thanks to the other two members of my supervising committee, Dr. Yiorgos

Kalogeras and Dr. Gregory Paschalidis for their important contribution in theoretical fields that I needed guidance with.

I would like to thank all members of the academic staff at the School of

English that I worked with all these years, especially those that were also my teachers during my time as a student there. I owe a special ‘thank you’ to Dr. Karin

Boklund-Lagopoulou for always being there when I needed help and advice, and, most of all, for teaching me that academic expertise and professionalism is not incompatible with kindness of heart; and to Dr. Ruth Parkin-Gounelas and Dr. Nicola

Rehling for guiding me with patience through my early attempts in research and writing, providing me with knowledge that later on made this dissertation significantly easier in several respects. I would also like to thank the personnel at the

School of English, both present and retired, not only for assisting me with the various vi

issues related to my studies, but also for keeping the School in order in difficult times. The librarians, administration, and technical support officers are the unnamed heroes behind all dissertations that make it to the end; for this they have my respect.

The support of all my friends both within and outside of the academe has been a special kind of help and support during this period. First, all my fellow PhD candidates that I happened to temporally coincide with all this time have been advisors on issues out of my scope at the time; but most importantly they offered me a professional and personal companionship that made me feel as being part of a larger whole, and that a dissertation is as much a personal as it is a collective experience. From all my friends outside the academe, all of which have helped me not to forget myself into research and writing, Alkis stands out as the closest one, for his moral support, as well as his occasional relapse into the role of IT technician.

My last and warmest ‘thank you’ goes to my family, to which I dedicate this work; my parents, Kostas and Anastasia, and my sisters, Evi and Eirini. I am forever indebted to them for bearing with me all these years and for supporting me in countless ways. Finally, I cannot but say that I truly owe the completion of this study to my Evangelia, for keeping me on my feet, for having faith in me even at times when I did not, and for being the one person who believed in me against all odds. vii

ABSTRACT

The present study endeavors to provide a new direction for cinema theory that will be based on recent findings in cognitive theory and visual perception, with a special focus on digital moviemaking. Assuming the application of linear perspective during the Renaissance as a landmark point in the history of imaging, the study associates Computer Generated Imagery (CGI) with various technologies that have been used over the years in both cinema and in its kin arts, photography and painting, while at the same time it reviews major theories of cinema like formalism and realism on the various uses that they made on such technologies. By establishing a common genealogy for cinema as a whole, and at the same time debunking the term “revolution” in digital cinema, the study justifies the need for a more encompassing, stable and objective theory applicable to all cinematic texts.

As regards the type of theory that meets these criteria, the research was based on applications of cognitive theory to cinema, specifically the work of Julian

Hochberg and his associate, Virginia Brooks, in search for a framework that is relieved from the influence of any ideological or philosophical trend that might limit its universal applicability and objectivity. Hochberg’s cognitive theory of schematic mapping was deemed to be the most appropriate one to address universalities of human perception that are based on the actual physiology of the human visual system, as regards both its function and its limitations, while at the same time explain all encounters with cinematic texts, from realism to extreme worldmaking.

Schematic maps, i.e. the mental formations created inside the human mind in the process of perceiving the world in an inescapably piecemeal fashion, are composed into non image-specific forms that are from then on applicable to all kinds of visual viii

perceptions. As regards the cinematic texts, those were selected on the assumption that an extreme case of visualization is required in order to test the validity of the theoretical model. The texts selected were CGI fantasy blockbuster Hollywood movies, as they manifest two distinct features: on one hand, they abide to the rules of classical continuity which emanates a sense of realism, while posing the challenge of this realism being entirely contradicted by the synthetic cinematic universes in which they are set.

It has been found that Hochberg’s model of schematic mapping adequately explains that synthetic worldmaking is able to function realistically exactly because it is based on the schematic maps that both moviemakers, CGI designers, and spectators alike share, due to their common physiological experience of the real world. The study distinguishes between the cinematic story and the cinematic world as the two constituent factors of any cinematic universe, and applies Hochberg’s framework as well as other theories of perception in key aspects of the cinematic experience, i.e. composition and montage, motion, and narrative. The main findings were that schematic mapping is more than a promising field on which to establish a cinema theory for the future for two main reasons: first, all assumptions are verified in terms of the framework being applicable to any kind of narrative cinematic text; and second, the model is not incompatible to other theories of cinema, older or contemporary, as it operates on a physiological level of human vision, thus providing a scientific objectivity that literature on this art is in need of. ix

NOTES ON TERMINOLOGY

This is a research that focuses on the digital present and future of cinematography that respects and, where possible, incorporates theory and writings of its analog past. It is therefore obvious that a problem arises regarding some of the terms from cinema theory that are used, mostly in relation to the word “film” which normally refers to the strip of celluloid from the analog era, but has become a generic term for any cinematic production. It should be noted that this issue has been, to the extent possible, taken into consideration throughout the present study. The terms “film,” “cinema,” “movie,” and their derivatives or compounds are used interchangeably but entirely consciously in each case; more specifically, “film” is reserved for discussing analog moviemaking or anything in the context of it, whereas “cinema” and “movie” have been used in a more encompassing manner. Only two exceptions were made: the first one involves the compound adjective “pro-filmic,” which refers to the reality that exists before and independently of its image, as the latter is recorded by a camera. The term was deemed far too solidified in the relevant theoretical context of “reality vs. lens realism” (in contrast to the non-standard “pro-cinematic”); therefore the term “pro-filmic” was retained and preferred, even when discussing digital works. As for the second exception, it was made for the cases in which the word “film” was found inside quotes from secondary sources; the quotes, as normal, were retained in their original wording.

In addition to the above, a note should be also made about the terms “perception” and “cognition,” as the difference between the two terms is often a bit shady. The present study generally follows the definitions given to these two terms by the American Psychological Association (APA), according to which perception refers to “[t]he processes that organize information in the sensory image and interpret it as having been produced by properties of objects or events in the external, three- dimensional world,” whereas cognition refers to “[p]rocesses of knowing, including attending, remembering, and reasoning; also the content of the processes, such as concepts and memories” (APA Online Glossary of Psychological Terms). In other words, perception refers primarily to the mechanisms of acquiring, recognizing and understanding sensory data from our everyday environment through our interaction with the physical world of our senses, whereas cognition refers to the multi-leveled processing of information which stems from that acquisition, e.g. making associations and inferences, creating memories, solving problems, etc. From these definitions it becomes obvious that the two processes are in constant cooperation with each other; this is the general assumption that this study embraces.

As regards the terms “motion” and “movement,” which are extensively used especially in Chapter 4, there was the issue of them being obviously more or less equivalent, but being used differently in relevant literature. Based on that, and in order not to interfere with the way each author uses these terms, in the present study they appear interchangeably depending on the context. Although motion was generally treated as a wider and more encompassing term, both were consistent with following a specific terminology found in relevant literature (e.g. “apparent motion,” “proxemic movement,” “gestural movement,” etc.). 1

CHAPTER 1

INTRODUCTION – A NEW THEORY FOR AN OLD ART?

What has happened to cinema in the digital age? Is it still here, still as close to reality as art ever got, still as entertaining as it used to be, or are we witnessing the offspring of its unholy marriage to Computer Generated Imagery (CGI)? This is a challenge of significant difficulty that contemporary theory of cinema has found itself confronted with over the past few decades. Before the advent of digitization, cinema theorists had addressed all kinds of miscellaneous issues of e.g. identity, representation, spectatorship, mass culture, ideology, etc., with many of them admittedly not having been easy tasks to tackle. But, in retrospect, it has become apparent that the task of theorists used to be easier than it is today, mainly because of the same underlying assumptions: in all cases, the medium was photographic, in the technical sense of the word, and thus had the same relation to pro-filmic reality.

It was only the method of manipulating the recording images that varied.

Considering the various approaches that have shaped cinema theory over the past one hundred years or so such as realism, formalism, structuralism, psychoanalysis, even early cognitive theory, no matter how different from each other or contradictory they may have been, they all worked with the same subject matter and assumed the same technical parameters: i.e. that cinema is a strip of celluloid, containing images that belong to an existent pro-filmic reality, with artificial post- filming interventions to the latter being too limited to be theoretically noteworthy under normal circumstances. Digital technology has changed all that; cinema has been transformed to such a degree, that its theory essentially had to start over. 2

Based on this acknowledgment, this introduction presents the main problems and issues that the present study will focus on, divided into three main parts. The first part introduces the main changes that digital technology has brought to the cinema industry and the theory problems that came along with it. The second part discusses the shortcomings of applying older or existing cinema theories to contemporary digital cinema, and justifies cognitive theory as the most appropriate discipline to address the new medium. Finally, the third part sketches the outline of the entire study, introducing its main points and the selected theoretical framework.

More specifically, the third part introduces the primary cognitive framework that has been selected, i.e. the work of Julian Hochberg and his associate Virginia Brooks, and establishes the grounds on which it was deemed appropriate for the purposes of the present study. Hochberg and Brooks’ work is significant for two main reasons: first, it provides a model that inquisitively passes through some of the major prominent theories of perception of the past, re-visiting the field under a new light and without either dismissing or accepting blindly any of them in its entirety. This is exactly the spirit that is needed for an innovative theoretical account of digital cinema, as such an account needs to be on reconciliatory rather than on rupturing terms with the past, if the genre is to have any chance of being recognized as equal to the rest of cinema. Finally, the structure and sequence of the Chapters of the study, which closes this third part of the introduction, clearly reveals the effort that was made to approach digital cinema through the insight of the history of art as a whole, and with the contribution of various cognitive accounts.

3

1.1. The Challenges Towards a Theory of Digital Cinematic Realities

There are two essential problems that contemporary cinema theory now faces. The first problem is the ability that imaging technology now has to create synthetic worlds and characters, an ability which obviously results in the severing of the connection between movie images and their real life referents. The second problem, which is a consequence of the first, is the fact that contemporary movie images are inherently fragmented in multiple ways, the most important of which is the ontological breach between CGI images and real-life objects. Because of the impressive capacities of computer graphics today, digital moviemaking has grown increasingly bold in creating worlds that are more life-like, complicated and visually coherent, but at the same time less similar to the actual world as the latter is picked up by human senses. Because of this contradiction, digital cinematography positions the spectator in an encounter with cinematic universes that become increasingly convincing as instances of worldmaking.

The word “digital” has generally been associated with blockbuster fantasy movies, and those will indeed be the focus of the present study; yet, it should also be kept in mind that digital technology has spread throughout the industry, facilitating practically all levels of production and distribution of the vast majority of movies.

The main difference from one movie to the other has now become an issue of the level of intervention of digital technology in each case: for example, some movies are entirely designed using computer graphics software, while others are merely distributed or projected using digital media. Consequently, some movies are more obviously “digital” than others, when, in fact, pure CGI as well as digital manipulation of recorded footage is actually there more often than spectators notice. Therefore, 4

before any approach even begins an attempt to provide a noteworthy and reliable theory for contemporary cinema, it is obliged to consider these two problems of ontology and fragmentation.

As regards the first issue, the loss of the ontological link between the cinematic image and its real-life referent, It is obvious that the invasion of the digital medium in moviemaking cannot be addressed merely by clinging to influential landmark film theories of the past, as the latter normally take into consideration movies that are photographically captured on analog film, i.e. a kind of cinema that is practically becoming marginal now. Technologies of imaging, today more than any other time, play a defining role in what product may be labeled realistic in cinema, and, even more, in the form that it will have, especially now that it is certain that the cinematic artwork is impossible to be conceived as a continuous entity as it was e.g. in the era of Italian Neorealism.

Gregory Currie, one of the main representatives of American cognitive film theory, has offered a readily available and very useful categorization of visual realism. In his 1995 book Image and Mind: Film, Philosophy and Cognitive Science he conveniently classifies all realistic approaches to cinema to three main categories, based on the type of realistic experience that each of them assumes based on the spectator’s point of view: Transparency, Illusionism and Likeness (20).1 First, as

1 Currie also acknowledges a fourth type of realism, which has to do with the assumption that the relations between characters reflect, and ergo have some affinity to, the social relationships between people in real life. In this sense, movies become realistic as examples of real-life situations. Nevertheless, Currie seems to admit his own academic insufficiency regarding the sociological background that an analysis of this type of realism would require (21). Although Currie assesses such a sense of realism as “perfectly intelligible” (21), his perceived unwillingness to further discuss it is probably indicative of the general aversion that American cognitivists demonstrate against the socio- ideological turn that cinema theory took during the 1970s under the influence of Continental philosophy and criticism, a turn which cognitivists see as overcomplicated, misleading and, to a certain extent, unsubstantiated. 5

regards transparency, Currie follows Kendall Walton’s famous explanation of the term, referring to cinema as a “transparent” medium through which the spectator literally sees the object(s) depicted, as seen “through a window or a lens” (20).2 The second category, illusionism, involves the notion that cinema has the capacity of deceiving the spectator into an illusion of the “reality and presentness [sic] of fictional characters and the events portrayed;” and finally, likeness refers to the experience of movie-watching as one that approximates “the normal experience of perceiving the real world” (20). In Currie’s perspective, the better part of cinema theory regarding realism is established on the juxtaposition or the combination of the above trends.

Currie is on a good path, but in order to explore what kind of realism, if any, digital cinema is closer to, the digitally transformed cinematic experience needs to be built on a definition or conception of realism that will easily accommodate the new medium alongside the old ones. Given the fact that the shift of the medium from analog to digital has brought significant changes to the way the cinematic image is experienced, this new approach to realism should not be expected to conform completely to any single one of Currie’s categories, practically all of which have been established having analog cinema in mind. On the other hand, transparency, illusionism and likeness do not necessarily need to be contradictory to one another, either; in fact the digital medium calls for a selection of traits from each one, owing to the profound and radical changes that the cinematic image has undergone due to digitization, both in terms of its ontological ground and its relation to viewer experience. As the film medium changes, so does the experience of movie-

2 Walton deploys his transparency thesis in his essay “Transparent Pictures: On the Nature of Photographic Realism.” 6

watching; and it seems that, to a certain extent, none of these three main trends makes complete sense either on its own or without serious modification when applied to digital cinema.

As regards the second issue, that of the fragmentation of the cinematic image, let us focus on what is probably the most common case in contemporary movie productions: movies that include portions of physical reality mixed with digitally created objects, or are set in CGI backgrounds. The process known as digital compositing simplifies the procedure of creating worlds in movies. Heterogeneous audiovisual CGI samples are combined with each other, as well as with recorded samples from the actual world, to produce a visual stream that resembles

“something that took place in real physical space and was filmed by a real camera,” while at the same time various techniques enhance this “reality effect” (Manovich,

New Media 136–137). Taking the Universal Capture (U-Cap) technique as an example, Manovich offers an idea of what kind of evolution of the movie image we are talking about:

[D]ifferent types of imagery are not mixed together but rather fused to

create a new kind of image. This image combines “the best” qualities of

two types of imagery that we normally understand as being ontologically

the opposites [sic]: live action recording and 3D computer animation.

(“Image Future” 3)

This hybrid new form of visual narratives, Manovich tells us, does not strive anymore, like previous movies, to hide the seams between digital graphics and live action; instead, “it explores the space between juxtaposition and complete integration” (“Hybrid Media” 3). Digital realism in cinema, therefore, stands in 7

between the mechanical and the natural; not only it constantly underlines the meaninglessness of separating them, but actually establishes its effect on their difference, eventually becoming more than each of them.

In effect, what this means is that a theory of contemporary digital cinema needs to be established on a wider and revised conception of realism; this brave new image is meant, and generally succeeds, to be perceived by spectators in the same way and on the same terms as images perceived in real life. As such, it challenges a significant part of the assumptions of older theories regarding what is perceived as real and how it should be addressed. This is why a theory of digital cinema, before it even starts considering issues of e.g. content and representation, must first and foremost start by answering a question, the answer to which used to be more or less obvious in the analog days: what kind of realistic experience exactly do spectators have when watching a contemporary movie? In the digital era, it is therefore mandatory that issues of visual perception and cognitive comprehension precede in urgency all other discussions.

1.2. The Rise of Cognitivism

The way out of the maze is obviously not an easy task. Of all the candidate approaches to a theory of cinema that meets the needs of the Twenty-first century, there have been a number of suitors, the most prominent of which being cognitive theory. Cognitivism places primary importance to the function of human cognition, and assumes active minds during movie-watching, an approach which contradicts structuralist theories of cinema that dominated writing on Cinema for decades.

These structuralist approaches, before the gradual fading of their momentum in the 8

mid-80s, essentially viewed the spectator as a passive receiver, hopelessly entangled in the web of ideologies. As rivals to such assumptions, cognitivist trends in cinema theory surfaced from the need to consider movie-watching from the viewpoint of thinking, active spectators. Almost a century after the first publication of

Munsterberg’s The Photoplay: A Psychological Study in 1916, cinema seems to have drawn a full circle to return to the human mind, where Munsterberg believed it to acquire its meaning in the first place.

Most contemporary approaches draw more or less on theories of cinema which are generally established and widespread; perhaps the elusiveness of the digital medium makes it logical for researchers to seek refuge to influential theorists of the past. But the problem remains and it is easy to understand why: if any of those theories could, on its own, support the weight of the changing medium, that theory would have probably prevailed with relative easiness. But the exact opposite has happened: even cognitive theory, which has emerged as a hopeful alternative capable of bringing new air to the rapidly decaying and complicated matrix of cinema studies, still struggles to establish a solid and reliable framework.

Those who disagree to the above statement are reminded of the cruel reality: the first prominent cognitive approach to cinema, introduced by David Bordwell as early as 1985 in his landmark book Narration in the Fiction Film, by now already aging and decaying, for all the refutation that it has undergone since, sadly and stubbornly remains probably the single most influential one among a multitude of other cognitive approaches to cinema.3 Despite its impact and the overall positive

3 Bordwell’s theory demonstrates both incredible insight for its time – often quoted in several parts in the present study – as well as surprising shortsightedness at the same time. It has undergone significant attacks over the decades, on several grounds. Although a review of such attacks would be 9

new breath that it gave to cinema theory, the Achilles’ heel in Bordwell’s theory has remained, until today, twofold: first, resting in its success, it did nothing to remedy the immense dispersal of cognitive theory variations. And if problems such as this can be forgiven to Bordwell as to any completely novel approach, what remains unforgivable is the fact that his theory had to keep undergoing maintenance and amendments ever since so that it could accommodate the inherent dispersal of cognitive theory on which it was based. And three decades is quite a significant amount of time for any discipline to remain so dispersed. For all these reasons, the present study cannot but respectfully avoid following Bordwell’s cinematic cognitivism, without of course denying its value for opening new directions to cinema studies. Throughout the study, it will be used as a pool of resources rather than a mentoring theory.

Of course, Bordwell’s shortcomings are simply indicative of a discipline that stubbornly refuses to be codified in a limited number of specific characteristics. Noël

Carroll tries to justify this situation by claiming that cognitivism is a stance, rather than a concrete and unified theory; cognitive research expands over a very wide area of domains, thereby making it futile to expect a common theoretical framework embraced by all cognitivists (62). But there is really no excuse for the 30-year delay.

Cognitivism may indeed be a very diverse field, but, out of the various trends within it, there are at least a couple of approaches, more applicable to cinema and

illuminating, such an endeavor would cover a rather long part of the present study, mainly due to the fact that Bordwell’s approach has been very influential, and extremely lengthy bibliography sprang from it. This would result in the present study including a very long critique of Bordwell’s thesis, and even more, on parts of cognitive theory that are beyond the scope of the theory of Hochberg and Brooks, selected and used here. This is far from the purpose of this study; for this reason, in the following chapters the critique on Bordwell will be limited only to those points that have been deemed to be false and incompatible with specific parts of Hochberg and Brooks’ approach. 10

pertaining to its features, that should have emerged already, besides that of

Bordwell. In the opposite camp for instance, psychoanalysis, despite its deadly flaws, has provided an incredibly coherent and withstanding theory of cinema which survives practically until today; it may be diminished as regards its impact and influence, but it is still intact and solid in its positions. Indeed, its resilience may have simply been the result more of its consistency and less of its capacity to provide sufficient answers to theoretical issues, but its birth and course over the years still constitutes a lesson that cognitivism has yet to learn.

But is that reason enough to give up on cognitive science altogether as regards its application to cinema theory? The answer is definitely “no.” Cognitive science, being the discipline charged with explaining the operations of the human mind and its mode of contact with the environment, is definitely the place to turn to, now that the main problem of cinema is the loss of the connection with a pro-filmic cinematic reality as a stable, tangible and trustworthy point of reference. The advent of synthetic realism in digital moviemaking has resulted in the production of movies that are perfectly comprehensible, and yet have nothing to do with reality as we know it; therefore, it revealed that the relationship that cinema as a genre has with the spectators’ experience of cinematic worlds, or even the real world for that matter, is more multi-faceted than originally thought, thus also betraying how fragile the terms “realism” or “realistic representation” are. Consequently, the invasion of digital graphics in cinema and its merging with actual recorded images has given a fatal blow to what used to be understood by the concept of a realistic narrative as representation of real-life situations. The cognitive trend which has been more prominently initiated in cinema studies since the early 1980s may still be hampered 11

by a lack of a specific set of core theoretical principles, but is definitely on a good path as regards giving answers that need to be built on scientific foundations.

Cognitivism, for all its problems, is today the best solution available for theorizing a kind of cinema that actually defies cinema theory as we know it.

The cognitive approach addressed the need to deliver cinema knowledge from the set of pre-existing theories, which, for cognitivists, lost their purpose somewhere along the way, shifting attention from movies themselves to explaining wider cultural structures. Bordwell started his own approach by correctly criticizing those theories for their vagueness; he uses for them the accumulative term “Grand

Theory,” in which he includes subject-position theory and culturalism, as well as the feud between them.4 Bordwell generally disapproves of Grand Theory, accusing the body of work contained in it of becoming too self-centered, complicated and ultimately incomprehensible, essentially failing to serve its explanatory purpose:

The rise of Theory crushed theories and discouraged theorizing. Grand

Theory created bad habits of mind. It encouraged argument from

4 According to Bordwell, the subject-position theory was the trend that replaced pure structuralism in the US in the 1970s, conspicuously coinciding with the time in which French film theorists started teaching in international programs. It was an amalgam of Althusserian Marxism, Lacanian psychoanalysis, Metzian semiotics etc., with existing disciplines in the US, such as feminism, soon incorporating and further expanding the former. Bordwell notes that the branches of the subject- position theory were built on common assumptions about the nature of the psyche and social structure, which revolved around the role of “the subject in language and social activity.” In short, the theory laid emphasis on the “subject” not as a sense of solid, personal identity or as an “immediate sense of one’s identity or self;” instead, the subject is viewed as a “category of knowing” which is acquired rather than innate, and always dependent on the social positioning of the individual within a network of relations to other subjects, tightly bound by systems of representation (Bordwell “Contemporary Film Studies” 6). As a consequence, film in the 1970s was theorized as a “semiotic system . . . considered to engage the spectator as a split subject, initiating a process in which conscious and unconscious interact” (7). Culturalism, on the other hand, sprang out of the critique against subject position theory. Within its circles, as Bordwell notes, the term “culture” in theoretical writings gradually replaced “ideology” and “society,” eventually covering “all spheres of social activity” (9). Unlike subject-position theory, culturalism advocated that “pervasive cultural mechanisms govern social and psychic functions of cinema” (9). Finally, culturalism laid emphasis on the historical aspect of social practices, as it is seen through “microhistories” rather than the “grand narratives” of subject position theory, a position which lays importance in the specific uses of texts rather than the texts themselves (10). 12

authority, ricochet associations, vague claims, dismissal of empirical

evidence, and the belief that preening self-presentation was a mode of

argument . . . Very likely, no one in the history of English ever published

prose as incomprehensible as that signed by Theorists. (Poetics of Cinema

2)

In the same spirit, Carroll also adopts a contemptuous attitude towards this body of writings. Similar to Bordwell, he attacks traditional film theory, for which he uses the sarcastically capitalized all-inclusive term “Theory,” describing it as an often incoherent pastiche of the academically diverse writings of theorists like Althusser,

Barthes, Kristeva, Derrida, Foucault, Lacan, etc. These writings, according to Carroll, were almost opportunistically used by universities to create a new discipline of “film studies,” a practice which ultimately ended up expanding the Theory way out of its proportions or what the academe could handle in the first place. The body of work produced under the Theory eventually derailed the course of cinema studies, drawing it away from the study of the structure of cinema itself and disregarding crucial aspects of the role of the spectator in the process of film watching. In

Carroll’s own words, “film studies has been flooded with repetitive decoctions of the

Theory in search of the same market in much the same way that consumers are confronted with so many marginally differentiated shampoos” (37). Carroll also notes that the gradual decline of the Theory in recent years has appeared because of its stiffness and its clinging on questions of “theorizing,” rather than on theoretical issues themselves (38). It can be argued that psychoanalytic theory, which comprises a large and highly important part of the Theory, can be charged with revolving instead of evolving; and it is also true that its attempts to meet the needs of 13

contemporary cinema have been scarce and hardly influential, at least in relation to its past glory. But at the end of the day it all comes down to the same fact: the evolution in cinema imaging and technology has been so dramatic over the past two decades that it has outran old-school theorists, and it ultimately crushed those whose accounts could not keep up with the massive changes in the industry. This is why cognitivism easily emerged as a new hope for cinema studies. This is also why, for all its flaws, Bordwell’s theory, although not embraced in the present study for the reasons mentioned earlier, at least deserves credit and respect for helping cinema theory break free from the eternal loop it was trapped in, so that cognitivism can at least have a fair shot.

Another reason for opting for cognitivism is the fact that it is so wide, that it can encompass many older disciplines from various areas. In other words, what is normally a flaw for a discipline can be turned into an asset under the right circumstances. Warren Buckland’s very straightforward account of the two main trends in cognitivism illustrates this point. Buckland divides cognitivists in two large groups, the American and the European, and explores the relationship of each group to what he calls “Modern Film Theory,” i.e. the body of work that emanates from structural linguistics, semiotics, Marxism and psychoanalysis. Each group has a differentiated attitude towards this past. As we have seen, American cognitivists, more well-known and influential, such as David Bordwell, Noël Carroll, David

Branigan and others, reject Modern Film Theory entirely; on the contrary, European cognitivists, e.g. Francesco Casetti, Michel Colin, Roger Odin and others choose a reform by bypassing the Theory’s later form, i.e. Post-structuralism, and instead find a good use for its early semiotic stages, productively combining those with cognitive 14

theory in what Buckland calls “cognitive film semiotics” (1–2). This latter case, although not explicitly explored in the present study, has great value in revealing the potentials that cognitivism in general has for addressing contemporary cinema: its wide theoretical span and, more importantly, the support it has from actual scientific data instead of notions and ideas expressed in complicated yet hollow and equivocal sophistries, make it a flexible, prestigious and reliable toolbox in resolving the perceptual riddle of CGI fantasy realism.

This is indeed a promising approach, but it may be a bit too early to head for that direction. The influence from the group of disciplines that comprise the Theory is too recent, and it could be risky at this point to flirt with its kin, i.e. semiotics. It is probably true that Buckland’s approach, and generally the European branch of cognitive science, is closer to a more reliable, unified and barricaded theory of digital cinema, as it manages to be both novel in its approach and conciliatory in relation to the past. The only problem is that digital cinema is a hard-core technological domain; any theoretical framework for pure CGI and compositing in cinema must incorporate more direct and clear scientific data, before it starts becoming more abstract. In other words, what digital cinema theory desperately needs right now is scientific substantiation of its cognitive impact as regards the mechanics of human visual reception, exactly because it poses tremendous challenges to notions of realism, which used to be a major point of reference for practically any theory so far.

15

1.3. Problems and Solutions: an Outline to the Present Study

1.3.1. Theoretical Framework

For all the above reasons, the present study clearly adopts a cognitive framework in cinema, but the effort was towards providing supporting experimental data from cognitive science that are not as notoriously abstract as the Theory used to be, and thus also not as easy to refute. At the same time, the framework selected was one that still maintains ties with past theories, thus also aiming at providing continuity for cinema theory as a whole. The cognitive approach that was selected was mainly that of Julian Hochberg and his close associate Virginia Brooks. Hochberg is an Emeritus Centennial Professor at Columbia University NY, and belongs to the wider group of American cognitivist academics. His work on visual perception has been among the most prominent ones in the field for many decades now. He has published work on the cognitive perception of cinema in the wider context of his research on visual perception and comprehension of images and the surrounding world in general. His approach evolves on the track initiated by the physics of

Hermann Von Helmholtz, as well as the Gestaltists, both of which he reviews, corrects, accepts and supplements in several ways. Hochberg’s position generally disagrees with “direct” theories – although without being straightforwardly dismissive of them – such as the work of J.J. Gibson, which support that we perceive the world and visual scenes directly and practically without intervention of mental operations. Contrary to that direct access, Hochberg believes that cognitive operations are at hand while we receive visual data from the environment, which shape the world for us into comprehensible structures. 16

Hochberg and Brooks’ view is that the perception of moving images in general, and thus also cinema in particular, is similar to normal perception of the world, and should thus be explored on the same grounds. Their body of work is built around the argument that our perception is always piecemeal, which means that we never perceive an image in its entirety at the same time, due to the physical limitations of the human visual system: we are only capable of seeing only specific parts of an image or scene. Our eyes collect spots from the optic array only where the fovea focuses in successive and rapid saccades; these spots provide clear and usable visual information, but are very small and limited in number compared both to the image itself and our optic array as a whole. On the contrary the images perceived from parts of the eye that are away from the fovea are significantly deteriorated as regards the quality of visual information. Eventually, what we perceive from this partial reception of images is a mental structure rather than an image, a schema which we fit into pre-existing knowledge of previously encountered schematic structures, which we then amend with new information: in short, this means that we do not see the world the way we think we do, because our visual and cognitive mechanisms work in an entirely different way. This is what Hochberg refers to by his famous phrase “the mind’s eye.” This feature of the human visual system and the way it affects our perception of images and scenes is hardly a recent discovery; and yet, surprisingly, it has not been taken into consideration or explored in any prominent way by cinema theorists over the decades. Overlooking this simple physiological and cognitive truth has obviously tremendous implications for our approach to cinema: its consideration could have easily changed the definition of cinematic realism and perception altogether, as it completely alters our knowledge 17

of what kind of visual data humans actually collect and perceive, and in what form.

In essence, it pulls the rug from under any attempt to theorize in favor or against realism, in any of its versions. More importantly, it compels any attempt of this sort to consider scientific data and truths instead of e.g. arguments based on confusing or hollow neologisms. It is more than certain that cinema theory has clearly had enough of those already, anyway.

The present study, therefore, will draw significantly, though not exclusively, on the relatively unknown work of Hochberg in the field of cinema, in order to provide further supporting documentation to kindred cinema cognitivism in general.

One of the reasons that Buckland’s Cognitive Semiotic approach is on a good path regarding the future of cinema theory, as mentioned earlier, is due to the close affinities that it has with Hochberg’s argument. More specifically, the cognitive semantics of George Lakoff and Mark Johnson seem to agree with the way Hochberg conceptualizes schemata, albeit with a more linguistic orientation. Lakoff and

Johnson’s schemata, which are taken into considerable consideration in Buckland’s approach, are defined as “cognitive structures that organize perceptual input into experiences,” thus representing in the mind both thought and language (Buckland

22). Just like Hochberg’s, Lakoff and Johnson’s schemata are image-based, albeit not bound by specific images, and “directly motivated by bodily experience” (22). For this reason, they fit very well a cognitive model of schemata tailored for cinema comprehension.

The fact that Hochberg’s account agrees with that of Lakoff & Johnson to a great extent is a promising indication towards a unified cognitive theory on both sides of the Atlantic. And yet, it is also indicative of the dispersal of the discipline; 18

how can a researcher explain, or forgive for that matter, that Buckland’s cognitive semiotics does not consider at all the work of Hochberg, a well-renowned expert in the field of visual perception since the middle of the Twentieth century? Cognitivism simply does not seem to be willing to summon all its available forces together. A counter argument could be that, at the time of the development of the early cognitive approaches to cinema, several advances in cognitive science were not yet available; but this does not really stand. Contributions by both Hermann von

Helmholtz and the Gestaltists, on which Hochberg and Brooks heavily rely on, were available from as early as the late Nineteenth century. Now that cinema theory strives to get back on its feet, ignoring one another is a luxury that cognitive theorists cannot afford any longer.

In this context, the present study builds its argumentation on two main foundations: first, that digital cinema today is not a different product than the cinema of the past, nor of the rest of imaging in general, but that changes in technology demand for more scientifically aggressive theorizing. This means that any specific area of digital cinema that is explored should take into consideration the arguments and history of related areas in analog cinema or visual media in general, as well as the changes in technology and what they entail for revieweing assumptions of the past. And second, that watching cinematic scenes is subject to the same perceptual mechanisms as seeing scenes in real life; this means that questions of ontology in cinema, i.e. whether cinematic images are real or not, if asked at all, are secondary: the real question is not whether they are real or not, but whether the mind comprehends them on the same grounds, a question which is at a completely different level. Using a statement that is as vague as it is useful, Bordwell 19

defines “realistic motivation” in a viewer’s disposition as something that “depends on what seems lifelike to someone versed in specific conventions” (Narration in the

Fiction Film 149). The primary knowledge gained by cognitive theory is that cinematic images are indeed as real perceptually as any other image. The fact that our mind can tell the difference between “real because I am there” and “real as if I were there” does not mean that the image is perceived as fake, at least as far as the human perceptual mechanism is concerned.

1.3.2. Content and Structure of Chapters

The general assumption of the present study is the way synthetic worlds are created with digital graphics as well as the cognitive operations through which these worlds are perceived. The movies that serve as examples are selected from the body of contemporary fantasy blockbusters, as the purpose is to research the impact on human cognition of the kind of worldmaking that does not look like scenes from real life. Since, as we have seen, digitization is a wide and encompassing term in the cinema industry today, the selection of the type of movies to focus on was made having in mind certain ones whose content shows a significant departure from the appearance of the physical world, in order for the study to explore the responses of human perception to non-realistic situations. Each chapter takes up a theme that is intrinsic to cinematic worldmaking in general, and explores it from the standpoint of digitization: technology, editing, motion, and narrative structure. These topics are all core areas that the digital shift has tremendously affected in moviemaking. Each topic corresponds to a chapter, and each chapter has two main axes: one artistic, so to speak, and one scientific: on the premises of the corresponding topic in each case, 20

the artistic axis links digital moviemaking to historical aspects of cinema in general, while the scientific axis links the topic to visual perception and provides a substantiation of the argument which is based on cognitive psychology.

Chapter 1 has the purpose of setting the foundations of a cinema theory for the digital age, while debunking the term “revolution” for digital cinema, as it immediately denotes a false separation from the history of analog visual arts. The

Chapter endeavors in this task by establishing the position of digital cinema in the same lineage as the rest of imaging; it does so by providing grounds for considering a common ancestry for all visual arts of the Twentieth century. The key for incorporating CGI in a common history of imaging is technology, as the latter has been a defining parameter of realistic accuracy in pictorial arts in general. The genealogy of digital cinema is equally a technological as well as an artistic one, and all its components go back to Renaissance perspective, the primary source of

Western realism. The complicated and various intersections among arts, sciences, technologies, etc., which paved the path towards the invention of cinema, provide the wider viewpoint that gives to synthetic realism, as well as to its experience by the spectator, its history, purpose and technical foundation. Due to this common ancestry line shared by all visual arts, the practices of digital cinema are presented as a reconciliatory force among trends of the past that have traditionally opposed one another on the basis of their assumptions regarding the optimum manipulation of movie content, i.e. realism and formalism. These two pillars of cinema theory seem to have been more preoccupied with proving each other wrong than to explore their areas of agreement. Digital cinema actually emerges as a digital avant-garde; with the aid of cognitive theory, it easily overturns their various polarities, such as the 21

notorious tradition between window and frame, which have vexed cinema theory for the better part of the last one hundred years. With this purpose in mind, the final part of the chapter begins bridging its content with the cognitive theory of Hochberg solidifying the ground in order to fully deploy the theory further down.

Chapter 2 picks up the discussion as an aftermath of the digital avant-garde, and explores the practices of classical realist editing and formalist montage, as well as their evolution and fusion in the digital era. The Chapter on one hand passes through these two pillar theories of cinema history, especially their attitude towards the manipulation of successive shots in relation to each other; on the other hand, it correlates the inherently fragmented nature of all cinema with the various levels of fragmentation of CGI in particular, and explores the implication of this fragmentation for building a solid theory for the visual perception of digital cinema. From the possible candidate cognitive theories that could address this issue, the chapter justifies the primacy of that by Hochberg, and explains its main points in detail. The most important point to be made here is that which describes human vision as inescapably piecemeal, i.e. with the visual system rendering images to the brain in the form of fragmented particles to be deciphered with the use of schemata; this is a key point which is used to approach both the cut as an essential mode of fragmentation in cinema, as well as other levels of fragmentation, as those were laid out at the beginning of the Chapter. The latter also addresses two essential characteristics of digital fantasy blockbusters, which may pose issues to the application of Hochberg’s theory: the cognitively demanding visual complication of digital worlds, and the constantly increasing speed of editing in the corresponding movies. Each of these two characteristics refers to a different level of fragmentation 22

within the general framework of digital moviemaking. Both of them are resolved by drawing on key points from within Hochberg’s theses. By the end of the Chapter, the main feature of digital cinema, i.e. its multi-leveled fragmentation on the basis of technology, image, ontology, editing, etc., is shown to be fully compatible with

Hochberg’s schematic account of visual perception.

Chapter 3 builds upon some of the points raised in Chapter 2 in relation to editing, and moves on to discuss the perception of motion in the cinema. The first part of the Chapter illustrates the importance of moving characters in general, as a feature in art that was historically associated with presence of life and animation, eventually becoming an obsession for both artists and engineers as well as their patrons. Based on this background, the Chapter moves on to describe the ongoing debate on whether or not the motion seen on the cinema screen is real or illusory, reviewing landmark positions on the theme, such as the Gestaltist contribution of

Max Wertheimer, the persistence of vision fallacy and its refutation by Joseph and

Barbara Anderson, as well as oppositions within the cognitive sphere, such as the one between Gregory Currie and Andrew Kania. The better part of the Chapter establishes a more focused and experimentally supported account of visual motion.

After listing the various modes of movement that a character can exhibit in a movie into the general categories of gestural and proxemic, as well as their sub-categories, the chapter explains how cognitive perception functions in each case in the same way as it does with the perception of motion in real life. Several parameters that enhance the realistic effect of motion in CGI are accounted for, and the results of experimental data are utilized in support of this thesis. The contribution of

Hochberg’s theory in this study of motion in digital cinema is very important; on one 23

hand, his account of mental schemata explains the perception of artificial motion the way it does with real motion. And on the other, exactly because this account can anticipate what kind of information can be perceived depending on the visual cues and circumstances in each case, it provides useful insight for CGI animators and editors as regards the optimal construction of animated synthetic worlds in cinema.

Chapter 4 focuses on the role of narrative in digital cinema. It thus completes the set of themes that cinematic worldmaking relies on, passing from motion in general to purposeful motion, which creates meaningful sequences of actions and, ultimately, establishes narrative structure. Despite the influences of formalist techniques and practices in CGI, digital fantasy blockbusters are presented as the descendants of Hollywood cinema; and also, despite their deviations, these movies still heavily rely on continuity style, which emerges as the most cognitive-friendly mode of visual storytelling, a feature which also explains its longevity and success.

Mainstream blockbuster movies aim primarily at telling comprehensible stories to vast audiences, using classic narrative structure in order to override the fact that the worlds depicted in them are very unrealistic in comparison to the spectator’s everyday reality. Mainstream blockbusters use the very specific and well-defined techniques of visual narration and seamless editing that characterizes the narrative flow in Classical cinema, so that they provide the necessary level of coherence, which ultimately creates the sense of a reliable cinematic world. In contrast to Bordwell’s

“gap-filling” cognitive model of spectator, Hochberg presents the more solid version of an active mind that does no gap-filling, and is able of making sense of narratives without any need for the redundant parts of the action, which, eventually, are not used at all, either by the movie itself or the spectator’s mind. This model is much 24

more convenient for considering synthetic worldmaking, of which the hypothetical totality of details and events has no real-life model to rely on, and is thus impossible both to render on screen and for spectators to imagine. This observation is connected to the positive effect of breakpoints and, generally, segmentation as regards the comprehension and retention of narrative structure, as segmentation of the visual narrative marks the points in which redundant information have been omitted. Finally, the Chapter reviews the close relation between the functions of human memory and narrative structure: the latter is a powerful schema and a catalytic factor in retaining memories, which actually enables the comprehension of oneself inside a spatiotemporal continuum. Because of this fact, continuity narrative perfectly matches the human cognitive mechanisms and this is why it works perfectly for fantasy blockbusters as well as for any other movie.

It is obvious therefore that the present study aspires to be more than yet another paper on cinema. On one hand, it draws on a number of disciplines that are not easy to co-exist; and on the other, great effort has been made into preserving and addressing both the artistic and the scientific aspects of the topic. This is because digital cinema, by its very nature, defies borders, categories and tags; in its search for a new identity, the last thing that cinema theory needs right now is exclusions. It is the author’s belief that the study has been successful in properly addressing all the major issues regarding digital worldmaking, and has provided productive thought for considering even more; the latter were deemed to be beyond the scopes of the present body of research and are briefly suggested in Chapter 6, which concludes the study. By the end of the very last page, digital cinema will have hopefully been theorized properly, and it will have found its own position in 25

contemporary culture as the locus where science and art have been so merrily wedded for the first time, but with little appreciation so far. 26

CHAPTER 2

TOWARDS DIGITAL TECHNOLOGY & VISIONS OF THE WORLD IN CINEMA

2.1. Bringing Ongoing Feuds to an End

This chapter aims to provide a solid substructure on which to attempt to theorize digital cinema in a way that will take into consideration all of its historical and technical particularities. The challenge in this endeavor is the fact that, for many years, theories of digital cinema left aside one aspect or the other. The formula proposed here does not draw solely on the history and specifics of one discipline, e.g. realism, formalism, painting etc. How can the codes of realism and continuity editing, for instance, apply in synthetic imagery, when their composition technique is completely different? It is true that Hollywood-style montage and visual narrative has prevailed, especially with the domination of the blockbuster culture; still, Soviet montage has passed down to cinema theory very interesting insight into the way those images make sense. This is true because, on one hand, most mainstream movies today use classical Hollywood linear narrative and seamless continuity montage, but on the other hand, the images in new CGI and fantasy movies are not realistic in the sense that they do not portray familiar landscapes, characters, objects, etc. The challenge to reconcile the two is further hampered by their different objectives.

The glue that will be used to bring together these two extreme approaches to visual art is the utilization of elements from the history and technology of imaging.

The tight, though often neglected, relationship between technology and art which dates back centuries but reached its creational peak about a hundred years ago 27

provides a promising point of departure for a unifying theory of cinema. According to

R.L. Rutsky in High Technē, Art & Technology, the late nineteenth – early twentieth century was the era when technology first joined art not only in instrumental but also in aesthetic terms, a point which marked the initiation of “machine aesthetics” in culture. Technology was never alien to art, but in contrast to past practices, this time the marriage between the two was both conscious and broadly manifest.

Modernism provided fertile ground for this conjunction: the result was “aesthetic modernism,” which brought together art and the machine as a celebration of form and technique (Rutsky 73). The fireworks did not last long, though. First came the degeneration of formalism due to political disillusionment; and then, when the theory of analog cinema, especially realism, lost its connection with its Renaissance- old, pseudo-realistic, technological ancestry, it took a wrong turn. The exaltation of photography-as-record indeed helped cinema gain independence from the other arts, but also deprived it of several of their expressive tools that were deemed too

“artificial” to fit the revered purity and directness of the lens. Too tightly bound with this ideal of photography-as-record, cinema had to wait years for its capacities to unfold: this became feasible when digital media ruptured the boundaries between art forms, and consequently between genres as well, and unleashed new theoretical potentials as regards cinema reception and comprehension.

Rutsky claims that mechanization brought by modernism is the “basis of mass production” (79), and this description perfectly matches the appeal and processes of both film and its closest of kin, photography. In the rivalry between realistic and formalistic practices, the former eventually prevailed in terms of the consumption of this mass production. However, an assumption of these two arts as essentially 28

realistic sounds hollow: true, photography and film were as close as possible to recording slices of real life; but in a wider historical perspective, in all attempts for visual representation of the world, realism may have been the original intention, but, eventually, formalist found its way in the process to a greater or lesser degree. The reason behind this phenomenon appears to be the inherent capacities of human perception as regards the reception of visual encounters, which applies to the entire history of visual art. This is why it is impossible to theorize digital imagery in cinema without the use of other visual arts; and, even more, without always keeping in mind that all of these other arts were created or developed by the tendency of the human mind towards calculation and structuring which was briefly revealed during the mechanization of art, despite the fact that it was again concealed afterwards.

Computers and digital media, which revive this revelation in various ways, are far more than yet another tool for imaging; they constitute a meta-perception of images, a simultaneous manifestation not just of the ways the human mind experiences images, but also of the internal mechanisms that the mind uses to experience them. Computers are indeed made in man’s own image.

For this reason, digital cinema can be approached neither on its own, i.e. as a technological miracle that took cinema or cultural theory by surprise, and certainly not as merely a technological feat that little has to do with real art. In fact, neglecting either of the two sides, art and technology, in an analysis of a product which is so difficult to define, inevitably leads to unbalanced conclusions. This is exactly the path that both realism and formalism took, as both of them almost deliberately chose to ignore what was common between them, opting instead for what was different. A theory of digital cinema, therefore, is a product of, and manifests itself as, a 29

technology of assemblage: it craves for combination of theories, just like it excels best when it exhibits its own combinatory capacities; as a result, no single cinema theory can hold it. For this reason, after we review both the arts and technologies that have made digital cinema possible, the chapter will explore the ways in which

CGI brings cinema theories and, above all, practices together; gradually it will be shown that cinema needs to be steered towards cognitive approaches that are beyond and above any single cinema theory, and also have the capacity of incorporating all of them without negating any of them.

2.2. Cinema: Photography, Painting and the New Medium

Digital cinema, simply understood, is the latest chapter in the history of imaging which started centuries ago, and it is no more of a revolution than what other technological or artistic breakthroughs were in their own time. The technology that made cinema possible in the first place indeed goes back to the invention of photography, but photography itself was the technological culmination of a process towards pictorial accuracy that actually lasted for centuries. In this wider view, photography as an art was much more than a coincidental experimentation on light and chemicals. Originally conceived and developed as a solution to the need for absolute realism, cinema was the hybrid of pre-existing forms of art, photography and painting, brought together by a combination of technological advances in the broader area of imaging. A definition of digital cinema will ultimately have to reconcile photographic realism and canvas realism; the reason is that the technical bonds that digital cinema shares with its immediate ancestor, analog film, are a bit overrated; and on top of that, these bonds are often so strict and inflexible that they 30

actually make the relation between the two even more problematic. By severing some of these bonds, digital cinema clearly overlaps with its even greater ancestors of imaging in several additional areas, in a way that the stand-off between analog and digital movies loses all meaning. It will be shown that the bonds that are severed are actually the ones that make digital cinema in particular a more obvious and intricate part of cinema and imaging as a whole, in comparison to the present situation.

In order to follow this tradition, it is therefore essential to trace the family tree of digital cinema. This way, it is possible to link the technical means to the purposes of their inventors, and thus find common grounds behind them. This family tree, simply put, goes back to analog cinema, and further back, to photography.

Photography actually has two ancestors, one technological and one artistic. The former involves the combination of pre-existing technologies of chemistry and optics, irrelevant to each other, a combination considered to have been first achieved by Nicéphore Niépce, in 1826.5 And as regards the latter ancestor, it was actually painting: Gombrich, for instance, describes the initial rivalry between photography and painting; the invention of the former actually played a crucial role in the evolution of the latter, as “painting did not have to do, any more, what a mechanical device could do much more effectively and affordably;” Gombrich states that, up to then, a painter was traditionally considered to be someone who could

5 The question of the fatherhood of photography is a bit complicated, as one branch in the history of art considers Niépce as its inventor, whereas another sees it as the sum of independent accomplishments simultaneously achieved by more than one people. Even if we limit the list of contributors to the modern era, thus putting aside all ancient names that are linked in one or the way to to the camera obscura, such as Aristotle, Euclid, Ibn Al-Haytham, Leonardo DaVinci, etc., we are still left with several names besides Niépce, like Louis Jacques Mandé, Louis Daguerre and Henry Fox Talbot. See Appendix II, pictures 1 and 2. 31

“immortalize in time the exterior form of things” (524), implying the fact that the notion of realism in painting was interwoven with recording reality as faithfully as possible. When photography singlehandedly defeated painting in this task, the latter passed through a crisis of identity, that eventually led to painters developing and experimenting on images beyond the reach of the photographic art of the time. This experimentation actually relieved painting from the burden of imitating reality, enabling artistic genres like impressionism to flourish by breaking the bonds of the institutionalized “academic” clichés (525), and in fact the bonds of forced realism itself.6 After all, Niépce’s invention had practical rather than artistic purposes:

Niépce was actually interested in lithography, and his experimentation aimed at satisfying that interest of his, rather than producing a new art.7 People were not in search of such a thing anyway; if anything, by derailing it from its long tradition of realism, photography did painting the favor of liberating its expressional potentials.

But the ancestry line of photography is much older that the nineteenth century. Niépce’s breakthrough was simply the crossroads where these innovations clashed with the ancient strive towards realism, and specifically from the moment that this strive picked up pace from as early as the Renaissance. The quest for realism that ignited the technological progress towards photography was the same as the one that inspired Filippo Brunelleschi to understand the structure of linear perspective, and Leon Battista Alberti afterwards to set out the mathematical rules

6 Gombrich uses Édouard Manet as an example of this break with tradition. Manet and his fellow radical artists believed that traditional methods of painting, which were thought to have achieved pictorial accuracy in depicting the world, were actually false, in fact defining realistic depiction by imposing their rules on it, rather than serving it (513). Gombrich eventually claims that “artists were forced more and more to explore areas where it was impossible for photography to follow. In reality, modern art would not have been what it is now without photography pushing it to that direction” (525). 7 David Travis notes about Niépce that, “[s]ince he lacked the artistic ability to draw his own pictures, he employed light sensitive varnishes to copy pictures already engraved” (15). 32

that define it, both of them in the Fifteenth century A.D. (Gombrich 185; 217).8 Even though Renaissance realism and Niépce’s breakthrough used different tools and equipment, they were essentially motivated by the same goal: the technological production of images as close to reality as possible. The mastery of linear perspective is a landmark in art history, perhaps even greater than the camera obscura – already an ancient invention by then – as it revolutionized the way the world was depicted in pictures: artists started applying it to their paintings in an attempt to revive, as Gombrich notes, the art of painting as a whole. Here, a conceptual leap needs to be made, from this turn in Western art, all the way to the invention of cinema: cinema, as the offspring of photography, is established consequently and inevitably on this entire tradition which carries and combines the foundations of Western painting with Niépce’s technology, sharing with both of them the same line of artistic and technological ancestry which was gradually built by the quest for realism. André Bazin recognized these common origins of cinema and painting; he saw it as the “need for illusion” that has haunted art ever since the

Renaissance (11–12).

Where does this equation leave digital cinema? One may reasonably argue that digital technology broke away from the tradition of realism because of its difference, as regards both the tools used and the resulting image. Nevertheless, the opposite is true: in this chain of developments in imaging, digital cinema is the first link that consciously reverts to the past for technical solutions as regards the realistic credibility of the produced image. William Mitchell in The Reconfigured Eye, for instance, includes the use of software that uses three-dimensional graphics to create

8 See Appendix II, picture 3. 33

objects or scenes as one of the basic methods by which a digital image is obtained;9 this kind of software is designed based on the mathematical rules of perspective (6).

This way it abides by the same original tendency of realistic depiction that motivated the development of all pictorial arts of the Western tradition ever since the 16th century, a tradition which culminated in the 1:1 realism that was later on attributed to photography as the first absolutely successful mirror of reality. Mitchell also contradicts the argument made by Edward Weston, the American photographer, on the contrast between the “workability of a painting” and the “closure of the photograph,” regarding the openness of the image to manipulation; the author believes that in the digital era it is easily understood that exactly the opposite holds:

So the art of the digital image cannot adequately be understood as

primarily a matter of capture and printing, as Weston conceived

photography. Intermediate processing of images plays a central role.

Computational tools for transforming, combining, altering and analyzing

images are as essential to the digital artist as brushes and pigments are

to a painter and an understanding of them is the foundation of the craft

of digital imaging. (6–7)

Mitchell’s argumentation clearly demonstrates the fact that the digital image is inconceivable without taking older media into consideration, as, he concludes, “the digital image blurs the customary distinctions between painting and photography and between mechanical and handmade pictures” (7).

9 The other two methods are (i) the use of “some sort of sensor array” that captures images, e.g. the sensor of a digital camera, and (ii) the use of a computer cursor to assign color values to arbitrarily manipulated pixels, as in “painting” software. In all three cases, the digital image requires that some sort of value is assigned to a grid of pixels (Mitchell 6). 34

Lev Manovich reads Mitchell closely and provides further arguments that illustrate the way in which digital cinema re-invents past technologies. Starting from

Mitchell’s claim that paintings and digital photographs alike share an “inherent mutability” which eventually eliminates their differences, Manovich expands this argument to include cinema:

Since a film is a series of photographs, it is appropriate to extend

Mitchell’s argument to digital film. Given that an artist is easily able to

manipulate digitized footage either as a whole or frame by frame, a film

in a general sense becomes a series of paintings. (The Language of New

Media 304).

Clearly, therefore, digital movies are closer to paintings than they are to photographs, yet still retaining the artistic and representational drive that created photography in the first place. With the inevitable application of mathematics in digital imaging, cinema firmly re-establishes the usually ignored or neglected bond it shares with Renaissance painting: the intention and the methodology are the same; it is the hardware tools that have changed, from paper and pencil to silicon chips. In both cases the outcome is images of things that look exactly as they would if they were real, only they are not. It so happened that perspective, a technological rather than an artistic achievement, has proven too hard to die, perhaps due to the constant and inherent effort of man to comprehend the universe in one or another form of measure. The leap that we were called to make is thus summarized as follows: digital graphics actually exist owing to Alberti’s treatise.10 Under this light, and contrary to what is often insisted upon, it is important to underline that the

10 See Appendix II, pictures 4-6; also, DVD-ROM, video file 01. 35

development of digital movies is actually not a deviation from the photographic origins of cinema, but a circular re-invention of its origins. Seen through this viewpoint, digital cinema gradually reveals one by one all the ways in which it is closely related to aspects, methodologies, and entire trends in the history of imaging.

Renaissance and digital imaging technologies even share the same shortcoming of being unblemished: the fact that the digital image is synthetic eliminates any traces of a recording apparatus that affects the image quality in one way or another.11 As a result, images actually appear to be too perfect to look real, at least in relation to what spectators understand as real based on their experience of the world. Alberti’s guideline on perspective was also similarly fail-proof: because of that, though, the method of perspective is a perfect but essentially deceiving image of an imperfect world, with lines that are always absolutely straight and trees that all have the same height. As William Mitchell implies, it actually came to speak only for itself instead of the objects it was intended to depict:

Perspective construction was reduced to an objective process that, if

followed faithfully, was guaranteed to produce accurate results. Like a

photographer, the perspective artist can control the image by varying the

parameters of station point, direction of gaze, angle of view, and

distance and inclination of the picture plane relative to the station point.

(117)

11 Despite the fact that this kind of alterations of the image in analog recording are often considered a pleasant aesthetic quality, nonetheless they are still degradations of the image in strict technological terms. But most importantly, as generations of spectators have consumed analog images, they have been involuntarily trained on what the recorded image should look like based on the analog mode. 36

The artificial elimination of all anomalies in a mathematical illustration of the world becomes too conspicuous, with the consequent risk of turning flawlessness into a flaw. This is exactly the problem that digital designers also try to hide using software enhancements and various virtual filters that mimic analog recording media.12

Besides, as Jay David Bolter and Richard Grusin note, there is still debate whether perspective is yet another arbitrarily designed system of Western pictorial tradition, which, as such, only works under specific social and cultural circumstances, or it actually follows the “intrinsic principles of the human vision and Euclidian space”

(72). Social constructionists aspire to the former position, asserting the artificiality of perspective in relation to the human mind; on the contrary, several art historians and psychologists are in favor of the latter position, and believe in such an intrinsic relation between perspective design and the mechanisms of human perception (72).

Despite the obvious dead-end, it is most probably true that in CGI cinema, especially in the mainstream fantasy genre, the audience probably cannot be expected to respond e.g. to a high-tech CGI setting without having been exposed to an analogously high-tech computerized society.13

Partly due to these shortcomings, and partly due to a kind of association between reality and the analog medium that reaches the point of brainwash, the move from analog to digital in cinema is often seen as a suspicious shift, from recording reality, to manipulation of reality, in a way similar to the much debated contrast between vinyl records and CDs in the ’90s. The aesthete advocates of the

12 See Appendix II, pictures 7 and 8. 13 There’s still a historical paradox that should be noted here, though: oddly enough, the Renaissance artists like e.g. Masaccio who used numerical science in the form of perspective to make their work look more convincingly “real” were not so ardently accused of compromising the art of painting, as it happens today with the fear of the digital medium intruding cinema and photography. 37

analog medium often condemn digital innovations roughly for two reasons; first, the easiness and speed of digital production is believed to undermine the quality standards of imaging, as the painstaking analog procedure is considered to be the normative pillar of the truly creative photographic art.14 Second, and most important, digital imagery in movies threatens, for some, to overthrow the capacity that cinema has to mechanically and objectively record reality, a capacity inherited from photography as its definitive feature, by replacing it with the composition of completely artificial images. In defense of the digital medium against these considerations, it needs to be clarified that digital technology was not an external factor that brought changes to cinema; the problem with such an assumption would be that it finitely ties the essence of cinema exclusively with analog modes of cinematic imaging, which is hardly true: it is obvious that, in the wider history of pictorial realism, cinema was the outcome of an idea and a line of progress much older than Niépce. In fact, as this history shows, analog cinema was simply another stage in the long lasting process of technological evolution in imaging, which, of course, did not always include lenses, celluloid and silver compounds. On what grounds, then, should digital cinema be cast out from the same tradition?

This kind of distrust against digital cinema is caused by uneasiness against an art which is being transformed to a point where it is too elusive to define and too difficult to place in the same line of tradition as its predecessors. But, in truth, the latter task is not as demanding as it appears; after all, digital cinema is simply a re- visiting of the same theme of rivalry between media in the history of imaging. It

14 Walter Benjamin in his landmark essay “The Work of Art in the Age of Mechanical Reproduction” was among the first major thinkers that condemned quick reproductions of works of art on the premises that the latter lost contact with the time and place of their creation, a feature that gives then their distinct and unique aura. 38

seems that art history is now repeating itself; digital technology changes cinema, the technological offspring of photography, from within. Just like in the nineteenth century photography, the art of “recording,” pushed painting, the art of “creating,” into a search of identity, today digital cinema, which shares impressively many features with painting, practically avenges painting for the crisis it underwent, by haunting and compromising the originally recording nature of film, thus forcing the entire art to re-invent itself. In other words, digital image completes a circle that justifies technological evolution in imaging for the sake of pictorial accuracy. And, as technology changes and evolves, so does this concept of pictorial accuracy.

2.3. Technologies of Reality

2.3.1. Technology in Aid of Realism

Digital cinema has been received by many as a revolution, when, in reality, the coincidence of art and technology is hardly news. After all, perspective in pictorial art was basically a technological rather than an artistic innovation. Many theorists agree on this observation, providing various aspects that prove it to be more diachronic than it seems at first. Amy Ione, for instance, uses the example of stereographic photography to stress the fact that science and art, although not actually overlapping, indeed had common areas of “contextual impact:” photography, for one thing, was simultaneously but independently developed by artists, i.e. Niépce and Daguerre in France, and scientists, i.e. William Fox Talbot in

England (86),15 a fact which shows a common interest on the matter coming from

15 Niépce is actually credited with inventing what is considered to be the first real photograph, although his actual intention was simply to find a way to copy engravings because his was not skilled in drawing. Daguerre entered the picture when he partnered with Niépce, probably in search of 39

apparently opposite directions. For Ione, this parallel interest in visual arts and technologies, from the evolution of optics and photography to the emergence of modernist painting and beyond, reveals a logical and contextual pairing between the two as parts of “a larger, more fundamental transformation occurring within

Western culture,” i.e. the rapid “modernization of vision” (88), a note which implies a much wider influence on various aspects of the human life. In the same spirit,

Roger Malina insightfully notices a similar connection between science and art in recent times with the invasion of digital technology in art, underlining that the introduction of computers in art is a novelty that simply repeats art history, as it brought changes in media pretty much the same way that photography had also done to pre-existing visual arts when it was introduced:

[T]he introduction of the computer is affecting art-making in pre-existing

or traditional art forms. Just as the introduction of the technology of

photography had multiple and profound effects on painting, so the

computer is affecting pre-existing art-forms. The computer is leading to

change both in static forms, such as painting, photography, sculpture,

poetry and literature, and in time-based art forms, such as kinetic art,

film, video, music, dance and theater. (33)

chemicals for recording the images that he used to improve his dioramas, a popular show in which “transparencies were transformed by the play of unseen mirrors, blinds, and colored glasses” (Mayor 241). When their partnership ended abruptly 4 years later with Niépce’s death, Daguerre continued where they had left off together, perfecting Niépce’s invention and finally presenting it to the public in 1839 as daguerreotype, which became the first commercially successful photographic process. Talbot, on the other hand, had no immediate interest in art, focusing instead on scientific experimentation for its own sake. His own breakthrough was the photographic process of the calotype, quite different than the daguerreotype, independently developed but also presented in 1839. The feature that eventually made the calotype prevail, despite the fact that the daguerreotype had better results, was the ability to reproduce the image using the negative print on the plate originally used for taking the picture (Mayor 241–242). 40

Malina thus supports that art forms may have less obvious affinities with the past, yet these affinities prove that no art form is really born out of thin air, no matter how different it may look compared to its predecessors. The introduction of new technologies does not create new art forms, even if it might seem so, but simply affects pre-existing ones to one degree or another. Eventually the offspring of such interaction between the two is erroneously received as a novelty, simply because the line of succession between the original artistic practice and its technological revival has been lost with time, or the original practices are transformed by the new tools beyond immediate recognition.

Despite the obvious impossibility, or even irrationality, to consider the history of imaging without the contribution of both art and science, still the introduction of computers in visual arts is being met with skepticism. A key reason for this is the elevated role of the “machine,” as a wider concept, which presumably threatens to debase the process of creation and, eventually, eliminate the importance of the artist. The relatively recent introduction of digital graphics in visual arts has made this fact more obvious, as computers seem to be attracting far more accusations of alienating the human factor than paint brushes or film cameras ever did in their own time.

Malina addresses this issue, commenting on the function of computers in shaping contemporary visual culture and art in relation to the role of the artist. He acknowledges that computers are primarily tools for manipulation, not of creation or representation; he notes that the artist still has, and should have, the central role in art, but now that computers are the new tools it has become apparent that artists have been left behind, so to speak. Malina further notes that in digital arts, unlike in 41

traditional plastic arts, the artwork is “not directly accessible to the human senses,” i.e. it is physically separated from its medium; this trait is common in cinema as well, and already creates an affinity between digital arts and cinema or photography (33).

This makes it quite easy for the digital medium to absorb the latter or vice versa, but at the same time it also detaches the artwork from the artist even further. But this detachment between artwork and artist probably makes little difference, anyway, and the reason is simple: the role of the artist was never part of the equation when computers were starting to be developed. Despite all these changes in art that digital graphics have made possible, computers are still tools, “not developed with the specific needs of artists in mind;” the artists’ involvement in the course of this technological development has been very limited (Malina 35).

Malina seems to have a point here. As digital technology transforms the traditional codes of creative conduct, artists generally find it difficult to accommodate themselves in the new conditions. Exactly like painters in the past, digital designers have no part in the development of new technologies, be it new paints or brushes, or computers. Therefore when the time comes to use in art what was originally designed to be a tool for work without any specific artistic intention in mind, artists can do little more than simply take the palette of functions and capacities of this tool for granted, a fact which eventually limits the artistic potentials of the computer (Malina 34). The direct consequence of this situation, according to

Malina, is the fact that, although technology has advanced, artistic creation has been

“short of revolutionary” and has exhausted its practices many years ago (34). Malina implies that artistic creation needs to be as advanced and cutting-edge as the technology that now supports it; as he claims, it is pointless or even wrong to apply 42

new technology to worn out artistic themes, instead of using the technology of the past, on which those themes were after all established (34). In other words, the fact that the digital means relies on visual technologies of the past should not be a reason for digital artists to reproduce common themes, but an opportunity to create completely new visual results. Malina rightly believes that technology in fact dictates artistic practice, based on its features or its limitations, and computers should not be considered an exception; cameras reached their full creative potentials only when artists became familiar with them, and the same thing will happen, or at least should happen, with computers as well. Nevertheless, Malina here seems to be overlooking an important side effect of digital technology: one of the characteristic features of the digital is its morphing ability, in other words its capacity to imitate all analog practices, alongside creating new ones of its own. This feature is precisely what enables digital media, and digital cinema in particular, to actually archive its own self by collecting, reproducing, celebrating, and thus elevating to the status of cult, visual products from the analog era that are worth salvaging. Indeed, as Malina notes, the artist needs to be more consciously involved in the design of the tools that he will inevitably come to use; but this conscious approach needs to be in harmony with the essentially archiving nature of the digital.

On this relation between technological tools and art, Malina draws on John

Andrew Berton who focuses on cinema, specifically on the massively important role that technological advances and innovations have played in the development from analog to digital movies. Berton notes that “[n]either computer, nor camera, was created with artistic interests in mind, but both were soon directed there despite their practical applications elsewhere” (6). Berton underlines that digital cinema is 43

received in the same way as were early films, i.e. with more attention being paid to its impressive technological capacities rather than its creative potentials, since “both the creator of the images and the impartial viewer are more interested in process and function than in content and concept” (5). Berton acknowledges that, although filmmakers, critics, and theorists eventually adapted and took new analog technologies and their artistic impact into consideration, digital movies produce a polarized debate: on one hand, a movie may be hailed only as an impressive technological achievement, with its conceptual or artistic parameter completely overlooked; or, conversely, it may be rejected on exactly the same grounds, i.e. as a visual firework, a short-lived spectacle void of any real creative grounds (5). Berton is skeptical towards both these approaches, claiming that the wiser way out of a dead- end like this is to view computers first as tools in general, and then as tools adopted by artists. In both types of media, regardless of the decades between them and the tools used in each case, it has always been the “unexpected realism” of images that ultimate won audiences, and, in a sense, rather undermined the effort of the artist as a creative mind behind those images; for Berton, it is precisely this neglect of artists’ role that has subordinated content to spectacle in the first place (6).

Laura Kipnis supports that the issue is quite more complicated than a simple technology vs. art debate, and adds social and economic parameters to it. Between the two extremes, i.e. the fear of the disappearance of narrative as we know it, and the skeptic opinion that this fear is simply “narrow technological determinism,”

Kipnis stresses the fact that such technologies are social, in the sense that whatever impact they might have, this impact is always confined and controlled by the “social 44

institutions and economic forces” that these technologies serve (597).16 One of the main reasons for this techno-fear, for Kipnis, is the fact that traditional film theory has made the mistake of treating film in an elitist way, consequently separating it from other visual narrative genres, like video: film has been considered more

“artistic” because of its painstaking photographic origins and objectives, in contrast to the easiness of technological gismos available to video, which presumably threatened to obliterate the noble art of cinema.17 Kipnis shows that such a separation not only proves oxymoronic, given the vast technological resources not only available but indeed necessary to moviemaking, but also seems almost ridiculous now that that digitization has made possible the absolute convergence of all kinds of media. Kipnis rightly observes that the ideologies and financial parameters that drive the cinema industry are too powerful for anyone to blame the change simply on an art-vs.-technology debate. From production, to casting, editing and distribution, digital movies are essentially cheaper and easier to handle, just as photographic portraits once were, when compared to painted ones.

Artistic innovations in general have benefited from technological advances in various ways over the centuries, and cinema in particular, instead of simply benefiting from technology, actually relied upon it. Cinema is perhaps the only art that was developed because of, and based on, the technology that preceded it. Both the third dimension and digital technology were conquests of science that later benefited art: Alberti managed to harness the mysteries of the realistic image by

16 Kipnis here reminds us that such concerns probably emerged when other technologies made their appearance, like sound or television, causing similar or analogous discussions. 17 In respect to this separation, Kipnis adds a very characteristic quote by a film philosopher, whom she does not name: “In shooting film you are shooting on precious metal, that is, silver. With video technologies you are shooting on silicon, which is essentially dirt” (601) 45

tying it down to rigid mathematics, thus using the language of science to speak of the visual experience of the world. His method consciously viewed the world as a set of principles possible to be deciphered. Since the image could not be mechanically recorded, perspective was devised as a recipe to reproduce it. On one hand, the discovery of perspective was such a major breakthrough, that it actually prescribed the methodology of realistic art for the better part of the last five hundred years or so; and on the other, and most importantly, it signifies a turning point in art history from which realistic depiction was associated with the practices of mathematical quantification and structured composition. And despite the fact that this change was brought about inside the realm of realism, it was formalism that took it to its full potentials, as we shall see in the following section.

2.3.2. Collage and Form

The realization emerging from the historical study of realistic representation is that Alberti’s breakthrough was in fact the thing that art purists of realism would actually consider an abomination: that, in the course of the Twentieth Century, it became apparent that the realistic scopes of art were actually more adherent to principles of formalism. Of course, such statements are too dangerously simplistic to cover the entire range of realistic art, nor is the intention of the present study to go into that direction. Nevertheless, as regards specifically the history of film and the ancestry line of digital imaging in cinema, it becomes clear that the affinities between synthetic photorealism and formalism are way too obvious to be ignored.

The emergence of formalist cinema in the early decades of the Twentieth Century was yet another summit conquered with the combination of art and technology. 46

Formalism is a crucial chapter in the historical evolution of cinema, somewhere between the emergence of perspective realism and digital cinema, as it provides invaluable insight into cinematic techniques and theoretical frameworks that now acquire new meaning under the light of digital graphics and specifically composition in contemporary cinema. As archiving and assemblage are essential to digital cinema, the creative mastery of composing together visual elements seems far from extinct.

One of the greatest affinities between formalist and realist cinema is the fact that both sprang out of earlier forms of art which had a special relationship with technology. The aesthetics of the “machine” on which formalism is founded started with futurism that marked the art of the early twentieth century in Russia; just like it happened with Renaissance realism, once again geometry became a tool and an inspiration, this time for futurist painters. Art had to adapt to a radically new conception of life and viewpoint of reality; and cinema, like photography had done in the past in its quest for pictorial realism, followed this trend by putting its wondrous technological apparatus to use. Futurism sprang well before the Russian Revolution, and served it for years, setting the foundations for the development of the montage style: a methodology of formalism as “a new art, appropriate to the machine age” using “subjects from modern life” and developing montage as an effective technique of “shocking juxtapositions” (Bordwell “The Idea of Montage” 11).18 Technology, on

18 According to Bordwell, the momentum that futurism gained with the supremacy of the Soviets reached its peak around 1919-1920, when futurism had to work out its position in the already stabilized, by then, political and social environment. Eventually, the traditionally elitist futurism was attacked by the proletariat itself, and split in two parts: those that advocated an objective art, free from ideology, e.g. Kazimir Malevich, and those that envisioned an art that is accessible by all people, and uses industrial materials and aesthetics, e.g. Vladimir Tatlin. Eventually, the latter fragment formed the Constructivists, who took over around 1922; constructivism is credited with theorizing, 47

the other hand, contributed to the formation of this entirely new approach to art both with its capacities and with its shortcomings.19 This formalist urge towards a re- configuration of artistic practice emerged right when photography and, consequently, cinema started breaking away from the tradition of artificial imitation of realism as it had been prescribed by the application of imaging technologies like perspective. The increasingly independent photographic arts, as discussed earlier, laid claim to a faithful and unmediated recording of reality, practically ignoring the fact that this reality was no less mediated than before: the role of the apparatus of the camera along with the distortions that it also caused was easily swept under the carpet.

The montage cinema of the formalists, despite its innovative and revolutionary nature, seems to have been far more respectful and acknowledging towards its technological and artistic origins than its realist sibling proved to be.

Unlike the realists, that forgot the technologically mediated ancestry of the camera equipment, formalist montage embraced technology. This forthright preference for the machine aesthetics was the real innovation, the avant-garde,20 which came in

developing and applying montage style (“The Idea of Montage” 13). Despite this, futurism can be safely considered the source of montage, both in terms of inspiration and methodology. 19 Specifically about the latter, Joyce mentions that soviet montage cinema was actually born out of the scarcity of both available film stock and cameras: in a creative need to keep the propaganda alive and the public entertained amidst political and social unrest, there were moviemakers that “worked on re-editing existing films (often European/American films and old Russian newsreels) to make them conform to the values of the Soviet state” (334). It should be noted here, though, that David Bordwell downgrades the role that the shortage of film stock played in the evolution of formalist montage (“The Idea of Montage” 10). 20 It should not be overlooked that the avant-garde was not merely an artistic revolution, but the reflection on art of a political radicalism that simmered across Europe for decades. As important as political development and ideology may be in relation to the avant-garde, the purpose of the present study is to research its technical and aesthetic affiliation to contemporary digital cinema. For a thorough and elaborate analysis of the political background behind the avant-garde, see Andreas Huyssen’s essay “The Hidden Dialectic: Avantgarde – Technology – Mass Culture” included in his book After the Great Divide: Modernism, Mass Culture, Postmodernism, as well as in Post-Impressionism to WWII, a collection of prominent essays on art history of the Post-Impressionist period. 48

direct contrast to realist trends which always tried to hide construction and mechanics underneath resemblance and flow. Lev Manovich attempts a very straightforward linkage between the avant-garde of the beginning of the Twentieth century and the contemporary digital avant-garde, which he directly associates with software. His first crucial observation is the fact that even the original avant-garde sprang out of what used to be new media in their own time, i.e. film, photography, etc. (“Avant-Garde as Software” 1–2). The period roughly between 1915-1930 actually exhausts the entire list of crucial innovations in the codification of visual communication, like montage, classical filmic language, etc., which have been recycled and perpetuated until the present day in all forms of mass culture thereafter; as Manovich puts it, “[m]ass visual culture only pushes further what was already invented, ‘intensifying’ particular techniques and mixing them together in new combinations” (2). In other words, in contrast to earlier cultural periods which introduced new “expressive vocabularies,” we use those same ones today with the use of computers to create our visual culture, essentially recognizing new media as old media (2–3). This recycling of the expressive vocabularies of media not only confirms, but also relies upon the archiving character of the digital.

The vision of leftist avant-garde artists, Manovich concludes, as well as the techniques devised to bring that vision to life during the less than two-decade period in which it officially lasted, is now realized in the assembling and synthesizing functions of computers, as well as in the ways they interact with users. In his study of early-twentieth century avant-garde techniques, Manovich coined the term visual atomism. This term, for Manovich, encompasses the possibility to construct a complex message from a collection and assemblage of distinct particles, the function 49

and impact of which is known in advance; this way the response to the complex message can be predicted more easily, as each of the constituting elements will be understood by the receiver, who, however, is required to employ cognitive capacities to re-construct it mentally.21 Visual atomism returned with much more effectiveness in computer media; however, it was transformed from a theory of visual impact to a model for a high-tech communication platform. For example, images are broken down to pixels for easier handling and transmission, 3D space is comprised by elemental polygon structures weaved together, and hyperlinks multiply the layers of information available to the user/viewer (“Avant-Garde as

Software” 5). Manovich goes into great pains to illustrate the ways in which digital imaging is a more self-conscious, more effective, and thus more successful application of the basic technological principles of the avant-garde. And, to a great extent he succeeds in doing so. His theory on New Media is a very convincing account on the ways techno-cultural aesthetics succeeded twice in the same century in revolutionizing the very concept of art.

In the same spirit, Andreas Huyssen writing on the avant-garde underlines the tremendous impact that twentieth century technologies of media and transportation had on mass culture, in the avant-garde’s attempt to overcome the art/life dichotomy and make art productive in the transformation of everyday life

(5). Huyssen observes that technology was the key influencing parameter in the development of the avant-garde, and the role that photography and cinema played in this process was catalytic, due to their essential relationship with technology:

21 Manovich mentions that this concept had been experimented upon even earlier, in late-nineteenth century painting. Georges Seurat, for instance, studied the impact of visual stimuli and applied them to his paintings (“Avant-Garde as Software” 5). 50

The invasion of the very fabric of the art object by technology and what

one may loosely call the technological imagination can best be grasped in

artistic practices such as collage, assemblage, montage and

photomontage; it finds its ultimate fulfillment in photography and film,

art forms which can not only be reproduced, but are in fact designed for

mechanical reproducibility. (9)

In fact, Huyssen adds, technology in the late nineteenth – early twentieth century may very well have been the very source for inspiring the avant-garde instead of simply shaping it, both through the gradual establishment of the “aestheticization of technics,” and the awe emanated by the beastly war machines of WWI (10). There are certain analogies between these conditions and the digital era. First of all, over the last 50 years of so, technology has been promoted as a factor of well-being in western capitalist societies, especially during the post-WWII period, when a sense of return to normal life was restored; this return was deeply associated with family entertainment and an abundance of appliances and tools, both of which were introduced to the family as smaller or greater technological innovations. And second, warfare has been prominently underlined by digital technologies as indication of supremacy and expertise, with Ronald Reagan’s Strategic Defense Initiative (SDI) project and the Gulf War standing out as primary examples. This entire era, from the early 1950s until the present day, has worshipped technology, leading to the role of computers in everyday life being gradually centralized. Consequently, art has also been affected by this high-tech society, with examples like video art, industrial and electronic music, animation, etc. gradually entering the creative scene. 51

Technology was the heart, but quite possibly also the Achilles’ heel of Soviet montage style. Mark Joyce notes that the advent of sound films with their technical complexity marked the decline of the genre (357).22 But the decline of formalism was probably not just because of that. The original avant-garde perhaps failed partly because it became too self-referential, with technique practically speaking only for itself at the later days of formalism, eventually falling into the fallacy of self- recycling; and partly because it forgot the internal need of the viewer to rely on linear narratives in visual storytelling.23 Especially this latter element is something that realist cinema, on the contrary, always remained aware of. The claims of the avant-garde to a viable notion of realism was that, if reality is truth, and truth is related to ideology, then formalism is equally “realistic” as realism, since its goals are driven by ideology. The digital avant-garde seems ready not to repeat that mistake: it retains the technical methodology of assemblage inherited by formalism, and at the same time directly addresses the mass culture industry, and specifically the blockbuster tradition, which, narrative-wise, is founded on principles of realist cinema. Formalism experimented with film form exactly the same way as digital cinema does, only the drive behind this experimentation is neither ideology-driven anymore, at least not as explicitly as it used to be, nor confined to politics. After all, the technology of digital cinema may very well be the unmaking of traditional realism, just as technology of sound was once the unmaking of the avant-garde in the past. Not only do images have less and less resemblance to the actual world, but

22 In order to record sound in a movie, it was initially necessary to use non-movable cameras and fixed microphones. Formalism could not reconcile these needs with its use of fast camera movement and rapid editing technique (Joyce 357). 23 The relationship between narrative structure and the human mind is thoroughly discussed in Chapter 5. 52

the implied manipulation of the rendered image that the digital medium connotes, leaves a distrustful taste to the viewer as regards the equivalence of the image to reality. This distrust should have been around for centuries, alongside various technologies of imitating reality, but somehow made it this far unnoticed; digital imagery has simply made it obvious.

2.4. The Digital Avant-Garde

A digital avant-garde emerges, the avant-garde of the beginning of the twenty-first century, which, like the avant-garde of the early twentieth century, is too problematic to theorize simply because theorists try to fit it into an existing model. This new avant-garde, however, is the first that revolutionizes art, not because it overturns the established codes, but because it reconciles them – and this may be too hard to grasp. It uses essential elements from both the main branches of cinema, formalism and realism, firmly tied together with digital technology.

Manovich, for instance, reviews this novel avant-garde trend and concludes that, on one hand, the original avant-garde never disappeared, and on the other, the revolution of new media was neither a revolution, at least in the way it is often thought to be, nor were the media used in it essentially new. The same, radically new techniques that “were harnessed to help the viewer to reveal the social structure behind the visible surfaces, to uncover the underlying struggle between the old and the new” revive today as the imaging and communication tools of a computer age (“Avant-Garde as Software” 9). Manovich thus shows us how wrong it is to try to theorize new media, or even simply experience them without taking into consideration the fact that they belong to the same lineage as old media. 53

This lineage is very easy to trace by taking the technical aspect as a main axis, whether we may call it technology, machine aesthetics, epistemology, science, etc., and following its level of interference with culture through the decades. One of the features of post-modernism was the fact that it facilitated that interference.

Especially after the 1950s, technology becomes fetishized, revealing itself as completely on a par with postmodernism; and the latter becomes a symptom of a consumer society that “gets rid of the avant-garde’s original politics and, through repeated use, makes avant-garde techniques appear totally natural” (Manovich,

“Avant-Garde as Software” 10). In the fertile ground of a society traumatized by

WWII, struggling to leave the old world behind, and craving for technology and innovation as symbols of rejuvenation of the shattered world, post-modernism planted technology, stripped it of any hard-core ideological weight, and integrated it into everyday life. The sense of prosperity provided by technological gizmos was the glue that helped connect the fragments left behind by the war, without looking for grand ideologies for support. The new high-tech society was a society of assemblage, of bits and pieces put together in mass culture; the original avant-garde dream of implementing art and life through technology started to become reality, albeit in a way that original formalists would probably find detestable.

The original avant-garde mostly exhibited itself in architecture and cinema, and the digital avant-garde has not changed this. This is reasonable, since these are both aesthetic forms that have special affinities with technology and, more generally, with the notion of modernism, due to the fact that they are born and developed in parallel to technological evolution. Specifically about architecture,

Rutsky refers to the enhanced influence of the engineer replacing that of the 54

architect, a tendency already apparent by the late nineteenth century (77–78).24 The importance of this replacement of the aesthetician by the engineer is echoed in today’s digital avant-garde moviemaking and New Media practices in general: the role of the digital designer in CGI applications, such as 3D animation in movies, simulations, gaming, internet etc., is considered more essential in the creative process than that of the artist. Given the fact that today’s cinematic spectacle is aimed at the masses of pop-corn eating audiences, eager to spend two hours for escapist spectacular entertainment, it would not perhaps be too much to realize that the thinker is being gradually replaced by the nerd.

Besides all these similarities, however, there is an essential difference between the formalist avant-garde and the new digital avant-garde. The former heavily relied on the juxtaposition of images and stimuli in montage, in order to elicit specific impressions from the audience. On the contrary, the digital mass culture industry more often than not seems to be targeted towards movies that do not favor juxtaposition in the same sense. Despite their lack of similarities with real-life environments, digital movies are not meant to be disturbing: they are easily grasped, and tend to employ a kind of editing that does not really contradict the classical seamless narrative techniques of traditional realism. Huyssen stresses this obsoleteness of the element of shock, which used to be essential for the avant- garde, and its replacement with a kind of imagery in movies that patronizes vision instead of challenging comprehension:

Not only is the historical avant-garde a thing of the past, but it is also

useless to try to revive it under any guise. Its artistic inventions and

24 Rutsky reads this observation in the writings of the architect Anatole de Baudot in 1889. 55

techniques have been absorbed and co-opted by Western mass

mediated culture in all its manifestations from Hollywood film, television,

advertising, industrial design, and architecture to the aesthetization [sic]

of technology and commodity aesthetics. (15)

Yet, although Huyssen notes that the traditional avant-garde is indeed probably gone, this is not entirely true; what is indeed lost is the aesthetic impact of the montage techniques, but the techniques themselves are indeed still around, only they have been submitted and adapted to other purposes. The contrast between the socialist past and the capitalist present of these techniques, sharp as it may be, is irrelevant for the purposes of this study. The important thing to underline is the often neglected technical parameter: the avant-garde techniques survived and re- surfaced in the digital era, even if they had to form an unholy alliance with classical realist narrative in order to re-enter the culture industry. It needs to be pointed out, though, that the re-appropriation of both the realist and formalist view in the digital medium means leaving behind theoretical assumptions that have brought the two in a clash with each other in the past. The digital avant-garde utilizing both of them simultaneously may very well also mean utilizing none of them entirely, as we shall see in the following section.

So, is perhaps the new media avant-garde a hoax, a mere re-writing of cultural history with a different pen? Manovich says that there are indeed two equally important groundbreaking aspects in it, only they are not to be found in form. The first has to do with the fact that, whereas old media offered new ways of representing reality, new media is about new ways of handling information; and the second has to do with the fact that new media does not offer new views of the 56

world, but presupposes previously stored media and offers new ways of accessing them. In this sense, Manovich sees new media as meta-media or post-media, as it relies on the manipulation of pre-existing material (“Avant-Garde as Software” 11).

Most important of all, the element that underlines the relation between media, not just digital and soviet avant-garde but also those that are as old as the Renaissance, is their common denominator of reflecting the external world, one way or another, and with one technique or the other; as Manovich puts it, “despite these differences, they are all concerned with the same project of reflecting the world,” and the project remains the same regardless of the tools used (12). Culture today is therefore not in the process of creating media unseen before, as “it has enough trouble dealing with all the already accumulated representations” of the world; new media is all about

“media access, generation, manipulation and analysis,” and does not create any new forms but simply changes the ways these forms are used (13). Harnessing and explaining the selection of these forms, as well as the ways in which they are put to use, may provide us with far more insight than studying the relation of analog or digital media to reality; it may even show us the ways that the selection and usage of these forms shape reality on our behalf, as well as the reception of this reality by the human mind. Ultimately, they will show the ways in which narratives are dependent not simply on methods and tools, but on the way these tools take advantage of preconditions that enable some sort of “realistic effect” in the mind of the spectators, which turns out to be medium-independent.

57

2.5. Looking Through Worlds Bounded by Technology

2.5.1. On Windows and Frames

From a technological viewpoint, the digital avant-garde is indeed a recent reconciliation of the original clash between realist and formalist versions of reality. In close relation to that, and despite their ideological, stylistic, thematic, or other differences, there is one more thing that both realist and formalist cinema take for granted: the kind of reality they shape and serve to the human eyes is one that is separated from the real world by a rectangular boundary. It is the same boundary that cinema inherited from the framing of reality during the times of portrait making, a framing which was only recently standardized for cinema in several screen aspect ratios over the course of the Twentieth century. All of these various aspect ratios have indeed been rectangular and in a landscape orientation, to match the horizontal area coverage of the human visual system, but allowed no tolerance for extreme experimentation: the several aspect ratios that were standardized during the years of Classical Hollywood have been passed down fairly unaltered to the digital era, despite all other various changes in the technology of cinema.25

Therefore, the role that this border around the image plays in the perception of the cinematic world has been identified from the early days of cinema, and has not been left without attention.

25 There have indeed been a number of aspect ratios that studios tried out. The antagonism between companies urged them to a race for screens as big as possible, believing that those would be welcomed as signs of technological superiority, which in turn, would be translated to increased revenues. Several sizes have been introduced over the years, like VistaVision and CinemaScope; but the general tendency has always been to go from square to wide screen. From the initially square early films, to the standardization of the “Academy ratio” (1.33:1), and then to expanding the image to the left and right sides either with masking or with anamorphic lenses, aspect ratios in cinema have come a long way to reach today’s generally common 1.85:1 ratio (Bordwell and Thompson 183–184). 58

Why is this rectangular border so important in cinema theory? The reason is that the definition that will be assigned to the border immediately affects the status of the world inside it as well. Formalism and realism viewed it in a different way; this resulted in each of the two having different attitudes towards the status and purpose of the cinematic image inside it, in relation to the real world outside it.

Vivian Sobchack, for instance, refers to the role of the border in order to separate formalism from realism based on the way traditional film theory underlines the purpose of the film in each case: formalism, on one hand, uses the metaphor of the

“frame” to underline the “camera’s perceptive nature” and the positive imposition of the artist’s personal and ordered vision on the chaotic mass of recorded images.

Realism, on the contrary, uses the metaphor of the “window” to exalt the “camera’s expressive nature” and “perceptual purity and openness,” and welcomes the raw presentation of the world in images (15–16). Consequently, formalism saw the truth of cinema in the message that it can convey through “cinematic ‘language,’” i.e. with development of montage techniques; realism, on the contrary, relies on “cinematic being,” advocating that cinema is its own message which draws its self-sustainability from the power of mise-en-scène (15). The interesting point that Sobchack makes is the dependence of each system upon the other: two poles in a dual system, each defining itself by implying the existence of the other. And, even more importantly,

Sobchack notes that there has been emphasis on such a “dual poetics” which constantly feeds this polarization, instead of working towards a realization of a single theoretical system that would productively utilize both of them (15–16). This polarization, in other words, is actually sustained by the very system that describes it; simply put, the fact that one exists because of the other is a reflection of their 59

interdependence, and therefore it is obligatory of any attempt towards a unifying theory to take both into consideration.

Thomas Elsaesser and Malte Hagener also discuss this almost notorious distinction between window and frame in relation to realism and formalism respectively, but they do find similarities between them besides their differences. As regards their similarities, the authors first of all note that in both cases cinema is

“ocular–specular (i.e. conditioned by optical access), transitive (one looks at something) and disembodied (the spectator maintains a safe distance)” (14). It is obvious that these similarities are elementary and common between the two, as they are bound to the traditional cinematic experience in terms of the technology available and used in both realism and formalism. Apart from them, nevertheless, there are also similarities that are established on the content and artistic impact of each style. In this context, Elsaesser and Hagener state that both the metaphors of window and frame aimed at consolidating the status of cinema as an art, making more manifest its relation to the Renaissance ideal of appreciating art by setting some border around it, thus creating a distance between the artwork and its observer (16). This immediately creates a position for the observer; both realism and formalism require him to be a conscious perceiver, albeit with only visual access to the content of the artwork, the meaning of which he will negotiate using his consciousness and reasoning (16). Both models, therefore, see the spectator as completely disembodied,26 and aim at the ways in which he can respond to the

26 The authors follow Charles F. Altman in this similarity, and assume a critical stance to it. They believe that because of this similar attitude, both models erroneously neglect several factors that may affect cinema production, like technology, as well as its reception, e.g. “differences in human perception, cultural conditioning and cognition” (16). Because of these shortcomings, the imagined spectator in both cases is also imaginary, in the sense that he is an entity impossible to be found in 60

structure of the movie, the wholeness and coherence of which these models presuppose (16). It is obvious, therefore, that the structure of the movie, which reflects a purposeful rather than arbitrary way in which its content is presented to the audience, is not accidental, or at least should not be considered as such.

It is the different realization of this otherwise common starting point that differentiates realism and formalism; and it is also what causes their differences as regards the attitude towards worldmaking, a difference which, as we shall see, is not so difficult to surpass, given the right viewpoint. The window and the frame have inherently different qualities, which Elsaesser and Hagener summarize in the phrase

“one looks through a window, but one looks at a frame” (14). On one hand, the window implies transparency, in Kendal Walton’s sense of the word: it aims at driving attention away from the border, effectively making it invisible, thus providing a sense of direct access to the content of the image; the frame, on the other hand, accentuates the presence of the border, as well as the special arrangement of whatever it encompasses, which deliberately makes the latter seem more artificial:

While the window directs the viewer to something behind or beyond

itself – ideally, the separating glass pane completely vanishes in the act

of looking – the frame draws attention both to the status of the

arrangement as artifact and to the image support itself . . . On the one

hand, the window as a medium effaces itself completely and becomes

reality: he is “not only disembodied, but exists mostly for the benefit of the theory he or she is supposed to exemplify” (16). These observations are very interesting in relation to audience analysis but they exceed the purposes and scopes of the present study. Nevertheless, the issue of coherence discussed in this argument is crucial in the perception of the cinematic world, and will be deployed extensively in Chapter 5, albeit not from an ideological viewpoint. 61

invisible, and on the other, the frame exhibits the medium in its material

specificity. (14–15)

The digital avant-garde seeks for ways that sidestep these differences, not by ignoring themselves or the ideologies behind them, but focusing on the human visual perception as a parameter able to be scientifically analyzed and experimentally supported. In this respect, it can be used to explain the ways in which cinema both allows a sense of direct access to its worlds, and at the same time maintain its obvious status as an artwork. A reconciliatory and, to some, heretic argument such as this, passes through two axes: first, the mode of presentation of the cinema world to the spectators, and second the elementary science behind visual perception that allows it to work. Let us see them in turn.

As regards the first axis, the answer has been already given decades ago: the most successful mode of presenting the cinema world to the spectators is found in the classical mode of cinematic storytelling. The fact that the digital mainstream moviemaking today generally follows the same rules of continuity that had been established by the mid-Twentieth century by Classical Hollywood is by no means an accident: plain and simple, it is so far the only mode of storytelling that is so appealing to audiences, possibly due to the paradox of “an unmediated view (the window) requir[ing] elaborate means and codified rules (the frame)” (Elsaesser and

Hagener 18). In the context of the present discussion, its significant value lies in the fact that it actually verifies the reconciliatory function of digital cinema as regards realism and formalism, or window and frame:

Classical cinema keeps its disembodied spectators at arm's length while

also drawing them in. It achieves its effects of transparency by the 62

concerted deployment of filmic means (montage, light, camera

placement, scale, special effects) which justify their profuse presence by

aiming at being noticed as little as possible. A maximum of technique and

technology seeks a minimum of attention for itself, thereby not only

masking the means of manipulation, but succeeding in creating a

transparency that simulates proximity and intimacy. (18)

Extending Elsaesser and Hagener’s argument, it is not difficult to recognize the value of classical mode of narration for both cinema in general and for digital cinema in particular. This value is manifold; first of all, the classical mode forgives the stiffness of both theories, which is often too obvious, by revealing the ways in which they can indeed coincide. Second, it makes the role of technology more conspicuous, bringing it on a par with content in a more prominent way. And third, by doing so, this increased consideration of the role of technology in cinema allows the creation of a comfortable room, in which digital cinema can find not just a niche, but its rightful place in the lineage of all visual arts: Elsaesser and Hagener verify that the classical style of narration is directly linked to the tradition of Renaissance perspective, as it is characterized by “managing distance and privileging apperception principally through the disembodied eye” (20). Seen through this viewpoint, it is not only difficult, but completely irrational to leave digital cinema out of the history of imaging, both for its own sake, as well as for the sake of the comprehension and appreciation of the entire art of cinema itself.27

As regards the second axis, the science of visual perception that makes possible the success of continuity narration, bridging the gap between window and

27 The critical role of the classical mode of narration and continuity editing in bringing digital cinema in this lineage is extensively discussed in Chapter 5. 63

frame requires a more daring leap to be made, one which addresses the cognitive processing of the ontological status of the cinematic universe. Realism and formalism of course had different attitudes towards the latter. For example,

Elsaesser and Hagener refer to Leo Braudy’s separation of films into open or closed ones: open films are presented and recognized as segments of a pre-existing, changing reality, of which spectators only get a glimpse; closed films, on the other hand, present universes which only contain those elements that play a specific functional role (16).28 In Braudy’s own words, the difference may be rephrased more simply as one “between finding a world and creating one” (qtd. in Elsaesser and

Hagener 17). The authors here draw a parallel between window and frame on one hand, and open and closed films on the other:

[T]he window offers a detail of a larger whole in which the elements

appear as if distributed in no particular way, so that the impression of

realism for the spectator is above all a function of transparency. By

contrast, foregrounding the frame shifts the attention to the organization

of the material. The window implies a diegetic world that extends

beyond the limit of the image while the frame delineates a filmic

composition that exists solely for the eyes of the beholder. (18)

28 According to the authors, the universes depicted in “closed” films are elaborate compositions of elements that play a very specific role on the filmic composition as a whole. For example, films by Georges Méliès consciously make a self-reflective use of the same cinematic tricks and techniques; moreover, all elements in films by Fritz Lang, Alfred Hitchcock and David Fincher seem to be placed there by the invisible hand of some higher agent that controls the flow of the film to the very last detail; this is what makes each and every one of them necessary (16). This creates a dependence as well as tension, at the same time, between on-screen and off-screen space (16). On the contrary, the cinematic universes in “open” films are slices of a cinematic reality which seems “as if what it depicts might continue in much the same way even if the camera were turned off, and life would continue to ebb and flow beyond the limits of the frame” (17). Jean Renoir’s long, flowing takes and multitude of characters are cited as exemplary of this type: unlike “closed” universes, this kind of cinematic reality does not seem to be controlled by agents external to the diegesis (17). 64

The polarity is more than prominent, and indeed very recognizable even by students of cinema studies. Yet, despite the obviously diametric opposition between worlds in windows and worlds in frames, blurring this opposition may actually be proven to be easier that it seems. This blurring is the leap that needs to be made if digital technology is to be appreciated in the context of the history of cinema and other visual art technologies. To get solid answers, all one needs to do is ask the proper questions: why see the kind of cinematic world in each case as different from the other? Why use as criteria arbitrary definitions for cinematic universes, instead of using elements that are scientifically observed, and thus not only more objective but vastly more difficult to contradict as well?

The answer lies in acknowledging that cinema theories have indeed addressed the role of technology over time, but mostly in isolation: those theories, usually affected by one or the other philosophy or ideology emerging at the time of their development, have tended to neglect the relation between technology and the physiology of human visual perception. By overlooking visual science, traditional cinema theories have been asking the wrong questions: practically all of them seem to have generally assumed that the cinematic universe is in one way or the other separate from our own. Indeed it is, albeit only in physical or ontological terms; but that is not the case from the viewpoint of our visual perception. Humans do not have separate visual systems for real life and for watching movies, which would also imply different and interchangeable perceptual mechanisms for each situation, e.g. art vs. real life, with different responses to what is being seen. Our visual system precedes visual technologies, which have been developed based on the pre-existing merits and shortcomings of this system. The human perceptual system is one, having a 65

multitude of functions and adapting capabilities; our theoretical shortcomings in identifying the flexibility of the system should not make us forget the starting point: that the human visual system, despite the various culturally bound conditionings, such as the influence of Renaissance perspective in Western culture, is a single, common, standard and measurable mechanism, relieved from any ideological, philosophical or other constraints, an objective and solid ground upon which to build an all-encompassing theory of cinema. Why try to define the qualities of cinematic universes as separate from our own, when, in fact, our visual perception and cognitive comprehension actually treat them as extensions of the real world? The value of advancing visual technology, making cinematic universes especially of fantasy as unreal visually as they have ever been, and yet narratively credible as worlds, has revealed that our perceptual capacities are the same in whatever movie we see. This, and presently only this, makes the science of visual perception the most reliable candidate so far to provide solutions for questions regarding cinema comprehension. In the following chapters, a solution such as this has been selected, based primarily on the work of Hochberg and Brooks on visual perception. In the last section of the present chapter, we shall get a glimpse of how such a solution may work for a purely technological feature of cinema, i.e. the function of the rectangular frame.

2.5.2. Re-routing the Role of Visual Perception in Cinema Theory: Beyond the

Window/Frame Polarity

The metaphor of windows and frames has proven quite influential, as it is a versatile polarity that has been adapted to support a number of opposing theses in 66

cinema theory, practically none of which seems to have been established on absolutely objective grounds, as described above. Traditional artistic theories of cinema generally ignored the actual neurological mechanism of the human vision.

The contribution of cognitive science in cinema theory has done the exact opposite: assertions are based on experimental, measurable findings. The example of the window and the frame is characteristically vivid: realism and formalism consumed themselves into disagreeing whether the universe on screen is, or should be, a crude copy of reality, or a message conveyed under the agency of the moviemaker giving order to chaos; none of the two sides bothered to verify if either argument can be supported by human physiology of vision.

For visual perception, the scene contained in the human optic array needs to be measured according to what is being comprehended. Taking for granted that

Classical style ensures a proper continuous flow of information on screen, posing no issues of comprehension, we immediately stumble upon a very real and basic problem: all images on screen are distorted, except when they are viewed from a very specific viewpoint, which can obviously be occupied only by one person at a time; it is similar to the optics of vanishing point in Renaissance perspective having a maximum perception effect only when the viewer occupies a certain position. How, then, can realism claim that cinema should offer a glimpse of reality as-it-is, when it is technically impossible to do so, since everything on screen is not only 2D, but also slanted for everyone but a handful of “privileged” spectators that happen to be sitting at the center of the theater?29 And how can formalism claim that cinema

29 In fact, James Cutting notes that no spectator can actually occupy this position (“Rigidity” 325); hence, no spectator can possibly ever be in this presumed optimal position, thus making transparency practically nothing more than a chimerical aspiration of realism. 67

should be the reality of the message conveyed, when the message will inescapably be delivered in a distorted way, which is out of the control of the moviemaker, since he is in no position to predict all possible seating arrangements during screening?

Theorizing cinema now that technology plays such a crucial role in movie- watching cannot ignore the fact that the image may be delivered to the spectators in unpredictably altered ways. Therefore, in search for good candidate approaches to support it, priority should be given to theories that take into account the cruel but largely ignored reality of movie-watching conditions. Hochberg and Brooks do take this into account. In their analysis of visual perception during movie-watching, they agree that any picture, moving or still, is indeed “in some sense a surrogate for whatever is being portrayed,” given the fact that the optic array that those pictures present is identical in several aspects to that of the scene itself (“Perception of

Motion Pictures” 207).30 This condition, according to the authors, “invites a reasonably straightforward definition of fidelity:” the latter can be assumed to be a measurable, predictable, and, above all, reproducible quality in pictures; it essentially means that proper attendance to qualities such as resolution, contrast, grain, seating distance and position, etc. can make the image look as if indeed the spectator is looking at the scene from an open window (207). Nevertheless, this is not the case. According to the authors, experimental studies have proven that neither viewing distance nor resolution plays any decisive role in the legibility of the image; accordingly, it has been found that human perception can tolerate a viewing

30 Hochberg and Brook’s note here refers to analog pictures, which is not the same as digital ones: in CGI, that optic array may not have existed at all. But that does not invalidate their argument; as we have seen, the digital reproduces the analog to look “realistic,” so it essentially also reproduces the kind of conditions that would have been occurring if the scene had been shot in reality instead of synthetically put together. 68

angle that is significantly wider than the presumed optimal position at the center of the theater, allowing proper visibility and comprehension of the image without the need for screened objects and their motion to be distorted accordingly over virtual space (208). Cutting in his own work in the field has named this phenomenon La

Gournerie’s paradox;31 it refers to the “[p]reservation of phenomenal identity and phenomenal shape of objects in slanted pictures,” which means that distortions of perspective generally go unnoticed by the spectators (“Rigidity” 323). Cutting overviews possible explanations to the paradox, concluding that one of the key reasons that human perception can easily accommodate such deviations from the optimal position of viewing up to a certain extent is simply because it is faulty in its measurements in the first place: “the human visual system is sufficiently inexact in its local measurements of optical projections to tolerate small distortions” (“Rigidity”

333).

It is obvious that such findings have severe implications for any argument that makes assumptions on what spectators see and what they perceive it as.

Speaking of motion pictures in particular, Hochberg and Brooks arrive at a critical conclusion: “[w]e cannot simply define a motion picture, then, in terms of the fidelity of the motions for which it is physically a surrogate” (“Perception of Motion

Pictures” 208), since the area that motion will cover in the spectator’s visual array is quite different than the equivalent motion in real space and time. So much, then, for the window vs. frame debate; this metaphor simply cannot work as naively as cinema theorists thought it did for decades. Only this time it is the human visual

31 The name is derived from Jules de La Gournerie, who made the original observation in 1859 that “viewing a perspective picture from a point other than its composition point (assuming the composition point to be in the center of the picture) should create systematic distortions in pictorial space” (Cutting, “Affine Distortions of Pictorial Space” 306). 69

system itself that disproves misguided assumptions about the role of the rectangular frame, not one or the other theorist or approach. The digital avant-garde, appropriating the techniques of both formalism and realism, and fusing them in a technologically wondrous realization of the classical style, proves that not only it is beyond such oppositions, but also that it is closer to a more objectively scientific explanation of movie-watching.

From a cognitive standpoint, which is as straightforward as that of Hochberg and Brooks’, things are fairly simple and fully on a par with the classical style in digital cinema which fuses the goals of both window and frame. The frame is simply one of the key parameters that creates and maintains motion, which is the absolutely essential characteristic of movies. Consider for example a tracking shot, which is the same regardless of whether it involves real recorded action or CGI: an object being tracked by a camera moving in parallel to it, which is identified with the rectangular border, is essentially standing still, since it occupies the exact same position in the spectator’s optic array, and hence does not simulate the motion detectors that are built in the human visual system. It is the motion of the background that creates illusory motion for the object tracked: if the background moves to the left, the otherwise still object appears to move to the right (Hochberg and Brooks, “Perception of Motion Pictures” 227). Several other examples can be used, in all of which the border plays a crucial role in positioning the spectator, metaphorically as well as literally, not just in front of, but also in relation to the depicted cinematic universe and the action taking place inside it. Under this light, the frame becomes a stable and trustworthy point of reference for the perception of 70

motion and, ergo, the progression of action. It is responsible for creating the sense of motion on which cinema is by definition established.

In the following chapters a similar approach will follow. With respect to the origins of cinema as partly-technological, partly-artistic, the cognitive approach that is adopted will draw progressively more on the work by Hochberg and Brooks, as well as on other psychologists of visual perception, in order to provide an approach to digital cinema that is as objective as possible. The Classical style has proven to be a very good combination of several aspects of realism and formalism; now, digital cinema, pushing cinematic worlds further away from reality and thus forcing a greater attention to the artificiality of cinema, projects advanced visual technology as a reason to really start explaining how the classical style really works. The increasing detachment of CGI from the real world allows a much greater distance of spectators from the content of the cinematic universe, which is necessary for a more sober reflection on how cinema actually works for them. The digital avant-garde is thus nothing more – or less – than a perfect opportunity to explore cinema at its fullest. The CGI software becomes a mold in which digital cinema is created: an alloy of art and technology much harder to break than any of its components; a sum that is much more than its parts, as the Gestaltists would say. All what movie theorists need to do is respect it as such. 71

CHAPTER 3

RE-COMPOSING REALITY

3.1. The World in Fragments: from Continuous to Contiguous

The breaking down of cinema to its composing particles has been a characteristic of the art practically ever since the earliest noteworthy works. As such, it is a unifying feature between the original avant-garde and its digital descendant, providing links that are historical, cultural, and technical. Bryan Alexander, reading

Janet Murray’s Hamlet on the Holodeck (1997), states that each time a new medium emerges, it appropriates expressive elements from pre-existing media and incorporates them in its own methodology, a tendency that Murray calls “additive art” (42). It is only after this process that the creators of the new medium start to explore the potentials and expressive traits that are specific to this new medium, and this is exactly what has happened with digital media: after cinema appropriated the techniques of the novel, it began following its own technical pathway, with the development of editing, lighting, etc. Digital cinema, in turn, also appropriates all these pre-existing aesthetic and technical conquests of analog filmmaking, and adds to them its own distinctive potentials of manipulation (42). Practically the entire toolbox of visual communication that digital cinema still uses was put together between 1915 and 1930, including montage for photography and cinema, collage in art, as well as the codification of classical filmic language (Manovich, “Avant-garde”

2). No real groundbreaking developments are marked after this period, all the way to the present day, as moviemakers still utilize the same arsenal of techniques, despite their reliance on the computer. And as regards mass visual culture as a whole, 72

Manovich notes that it “only pushes further what was already invented,

‘intensifying’ particular techniques and mixing them together in new combinations”

(2).

It is therefore clear that the perception of image in digital cinema carries within it aspects and parts of the entire lineage of practicing and technologies of imaging. For this reason, it is important to locate and utilize one key characteristic element as a central axis for all those converging aspects and disciplines; without such a unifying element, any theoretical approach to digital cinema will probably be nothing more than an awkward pastiche of bits and pieces from theories of other arts. Luckily enough, this characteristic is readily available by the medium itself: the inherent fragmentation of the digital movie. That is, in order to take this discussion one step further, we need to always keep in mind that the digital means entails broken pieces put together. This is a fact that defines the medium entirely; therefore a discussion on the ontology of the digital image cannot but take it as a starting point. And along with it comes the realization that the world portrayed in movies, any world for that matter, is one given to us in a piecemeal fashion; movies contain shattered universes, either deliberately, as in formalist experimentations, or inevitably, as in realist continuities.

This fragmentation was not, of course, introduced with the digital image; in fact, it was around from the time cinema made its very first appearance. The late nineteenth century was marked by the invention of the assembly line, which indeed marked the revolutionary intrusion of machines in society, but can oddly also be seen as a continuation of the scientific rationality that was initiated centuries ago.

Similar to linear perspective, Rutsky sees the standardization process introduced 73

with the assembly line as the outcome of a “tendency towards rationalization” that had begun as early as the Renaissance (79). Thus, along with technical tradition, the

Renaissance also passed down to centuries of evolution in imaging this early attitude of mechanization of the process of production, cultural or otherwise; having reached its symbolic culmination in the logic of the assembly line, this attitude was realized in digital cinema primarily in editing, excellently visualized in the workflow of digital editing software like Final Cut Pro. As Rutsky adds, the original establishing idea of this centuries-old rationalization was that an item is immediately reduced to its elementary composing particles “usually represented in quantitative or geometric forms,” so that it becomes calculable, countable, and multipliable, and therefore mechanized; this description by Rutsky of breaking down of an item, physical or mental, into fragments, able to be re-shaped and manipulated at will (79), is exactly what digital editing software does in imaging today.

This fragmentation of the synthetic image is expressed in the ways in which digital images are composed. Depending on the proportion of synthetic vs. real footage contained in them, they may consist of three broad categories or cases: (i) purely digital images, (ii) images previously shot with an analog camera and were subsequently digitized (or specific parts or footage of those), or (iii) a combination of the two. But digital cinema in particular extends this fragmentation even further, as the entire process, from the beginning all the way to the finished product, is one of constantly composing elements together. There are thus several levels of fragmentation to consider besides the obvious ones. We shall identify four of these levels here. 74

The first level of fragmentation works on the elementary level of the technical creation of the image. It starts with case (i), purely digital images, which involves the use of a digital palette in a computer for the “assignment of an integer value to a pixel in order to specify (according to some coding scheme) its tone or color” (Mitchell 5); in this case, the product in all its stages will always be purely artificial, as it entirely “originate[s] in numerical form” (Manovich, New Media 28), which is a fairly straightforward scenario. The complication arises in case (ii) which involves digitized pictures, but also in (iii), in which the product is partly digital and partly digitized. In (ii) and (iii), the image or part of it, respectively, that is initially captured with an analog apparatus is traditionally considered ontologically continuous, and must be subsequently digitized, i.e. broken down into arrays of pixels, i.e. non-continuous particles that are capable of being represented numerically; this process consists of two steps, sampling and quantization

(Manovich, New Media 28). Sampling will entail turning “continuous data into discrete data,” i.e. pixels; from this point onwards, the procedure is similar to (i), with each newly created pixel being assigned a numerical value that will determine its visual characteristics (28). Once this process is complete, purely digital data as in case (i) are technically identical to digitized ones as in (ii), and they are thereafter considered exactly the same, at least as far as the computer is concerned.32

The second level of fragmentation works within the elements comprising the final image, i.e. referring to the creation of the partly or entirely synthetic items inside it, as the case may be. Thus it involves the shapes, forms, creatures, etc., that are constructed with disparate visual elements, such as “pixels, polygons, voxels,

32 See DVD-ROM, video file 02. 75

characters, scripts” that are indeed “assembled into larger-scale objects, but continue to maintain their separate identities” (Manovich, New Media 30).33 So, for instance, the character of the Gollum in The Lord of the Rings trilogy is the assemblage of the following: polygonal structures formed by sets of pixels, for purposes of animation modeling; the shape, gestures, motion, facial expression and voice of the actor Andy Serkis; and arrays of pixels assigned with values that simulate color, texture and shadow. Each of the above is a separate and generally recognizable item on its own, and yet they all together form a new, unique entity.34

The same also applies e.g. to buildings, cities and landscapes, crafts of any sort, etc.

It is important to note that after these elements are combined, they might be recognizable on their own, like e.g. voices or colors, but their individuality is eradicated as they are fused within the entirety of the figure that they form together with other elements. It is easy, for instance, to isolate and recognize the voice of

Serkis, if one pays specific attention to it, but the figure of Gollum prevails, and eventually the spectator “chooses” to fuse the voice with the CGI image and assign it to the Gollum.

The third level of fragmentation involves whatever appears simultaneously in the same frame, in a kind of an artificial, post-production mise-en-scène: the visual co-existence of synthetic items, artificially composed in the second level discussed above, with items already recorded, i.e. purely digital imagery combined with analog imagery which is subsequently digitized. This practice is generally known as digital compositing. Visual graphics in cinema today, especially fantasy blockbusters, mostly

33 Manovich notes that this is a characteristic of all New Media; based on the fact that contemporary digital practice blends all kinds of genres together, digital cinema qualifies for New Media status in relation to this characteristic. 34 See DVD-ROM, video file 03. 76

consist of this latter case, as purely digital images in cinema are practically used only in animation movies. It is only the ratio between the two kinds of images that differs from one production to the other. The process of digital compositing in a specific visual project involves the combination of elements digitally synthesized for the purposes of this project, with elements selected from databases of stock footage

(Manovich, New Media 136), or even footage shot for this project specifically.35 The visual result is a composite mise-en-scène made of completely heterogeneous elements, some of which are CGI, while others are digitized material. Several techniques are applied to ensure that these elements are blended together properly, so that they give the impression that they belong to the same space and time: perspective alignment, color modification, along with tone, contrast and shadows, etc.; the presence of a camera is also simulated, by adding grain, virtual camera movement on the landscape, artificial blurring for some of the objects for depth-of- field effect (137), and even lens flare and trembling zooming-in and out which gives the impression of an unstable hand holding a camera. Once these diverse objects are weaved together, they form a new, unified image: their individual properties and origin are lost, eventually forming a “single seamless image, sound, space, or scene”

(136). Compositing has been quite the same in process and principle as practices

35 One of the most common processes for most movie productions today involves actors and all other live shooting being recorded on blue or green backgrounds, called live plates; these surfaces make it very easy to add subsequently any digital or digitized background from stock footage (Manovich, New Media 136). The technology is called Chroma Key, or simply Green Screen technology. The original techniques actually involved blue screens; the color blue was selected for the backdrop because normally blue is not expected to be visible in human skin tones. The technology was used from as early as the silent period, and was later perfected by Petro Vlachos in the 1950s, with the introduction of green backdrops. For a concise history, function and applications of the green or blue screen, see Mark Sawicki, Filming the Fantastic (157-197). For an example of how green screens are used in fantasy cinema, see DVD-ROM, video file 04. There is also a very interesting example of the way compositing can actually enhance and alter a film shot decades ago; in the digitally remastered editions of the original Star Wars IV: A New Hope, George Lucas included the initially deleted scene of Han Solo (Harrison Ford) talking to the alien Jabba the Hutt in a newly composited sequence. For this scene, see DVD-ROM, video file 05. 77

applied in its analog ancestor; the digital means only changed its technical aspect, as now compositing describes the “ability to remove and replace pixels from a digitized image” (Lefebvre and Furstenau 82). Based on all the above, it is no surprise that digital compositing is a unique visual product that has raised all kinds of debates regarding the authenticity of images, and has therefore been the cornerstone of all relevant literature.

The fourth and final level of fragmentation operates on the level of the flow of the movie. As Manovich notes, several arts are considered continuous anyway, such as classical sculpture or photography, but others, such as cinema, are somewhere in between: as we have seen in the previous chapter, cinema has traditionally been considered a photographic medium, which instantly implies that each of its constituent parts, the frames, are continuous in form (New Media 28). But it is now easy to raise two major objections to this claim: first of all, based on what has just been described regarding the levels of fragmentation, contemporary movies are only artificially continuous, and cannot be considered anything close to naturally continuous like other arts, since they are fragmented even at a level as low as their elementary particles, the pixels. And interestingly enough, this does not apply only to digital cinema; as Manovich notes, digital compositing is nothing different than the descendant of techniques like keying or optical printing, widely used in analog movie-making (New Media 138).36 Second, and most importantly, a movie, whether an analog or a digital one, is almost never comprised of one single take in each

36 Manovich adds that a major difference between these techniques and digital compositing is the fact that the former never became the standard in cinema production, like the latter did. We can accept this difference, but it still does not affect the great similarities between the two; these similarities are enough to allow a solid affiliation between analog and digital cinema on the basis of continuous vs. fragmented image. 78

scene: even if there weren’t the stylistic or narrative aspects of movies, based on which moviemakers break stories into scenes and scenes into shots, there are still technical limitations in the duration of shooting and projecting a movie, that necessarily separate any of them into parts.37 First, as regards digitization of analog material, Manovich notes that sampling also carries within it a manipulation of time, as it becomes clear that the process of sampling begins a lot earlier than the moment the movie is eventually inserted into the digitization software: “each frame is a continuous photograph, but time is broken into a number of samples (frames)”

(New Media 28). The sampling Manovich refers to is actually one of fragments from the pro-filmic reality, which the moviemaker selects and weaves together; in fact, it is not just pieces of time, but also of place: even if the pieces belong to one unified pro-filmic space, successive framing of parts of that space is also a process of fragmentation, which demands from the spectator to put the pieces back together inside his mind. And second, digital movies are of course no exception; they too submit to the laws of visual narrative and montage, which necessarily break up time and space irrevocably. It matters little, if at all, that the imagery in each frame may be synthetic, real, or a combination of the two.

3.2. Addressing Cinema Editing, Now and Then

This focus on editing, either in positions in favor of it like Rudolf Arnheim’s, or against it like André Bazin’s, reveals its defining force in cinema, from its first days until the present. Let us not forget the fact that both formalist montage and classical

37 Even now, that double projectors in theaters ensure that the flow will not be disrupted, movies distributed in celluloid reels require more than one of those. In fact, the largest part of movie production today is still shot on celluloid and subsequently digitized and composited. 79

editing, two theoretical approaches with completely opposite orientations, sprang from exactly the same sources: the pioneering works of Edwin S. Porter and W.H.

Griffith. Exploration of the works of Porter and Griffith reveals cinema as the oxymoronic case of a methodologically concrete art, founded upon the tearing apart of its elements, right from the start. On one hand, Porter’s work is of tremendous importance in the context of digital compositing and editing: Porter made his Life of an American Fireman (1902) out of pieces of film stock already shot previously, weaved together in an artificially coherent storyline.38 This astonishing and innovative visual product meant that “the meaning of a shot was not necessarily self- contained but could be modified by joining the shot to others” (Reisz and Millar 5); most importantly, it immediately designates Porter as the technical and cultural forefather of digital compositing and editing, the idea behind which being a sense of coherence we defined earlier in essentially the same way. Griffith, on the other hand, is credited with realizing that “a film sequence must be made up of incomplete shots whose order and selection are governed by dramatic necessity” (Reisz and

Millar 10). The value of his work lies in using fragments of footage and, mostly, in the methods he used to connect them, as realized in his landmark films Birth of a Nation

(1915) and Intolerance (1916). Seeking to develop Porter’s original idea of continuity, he made significant advances to techniques such as match cut, eye-line match, fade- ins and outs, iris-ins and outs, etc., using editing for the first time to create variations in the dramatic impact of his narrative (Reisz and Millar 7; Fabe 6). Digital blockbusters not only have inherited this basic editing methodology as it runs

38 Porter had worked for Thomas Edison as a cameraman. The pieces of film used in The Life of an American Fireman came from his search in Edison’s old film stock for material having even a slight potential for thematic coherence, so that it would be easier to stitch together (Reisz and Millar 4–5). The entire short film by Porter is available in the DVD-ROM, video file 06. 80

through an entire century of moviemaking, but demonstrate its value by still simulating it artificially in entirely synthetic shots. In the context of Griffith’s effect on classical editing, Reisz and Millar stress the fact that that, since Griffith’s time,

“[t]he principle of using long, medium and close shots for various degrees of emphasis has remained substantially the same” (29). With editing methods gradually being passed down to digital movies, from Porter and Griffith, to George Lucas, Peter

Jackson, and the Wachowskis, cinema has made a full circle all the way back to its methodological roots.

The other important lost link between digital practice in cinema today and

Porter & Griffith, especially the latter of the two, should be sought in the kind of editing that invites associations between composed parts, relying heavily on mental operations. Both variations of the original avant-garde, the Kuleshov/Pudovkin and the Eisenstein/Vertov schools, were impressed by Griffith’s work and his assumptions about the psychological outcome of shots placed in associations; but also both of them sought to develop and interpret that work in a way that it would serve their ideological aspirations. On one hand, Lev Kuleshov had experimented with editing together unrelated visual material in various combinations, to show that spatial and temporal continuity in cinema is an illusion: meaning and flow was considered as not deriving from photographic reality, but stemming from the celluloid strips and the way the film director arranges them (Cook 146–147).

Therefore, for Kuleshov, the power that montage has to create associations between the composed parts is clearly a perceptual process, which elevates montage to an

“act of consciousness” for both the filmmaker and the audience (147). Additionally,

Vsevolod Pudovkin, also of the Kuleshov school, believed that Griffith’s approach to 81

continuity would be far more effective if each shot had a new focus, with emphasis placed on the details embellishing each shot rather than on long takes, and meaning deriving from the juxtaposition of shots (Reisz and Millar 12–13).

Contrary to the above positions, Sergei Eisenstein did not believe in such continuity, even in its formalist interpretation: aiming primarily at shocking the audience in every shot, he established his entire theory of filmmaking on the notion of conflict, i.e. shots in “opposition, contradiction or collision” as well as “their resulting fusion in the spectator’s mind to synthesis” (Ben-Shaul Key Concepts 75).

The emotional, psychological and intellectual manipulation of the audience was attempted with a constant interchange of types of shots placed in geometrical and rhythmic oppositions (75). This practice demonstrates his contempt for mimetic realism, as he assumed that the force of meanings would be communicated from one shot to the other and would be forged together into a wider, more complete message. Finally, Dziga Vertov, also an advocate of aggressive montage like

Eisenstein, believed in film methodology as “dialectic interaction between

filmmaker, film technique and material social life, in order to conduct a ‘communist decoding of reality’” (Ben-Shaul Key Concepts 78). He believed that film should capture reality as it truly is, and subsequently editing should “arrange this reality into an expressive and persuasive whole” (Cook 142–143). In all the above trends, comprehension of filmic reality was left to the hands (or discretion, for that matter) of the spectator, and was essentially tied to the powers of technology.

Kuleshov called it creative geography, Pudovkin constructive editing,

Eisenstein intellectual montage, and Vertov kino-glaz (cinema-eye), but it was essentially the same approach; are we at liberty to call it digital compositing and 82

editing? It is true that digital manipulations today do not bear the ideological burden of the Soviet avant-garde; if one desperately wants to find a common motive between them, that would be the fact that, just as soviet avant-garde was the vanguard of a political propaganda aimed at the masses, so too digital avant-garde, as a mainstream product, targets the massive consumption of pop culture. But there is no need to resolve to such awkward simplifications; a mere alignment of the two based on psychology of perception is more than enough. Indeed, digital compositing and editing also relies on technology and the combination of shots and other elements unrelated to each other; indeed, the spectators are also expected to acknowledge the rules of digital worldmaking on their own; indeed, this task is also demanding because the resulting imagery often bears little resemblance to normal reality, or it dramatically transforms its elements; and, indeed, the spectator also needs to concentrate and rely on what he knows, and how the building blocks in the movie are connected to each other, regardless of which of the four levels of fragmentation they belong to.

The point is that, if one expects to find an ideological or philosophical analogy between soviet and digital avant-garde editing, he is likely to be disappointed; but once the focus is shifted to editing as a technique, and, most importantly, to what it demands from the spectator’s perceptual resources, the similarities are remarkable.

This concentration solely on the technique, methodology, and technology enables us to flatten all formalist theories together, and thus draw the analogy with digital cinema more easily; after all, the theoretical disagreements between formalist theories were formed mainly on ideological grounds, which is not of interest in the 83

present study, nor can it be paralleled to contemporary digital blockbuster cinema.39

And by using cognitive theory to address the comparison between avant-garde, and digital compositing and editing, we are actually taking advantage of a link already known: the common interest, underlined by Ben-Shaul, that formalists and cognitivists share for dialectical film montage, on the grounds of its use of

“asymmetrical or contradictory film transitions” which draw the spectator’s attention to the formal qualities in the structure of a movie (Key Concepts 69). Still, the technical similarities between traditional and digital avant-garde should not distract us from the fact that, admittedly, the former indeed demanded too much from audiences40 and this may be one of the reasons for its decline. Nevertheless, those similarities still establish it as the best starting point for comprehending visual perception in digital motion pictures.

The digital is a disturbing demonstration of the fact that movie data are nothing but assemblies of particles, which form bigger, assorted entities, used next to one another to form scenes that are weaved together so that they form visual renderings of cinematic worlds. Movies today, as parts of new media, are established on the idea of breaking, composing and glueing things together, and are by definition composed of “independent parts, each of which consists of smaller

39 After all, Julian Hochberg and Virginia Brooks notice that the approaches to montage provided by the major theoretical opponents within formalist theory, Eisenstein and Pudovkin, are conspicuously not as stark as the moviemakers themselves or anyone else might initially think (“Perception of Motion Pictures” 264). Also, Bordwell states that all four formalist theories mentioned here are practically the same in that they “assumed that filmic meaning is built out of an assemblage of shots which creates a new synthesis, an overall meaning that lies not within each part but in the very fact of juxtaposition” (“The Idea of Montage” 9). 40 Eisenstein probably overestimated the cognitive results of his technique, or was altogether mistaken about it. Hochberg and Brooks note that there is now “no reason to believe that without a specific effort at construal by the viewer anything other than a meaningless flight of visual fragments, relieved by an occasional meaning that chunks the montage into a memorable unit, will be perceived” (“Perception of Motion Pictures” 265). 84

independent parts, and so on, down to the level of the smallest ‘atoms' – pixels, 3-D points, or text characters” (Manovich, New Media 31). As such, they conform to the practices of what Mitchell calls “the age of electrobricolage” (7). It is crucial to notice that this technical and ontological leap from continuous to fragmented data in visual culture has severe implications for human comprehension of visual stimuli, as vision is considered to provide us with the most direct and trustworthy access to the world, despite the fact that such a claim is debated and often proven wrong. This breaking down of knowledge of the world into fragments that started several centuries ago aimed at the quantification of the very knowledge of what it means for something to be real and perceived as such. The word that best describes this trait is synthesis, with its Greek etymology41 implying a twofold meaning: on one hand, it means composition of elements, but on the other it signifies that the result of this composition is an artificial creation. In digital cinema, with these two aspects put together, synthesis comes to mean a potentially real-like image, but the constituent parts of which are not necessarily founded in what we perceive as solid reality in the environment around us. Digital technology was the culmination of this tendency to comprehend reality by dismantling it, but it also brought with it the ability to control and re-shape the resulting knowledge. This side effect scaled down knowledge of reality to a set of properties that, once expressed into a checklist, could also be replicated and, consequently, also modified. The logical outcome of this situation is that the sense of objectivity in visual reality, especially in art, began to be compromised: diminished from its throne to being conceived as a set of qualities

41 This comes from the Greek word συνθέτω (συν+θέτω), the etymology of which is “to put things together, creating a whole from parts;” this process automatically implies that the final product is artificial. 85

open to modification, it also became more open to different theoretical approaches.

Cinema, in particular, was no short of those.

3.3. Approaching Theories of Visual Fragmentation

Both the classical editing and the formalist montage fractions saw in the fragmentation of the movie image an opportunity to convey what each of them considered as a proper rendering of reality, or an aspect of it. Consequently, various theories of perception have been employed in each case to account for the ways in which the perception of moving images in cinema resembles that of the real world in normal conditions. The main objective of cinema has been to convey some sort of message in a visually legible manner, despite the fact that both the kind of message and the mode of delivery are different in each school; therefore, such a statement already implies that the delivery of the message is the primary objective here, and that any of the delivery options that is to be selected must agree with the ways perception operates in humans, so that the meaning is compatible with human perceptual mechanisms and gets through to the recipient successfully. This is a very practical starting point for approaching a theory of perception of digital cinema, as it can easily accommodate any attitude to movie construction, realist or formalist, as long as it is grounded on a well-organized and substantiated message-and-delivery matrix.

As the technological development of cinema has outpaced the various individual film theories of the analog era, it quickly becomes obvious that, in relation to digital cinema specifically, a theoretical approach is needed that combines old approaches and disciplines with new ones, taking into consideration the form, 86

essence and lineage of the medium, as it has been addressed so far in the present study. In other words, it is imperative that a theoretical framework for the perception of digital cinema should be able to fulfill two essential prerequisites: first, it must support and explain the multi-leveled fragmentation of a technical novelty, such as the digital image, without doing away with useful insights from the past. And second, as regards the final product, it needs to account for the way these images are comprehended, since, on top of the fantasy factor that is generally prevalent in

CGI cinema, such a fragmentation also produces images that do not resemble those of the normal environment. Respecting these prerequisites, we can review four candidate theories based on the role they assign to mental processes that are required for non-normal, so to speak, images, to be comprehended: Helmholtz’s theory of vision, Gestalt theory of perception, Gibson’s Ecological theory, and

Lakoff’s and Johnson’s cognitive semantics. All of these fractions have been associated one way or another with film comprehension; and despite their differences, as Julian Hochberg demonstrates in his “Levels of Perceptual

Organization,” they also overlap with each other, depending on the way they address specific areas within their wider disciplines. In the sections that follow we will first review the initial foundation of Hochberg’s model, i.e. Helmholtz’s theory; then we will move on to Gibson’s “direct” model and the Gestalt theory, in order to show how the two contradicted Helmholtz and whether these contradictions have been well-founded or not. Finally before deploying Hochberg’s model itself, we will review Lakoff and Johnson’s “cognitive semantics” based on the conspicuously overlapping areas between them.

87

3.3.1. Helmholtz’s theory of Perception

The first candidate theory for the purposes of this study is Hermann von

Helmholtz’s theory of perception, founded on his research on optics. Born and raised in the course of the Nineteenth century, Helmholtz favored John Locke’s empiricism over Immanuel Kant’s nativism,42 but his theory often relapses to subtle Kantian insights. Kant’s nativism had already started to gradually succumb to the emerging seventeenth- and eighteenth-century empiricism; perception of form after Locke and all the way into the Twentieth century followed a more positivist path (Andrew,

Concepts 25), which, contrary to any assumptions of innate knowledge, generally held that the mind is “a blank slate ready to be written upon by experience” (Gordon

30). Such positivism was generally atomistic,43 viewing perception as the “product of long experience that stimuli evoke;” thus it defined perception as an ongoing additive process, lasting for years, that combines visual stimuli with all other bodily senses and one’s sense of presence in the world (Andrew, Concepts 25). Amidst this clash between nativism and empiricism, Helmholtz is an interesting case of a physiologist, an empirical scientist in his own preference, who also benefitted from

42 This was also true of other structuralist models around his time. Structuralist accounts of visual perception, which were of the empiricist kind, assumed that “our visual experiences consist of (i) sensations of different colors — light, shade, hue — and (ii) images, or memories of those sensations” (Hochberg, “The Representation of Things and People” 150). Any scene in the environment emits stimuli elements, each of which creates its own basic and elementary sensation; the percept is comprised by the sum of these sensations, essentially breaking up the perception of the world to a mosaic (Gordon 11). Nativist Approaches to perception on the other hand, following Kant, had assumed that certain parameters of perception, like time and space in which motion occurs, are not themselves perceived; instead, they are “a priori intuitions,” innate knowledge that is taken for granted, “superimposed upon reality by our minds” (Gordon 8). 43 Positivism is essentially atomistic. This approach treats any object under scrutiny as capable to be broken down to separate “observable units,” which means that all objects of observation can be scaled down to “atomic units” (Delanty 12). By analogy, the atomistic approach of positivists to reality means that the latter is open to fragmentation for easier and more effective study. 88

an environment infused with Kantian philosophy of vision.44 Helmholtz’s formulation of the notion of unconscious inference is central in his model and dictates his entire theory of perception. His model is established on the assumption that “perception draws on the same cognitive mechanisms as do ordinary reasoning and scientific inference” (Hatfield 116). The model of unconscious inference works as such: sensory signals (objects, events, etc,) are received from the environment, and are broken down to “independent, simple, but unnoticeable sensations” [emphasis added] (Hochberg qtd. in Anderson Reality 18–19). As the visual system receives these sensations from the world, it constantly performs unconscious estimations based on prior knowledge in order to determine the characteristics of the perceived object or scene. It is important to note that only the resulting characteristics are actually experienced, as neither the sensations nor the estimations are conscious actions (Hochberg, “Higher-Order Stimuli” 189).

Helmholtz established his theory on his famous rule of perception, which states that “we perceive that object or event which would, under normal seeing conditions, be most likely to produce the pattern of sensations that we receive”

(Hochberg, “On Cognition in Perception” 127). Under this description, it derives that the object that is perceived is not dictated by any presumably objective and 1:1 equivalent sensation, or sets of those, but the other way around: the sensations dictate to the viewer a certain pattern, which is translated in his mind into the closest corresponding object. This immediately means that, as the sensations change, even slightly, so do the produced patterns and, consequently, the perceived

44 Hochberg stresses the Kantian influence in Helmholtz’s work, noting that both his teacher, Johannes Müller, and his father were Kantians (“Levels” 280). 89

objects.45 In this respect, Hochberg states that unconscious inference means “[t]o fit a perceived object or event to the sensory data” [emphasis added] (Hochberg, “On

Cognition in Perception” 127). Since, therefore, perceptions are “fitted” to sensations instead of the other way round, it can be deduced that the results of mental operations are crucial in perception:46 the mind will keep trying to make sense even of complex sensations, i.e. it will try to fit them to already existent structures. Functioning in such a purposeful way in cases of problematic viewing, our mental operations will struggle to create comprehensible perceptions even from sensations that are distorted or not normal in any other way (Cutting, “Rigidity”

325).

Unconscious inference was a breakthrough by which Helmholtz associated perception to mental representations, realizing that an exclusively physiological explanation of perception was not adequate. It also enabled him to avoid the fallacy which was common in other structuralist theories, that a percept is simply the sum of the sensations it produces: by utilizing experience and memory as major contributing factors in his theory, which together constantly “correct and enhance the momentary effects of stimulation,” he defined the skill of perceiving as one that, to a great extent, “must be learned” (Gordon 11). But although it is not explicit, his framework indeed seems to imply some set of perceptual subroutines already existent inside the mind, a feature which is generally in discord with his empiricist self-aspirations. This implication, i.e. that cognitive processes silently intervene at some stage in the process of perception, is also stressed by Hochberg as he discusses

45 For a schematic example of this point, see Appendix II, picture 9. 46 Despite the fact that Helmholtz’s approach is an empirical one, Hochberg notes that it can still be read through Kant’s schemata (Hochberg “Levels” 280). 90

the critique that Helmholtz’s theory faced over the decades. Hochberg notes that polemics of the unconscious inference hypothesis did not quite succeed, as, in one or the other way, cognitive processes must intervene at some point (Hochberg, “On

Cognition in Perception” 133). Gestalt theory and Gibson’s ecological approach, each with its merits and shortcomings, are very interested examples of these polemics.

3.3.2. Gestalt theory of Perception

It is this characteristic of Helmholtz’s theory that brings him in an interesting conversation with the second candidate theory of this study, Gestalt theory. Cinema, since its birth and all the way to the 1940s, was theoretically addressed by psychologists and sensory physiologists using variations of Helmholtz’s rule of perception, from which it derives that the basis of perception is not located anymore on the retinal image or the optic array, but on the way the mind processes sensations to form mental representations, just like normal problem solving inferences, only in this case unconsciously applied (Hochberg and Brooks,

“Perception of Motion Pictures” 211–212). Gestalt theory also emerged during the early days of cinema, in the conditions that were fruitful for its application to the newly formed art, and thus the two share intimate ties ever since (26). It is placed here under consideration, as one of its main proponents, Rudolf Arnheim, used its framework to prioritize the role of human cognition in the process of comprehension of cinema, an art whose form is shattered by technology.47 Gestalt theory and its

47 An interesting link can be noted here between Arnheim and Helmholtz, in relation to their response to art. Arnheim felt that art, specifically cinematic art, was the premises on which the application of Gestalt theory can be fruitful. Similarly, Helmholtz first started developing the elements that would later formulate the unconscious inference hypothesis when he was studying the response of observers to art; he found that “the global perception of a work of art depended on unconscious 91

regressing to Kantian nativism for inspiration were among the factors that caused the tradition of empiricism, such as that of Locke, to start withering (Andrew

Concepts 26). Introduced by Christian von Ehrenfels and Edgar Rubin, but acknowledged as truly starting with Max Wertheimer, Gestaltism considered perception as “innately determined” (Gordon 8).48 In contrast to positivist approaches, atomic units that are found in configurations were considered more important than those in isolation, advocating the existence of forms that are innate in human cognition, “structured inside the physiology of the eye and the neural patterns of the human brain;” such forms are elementary in nature and thus geometrical in form, and are involuntarily brought into perception when they are provoked by stimuli from the environment (Andrew Concepts 26).

Gestalt theory has several points that may come in handy when one studies a fragmented kind of art like cinema. First and foremost comes the Gestaltists’ emphasis on “wholes over parts, patterns over individual sensations” (Andrew, Film

Theories 29). Wertheimer, studying the successive stimuli producing a sense of movement in what is known as the phi phenomenon,49 stated that these stimuli were indeed separate, but when they are placed in “temporal and spatial relationships,” a quality arises between the stimuli that “differs from (is over and

visual analysis of details, the inaccuracy of one of which was capable of producing an unfavorable esthetic impression” (Meulders 86). As an example of this, Helmholtz mentioned in one of his lectures that he had once seen in the Berlin Museum a statue of Apollo, that he could not appreciate fully, without knowing exactly why; eventually he discovered that it was due to an anatomical error in the form of the statue, which he did not notice initially (86). 48 Gordon notes that this argument was central in Gestalt theory, despite the fact that it appeared about an entire century after Kant’s death. 49 The phi phenomenon is introduced in more detail on p. 145, as the context is more suitable there. For the reader’s convenience, there is an entry on the phi phenomenon in the Appendix I (Glossary). 92

above) the sum of the parts acting in isolation;” this sum acquires a Gestaltqualität,50 a “form quality” (Gordon 13). This became the most famous dictum of the

Gestaltists, as well as one of their cornerstone propositions, which urged them to seek and define several laws of grouping, in order to properly demonstrate the visual effects of their theses. The second one was the famous figure-ground distinction; put simply, it describes the easiness by which a perceiver immediately distinguishes between a figure against the ground it lays, and the other way round. Defined like this, it was basically a rejection of structuralist assumptions of sensations, as, to the

Gestaltists, it revealed that perception is dynamic and constantly adapting itself to identifying the most normal or familiar percept in each case (Gordon 15). It also linked the Gestaltists to their nativist background, as it revealed a feature of the human perception that is characterized by “valuable economy” and, being innate, does not need to be learned (15). Indeed, therefore, the Gestalt theory favored grouping over individuality, and its proponents set forth to discover and theoretically solidify the codes by which members in a group were arranged.

The study of these assumptions revealed the existence of a “general underlying principle” that governed all of their various modes of grouping elements.

This was expressed in the Gestalt law of Prägnanz or figural goodness, which explained the tendency that perception has, “wherever possible, towards simplicity, symmetry, and wholeness” (Gordon 17). The quality of Prägnanz on one hand pushes grouped stimuli into a tension which is created among them, in effect forcing them to reveal clear patterns for the perceiver; and on the other, it also demonstrates perceptual constancy, i.e. the “tendency of perception to be veridical”

50 The term was first introduced by Ehrenfels as early as 1890, who noticed that “many groups of stimuli acquire a pattern quality that differs from the parts when seen in isolation” (13). 93

(19), i.e. the unconscious effort of the visual system always to make some kind of familiar sense out of whatever it sees. In turn, this tendency towards familiarity that

Prägnanz has, is governed by the minimum principle, and old idea51 reworked by the

Gestaltists: “We perceive just that organization (shape, scene, or movement) that is simplest (according to some specification)” (Hochberg, “Levels” 282). Because of the effect of the perceptual constancy, according to the theory, perception bypasses and ignores the effects of individual stimuli, and instead focuses on the greater environment, as a “dynamic field” in which stimuli are arranged (Gordon 19–20).52

The Gestaltists, therefore, placed great importance to the innate tendency of the human mind towards making meaning, even from visually challenging scenes.

It was this latter confidence in human cognitive capacities that helped the

Gestalt theory soon established itself and also gain a considerable leverage in the visual arts. An equally important factor of the success of the theory was the fact that the Gestaltists always used explicit visual examples to support their arguments. They were also favored by the conditions in which the theory emerged; the Gestaltists found fertile ground to adopt a critical stance towards their preceding and contemporary structuralist accounts, as the theoretical shortcomings of structuralism had already began to appear. The Gestaltists, for example, had fundamental disagreements with structuralism: first, on the necessity that structuralism posed for constantly practicing the identification of sensations in order

51 The minimum principle has its roots back in the mathematics of ancient Greece, as well as in the mid-eighteenth century, when Maupertius announced what he called “the law of least action,” which stated that “[i]f there occurs some change in nature, the amount of action necessary for this change must be as small as possible” (Gordon 10). This idea of a “minimum effort” for natural processes was to be manifested in various ways for many decades to come, and provided the foundation for the central core of Gestalt theory (10). 52 Gordon provides an example of this function: owing to perceptual constancy, objects do not appear as shrinking when they move away from the observer (19). 94

for those not to be confused with one another; and second, on the “mosaic view of perception” (Gordon 11).53 On the other hand, despite their acceptance of the role of experience, the Gestaltists also downplayed openly Helmholtz’s emphasis on learning as a source of memory and experience, which was a fundamental assertion in his own model, but they were not hostile towards the role of experience specifically (Gordon 11). Interestingly enough, this emphasis by the Gestaltists’ on

“experience rather than data” (15) creates a conspicuous link with Helmholtz’s utilization of experience and memory as factors that constantly adjust sensations, which was his own attempt to fine-tune this structuralist fallacy. It was this subtle nativist twist in Helmholtz’s theory that has provided enough ground for a theoretical framework that partly reconciles Gestalt theory with his own approach.

3.3.3. Gibson’s Ecological theory of Perception

The third candidate theory is J.J. Gibson’s ecological approach, as deployed in his landmark work The Ecological Approach to Visual Perception. It is also the most recent one of the three theories reviewed so far. Joseph Anderson has been a major proponent of the application of Gibson’s theory to cinema, claiming that the difficulty that it faced to be accepted ever since its publication has been its groundbreaking opposition to the majority of perception theories that have traditionally followed Helmholtz’s account (Reality 18). Although he agreed with the optical physiology described by Helmholtz, Gibson’s main objection to Helmholtz’s framework was based on the former’s denial of intermediate interpretative stages between the stimulus and perception, opting instead for a more direct explanation

53 See foonote 42. 95

for perception. Consequently, he rejected Helmholtz’s unconscious inference, because it intervened between stimuli and perception, and instead believed that the latter is direct, unmediated, and requiring no interpretation of the visual stimuli. In fact, according to Anderson, this opposition between direct and mediated perception is the main line that separates Gibson’s theory from traditional perceptual psychology (Reality 19). Gibson’s theory is established on a biological basis, claiming that each and every capacity of humans has been acquired through the evolution of the species for some or other reason of survival. The cornerstone of this theory is the ground, both literally and metaphorically, as the most stable point of reference in humans’ contact with their environment, which Gibson felt is now permanently wired to perception (Reality 17–18).

Gibson’s theory has two important parameters, which are connected with the use he makes of the terms affordance and invariant. First of all, in the context of ecological theory, affordances are the qualities of perceptions that bear some meaning to the observer in relation to his immediate environment, as humans are tuned to pick up information that guide them towards taking the proper steps for their survival (Butler 65; Gordon 154). On similar grounds, Anderson defines affordances as the things that we seek in our environments and are useful for our lives, as opposed to a random collection of purposeless information; affordances, he notes, are embedded in our perception of an object or event (Reality 40–41).54

Gibson’s novelty in introducing the theory of affordances and invariants is the fact that, despite them being abstract properties, he defined them as “perceived directly, without prior synthesis or analysis . . . directly from the pattern of stimulation arising

54 Gordon mentions, for instance, objects having the affordance of being “graspable or throwable” [sic] (154). 96

from the object” (Gordon 155). A skeptic seeking to find weaknesses in Gibson’s model would probably note that affordances imply some sort of learning ability and process: a meaning or usable quality of an object is surely not expected to be a disposable, one-time experience, as this would render it completely useless for evolution, thus negating the elementary assumptions of ecological theory. Gibson and his followers have replied that learning indeed exists, but it is a concept much wider than individual lifetimes, expanding to the entire length of evolution of a species: for humans, after millennia of this evolution, proper response to specific affordances is a property that has been hard-wired to our visual system as its default function, indeed very much like any other organ in our bodies has learned to perform a specific biological task (Gordon 156).

The second important parameter in Gibson’s theoretical model is the assumption that the information readily found in the environment suffices for perception, in the form of “ambient array of light reflected from objects on the world” (Anderson Reality 29), rather than “building a percept bottom–up from separate sensations” (Richards 1025). Gibson’s model, by its nature, relies substantially on the role of movement in the perception of the environment;55 based on this, it assumes that as objects move in the visual array, their patterns and textures as they reach the eye are constantly changing, but they do so governed by complex yet “lawful regularities,” dictated by the correlation of adjacent parts in the patterns (Gordon 151). This means that there is a tremendous amount of information that changes, but also that “over and above the behavior of each

55 Gibson was initially recruited to assist pilots during WWII in flying and landing aircrafts safely. His theory was developed based on his studies on pilot behavior; therefore it included observers in motion. 97

texture element” there is another set of information which remains stable and thus provides utilizable spatial information (Gordon 152). These constitute what Gibson called invariants: “higher-order properties of patterns of stimulation that remain constant during changes associated with the observer, the environment, or both”

(153). Based on this thought, Gibson defined visual information as a direct process:

Information, then, consists of patterns of actual relationships between

objects in the world. It is not something added, deduced, or inferred

from raw data. The information contained in patterns of light is

encountered directly by the visual system and processed immediately

and continuously without the necessity of logical constructions such as

deduction or inference. (Anderson Reality 29)

Consequently, such a definition for visual information eliminates mental schemata, grouped either under “representation” or any other term, because it rejects the symbolic system that is necessarily assumed by them. It thus delineates a kind of access to the world based solely on raw data and the associations created between them, as the light they reflect gradually changes direction and angle to our eyes.

Gibson’s concept of invariants is complex and difficult to grasp, but it is also considered his major contribution to psychology.

Overall, in Hochberg’s view, Gibson’s direct model of perception seems rather problematic. In order to show this, Hochberg starts from Gibson’s critique of

Helmholtz’s unconscious inference, especially as regards the role of cognitive processes of perception. The fact that Gibson’s model essentially denies these cognitive processes in favor of a direct access to the world left him with a number of 98

parts in his theory that he could not fully explain.56 For example, in trying to explain what happens in cases in which parts of an object are occluded from sight, thus hindering direct perception of this object entirely, Gibson calls on the function of expectations, “using language almost identical to that of [J.S.] Mill57 and Helmholtz”

(Hochberg, “On Cognition and Perception” 131). Also, the fact that clear stimuli are received primarily by the fovea, the direction of which needs to be selective and purposeful, shows that there must be underlying schemata already existent inside the mind that guide this movement towards picking up specific parts of the array that will complete the image as quickly and efficiently as possible (131).58 It is obvious therefore that the direct model of perception cannot function properly, at least not in the strict borders that Gibson’s theory limits it by denying the role that cognitive operations also play in the process. In fact, when reviewing Gibson’s model, Hochberg cannot see any other way for a person to acquire cognitive processes, except as parts of the process of perception in the presence of stimulus information (129).

56 As Hochberg notes, this is the same shortcoming found on approaches that contradict Gibson, such as those of Richard Langton Gregory, William Epstein and Irvin Rock: although they keep bringing into attention the argument that there must indeed be some cognitive problem-solving process mediating between the reception of the stimulus and the perception of an object, they still fail to address properly the exact characteristics of this intermediary stage; as a consequence, the way in which stimulus is related to what is perceived remains generally vague in all of them (“On Cognition and Perception” 131). 57 The British philosopher John Stuart Mill was in agreement with Helmholtz as regards the fact that “channels of direct sensory response, which analyze the patterns of stimulation into raw sensory responses, do not contribute directly to experience” (Hochberg, “On Cognition and Perception” 127). 58 This limitation of the fovea will be fully analyzed in relation to cognitive operations, in the discussion of Hochberg’s model further down, as this partial or “piecemeal” viewing of objects is one of the cornerstones of his own model. 99

3.3.4. Lakoff and Johnson’s Cognitive Semantics

The fourth and final cognitive theory, as well as one of the most promising ones in terms of visual perception is the Cognitive Semantics of George Lakoff and

Mark Johnson. The essential difference with the previous theories is that Lakoff and

Johnson’s approach to cognitivism is one that does not reject the function of language; therefore, it is generally not compatible with the American school of cognitive film theory, such as the constructivist account of Bordwell. According to

Buckland’s account, Lakoff and Johnson’s model reviews and builds upon several of the basic premises of the Language Analysis tradition, established mainly on the writings of Ferdinand de Saussure, Jürgen Habermas and C.S. Peirce among others.

The original goal of the Language Analysis tradition was to “transform questions about epistemology and the mind into questions about language and meaning,” essentially subverting the 300-year long epistemological idealism spanning from

Descartes, to Kant, and finally to Hegel (Cognitive Semiotics 15). In contrast to this tradition, which revolved around the concept of mental images, the Language

Analysis tradition proposed “language, signs and the process of semiosis” as a viable alternative (Cognitive Semiotics 15). The immediate and serious consequence of this shift is the loss of the potential for absolute objectivity that epistemology advocated: where the latter claimed that one is capable of having direct access to the thoughts inside his own mind via a “first-person” perspective, the Language analysis tradition states that the only access to those thoughts is possible through the use of language, which is a crucial parameter of human existence in general:

Habermas – and the Language Analysis tradition generally – decenters

subjectivity by arguing that it is not master of its own house, but is 100

dependent on something that is a priori and intersubjective – language.

And language is fundamental because it functions to represent (or

disclose) reality, establishes interpersonal relations, and guides personal

expression. (Buckland, Cognitive Semiotics 16–17)

This prioritizing of language essentially displaces human reason from the center of the universe, by assigning to it a “third-person” perspective, which is always and inescapably mediated. Buckland notes that cognitivists in general follow the principles of epistemology, whereas semioticians adopt positions from the Language

Analysis tradition (Cognitive Semiotics 17).

In the midst of this feud, what Lakoff and Johnson have done is quite interesting, as, on one hand, their work places them within this reforming Language

Analysis realm, but on the other it extends to the physicality of the human body and its experience of the external world. Although the authors operate within a cognitive framework, which is traditionally epistemological, they incorporate the function of language as laid out by semioticians, but refuse the latter’s belief that thought as an abstract entity exists within an “autonomous realm;” instead, they trace it back to the basic functions of the body, which acquires its knowledge from its physical interaction with the external world, since “both thought and language are fundamentally motivated by bodily experience” (Buckland Cognitive Semiotics 39).

Therefore, Lakoff and Johnson on one hand attempt to overcome the Cartesian epistemological division between mind and body, while at the same time preserving the importance of human experience in comprehending the world:

[C]ognitive semantics mediates between dualism and its polar opposite,

eliminative behaviorism, which repudiates the existence of any mental 101

events or states by reducing them to bodily behavior. In rejecting both

dualism and eliminative behaviorism, cognitive semantics defines itself as

a form of mental materialism, in which mental phenomena are dependent

on (or are realized by) the material (the body), but are not mechanically

reducible to the material. (Buckland, Cognitive Semiotics 40)59

In short, Lakoff and Johnson oppose epistemological idealism because it assigned practically no importance on language as regards knowledge, stripping the latter from all functions related with imagination, and acknowledging only the role of experience. And, on the contrary, they also revised significantly several of the assumptions of the Language Analysis tradition; the reason was that the Language

Analysis Tradition seems to have assumed too much from language, essentially forgetting the fact that language is affected by the ways the human body functions inside its physical and social environment.

The importance of Lakoff and Johnson’s model for the present study is that it introduces a concept for mental representation that incorporates schematic constructions. The authors disagreed with objectivism which considered knowledge as “propositions consisting of meaningless symbols” inside the mind, that correspond to external reality though which they also acquire their meaning

(Buckland Cognitive Semiotics 39). Instead, they replace in their model these abstract symbols with schemata, which they define as “cognitive structures that organize perceptual input into experiences” (40). These schematic constructions aid

59 Descartes did not, or could not, assign to the mind the same mechanical attributes that he did to the body, defining instead the res cogitans or “thinking substance” as its operating force, and thus raising questions of interaction between the two: the operations of the human condition, split between the mind and the body, were therefore deemed incompatible with each other (Buckland, Cognitive Semiotics 40). 102

perceptual stimuli from the environment to be transformed into experiences, and, unlike objectivist symbols, they are meaningful because they are established on “the body’s innate sensory-motor capacities” (40). The immediate consequence of this, which poses an essential difference with objectivism, is that schemata conceived this way are image-based, because they transfer the sense of one’s own body and motion to thought; or, as Johnson calls it, they create the image of “the body in the mind” (40). Most importantly, they are “cognitively real because they are directly motivated (non-arbitrary) and inherently meaningful” (40).

This account is quite interesting is relation to the present discussion for two main reasons. The first one is the connection that it makes between physical body movements to schemata, as a means of establishing a communication between the mind and the external world. And the second one is the way Lakoff and Johnson bypass the stiffness of abstract linguistic account of knowledge by taking into account metaphor and metonymy, which they see as “imaginative projective devices that [are] fundamental to cognitive reasoning” (Buckland Cognitive Semiotics 31). To elaborate on these two reasons: cinema, apart from being an artwork with motion as its definitive and differentiating trait, is one that takes advantage of the framing of the image in order to guide the viewpoint of the spectator, and thus, consequently, also to position him inside the virtual cinematic world. This fact immediately grants schemata an unprecedented level of sharing across all spectators: since schemata are tied with the structure and kinaesthetic function of the human body, and are thus a species-wide trait, they immediately render the experience of cinema as a highly common one; this important observation allows schemata to become a very good foundation for a universal approach to cinema watching. In other words, they 103

can be used to create a master-theory of cinema, based on experience; and at the same time, the fact that they also utilize the functions of metaphor and metonymy, also allows ample room for content-based sub-theories, which may provide more subjective approaches and interpretations to specific issues.

All four theories discussed in this section have been selected on their direct or indirect relation to the technology of cinema, especially as regards its fragmentation, as well as to their relation with issues of mental representation. It is no surprise that psychological theories of perception have tried to accommodate cinema to their principles: Arnheim and Mitry have been ardent supporters of

Gestalt theory, Joseph Anderson has taken up Gibson’s ecological route, while Julian

Hochberg has begun from Helmholtz’s theory to reveal the latter’s often reluctant, yet conspicuous common areas with the other two. What is more, Lakoff and

Johnson’s image-based schemata are very much akin to the ones conceived by

Hochberg and Brooks, an observation which distantiates all these four authors from mainstream cognitive accounts of cinema like that of Bordwell.60 For these reasons,

Hochberg is a wise starting point for an approach to digital cinema: on one hand, his theory is the most recent one, ergo the most probable to have taken into consideration all others, and the most appropriate for a contemporary art seeking a scientific and cultural identity to define itself by; and on the other, Hochberg has devoted a part of his work specifically to cinema, demonstrating solid ways in which

60 Just like Lakoff and Johnson, Hochberg and Brooks also define schemata as image-based. But it needs to be stressed that in both cases, image schemata of this sort refer to abstract guiding patterns rather than objects themselves; they are non-specific structures of images rather than specific items, events, etc. In the case of Lakoff and Johnson, the function of metaphor and metonymy makes image schemata extremely versatile as guides to cognitive comprehension of the world. This feature marks the most crucial difference between their own and Bordwell’s top-down model, whose own schemata are significantly less flexible as they are transcendental, static and finite (Buckland 29-32) For a detailed account of this difference, see section 5.4.1 in Chapter 5. 104

the latter can be finally approached and defended by crude science rather than the vague and often incomprehensible philosophical sophistry that has been contaminating literal, cultural, and cinema studies for decades.

3.4. Approaching Mental Schemata: Julian Hochberg on Perception

The work of Hochberg, along with Virginia Brooks, being an application of cognitive psychology applied to film, assumes active mental operations on behalf of the spectator, in the latter’s effort to comprehend the moving 3D world of a movie in its entirety, both as regards narrative space and succession of events. The difference with the larger body of cognitive film theory, according to Ben-Shaul, is the fact that Hochberg and Brooks’ work considers that the spectator starts from bottom-up hypotheses, which are automatic, precede, and thus also affect top-down process (Key Concepts 60).61 This observation derives from “spatial and rhythmical” disruptions that editing and two-dimensional surfaces produce, as they transfer motion from real life to the cinematic image:

Since these disruptions occur on the perceptual level, spectators cannot

consciously control them (as they cannot avoid seeing the projected film

in continuous motion or as three-dimensional). In being uncontrolled by

spectators, these ‘disruptions’ overpower top-down intentions. Thus,

slowing movement on the perceptual level undermines a top-down

intention to impart thematic urgency. (Key Concepts 60–61).

61 Top-down processes rely on background knowledge to assign expected meaning to a certain visual stimulus, whereas bottom-up processes, doing the opposite, rely on visual stimuli, which are combined to form a unified conclusion. These processes do not operate on an “either-or” basis; instead, the mind applies them simultaneously and in a cyclical mode: the mind perceives stimuli, applies hypotheses to them based on schemata, checks the validity of theses hypotheses, enhances them with further stimuli, re-evaluates them if necessary, and so on so forth. 105

By assuming an essentially “active and aware” spectator, other cognitive film theorists essentially sprang out of critique of their predecessors (Key Concepts 61).

More specifically, Ben Shaul notes that the majority of work being done on cognitive approach to cinema moves on the opposite direction, i.e. focusing on the top-down processes at hand during the spectator’s perception of a movie, as the spectator is affected by aspects of genre, narrative and style (Key Concepts 61). Consider

Bordwell for instance, and his leading film cognitivist school: his account is top- down, describing the way the spectator interacts with the content of a movie in order to construct its meaning based on his pre-conceived schemata. But this is exactly one of the main weaknesses in his theory: as Buckland notes, Bordwell’s model reduces all perceptual data to not much more than simply “a set of cues interacting with the spectator’s cognitive capacity” (Cognitive Semiotics 30). On the contrary, this preoccupation of Hochberg and Brooks with editing, its effects on content, and its consequences for perception should provide a more original and accurate theoretical background for digital cinema. The fact that their model considers bottom up processes leaves ample room for an approach to schemata that delivers them from the stiffness of carrying only specific and finite meanings; in contrast to Bordwell’s, Hochberg and Brooks’ model delineates schemata as far more versatile and adaptable.62

Viewers of contemporary digital blockbusters have a much more demanding task to perform than those of classic analog movies. Echoing Gestalt assumptions, montage theory aspired to a kind of cinema that could not do without cuts, as those

62 In the context of the present study, and in relation to visual narratives in general such as cinema, probably the most important consequence of Bordwell’s conception of schemata is their finiteness; they cannot be consider as image-based, flexible or adaptable. For a more elaborate account of the weaknesses of Bordwell’s in this respect, see section 5.4.1 in Chapter 5. 106

were believed to “evoke a constellation of meanings larger than the sum of their parts” (Prince and Hensley 71). Thus, at the time of movies realizing the ideological vision of Soviet montage theorists, spectators were expected to struggle with some higher meaning made out of images brought to them in patchwork fashion. But all of those images were shot with a real camera; the lack of synthetic imagery in avant- garde cinema poses a key difference with digital compositing, as in the latter case images do not belong to any pro-filmic, photographic reality, but originate in the imagination of a team of animators and special CGI designers. This fact poses an issue which is not different but additional to the demands of analog film: not only spectators have to comprehend and perceive images given to them in pieces, as it happened in the analog days anyway, but now they also have to do so with images that they have never seen before.

Hochberg raised several assumptions regarding the mechanisms of human perception that apply to a fragmented art like cinema. Along with Brooks, his inevitable interest in motion pictures rose, among other reasons, from the similarities that movie-watching bears to the way we perceive the world: “motion pictures permit us to study the processes by which we comprehend, anticipate, and store the visible contents of the world” (“Perception of Motion Pictures” 206). His most crucial point regarding perception in general has been the fact that the latter works through successive glances instead of all at once. Affected by his Gestaltist background, he and Edward McAlister initially tried to quantify the quality of

Prägnanz, by finding a “a global simplicity metric that could account for perceived organization;” this is an essentially wholistic approach, as it assigns primary importance to entire patterns rather than specific parts or points of them (Peterson 107

419).63 But his subsequent study of Roger Penrose’s physically impossible figures64 showed him that perception of 3D structures do not verify his and McAlister’s initial findings. On top of that, Hochberg was fascinated by the famous Parks effect,65 which gave him insight as regards the picking up of visual information during the process of looking, leading him to withdraw from his initial wholistic approach, and move towards a piecemeal approach:

[T]he visual perceptual system can, under the proper conditions,

assemble a set of partial views that fall on the same retinal area over a

period of time into a single simultaneous form or scene; and that such

scenes synthesized in space out of time must comprise a large part of the

normal visual experience on which our attempts at perceptual laws are

based. (“In the Mind’s Eye” 76–78)

This assertion essentially states that perception of an object or scene is actually a cognitive entity, which is mentally reconstructed from the combination of several successive and partial views of that object or scene, and with the aid of memory. The fact that we see each part of an image separately from the whole thing, and thus we process it individually before moving on to the next part, is expressed in Hochberg’s

63 Hochberg and McAlister’s co-authored effort was published in 1953 in the essay “A Quantitative Approach to Figural ‘Goodness.’” 64 See Appendix II, pictures 10 and 11. 65 In 1965, Theodore E. Parks in his paper “Post-retinal visual storage” reported an experimental finding on partial viewing. Parks’ experiment involved a pattern hidden behind a surface that only had a narrow cut on it; the pattern was moving, whereas the surface with the cut remained stationary, thus enabling only a successive view of parts of the pattern as it passed behind the cut. Viewers were nevertheless able to recognize the entire pattern through the narrow cut (Hochberg, “In the Mind’s Eye” 76). Hochberg conducted variations of Parks’ experiment using aperture viewing, probably the most important of which in his own famous experiment with the cross: a cross is seen moving behind a smaller and stationary round hole, which allows only one part of it to be revealed each time; viewers had no trouble recognizing the shape of the cross, as long as they were already familiar with a wider or complete image of it beforehand (Hochberg and Brooks, “Perception of Motion Pictures” 247). For illustrations of the Parks effect and Hochberg’s aperture viewing experiments see Appendix II, pictures 12 and 13 respectively. 108

famous phrase “our perceptions of objects are not everywhere dense” (“How big is a

Stimulus?” 324). Indeed, Penrose’s impossible figures produce illusions exactly because our visual system cannot see and process the entire shape at once; each part of them makes its own three-dimensional sense, and thus our mental representation of the figure assumes it to be correct, despite the fact that it is physically impossible.

After the dismissal of his early work on Prägnanz, Hochberg builds a model that includes mental images, thus defining the perception of images as abstract cognitive impressions of what we see. Following Helmholtz’s model of the successive stages from stimulus to perception, he lists three levels in perception: (a), the environment provides the visual system with “patterns of stimulation;” (b), the system reacts with “specific nerve energies” which pinpoint the effects that a certain stimulation has to our perceptual system, but they themselves are not yet

“psychologically accessible;” and (c), we have the formulation of mental structures: the experience of the other two stages teaches perception to anticipate specific patterns, much like Helmholtz’s unconscious inference (“On Cognition in Perception”

127). This is where the original model is complemented by Hochberg’s piecemeal view of perception; the mental structure, called schematic map, contains both the samplings of an object or scene, but also the expectancies that are raised in the mind of the observer regarding the object or scene, which derive from those samplings; or, as Hochberg plainly puts it, it is “the ‘glue’ by which successive glimpses are joined into a single perceptual structure” (“In the Mind’s Eye” 88–89). Schematic maps should be understood, on one hand, as abstractions containing information about visual structure, as well as all spatiotemporal information that will enable the 109

viewer to integrate and merge his successive views as belonging to the same object or scene, ergo also to the same space and time as well. In the process of saccades scanning an object or scene and picking up bits and pieces of its image, the mind not only elicits the mental schema that it deems most appropriate by which to perceive the object or scene correctly, but also directs the visual system on what kind of information it should next be looking for and where in the optic array, so that the schema is tested based on the sampling process that saccades carry out. In this respect, schematic maps are complex structures “entailing the processes involved in fitting an interpretation to the contents of the current view, motor plans regarding where to look next, and expectations regarding what will be found there” (Peterson

423). It is obvious, therefore, that schematic maps should consequently not be thought of as being mental picture-copies of the objects or scenes one sees. Instead, they are “schematic, not dense and consistent as is physical structure,” and definitely not concrete mental images corresponding to whatever is being seen; thus they bear none of the characteristics of the real world (Hochberg, “On Cognition in

Perception” 132; Hochberg and Brooks, “Movies in the Mind’s Eye” 387).

There are two important and related parameters to consider regarding schematic maps: schemata need to be already there, from prior knowledge, in order to be elicited during perception; therefore, we are interested, first, in the way they are acquired, and second, in the way they affect the direction of the saccades. As regards the first parameter, Hochberg was open to the starting assumption of “basic and relatively unalterable form-perception mechanisms, inherited or established early in the individual’s life” (“In the Mind’s Eye” 91). Assuming that this statement is correct, learning can be thought of as a process that starts with elementary and 110

crude perceptions of the world during infancy, and is constantly enriched by the integration of adjusting attentional eye movements on top of previous ones, without affecting previously acquired forms: “perceptual learning produces changes in the way in which momentary glimpses are stored, not changes in the form as registered in those glimpses” (94). This way, schemata become structures that also gradually incorporate, apart from the information about the scene or object, information about the eye movements themselves: our internal representation of the world as schemata therefore integrates knowledge of those movements, as well as their consequences (Hayhoe 450). This brings us to the second parameter: as form and saccade are both hardwired in schematic maps, the latter significantly affect where the next saccade will be, in order for the schema to be tested, a condition which places great importance to previous experience for the correct application of a schema:

[Hochberg] introduced the idea that perception in the momentary glance

works recursively with the perception of schemata in a mutual

relationship. For example, the processes of perception in a glance might

automatically evoke hypotheses (learned or innately acquired) about

what lies outside the scope of spatial resolution on that glance, leading

to a bias in where the next glance will be placed and what might be

expected to be found there. In turn, having an expectation of a particular

schema in a scene would lead to the testing of specific hypotheses in

subsequent glances. This recursive aspect to Hochberg’s theory made it

very clear that vision in almost every circumstance is influenced as much 111

by what lies in the mind as by what lies in the eyes of the beholder.

[emphasis added] (Enns and Austen 440)66

This clear and concise summary of the way schemata are elicited and affect perception of the world underlines the basic propositions of Hochberg’s theory. It is no surprise that Hochberg and Brooks have worked extensively on the effects that cutting in film has to the viewer’s comprehension of the cinematic world; as digital cinema is the definite art of vision in pieces, it calls for a testing of their propositions.

All this analysis generally refers to the actual visual and cognitive mechanism of humans acquiring schemata as they grow up. But the million dollar question here is exactly what kind of schemata these are. The answer to this question indeed helps to complete the puzzle of a cognitive model for moving images that can be applied species-wide. One possible explanation can be sought in the cognitive model of

Lakoff and Johnson. As we have noted earlier, Lakoff and Johnson’s approach is quite similar to that of Hochberg and Brooks as regards, on one hand, the fact that schemata are conceived in their model as image-based, and on the other the way they are believed to be processed by the human mind. These schemata, because of the fact that they are inextricably tied by the function of the human body, which is a shared human experience, constitute a stable point of reference that all humans abide to, and to which the explanation of any visualization, of any object, of any weird looking world must return to. Lakoff and Johnson have codified a number of types for these schemata; these codifications originate directly from the human

66 Hochberg, in the body of his work, has provided several similar definitions of the model of schemata. One of the most simple and straightforward ones states the following: “given eye movements are made in order to answer visual questions, the viewer must keep some account of the body of ‘questions and answers.’ Such an account comprises a ‘schema’ of the object or scene” [emphasis added] (“Perception of Pictorial Representations” 373). 112

body, and subsequently undergo metaphor and metonymy to be “translated” to different concepts: (i) the container or in-out schema, which is connected to the concept of “interior,” “boundary,” and “exterior,” and which enables the awareness of our autonomous physical bodies in relation to the external world; (ii) the part- whole schema, which is established on the fact that we perceive our bodies as having different parts (iii) the link schema, which is related to our position in relation to other people; (iv) the center-periphery schema, which is related the way we conceive our bodies as having more “central” or important parts for our survival, such as the internal organs, as well as “peripheral” or less important ones, such as the limbs; and (v) the source-path-goal schema, which is related to the movement of our bodies from one place to the other. Respectively, metaphors for these schemata apply for instance to (i) the way we conceive various states of mind, as “contained” in our mind; (ii) our family as a “whole” having different “parts,” its members; (iii) our social interactions or “links” to other people; (iv) theories as having core- and side-arguments; and (v) our “movement” in life towards goals purposes (Buckland

42-44). The structure, function, motion, and, ultimately, awareness of the human body dictates viewpoints (both physical and mental), shapes, concepts, and narratives in such a forceful way that the metaphors and metonymies are carried along by designers of CGI worlds into the process of worldmaking in such a way that the resulting universes cannot but obey to the same schemata as our real, everyday world. It is neither an option, nor an accident; it is the result of the finite way that humans are programmed to experience the world around them.

113

3.5. Digital Editing and Perceptual Re-construction: Applying Hochberg’s

Model

3.5.1. Then and Now: Changes to Consider

It should be clear that the very conception of cinema as a fragmented, non- continuous art agrees with the way perception is also piecemeal and constructive as

Hochberg has demonstrated. Hochberg and Brooks may have worked with analog cinema, but still their findings apply to digital cinema as well, since, as discussed earlier in the present study, digital cinema has inherited editing codes from both classical and formalist practices. Moreover, Hochberg and Brooks themselves acknowledge the application of their theory to any kind of cinematic image, with special emphasis on the fact that “no imagined layout, world, or universe is beyond visual experience” (“Perception of Motion Pictures” 207), especially considering the extent and sophistication of CGI special effects today. Opening their essay “Film

Cutting and Visual Momentum,” Hochberg and Brooks explicitly state that they were drawn to the study of cinema exactly because of the fact that the medium uses non- overlapping views of the world that it portrays, thus making both the moviemaker’s techniques and purposes interesting points to consider in their study; they believed that this analogy provides insight into the way we comprehend the world, as it shares the same skills that we apply in our “normal purposeful perceptual inquiries”

(207). For this purpose, together they applied to cinema Hochberg’s initial findings from aperture viewing experiments67 to the way extensive cutting affects mental representation of cinematic space beyond the screen, attempting to delineate the way the movie environment becomes a meaningful flow of objects, characters,

67 See footnote 65. 114

events and places, despite the fact that it is comprised of non-overlapping, piecemeal views.

Film cutting has been an essential feature of cinema ever since its beginnings, and the fact that the process actually does not even involve actual film any longer has made it even more visually obvious and prominent. Both major variations of film cutting, classical continuity editing and soviet montage, utilized it in ways governed by very specific rules and methods, regardless of the fact that differences in target groups and ideological objectives resulted in completely different realizations of editing in each case. It is also important to stress that, despite those differences, both sides were striving with the same goal in mind: to render on screen what they believed was the best version of reality that the medium could possibly offer.

Classical realists used mise-en-scène and seamless continuity to unite pieces of action and events in what they believed was a rendering of the narrative close to normal perception; prominent masters of Soviet montage went the exact opposite way: they celebrated technology as they believed that it could enhance visual perception.68 The fact that this “correctness” of representation of reality, both in continuity editing and in formalist montage, passed through technology, creates the affinities discussed in Chapter 2, but is also raises a few points to consider when applying the same model of perception to both. Thus it is wise to pinpoint both the similarities between analog and digital movies that make possible the application of

68 Vertov’s kino-eye, for instance, presented slices or particles of the raw truth of life, orchestrated by the moviemaker to a totality that celebrated the camera as the eye that sees and reveals a cinematic truth about the word, which is inaccessible by the human eyes. Vertov, in fact, had to defend his kino- eye technique against accusations of “formalism,” in the negative sense that the term acquired after 1934, claiming that it was an instrument for searching and revealing the unseen and unsaid truth of the world, the “kino-pravda” (Rutsky 95). 115

Hochberg’s model to both, but also the differences that raise challenges to this application.

As regards the similarities, the link with technology, as well as all other similarities mentioned so far, renders the model of visual perception discussed here as one that is open to application to digital editing as well. Once the material is digitized, it is automatically translated in its entirety to 0/1 values, which is the only language that the computer can understand. The computer as an editing machine does not distinguish between purely digital and digitized analog material (Lefebvre and Furstenau 82–83). Just as it happened in the days of analog editing, the computer flattens pro-filmic reality with synthetic imagery to create yet another version of reality, which does not compete with the normal reality of our senses like older cinematic modes did. In Manovich’s words, “synthetic computer generated imagery is not an inferior representation of our reality but a realistic representation of a different reality” (New Media 202). This environment is still cinematic, following spatial, temporal and narrative rules; as such, it is governed by the laws of screening any kind of real space, and therefore should make no difference to normal perception in terms of Hochberg’s model.

As regards the differences, on the other hand, the technology used in digital cinema also creates two discrepancies in relation to analog cinema, in terms of image perception, which cannot be ignored. Both differentiating features are related with the significantly increased complexity in digital cinema, in comparison to analog cinema: the first differentiating feature is related with the content of digital movies, which includes more and more non-existent and imaginary synthetic items. This departure from familiarity with the real world demands from viewers increased 116

cognitive functions of comprehension of a cinematic universe which is alien to what they are used to from their normal experiences in their real lives. The second differentiating feature is related with the constantly increasing speed of editing, which does not always allow the necessary time for viewers to fully absorb the content of moving images. These two major and acknowledged differences may undermine our attempts to apply Hochberg’s perceptual model to digital cinema.

The solutions to these potential issues will be sought within the model itself, despite the fact that it does not address these characteristics of digital cinema specifically.

We will discuss them in turn, and in the process we will be applying the model.

3.5.2. Addressing the First Differentiating Feature: the Cognitively

Demanding Environment of Digital Worlds

The visual extravagance of digital blockbusters usually makes them more challenging in perceptual terms compared to movies in realistic environments. This is because of the imaginary places they contain, or the points of view to which they place viewers which are normally impossible to occupy, or, usually, both. Therefore, mechanisms are required for viewers not only to make sense of where they are, but also to preserve that knowledge: viewers are navigated through an environment which is not only non-existent, therefore posing problems of recalling from memory the equivalent schema, but also presented in pieces which need to be put back together, i.e. the frames. The problem is further intensified by our physiology constraints: our visual system picks up significantly fewer details as we move further away from the retina, which means that we perceive in specific detail only the parts of images that the center of the retina sees. Consequently, images are perceived one 117

small part at a time, and never in their full detail.69 Normally, this is not a problem: we usually utilize only a few glances of a scene or object, because they are enough for perceiving it in its totality. But it is certain that this totality is not everywhere detailed, as a few glances permit only a small quantity of detail from each object to be picked up by the center of vision (Hochberg and Brooks, “Perception of Motion

Pictures” 246). Digital blockbusters, due to the fact that their content is not included in the familiar reality of viewers, should be expected to require a significantly larger effort in order for images to be properly perceived and reconstructed from those partial details. The application of schematic maps in Hochberg’s model of piecemeal perception should thus be expected to be a much more complicated process compared to that of viewing a movie that makes little or no use of extensive CGI worldmaking.

It was discussed earlier that an important feature of the model of schematic maps is the fact that the direction of successive saccades is governed by the viewer’s expectations, as to what piece of information should follow each view. The visual system directs the eyes towards the next piece of information that is more likely to assist the mind in eliciting the proper schematic map and building a cognitively comprehensible image. This assumption, as founded on human perception, is enough to affect editing, and thus produce good or bad cuts. Hochberg and Brooks note that montage in most movies, including digital ones, generally produces non-

69 This limitation is known as “retinal inhomogeneity" (Sedgwick 573). Hochberg notes that Gestalt theory completely ignored these constraints by practically assuming a simultaneous perception of entire figures; in reality, “the separate parts of the figure all have to be brought at different times to the central part of the retina, the fovea, if they are to be seen in full clarity of detail” (“The Representation of Things and People” 158). For a picture simulating visual inhomogeneity, see Appendix II, picture 14. 118

overlapping shots,70 which is exactly where the problem is located; in their own words, “[m]ost cuts do not bridge overlapping shots” since montage “provides sequences of views that are not related to each other by overlap or by common background. Only some mental representation can relate them” (Hochberg and

Brooks, “Perception of Motion Pictures” 259). In overlapping shots, viewers usually rely on landmarks, i.e. recognizable points of interest that function as anchors of meaning for the rest of the contents in the visual array (Hochberg and Brooks,

“Perception of Motion Pictures” 255–256).71 For example, in an interview scene, the camera will normally keep changing direction at an angle close to 180°, from the interviewer to the candidate; but the two sides of the same desk they are using will be picked up by the visual system, and a unified space will be constructed.72 Other known techniques include the consistency of the direction of the camera in moving scenes, or, more importantly, the direction of the actors’ gaze; the latter signifies for the viewer a very powerful vector that guides his next saccade, so powerful that it may even undermine legible landmarks, unless perhaps those are made extremely prominent (“Perception of Motion Pictures” 256–258). It should be obvious, then, where the problem lies with non-overlapping shots: there are no landmarks, at least

70 The terms overlapping and non-overlapping shots are used by Hochberg and Brooks to denote successive shots which either share or do not share, respectively, any common features inside their frames (objects, parts of the setting or background, people, etc.). These common features, as we will see further down, when they exist, act as landmarks used by our perceptual mechanism as it scans the image for meaning. 71 It is important to note here that overlapping shots using really obvious landmarks in this fashion, especially having landmarks appearing in places inside the consecutive frame that are really close to each other, are usually avoided in cinema, because they produce the effect of undesired apparent motion (discussed in Chapter 4). Hochberg and Brooks analyze ways in which bad cutting in cinema may have this effect and produce false perception in an edited scene (“Perception of Motion Pictures” 250–251). A very characteristic case of misleading apparent motion caused by overlapping shots is the jump cut. For an example of the jump cut, see DVD-ROM, video file 07. 72 For Hochberg’s own illustrations of overlapping shots see Appendix II, picture 15. For a visual example, see CR-ROM, video file 08. In this movie clip the metal jar that Galadriel (Kate Blanchett) is holding is an obvious landmark that creates overlapping between shots. 119

obvious ones, unless they are provided by some other means.73 Hochberg and

Brooks see the piecemeal view of edited shots in cinema as comprising a space that is much bigger than the screen; viewers are perfectly capable of re-constructing that space mentally, provided that they already have some clues about the unified space, in order to elicit the correct schema to match this piecemeal view. And that schema is provided by the establishing shot.74 The establishing shot has been a characteristic of opening scenes in both analog and digital cinema, in fact having the same function in both; but in digital cinema it is far more important in screening synthetic worlds than it used to be in the days analog recording.

Moviemakers make use of an establishing shot prior to the commencing of each segment of non-overlapping views, in order to provide viewers with a schema that navigates them in the space where the action is about to take place (“Film

Cutting and Visual Momentum” 207).75 As it was made obvious from Hochberg’s experiment of the aperture viewing of the cross,76 familiarity with as much space beyond the screen as possible is crucial for recalling the complete schema from pieces, and this prior knowledge is exactly what establishing shots serve in a movie.

But this prior knowledge has to be re-tested and renewed all the time in the course of the movie, as we are innately fine-tuned with an impetus to constantly seek new information about our environment. This impetus is called visual momentum; it is strongly connected to the notion of montage, and it has a dual function in cinema:

73 For an alternative to the overlapping shot, see DVD-ROM, video file 09. The trick of having Neo (Keanu Reeves) reflected on Morpheus’ (Lawrence Fishburne) sunglasses enables the viewer to situate the two characters in the same room facing each other even without the need for a long shot that shows them together. 74 This could be any open shot with a lot of visual information in in, e.g. a long or extreme long shot. 75 It is interesting to point out here that the phrase “the ‘glue’ that joins the successive views” is used to describe this establishing shot , which is exactly the same phrase that Hochberg has also used in his description of the schematic maps. See p. 108 in the present study for the exact reference. 76 See footnote 65. 120

not only can it be used to explain the usefulness of establishing shots, but also the need for constantly changing angles and views when editing a movie, even for no apparent reason, so that the viewer’s interest is maintained, a need that moviemakers and editors respect entirely.77 Therefore, visual momentum is the

“motivating factor” that enables the viewer to continuously search, draw and test schematic maps (“Film Cutting and Visual Momentum” 207); it explains why edited scenes, that stimulate this search, are interesting, while those that are static are considered dull. Visual momentum in cinema works best when the schema has already been established at some point earlier: this is also why digital blockbusters today not only rely on establishing shots prior to edited ones just like analog ones did, but it appears that they have intensified their use. Opening as well as intermediary scenes from some of the most highly profitable digital movies demonstrate a conscious effort to present the movie world in as many views as possible. In the days of analog fantasy cinema, on the contrary, this was either too expensive, not convincing enough, or simply impossible to do to such a degree.

The examples are more than abundant. Perhaps the most striking one is the digitally remastered edition of the original Star Wars trilogy released in movie theaters in 1997 and in DVD in 2004. The new edition not only boasted crystal-clear image and sound throughout the entire movie, but George Lucas also seized the opportunity to include a number of CGI establishing shots from locations in the narrative that were impossible to show in the original release, such as the new extended scenes in the town of Mos Eisley in Episode IV: A New Hope or the planet

77 The importance of visual momentum as a contribution to the perception of motion pictures is evident from the fact that it was the first and possibly the only explanation to this date for the reasons a static shot in a movie, as moviemakers say, “goes dead” unless it is soon revived by a change in viewpoint (Tan 567). 121

Coruscant in Episode VI: The Return of the Jedi.78 Two more recent examples are Zack

Snyder’s Man of Steel (2013) and J.J. Abrams’ Star Trek IX (2009). As both of them are reboots of well-known franchises, their directors obviously wanted to be explicit about their own visions of settings very familiar to the audiences. Snyder in Man of

Steel devotes a quite long sequence to featuring the planet Krypton, which is the only fantasy realm in this cinematic narrative, as the rest of the movie takes place on

Earth. Therefore, he devoted substantially more CGI external scenes, some of which he presented via the device of having POV79 shots of Jor-El (Russell Crowe) flying in the back of a winged creature. In a similar fashion, Abrams significantly highlights the way he presents his own USS Enterprise, as it is obviously one of the most iconic things in the history of science fiction.80 It is clear, therefore, that one of the main reasons that digital cinema makes this extensive use of long shots of inexistent places, such as spacecrafts, planets, fantasy realms, etc. is because they are enormously cheaper to create with the use of CGI, given their detail and size, and far more convincing and impressive; and most importantly, with the increasing complexity of cinematic worldmaking, they are also practically mandatory for the viewers’ orientation and attention.

Visual momentum affects other characteristics of piecemeal perception, i.e. the fact that the information that this active search entails is not random, as well as connected with speed and complexity of stimuli, a characteristic which we have shown to be inherent in digital movies. Visual momentum explains the general tendency of vision to look for information, which is broken down to two separate but

78 See Appendix II, pictures 16 and 17 respectively. 79 A common acronym for “point-of-view” camera angles. 80 See DVD-ROM, video files 10 and 11 respectively. 122

interrelated functions: Peripheral Search Guidance (PSG) and Cognitive Search

Guidance (CSG). These two functions together, which will be explained in detail in the following paragraph, deliver the best possible combination and ordering of saccades in order for visual perception to be as effective as possible even in non- overlapping views. They are interchangeable but also complementing each other in the search for visual information, and are extremely useful in cases in which the visual array keeps changing unexpectedly in terms of both speed and content, as in digital blockbusters.

These two functions are perfectly applicable to digital cinema, because both image construction and combination is more differentiated in CGI, thus welcoming the test of the theory. Each time there is a new cut, the viewer needs to decide very quickly whether it opens a new scene altogether, in which case there is a new search for orientation, or it is a different angle of the previous shot, in which case previously located landmarks may be applied (“Perception of Motion Pictures” 261). Initially, the role of PSG is to pick up more general information from peripheral vision by locating any familiar parts of the image that are more likely to act as landmarks for subsequent views; then, CSG takes over, which tries to answer “more cognitively driven questions” about the meaning and content of the parts of the image close to the landmarks (Hochberg, “Components of Literacy” 126; Hochberg and Brooks,

“Film Cutting and Visual Momentum” 216). These two functions are interchanged in the process of perception each time the visual array is altered and can very well explain a number of issues concerning synthetic imagery as well. This is because the content of composited images partially derives from the real world; the content of digital narratives, no matter the level of CGI they incorporate, always derives 123

partially from the object and scene experience of the real world. Designers of visual graphics are human beings; therefore they reproduce and embellish images from their own memory, dictated by the schemata they have been constantly acquiring and amending since their own birth. It is logical then, that synthetic imagery still comprises of largely recognizable parts, textures, shapes, etc., coming from a shared and commonly experienced sense of reality.81 The difference lies not in what they are using, but in how they are using it: those elements are known, but they are merged by CGI designers in combinations that are not found in real life, hence their non-familiar looks. If that was not the case, they would have been creating images that make no visual sense whatsoever, a condition that may be welcome in abstract art for aesthetic, philosophical or other reasons, but certainly has no place in commercial cinema.

Using the description of PSG and CSG we can account for the viewers’ visual functions as they watch a digital blockbuster on the basis of normal perception; but most importantly, we can also provide an explanation for the perception of CGI compositing. Images of the latter kind are normally expected to be put together using stock imagery of recognizable objects along with synthetic ones, some of which

81 This observation provides another link between Hochberg’s model and that of Lakoff and Johnson, based on the type of schemata that they both assume. As we have seen in section 3.3.4, in Lakoff and Johnson’s model, schemata are acquired from the experience one has with the space around him, using the human body as a point of reference. The experience of space is thus a collection (and a re- collection, for that matter) of “simple structures that arise from the body, up-down, back-front, center-periphery, part-whole, inside-outside, paths, links, force and so on,” which are completely dependent and “constrained by the dimensions of the human body” (Buckland 42). The purpose of the two authors was to seek for a model that would be universal for all humans, and thus bypass “radical relativism and subjectivism;” the result was a model that conceives of image schemata as abstract but not arbitrary, as they are always based on the common and shared experience of the functions of the human body, which is the same for everyone under normal conditions (42). It is exactly this universality and objectivity that Hochberg’s model also addresses, in its own attempt to define the way humans actually see images, not the way they choose to explain them, as other philosophical or theoretical trends commonly do; and the fact that both Hochberg & Brooks and Lakoff & Johnson arrive at image-based definitions for the schemata is obviously not accidental. 124

will not be familiar from the experience of the real world. During the initial scenes, and with the possible aid of establishing shots, PSG is used to trace the former of those objects in a normally coherent and clearly defined space. This recognition takes place on the basis of known objects, as well as features such as light patterns, texture, edges and size, unconscious attention to laws of physics, etc. A spacecraft, for instance, may have any awkward shape the CGI designer wishes, but it will be normally perceived by visual cues in the background (stars, planets, a familiar view of earth from space, etc.), as well as knowledge and recognition of generic concepts like cockpit, engine, antenna, etc; also, despite their alterations, the planet Pandora in Avatar is still essentially a tropical rain forest with native tribes and exotic animals, and the realm of the Shire in The Lord of the Rings trilogy is really nothing more than a picturesque rural British village.82 Even a completely artificial and blatantly synthetic environment will work in the same way: the racing arena in Tron: Legacy

(2010) is impressively realistic despite the fact that it practically comprises only of patterns of light; yet, these patterns still obey to the structural laws of perspective, and the arena itself is shaped like a normal stadium, both of which elements create a familiar landscape for viewers.83 Let us also not forget the fact that fantasy blockbusters very often make use of small captions in establishing shots, which are immediately picked up by human vision; via the process of visual momentum, the fovea will be directed towards the caption because our perceptive mechanisms recognizes the script at the bottom of the screen as potential source of information.

It is important for CSG to operate on making use of the captions; with the additional information that the latter provide, it becomes easier for the spectator to elicit a

82 See DVD-ROM, video files 12 and 13 respectively. 83 See DVD-ROM, video file 14. 125

schematic map that quickly establishes a spatial familiarity with what is being screened; this familiarity will not only secure that the scene is comprehensible on its own, but will also provide landmarks which will enable the viewer to mentally re- construct the off-screen space of the scene, immediately after the first cut away from the safety of the establishing shot.

After these landmarks are set, CSG scans the image with additional saccades for content-specific information; this is the stage where CGI items are contextualized in the visual and narrative content of the movie, and are recorded in memory for use later in the story world. Creatures, objects, places, etc. are placed under visual scrutiny and are analyzed visually as well as functionally, always obeying the worldmaking rules of the storyline. Because of the limited time available to absorb and contextualize the usage of world-specific items, clarifications given in the dialogues are often used to facilitate visual perception, while other times any help is deemed redundant: in Star Trek (2009) and Star Trek Into Darkness (2013), for instance, alien races like the Romulans or the Klingons, or objects like phaser guns are not explicitly introduced, as they are already considered contextualized and well- established in the Star Trek universe since the beginning of the franchise several decades ago. Inexistent objects, creatures, etc. are thus perceived via their cognitive contextualization within known objects and forms; expressed in terms of the model reviewed here, they are incorporated to familiar schematic maps. This may indeed be a fuzzy notion to grasp, and can be related to Penrose’s impossible 3D objects for theoretical backing. Applying his study of those objects to all others, Hochberg emphatically stresses the difference between objects themselves and our perceptions of them, and warns about the fallacy of assuming that “the perceptual 126

response to an object is in any sense an object” (“How Big Is a Stimulus?” 324).

Penrose’s impossible objects, like normal ones, give us the sense of a mathematical integrity of structure, because we perceive them partially. It just so happens that these objects also prove that perception of all objects is a collection of attributes, with no complete internal model of the object existing in the perceiver’s mind; in short, attributes of all objects are the result of “our intention to perceive” (324–325).

We do not need to have beforehand a complete and specific image of the Gollum for it to make visual sense in the universe of The Lord of the Rings. Non-existent CGI objects and creatures can thus be defined as collections of attributes, which will be validated in the context of the cinematic universe by the constant intention and effort of the viewer to make sense of them. This validation will take place, on one hand, based on a certain functionality that the object or creature has in the world of the movie; and on the other, it will be facilitated by the fact that it will be recognized because the CGI designer has necessarily established the design based on attributes from real life, shape, texture, etc., which are also shared by the viewers.84

3.5.3. Addressing the Second Differentiating Feature: Rapid Editing

The second crucial feature that differentiates traditional montage and editing from digital one is the constantly increasing speed between cuts. The connection between fast editing and the tone that moviemakers wish to communicate has been observed since the silent years. According to Hochberg and Brooks, fast pace rhythm has been proven to be connected with action; both Eisenstein and Pudovkin had noticed and, in fact, agreed upon the “emotional or affective effects of cutting”

84 The comment of footnote 81 regarding the similarities between Hochberg’s model and that of Lakoff and Johnson obviously also applies here. 127

despite their theoretical disagreement in the way this editing should eventually be realized (“Perception of Motion Pictures” 272). Editing in contemporary digital blockbusters constantly becomes significantly faster than in classical cinema, and even faster than soviet avant-garde, with a far shorter average shot length, a characteristic that Bordwell includes, along with other features, into what he calls intensified continuity.85 This feature of rapid editing has come to be considered absolutely necessary for a movie to perform well, revenue-wise (Messaris 3; Reisz and Millar 41).86 There are several explanations offered as to why this has happened;87 but sources seem to converge to the effect of digital technology, which has made editing both faster and cheaper (Messaris 3; Bordwell, “Intensified

Continuity” 23; Lefebvre and Furstenau 71).

Fast-paced editing poses an issue with applying Hochberg’s model of schematic maps to digital cinema, as the increasing speed of cutting in contemporary digital movies has been proven to affect perception. The process of perceiving the fragmented movie space through editing entails picking up a considerable amount of information, but the rhythm of picking it up is not stable; according to Hochberg and

Brooks “we seem not to treat all information equally: certain aspects act faster (or

85 Bordwell provides a historical account of the constantly shortening average shot length (ASL) over the last sixty years of the Twentieth century. According to his account, ASL has dropped from an average of eight to eleven seconds from 1930 to 1960, to an average of three to six seconds at the end of the century, with examples even dropping as low as 1.8 seconds (“Intensified Continuity” 16– 17) 86 In stressing this necessity, Messaris actually notes that Steven Spielberg, a director obviously in a position to make whatever choices he pleases, had to refer in an interview over and over again to his decision not to apply rapid editing to Indiana Jones and the Kingdom of the Crystal Skull (2008). 87 Bordwell mentions various possible reasons for this, for example the change in aesthetics brought by television; the producers’ demand for multiple alternative takes of a scene in order for more options to be available in post-processing; multiple-camera filming; etc. (“Intensified Continuity” 22– 23). Messaris adds, apart from the easiness that the digital medium provides, the shift of Hollywood towards young male audiences, which demanded faster-cut movies as those are generally thought to denote action; and the general pressure that cinema is under, from other forms of entertainment, like gaming, that are more fast paced anyway (5). 128

more strongly) than others” (“Perception of Motion Pictures” 254). Elsewhere,

Hochberg and Brooks relate this statement to what we have seen to be the functions of PSG and CSG:

There are some results that suggest that the course of active looking, and

of the visual momentum that presumably impels it, can be partitioned

into components: a fast component that brings the eye to those

peripherally visible regions that promise to be informative or to act as

landmarks, and a more sustained component that directs the eye to

obtain more detailed information about the main features that have

already been located. [emphasis added] (“Film Cutting and Visual

Momentum” 225)

If we assume the existence of an optimal rate of cuts in a scene, which retains the sense of action yet permits the best possible combination of collecting, retaining, and combining information, it is clear that changing the editing speed in a movie will definitely affect the comprehension of the image as well. Not only do we not collect the entire amount of visual information available in our visual array, due to retinal inhomogeneity and the storage constraints of our working memory bank, but we also dump a large amount of that information as soon as our perceptual mechanism determines that it is not useful to us anymore:

It is generally agreed that, following a change in gaze position, observers

retain in memory only a small number of items, or “object files,”

consistent with the capacity limits of visual working memory, together

with information about scene “gist” and other higher-level semantic

information. (Hayhoe 448) 129

On one hand, retaining a very small amount of that information rather than an entire copy-image of whatever we see is a function that indeed agrees with the assumption that perception utilizes schematic maps, and makes our visual system highly economical, as Helene Intraub notes (“Visual Scenes” 217–218). Hochberg and

Brooks themselves also note that omission of unnecessary events in motion pictures is an effective usage of valuable screening time which demonstrates “an economy that is the essence of an art form” (“Perception of Motion Pictures” 234). But on the other hand, in the context of a cultural product like fast-edited digital movies, it inevitably raises questions as to whether the available information is enough for properly viewing the movie. Moreover, in addition to cutting rate, Hochberg and

Brooks in their study also found the complexity of the contents of a shot to be a factor that intensifies visual momentum (“Film Cutting and Visual Momentum” 225;

“Perception of Motion Pictures” 275).88 We have already addressed in the previous section the fact that digital movies tend to increase the amount of available visual information, both qualitatively and quantitatively, and the cognitive demands that this entails. Therefore, it is reasonable to ask: what is the usefulness, if any, of an editing method that delivers what it seems to be a perceptually deficient visual product?

This visual deficiency of digital fast-paced editing is evident both in practice and in relevant perceptual psychology literature. On one hand, there are numerous examples of movies infused with CGI that contain action scenes in which practically

88 Study of these factors and their effect on visual momentum was based on the notion of affective cutting tone, an “aesthetic arousal variable” in Hochberg and Brooks’ words, introduced by Raymond Spottiswoode in 1933. Based on Spottiswoode’s own movie-viewing experience, the term referred to the fact that “[s]equences of simple scenes would sustain high affective cutting tone when presented at a fast cutting rate and would be reduced at slower rates, whereas the reverse would be true with the more complex scenes” (Hochberg and Brooks, “Film Cutting and Visual Momentum” 208) 130

little or no actual detail can be discerned: the highway chase in The Wachowski’s The

Matrix Reloaded (2003), fight scenes in Michael Bay’s Transformers (2007),

Babydoll’s (Emily Browning) combat visions in Zack Snyder’s Sucker Punch (2011), and many more.89 On the other hand, relevant literature has already noticed and analyzed the fact that we only notice certain visual stimuli anyway, and only under specific conditions, a condition termed change blindness. According to Daniel Simons and Daniel Levin, change blindness refers to the observation that we are generally unable to detect changes from one view of a scene to the next, unless we attend specifically to those changes, suggesting that we are also unable to retain many of visual details from one view to the next. Although change detection is highly dependent on attention to objects at the center of vision, research indicates that our visual system, on one hand, is not likely to detect changes, if it attends to more than one central object at a time, regardless of their position in vision (265). And even with one point of attention, we are still blind to changes if they do not affect “the gist of the scene;” if that gist remains the same, we unconsciously assume that details also remain the same (266). It is obvious that movies are not only established on visual momentum, which precisely means constantly shifting the viewer’s focus, especially in digital blockbusters, but they also rely on the invisibility of continuity editing to sustain the general visual theme of each scene without viewers noticing potential mistakes. It is also obvious that Hochberg’s entire model of schematic maps relies on the fact that we cannot retain details, but more general and abstract structures of what we see. Digital cinema intensifies this issue, which has been already addressed in relation to viewers’ sensitivity to picking up errors in continuity

89 See DVD-ROM, video files 15, 16 and 17 respectively. 131

as a common characteristic of all moviemaking. It exemplifies our general inability to pick up many details from the visual world, which is the cost of having such an economical system of perception (Intraub, “Scene Perception” 455).

Based on all this, the question now appears more imperative: why does digital moviemaking insists on an editing mode that obliterates attention to details?

One possible answer could be that cinema production is still experimenting with speed: knowing for certain that action is intensified with increasing speed, moviemakers try to deliver as much action as possible, testing the limits of viewers’ cognitive capacities, until a presumed golden mean is achieved between speed of cutting and effectiveness of action scenes. But surely this assumption is false: instead of a tendency towards this presumed golden mean, Bordwell’s research demonstrates that blockbusters still tend to become even faster, despite the fact that experimental findings on speed of perception have long ago set clear limits to the amount of information an observer can retain from changing visual arrays.

Besides, on a more practical basis, this is the era of rapid information: if audiences leave movie theaters displeased by any aspect of a much-anticipated blockbuster, including speed of course, it will be all over the internet in a matter of days or less, depending on the hype of each movie. It is easy for producers and moviemakers to know quickly what they did wrong. This leaves us with one reasonable assumption: if a mode of moviemaking is chosen that undermines attention to detail, this is probably because it is done on purpose; details in digital cinema are meant to be obscure and go unnoticed.

Digital cinema was never meant to be about mise-en-scène masterpieces like those of Godard or Antonioni; it is supposed to be about trouble-free entertainment, 132

primarily of younger audiences. This attitude, by definition, shifts the focus of those movies: from attention to scenic details and composition, to a wider, more generally conceived fidelity to the cinematic universe that the movie takes place in. In the case of movies based on Marvel Comics, for instance, moviemakers do not rely on very specific details in each shot, but on maintaining a typical sense of continuity in a half- convincing Marvel Universe. Movies advertise their use of advanced CGI, but this does not mean that the level of detail is state-of-the-art in all CGI scenes throughout the movie; and there is no reason for it to be. Even if CGI technology is capable of rendering sufficient visual detail in all parts of a movie, the very fact that the movie will anyway be fast-edited makes this option redundant as viewers will never notice it due to the limitations of human vision. Things are a bit different in long shots: a lot more attention is paid there by designers, as even a few seconds will suffice to reveal imperfections in the synthetic photographic-like fidelity of CGI, which will be enough to destroy the illusion, and the movie to be disreputed as “cheap in graphics.” Of course, this observation depends on the time a movie was made: for example, today viewers may sneer at the CGI long shots of dinosaurs in Steven Spielberg’s Jurassic

Park (1993), or the alley duel between Neo and the viral Agent Smith in the

Watchowskis’ The Matrix Reloaded (2003),90 which are hardly convincing anymore, but were state-of-the-art CGI effects when those movies were produced. Still, long shots back then compensated, as they do now, and balanced whatever visual information could not be shown in fast scenes. They conceal all kinds of mistakes caused by rapid editing: continuity, structure, form, etc. In a synthetic visual environment, it should be far more likely that visual inconsistencies will occur, as the

90 See DVD-ROM, video files 18 and 19 respectively. 133

composition of the image relies on the creational capacities of visual designers. A mechanical apparatus will never “forget” to capture the scene that lies in front of it; but not all visual details are that easy to incorporate in digital movies.

The use of speed to cover mistakes is neither a privilege of digital movies, nor of all contemporary moviemaking in general.91 The only difference is that Hochberg’s model and experimental research on change blindness provide it with experimental substantiation. The use of editing to cover mistakes was explicitly noted as advice for moviemaking even as early as in the writings of Kuleshov, and is even incorporated in contemporary editing practice as a suggested technique that will make viewers blink and thus not notice a cut (Simons and Levin 264). This methodology is practically identical to what perceptual psychology includes in the checklist of things that produce change blindness:

Basically, interrupting an observer’s view of a scene by an eye blink, a

brief flicker in the image, a brief occlusion by a passing object, or a

change in viewing position can render the observer insensitive to

changes in the location and identity of many objects that are not the

current focus of attention. (Enns and Austen 440)

It makes perfect sense to argue that techniques such as these in moviemaking are multiplied as editing becomes faster: as the demands of visualization increase in rendering fantasy worlds, so do the chances of making mistakes in animation and worldmaking. Fast editing in digital movies is an easy way to cover all these mistakes, both those that are known and those that are presumed to have occurred but have

91 In this respect Messaris refers to Bordwell’s comment about Paul Greengrass’s The Bourne Ultimatum (2007); Bordwell has claimed that extremely fast editing conceals the overall bad directing of the movie (Messaris 5). 134

gone unnoticed by the creators of a movie. If fast-paced editing also works in favor of action, so much the better. 135

CHAPTER 4

MOTION: BRINGING IMAGES TO LIFE

4.1. Motion as Life: from Automata to Cybernetics

In a discussion about how synthetic imagery in cinema affects the viewers’ perception and sense of reality, one of the most important strategies is locating those features that actually denote life and produce a realistic response. Human beings are accustomed from real life experiences into assessing reality, or at least proximity to it, based on certain characteristics which may or may not be exhibited by what they are looking at. One of the most powerful visual cues to denote life is motion. Due to retinal inhomogeneity and the fact that the human visual field is spatially limited (i.e. not covering 360°), the world is always seen as if through an opening, with its clarity not even being equally clear over its entire surface; this condition makes motion a critical factor in the perception of scenes (Intraub “Scene

Perception” 454). Humans are extremely sensitive to motion and there is good reason for that; in the course of our evolution, perfecting the acuteness to detect motion was the thin line that separated food from starvation, and life from death.92

With motion being such a crucial element of everyday life for millennia, it is therefore not a surprise that humans have evolved to identify motion as a signal of life in their immediate environment. Such a signal alerts the perceiver in a way that actually situates him consciously inside this environment; consequently, a signal of motion means a signal from reality. It is difficult to imagine this feature ever being

92 This is essentially an ecological approach; it is very much akin to that on which J.J. Gibson and, subsequently, Joseph and Barbara Anderson established their own theory of cinema perception. For a deployment of this theory please refer to The Reality of Illusion by Joseph and Barbara Anderson. 136

transmitted with a paralleled intensity by any still picture. Despite their similarities and kinship, still images and motion pictures definitely do not share the same realistic impact, due to the fact that the former are incapable of stimulating the same primordial, instinctual and ontological connection between man and his constantly moving environment. The fact that digital motion pictures move, therefore, is clearly an advantage as regards their realistic perception; but is that enough to counterbalance the blatant artificiality of their synthetic content? This is the question that this Chapter will address.

The importance of motion in the way objects in the world are perceived is revealed by the attention motion has attracted over time. The connection of motion to life is manifested in the long-lasting efforts of man over the centuries to create automata, machines that could move perpetually without any exterior force keeping them running. Silvio A. Bedini, for instance, provides an exceptional summary of this quest for perpetual motion, as well as to the role it played to the evolution of modern technology. The accomplishment of perpetual motion is explained in terms of man’s desire to “simulate nature and domesticate natural forces” by imitating natural functions of animals, such as flight, underwater swimming, etc, which would provide him with control over nature (Bedini 24).93 But on a wider perspective, this control over nature is essentially a control over mortality: achieving perpetual motion reveals man’s desire to unlock the secrets of life itself. Historically, the obsession with automata is revived and sustained during the Renaissance, demonstrating the general “rise of mechanicism” of the era (24). Bedini describes in detail the way Renaissance engineers and their patrons brought back from oblivion

93 See Appendix II, picture 18. 137

the studies of Heron, Ctesibius and Philon, commencing in a quest for perpetual motion that lasted for centuries, and attempting to produce all sorts of mechanical animals and machines featuring initially hydraulic and pneumatic and later clockwork types of motivating mechanisms. Bedini comments that most of these machines caused a lot of exaggerated word-of-mouth hype in their time; this, to him, is an indication that, if they existed at all, these machines were probably deceiving illusions rather than true automata (31). Nevertheless, real or not, their exaggeration does reveal the general preoccupation with animating synthetic figures, as well as the immense fascination of people being in the threshold of knowing that the figures are not alive, but at the same time functioning as such.

The eighteenth century was the time when the creation of automata became more bold, as inventors began experimenting with giving them a truly functional human form. These early androids are a clear indication that the ultimate goal behind their creation was to simulate life in some way. Bedini lists several known androids of this sort that were invented during this period, which were complicated devices despite the fact that they could perform, at best, only a handful of tasks, such as writing a specific poem or drawing a couple of pre-programmed sketches

(37–39).94 The fact that the form of the automata expanded from miniature soldiers, animals and fountains, to full-scale human forms reveals that the goals of inventors of the time were, by then, motivated by an enhanced confidence in the capacities of the machine and human knowledge, probably influenced by the general spirit of the

Enlightenment:

94 See Appendix II, picture 19. 138

It is interesting to speculate how many among the countless number of

mechanicians [sic] who worked to produce automata for whatever

purpose may have secretly nurtured an ambition to go a little beyond.

Certainly the temptation was always at hand to attempt to create life

itself in giving birth to these lifeless figures by a combination of alchemy

and mechanics. (Bedini 40)

This obsession with automata, especially human forms that could move autonomously, is therefore the higher aspiration of inventors to create life, or defeat death, whichever way one wishes to see it. In fact, it was not limited to man-made machines; it is also evident, for instance, in the impact of Luigi Galvani’s experiments, who animated dead animal body parts using electricity: it was a side effect of his scientific studies that the motion produced by the electrical stimulation was received by many as a metaphysical expression of revival. Spectators of androids in action and Galvani’s electricity experiments were almost certainly aware of the fact that the former were machines and the latter involved dead animals; regardless of that, though, fascination with artificial motion clearly shows that after the eighteenth century Galvani’s experiments truly became a sign of scientific progress that signified some kind of control over nature.

The above observation about the eighteenth century also reveals a consistency with the lineage of imaging, as this was set forth in Chapter 1. On one hand, during the Renaissance, the turn to ancient Greek culture and technology lead to a fascination with mechanics that culminated in the creation of automata; it was this same preoccupation with construction, quantification and measuring that gave birth to perspective, the forefather of contemporary visual cybernetics, and 139

ultimately, digital image as well. And on the other hand, the rise of android creation during the eighteenth century coincides with the time when photography was invented; under this light, both of them seem to be instances of an effort to capture, manipulate, animate, and multiply the human form, an effort that also reveals man’s metaphysical aspiration to reach the divine. Of course, the science of the time was still oscillating between metaphysical and more pragmatic explanations for motion.

Helmholtz, for instance, whose work was a major source of influence in Hochberg’s work, also studied automata and perpetual motion, in the context of his studies trying to refute the validity of vitalistic approaches95 to science, especially that of

Georg Ernst Stahl. Helmholtz infused electricity to frog cadavers to demonstrate that there was really no need “to postulate the intervention of metaphysical forces in a living world of which the coherence and dynamism could be explained in terms of physicochemical processes” (Meulders 66). By claiming this, Helmholtz attempted to orient research towards a form of science more reliant than the one vitalism could offer.

This contradiction between the two explanations for motion, the metaphysical and the more pragmatic one, was therefore also a contradiction of what constitutes life. The eighteenth century was probably the first time that science was deemed to be developed enough to address this issue, probably because of the slackening grip of religion on the attempts made to answer the major questions of life and existence. Even more, it is indicative of the fact that the perception of life is

95 Vitalism advocated the essential difference between animate and inanimate things based on the assumption of a “non physical element,” a certain life-giving force like a fluid or “spirit.” Vitalism opposed the “mechanistic” view of Descartes and his successors that animals and humans are some sort of automata, whose only difference from man-made devices is the degree of their complexity (Bechtel and Richardson 1051). 140

intricately tied with the perception of motion. And although science today does not address the matter so vigorously, the question is still around: the recent development of photorealism in digital graphics and the questions of virtual reality that it also raised, clearly betray the silent acknowledgement that the human perceptual system may easily be tricked into believing the veridicality of objects in motion as real; or even worse, it may even blur the very distinction between reality and non-reality, altogether. In fact, the case with Galvani’s experiment is very close to that of motion in digital cinema, in terms of animation producing a life-like effect: it creates a similar sense of seeing living things, whereas in reality viewers are perfectly aware that what they see is entirely synthetic and manipulated by technology. This state of “in-between-ness” is actually the experience spectators seek by going to movie theaters in the first place. This thin line between what one knows and what he perceives, or is fooled to perceive for that matter, is at the center of discussion of CGI in cinema and other digital media, because it engages viewers in interacting mentally – and sometimes physically as well – with an oxymoronic environment: one that is amazingly unreal, yet calls upon their cognitive perception of real space and real objects. It should be obvious by now that when perceiving motion, as is the case with other perceptual functions of the mind, the intention of the viewer defines to a great extent the ontological quality of the stimulus, i.e. whether what is being seen is an illusion of motion or actual motion.

The same was probably the case in the early days of cinema as well. Viewers of the first film reels shown in fairs inside shabby tents, were drawn to the new medium precisely because of motion, which was the feature that distinguished it from photography. Those early films were considered a technological miracle of 141

recording real, living things, but of course viewers were aware that reality is not black-and-white, speckled, mute or blurry. As it happens now with CGI, motion was enough to do the trick, or at least did a good part of the job. Moviemakers very soon realized that cinema had far greater potentials than a mere recording of people walking around in parks and city streets.

Coupled with the audiences’ attraction to a moving spectacle, especially following the freak show tradition of theme parks, it was only a matter of time that fantasy cinema was established. Nevertheless, although creatures and monsters were relatively easy to build, once again it was their motion that had to be figured out. The problem of animating these fantasy worlds and beings was practically born along with the tradition of horror and fantasy itself. And from that problem and its solution, cinema was endowed with several noteworthy techniques that are still around today, such as stop-motion animation.96 Utilized in horror and sci-fi films of the classic era and featuring some of the most interesting b-monsters in the history of cinema, stop-motion was a technical feat of the time, as it required elaborate and time-consuming workload. The images that this method finally produces and which reach the eye are composed by bits and pieces of a completely different reality, manipulated by the moviemaker in such a way that it gives the deceiving impression of unity, life and coordination. Despite the fact that stop-motion seems quite crude and choppy today, it demonstrates the need to animate cinematic forms, even in a not-so-perfect manner, as animation affects quite significantly the level of realistic

96 Stop-motion animation requires shooting motionless puppets for a split second at a time on such positions that when the stills are projected rapidly together as a string of images they give the impression of live movement. For one of the best examples of original stop-motion in cinema, see DVD-ROM, video file 20; it features the skeleton fight scene from Don Chaffey’s Jason and the Argonauts (1963), in which the special effects by Ray Harryhausen are considered his masterpiece. 142

effect of an image. From a certain viewpoint, stop-motion is the forerunner of contemporary digital graphics. In fact, the way digital forms are animated today reminds in many respects the way puppets used to be animated.

4.2. The Debate on the Reality of Motion

This attention to detail, and, most importantly, the success or not of the final product, has been central to the impression of reality that moving images produce, digital or otherwise. Ever since the initial momentum of photography started to slow down, and cinema to gradually take over the fascination of audiences, the perception of apparent motion drew the attention of research on the perception of pictures. Several theories of perception of motion have been applied to cinema motion, with a varying degree of success or influence. These early accounts may disagree in several areas, but appear to agree in one key feature: they all treat motion seen in cinema as apparent motion. Cinema was traditionally treated as an illusionistic medium, and, as such, the moving objects inside it were also seen as illusions, and no real motion was thought to take place. With the passing of decades and the theories of cinema growing more mature, other theories were willing to accept that motion in cinema is real, like any other motion perceived in real life.

Thus, two aspects of reality of cinematic motion arise: the technical and the psychological one. The technical reality of motion in cinema refers to the question whether this motion actually takes place before our eyes, i.e. it is a physical displacement of an object from one position to another. This aspect, therefore, concerns motion as an ontologically real event. The psychological aspect, on the other hand, defines reality of motion not in terms of an ontological or physical event, 143

but in terms of what we perceive; in simple words, cinematic motion is as real as any other motion we pick up from our normal environment, since it stimulates our perceptual system in the same way, and it elicits the same response, e.g. physiological, emotional, reflexive, etc.

This distinction is neither a novel nor an unknown one, but its significance has changed over the decades. The reason for this change is the introduction of digital technology in cinema, both in terms of the implementation of CGI in the motion picture, as well as the new modes of projection in digital theaters. This physical change in the medium, from photochemical recording of pictures to their synthetic creation, has inevitably brought with it the need to re-evaluate the level or kind of reality, so to speak, of the cinematic image. In terms of motion in particular, the problem arises from the fact that all traditional theories took for granted analog projection of celluloid reels; after all, with no other technological alternative available, the analog form of recording and projection was taken for granted as the definitive technology of cinema. Consequently, all these theories started from two main features of cinema technology: (a) the fact that film frames were recorded successive stills of actual motion that indeed took place physically in the pro-filmic reality; this applies to early fantasy films as well, since the better part of special effects of the time, from Méliès to stop-motion monsters of the 1940s, relied precisely on cutting and gluing together film strips. And (b) seeing movement on cinema relies on stroboscopic motion, i.e. the apparent motion caused by the succession of film frames at specific intervals.97 It is obvious, therefore, that the issue

97 Hochberg defines stroboscopic motion as “successive small spatial changes in the static patterns of luminance [that] appear as movements from one place to another.” He notes that this is the exact same effect that produces apparent motion in both cinema and television (albeit with an aesthetic 144

of the perception of motion in cinema in terms of it being real or not, needs a revision, in order for theory to keep pace with new technologies of moviemaking.

It appears that it all comes down to the way one defines reality or realism.

Traditional theories of cinema addressing motion among other things, beyond their individual merits, shortcomings, disagreements, etc., commonly shared pro-filmic reality as a stable point of reference against which they could always test the validity of their claims. Digital cinema does not have pro-filmic reality; it is relieved from the need to capture motion with a recording apparatus that took place at a certain place and time. Nor does it necessarily rely on the intermittent succession of frames for projecting the movie, as digital projectors produce a constant and uninterrupted beam of light.98 So, is motion in digital cinema more real, less real or a entirely different story than that of analog cinema? Reality may have managed to hold its grounds for decades amidst fierce cinema theory debates, but in the digital era there appears to be no escape any more for its dismantling: among cognitive film theorists,

Gregory Currie, for instance, claims that realism is a matter of degree, and admits that “[o]ne way to argue that there is real movement of cinematic images would be

distance between the two due to the different refresh rate of the images in each case) (Hochberg, “Perception of Motion Pictures” 208-209). He stresses that stroboscopic motion has been repeatedly but erroneously attributed to the false “persistence of vision” thesis, which we will discuss further down in this chapter. Following the term that Hochberg himself uses, the term “stroboscopic” will be used hereafter in the present study to refer to this effect of apparent motion on screen, instead of “intermittent” which might be considered semantically more accurate; the latter term will be reserved for referring to the mechanism of stopping and moving the celluloid frames both in the recording and the projection cinematic apparatus. 98 It is true that the vast majority of movies today are projected in theaters using traditional celluloid film reel projectors; even digital movies that are partially or entirely created in computers with the use of software are printed on actual film during post-production, and are still distributed in this manner. The present study takes this issue into consideration, despite the fact that it is considered merely a temporary convenience of studios and distributors due to financial reasons. At the same time, the study focuses also on the fact that not only digital technology allows uninterrupted projection, but also that this new mode of distribution and projection is gradually expanding, as more and more theaters become equipped with digital means for distribution and projection. Put simply, we are now in a transitory period which forces us to consider both the analog and the digital mode. 145

to adopt very liberal criteria of reality” (“Film, Reality, and Illusion” 336). Thus, under the condition that ontological reconciliations or compromises might need to be made, let us attempt an answer to this topic by addressing some of the most prominent theses on cinematic motion, and see if they still count for anything in the digital era.

4.3. What Cinematic Motion Is Not: an Ongoing History

In what is probably the first serious attempt to theorize apparent motion,

Wertheimer studied and explained three stages of apparent motion: (i) beta movement, (ii) partial movement, and (iii) phi movement or, as it is more commonly known, phi phenomenon (Anderson and Anderson 5). Each one functions under specific laboratory conditions, which produce different responses. In the first case, showing to viewers an object first in position A and then in position B gives the impression that the object has moved from A to B. The second case may have various effects; e.g. the object may be seen as moving at a short distance, or two objects seen moving instead of one. An in the third and most important case, phi movement, motion itself is considered to be purely perceived, not an object moving: it is the case of “movement which connects the objects and has direction between them but seems not in itself to be an object” (Steinman, Pizlo, and Pizlo 2259).99

Wertheimer’s experimentation on motion involved the succession of visual stimuli at various intervals, and found that the above phenomena occurred when those intervals were just right in terms of coordination with the perceivers’ visual systems: as the speed of succession became slower and, therefore, the intervals increased,

99 The most common confusion is between beta movement and phi movement. For animated visual examples for each, see DVD-ROM, gif files 21 and 22. 146

the experience gradually shifted from perceiving the two objects simultaneously as distinct from one another to perceiving phi, “pure movement that is seen without a moving object” (2259). The importance of Phi movement, was an exciting finding that constituted one of the cornerstones of Gestalt theory: phi movement “arises as the result of temporal and spatial relationships between stimuli;” in this sense, the resulting perception of motion is an effect greater than each of the stimuli that were used in the experiment, a new form that “differs from (is over and above) the sum of the parts acting in isolation” thus having a Gestaltqualität (Gordon 13).

The phi phenomenon is a notion that was to become a centralized concept in the Gestalt theory of perception, and, consequently, to other theories that were influenced by the Gestaltists over the years. This is where its importance lies in relation, not only to cinema, but also to studies of motion in general: Wertheimer’s findings were a groundbreaking innovation, directly challenging dominant views of the time that followed the associationism advocated by Locke and Berkeley:100 where associationism relied on the combination of elements to explain the perception of motion as a mental event, Wertheimer’s study showed that motion in particular is a sensation on its own, directly perceived rather than mentally composed (Hochberg and Brooks “Perception of Motion Pictures” 210–211). On one hand, beta movement was an important finding as it was shown that, in proper intervals and under certain conditions, it cannot be distinguished from real

100 The philosophical trend of associationism was the main expression of empiricists (see footnote 42). It endeavored in defining the elements that constitute mental images. Associationists believed that all our thoughts originate in sensations, “elementary sensory experiences” which are combined with each other to produce thoughts. Therefore, new sensations are added on top of older ones to produce new thoughts. Associationists considered sensations as being finite in number, but their combinations being able to produce the infinite number of thoughts and images we perceive and have in our minds (Hochberg “Psychophysics” 33). 147

movement (210). And on the other, on top of this observation, the experience of pure phi showed that the perception of motion was a direct sensation on its own

“with its own physiological basis,” rather than the product of mental associations

(211). Wertheimer’s discovery has a dual impact for cinema studies, one that is positive and one that initially appears a bit troubling: on one hand, it is truly credited with an originality that marked the deliverance of visual perception form fossilized theories of thought and cognition; but on the other, it seems that it contrasts

Hochberg’s model of schematic maps, which consequently means that it also opposes the model of digital cinema built here. In order to demonstrate the opposite, i.e. that these phenomena complement Hochberg’s model rather than contradicting it, we need to delve a little bit deeper into why cinematic motion was not the same after Wertheimer.

Perhaps the most prominent theory of perception of cinematic motion that sprang from associationist theories was the persistence of vision thesis. The persistence of vision remained for decades the most widely spread explanation for explaining why and how movie spectators see objects moving on screen, and still appears to be alive in cinema theory literature, despite the more recent findings that completely disprove it. Anderson and Anderson, who have ardently written against any validity of the persistence of vision, define it as follows: “when the human eye is presented with a rapid succession of slightly different images, there is a brief period during which each image, after its disappearance, persists upon the retina, allowing that image to blend smoothly with the next image” (2). This assumption has been proven to be impossible to stand, as such a fusion or blending of images one on top of the other would not produce apparent motion, but instead a superimposition or 148

“piling up” of images, one on top of the other (Anderson and Anderson 3; Hochberg and Brooks, “The Perception of Motion Pictures” 208–209). In its attempt to answer the question why we do see continuous motion when in fact we know that what we see is a succession of still frames, the fallacy of the persistence of vision theory lies, according to Anderson and Anderson, to the fact that its proponents confuse apparent motion with flicker fusion, failing to see that they are two different things: the appearance of motion is different from “fusing of the flickering light,” which is now common knowledge in perceptual psychology (2).101

The potentials that cognitive psychology has in explaining several puzzling aspects of the moving image in cinema seem to have been ignored to a smaller or greater extent for a long period of time. In pursuing a viable theory for digital cinema, one that accommodates existing theories of apparent motion but makes no compromises as regards the acknowledgement that several technical aspects of the medium have irrevocably changed, it is fortunate that most of the dirty work of dismissing scientifically erroneous theses, such as the persistence of vision, has already been completed. True, we know what cinematic motion is not, but we still have not answered the main question of what cinematic motion is, and as stressed earlier, the key problem remains: the known and prominent theories of cinema that deal with cinematic motion are established on a technology which is dying, i.e. the succession of film frames.102 In both cases, the result remains the same; there is motion on the screen. We cannot ignore the fact that digital cinema, at least today,

101 Anderson and Anderson also attribute the longevity and success of the persistence of vision theory among other reasons to “a lack of careful scholarship among film writers:” being content and absorbed by Marxist and psychoanalytic models of mind, scholars remained “victims of blissful ignorance,” not seeking for alternative explanations in psychology and physiology (3–7). 102 This change of technology from rapid succession of still images to a continuous projection of motion applies to cinema as well as to new technologies of TV panels. 149

is still in a transitory period: on one hand, digital movies are still printed on celluloid in post-production and distributed in theaters with analog projecting systems; in such cases, we may indeed apply any plausible theory that takes into account stroboscopic motion, as we are still talking about intermittent light and change of still frames. But on the other, digital distribution and projection escapes that confinement, as projection of light is continuous and celluloid has been replaced by changing pixel values. This is a tricky predicament: how do we define if this motion can be considered real or not, and on what theoretical premises? Does the technology used in each case affect the answer to this question, or should we simply consider the final impression of motion that viewers perceive, in order to attempt an ontological approach to the reality status of cinematic motion? Unfortunately, even within the vast realm of cognitive studies, there is no consensus to this issue. In fact, exactly the opposite is true: prominent figures of cognitive film theory have completely opposite positions as regards the reality or not of motion in cinema.

On a careful consideration, there aren’t many alternatives to choose from: motion in cinema may either be an illusion, or it may be real; or, it may be something in between which of course needs more elaboration from whoever brave theorist attempts it. A good example of the former two opposing theses is the contradiction between Andrew Kania and Gregory Currie, whereas an example of the latter, more in-between positions can be found in Joseph and Barbara Anderson. Currie believes that cinematic motion is actually no different in any way from real motion, whereas

Kania directly disagrees with him, providing a set of counter arguments to support that Currie’s positions are far-fetched and hasty, and that cinematic motion is an 150

illusion. In the context of illusionist theories,103 Currie recognizes two kinds of illusionism in cinema studies: cognitive or “strong” illusionism, and perceptual or

“weak” illusionism. The former refers to a “state of mind essentially involving a false belief,” i.e. a condition in which the mind consciously mistakes an illusion as true; the latter, on the other hand, refers to cases in which the viewer is aware of the illusion, despite the fact that he continues to perceive it as such. Currie believes that cinematic motion is classified by illusionists under this latter category (“Film, Reality, and Illusion” 333).104 Eventually, Currie rejects both kinds of illusion; by doing so, he states that it is real motion that movie viewers perceive. On the opposite side, Kania provides what he believes to be a dismantling of Currie’s thesis. He addresses

Currie’s arguments one by one, and rejects them either as weak or as implausible.

Yet, he does agree that they do have a rational basis in the scenario of a continuous projection of light; interestingly enough, this scenario that Kania seems to consider as being hypothetical at the time of his writing, is now already a reality, something which we need to take into account. Finally, as an example of a middle position between Currie and Kania, Joseph and Barbara Anderson also believe that cinematic motion is an illusion, but in order to support their claim they address areas of psychology that ultimately bring their argument to the conclusion that motion is only real in perceptual terms, but not in technical. After we attempt an account of cinematic motion in digital cinema based on Hochberg’s model, which is a

103 Currie establishes his line of argumentation on an attack that he deploys against what he calls illusionism theses, mainly represented by Marxist scholars, which advocate that movies “engenders in the viewer an illusion of the reality and presentness [sic] of fictional characters and events portrayed” (“Film, Reality, and Illusion” 326). 104 A good example of this is the Müller-Lyer illusion: two lines are placed in parallel, each having arrowheads in their tips. The arrowheads of the first line point inward, while those of the other point outward. The lines are of exactly equal length, but the arrowheads pointing at different directions cause one to look longer than the other. The viewer cannot help seeing the lines as different in length, despite the fact that he is perfectly aware of the illusion. See Appendix II, picture 20. 151

reconciliatory one and manages to fit together opposing arguments, at the end of the chapter we shall re-consider these three positions.

4.4. Addressing the Variations of Cinematic Motion

The fact that cinema, unlike most of the other arts, is an essentially fragmented medium, as we have repeatedly stressed, hampers the effort for a definition of cinematic motion. In a medium as fragmented as cinema, the study of motion is multifaceted: the breaking of the movie into different shots with the use of cuts immediately dictates different kinds of motion that need to be taken into consideration. This existence of different kinds of motion applies to both recorded footage as well as CGI animation; it is the explanation given in each case that differs.

We can distinguish between (a) the way a figure on screen exhibits motion in relation to itself, i.e. how a form, character, etc. is animated; and (b) the movement of these figures inside the cinematic space, i.e. from one location to the other.

Within this latter category, we have a further breaking down of sub-categories: (b1) motion in relation to cinematic space, i.e. from one part of the cinema screen to the other. This cinematic space may be visible all at once in a specific steady shot, e.g. in an extreme long, long, or medium shot, or inside an imagined space, i.e. one which appears and disappears through the frame, as if through the window of a moving train. And then there is (b2) which is a technical side effect, and involves the movement of a figure inside non-unified space caused by editing, i.e. cutting from one shot to the other. This latter motion may be deliberate or not, depending on the movie; yet, it still counts as motion as far as the human visual system is concerned. 152

Providing movement to CGI figures is really a descendant of older animating techniques. In relation to the above classification, we can note that all synthetic models, no matter whether they are analog or digital, require two main kinds of movement that the animator needs to consider: gestural and proxemic movement:

Two kinds of movement are relevant here: gestural movement, which, as

I have already explained, is movement pertaining to the "body" of the

figure; and proxemic movement, which is movement of the figure as a

whole from one virtual location to another. (Tillis 186)

It is obvious that we can use the term gestural movement for category (a) as described earlier, and that both categories (b1) and (b2) are easily accommodated together by proxemic movement. Therefore for the purposes of this study we can freely appropriate the two terms presented by Tillis.

The author describes the way each of these two kinds of motion is created with the use of software, noting that in most cases they are combined together.

Gestural movements are controlled by a process called kinematics. Originally, forward kinematics was applied, which required the animator to feed commands for each part of the desired motion in the figure, starting with the larger parts of the limbs and then moving on to the smaller ones. But the gradual sophistication of software enabled the development and application of inverse kinematics, which only requires the animator to use the mouse pointer to “grab” e.g. the limb of the figure he wants to move, and drag it to the desired point; the software is not only already fed with the necessary subroutines and commands to apply motion properly in the entire part of the limb, but movements can be stored in libraries for later use; this, for Tillis, signals the unmistakable way in which the digital animator becomes 153

analogous to the puppeteer of the past (186–187).105 As for proxemic movement, the animator is required to consider parameters such as the route that the figure will follow inside the virtual space, its speed, which will define the number of frames necessary for the movement to be completed, whether there will be objects occluded by the figure passing in front of them, its change of size as it moves farther or closer, which requires a re-rendering of the figure in each frame, etc.; and on top of those, there are also the “‘physical’ qualities” of the figures, e.g. their virtual weight, gravity, texture and material, etc. (187). And of course, in moviemaking there is also the parameter of editing which may completely modify the viewer’s perception of cinematic space and the characters’ position in it. It is obvious, therefore, that moving a synthetic figure both in relation to itself and to the rest of the virtual environment is a really complex procedure that requires a lot of effort and focus on detail in order to be convincing. Details can indeed be lost or deliberately obscured with the use of rapid editing, but in any digital movie there will eventually be a time when slower scenes will need the attention of the animator. We will borrow the terms used by Tillis in our attempt to address each kind of cinematic movement using a model of motion perception based on Hochberg and Brooks’ insights.

4.5. Gestural Movement

4.5.1. Biological Motion and Motion Capture Technology

The main problem with gestural movement is the changes in the technology of cinema which render older theories of perception obsolete. Contrary to analog

105 For an instructional video on the concepts of forward and inverse kinematics, see DVD-ROM, video file 23. 154

films which utilized solely the technology of celluloid frames, movies today may either still rely on this technology, or be shot, stored and projected using exclusively digital means, such as hard discs, broadband internet connections, and digital projectors. Yet, despite the change in technology, the experience of moving pictures by spectators still remains the same. Moreover, we should not forget that fantasy blockbusters, as one of the main areas of cinema where digital technology is applied, include figures and images that are not found in real life, and hence they are not expected to be recognizable immediately, and yet, most of the times, they are. This characteristic, therefore, poses an additional problem: a model is required that does not only solve the riddle of cinematic motion, but also explains how these figures are recognized by viewers as living entities in the context of digital cinematic worldmaking. What is perception of motion in cinema narratives worth, if the only thing one sees is scrambled or lifeless images? And how would that possibly explain cinematic worldmaking? Advances in psychology over the recent years have provided interesting insights, not available in the past, that do indeed offer answers.

The solution to this dual problem of cinematic motion proposed in the present study utilizes the concept of biological motion.

In the literature of psychology, biological motion is defined as the study of the movement of living organisms “by displaying points of light representing the major joints of the body” (Hiris 1). A major pioneer in the field was Gunnar

Johansson. His experiments involved filming people covered in black suits and walking in completely dark rooms, with only a limited number of points of light attached to specific parts of their bodies, such as the limbs and head, so that viewers of the footage could only see the moving lights as the filmed people were walking in 155

the dark. Johansson studied extensively the way these points of light are enough to enable an accurate perception of body movement,106 and significant contributions to the study are currently made by Nikolaus Troje and his associates. The study of biological motion has revealed that the human visual system is extremely sensitive in perceiving other human forms moving (Pinto and Shiffrar 293). Even more interesting is the fact that, apart from human movement in general, biological motion may also provide additional detailed information such as the moving figure’s

“sex, emotion, identity and intentions” (Hiris 13). The sensitivity of humans to biological motion perception is considered a key factor in the evolution of our species, as it enabled the quick detection of other group members, but most importantly, of other potential predators and preys (Troje 2; Lu 1); as such, it is not confined to human forms, but also works for the perception of other animals (Ikeda,

Blake, and Watanabe 1936; Troje and Chang 1). With this mechanism having been passed to us from our ancestors in the process of our evolution, biological motion detection now still serves our survival, and also enables our social interaction as it allows us both to recognize people as well as their intentions (Ikeda, Blake, and

Watanabe 1935; Thompson and Parasuraman 4; Troje 9).107

It is clear that the existence of such a perceptual mechanism for humans and other animals cannot be exaggerated; in fact, studies have suggested that the perception of biological motion is potentially an innate mechanism, evolutionarily

106 See DVD-ROM, video file 24; an original filming of Johansson’s experiments on biological motion with his own commentary. It is also very interesting to note here that Johansson’s research, findings and conclusions reveal his deep influence by Gestalt psychology, as his mentor was David Katz (Hunt and Halper 1). 107 Despite the fact that it is not explicitly stated, this observation has a clearly ecological orientation. It is strongly connected to Gibson’s affordances, described in the previous chapter as “opportunities for action” prompted by specific stimuli from the environment. 156

hard-wired in our perceptual system (Troje and Chang 16). It appears, therefore, that such an innate system, one that enables the perception of motion of living things, could potentially prove extremely useful in detecting and identifying life in general, both natural and artificial. Perception of biological motion, like all other mechanisms, is triggered when visual stimuli meet certain criteria; logic dictates that, if artificial images are also made to meet the same criteria, viewers will perceive artificial life moving, just like they do with real living people and animals moving.

In fact, CGI and animation already utilize a motion recording technology that works on the same principles as the one used by Johansson. Digital cinema industry refers to this technology as motion capture or “mocap;” it has already been used with tremendous success in movies such as Robert Zemeckis’ The Polar Express

(2004) and Beowulf (2007), Peter Jacksons The Lord of the Rings trilogy (2001, 2002 and 2003) and James Cameron’s Avatar (2009). Mocap is technologically far beyond

Johansson’s light bulbs strapped on people dressed in black, and yet the principle is essentially the same. It involves the use of numerous sensors, instead of lights, strapped on the actors’ bodies; the actors perform inside specially constructed rooms, equipped with a large number of cameras around them, which pick up and record the signal of each sensor, thus mapping all signals together to create a synthetic moving image of the actor inside a CGI environment, with the use of proper software. The initial visual result of this crude accumulation data is a vector space within which one can see the structural forms of the actors moving and performing, as they are composed by the multiple signals of the sensors. This makes the work of CGI animators significantly easier: instead of guessing and designing themselves the proper position, movement, size, volume, etc. of digital figures, they 157

now have an extremely precise vector model as a guide to where exactly each of the figures’ body parts would be and how it would move if it were in real space and time.

In fact, one of the latest developments of this technology is performance capture, excessively used in Avatar in combination with mocap; this technique uses points painted on specific parts of the actors’ faces, along with a wearable apparatus that holds a camera right in front of their faces. This enables the mapping of even minute facial motions and expressions so that the actors’ performance is actually recorded.

This “digitized” performance is subsequently transferred to the CGI skins that the characters of the movie have, and enable an incredibly life-like performance by those digital characters.108

This makes the movement of CGI figures inside cinematic space unprecedentedly natural and life-like. Johansson’s experiments on biological motion demonstrated that the moving structure of dots is more than enough for the perceiver to clearly discern the figure of a living body moving, despite the complete lack of color, texture, shadows, etc. (Pinto and Shiffrar 294); mocap takes advantage of the fact that this moving sub-structure will produce the proper kinematic result, which will not be influenced by the exterior form that the figure will take.109 All that animators need to do afterwards is to decide the exterior form of these figures, e.g. their color, skin, texture, clothing, shade, etc, in a sense pretty much like dressing the frame of a scarecrow; as the natural structure of these forms will remain intact

108 For illustrations of performance capture technology in Avatar, see Appendix II, pictures 21 and 22; for the way this effect was actually used on set during the production of the film, as well as other illuminating details on its special effects, see DVD-ROM, video file 25. 109 It should be noted though, that recent experiments on biological motion have shown that the points themselves need to remain simple in form; using moving points that are complex in form might interfere with the perception of biological motion as the human visual system will have to allocate resources to discern each of these figures as well (Hunt and Halper 2–3). This is not a problem in mocap, as the surface CGI form that the figure will take is usually uniform, coherent and immediately recognizable; this allows the underlying structure to prevail over the surface form. 158

underneath, any exterior form that they may be given, depending on the type of movie that is being made, will also inherit this naturalness of motion. Consequently, it will also inherit the naturalness of the gestures of a real person or animal.

The power of the moving human form is compelling, and catalytic in perceiving the moving CGI figures on screen in the same terms as living entities.

Experimental research on biological motion reveals a number of reasons for which this happens. First of all, it needs to be pointed out that the term “biological” is used to distinguish the perception of motion of this sort from mechanical or robotic motion, which is not perceived as belonging to a living entity. This distinction on one hand explains the fact that the stimulus created by biological motion creates discernible “patterns of motion characteristic of biological systems;” on the other hand, and most importantly, it constitutes evidence that the human brain is equipped with specific mechanisms for picking up and making use of this kind of motion, separately from artificial motion produced e.g. by robotic or other means

(Binder, Hirokawa, and Windhorst 408). As far as digital animation in cinema is concerned, the utilization of mocap in mapping motion ensures exactly this pattern of something alive moving in space; therefore, viewers should be expected to experience biological motion from CGI figures, as the underlying structure of those figures is entirely based on actual biological motion.

4.5.2. Associating Biological Motion with Hochberg’s Model of Perception

There are a number of parameters for biological motion to be properly perceived; they are the same parameters that also explain how biological motion works in digital cinema. The general observation on these parameters is the fact that 159

biological motion perception is a multi-leveled process, in which both individual limb movement and entire body structure play separate roles. Delineating a multi-leveled model of this sort, Troje identifies four stages in biological motion recognition: (i) Life detection, (ii) Structure-from-motion, (iii) Action recognition, and (iv) Style recognition (8–10). The first level, life detection, refers to the extraction of information from the ballistic movements of separate limbs, which manifest

“signatures imprinted on the movements of an animate agent by the physical laws related to inertia and gravity” (3). It is hypothesized that the ways in which physical laws related with inertia and gravity affect the swinging movement of limbs is encoded in the motion of animate beings, which human perception is programmed to pick up (Troje and Chang 7). In simple words, the human visual system is already aware, so to speak, of the way gravity constrains and thus betrays the motion of people and animals. Once a living being is detected, our perception moves to the second level, structure-from-motion, in which all this limb motion information is organized by our perception into an elementary yet “coherent, articulated body structure,” e.g. a man or a specific animal (Troje 9). In the last two levels, more specific information is added: in action recognition, the specific action or intention of the moving being is detected, and finally, in style recognition an identity is created for the perceived moving being, detecting and utilizing more complex information regarding sex, emotional condition, personality, etc. (9). It is obvious that the entire model that Troje describes is built on the motion of the limbs which signals the primary mechanism for detecting life itself, as “[t]he rhythmic, either ballistic or pendular motion typical for the limbs of animals is very unlikely to be produced by inanimate objects” (Troje and Chang 7). This innate mechanism for recognizing 160

biological motion as different from motion produced by inanimate beings is what makes all the difference in digital cinema between simple CGI animation and worldmaking practice.

A more carefully consideration of the second level specifically, structure- from-motion, reveals an even tighter relation between CGI and mocap, as the recognition process mechanism is fully deployed in the detection of the structure and form of a human body in its totality. Experimental research, such as that conducted by Hongjing Lu, has shown that body structure and motion information, i.e. the kind that derives only from the physics of movement, are two separate parameters in the perception of biological motion, and that the former is indeed a prerequisite for the latter; in fact, there is now “direct psychophysical evidence” that visual information that comes solely from motion information does not suffice for the perception of biological motion:

[M]otion information is insufficient, and structural information is

necessary, for the identification of biological motion . . . [T]he present

findings support the hypothesis that the representation of biological

movement cannot be solely based upon the motion trajectory of joints.

Evidence consistent with this conclusion has also been provided . . .

suggesting that computational models of action perception must utilize

structural information. (Lu 11–12)

In the context of cinema viewing, it is therefore obvious that simply creating a moving synthetic image with the use of software may provide visual support for the narrative, but taking it to the next level, i.e. worldmaking, requires more intimate ties with reality. The structural form of the human body is a powerful agent that can 161

indeed provide those ties, as it triggers the perception of biological motion.

Therefore, in mocap the integrity of the underlying structural form of an actual, performing actor will ensure the proper perception of biological movement for whatever CGI surface appearance is given. What is even more interesting to note here is the fact that this visual event is not a trick or illusion of real motion; viewers actually see real motion of real people, simply because mocap is entirely mapped on the real movement of actors.

Similarly to this contrast between motion information and body structure, there is also the contrast between motion information, which is a stimulus factor, and non-stimulus factors, i.e. those that are not merely raw motion data but reveal intentions, action, identity, etc. As we have seen, in Troje’s model such factors come into play in the last two stages, i.e. action recognition and style recognition. It is obvious that in these stages, since we are talking about the recognition of specific behavioral patterns beyond the raw stimuli movement, more cognitive mechanisms take over: action and style recognition is, in effect, recourse to memory banks in order to find similar patterns that will guide the viewer’s response. Detection of biological motion is a mechanism that has been evolutionarily developed to ensure that the perceiver will recognize intentions of other living beings in his environment for reasons of survival or proper social conduct; storing experiences in memory banks and eliciting them when motion around him exhibits familiar patterns is essential for the entire mechanism to work.

Similarly to the above, and as regards the role of memory, Hochberg and

Brooks have also agreed that for the study on biological motion using light points, a model is needed that explains the extraction of information related with action, 162

gender, identity, or behavior. A simple geometrical model, e.g. one that only takes into consideration raw motion stimuli and uses only a vector information model, may, at best, simply provide information about the virtual lines that connect the light points (“Perception of Motion Pictures” 232). In fact, the sustainability of the biological motion detection mechanism is based on the way raw motion information is combined with action and style to produce kinetic-behavioral patterns that are stored in memory for later use:

[I]t is through memory, or other after-the-fact processes, that all our

responses to moving stimuli must eventually exercise their cognitive and

behavioral consequences. Geometrical motions themselves, unless they

form some simple and predictable pattern, are quickly lost, and no

explanation of how we perceive motion pictures (or the world more

generally) can claim success if it stops with the detectors, or the vector

analyses discussed so far. (232)

In more simple words, motion information is easier to recall from memory and made use of, if it is based on elements that are indeed more memorable, like, e.g. segmentation into discernible parts and obvious course towards some purposeful end.110 Biological motion should be expected to function under the same principle: recording the motion pattern will be based on having some usefulness to the perceiver.

110 The authors refer to the effects of the famous Heider – Simmel film on viewers. In 1940, Fritz Heider and Mary-Ann Simmel created a short film in which only geometrical shapes and lines appeared, but in clearly discernible behavioral movements. Despite the visual minimalism of the content of the film, viewers were able to recognize and follow a line of events with the shapes and lines as protagonists. The results were published in their essay “An Experimental Study of Apparent Behavior” (1944). We will review the effects of segmentation of action based on the Heider – Simmel film in Chapter 5 in relation to narrative. For the entire short animation by Heider and Simmel, see DVD-ROM, video file 26. 163

Let us use this information for a final important note, regarding the number of illuminated points – or sensors, in this case – used in the mapping of human motion. The Heider-Simmel film proves that only a limited number of points is enough to produce meaningful motion that can be stored in memory; but, in addition to that, original and subsequent laboratory experimentation on the minimum number of illuminated points that are required for biological motion to be perceived has shown that recognition of biological motion increases dramatically as the number of points also increases (Neri, Morrone, and Burr 894). This underlines the fact that the practice of using a large number of sensors in mocap not only does not contradict the experimental practice on biological motion, which traditionally only uses a limited number of illuminated points, but in fact verifies it: making animated figures on screen as life-like and natural as possible welcomes the use of more sensors than those used only in the joints, and the experimental data on the number of points provides evidence for this. The more points, the clearer the underlying human form is; therefore, not only its recognition by the viewer is facilitated significantly, but the produced biological motion is even more manifest.

Furthermore, Troje’s model also agrees with Hochberg’s piecemeal view of perception. Hochberg’s view stresses the utilization of information that comes primarily from foveal vision rather than peripheral vision. In section 3.4 of Chapter 3 of the present study it was described that the visual system first picks up only pieces of the visual array which it subsequently combines with expectations deriving from memory and experience, in order to create schematic maps for the perceived objects, i.e. cognitive interpretations of what is being seen. It was also noted that this sampling of visual data is economical, i.e. makes use of the least possible 164

information for making these cognitive assumptions, and takes place primarily in the center of the fovea rather than in peripheral vision. Comparing and correlating

Hochberg’s view with that of Troje reveals a number of analogies. First, Hochberg’s collection of raw motion stimuli corresponds to the first two of the four levels in

Troje’s model, life detection and structure-from-motion. And second, the ignition of memory that Hochberg describes, as well as the application of schematic maps, correspond to the last two levels in Troje’s model, action recognition and style recognition, which also require the intervention of cognitive mechanisms in order for the perceiver to deduce the identity of the moving being based on non-stimulus information. As regards the first two levels, Troje notes that life detection is very sensitive and works well both in the fovea and the periphery, but structure-from- motion only works well in the fovea (Troje 9; Ikeda, Blake, and Watanabe 1940). As we have seen previously, structure-from-motion is a crucial step in the recognition of biological motion, as it involves the establishment of a complete physical structure, a prerequisite for biological motion perception right before the application of more specific non-stimulus information of identity. Just like in Hochberg’s model of perception, Troje’s structure-from-motion is also the result of visual sampling, which is followed by the construction of schematic maps with the use of memory. The similarity is simply too conspicuous to ignore.

Hochberg’s piecemeal view was used to approach the fact that digital imaging creates a demanding visual experience and yet viewers respond to it positively. In that view, the use of landmarks explained both the perception of unreal environments, as well as fast editing. Now, the common ground that is revealed between Hochberg’s view and Troje’s model explains animated motion on the same 165

premises. The visual system is expected to respond in the same fashion in movie viewing as in real life. In the first level, peripheral view will be poor, providing only vague stimuli of motion; but it will attract the attention of the fovea, because of our innate instinct for survival. In the second level, the fovea will sample the CGI character or figure, recognizing the human structure provided by its mocap-powered pattern of movement. And finally, it will pick up non-stimulus elements such as texture, gender, intentions, etc., provided by memory, to raise a schematic map for the CGI form, which may be seen for the first time, but will always be established on prior experience. The CGI character will thus be a new image perceived, but it will be recognizable simply because it will be based on the same innate evolutionary mechanisms of perceiving familiarities in the environment, and fitted on the schematic maps of prior cognitive knowledge. It should be stressed though that this observation on the similarities between the two models focuses on the recognition of biological motion and the living structure that it entails; when it comes to the question whether this motion interacts with the space around objects, Hochberg disagrees that it can function as a cue for the perception of 3D space, as we will see in the next section.

4.6. Proxemic Movement

4.6.1. Placing Moving Objects in Space

Perceiving proxemic movement inside virtual space needs a different approach than gestural movement. Findings on biological motion indeed can explain the realistic perception of CGI people and beings by taking advantage of the structure of the real actor or animal underneath, and the easiness by which viewers 166

identify it; but the displacement of that figure inside space is an entirely different story. The reason is simple: gestural movement is confined to the way a solid structure moves in relation to itself, i.e. the motion of its parts in relation to one another. But proxemic movement involves the motion of the entire structure in relation to its surrounding environment. And by environment we mean not only the virtual cinematic space inside which action is taking place, but also the rectangular frame of the cinematic image that separates the cinematic world from the world of the viewer. The frame creates an additional point of reference for the perception of motion inside the cinematic world, which is also picked up by the viewer’s visual system and thus needs to be taken into consideration. In this view, we need to refer to two distinct uses of the frame when it comes to present cinematic events: the shot, which involves one continuous and uninterrupted line of action, and the sequence or scene, which involves several shots edited together with cuts in between or other transition devices. In both cases, the motion involved is not merely in relation to itself, as is gestural movement, but also, or even primarily, in relation to the surrounding space. The perceptual mechanism in each case is not the same; most importantly, applying biological motion will not help us with proxemic movement as it did with gestural movement.

To demonstrate the correlation between object and its surrounding space, consider for instance the example of structured motion produced from moving points, like the sensors or lights used in biological motion. The power of the moving human form, assisted by an increased number of sensors or lights, was enough for viewers to extract the shape as well as identity traits of the walking form.

Nevertheless, we need to take into consideration two facts: first, in digital fantasy 167

movies, objects that move are not necessarily known human or animal forms; and second, all moving objects have to be placed inside a coherent spatial environment.

The task is not always as simple as it was with gestural movement, which was explained by biological movement. For example, both Johansson and Gibson support a rigidity principle for the perception of objects, i.e. that “we normally perceive that rigid 3-D layout or object (if one exists) that will fit the moving 2-D pattern (we perceive the invariant under transformation)” (Hochberg and Brooks “Perception of

Motion Pictures” 236). But it has been experimentally proven subsequently that viewers cannot extract structure-from-motion, if there are no proper pictorial depth cues, e.g. because depth cues are either altogether absent, or existing but conflicting with the information that the observed motion produces (236). Pictorial depth cues in the perception of structure-from-motion, hence to motion in cinema as a 2D surface, which Hochberg and Brooks stress but both Johansson and Gibson seem to have underplayed, are of special importance in the study of digital movies: if anything, they triumphantly reveal the crucial link between digital worldmaking and the tradition of Renaissance perspective as described in Chapter 1.

The perception of motion in space has been a major theme in perceptual psychology, especially since the introduction of cinema. Several theories have addressed the issue, proposing explanations that vary from exclusively cognitive operations that mediate between the stimulus and the perception of motion, all the way to the other extreme which advocates that we experience a direct and unmediated perception of moving objects. Hochberg and Brooks build their own theory by navigating between those theories, addressing the shortcomings in each case. First, they review early Helmholtzian accounts of perception of motion, which 168

are based on his rule of perception, and, as such, see motion as a “mental representation of a set of circumstances in the distal world” rather than a direct sensory event which can be described simply in terms of the retinal image

(“Perception of Motion Pictures” 211).111 Subsequently, Hochberg and Brooks also review direct theories of motion, which began with Wertheimer; in fact, among other things, Wertheimer’s work aimed at dismantling the Helmholtzian mental representation model by claiming that motion is directly perceived. Using his study of the phi phenomenon, as well as the findings that stroboscopic motion and continuous motion – like the one we experience in real life – are indistinguishable from each other under the proper circumstances, Wertheimer concluded that

“motion experience was a sensation as direct as red, green, or any other, with its own underlying physiological basis” (Hochberg and Brooks “Perception of Motion

Pictures” 211). Variations of the same tradition of direct perception continue in the studies of Gibson and Johansson; for example, Gibson argued that “under natural

(not manmade) conditions such motion-produced visual information is the inherently direct and error-free source of our perceptions of our environments and their content” [emphasis added] (Hochberg and Brooks “Perception of Motion

Pictures” 236).112

111 More specifically, the Helmholtzian model considers motion as one of the distal properties of the world one perceives. These properties, which also include form, distance, direction of gaze, etc., are measurable in the real world, but are not picked up by the photoreceptors of the human eye directly; the latter picks up sensations, i.e. the patterns of light that stimulate the retina, and the distal properties are added later, as the human perceptual system creates the image that most simply fits the stimulus pattern. Therefore, motion, like other distal properties, is not consciously perceived in Helmholtz’s model, but instead is a mental property which is added later in the process of perception. See Hochberg and Brooks, “The Perception of Motion Pictures” in Friedman and Carterette p. 211. 112 Gibson’s proposal was based on the famous “invariant under transformation.” More specifically, his model supports that “one responds to views of the world, which undergo continuous transformations as one moves and as viewpoints change over time, by extracting the structure within that changing array of light that remains invariant under those transformations, and which therefore 169

Hochberg and Brooks’ approach moves in between mental representation and direct perception, proposing a model that accommodates the two extremes and can be more flexibly applied to motion pictures. More specifically, the authors see minor or major shortcomings in both of them, if they are to be used on their own:

We can admit that perception involves some process of construal that fits

but is not determined by stimulus information, without turning to a naive

view of how problem solving works (logic does not generally play a large

role, as is implied by "inference"), and we can admit the existence of

direct sensory response to some aspect of the world, like motion, without

ruling out more cognitive paths to the same perceptual consequences.

(“Perception of Motion Pictures” 212–213)

The fact that watching and following digital movies is such a demanding visual task dictates the importance of navigating inside these different approaches simultaneously. When it comes to the perception of moving objects in relation to their space, virtual or real, there is a huge difference between attributing it to purely cognitive or to exclusively direct explanations. The answer to the issue will be indicative of the nature of our perceptual knowledge about our environment in general. The following section will review the way Hochberg and Brooks reconcile the two extreme positions, as well as the way this reconciliation may be applied in digital cinematic motion.

As noted earlier, when discussing spatial motion in relation to a medium such as cinema, where only a rectangular fragment of the cinematic world is available at any given time, there are two kinds of movement to consider: motion from one specifies the structure of the invariant surfaces of the world” [emphasis added] (Hochberg and Brooks “Perception of Motion Pictures” 245). 170

position to another within this rectangular screen, i.e. within the same shot, and motion from one shot to the other, i.e. within the same edited scene. The implications of this separation are obvious: since a single shot covers a much smaller virtual cinematic space than what an edited scene normally covers, the former kind of motion is usually perceived as the object covering a smaller virtual distance than the latter.113 In digital fantasy cinema the construction of space is obviously of crucial importance for preserving the overall credibility of the virtual world; therefore, the perception of motion within that world also needs to produce a realistic effect, in order to sustain and even further facilitate this credibility. Given the importance of motion for the impression of reality and life, as described early in the present chapter, motion in unreal environments such as CGI worldmaking is a more challenging area to test this hypothesis. Therefore, it is important to see the relation between proxemic motion in cinema and in real life, i.e. whether the former is perceived as real or illusory, and the way it thus interacts with virtual cinematic space as far as the viewer’s perception is concerned. And of course, it is important to explain the mechanism behind each kind of motion.

4.6.2. Moving in Cinematic Space Inside the Same Shot

The first kind of proxemic movement, i.e. the one which occurs within a shot, is a relatively easy task to handle. The reason is that it is a lot closer to the motion

113 As stressed here, this distance that is covered from one shot to the other, i.e. between cuts, is virtual, i.e. corresponding to the presumed space of the cinematic world that the viewer reconstructs in his mind. In reality, i.e. in terms of the actual displacement of objects in the viewer’s visual array on the cinematic screen, this motion may be equal to, or even smaller than a motion that occurs within the same shot. This obviously depends on the kind of shots that are edited. What is important here is the fact that shots edited together create the sense of larger space, and ergo the motion across cuts is perceived as covering a longer distance. 171

seen in real life, and therefore it should be easier to establish a connection between the two. As introduced in the previous section, Hochberg and Brooks discuss and reconcile two extreme positions: direct perception of motion through photoreceptors, and indirect perception of motion, through some process of mental representation of motion. Initially there seems to be laboratory evidence that supports the former, claiming that there are “receptor mechanisms that respond directly to motion,” most importantly to stroboscopic motion of the kind cinema movie projection is comprised, and that these receptors are located in specific cells which are activated when the retina picks up movement (“Perception of Motion

Pictures” 215). This is a well-known fact in perceptual psychology:

Under appropriate conditions, beta movement is indistinguishable from

real movement . . . In general, the perceptual system treats a succession

of static views, taken with fine enough temporal grain, as being

equivalent to the continuous movement from which those views are

sampled. (Hochberg and Brooks, “The Perception of Motion Pictures”

210; “Movies in the Mind’s Eye” 376)

In simple words, when the real-life continuous motion of an object is recorded, thus producing a succession of frames each with a static view of the objects in it, the projection of these frames will be indistinguishable from real motion as far as the human visual system is concerned, as long as a certain velocity of this projection is maintained. Within a shot, the succession of frames is not interrupted by editing; therefore, apparent motion must be strongly connected with this effect.

Moreover, Hochberg notes an additional and interesting finding in experiments on stroboscopic motion that supports the direct position for 172

displacements within a shot: it seems that, when confronted with successive views of different objects, with short intervals between views and small distances between the objects themselves, the human visual system is unable to detect specific traits and characteristics of each object appearing in each view:

[V]iewers show no preference for an apparent motion between objects of

the same form or color over apparent motion between objects that differ

in these attributes . . . The motion occurs between nearest successive

contours, and does not at all preserve phenomenal identity. [emphasis

added] (“The Perception of Motion Pictures” 215–217)

The result is that motion is seen between non-corresponding objects, simply because they are the closest to each other, regardless of the fact that they may be different in form or color. This proximity phenomenon perfectly explains the fact that the human visual system has no trouble seeing motion between objects appearing in successive frames, with the condition that the intervals between those frames are just right.114 It clearly demonstrates that the human visual system will see motion between both objects regardless of them being identical or not, as long as they are the nearest to each other in a successive set of frames. In cinema projection, frames that belong to the same shot are a perfect example of this condition: an object or

114 Hochberg and Brooks note that a Gestalt approach would easily but erroneously reply to the apparent motion between frames. Since successive frames of the same object include more than one identifiable objects, the question would arise as to how we predict and anticipate where each point would move next; the Gestalt approach, following Helmholtz, that “one perceives whatever requires the fewest changes” initially seems plausible, and would explain how an object located in point A in one frame occupies point B in the next, or in those subsequent to that. The problem is that this position implies that viewers see motion between corresponding objects; in contrast to this, it is proven that the exact opposite happens, as “at least for short displacements, and over short intervals of time, motion occurs between the nearest luminance-difference contours of the same sign . . . regardless of color or features of form” (“Perception of Motion Pictures” 217). Cinema projection obviously meets these requirements of short displacements and intervals of time, therefore a Gestalt approach would not work in this context. The displacement of objects here is discussed as one occurring from one frame to the next; it does not cover cuts unless we have a cut in which the adjacent shots are almost identical to one another. 173

character is presented in each frame in a slightly displaced location; in an intermittent projection of frames, the succession views of this character or object will be received as apparent stroboscopic motion by the viewer, as each view of the object will be very close to the next and will qualify as the nearest neighboring object.115 This kind of apparent motion, because it is picked up directly by photoreceptors in the retina, does not require claims to any mental representation processes for motion to be perceived. Therefore, it covers one end to the reconciliatory model that Hochberg and Brooks describe.

This is indeed a solid phenomenon, but it appears to belong to a different era

– at least at first glance. The reason is that it belongs to the vast body of theory of cinematic motion, which has been written and tailored on analog cinema; as such, its applicability in digital cinema should not be taken for granted without some elaboration. A solid triangle has to be formed, with a connection between its corners: on one corner, there is this proximity phenomenon that enables motion within a shot to be experienced, despite the fact that it is produced by non- continuous fragments, i.e. the successive frames. On the second corner there is digital cinema, which may be printed on celluloid during post production for reasons of distribution, but, then again, may just as well be stored in a hard drive and shown with a digital projector, the image of which does not have any fragmentation, as there are no frames.116 And on the third corner, there is the experience of real

115 This phenomenon of apparent motion between non-corresponding objects works in the viewer’s favor when it comes to motion within a shot, and explains apparent motion just fine. But it is obvious that it will also arise across shots; the problem is that in this case the objects identified with each other from one frame to the next will not necessarily be the same, as it happens within shots. This identification of objects in edited scenes may be unwanted and problematic in the process of watching a movie and will be discussed further down. 116 Fragmentation still occurs, based on the various levels of fragmentation that were discussed in Chapter 3, but in this case they do not apply to the level of frames. 174

motion, which is what the viewer needs to have in order for the sustainability of the fantasy CGI world as an animated universe to work; in other words, both the other two corners need to agree with this third one in order for cinematic worldmaking to work as perceptually real. The connection between the three corners needs to incorporate the current digital reality of cinema, and the technological alternatives that it has introduced.

For the first couple, i.e. the relation between the proximity phenomenon and real motion, things seem to be fairly simple. Hochberg and Brooks cite a number of recent studies117 that reveal the fact that stroboscopic motion utilizes exactly the same visual mechanisms as motion that occurs in real life. In these studies it is experimentally proven that “the real and the apparent motion have the same underlying basis” (“The Perception of Motion Pictures” 217). This finding simply means that, as far as human visual perception is concerned, motion in cinema is just as real as is motion that is seen in real life, since, under proper conditions of speed of the intervals and distance of displacement, the human eyes simply cannot see the difference between apparent motion and real motion. This statement opens the way for establishing the second couple, motion in digital movies and motion in real life.

As regards the projection of digital cinema, as we have seen, there are two alternatives: digital movies are either printed in celluloid during post production and are thus shown in theaters using traditional projectors as any other movie; or, still less commonly but with an increasing rate, they are digitally stored, distributed and projected using hard drives and digital projectors. In the first case, digital movies will

117 See E. H. Adelson & J. Bergen, “Spatiotemporal Energy Models for the Perception of Motion” (1985); P. Burt & G. Sperling, “Time Distance and Feature Trade-offs in Visual Apparent Motion” (1981); and A.B. Watson, & A.J. Ahumada Jr, “Model of Human Visual-motion Sensing” (1985).

175

behave like any other analog film: they will be projected in stroboscopic mode, and will therefore only need to abide to the conditions of speed of intervals and proximity of views of objects inside frames, as described above. In the second case, digital projection is by definition a continuous process anyway, with no individual frames, and therefore also with no intervals and no split seconds of darkness between frames. The human visual system in this case does not at all need to tackle with perceiving apparent stroboscopic motion, simply because there is no stroboscopic projection of frames. It is obvious, therefore, that in either case, proxemic movement within shots in digital movies will be perceived exactly in the same way as real motion. This line of thought completes the triangle: digital movies project proxemic movement as real as any analog film does, as long as the same projecting requirements are met, without this condition being hindered by the recent technological developments in storing, distribution and projection.

4.6.3. Moving in Cinematic Space Across Edited Shots

The second kind of proxemic movement, the one that involves motion between edited shots, is trickier and vastly more problematic in moviemaking. It is logical to assume that in edited shots it is significantly easier to disrupt and undermine the conditions of proximity and interval speed that produce apparent motion within shots, just as it is also easier to apply them accidentally in cases where they are not welcome. For example, if a moviemaker wishes to produce uninterrupted motion that expands from one shot to the next, he will need to be a lot more careful with gluing the shots together; otherwise, editing shots in a way that produces a match cut will result in apparent motion between non- 176

corresponding objects across different shots, which is normally an undesired situation in moviemaking. It is reasonable to assume that, as far as motion across edited shots is concerned, the human perceptual system should be expected to process more complex visual streams, and consequently reliance on mere direct stimuli from photoreceptors is probably inadequate. For this reason, addressing motion across shots covers the second end to Hochberg and Brooks’ reconciliatory model: the previous section addressed motion within a shot and utilized, among other things, direct sensory response to stimuli, whereas this section addresses motion across shots and will give more weight to mental representation. Just like mainstream moviemaking normally comprising of a succession of both shots and scenes, perception of motion is revealed as a versatile process that responds to both of them by relying on the interchange between simple, direct perception and more elaborate mental operations.

In order to accommodate motion across edited shots in a scene, along with motion within a shot, Hochberg and Brooks start by drawing on two-system theories of motion perception. Expressed in several variations, this family of theories supports that humans use two separate systems for perceiving motion, “both a primitive motion-detection response, and a more detailed form-dependent process”

(“Perception of Motion Pictures” 218). We have seen in the previous section that the direct, motion-detection system serves us quite well in explaining the perception of proxemic movement within a shot, as it relies on the same mechanism with which humans detect motion in real life. But there are certain conditions of motion which cannot be explained simply by a direct response to stimuli, and motion across cuts is one of them. Hochberg and Brooks note that transition from one shot to the other 177

introduces a different array of visual information to the viewer: for example, it could introduce the same objects from the previous shot in different locations, or a new set of objects altogether, etc. (“Perception of Motion Pictures” 221). As regards the actual displacement of objects on the screen, motion within a shot is short-range, i.e. operating “over short distances and brief durations,” whereas motion across shots is long-range, i.e. covering “long distances and long durations” (Cavanagh and Mather

103). These different kinds reflect the function of separate mechanisms in human visual perception:

[W]e want to point out that apparent motion is normally short-range

within a shot, and therefore presumably reflects the simple action of the

motion-detector system. The recording of most motions within a shot is

therefore essentially automatic and need not concern the filmmaker; but

across a cut, apparent motion is likely to be mostly long-range in extent,

is usually unwanted and unforeseen by the filmmaker, and is very

different – and far from simple – in its characteristics. (Hochberg and

Brooks, “The Perception of Motion Pictures” 220–221)

Therefore, perceiving motion across cuts cannot be automatic, as it normally is within a shot, since it is most probably dependent on the latter system for perceiving motion, i.e. the form-dependent one: when using this system, viewers demonstrate a higher degree of attention to the identity of objects that are displaced. This factor of attention as a whole, as well as the specific areas of attention that it is broken down to, marks the essential difference between motion across shots from motion within a shot, as, in the former case, attention requires an more enhanced active cognitive process. 178

In the context of digital movies and specifically worldmaking in CGI fantasy, a discussion about apparent motion being dependent upon the recognition of objects is significantly more important than in analog cinema. As we have seen in section

3.5.2 of Chapter 3, the recognition of objects in unreal environments becomes possible due to the piecemeal nature of human perception. Briefly, the argument there stated that the recognition of inexistent figures takes place as the retina scans the screen for recognizable objects, fits them in pre-existing and known mental schemata, and contextualizes the CGI design and figures in that schema. This way, digital objects of fantasy are fused with real or realistic ones within a revised, in a sense, version of the original schema. Nevertheless, despite the fact that, as we have seen, visual momentum guides the retina towards informative areas of the image as quickly as possible and in the most economical way, it is logical to consider that the recognition of objects takes more time when those are seen for the first time than those that are known and normally exist in real life. The design of a CGI object by animators should, and generally does, take into consideration factors of its form, the capacity to draw the viewer’s attention, and the expectations that this form creates.

These expectations will initially establish its contextualization inside the synthetic world, as explained previously, and will subsequently affect the way the viewer will perceive its motion or displacement.

Why do we need to focus on these factors to determine proper motion? The reason is simple: since proximal displacements on screen are perceived by two different systems, which, in turn, produce short-range and long-range apparent motion depending on distance and intervals, there is no guarantee that the correct direction, or even the correct object moving, will always be the one that is actually 179

being perceived. Consider the following scenario, which is not at all rare in moviemaking: in a certain shot, an object occupies a position on the far left of the screen. Then, after a cut that introduces a new shot, a completely different object occupies a position also at the far left of the screen, perhaps with a very slight difference in position. If another cut introduces a third shot very quickly, before the long-range system has time to scrutinize and identify the second object, the short- range system will identify the object in all three shots as being one and the same.

The short-range system works directly, based on primitive detection of motion by photoreceptors on the retina, whereas the long-range system needs time to identify the objects; without enough time available, and with very small spatial distance between objects, the short range system obviously takes over:

Where such meaningful, long-range factors are in conflict with short-

range determinants, the latter seem in general to prevail. Within no

more than 80-100 ms after an abrupt change of view, 16 apparent

motions occur between nearby successive contours of the same sign (or

contrast direction). Detailed shapes themselves take longer to be

recognized: the very earliest response seems to be to “blobs,” that is,

regions defined only by their low spatial frequency components and

lacking in detail. (Hochberg and Brooks, “Perception of Motion Pictures”

222)

In digital movies, therefore, where the medium allows for worldmaking that is significantly more detailed and elaborate than in analog films, animators and directors need to take into consideration these factors that contribute to the recognition of motion on screen. We have seen in Chapter 3 that, on one hand, jump 180

cuts are considered undesired and sloppy editing in contemporary moviemaking, and on the other that fast cuts are one of the main features of digital blockbuster editing.118 It is obvious that, the faster the cut, the more careful moviemakers need to be, taking into consideration the two systems for perceiving apparent motion in combination with the increasing complexity of digital objects and worlds.

When editing a movie as fast as blockbusters are generally edited today, it is certain that the human visual system simply cannot handle the amount of visual information necessary for absorbing and recognizing all CGI objects in their full detail. It is very common for an object, especially a synthetic, non-existent one, to be presented in a way that causes ambiguity as to the direction of motion.119 This ambiguity needs to be resolved by incorporating the parameter of mental representation in the proposed model, i.e. a process that explains how and why the mind will choose one alternative over the other. In Hochberg and Brooks’ own words, such a model, if it claims to explain apparent motion in ambiguous images,

“must therefore include the object's shape, the viewer's attention to that shape, and some process that chooses between alternatives not simultaneously present in the stimulation (therefore, between ‘mental representations’)” (“Perception of Motion

Pictures” 222). An exclusively direct model, e.g. Gibson’s and its variations,120 is not enough; the theory of invariants under transformation simply cannot explain the

118 See footnote 71 and section 3.5.3 respectively. 119 Hochberg and Brooks provide a sketchy example to illustrate this point. They use Joseph Jastrow’s famous “duck-rabbit” illusion, in which a shape may appear to be either a duck facing leftward or a rabbit facing rightward, depending in where the viewer attends. It is crucial to stress here that only one view is possible at a time, even if the viewer is directed towards noticing both views. In any case, it is up to the viewer’s attention that the object becomes meaningful towards one or the other direction; and, most importantly, the way the object faces determines the tendency to perceive it as moving towards that direction (“Perception of Motion Pictures” 223). See Jastrow’s sketch in Appendix II, picture 23. 120 See for instance the work of Shaw and Pittenger on direct perception. 181

perception of motion from one shot to the other without the intervention of mental operations that are both slower and more directed towards the phenomenal identity of objects.121

One rough way to partially counterbalance this problem is to make CGI objects and figures partially resemble to a smaller or larger degree known items from real life. It is easy to see this resemblance: the Na’vi tribe in Avatar, the Thark tribe in John Carter, the mutants in the X-Men series, the “Shrimps” in District 9, to mention just a few, all retain the elementary structure of the human form despite the elaborate CGI suit that they are made to wear.122 Other examples could include the Pandora creatures in Avatar that clearly resemble real jungle animals, or the starfighters and cruisers in Star Wars which are made aerodynamic when all action takes place in space, where in reality the absence of air would eradicate the need for aerodynamic vessels. One reason for this practice of making an object remind, even in a sketchy way, of something that exist in real life, as we have seen, is to anchor and contextualize it in a coherent cinematic universe. But by doing so, it also enables this object to inherit the functional traits of the original object. As it happens with the duck-rabbit illusion, this characteristic creates a bias to the viewer to perceive the object in a certain way. The fact that “we tend to see the depicted persons or objects move in the directions they face” (Hochberg and Brooks “Perception of

Motion Pictures” 222) should work for any figure with what can be recognized as a face; also, the fact that “[o]bjects that are characteristically mobile (cars, airplanes,

121 Ed S. Tan stresses the need for “inference and knowledge use” for perceiving non-overlapping shots, a feature which is denied in direct models such as that of Gibson; and more importantly, he mentions that Gibson himself admitted that it was a difficult riddle for himself to fit his “invariants” in such kind of editing, and explicitly stated that “‘[t]here is no overlap of structure between such alternating shots, but there must be some common invariants. What are they?’” (qtd. in Tan 565). 122 See Appendix II, pictures 24-27. 182

people) are more prone to be perceived as moving” (226) should also count for things that are digitally built to resemble such an object. In simple words, a perfect explanation for CGI design being based on actual objects leads to a kind of vicious circle: on one hand, digital designers and animators use forms that are already known from real life, because they are simply utilizing their own internalized schemata in a creative manner, since, of course, memory and experience cannot be evaded; and on the other, the use of these known forms ensures that viewers will be able to respond positively to the appearance of the CGI objects.123 It should be noted though, that apart from the common reality of schemata that audiences share with designers, recognition and response to CGI objects requires an elevated level of literacy and a kind of visual training, as digital worlds become increasingly complex.

In summary, this is the way that Hochberg and Brook’s model can be used for the perception of motion specifically, without making any compromises or leaving any loose ends. This model unites opposing theories of perception, the one of direct and unmediated perception, and the one that involves mental representation, simply by clarifying the distinction between modes of motion perception that have anyway been proven to be separate, i.e. short-range and long-range apparent motion. This distinction enables us to return and briefly review the theories of cinematic motion addressed earlier in the chapter: the ones by Currie, Kania and

Anderson & Anderson.

123 As we have discussed in Section 3.4 of Chapter 3, Lakoff and Johnson’s model fits Hochberg’s model in a quite satisfactory way as regards the actual nature and conceptual origin of these internalized, species-wide schemata. As a reminding note, it was noted earlier in the present study that Hochberg’s approach mostly explains the physical parameter of the visual and cognitive mechanism that humans use, whereas Lakoff and Johnson’s cognitive semantics define exactly what kind of schemata these are and how they intervene in our perception of the world. 183

The first two, Currie and Kania, expressed the most direct opposition between a realistic and an illusionistic approach respectively, but after elaborating on Johannson’s, Troje’s, and Hochberg & Brooks’ models, it is obvious that this opposition is simply a misunderstanding: both Currie’s and Kania’s papers demonstrate ignorance of solid experimental data on motion perception from psychology. Therefore, they both resolve to one or the other end of an “either-or” polarization, which is caused by the theoretical scope that they apparently utilize, which does little more than move around a vague conception of “cognitive functions,” with little actual evidence from cognitive psychology to support it. But reality of visual perception is not that simple: the perception of both real and cinematic motion indeed takes into account cognitive functions that may or may not follow the physical stimulation of the eyes, but building an argument which ignores experimental evidence on the latter is not simply unwise, but also irrational. It is also important to note that both Currie’s and Kania’s positions were laid on paper at a time when Hochberg and Brooks’ approach had already been around for many years.

As for Joseph and Barbara Anderson’s approach, it seems that it is the closest one to that of Hochberg and Brooks, and they almost reach the right conclusion, but leave some important elements aside. They, too, acknowledge the fact that short-range motion and real motion operate under the same perceptual mechanism, with the human visual system being unable to distinguish between the two; therefore, they also reach the conclusion that, as far as our conscious knowledge is concerned, motion in cinema is an illusion, but as far as the human visual system is concerned, it is as real as motion in real life. Nevertheless, they focus on the difference between short-range and long-range apparent motion but they seem to ignore the operations 184

that the latter may have in the perception of a movie, in the way that Hochberg and

Brooks apply it to motion seen between edited shots. The reason for this shortcoming in Anderson and Anderson’s model is the fact they draw on Gibson’s direct approach, which makes their model focus on direct photoreceptor stimulation, and leave aside cognitive functions. But how can a direct model come through under today’s circumstances? With digital cinema spreading and still being in need of a solid theoretical framework, the content of movies becomes more and more complex, and their technology dramatically changes. It is obvious that

Hochberg and Brooks’ model is a far more likely candidate to support the weight of these changes, as it leaves ample room to both direct perception and cognitive operations for explaining cinema motion in its latest manifestations.

185

CHAPTER 5

THE POWER OF STORYTELLING

5.1. Motion with a Purpose: From ‘Moving Around’ to ‘Going Somewhere’

As described in the previous chapter, the power that motion has to denote animation, i.e. the sense of some sort of life present, is a breath-giving force in cinema. Humans are evolutionarily fine-tuned to respond to motion because they are immersed in a universe which is constantly in flux; as beings equipped with the capacity to move, they understand themselves as parts of a non-static world. We have seen that one of the main reasons why cinema is so appealing is exactly the fact that it replicates this kind of world. But the evolution from automata to modern digital graphics in cinema indicates that motion is perceptually effective when it is charged with some sort of meaning: the strive for perfecting motion, and turning it from a repetitive mechanical routine to a recognizably purposeful string of actions, i.e. an event, also betrays a tendency to comply with sets of perceptual subroutines that humans respond to in a better way, making this kind of motion easier both to comprehend and to remember. This is where the role of narrative comes into play: the power of visual storytelling in cinema lies in the fact that humans have evolved to respond to causal structures, as well as other textual relationships through which the viewer navigates in the process of watching a narrative movie using cognitive reasoning.

Causal strings of actions are part of the way humans situate themselves in their surrounding world, as parts, agents, or receivers of a network of purposeful deeds. In Lakoff and Johnson’s model of cognitive semantics, the source-path-goal 186

schema is one of the major ones that human cognition learns to obey as we evolve into comprehending the world via our awareness of our bodily structure and functions. The source-path-goal schema, as we have seen in section 3.4 of Chapter

3, evolves from our actual physical motion from one place to another in a particular direction; this motion may be hampered by various kinds of obstacles or difficulties, in a process which is projected, through metaphor and metonymy, into our actions towards specific goals, ambitions etc. in life (Buckland 44). After having discussed the importance of motion in cinema comprehension in the previous chapter, approaching narrative through such a cognitive framework makes perfect sense; the acknowledgement of the importance of purposeful motion, even simplistic such as that of the automata, enables us to explain the bigger picture, i.e. how narratives in motion pictures are actually perfect examples of the source-path-goal schema, as it transferred from simple movements to an actual plot. Of course, it should be acknowledged that neither are all movies comprised of moving scenes, nor do they have a tight cause-effect structure upon which events and actions are placed. But in the context of digital fantasy, classical narrative structure clearly prevails in movie- making practice today at a significant degree; and a study that uses normal perceptual operations to explain movie-watching should not be penalized for focusing primarily on this vast majority of moviemaking.

Narrative theory, in general, is a branch of humanities that addresses, among other things, the function of storytelling patterns, i.e. the parameters and devices that define entire categories of stories, no matter how the latter are made or told.

Therefore, the question here is whether we need a separate narrative theory for digital movies, or there are general rules of narrativity, pertinent to moviemaking in 187

general, that may apply just as well to digital visual texts too. Why do we need to make such a separation? Is not digital moviemaking part of the much discussed and already delineated film narrative tradition? Does their different technology really require a different approach as well? All these questions arise from the fact that all past theory of film narrative dealt with analog films that had either a stronger or a different kind of connection to pro-filmic reality than CGI does. To clarify: on one hand, the fact that movies in the past were analog and were shot in celluloid that captured a scene happening in front of the camera, in a way ensured that the ontological bond between pro-filmic and filmic events was retained; thus, presumably, with analog films it was easier or more natural for audiences to reconstruct in their minds a cinematic world that indeed resembled their own very much, as it was made from bits and pieces of it. And on the other, even in many cases of formalism or avant-garde narratives, the pro-filmic reality was indeed treated as a building block of something different, but these filmic blocks, again, were recognizable and the composition was obvious; in fact it took effort to deliberately make them look artificial. In digital fantasy movies none of these two conditions are met: with the use of CGI the pro-filmic reality has been either limited or altered to being non-recognizable as such, with the cinematic image thus loosening or altogether losing its pro-filmic referent; and, what is more, the selection and composition of bits and pieces of CGI and real-world footage aims for complete assimilation and homogeneity rather than obvious artificiality. At the level of narrative, then, digital movies may pose a problem; worldmaking may be successful as a concept or artwork, i.e. an image eventually realized in colorful and impressive- 188

to-see CGI, but following a plausible storyline inside such a world could be a different story altogether.

The answer here is to study the possibility of narrative universalities that the human mind responds to when watching a movie, regardless of whether this movie has fantasy or realistic content. The importance of the theory of schematic maps in digital worldmaking is obvious in the context of narrative. Consider for instance the effect of the Heidel – Simmel film: a simple 2-D animation, without any photographic realism whatsoever, which nevertheless manages to display a rudimentary setting, as well as recognizable characters and clearly intentional actions.124 The implications are crucial for fantasy moviemaking: the Heidel – Simmel film effectively proves that, when constructing a narrative, the external form of things, setting and characters is secondary to the proper connection and flow of events. In digital fantasy, this finding is extremely important, as it demonstrates the fact that it is easy to deliver a proper, comprehensible story without the need for absolute visual realism of the photographic type. Memory stores events in a schematic fashion, which are recalled and fitted on whatever is being shown. Instances such as the Heider – Simmel film clearly indicate that such meaningful structure supersedes external form. Just like with biological motion, human cognition proves to be capable of recognizing and applying multiple layers of schematic information, far more complicated than the input of visual information on the surface, limited or detailed alike.

Ideally, then, a theory of narrative comprehension would work for all narrative cinema, regardless of whether the movie is set in a realistic or a fantasy background. This is what this chapter tries to achieve: a common denominator for

124 See footnote 110. For the short animation by Heider and Simmel, see DVD-ROM, video file 26. 189

the way the mind comprehends narratives, that enables it to draw equally proper and accurate inferences about the plot without the unreal or unrealistic background placing any real obstacles in this cognitive process. In order to provide such a link, the chapter will first show that digital blockbusters never really broke the tradition with classical Hollywood narrative cinema; instead, they infused it with elements that actually made it more versatile, both thematically and aesthetically.

Subsequently, a model of perception will be used, based on Hochberg and Brooks and following the previous discussions on movie editing and motion, in order to define universalities of narrative comprehension. And finally, the study will address the link between narrative and factors such as memory and schema recalling.

5.2. From Old, to New, to Digital

5.2.1. The Old and the New…or the Same?

Is it possible to treat digital fantasy cinema in the same way as any other realist movie with a classical narrative, and thus connect both of them with elementary human cognitive mechanisms on the same theoretical premises?

Admittedly, the fact that human cognitive comprehension responds so well to causal narrative structure reveals the reasons for the success of classical narrative, as the latter has been the definitively proper mode of seamless causal storytelling.

Therefore, if it can be shown that the classical narrative mode shares this affinity with digital cinema, then, inevitably, the two will also share the same positive response by the human cognitive mechanisms.

Regardless of where one stands in relation to the status of digital cinema today, there are two observations that are undeniable and that need to be accepted 190

as starting points in order to attempt a connection between analog classicism and digital fantasy: first of all, American cinema, and, even more, digital blockbusters, globally dominate the moviemaking industry today to such a degree, that it would be a serious fallacy not to take them as a starting point for this discussion – which is what the present study has taken into consideration. Considering the figures, i.e. revenue, number of projects produced, etc., all other movies, no matter how well- constructed or successful they may be, are mere fireworks compared to the massive empire of American moviemaking industry. The second observation is that the prevailing mode of cinema storytelling as a whole is the classical Hollywood style.

The assumption is obvious: when speaking about cinema in general, in reality it is mainstream cinema that one has in mind; and, in turn, when speaking about the latter, one refers to movies shot in the classical Hollywood continuity style, or a variation of it. Discussions about any other kind of moviemaking need to specify type or origin, which makes those movies by definition secondary and marginal compared to the American cinema industry. Because of the obvious validity of these two observations, it is only natural that the American digital blockbuster, for the purposes of this study, cannot be considered anything else but the norm in cinema.

In order to verify this claim, the new challenge that arises here is to place digital cinema in the line of tradition of movie narrative, besides that of technology.125 Movie narrative is the generic concept by which a cinematic universe is conveyed to the viewer; and the royalty, so to speak, of movie narratives, is the undying and practically fail-proof method of Classical Hollywood style. Before we can ask whether digital mainstream cinema can be placed there at all, we need to see in

125 The tradition of technology, and digital cinema inside it, has been extensively discussed in Chapter 2. 191

brief what happened to the classical narrative tradition itself, if it still survives today, and in what form. Although there is a consensus that contemporary mainstream cinema is indeed the descendant of classical cinema methodology, there is some disagreement on a theoretical basis as to the way the two are actually connected. On one hand, the fraction represented by David Bordwell and his associates, ardently defends the view that classical narrative is so deeply rooted and powerful that it has barely changed form until the present day, even after the rising of the New

Hollywood in the 1960s. Bordwell supports that the longevity of the classical mode is established in the fact that, to the present day, it is virtually the only mode of movie storytelling that agrees with the way the human mind is predisposed towards receiving and processing visual information. In the same spirit, Kristin Thompson, in her chapter on post-1960s cinema in The Classical Hollywood Cinema, refutes those that see the American cinema of the 1960s onwards as a break from the classical tradition. The other fraction, on the other hand, led by Thomas Elsaesser and

Warren Buckland, also sees a continuation in the classical tradition, but also believe that the main feature of this late period of American cinema is self-reflection, in the form of a more conscious usage of trends from the past. It is obvious that the two fractions are not impossible to co-exist, at least in the case of digital cinema specifically.

Both fractions are right in stressing that the digital blockbuster, in both its older analog and the more recent digital form, indeed emerged from the ashes of old

Hollywood. Among the main reasons that led studios towards blockbuster productions were the rise of television leading to a decline in cinema tickets, the gradual move from the “unit-production system” to the “unit-package system” after 192

WWII, the complete failure of technologies like aromavision or smell-o-vision to produce the kind of life-like experience that studios hoped it would attract masses of movie-goers, and the aftermath of the Paramount Decree.126 Under these pressures, the studios were oriented towards “fewer, specialized projects,” a tendency which was translated to single films expected to excel in revenues; eventually, this tendency for fewer, more expensive and more sophisticated productions, created the conditions for the birth of the blockbuster (Bordwell et al, The Classical

Hollywood Cinema 368). The blockbuster was not born in a single day, nor was it an overnight discovery of the industry. It took several years for studios to realize the financial potentials of a handful of expensive blockbusters vs. a mass of cheaper films; and most importantly, it took them even longer, until the success of Star Wars in 1977, to discover another virtual goldmine, the franchise. Until that time had come, film production following the collapse of the studio system constantly experimented with both style and financing modes. Amidst these conditions, the foundations were set for film production as we know it today, including digital moviemaking, despite the fact that elaborate digital graphics themselves appeared in movie production quite later.

126 Up to the mid-1940s, the major film studios in the US controlled not only the production but also the distribution and exhibition of films across the country, with their privately owned theater chains, as well as with the method of “block booking:” independent theater owners were demanded to prepay studios for the rights to exhibiting a specific number of their films. This method ensured massive revenues for studios, surprisingly coming from films that had not even been made at the time the contract was signed. In 1948, under US Government accusations of violation of antitrust legislation, the US Supreme Court ruled against the major studios, forcing the latter to move from the “unit-production system” to the “unit-package system.” According to Bordwell and Staiger, the former involved massive film production by a single company which relied on its own technical or other resources; on the contrary, the newly introduced “unit-package system” focused on fewer film projects controlled and organized by a producer who hired external crews and personnel specializing in specific tasks. The shift from one system to the other took place after WWII, essentially ending the creation of films of the “unit-production” type by the mid-1950s, and replacing it with more focused projects, eventually forming the blockbuster film type (The Classical Hollywood Cinema 330). The 1948 ruling of the Supreme Court has been known as the Paramount Decree. 193

It was in the later period of the New Hollywood, from 1975 onwards,127 that the blockbuster was consolidated as a film production practice; and, notably, it is by the standards of the blockbuster that movies with extensive digital graphics are usually made until today, at least those intended for mass audiences and not considered independent or artistic works. After the enormous and unprecedented success of George Lucas’ Star Wars in 1977, the cinema industry was never going to be the same again. Apart from the aesthetic and narrative changes that it introduced, it was because of Star Wars that Hollywood industry started relying once again on expensive blockbusters (Metz 387). Digital movies, including comic book adaptations, fantasy, and video game spin-offs, are the offspring of this situation, because of the rise of the audiences’ demand for more and more spectacle – and what spectacle is more eye-catching that CGI? It was Lucas’ willingness to invest the profits from Star Wars to the creation of Industrial Light and Magic (ILM), his own special effects company, which “became the state of the art facility for making special effects part of standard practice in the creating of high budget Hollywood films,” thus bringing innovation and high-end spectacle to the special-effects sector

127 Walter Metz identifies three main parts in the period we call Post-Classicism: (a) 1960-1968, during which the industry sought for a new identity along the emerging technology of television; (b) 1967- 1975, which brought the new breed of moviemakers that formed the “Hollywood Renaissance;” and (c) 1975 onwards, beginning with the release of Steven Spielberg’s Jaws, and characterized by a more conscious regression to Old Hollywood styles of moviemaking. The re-establishment of the “hero” in a classical three-act storyline re-anchors moviemaking to Old Hollywood narratives and, therefore, comprises a shift from the pessimism usually found in the Hollywood Renaissance. Metz believes that Jaws, although continuing the New Hollywood line of movies, was the beginning of the end of the Hollywood Renaissance (375–387). Thomas Schatz also stresses the importance of Jaws as a landmark in the history of American cinema, noting that its success underpinned both the notions of blockbuster and franchise. For more information, see Thomas Schatz, “Just when You Thought it was Safe; Jaws & the New Hollywood; no Longer just a Blockbuster, a Movie is a Marketing Extravanganza Series: 1976-1996: 20 Years of Pop Culture.” 194

of moviemaking, that had remained practically stagnant for decades (Metz 388).128

Other special effects companies sprang after ILM because of the success of the latter, some of them even benefitting from the expertise of former ILM employees that they recruited, and altogether contributing to CGI as “one of the most important technological shifts in the history of Hollywood since the coming of sound” (389). In short, it was the period of New Hollywood in general, and Lucas’s vision in particular that can be credited with the creation and establishment of CGI itself as a sine qua non parameter of cinema, as it is now an extremely profitable product of the Hollywood movie industry. The two pillars of contemporary moviemaking practice are the classical period and its revival during the New

Hollywood period, and the rise of CGI special effects; this is why they cannot be considered as separate from each other.

It is obvious therefore that contemporary digital cinema is an after effect of a trend towards unification that has its roots as early as the 1960s. Cinema is still affected from the rise of that new practice in imaging, which shifted its attention to re-using pre-existing “images and image forms,” and at the same time began reconciling the opposing trends of mass culture and modernist practice in art (Darley

131). A key factor was the abandonment of abstraction and the return to devices of representation, such as “allusion, parody, pastiche, citation and quotation,” practices which were closely related with the practice of montage, brought back from the oblivion of the avant-garde and reworked along the lines of contemporary culture

(131). In a general spirit of discontent with the commodity culture promoted after

128 ILM’s innovations in digital and other special effects gradually replaced traditional special effects techniques so successfully that earned the company nine Academy Awards in the 1980s (Redmond 136). 195

WWII, the new trend involves “practices ('self-referentiality' in its broadest sense) which return to copying, repeating, recombining, re-situating, quoting, referring to pre-existing texts – that is to say, to the forms, idioms, styles and images of this greatly expanded cultural horizon” (132). As Darley notes, in a peculiar yet innovative way, the grand opposition between the naive realism of mass visual culture and the anti-narrative montage tradition of modernism – which has prevailed for much of this century – began to disintegrate; and digital culture has become a brave new territory for the practice of intertextuality and cross-reference, gladly offering its technological easiness.

5.2.2. CGI in the Narrative Vs. Spectacle Debate

The integration of special effects technology, especially CGI, thus came to be considered one of the main new features introduced in New Hollywood.129

Nevertheless, as discussed in Chapter 2, here too technological innovation should not be considered as revolutionary as some may think; doing so leads to exactly the kind of break which is being refuted here, i.e. between contemporary and past cinema. Such a break not only is unnecessary as much as it is erroneous, but it also undermines the positioning of digital cinema as a continuation of classical moviemaking. The act of using and promoting the latest technology available provides no grounds whatsoever for separating digital cinema from its analog past: after all, Bordwell et al show that advertising the use of technological innovations is

129 Bordwell et al mention a number of “technological feats” in this context that are now considered characteristic of the New Hollywood: “sophisticated special effects, new camera supports (Panaflex, Steadicam, the Louma crane), television viewfinding, time-coded synchronization, computer-assisted storyboarding, and expanded multitrack sound recording” (The Classical Hollywood Cinema 373).

196

no news in the history of Hollywood when it comes to promoting new movies; and, as regards recent innovations in particular, those have variably been sanctioned “in the name of economy, realism, unobtrusiveness, spectacle, and narrative supremacy” (The Classical Hollywood Cinema 373). And, to make matters worse for polemicists of digital technology, it is undeniable that it was technology that came to the rescue of the cinema industry a number of times after the collapse of the studio system, from the blockbuster to the home video, and then to the multiplex hype and the DVD, in what Bordwell calls “a tale of last minute rescues” (The Way Hollywood

Tells It 4). Even earlier, during the first decades of moviemaking, the industry presented one development after the other, none of which was ever markedly named a “revolution,” perhaps not even the coming of sound, which is probably the ultimate advance in moviemaking of the first half of the twentieth century. Abrams et al note the astonishing rate by which CGI is advancing ever since the early 1980s, constantly gaining more and more of the run time of films, and constantly being improved with novelties introduced practically at an annual rate (86). But at the point in which they call this development in moviemaking a “digital revolution” (86) we should be inclined to disagree; moviemakers have done nothing different than what their predecessors did since the beginning of the history of cinema, i.e. keeping the art of cinema at pace with the available technology, as well as with the demands of the audience. Still, it is easy to understand and accept the fact that the term

“digital revolution” refers to contemporary culture in general, and cinema has been simply albeit erroneously dragged along in this trend.

Therefore, in contrast to any claims of discontinuation between old and new modes of moviemaking, not only do the structures of classical narrative still draw 197

breath, but they also do not even seem to be incompatible with the sensory overdose that digital cinema offers to spectators. Specifically as regards the blockbuster culture, which usually is all about CGI-overloaded movies, classical narrative seems to have facilitated such productions rather than inhibit them, as the codes by which a spectator perceives and receives a movie had already been perfected decades ago. By the time New Hollywood started bending the rules of the game, moviemakers already knew how to put good stories on screen; CGI, which was brought with the blockbuster, was simply a way to embellish those stories with something far more elaborate than anything used up to then, a technological maturity of visual storytelling much anticipated by special-effects visionaries as early as Méliès. Bordwell, Thomson and Staiger are positive that Classical Hollywood has proved too hard to die in recent movie production. It comes without saying that this statement includes the digital blockbuster type as well. In The Classical Hollywood

Cinema, the authors comment on film production after the 1960s, noting that movies are still made primarily with the same principles established during the

Classical years (369–370), which do not contradict effective spectacle. In a more recent work, Bordwell insists that classical construction has all but disappeared in the post-classical era; the revered linearity of the classical way is still found in contemporary and recent Hollywood action blockbusters, proving that “narrative and spectacle aren’t mutually exclusive concepts” (Making Meaning 104). Elsaesser and Buckland support exactly the same view; following a narrative analysis of action movies,130 the authors conclude that, even if there is an excessive use of spectacle

130 Elsaesser and Buckland provide a lengthy analysis of Die Hard (1988) which is not a fantasy movie, nor does it have excessive application – if any – of CGI. Nevertheless, since their argument is wider and revolves around post-classical cinema and spectacle in general, which cannot be conceived 198

devices, this is by no means at the expense of storytelling; therefore, even if there was opposition between classical vs. post-classical cinema, that opposition could never hold on the basis of narrative vs. spectacle (60–61).131

Still, it does not really require a very perceptive eye to see that blockbusters indeed show disregard for the rules of classical storytelling in a number of cases.

Such deviations usually have to do with alterations that aim to minimize the temporal distance between two action scenes. These alterations often involve omissions that indeed take their toll at a solid and tight cause-effect string of narrative events, of the kind the classical films were famous for.132 The issue, though, is not that classical narrative is discarded and slowly diminishing, gradually giving its place to digital or other spectacle as the sole characteristic feature of contemporary blockbuster; we can admit a loosening of classical narrative orthodoxy but this is not because audiences do not care for stories anymore. The symptom is not due to the lack of stories, but exactly the opposite: it is due to their multitude. Audiences have been constantly bombarded with so many plotlines, that patterns of plots are now discernible and familiar, and thus several parts of them are obsolete because they are not required anymore in order for the narrative to acquire a meaningful flow, as

without the contribution of CGI movies, the argument can be considered applicable and valid for the purposes of the present study as well. 131 The authors test the narrative vs. spectacle hypothesis against both the Aristotelian (i.e. linear) model of narrative, that Bordwell employs, as well as the Oedipal (i.e. structuralist) one, and find it equally problematic. The latter exceeds the focus and purposes of this study, therefore there will not be any further reference to it; still, it is very interesting to note that even these two completely different aspects of narrative theory are proven equally hostile against a polarization between narrative and spectacle. 132 This is, for instance, a relatively obvious situation in Luc Besson’s The Fifth Element (1997), starring Bruce Willis; in fact, Peter Lunenfeld notes that, when Willis was asked about “the amiably shambling incoherence” of the movie, the actor laughed and replied that nobody cares for stories anymore (151). 199

it was the case in the past. This is what Peter Verstraten reads in his review of

Lunenfeld’s arguments:

[T]he much-debated crisis of (classic) film narrative has more to do with

the sheer quantity of stories in the contemporary information age than

with indifference: nowadays, simple references suffice . . . We can easily

jump from A to C while omitting B because the visually literate viewer

has already trodden that particular path many times . . . [T]he typical

standard elements of a story no longer require elaborate psychological

motivations. (154–155)

Classical storylines annoy critics either because of their commonplace banality, like e.g. plotlines that re-surface in many movies, or, on the contrary, because of alterations in their norms. In any of these cases, classical narratives prove that they matter in contemporary blockbusters, and in fact contribute to their success. Digital blockbusters may appear to favor spectacle over story; but in reality they simply underline the importance of classical narrative literacy, rather than maintaining classical narrative methodology. They may not always follow classical patterns faithfully, but they rely significantly on the familiarity of audiences with “references to overtly familiar plot devices;” this literacy of the audience with “story patterns from other films” will enable them to tolerate gaps that do not require to take up any screening time, and causality between screened events will find its way as the viewer will elicit these patterns from his knowledge of storylines from previous movies (Verstraten 155). Consequently, instead of digital blockbusters being a debase or contempt of classical storyline, such an approach shows that they actually praise it and step firmly on it before they even begin to deploy the visual delight of 200

their synthetic universes on screen. Therefore, even if there is indeed any tendency in movies to show a withering interest in stories, the blockbuster as a kind of moviemaking is not the reason.

5.3. Coherence as the Prerequisite for a Unified Theory of Narrative

Comprehension

5.3.1. Accommodating Oneself in a Fantasy Movie World

It can be easily deduced from all the above that the form of mainstream cinema over the decades has indeed undergone several modifications, but the focus on the basic principles of classical storytelling has been maintained one way or the other. Therefore, the establishment and application of a cognitive model for cinema narrative comprehension with respect to digital blockbusters does not need to be separate from explanations applied to analog continuity moviemaking. A unified theory of narrative can be based on cognitive comprehension, which is a neutral point of reference for both analog and digital cinema viewing; but it should be kept in mind that taking cognitive comprehension as a common point of departure does not mean that it is easy to sweep under the carpet and pretend to ignore the fact that digital narratives have a different, much more demanding and elaborate kind of worldmaking than analog ones.

We can divide any narrative movie into two broad, yet separate and clearly defined parts, the cinematic story and the cinematic world. The former refers to the set of interwoven events and actions in the movie, whereas the latter to the entire world in which they take place. In analog movies with a realistic setting, both the cinematic story and the cinematic world are recognizable by spectators: the events 201

are generally inspired by real life experiences, and the setting is a realistic background, which is normally a known location, or perhaps an imaginary one that could easily be real somewhere in the known world. The elements of the movie content may be re-ordered and subject to the interpretation of the viewer, but they generally remain highly familiar because they are up to 100% derived from a well- known pro-filmic reality. But in digital fantasy, there is no such balance between cinematic story and cinematic world; the story is also one comprised by recognizable events, but the world in which those take place exhibits a much more varied degree of departure from resemblance to locations in the real world: the spectator will not simply have to recognize objects, settings, and situations that are a good likeness of his own reality, as it happens in classical analog films, but he will have to create and assign entirely new meanings to visual and narrative elements that he has never encountered before. The key lies in the fact that the digital agrees with the analog as regards the role of the cinematic story: the reason the cinematic story is generally kept intact in the majority of contemporary digital blockbusters, despite the non- realistic cinematic world in which it is set, is indicative of the importance of narrative structure in movie comprehension. A properly structured story appears to be a solidifying agent, a powerful substance that holds together the cinematic world and provides it with the necessary degree of credibility.

Computer Generated Imagery by nature requires a much higher mental effort on behalf of the viewer in order to become intelligible. When the viewer visits the movie theater he is normally predisposed towards constructing a story in his mind out of what he is about to see. Synthetic worlds are one thing, but stories and imaginary scenarios built within them are something else: they require elaborate 202

and careful event processing on behalf of the viewer, as, not only the visual part, but also its rules, conventions, relationships and behavior of characters need to be composed and comprehended by putting together bits and pieces of the narrative.

Eventually the cinematic story will form an internal frame, an infrastructure that will keep the cinematic world from falling apart.

In digital fantasy blockbusters the input of stimuli is expected to be massive and the mental hypotheses applied by spectators to be more complex and complicated. Fantasy CGI movies may be all about worlds that audiences have never encountered in real life; but proper narrative flow ensures that these movies also enable the spectator to mentally navigate himself in the cinematic universe using processes by which he also uses to encounter the real world in the first place. If a person suddenly woke up in extremely elaborate CGI locations like Lothlórien, Zion,

Coruscant, or Pandora,133 he would probably feel completely disoriented, lost, and bewildered, because he would have been immediately inserted into a world the rules of which he would not know. Cognition immediately takes over: the person would probably seek for familiarities with his own world and experiences. Holding on to them, he would gradually be introduced to the new conditions he found himself into. But how is it that this seems to happen a lot faster in digital fantasy movies, even in cases where spectators have not even seen the movie trailer and therefore have no idea what they are about to see? Of course the cinema-goer that consciously visits the movie theater is a bit arbitrarily compared to one that wakes

133 The forest of Lothlórien is one of the Elven realms of Middle Earth in The Lord of the Rings trilogy. Zion is the last post-Apocalyptic underground city of humans in the Matrix. Coruscant is an all-urban planet and headquarters of the Republic in the Star Wars universe. Finally, Pandora is the home planet of the Na’vi alien race in Avatar. Far from exhausting the list, these worlds are perfect examples of elaborate CGI visual worldmaking in digital movies. 203

up into unfamiliar surroundings without intending to do so, but this comparison hopefully demonstrates that, as is the case for the unfortunate traveler, acquaintance with the new environment indeed needs a known point of reference for the movie viewer as well: where the traveler would be lost and bewildered, the spectator can appreciate the spectacular novelty of the surface, i.e. the cinematic world or the visual part of the movie, as long as he is constantly supported by a familiar infrastructure, i.e. the cinematic story or the event patterns in the movie.

With the danger of slipping to a narrative vs. spectacle controversy which has been refuted in the previous section, it could be said in plain words that the story is there to support the show, and as such, narrative will always more or less rely on the ways the efforts of the human mind are by default oriented towards organizing stimuli into a coherent and meaningful flow. And this, by definition, requires on behalf of the spectator both an enhanced awareness and compromise with the conditions of the cinematic universe that the movie text poses.

5.3.2. The Importance of Narrative Familiarity and Coherence in Synthetic

Worldmaking

A cognitive model of comprehension implies negotiation of meaning coming from visual stimuli; and, in turn, negotiating meaning implies that the spectator has his own cards to play with. In the present context, these cards are the knowledge that the movie spectator comes pre-equipped with when he enters the movie theater. This negotiation between existing and new information must conclude to a vision of the cinematic world that renders it a credible realm; in other words, this world needs to be possible at least under the circumstances of the cinematic story: 204

For a text to project a world, this world has to be accessible from the

actual world, which means that we must be able to use our image of this

actual world – our personal encyclopedia, Umberto Eco would say – to fill

the gaps of the text. (Which I named, following David Lewis, the principle

of minimal departure.) This encyclopedia depends on what we learn in

books as much as on our direct experience of life. It can also happen that a

text refers not only to our knowledge of the actual world and its history,

but also to our familiarity with other fictional worlds. (Ryan, “Possible

Worlds” 1)

Ryan’s concept of the minimal departure is a key parameter in accepting the credibility of the visual narrative encountered in a digital fantasy movie. In a movie featuring a universe that is visually and thematically departed from that of the actual world, the minimal departure is an anchor to the pre-existing knowledge of the spectators, which ensures that the cinematic world and the cinematic story will blend together smoothly. On the basis of how successful this worldmaking is, Ryan recognizes three categories of texts:134 (i) texts that feature logically coherent worlds, presented realistically; (ii) the “Swiss cheese” texts, whereby any gaps in the rationality of the featured world is clearly marked, contained and controlled, thus allowing the viewer to apply the principle of minimal departure to explain those gaps with his prior knowledge;135 and (iii) texts without any world featured, such as

134 Ryan’s argument is about texts of fiction in general. There is no reason not to apply her insightful categorization to cinematic texts as well. 135 There is an important note to make here. This “explaining” of gaps is not gap-filling; as we will discuss in section 5.4.2 of the present Chapter, Hochberg demonstrates that events that are not explicitly shown in visual narratives are not reconstructed in the mind of spectators. The “Swiss cheese” holes do not constitute specific events, but rational explanations given to the relations between features of the fictional world, as the spectator’s mind attempts to eliminate any arbitrariness in it. 205

abstract painting, which are deliberately and “systematically built on obliteration and contradiction” (“Possible Worlds” 1). Digital fantasy movies belong to the second category; they use continuity editing and normal causal structure in order to be presented as an ordered and principled world. The inevitable gaps are rationalized by the spectator’s prior knowledge, his effort toward building a coherent meaning via visual momentum, as well as by application and constant amendment of schematic maps which he is already familiar with.

It is only fair to assume that the principle of minimal departure does not apply only to the cinematic world, but also to the cinematic story, which is brought to the audience via the narrative. Both cinematic world and cinematic story in fantasy universes need to be supported by a delicately maintained minimal departure. The reason is simple: borrowing Bordwell’s terminology, we can say that each narrative is a syuzhet, and the fabula underneath it will definitely contain holes and gaps, because the totality of events in any fictional world is too vast to be conveyed to the receivers (and, in fact, does not even need to be conveyed in its totality, anyway, as we shall see in the next section); therefore, specifically in a fantasy universe, these assumed non-conveyed events should be expected to be substantially more, and some of them also quite different in type than those encountered in real life. The fantasy syuzhet reveals a universe that needs a significantly more elaborate decoding of events, even if these events are organized in a “continuity” type of narrative structure. The reliance on recognizable patterns of events from real life becomes at the same time more fragile and more mandatory, as the spectator is guided through the rules of the synthetic universe much more slowly than in movies representing normal reality, the rules of which he already knows from 206

real life to a far greater extent. This reliance to recognizable patterns, nevertheless, is not gap-filling; it is simply a point of departure for the spectator to associate the structure of events to one another, without imagining or recreating inside his mind events that are not screened, as, in an elaborate fictional world such as CGI universes, this is practically impossible to happen.

Digital fantasy narratives, therefore, demand that they are made with a lot more attention to detail, so that they present a satisfactory level of coherence. It was mentioned earlier that the cinematic story is the infrastructure, a necessary condition for the surface appearance of the cinematic world to be easily accepted.

Therefore, digital worlds above all else need to demonstrate coherence, which will counterbalance as many gaps as possible, and will also make it easy to tackle the ones that are left open. On one hand, the spectator comes equipped with a perceptual logic or reasoning dictated by his constant encounter with a tangible world around him, which is, in Lakoff and Johnson’s view, the outcome of the spectators former bodily experiences of the living world; and on the other, he is now confronted with universes that are partly or entirely virtual, which means that he has never had any tangible experience of the same sort with them. Coherence will help the viewer handle the struggle between the physical intangibility of these universes, and his own innately built efforts to make sense of them using the logic acquired through his senses and normal perception. As James Walters puts it, artificial universes “seek to shape a world that is credible from elements that are unfamiliar, strange and unrecognizable to us. The make-believe must still make sense” (113).

Coherence in digital movies takes one thing for granted: the viewer will have to do a lot of work completing the puzzle of a meaningful fabula in his mind, as that fabula 207

will have to accommodate his normal cognition in the laws of a newly-encountered universe. Therefore, coherence is, to a significant degree, a personal issue for the spectator to resolve.

This coherence goes by many names. Bordwell, for instance, assigns it to realistic motivation, a feature which “depends on what seems lifelike to someone versed in specific conventions” (Narration 149). This means that coherence in this case will rely on good faith, on behalf of the spectator, to the structural laws of the cinematic universe: whatever seems logical inside this universe will have to work in its favor, even if it does not seem equally logical or plausible in real life. George

Wilson similarly locates coherence in the power of classical narrative itself, in which

“there is a promise to depict a set of events, acts and situations which will turn out to have an internal explanatory coherence” (qtd. in Walters 114). In other words, the celebrated visual-compositional reliability of the classical mode also provides a sense of reliability in the unity of the entire cinematic universe: this universe may be artificial, but the classical structure will not allow that to be revealed. A similar approach is provided by Ryan who demands the viewer to “play the game,” i.e. to persuade himself temporarily but consciously that the fictional universe exists:

My own solution lies on a disjunction between what the reader knows and

what the reader imagines. The reader knows that fictional worlds are

incomplete, but when he “plays the game,” when he submerges in a

fiction, he pretends to believe that this world is complete . . . In the act of

imagination, . . . lacks are an epistemological question and not an

ontological one. (“Possible Worlds” 2) 208

For Ryan, a narrative takes for granted that the world in which the events take place is the actual world. It is the spectators that eventually make the narrative happen, but the events are not screened have to be the ones that provide the necessary level of coherence to that world, without the need for those that are not screened.

Coherence, in this respect, is a rather wide concept, encompassing a mode of assembly of all elements that will give the movie an aura of credibility, even if what passes through the silver screen has nothing to do with real life.

To play a little bit with Todorov’s definition of the fantastic, we can now be certain that digital fantasy cinema is already well over the “hesitation” between the natural and the supernatural that the theorist proposed as a definitive trait of the genre.136 Instead of hesitating between the two, digital movies go for both the natural and the supernatural and know when, how, and why to keep them together.

The world of the movie narrative is purely unreal but the way it is going to be marketed and served to the audience does not – and should not – need to be. This way, the synthetic fantasy of movies like Star Wars, Avatar or The Matrix are bridged to the reality of the spectator, and thus expand into it. This condition consequently leads to moviemakers using and tampering with these elements more consciously.

One can arguably wonder, though, as to how far this tampering can go before it crosses the line and becomes unsuccessful. It seems that there is no real recipe;

Bordwell is confident in the capacities of classical cinema that have enabled it to survive these alterations; the solid resilience of the classical mode of storytelling through all these decades lies on “a coherent approach to genre, plot, and style that

136 According to Todorov, one of the conditions that the fantastic text requires is that it “must oblige the reader to consider the world of the characters as a world of living persons and to hesitate between a natural or supernatural explanation of the events described.” Citation retrieved from the website of the University of North Carolina at Chapel Hill. 209

can assimilate a great range of thematic material” (The Way Hollywood Tells It 14).

Classical cinema indeed incorporates a set of rules, but, as Bordwell very successfully puts it, “[t]he classical system is less like the Ten Commandments and more like a restaurant menu” (The Way Hollywood Tells It 14); it relies on a loved recipe and known ingredients, executed with creatively innovative variations.

5.4. Cognitive Comprehension of Continuity Narrative Sequences in Movies

5.4.1. Dos and Don’ts in Applying Cognitive Psychology in Cinema Narratives

The Heider-Simmel film is a wonderful demonstration of this power of coherence on the level of narrative. A string of actions that acquire meaning through each other in relation to the order in which they are presented proves that the existence of a set of schematic events, despite them being in abstract form, constitutes mental structures that are perfectly adequate in assigning specific meaning to experienced actions, even without the need for intermediate or secondary actions to be screened. As we have seen in Chapter 4, the distinguishing factor of movies as an art is motion, and this motion needs to acquire meaning from cognitive levels that are far deeper than a simple tracking of changes in its trajectory.137 In order for this to happen, and for memory to take over and apply schematic mapping to events and their interrelations, these actions first need to be parsed and ordered, and then recognized and evaluated as parts of a string of events with a more or less clear target:

The fact is that we must parse most of the motion patterns we encounter

in terms of purposeful acts, not in terms of physically specifiable

137 As we have seen in Chapter 4, this is a point where Hochberg and Brooks disagree with the way Johansson provided a “direct” explanation of biological motion. 210

trajectories. They are not the same, in that the identical act can be

expressed and represented by very different physical motions . . . The

story structure encodes the too-numerous trajectories as a smaller

number of more distinctive and familiar, purposeful actions. These are the

same units by which the motions of people and animals achieve

recognizable organization (and which are perhaps learned by infants even

before they learn simpler physical motion). All of the unrelated

movements are immediately meaningful, if the viewer has undertaken to

construe them as purposeful and expressive actions. (Hochberg and

Brooks “Movies in the Mind's Eye” 382)

This parsing of actions will dictate the application of schematic mapping on the events of a movie, based on memory of already experienced events, which have formed, in the mind of the spectator, clusters of event patterns readily available to assist in interpreting new experiences. This is something that needs to be taken for granted before any attempt to apply a cognitive interpretation to movie-watching: If we are to understand movie-watching as a cognitively active event on behalf of the spectator, we also need to accept his internalized knowledge being in constant conversation with the movie content, with this conversation being a continuous application of schematic mapping to visual narrative events. But still, this is a somewhat general statement; the actual way of applying a model of schematic maps in cinema viewing requires further elaboration, not only with regards to how it should be approached, but also to how it should not.

This need for elaboration has created shortcomings in the realm of cognitive cinema theories that reveal reasons why the latter have failed to agree with each 211

other in several details, despite the fact that the application of schematic maps in narrative theory has been around for quite some time now. A quick look in

Bordwell’s model, which is perhaps the most prominent one that advocates an actively thinking movie spectator, serves as an example. Bordwell’s model was undeniably a huge breakthrough, and has proven extremely influential, but is definitely not without hiccups. His constructivist account of cinema spectatorship is famous for an insistence on a spectator that is “real,” “active,” and “executing a variety of operations” at the time of movie-watching (Narration 29–30). Denying the existence of any real or solid evidence that the structure of movies may be grounded on linguistic foundations, i.e. the kind of a universal language of cinema that structuralists and post-structuralists envisioned, Bordwell delineates movie-watching as a process of constant interaction between the spectator’s mind and the movie narrative. He essentially replaces the structuralist model of syntactic items with a set of schemata, which he nicely defines as innate “organized clusters of knowledge”

(Narration 31). The spectator carries these schemata with him in the movie theater as his own pre-acquired knowledge, and “constructs a perceptual judgment on the basis of nonconscious [sic] inferences;” in other words, in the process of movie- watching, the spectator receives input from the movie, uses schemata to make non- conscious hypotheses about the cinematic story, and in the course of the narrative constantly re-negotiates these hypotheses for either validation or re-orientation towards new ones (31). As we have seen, the process of inference-making, according to constructivist psychology, may be top-down or bottom-up.138 Bordwell stresses the fact that both processes are inferential, since “‘perceptual conclusions’ about the

138 See footnote 61. 212

stimulus are drawn, often inductively, on the basis of ‘premises’ furnished by the data, by internalized rules, or by prior knowledge” (31).

The problem with the way Bordwell applies schemata in his model lies in the assumptions that this model makes as regards what the human mind does with these schemata in relation to comprehension of the movie narrative, and the weight it assigns to mental representation. On one hand, Bordwell is correct, and in agreement with Hochberg and Brooks, as regards the general idea of schematic perception of movies, and the necessity of applying pre-set mental structures while watching them:

Without a mental structure in which to place the series of shots — their

order as parts of an event, or as sample views of some spatial layout — the

series is not rememberable [sic]. But it is rememberable given such a

structure and the effort to apply it (that is, the attentional resources). That

argues that mental structure is involved in the process of event perception

itself . . . (Hochberg and Brooks, “Movies in the Mind's Eye” 387)

Thus Bordwell is on the same page as Hochberg and Brooks as regards the fact that meaning derives from the content of a movie constantly being evaluated by the spectator against available mental structures. But his primary focus on the function of editing cuts in relation to the distinction between fabula and syuzhet, i.e. the temporal inconsistency between the story itself and the actual screening time, eventually seems to be causing his model to drift away from the actual target, which is the role and function of mental representations: instead of explaining the workings and nature of mental representation, Bordwell’s model ends up focusing primarily on “abstract story structure,” rather than actual cognitive processes; his 213

account focuses on “the structure of the narrative film itself . . . and not the viewer’s mental representation” (Hochberg and Brooks “Movies in the Mind's Eye” 387).

The fact that Bordwell allowed neo-formalism theory of art to overshadow cognitive science in his model was a fallacy that had as a direct consequence his misconception of the concept of schema itself. Buckland reveals this misconception in his comparison of Bordwell’s schemata to those of Lakoff and Johnson. As we have seen in section 3.3.4 of Chapter 3, Lakoff and Johnson conceptualize schemata quite similarly to Hochberg and Brooks, in terms of the fact that they take physical motion of the body into account, as well as in terms of the flexibility of their argument, which derives from incorporating in the model the processes of metaphor and metonymy. In both Hochberg’s and Lakof & Johnson’s case the difference with

Bordwell’s schemata is that the former are image-based, as well as the fact that they are immediately connected to the function of the human body rather than being disembodied and transcendental:

[Bordwell] conceives schemata as transcendental, functioning to

construct literal meaning only, and isolates them from both language and

the body. George Lakoff and Mark Johnson argue that schemata conceived

in this way are static structures simply imposed upon perceptual input to

give it meaning . . . In contrast to Bordwell’s, Lakoff’s and Johnson’s

schemata are image-based, are embodied and inherently meaningful (are

constituted by the structure of the body) rather than being

transcendental, are based on metaphor and metonymy, and are dynamic

rather than static. (Buckland Cognitive Semiotics 31) 214

In other words, although Bordwell simply followed the mainstream cognitive trends of his time, today we cannot but point out that his concept of schemata is outdated, as it is entirely inflexible. As Buckland notes, his schemata rest too much on a rather finite literal meaning, without taking into consideration the “imaginative projective devices” that are inherent to active cognitive processing of information; this function is exactly what Lakoff and Johnson assign to the role of metaphor and metonymy

(Buckland Cognitive Semiotics 31).

Essentially, what Bordwell did was to insist on the structure of the story as if that story was a stimulus ready to be consumed, with a more or less pre-determined cognitive outcome, since he believed that the cause-effect schema would absorb and translate the story structure in a prescribed manner. Although he was probably correct in the power of this schema and its function when a spectator watches a movie narrative, he seems to imply that the mental representation can be deduced from movie structure, will be identical for all spectators, and is thus not really worth exploring. Consequently, it appears that the neo-formalist preoccupations of

Bordwell’s model, in spite of its correct assumptions and well-defined scope as well as its convincingness regarding the power of the cause-effect schema, tipped the scales towards the structural aspect of movies rather than their content. As a consequence, the application of cognitive theory malfunctions in Bordwell’s model, eventually depriving his theory from providing a conclusive scientific account of cinema perception that is based primarily on the actual psychology of perception, and secondarily on theory of art, as it should have. The result is that, despite its convincingness, influence and overall effect, Bordwell’s model sadly remains 215

symptomatic of the general dispersal and lack of consensus that is manifested in cognitive trends in cinema theory.

But an approach directly opposite to that of Bordwell is just as wrong.

Hochberg and Brooks stress that Bordwell indeed neglected the actual nature of the operation of mental structures in favor of the form of the story, but to assume too much from these mental structures is just as erroneous. The authors warn us that accepting the function of mental representations in movie-watching means that we should first comprehend the difference between these mental representations and reality: our real life experiences as well as movie-watching is stored in our minds not in the form of a complete and identical copy of the world we encounter, but in the piecemeal fashion of schematic patterns. In simpler words, this means that our piecemeal perception is fitted on sketchy mental structures which, as such, do not share the same physical (spatial or temporal) characteristics as either the world in reality or that on the screen; “[t]he mental structures fitted to our successive glimpses at the screen and at the world cannot have the characteristics of the world itself (despite assertions that they do)” (Hochberg and Brooks “Movies in the Mind's

Eye” 387).139 Specifically in the context of movie-watching, any cognitive model that overlooks this essential difference between the cinematic world and the experience of this world, would erroneously assume that the mind reconstructs the fabula in its entirety, i.e. as a linear mental representation of both things screened and things not

139 More specifically, Hochberg and Brooks stress that that the physics we experience in daily life are not identical to our mental representations of them: “[m]ovements as we remember or anticipate them do not continue to run off in time, nor do remembered or anticipated layouts continue to extend in space . . . [W]e start to lose extended time and space when the supporting input ceases. In any case, it certainly does not take some 90 minutes to review in our minds the average movie’s representation” (“Movies in the Mind’s Eye” 387). The authors also provide two examples of models that assume exactly what themselves are ardently refuting: Irvin Rock, “The Logic of ‘The logic of Perception;’” and Roger N. Shepard, ‘‘Ecological Constraints on Internal Representation: Resonant Kinematics of Perceiving, Imagining, Thinking, and Dreaming.’’ 216

screened, which are filled in all detail by the mind automatically while watching a movie. As a consequence, such models imply that the mental representation created for the movie is extremely elaborate, as it is assumed to correspond to a 1:1 temporal and spatial reflection of the cinematic universe, gradually put together during the screening time. Hochberg and Brooks’ piecemeal model, on the other hand, denies the possibility as well as the necessity for such a level of detail in mental representations, emphatically stressing that we “simply cannot take either the moving picture or the events and space it represents as a faithful model for the film’s mental representation” (“Movies in the Mind's Eye” 387). Instead, they opt for a model of mental representation that makes provisions for the fact that, on one hand, human memory is simply unable to retain a massive amount of information, but on the other, it may very well function and respond to narratives even with blank spots in what it actually retains.

In their explanation of narrative comprehension during movie-watching,

Hochberg and Brooks take as a cornerstone the fact that what the spectator retains from the sequence on the screen has little, if anything, to do with what he actually sees:

[R]epresented movements and extents do not long outlast their

presentation on the screen. This means that, very shortly after it has

occurred, the representation of an event, or of a part of an event, is

different from the perception obtained during the event itself. Some

specific physical information about space and time is lost with time. We

assume that such losses occur as well after each change in direction or 217

speed, or after each cutaway or change in scene. (“Movies in the Mind's

Eye” 382)

This assumption renders two things essential: first, the elusiveness of the actual details of events renders the use of mental representations in the form of schematic events necessary for absorbing and comprehending the content of the movie; and second, as demonstrated by the Heider-Simmel film, that these events must be parsed into recognizable patterns of actions that manifest purpose. A movement in space is transformed from indeterminate to a solid memory only when it is

“regenerated when needed” by being fitted in a schematic event (“Movies in the

Mind’s Eye” 382). This, in turn, raises two further important questions: (a) in what way should we think of the actual schematic events that are retained, since the limited resources of memory do not allow us to use either the fabula or the syuzhet; and (b) whether some cuts or other moviemaking breakpoints, that aid the spectator to separate events into meaningful actions, are more important than others in this respect. We shall address these two issues in turn.

5.4.2. A Cognitive Notation Model for Visual Narratives: Preserving Only

What Matters

Hochberg and Brooks take two things as starting points: first, that movies should be studied as perceptual experiences during their first viewing, which for most people and most movies will also be the only viewing (“Movies in the Mind's

Eye” 389). And second, that the Heider-Simmel film teaches us that segmenting events into identifiable clusters of actions explains the success and effectiveness of editing, especially if the latter aims at continuity. The model that they propose for 218

the perception of movies does not seem to have been put together having any particular kind of movies in mind. In fact, if anything, the authors most probably did not have in mind digital blockbusters specifically. And yet, their model appears to be a far more appropriate one for this specific kind of cinema, exactly because of the elaborate worldmaking and challenges of coherence that it demonstrates.

There are a number of reasons that welcome the application of this model in digital movies as regards narrative. First of all, the model works as a cognitive approach to cinema because it strictly focuses on cognitive science and experimentation, without being contaminated by any abstractions crawling inside it, passed down from the structuralist or post-structuralist heritage of cinema theory.

Second, as we have seen, mainstream digital cinema does not deviate from the recipe for successful moviemaking that has been crystallized since the time of continuity editing, despite the facelift that the latter has undergone in order to meet contemporary technology and marketing needs. And most importantly, the model is well suited for digital cinema because of the way it visualizes the form that narrative takes as a mental representation. Contrary to both approaches, i.e. that either assume the spectator to be in a constant and detailed gap-filling process, or that focus more on the structural aspect of movies themselves, Hochberg and Brooks’ schematic model argues for the exact opposite, i.e. the lack of necessity of filling in every little detail that does not take any screening time. In a kind of narrative such as digital fantasy, where the worldmaking process is bound to leave a lot more gaps unattended compared to a narrative that is set in a realistic version of the world portrayed, this model explains exactly why and how the departure of fantasy from our real-life experience does not bother us in movie-watching: the synthetic world 219

appears as complete and coherent, because we do not really need to be aware of everything that is omitted inside it.

Instead of a kind of mental representation that reconstructs the narrative in its entirety, Hochberg and Brooks believe that spectators neither fill in themselves any elements missing from the narrative, nor do they have a schematic yet complete fabula in their mind to use for constant cross reference. Drawing on assumptions about the perception of narrative events during the process of reading,140 the authors believe that we retain and attend to far fewer events that what some might think:

While reading online [i.e. in real time] . . . events that are distant in the

narrative may go unconsulted, and unmentioned details are not filled in,

so long as no inconsistency is encountered. When someone watches a film

or tape unfold at its own pace, it seems even more likely that no filling in is

done and no overall structure is consulted, so long as the new input is

consistent with the immediately preceding or local context. And a well-

made film, intended like most films in history for a single viewing, must

therefore in general be locally comprehensible. (“Movies in the Mind's

Eye” 388)

The authors compare two scenes featuring exactly the same event, with one of them presenting all actions on screen in one continuous panning shot, while the other

140 More specifically, Hochberg and Brooks refer to Gail McKoon and Roger Ratcliff, who propose a minimalist hypothesis as regards the processes of inferences made during reading a text. The minimalist hypothesis assumes that, in the process of reading, the only kind of inferences that are encoded automatically are those that are “based on easily available information, either from explicit statements in the text or from general knowledge, and those that are required to make statements in the text locally coherent” [emphasis added]. This assumption contradicts constructionist views of text processing, which assume that during reading the reader draws on additional inferences in order to fully reconstruct a complete mental representation of all the events that the text describes (“Movies in the Mind's Eye”440). 220

using edited shots and cues like gaze direction of actors; they believe that, despite the difference in the way each movie presents its events, as well as in the cognitive processes involved in each case, there will be no actual difference between the two versions as regards the story structure itself. In other words, despite the different perceptual mechanisms employed by the spectator in each version, the way the story is comprehended in the panning shot version will not differ from the one in the edited version (“Movies in the Mind’s Eye” 388). This essentially underlines, on one hand, the abstract and concise form in which schematic events are retained in our minds, instead of having a 1-1 correspondence with the events themselves; and on the other, it demonstrates the limited capacities of human memory to retain details of events that are tracked online, i.e. in real time.

Let us further explore this idea in relation to synthetic worlds in digital blockbusters. It is a fact that most, if not all, mainstream movies, digital and analog, realistic and fantastic, are made with a narrative economy in mind which compresses the fabula within the screening time available. This, obviously, leads to the omission of events that are unnecessary for the promotion of action. But the crucial difference between analog/realist and digital/fantasy movies is the fact that the narrative in the former should normally be expected to be easier to follow and reconstruct mentally, due to the similarity between the cinematic universe they depict, and the real world.

Synthetic worldmaking, on the other hand, obeys laws that the viewer needs to learn and accept as quickly as possible, if he is to properly follow the course of the narrative in a cinematic universe that is presented as real, even if this reality is on an

“as-if” basis. Constructed, synthetic worlds of this kind, are bound to include events, situations, places, etc. that are not screened, but at the same time they are 221

increasingly difficult to imagine or deduced in order to engage in a process of gap- filling, at least to one that is as extremely detailed as Bordwell implies. Therefore, a model such as that of Hochberg and Brooks perfectly explains how it is that synthetic fantasy can function just as well as realist analog filmmaking: put simply, the model does not need to explain how omissions of events from the narrative flow are compensated, as it obliterates the need to proceed to any gap-filling altogether. By doing so, the main problem that it manages to tackle is the fact that many events in fantasy worlds are completely imaginary, often contradicting laws of time, space or physics in general, and therefore spectators are impossible to have had any kind of real-life experience of them prior to watching the movie. The value of Hochberg and

Brooks’ model lies in the fact that it renders redundant any need whatsoever to explain how could it ever be possible for a spectator to rely on experiences inexistent or impossible to have occurred, in order for him to fill in the events omitted from screening time. The fact that the digital blockbuster is established on the basic rules of continuity editing demonstrates and proves what was difficult to prove with realist movies: the fact that several events are not screened does not mean that they are easy to elicit and be filled; it simply means that they are not at all required by spectators, as long as the events are are indeed screened have the necessary internal coherence and connection to each other.

The immediate consequence of such a model of mental representation for narrative that has to be economical, for reasons of limited screening time, while still not requiring any filling-in, is that action is expected to be segmented. And for this mental structure of narrative, which is abstract and schematic, Hochberg and Brooks underline the need for conceiving a mode of modeling it, i.e. a notation system that 222

is able to handle, clarify and make obvious the proper breakpoints in which the narrative is segmented. This is of course no new idea in cinema theory, and more than enough notation systems of this sort have been proposed over the decades, normally having been inspired by linguistics. Psychology did not escape this influence: J.M. Carroll and T.G. Bever, for instance, acknowledge the fact that structural segmentation in movies is a notation system, very similar to the syntactic structure of sentences in that it “organizes viewers’ perceptual processing,” thus enabling them to elicit specific meaning from edited visual sequences; in fact, Carroll and Bever believe that the study of sentence perception can give insight to movie perception as well (1053). Nevertheless, Hochberg and Brooks believe that reliance on linguistic models for movie perception is inadequate, especially in the form that cinema scholars and cognitive scientists alike usually deploy, i.e. in models of narrative notation that use “words and tree diagrams” (“Movies in the Mind’s Eye”

390). Instead, according to the authors, a proper system of visual narrative notation should be visual instead of linguistic, in order to correspond to the way the perception and memory of spectators actually work:

Perhaps it should consist of brief high points or action features

economically sampled from the flow of events; it will be relatively

schematic, since details are not normally maintained unless needed; it will

be mostly ego-centered, or camera-centered, with a definite viewpoint

and 2D composition as distinguished from an object-centered layout

specified in 3D coordinates . . . (“Movies in the Mind’s Eye” 390)

Such a visual model for mental representations is quite similar in conception to the image-based schemata that Lakoff and Johnson delineate. Just like Hochberg, Lakoff 223

and Johnson stress the insufficiency of arbitrary – in the linguistic sense – symbols for conceiving schemata, and adopt image-based ones, despite them being abstract, as they are considered more appropriate due to their immediate connection to the human body.

It is very interesting to read that Hochberg and Brooks find comic book strips to be a very good approximation of what they have in mind about such a notation system, given the characteristics that they assign to it. Hochberg and Brooks state that perhaps the reason comic books are popular is because they are very close to the way in which we actually perceive the visual world (“Movies in the Mind’s Eye”

390); to this we will add that the immense popularity of comic books has been further translated to cinema, as a second-order appreciation of this form. The popularity of movies based on comic books, being not only the definitive but also the most marketed products of digital cinema today, demonstrates in two ways the validity of this model: both on the level of content and on the level of form. First, on the level of content, the fantasy worldmaking that they include as we have seen works without the spectators having to reconstruct those parts of the visual narrative for which they have absolutely no prior experience. A comic book script represents what Hochberg and Brooks call an “annotated ‘shot script,’” perfectly matching a kind of visual representation in which “the knowledgeable viewer can, if necessary, reconstruct and insert additional boxes between the major breakpoints”

[emphasis in the original] (“Movies in the Mind’s Eye” 390). And on the level of form, such movies resemble their printed ancestors, and share with them an analogous popularity, exactly because their narrative structure inherits from comic books this correspondence with the way the human mind segments events. Combining the 224

relatively simple narrative structure of these movies with the increasing rate of segmentation in digital blockbuster in general that we discussed in Chapter 3 makes it easy for us to argue that the rapid segmentation of action events in digital fantasy does not bother spectators because this is exactly the way they perceive visual events anyhow. Nevertheless, the issue raised in section 3.5.3 of Chapter 3 remains in relation to the increasing speed of editing: if the rate of segmentation crosses a specific threshold, the result might be non-intelligible. Due to the relevant novelty of digital technology in cinema, we are most probably now at a period in which contemporary movie making is still exploring the limits of this speed threshold, searching for the ideal duration of each shot. But this period is transitory; this search should be expected to come to a conclusive end fairly soon, with the speed of editing probably stabilizing across the larger part of mainstream moviemaking, at least for action genres which require faster editing, such as digital blockbusters.

5.4.3. To Cut or Not to Cut? The Importance of Breakpoints for Narrative

Comprehension in Action Sequences

Conceiving of a visual notation system appropriate for movie narratives, which includes particles of visual action, means that each particle acquires its importance exactly because of its individuality, i.e. because of its separation from others. So far, we have encountered the following findings about movie structure and comprehension: (a) action is segmented in smaller, more recognizable clusters;

(b) these clusters are placed in a causal order, each providing a visual “answer” to the visual “question” that has been posed by the preceding one; (c) this ordering elicits causal schemata, which are essential for the comprehension of any narrative 225

movie; and finally, (d) the power and effect of narrative schemata are so great that they manage to overcome issues of appearance or setting in a movie, which in practice means that even movies set in unreal, synthetic worlds of fantasy can appear as natural and realistic. Hochberg and Brooks stress the fact that in the

Heider-Simmel film action is “punctuated or segmented,” which is the same as what happens with human actions in real life, in terms of the way the human mind understands them: observed events are broken down to individual actions, connected to each other with “hierarchical, ‘in-order-to’ relationships,” inside encompassing “major plan schemas”141 which provide the general framework that makes this sequence of action meaningful (“Perception of Motion Pictures” 267).142

Spectators recognize events by picking up the breakpoints between them, and use those events to form a mental representation of the narrative; this mental representation is recognizable due to a pre-existent, encompassing abstract plan schema, dictated from real life experience, which acquires its structure primarily from breakpoints, while all the intermediate actions are redundant (268).143 All behaviors trigger actions, actions constitute events, and events are assembled into narratives; and this process, from behaviors to narrative, is hierarchically structured.

141 Hochberg and Brooks use “schemas” as the plural form of schema. The present study on the other hand has used the term “schemata,” following the general tendency of the literature of cognitive studies in cinema; the form “schemas” has been retained only when found inside quotes. It should be noted that in English both these forms for the plural are accepted as correct. 142 This is a reference to E.H. Lichtenstein and W.F. Brewer. In their paper “Memory for Goal Directed Events” the authors present their findings in relation to viewing and retaining goal-oriented and non- goal-oriented actions. They note that in their experiments “the recall of goal-directed actions was superior to that of non-goal-directed actions, supporting the hypothesis that plan schemata are used in the recall of events;” additionally, their experiments showed that “plan schemata” play a crucial role in the way the fabula is put together in the mind of the spectator as “the temporal order of actions in observed events is reconstructed from the underlying Plan schema” (412). 143 It should also be noted here that omission of trivial events is a standard characteristic in the structure of any narrative, as it sustains narrative economy. The preservation of this economy is not something new; in fact, it is known to storytellers for millennia now. The experimental findings cited here simply provide scientific explanation as to why it happens and how it works. 226

Stories are comprehended and remembered because viewers recognize, and even anticipate, the breakpoints between events; since these events are normally patterned on real-life experiences in the majority of narrative movies, synthetic and analog alike, the effect of the cinematic story taking place in a credible place and time is achieved, regardless of the cinematic world around it.

James Cutting offers a somewhat similar approach to the primary function of the cinematic story over the cinematic world, focusing on the way action is segmented. In Chapter 2, we referred to the explanation he provides as to why cuts work for spectators. In the context of narrative and the movie world, Cutting mentions that “continuity means keeping track of what is in each shot and making sure that the world that is projected on the screen appears coherent;” but he also states that spectators are not so much interested in this coherence of the background as their attention is focused on the actions of characters rather than the details of the world around them (“Perceiving Scenes” 14).144 It is obvious that

Cutting essentially states in a bit different way what has been claimed earlier in the present Chapter about coherence. Cutting seems to be attributing coherence only to the visual aspect of the cinematic world, adding that a strict coherence of this sort is not necessary, since proper narrative continuity will provide the necessary credibility to the screened events. What we have argued on the other hand, on similar grounds, is also the fact that continuity narrative, as a feature of the cinematic story, will provide the necessary infrastructure that will hold together every other aspect of the cinematic universe, aided, to a varied degree, by other factors such as e.g. knowledge of genre conventions or previous narratives, attention to detail during

144 Cutting here refers to the experiments conducted by Simons and Levin in relation to change blindness, to which we referred in section 3.5.3 of Chapter 3. 227

CGI design, etc. In other words, when Cutting actually uses the word “coherence” he refers to it as only a visual matter, which is plainly wrong; it is as if he implicitly admits that coherence depends solely, or at least primarily on the visual design or the setting. Obviously, coherence cannot be assigned only to that; it is a cognitive percept, a wider sense of order in a visual narrative that derives from stimulations that are much more encompassing than setting only. On the other hand, he might have chosen poorly the word “coherence;” and this of course does not negate the fact that he is right as regards the importance of action and its proper segmentation: proper cuts will deliver proper continuity, and will enable the triggering of a plan schema in a specific scene.

The above definition of coherence as a general sense of order is established on several factors that address patterns of pre-existing knowledge that the spectator carries with him inside the movie theater. Two of the most important parameters, which severely affect the way movies are segmented into actions edited together in cause-effect relationships, are (i) social pre-disposition, and (ii) familiarity with genre conventions. Hochberg and Brooks directly stress the importance of both of them in narrative continuity:

In film, familiarity with the medium, the genre, and indeed the individual

passage, as well as familiarity with life, should provide the local inferences

routinely needed to bridge cuts and elisions, without reference to the

story structure. But knowing what the viewer expects locally also gives the

filmmaker the opportunity to invite inferences about the story: to the

engaged audience, emphases or violations of expectations that are 228

unexplained by their context but are apparently deliberate probably invite

inference. (“Perception of Motion Pictures” 271)

In other words, segmenting action at the right point triggers cognitive mechanisms, as the spectators respond to sets of real cause-effect conventions that they are familiar with from older movies, as well as from their real-life encounters. Reliance on such genre and social conventions ensure, to a certain extent, that the narrative is properly presented to the audience as a familiar string of events, which is a prerequisite that precedes in importance everything else in the cinematic universe, no matter how bizarre or complex the synthetic world is.

As regards both genre and social conventions, Cutting is also in agreement with Hochberg and Brooks, noting that the moviemaker should “first win over the viewer with the narrative,” and then use this as a basis on which to present the rest of the visual material. In relation to genre, Cutting stresses a fact that is extremely important for synthetic worldmaking: the role of information found in the context of narrative. He believes that contextual information is essential for the comprehension of all editing in narratives; simply bringing segmented shots together is neither expected nor verified to work meaningfully for spectators (“Perceiving Scenes”

15).145 The importance of genre conventions and knowledge of previous narratives in digital cinema is extremely important. Let us not forget the fact that digital cinema is the offspring of post-classical cinema, which, in turn, was a mastery of assembling in

145 Both Cutting and Hochberg & Brooks note this in opposition to Soviet montage and its assumptions. Besides any artistic and aesthetic value that Soviet montage might have, all three authors agree that Eisenstein’s and Kuleshov’s assumptions cannot be verified, because juxtaposed shots in their films are meaningless without context. Cutting attributes this to the spectators being in need of the “big picture,” i.e. a longer narrative to guide them through what they are watching (“Perceiving Scenes” 15), while Hochberg and Brooks stress that the spectator needs to apply specific and precise interpretations of what they see using personal experience of associations, otherwise the movie on its own will be nothing but “a meaningless flight of visual fragments” (“Perception of Motion Pictures” 265). 229

bits and pieces the entire past of older movie narratives, in practically all levels: genre, editing rhythm, visual references, exploitation of older themes with franchises, etc. Digital technology has facilitated this process tremendously, as we have already discussed in Chapter 2, but it works exactly because the liberties taken are primarily on the level of spectacle, while, on the contrary, significantly fewer liberties have been allowed in linear narrative continuity, as well as in the connection of segments together: the rules to follow in order to succeed in moviemaking, digital or analog, fantasy or realism, by now comprise the proper methodology for segmenting action in continuity editing in general. And this brings us from genre to what Cutting believes about the role of spectators’ personal social experience: this methodology, the recipe for success, is not a strict and academic language of cinema, but simply a set of proper manners that the spectator feels naturally accustomed to, much like those from real life (“Perceiving Scenes” 15).146 In other words, the cornerstone of rendering any worldmaking credible, no matter how demanding, unreal, complicated, etc. it may be, is the foundation of its events on patterns that stem from the spectator’s social interactions.

Segmentation therefore is far from an arbitrary process; it is a stimulation to which the spectators are ready to respond actively. A discussion on whether motion on screen is perceived as real or not is way behind us now; events on screen, as has already been discussed in Chapter 4, are not simply motion, and can definitely not be explained solely by physics, i.e. on the grounds of vector analysis: as we have seen,

146 For example, Cutting notes that breaking the 180° axis rule in a conversation is one of the conventions that have slipped in Hollywood-style editing from an analogous convention of manners from real life: spectators are aware that it is rude to stand in between two conversants, so they feel natural with the camera not doing so either. Surprisingly enough, Cutting did not take advantage of this point of his in order to explain and emphasize the difference in the way we experience a real life conversation than a cinematic one, where over-the-shoulder shots are used. 230

motions we perceive also carry meaning manifesting character, intentions, gender, etc. The authors believe that it is very likely that spectators’ perception of on-screen motions is governed by the same rules that apply to real-life motion. Based on this, edited events on screen, which are obviously assembled, meaning-carrying sets of segmented motion, are “driven not by physical kinematics but by biological and social factors” (“Perception of Motion Pictures” 268). The authors report that spectators, while watching clips, demonstrate agreement as regards the breakpoints of action; this large degree of consensus betrays the fact that these breakpoints obey to larger, encompassing “plan schemas,” with all unimportant action being redundant (267–268). On one hand, this redundancy of visual information makes schemata quite economical; and on the other, it makes editing a quite predictable process, capable of yielding proper meaning out of any context, synthetic and realistic alike. In fact, more than that: it makes continuity editing a prerequisite for success, as it is patterned on precisely the way the human mind comprehends events anyhow. The next section reviews more closely the intricate relation between human cognition, memory, and narrative patterns.

5.5. The Mind We Inhabit: the Function of Human Memory as Narrative

5.5.1. Perceiving the World As Narratives and Vice Versa

Hochberg and Brooks argue that the events we encounter in our real-life experience of the world have the same source as most, if not all, the events we encounter in edited movies: they both originate in an experience and comprehension of events which, as noted earlier, originate on biological and social factors rather than simple physics (“Perception of Motion Pictures” 267). This 231

effectively means that the way we perceive events in real life and in movies is like a pair of communicating vessels, constantly affecting each other. As we have seen, the narrative substratum that supports the synthetic visual extravagance in CGI movies of science fiction and fantasy is established on the same classical code of visual communication that has proved its worth ever since the early days of cinema; it has also been shown that remaining faithful to this code of known narrative patterns actually helps the cinematic world look more real. Consequently, apart from being established solely on visual features such as setting, special effects, attention to detail, etc., worldmaking in cinema also relies on principles of cognition that are more fundamental and internal to the human mind than differences of external appearance, and are thus related to the way the mind perceives and connects to the world in general. This connection between human perception and the real world, according to cognitive psychologists, is also made possible with the mind employing narrative patterns. To comprehend this, it needs to be acknowledged that narrative is not simply an element of fiction, but a term that is much wider and more encompassing, in fact “a contingent universal of human experience” (Bordwell,

Poetics of Cinema 86). Applications of cognitive psychology in cinema challenge us to combine the following propositions: (a) the human mind comprehends the world and itself via narrative structures; and (b) it is exactly because of this feature of the mind that narratives enable synthetic worlds to become credible. Thus, it is only reasonable to assume that humans apply to cinematic universes, CGI or real-like, the same innate devices or narrative structures that enable them to comprehend the real world in the first place.147

147 Accepting this proposition is a risky endeavor: it means that we are also acknowledging the 232

Classical narrative is indeed a convention that facilitates comprehension in digital blockbusters as much as in any other type of movie, but it is not the human mind that was trained to follow this convention; rather the opposite is true. The methodology of the classical style reveals the fact that human artistic creations reflect wider principles that govern one’s perception and self-comprehension. Kay

Young and Jeffrey L. Saver describe the way the human mind acquires a sense of self within the world through memories, which it organizes in the form of narratives; this essentially makes life itself being experienced as a long story. Following Aristotle, the authors are concerned with the connection between the “I” and the story, i.e. the way experiencing life and storytelling are interwoven and mutually dependent; the only difference is that fiction is presented to us as narrative, so we consciously perceive it as such, whereas “the storytelling we experience as an event in life can lose its appearance as narrative by virtue of its integration in life” (72). Still, despite our consciousness getting in the way and forcing a “geometrical” way of seeing and thinking, we comprehend ourselves and the world as parts of a long story, which are the constituent elements of our consciousness itself. Following Barthes’ comment about the position of narrative in the domain of culture as well as its importance for the self,148 Young and Saver suggest that, insofar as the concept of narrative is a key

existence of bio-social explanation for the success of classical narrative; even worse, it opens the door to an interpretation of reality that is connected to individual perception, and thus released from any objectivity whatsoever. In effect, despite these risks, such an assumption can actually provide neuroscientific support to long-existing assumptions. 148 “[N]arrative is present in every age, in every place, in every society; it begins with the very history of mankind and there nowhere is nor has been a people without narrative [sic]. All classes, all human groups, have their narratives, enjoyment of which is very often shared by men with different, even opposing, cultural backgrounds . . . Caring nothing for the division between good and bad literature, narrative is international, transhistorical, transcultural: it is simply there, like life itself.” (Barthes qtd. in Young and Saver 73). Although Barthes’ analysis, taken from his Introduction to the Structural Analysis of Narratives, is from a structural point of view, which should normally be expected to clash 233

feature in human development, it is also “an inescapable constituent of culture” as a whole (73).

The importance of narrative and storytelling in relation to human cognition is also supported in the work of Roger Schank, who asserts that “storytelling and understanding are functionally the same thing” (qtd. in Young and Saver 73).

According to Schank, the past is recreated mentally in the form of stories; by narrating our experiences we give them a structure which is necessary for their retention in memory: in Schank’s words, “[w]e need to tell someone else a story that describes our experience because the process of creating the story also creates the memory structure that will contain the gist of the story for the rest of our lives" (qtd. in Young and Saver 74). Even though the actual words that we use will not be retained, re-telling the experience as a story enables us to distill from that experience its gist, its meaning or significance in our lives, a “cognitive space or an environment for thought” as Young and Saver call it. Most importantly, it also enables us to retain this gist in our memory reserves as something that defines us; when we encounter a situation that resembles the original experience, the effect of that experience which is stored in our mind guides us to function appropriately (74).

Therefore, for Schank, “[w]hat enables the making of memories is the telling of stories;” and, since memories are a vital part of what constitutes the human consciousness, one’s sense of self is actually made up of stories (74), which is a metaphor as much as it is actuality. As exemplified by children learning to recollect their past gradually while growing up, acquiring language, and interacting with their parents, narrative is used here to define human experience itself as a series of with cognitive theorizing, it is still enlightening as regards the status of narrative for human mind and culture. 234

memories, elicited in the mind upon request; since we organize our survival skills based on these recollections of things that we have learned and lived, “to be without one's stories is to be without knowledge of one's life” (74).

Interestingly enough, the function of memory is similar to the usage of the word in Information Technology. Memory is, to the human mind, what it is to a computer: a stored reservoir of data that is elicited on demand in order to organize the input of material into usable information. Similarly, for the mind, memory is the stock of experiences and schemata that organizes perceptual stimuli into a unified world, thus enabling a person’s self-realization within a cultural, spatial and historical continuum. Memory sets in motion the almighty causal schema as well as other schemata, empowering a person to function in his everyday environment based on applications of schematic maps: remembering what happened the last time one tried something, will enable him to learn from that memory and make adjustments the next time; and this is essentially the same idea behind moviemaking:

Something like this idea is found in many traditional cognitive theories,

which take the testing of mental structures as central to the perceptual

process. To some filmmakers, good editing poses first a visual question

and then a visual answer. (Hochberg and Brooks, “Movies in the Mind’s

Eye” 387)

Therefore, in movie-watching, memory activates the cause-effect schema as an easily recognizable cultural patterning, to which the spectator is pre-programmed, can hold on, and thus use as a point of reference, while watching a narrative movie – any narrative movie, for that matter. Through this viewpoint, narrative and memory are two powerful agents that provide the much desired coherence in the real world; 235

and by doing so, they also enable the same coherence in a fictional universe, the rules of which can be mastered in the same way, no matter how weird or unreal this universe may look like. In such a model, narrative emerges as an essential organizational agent in the human cognition that provides (a) coherence to the real world, via the elementary schema of causality; (b) coherence to fictional universes, via verisimilitude to those real-world schematic patterns of causality; and (c) the

Swiss-knife of skills that the spectator needs to adapt and situate oneself in any of these environments. A CGI movie, therefore, with its increased easiness in appropriating and fusing previous genres, narratives, settings, etc., takes advantage of the library of cultural patterns that are already innate in the human mind. In the following section we will see in more detail how memory establishes narrative structure in the mind as self-realization in real life, thus aiding spectators to follow narrative patterns in visual fiction as well.

5.5.2. Applying Theory of Memory to Comprehending Movie Structure

Memory is a complicated and definitely not a unified concept in cognitive psychology. In fact, cognitive psychologists have come up with various models for dividing it into categories; and on top fop that, the fact that some of these categories overlap make it even more difficult to translate any of these models to a solid theory of cinema comprehension. Still, memory is far too important a term to be overlooked here, due to its strong connection to both narrative and cognition.

Katherine Nelson, for instance, presents findings on the interdependence between narrative and memory, stressing that narrative form significantly facilitates the 236

retaining of memories by the human mind (10).149 Nelson utilizes various different concepts of memory from cognitive psychology, in order to provide grounds for her argument.150 The types of memory discussed in Nelson’s paper are (i) episodic, (ii) semantic, (iii) generic, and (iv) autobiographical memory. The way each type of memory connects to the other reveals how the person connects to the real world, and, ergo, can be used to reveal how he connects to synthetic universes in cinema narratives. We will explain them in turn.

The first two types, episodic and semantic memory, were originally described with their differences defined by Endel Tulving. In Tulving’s account, episodic memory is that which retains unique and spatiotemporally specific events from a person’s past (385–386). Therefore, episodic memory is the collection and storing of data regarding specific events that took place only once, in clearly marked time and place. On the contrary, semantic memory is a “mental thesaurus,” i.e. a person’s set of knowledge of symbols, particularly verbal, as well the relations between them and their referents, the code and rules of managing these symbols and associating them with each other, etc.; all these attributes make semantic memory necessary for the use of language (385–386). Under this scope, semantic memory is the one which refers to one’s factual knowledge of the world, which is general, non-specific, and not linked to individual events; this means that semantic memories are acquired at

149 Nelson refers to studies comparing styles of mother talk. “Pragmatic” mothers, when speaking to their children aged 18 months – 2 years, referred to past events in a more instrumental fashion (“where did you put your mittens?”); “elaborative” mothers, on the other hand, referred to them in the form of linearly ordered episodes, i.e. in the form of storytelling. The study revealed that children in the latter case retained significantly more memories from the events their mothers referred to. 150 Some of the terms in Nelson’s account were originally used by Endel Tulving as early as 1972; yet, according to Nelson, they generally apply to Larry Squire’s declarative memory system. Analysis of the latter exceeds the purposes of this study; for more information, see Squire et al, “The Structure and Organization of Memory.” 237

some point in one’s life through an event he experiences, but from that point on that specific event is not anymore attached to those memories (385–389).151

The third type, generic memory, is also discussed by Nelson on the basis of

Tulving’s model.152 Generic memories are very significant in the context of the present study. They provide general outlines of schematic events, upon which specific events are fitted: they are “derived from experience that sketches the general outline of a familiar event without providing details of the specific time or place when such an event happened, whether once or many times” (7). In this respect, generic memories are more abstract than episodic, and more specific than semantic ones. Specifically in relation to episodic ones, generic memories do not involve familiar events, but schematic frameworks for strings of actions, which are simply functional, repetitive, and with no life-defining significance for the person’s self; in plain words, what distinguishes episodic from generic memory is that episodic memory provides “the sense that ‘something happened one time’ in contrast to the generic ‘things happen this way’” (Nelson 7). It is obvious that generic memory refers to remembering events without retaining any specific spatiotemporal

151 Tulving provides a couple of examples for each kind of memory: “the following memory claims are based on mnemonic information stored in episodic memory: (a) I remember seeing a flash of light a short while ago, followed by a loud sound a few seconds later; (b) Last year, while on my summer vacation, I met a retired sea captain who knew more jokes than any other person I have ever met; (c) I remember that I have an appointment with a student at 9:30 tomorrow morning . . . Each of these statements refers to a personal experience that is remembered in its temporal-spatial relation to other such experiences. Now, consider some illustrations of the nature of information handled by the semantic memory system: (a) I remember that the chemical formula for common table salt is NaCl; (b) I know that summers are usually quite hot in Katmandu . . . Although some of these statements refer to the speaker’s ‘knowledge’ rather than his ‘remembering,’ all of them can be regarded as memory statements in that their content clearly depends upon information entered into the semantic memory at some earlier time. Unlike episodic memory claims, these statements do not refer to personally experienced unique episodes. Rather, content words in these statements represent linguistic translations of information retrieved about general concepts and their interrelations” (386– 387). 152 Nelson states that her argument draws on Tulving’s model, but she mentions that Tulving does not explicitly refer to generic memory. 238

details whatsoever, and therefore having no specific importance for the individual on its own, which is what distinguishes it from episodic memory. But it is also obvious that, without the schematic blueprint that generic memories provide, episodic memories cannot be sustained as meaningful strings of events.

Finally, autobiographical memory refers to remembrance of events which have self-defining importance for a person’s life-story; it could be seen as an interweaving of episodic and generic memories (Young and Saver 74), a kind of event-plus-background-plus-significance. Each of the previous three kinds of memory, i.e. episodic, semantic and generic, connects to autobiographical memory in a varying degree. First of all, episodic memories highly contribute to a person’s self-sustained autobiographical memory, which comes to comprise the set of events that are stored and related to his “personal past” and “personal identity,” and can be retrieved and narrated by the person in a linear order (Tulving 389). Second, as regards semantic memory, this consists not of events but of abstract notions and their relations, “objects — general and specific, living and dead, past and present, simple and complex — concepts, relations, quantities, events, facts, propositions, and so on, detached from autobiographical reference” [emphasis added] (388). Thus the relation between semantic memory and autobiographical memory is more loose and indirect.153 And third, generic memory is the layer on which all memories are laid before they become autobiographic ones: the pool of generic memories provides general schemata for strings of events, which effectively comes to constitute a

153 In fact, Tulving notes semantic memory operates within episodic; in his own words, “[i]f a person possesses some semantic memory information, he obviously must have learned it, either directly or indirectly, at an earlier time, but he need not possess any mnemonic information about the episode of such learning” (389) 239

network of possible relations between actions, out of which an event becomes memorable, i.e. episodic, and, eventually, important in one’s life.

Let us put this information to good use. As we discussed earlier, comprehension of a synthetic universe requires that the movie exhibits a certain level of coherence; this coherence is maintained by the spectator’s mind constantly negotiating familiar event patterns with the use of schemata, regardless of whether the surface form, the world in which these events take place, is realistic or not in comparison to the experience of real life. These schemata are established on narrative patterns which form the infrastructure of the cinematic universe, and are the starting point for accepting the surface as a visually credible world. We will need to explore whether the structure of memory, as described earlier, is indeed reflected in the way we make movies, a fact which will further strengthen the arguments of the previous section regarding the narrative construction of our own personal reality, and the fact that we tend to project this narrative construction of our worldview in our works of art as well.

First of all, we will discuss the function of episodic and autobiographical memory in the context of movie-watching. As we have seen, blockbusters following classical narrative patterns generally follow a mechanism that also requires fitting events in linear order so that they become meaningful; movies rely on causal and spatiotemporal relations between events to maintain their coherence. This fact directly reflects the relation between episodic and autobiographical memory: just like episodic memory evolves into autobiographical memory by forcing shattered actions to form a coherent and memorable sequence of important events, the movie facilitates the retention of the plot by presenting them in a causal and linear order. 240

By doing so, it actually simulates what the mind does every day in its effort to form, store and retain events that are worth remembering.

Second, movie-watching also has analogies with the function of generic memory. We mentioned earlier that generic memory provides the general blueprint on which episodic memories are fitted, before they acquire a more specific autobiographical significance. Narrative formulas in movies are similar in that they are also event patterns relieved of any spatiotemporal specificity, upon which the plot of a movie is fitted. In other words, narrative formulas resemble generic memory in that they also provide conceptions of events solid enough to form the necessary library of event patterns that the viewer will recognize and follow in a movie, but also abstract enough so that they can be fitted in any movie; their specific materialization in one movie does not interfere with that in another. Movie coherence, as we have seen, depends on recognizable narrative patterns that support any realistic or fantasy surface; in that it resembles the readily available patterns of generic memory, as they both have the usability of a functional infrastructure for surface events. The mass of generic memories are mostly functional events that form the backdrop against which a few of them will acquire autobiographical significance; similarly, minor or merely functional events in movies provide the backdrop that sustains the importance of the main complications in the plot. With the net of functional events promoting complication, all events become causally meaningful to each other and, subsequently, make the synthetic universe credible. By doing so, they make the CGI surface form almost as easy to accept and follow as any narrative movie that uses real-life setting. 241

Finally, semantic memory will weave together individual elements from that surface form: the semantic referents of objects, as well as aesthetic, structural, conceptual items etc., and the dynamic relations between them will be composed on the fly on the basis of previous factual knowledge combined with new semantic relations derived from the movie events. For example, the concepts of “space” and

“travel” are previous semantic knowledge; but the concepts of “warp drive” and

“hyper-space” are combinations of that previous knowledge with events inside the movie that support them (e.g. the ability of space ships travelling many times faster than light) as well as CGI visual aid (e.g. the stretching of star trails when ships travel in such speed).154 Eventually, the workings of memory reveal that synthetic movie image not only works for human cognition as any realistic motion picture narrative does, but, most importantly, that both of them obey rules that are interwoven with the mode and form in which humans perceive, evaluate and store events and their patterns.

A rational skeptic will immediately ask: if the processing of input from movie- watching reflects the same mental mechanisms of episodic, semantic, and generic memory that enable a person to function in his real-life environment, why do spectators do not physically respond to movie scenes to a point of taking action as they would do in real life?155 It is true that spectators normally do not demonstrate any motor responses during movie-watching, like for example the usually instinctual reaction to objects moving towards them or to point-of-view shots of, say, falling off

154 See Appendix II, pictures 28-29. 155 It is true that spectators indeed manifest emotional and mental reactions to cinematic narratives, but it is quite uncommon for them to show any physical reaction, perhaps with the limited exception of tears, laughing, jerking when suddenly terrified, heart palpitations, which are generally involuntary and do not entail complete bodily movement as it would happen when actually witnessing a situation. 242

a rooftop. Today we generally accept that, despite the great degree of verisimilitude and faithfulness in realistic representations in real-life narratives, deep down spectators do not to respond by taking any action because they remain aware that these narratives are complete make-believe. But how does this knowledge work? An explanation to this phenomenon can be found in Young and Saver; the authors refer to the “functionally advantageous” human mental capacity of decoupling, which is defined as a “separation of mental action from physical action” (82). Decoupling has been developed inside human cognition through evolution, and enables one to

“explore different response options to imagined mental scenarios without engaging the motor apparatus and actually enacting each envisioned behavior” (82). As the authors note, our ability to imagine literature and the arts could indeed stem from this ability to imagine narratives which may be based on our experience, but remain separate from it. Due to the function of decoupling, accepting a synthetic cinematic universe as credible and behaving as if it were real are two functions that remain detached from each other, at least in the majority of movies at this time.156

5.5.3. Memory in the Context of Story vs. Form

One might think that, unlike movies with a real-life aesthetics and theme, digital movies with extensive CGI universe construction do not divide the burdens of worldmaking equally between narrative and visual elements. In other words, a reasonable thought would be that narrative may be very important in credible

156 This situation might change with the expansion and constant improvement of 3D technologies in cinema. Such technologies are designed to change the way spectators interact with the content of movies; this way, decoupling may soon be revised for cinematic fiction. That being said, it is highly unlikely that spectators will start reacting physically to motion pictures because they will perceive them as real; 3D technology will most likely increase their instinctual or reflexive physical reactions. 243

worldmaking, but the job of the moviemaker, give or take, is merely to select and order events without messing up the syuzhet; since plot complications will allude to the spectators’ mental library of events, autobiographic memory will do the rest for connecting them, just like it happens with real-life memories. A credible and coherent synthetic universe is, for moviemakers and audiences respectively, as much a creative as it is a cognitive challenge; in this context, narrative actually proves to be more important than form. The reason is that, according to Tulving, semantic memory “has two sources, perception and thought” (388) which means that it comes after the experience of an event, despite the fact that we do not always retain that event.157 After the input of events, comes their mental processing, from which eventually semantic meanings between individual elements will be distilled and will form background semantic knowledge. But no matter how ingenious the imagination of the creative team of a movie production is, unreal CGI cinematic universes are perceived as credible full-scale worlds owing primarily to a plausible storyline; and the measure of plausibility is essentially dictated by the function of autobiographical and generic memories. The latter will enable spectators to adapt to a universe which may be unfamiliar in form, but still exhibits the same causal and spatiotemporal rules of storytelling as those governing the perception of real life and existence, i.e. via memory structure. The surface form eventually adapts to the underlying story (and not vice versa), provided that the latter is successful in its

157 This is an important issue for psychologists studying infants up to 2 years of age. The retaining of memories during that period, and consequently remembering them, is an extremely seldom phenomenon; yet, the formation of language is established on semantic relations, which need to be somehow derived from observation, input and processing of life experiences, even of those experiences which will not be remembered afterwards by the child. This reference is for informational purposes only and exceeds the scopes of the present study; for further information and proposed solutions, see Newcombe et al, “Development of Episodic and Autobiographical Memory: a Cognitive Neuroscience Perspective.” 244

structural connection of events. Thus, in order for a self-sustained cinematic universe to emerge in science fiction and fantasy worlds, the existing background of autobiographical and generic memories from real life and previous narratives is enough; it eliminates the need for the spectator to be familiar in advance with the entire range of semantic features like the places, characters, and concepts of the cinematic world. Those will be formed and will find their place as composing elements of this universe on their own, once the spectator’s cognitive function is absorbed by the neatness and logical structure of narrative. Of course, if those semantic features are familiar from prequels, as in the case of the Star Wars saga, so much for the better.

An unsuccessful case of narrative serving form is Roger Christian’s Battlefield

Earth: A Saga of the Year 3000 (2000). For all its all-star cast (John Travolta, Forest

Whittaker, Barry Pepper) and quite successful integration of digital graphics in real settings (via compositing), the movie was a complete disaster in the box office exactly because it had faulty constructions of both the cinematic story and the cinematic world. An example of the faulty cinematic story is the movie showing humans, turned into completely uneducated primitive slaves one thousand years ago by the alien race of the Psychlos, but nevertheless learning to arm nuclear bombs and fly Harrier fighter jets literally within a day, and, in fact, using them to defeat the Psychlos. Similarly, an example of the faulty cinematic world is the image of the long-destroyed earth given in the movie, in which jets still have fuel and fly, batteries are still charged, books haven’t decayed in the open air, etc. The movie relies too much on the comparison between the current and the future dystopian image of Earth, but does not attend to the details that this comparison will inevitably 245

call attention to, especially the infrastructure, i.e. the cinematic story: as the latter is not at all plausible, it drags down with it the cinematic world as well. Therefore, despite the fact that it is science fiction and thus allows for something more than a grain of salt in the story, the sense of dystopia never worked successfully, making the movie hardly convincing and really difficult for the spectator to be immersed into a credible cinematic universe.158

Each time a digital blockbuster uses a theme for the first time, two things occur in relation to storing and accessing new data: first, as we have seen earlier in this chapter, a set of event patterns will enable the viewers to make sense of the narrative world in the movie using their mental libraries, acquired from both personal experience and knowledge of older narratives. And second, the movie, at least a successful one, will establish a stylistic background, i.e. an aesthetic norm for its genre. For example, The Matrix (1999), Star Wars (1977), Star Trek (1979), The

Fellowship of the Ring (2001), and Sin City (2005), all set visual and aesthetic standards that have been re-used in subsequent movie productions, not necessarily of the same franchises as them. When a narrative movie introduces a new CGI world for the first time, we could assume that the functions related to autobiographical and generic memories are indeed more important than those related to semantic memory, at least initially, as they will have to trigger the operation of familiar or stock narrative formulas; only if they succeed, they will subsequently support the external verisimilitude of the digital universe. The aesthetic part of this universe along with background knowledge about it, the “cinematic world” as we have called them combined, will be acquired gradually in the process of watching the movie, and

158 See DVD-ROM, video file 27 for the trailer of the movie. 246

will be stored in semantic memory. When a movie is a success, those aesthetic themes will tend to be re-used, enabling semantic memory to facilitate reception of the subsequent movie with less risk of audiences rejecting its verisimilitude.

Therefore it could be proposed here that, in digital blockbusters, the operations that support autobiographical and generic memories are important for the success of a new movie as well, whereas semantic memory is important for cashing in that success, probably – but not necessarily – with the launching of a franchise.159

5.5.4. Weaving Worlds: the Role of Memory in Narrative Worldmaking

An argument that challenges the points raised so far would be that autobiographical, generic and semantic memory are generally a personal stock of each individual mind. Such a claim consequently makes movie perception also a personal matter, thus posing challenges to any claims for a universal application of the entire argument. Since each spectator uses his own reserve of memories, how can we claim that movies will be perceived in the same way by everyone? The answer to this issue has a social explanation. In section 2.2 of Chapter 2 it was stressed, following Bolter and Grusin, that the tradition of perspective, which ultimately created the entire tradition of cinema, analog and digital alike, is culturally specific and should not be expected to apply to all societies, places, and cultures; but it certainly does not affect only individual people. The human mind is developed within the social framework in which it operates; thus, claims about image

159 For example, Star Wars, Star Trek and The Lord of the Rings have been very successful franchises (the first two by expanding their universes into new movies and parallel stories, and the third, apart from the trilogy, with the release of The Hobbit: an Unexpected Journey (2012), The Hobbit: the Desolation of Smaug (2013) and The Hobbit: the Battle of Five Armies (2014). One sequel to Sin City has already been released in 2014, but the same kind of impressive digital aesthetics were very quickly used in 300 as early as in the following year (2006). See DVD-ROM, video files 28 and 29. 247

comprehension should be expected to apply to entire societies as much as they do to individuals and vice versa. Memory and cognition are developed via interaction with the environment, which ultimately fine-tunes the mind so that it conforms to its norms. What the present study claims about the role and importance of narrative patterns may indeed not be applicable to all people and all cultures in the world, but it certainly applies to entire societies sharing the same line of cultural tradition. The

Western conception of realism, as it has been laid out in Chapter 2, is a good example of a widely shared culture of this sort.

In relation to this view, Jerome Bruner stresses the importance of certain psychological theories on the relation between man and his culture. According to

Bruner, recent theories of this kind have refuted Jean Piaget, whose account treated mental development as a unilinear process of several stages that is universal and common for all people: for Piaget, mastery of a certain developmental stage is necessary before moving to the next (Bruner 1–2). On the contrary, these more recent theories which challenge Piaget’s approach see this gradual growth of knowledge as clustered, so to speak; unlike Piaget’s conception, these clusters are independent sets of knowledge with no linear relation to each other. They are developed independently and for specific purposes each; therefore, according to this view, completion of a specific cluster or set of skills that provide a certain kind of knowledge, does not imply either the prior or the subsequent completion of others

(Bruner 2). These domains of knowledge constitute, in a sense, a culture’s “treasury of tool kits” passed from one generation to another (2). The importance of such a view, Bruner adds, is that it places man into the evolutionary tradition of the culture he lives in, and, by extension, in the same collective traditional matrix as his 248

ancestors. But on the other hand, it defines knowledge as necessarily dependent on specific viewpoints; this way, it also questions the “universality of knowledge from one domain to the other,” and, most importantly, the “universal translatability” of knowledge across cultures (3). For this reason, as Bolter and Grusin note, comprehension of cultural products is by necessity culturally bound. By extension, this theory is also compatible with the view that one’s representation of reality is necessarily mediated by the cultural products one is exposed to, like “language and other symbolic systems” (Bruner 3).160

Now let us connect these points to memory and narrative. From summarizing what has been discussed so far, the following propositions arise: (1) the environment affects one’s representation of reality and triggers the development of a kind of knowledge which is personal as well as communal; and (2) memory is the product of this interaction between the mind and its environment, and it takes the form of narrative. Therefore, based on the above, it should be expected that memory has extremely close affinities with this representation of reality, shared by individuals and the society they live in; and, most importantly, that this representation becomes highly recognizable and conspicuous in narrative creations within that society.

Narrative thus makes worldmaking very easy both to comprehend and to communicate to others, since it matches our cognitive mechanisms, which we share with other members of our society.

160 This idea was originally developed by Vygotsky, and in one of its latest versions by John Seely Brown and Alan Collins it is called “distributed intelligence.” According to this, an individual’s working intelligence never works alone; instead, it is highly affected by one’s “reference books, notes, computer programs and databases, or most important of all, the network of friends, colleagues, or mentors” (Bruner 3). 249

Bruner reflects on the fact that most studies regarding the way the human mind comes to know reality revolve around the physical world, while very few of them are concerned with the comprehension of the social world. With observation being the primary tool for the former, he proposes narrative as the tool for the latter. He describes narrative as the instrument by which humans actually organize their existence and memory of past events, in forms such as “stories, excuses, myths, reasons for doing and not doing,” etc. (4). Therefore, for Bruner, since narrative places all thoughts and experiences of the individual in a certain order, it also constitutes the device by which the human mind delineates the very existence of the self, both in personal and social terms. The difference between accessing physical reality via observation vs. social reality via narrative is the fact that narrative is not governed by scientific laws of correctness or falsification, but by degrees of verisimilitude; this means that narratives are “a version of reality whose acceptability is governed by convention and ‘narrative necessity,’” rather than true- false assertions (4). Bruner notes that it is not only difficult, but probably unimportant to try to separate narrative as a mental process from narrative as the discourse that gives it form: just as our experience of the natural world tends to imitate the categories of familiar science, so our experience of human affairs comes to take the form of the narratives we use in telling about them (5).

These possibilities of narrative came into prominence when cognitive psychologists realized its potentials not merely as a tool for representing reality, but of actually constructing it mentally. Young and Saver agree with Bruner in that he sees narrative as an instrument that the mind employs in its interaction with the world, through which our experience of life and the very notion of reality itself 250

become meaningful notions (Young and Saver 75). This account of narrative as a worldmaking device follows closely the one originally delineated by Bruner; Young and Saver agree with him that narrative severely affects the way humans experience the environment. But their scope is to provide the necessary neurobiologic substantiation of this account, which is missing from Bruner’s account (75). The authors refer to studies in cognitive neuroscience which show that narrative is mediated in the human nervous system by a “regionally distributed neural network,” which allocates specific components of the formulation of narratives to corresponding areas in the human brain;161 the consequence of damages to these areas is cases of dysnarrativia, i.e. states of “narrative impairment” (75–78).162 It is precisely due the findings on patients with any form of dysnarrativia that narrative is considered here to be “the fundamental mode of organizing human experience”

(78). Narrative and storytelling structures provide patterns which enable humans to organize past, present and future based on observation. As Young and Saver assert,

“[c]onsciousness needs a narrative structure to create a sense of self based on the features of storytelling, like coherence, consequence, consecution” (79). And most importantly, according to S. Zola-Morgan, this process of “framing and recall of experience is a dynamic, variable and vulnerable process,” in the sense that it may

161 More specifically, the main parts of this system are: 1) the amygdalo-hippocampal system, where episodic and autobiographic memories are initially arranged; 2) the left peri-Sylvian region where language is formulated; and 3) the frontal cortices and their subcortical connections, where individual entities and events are organized into real and fictional (imagined) temporal narrative frames (Young and Saver 75). A more detailed analysis of this system and its functions has been deemed to be beyond the scopes of the present study. 162 Dysnarrativia is caused by damages to parts of the brain that belong to this neural network: under the effect of this disorder, the symptoms of patients that Young and Saver recount range from inability to recollect and narrate incidents in their lives, to fabricating recollections that actually never took place but which they experience as real memories, inability to make choices or predictions based on similar past events, and many more. The observed tight connection between “narrativity and personhood” leads to the conclusion drawn from these studies that people who have lost the ability to form and handle narratives “have lost their selves” (75-78). 251

change over time, susceptible to the emotions, cognition and social parameters at the time the memory is recalled (Young and Saver 79). In terms of movie-watching, this observation is crucial, as fantasy and SF movies heavily depend on their story in order for the worldmaking in them to function properly. Movies work the way they do because of the fact that narrative structure is hard-wired in our sense of existence: “to desire narrative reflects a fundamental desire for life and self that finds its source in our neurologic make-up” (80). 252

CHAPTER 6

CLOSING REMARKS

6.1. Expanding the Present Study

The study has focused on mainstream digital blockbusters because they were deemed the most appropriate for elaborating in the theory used, plus the fact that they are the most productive area of application of CGI today. But it needs to be acknowledged that the study has barely scratched the surface of the topic; there are a number of other themes, beyond those explored in the chapters of the present study, that are pertinent in several ways to the main thesis, and yet they had to be left out. For example, a major area in which the present study could further expand on is the comparison between mainstream cinema on one hand, and the more alternative productions on the other, based on the ways each utilizes digital technology aesthetically or thematically.

CGI application seems to be moving mainly in two directions. On one hand, there is mainstream blockbuster cinema, grossly dominated by the Hollywood entertainment industry.163 The American cinema production companies have proven extremely keen in creating visually demanding movies, most of which would have been either extremely expensive or completely impossible with the use of traditional construction and special effects techniques. Recent examples of such movies are The

Matrix trilogy (1999 for the first one, 2003 for the two sequels), the Lord of the Rings trilogy (2001, 2002 and 2003), the Hobbit trilogy (2012, 2013 and 2014), the prequel

163 This is true despite the gradual emergence of digital cinematography from Europe and Asia. Some noteworthy examples of digital movies from countries outside the US are the Fifth Element (France, 1997), Immortel (France, Italy and UK, 2004), Nochnoy Dozor (a.k.a Night Watch; Russia, 2005), etc. 253

trilogy of the Star Wars saga (1999, 2002 and 2005), graphic novel adaptations like

Sin City (2005) and 300 (2006), as well as their sequels, Sin City 2: a Dame to Kill For

(2014) and 300: Rise of an Empire (2014) Beowulf (2007), Avatar (2009), etc.164 On the opposite side of such mainstream movies are those that use digital technology to explore the potentials of the digital image in general in a more artistic, as some would call it, or creative manner, often having aesthetic results that, in a sense, follow the tradition of auteur-style cinematography. Auteur works are often characterized by defiance of realistic narrative or visual norms, and a more introvert overall style that is not addressed to multiplex audiences. Instances of such movies are e.g. Peter Greenaway’s Prospero’s Books (1991), The Pillow Book (1997), and the

Tulse Luper Suitcases (2003 for the first part, 2004 for the other two parts), Darren

Aronofsky’s The Fountain (2006), Wong Kar-Wai’s 2046 (2004), etc. These two broad categories of digital cinema, i.e. mainstream blockbusters and contemporary auteur works, exhibit very different applications of cutting-edge visual technology, and address very different target groups; nevertheless, they give a quite clear picture of the two directions that cinema is taking, i.e. either as a pop/commercial enterprise for the masses or as an artistic creation for the few, which may not be defined by the numbers like e.g. opening day revenue, cost of visual effects or artists’ wages, but still strives for impressiveness albeit a kind that operates on a different level.

The field of research towards this direction is indeed quite promising in approaching what human cognition comprehends as real or realistic. This latter

164 It is probably not accidental that many of the movies mentioned here are trilogies. With the exception of The Lord of the Rings, which was a trilogy book anyway, most of the other movies were subsequently expanded into sequels because of their huge success, caused by the impressive cinematic experience they provided. Even in the case of 300, which was probably thought to be impossible to have a sequel, as it is based on a story with a finite historical closure (the Battle at Thermopylae), producers have cashed in its success in a far inferior sequel. 254

category, “digital auteurism,” understands cinematic realism as presentation of levels or aspects of reality that are otherwise expected to be inaccessible by normal human perception, therefore allowing the auteur moviemaker to grant them any form that he feels is appropriate to his aesthetic aim. Digital technology expands the possibilities of such movies, by allowing visionary creators to easily and accurately create often impressively conceived artistic mise-en-scène. Obviously this type of realism, which deals with the kind of reality that exists under the surface of things, serves an approach to cinematic realism that is quite different than the one used in continuity narratives, and ergo would not be expected to make any similar “realistic” sense. That is, instead of using digital technology to replicate a flow of images that conforms to the way the human eye is used to seeing reality, it utilizes visual tools to enable an enhanced perception of the available images, one that reveals the viewpoint of the moviemaker and the messages that it carries. By often bypassing the normal structure of the visual world, these movies force the spectator’s cognition to conform to the director’s own vision of realism; instead of being pampered into an easily digested narrative flow, the spectator’s mind is trained to interact with a quantitatively and qualitatively extravagant pool of cinematic information. A cognitivist endeavor to answer these issues should be a real treat for the researcher.

Another major issue that has been left out of the discussion of digital cinema is that of interactivity. Digital graphics has been an area where interactivity has thrived over the past two decades, so it is perfectly natural for one to wonder about the reason it has been left out of the present study. The answer to this lies in the question itself: interactivity is far too wide and complicated a topic to be squeezed in 255

the span of a single chapter, and any more than that would create imbalance to the distribution of the themes in the study. It is true that technological breakthroughs, which gave birth to digital graphic in the first place, aim among other things at

“extending or expanding our information collecting systems; that is, our senses”

(Malina 36). This means that not only is interactivity already embedded in technology in the first place, but that cognitive science is the definitive discipline for studying it. Digital graphics are at the core of all contemporary applications of interactivity, of which 3D movies are but a small fragment, a fact which further justifies the need for further research on their connection. On one hand, interactivity could easily be divided into physical and mental, which dramatically increases the complication of analyzing it, and ergo its length of analysis; and on the other, it applies to practically all levels of visual technologies, from simple software interfaces to industrial design, from popular entertainment to military training, and from everyday apparatuses like Google Glass to advanced bionic technology.

It should be obvious therefore that taking up this theme would inescapably threaten to drift the present study away from its main area, which is digital narrative cinema. Even if the research had only expanded to the gaming industry, as it is not merely the most adjacent one to that of cinema but also practically fused with it by now, still the expansion would have been significant, and most probably a chapter’s length would not do justice to its potentials for theoretical depth. After all, interactivity is hard-wired to the entire Western conception of realism itself: the technology of perspective placed the eye of the receiver in a key position in relation to the work of art, in order for its effect to work. The consequence of this, as many theorists argue, was the entire Renaissance conception of man as the center of the 256

universe (Malina 34). As presented in Chapter 2, it was the same visual culture that culminated in CGI, set the foundations for 3D design, and, ultimately, passed down to it a similarly central position of the individual, who now interacts with the work of art and has thus been elevated from the status of observer to that of user. Indeed, it already sounds like quite a field for deploying cognitive science in all its potentials.

6.2. On Umbrellas and Rainy Days

To answer, then, the very first question that opened the present study: what has happened to cinema? Well, this is what happened: it became attuned to the needs of contemporary culture, capturing attention and imagination much in the same way as the first attempts to record and project motion on a screen in the last decades of the Nineteenth century. Once again, cinema mustered all the available technology and does what it has always done best: it is still the most inclusive, expressive, multi-faceted, wondrous and flexible means of entertainment that art and culture has ever created, the converging point for all the histories of visual and other arts and technologies; in fact the theater of the human mind itself. And as such, it remained stubbornly intangible, puzzling and controversial, and probably the most theoretically abused art form in history. In the latest chapter of this long course, digital technology has merely been a medium rather than an agent. It has done nothing more than reveal at the same time all the merits and flaws that cinema had been accumulating over the decades, some of which it inherited from its ancestors over the centuries.

Cinema defies time, because, undoubtedly, it is a highly versatile art. It has the unique capacity of being adaptable to practically any kind of audience, 257

circumstance and purpose. As it was described in Chapter 2, it is an art that was put together from the collaboration of different art forms, and, as such, shares both the merits and shortcomings of those. It took an entire century of movie productions until cinema managed to take a form that best expresses this multitude of techniques and trends. In other words, each form that cinema took since its creation was, in a sense, another experimentation of what cinema can or should be, according to the proponents of that form. Digitization could be seen as the definitive form of cinema, because for the first time all those forms are united in one single product. It is as if the most usable part of each trend was distilled and utilized in a

“master form” of cinema, one that combines the best features accumulated from an entire century of movie creation.

The consequence for this versatility is, as we have seen, the defiance that cinema shows for theory. Each moviemaking practice and trend emerged from, or accompanied, a theoretical framework that had a specific set of assumptions to support. In this respect, the more distinctly recognizable cinema of the past, like e.g.

Russian avant-garde, Italian neo-realism, Cinéma vérité, etc. was also a more conscious one; it had a solid ground on which it was established, and a certain goal to fulfill. This is probably why one may, to the present day, claim to be a fan of, say,

German Expressionist cinema, but it does indeed sound odd for one to claim that he is a fan of digital cinema. It is as if CGI has stripped cinema of the seriousness or weight that it used to carry back in the day when it was the cultural reflection of a social, political, or other statement. By combining only selectively the best of what each trend had to offer, digital cinema has obviously not been able to incorporate the theories that used to sustain those trends. Its thematic and expressional 258

versatility reflects its triumph of being practically the first art that concerns all people of modern societies. Eventually, it paid the price of all products that appeal to a large number of people; it became increasingly and extremely difficult to identify its merits inside the havoc of pop culture and mass consumption industry.

Technology and its impact is the only point of reference that a theory for digital cinema may be based on. The present study has pre-supposed that it is pointless to theorize cinema without placing at the center of this theory the perception of the image. This perception is what eventually defines the kind of spectacle that cinema is, and it is truly bounded and dependent on technological innovation. Most of all, a kind of theory that focuses on perception and elevates the role of the active mind proves that the spectator is not secondary to the medium, as e.g. Psychoanalytic theory has suggested. A theory that is based on the physics of visual perception and cognitive comprehension does not merely provide the stability of scientific objectivity that digital cinema craves for right now; nor is it simply a temporary patch until a “real” theory comes up. At this time, it is the only kind of theory that may address both areas of movie production and spectatorship, without getting in the way of other trends that may wish to elaborate on other case-specific issues, like for instance social or cultural aspects of cinema. Digital cinema is a generic and all-encompassing phrase that describes the better part of movie production today; each specific kind of cinema within it is simply a different manifestation of an otherwise industry-wide practice. Therefore, correspondingly, the genericness of digital cinema as an umbrella term, also needs an umbrella theory; a cognitive approach like that of Hochberg may serve to explain the general nature of CGI, while a more specific theory on, say, the political messages in movies, 259

may still focus on just that. The difference between a theory of visual perception and all those other theories will always remain that of a theory that applies to the entire medium, in contrast to others that cater for a specific academic target group within them.

The cognitive approach of Julian Hochberg has proven its worth. It has provided cinema theory with quite straightforward ideas and insights, relieved from any obscurities of narcissistically self-reflexive philosophies, or high-maintenance publications of neo-culturalist jibber-jabber. The idea of schematic mapping and all the notions that revolve around it in the matrix of Hochberg’s cognitive framework provide a substantiation for a new conception of realism, inaugurated by digital graphics, which is not easily outweighed by anything else that cinema theorizing has to offer at present. The more non-realistic cinema grows, in terms of the real world, the furthest away it drifts from traditional theories of representation. Hochberg’s schematic maps are practically capable of explaining any form of digital cinema out there, as it seeks the answer in factors that are inherent to the physiological mechanisms of human perception, and, as such, are not bound by cultural specificities. No matter how much cinema has changed or continues to change, the schematic foundation of human perception will never be outdated as a point of reference, as it is a function of the visual system itself. This is indeed, or hopefully, what everyone seeks in academic knowledge: an all-purpose, all-weather approach to turn to; the shelter of science for the rainy days of theory. 260

WORKS CITED

A. Books, Articles and Miscellaneous

Abrams, Nathan, Ian A. F. Bell, and Jan Udris. Studying Film. London: Arnold, 2001.

Print.

Adelson, E. H., and J. R. Bergen. “Spatiotemporal Energy Models for the Perception

of Motion.” Optical Society of America, Journal, A: Optics and Image Science 2

(1985): 284–299. Print.

Alexander, Bryan. The New Digital Storytelling: Creating Narratives with New Media.

Santa Barbara, CA: Praeger, 2011. Google Books. Web. 10 July 2013.

Anderson, Joseph. The Reality of Illusion: An Ecological Approach to Cognitive Film

Theory. Carbondale: SIU Press, 1996. Print.

Anderson, Joseph, and Barbara Fisher Anderson. “The Myth of Persistence of Vision

Revisited.” Academic Computing @ Evergreen. n. d., 3 Jan. 2006. Web. 15 July

2012.

Andrew, Dudley. Concepts in Film Theory. Oxford: Oxford University Press, 1984.

Print.

Andrew, J. Dudley. The Major Film Theories: An Introduction. London: Oxford

University Press, 1976. Print.

Bazin, André. What Is Cinema? Vol I. Ed. & trans. by Hugh Gray. Berkeley, CA:

University of California Press, 2005. Print. 2 vols.

Bechtel, William, and Robert E. Richardson. “Vitalism.” Ed. Edward Craig. The Shorter

Routledge Encyclopedia of Philosophy, 2005. Print. 261

Bedini, Silvio A. “The Role of Automata in the History of Technology.” Technology

and Culture 5.1 (1964): 24–42. JSTOR. Web. 7 Sept. 2013.

Benjamin, Walter. “The Work of Art in the Age of Mechanical Reproduction.” The

Continental Aesthetics Reader. Ed. Clive Cazeaux. London: Routledge, 2000.

322–343. Print.

Ben-Shaul, Nitzan. Film: The Key Concepts. Oxford: Berg, 2007. Print.

Berton, John Andrew. “Film Theory for the Digital World: Connecting the Masters to

the New Digital Cinema.” Leonardo. Supplemental Issue 3 (1990): 5–11.

JSTOR. Web. 20 Mar. 2011.

Binder, Marc D., Nobutaka Hirokawa, and Uwe Windhorst, eds. “Biological Motion

Processing.” Encyclopedia of Neuroscience. Springer Berlin Heidelberg, 2009.

408–408. Springer Link. Web. 7 Oct. 2013.

Bolter, J. David, and Richard Grusin. Remediation: Understanding New Media.

Cambridge, MA: MIT Press, 2000.

Bordwell, David. “Contemporary Film Theory and the Vicissitudes of Grand Theory.” Post-

Theory: Reconstructing Film Studies. Ed. David Bordwell and Noël Carroll. Madison:

University of Wisconsin Press, 1996. 3–36. Print.

---. “Intensified Continuity: Visual Style in Contemporary American Film.” Film

Quarterly 55.3 (2002): 16–28. JSTOR. Web. 20 July 2013.

---. Making Meaning: Inference and Rhetoric in the Interpretation of Cinema.

Cambridge, MA: Harvard University Press, 1989. Print.

---. Narration in the Fiction Film. Madison: University of Wisconsin Press, 1985.

---. Poetics of Cinema. New York: Routledge, 2008. Print. 262

---. “The Idea of Montage in Soviet Art and Film.” Cinema Journal 11.2 (1972): 9–17.

JSTOR. Web. 13 Nov. 2011.

---. The Way Hollywood Tells It: Story and Style in Modern Movies. Berkeley:

University of California Press, 2006. Print.

Bordwell, David, Janet Staiger, and Kristin Thompson. The Classical Hollywood

Cinema: Film Style & Mode of Production to 1960. London: Routledge, 1985.

Print.

Bordwell, David, and Kristin Thompson. Film Art: An Introduction. 8th ed. New York:

McGraw - Hill, 2008. Print.

Bruner, Jerome. “The Narrative Construction of Reality.” Critical Inquiry 18.1 (1991):

1–21. Print.

Buckland, Warren. The Cognitive Semiotics of Film. Cambridge University Press,

2000. Print.

Burt, Peter, and George Sperling. “Time, Distance, and Feature Trade-Offs in Visual

Apparent Motion.” Psychological Review 88.2 (1981): 171–195.

Butler, Brian E. “Seeing Ecology and Seeing as Ecology: On Brereton’s Hollywood

Utopia and the Anderson’s Moving Image Theory.” Film-Philosophy 11.1

(2007): 61–69. Print.

Carroll, J. M., and T. G. Bever. “Segmentation in Cinema Perception.” Science

191.4231 (1976): 1053–1055. JSTOR. Web. 8 May 2013.

Carroll, Noël. “Prospects for Film Theory: A Personal Assessment.” Post-Theory:

Reconstructing Film Studies. Ed. David Bordwell and Noël Carroll. Madison:

University of Wisconsin Press, 1996. 37–68. Print. 263

Cavanagh, Patrick, and George Mather. “Motion: The Long and Short of It.” Spatial

Vision 4.2/3 (1989): 103–129. Print.

Cook, David A. History of Narrative Film. 2nd ed. New York: Norton, 1990. Print.

Currie, Gregory. “Film, Reality, and Illusion.” Post-Theory: Reconstructing Film

Studies. Ed. David Bordwell and Noël Carroll. Madison: University of

Wisconsin Press, 1996. 325–344. Print. Wisconsin Studies in Film.

---. Image and Mind: Film Philosophy and Cognitive Science. Cambridge: Cambridge

University Press, 1995.

Cutting, James E. “Affine Distortions of Pictorial Space: Some Predictions for

Goldstein (1987) That La Gournerie (1859) Might Have Made.” Journal of

Experimental Psychology 14.2 (1988): 305–311. Ovid (Journals@Ovid). Web. 5

Jan. 2014.

---. “Perceiving Scenes in Film and in the World.” Moving Image Theory: Ecological

Considerations. Ed. Joseph D. Anderson and Barbara Fisher Anderson.

Carbondale: SIU Press, 2007. 9–27. Print.

---. “Rigidity in Cinema Seen From the Front Row, Side Aisle.” Journal of Experimental

Psychology 13.3 (1987): 323–334. Print.

Darley, Andrew. Visual Digital Culture: Surface Play and Spectacle in New Media

Genres. London: Routledge, 2000. Print.

Delanty, Gerard. Social Science: Beyond Constructivism and Realism. Buckingham:

Open University Press, 1997. Print.

Elsaesser, Thomas, and Warren Buckland. Studying Contemporary American Film: A

Guide to Movie Analysis. London: Arnold, 2002. 264

Elsaesser, Thomas, and Malte Hagener. Film Theory: An Introduction through the

Senses. New York: Routledge, 2010. Print.

Enns, James T., and Erin Austen. “Mental Schemata and the Limits of Perception.” In

the Mind’s Eye: Julian Hochberg on the Perception of Pictures, Films, and the

World. Ed. Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New York:

Oxford University Press, 2007. 439–447. Print.

Fabe, Marilyn. Closely Watched Films: An Introduction to the Art of Narrative Film

Technique. Berkeley, CA: University of California Press, 2004. Print.

Gibson, James Jerome. The Ecological Approach to Visual Perception. New York:

Psychology Press/Taylor & Francis, 1986. Print.

Gombrich, E.H. The Story of Art (Pocket Edition). London: Phaidon, 2006. Print.

Gordon, Ian E. Theories of Visual Perception. New York: Psychology Press, 2004.

Print.

Hatfield, Gary. “Perception as Unconscious Inference.” Perception and the Physical

World. Ed. Dieter Heyer and Reiner Mausfeld. Chichester: John Wiley & Sons,

Ltd, 2005. 113–143. Wiley Online Library. Web. 30 Apr. 2013.

Hayhoe, Mary M. “Integration of Visual Information Across Saccades.” In the Mind’s

Eye: Julian Hochberg on the Perception of Pictures, Films, and the World. Ed.

Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New York: Oxford

University Press, 2007. 448–453. Print.

Heeger. David. “Perception Lecture Notes: The Retina.” The Center for Neural Science

at NYU, n.d. Web. 7 Oct. 2013 265

Heider, Fritz, and Marianne Simmel. “An Experimental Study of Apparent Behavior.”

The American Journal of Psychology 57.2 (1944): 243–259. JSTOR. Web. 15

Oct. 2013.

Hiris, Eric. “Detection of Biological and Nonbiological Motion.” Journal of Vision 7.12

(2007): 4. www.journalofvision.org. Web. 7 Oct. 2013.

Hochberg, Julian. “Components of Literacy: Speculations and Exploratory Research.”

In the Mind’s Eye: Julian Hochberg on the Perception of Pictures, Films, and

the World. Ed. Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New

York: Oxford University Press, 2007. 125–138. Print.

---. “Higher-Order Stimuli and Inter-Response Coupling in the Perception of the

Visual World.” In the Mind’s Eye: Julian Hochberg on the Perception of

Pictures, Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and H.

A. Sedgwick. New York: Oxford University Press, 2007. 186–205. Print.

---. “How Big Is a Stimulus?” In the Mind’s Eye: Julian Hochberg on the Perception of

Pictures, Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and H.

A. Sedgwick. New York: Oxford University Press, 2007. 302–328. Print.

---. “In the Mind’s Eye.” In the Mind’s Eye: Julian Hochberg on the Perception of

Pictures, Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and H.

A. Sedgwick. New York: Oxford University Press, 2007. 76–78. Print.

---. “Levels of Perceptual Organization.” In the Mind’s Eye: Julian Hochberg on the

Perception of Pictures, Films, and the World. Ed. Mary A. Peterson, Barbara

Gillam, and H. A. Sedgwick. New York: Oxford University Press, 2007. 275–

301. Print. 266

---. “On Cognition in Perception: Perceptual Coupling and Unconscious Inference.”

Cognition 10.1–3 (1981): 127–134. ScienceDirect. Web. 1 May 2013.

---. “The Perception of Pictorial Representations.” Social Research 51.4 (1984): 841.

Print.

---. “The Psychophysics of Pictorial Perception.” In the Mind’s Eye: Julian Hochberg

on the Perception of Pictures, Films, and the World. Ed. Mary A. Peterson,

Barbara Gillam, and H. A. Sedgwick. New York: Oxford University Press, 2007.

30–59. Print.

---. “The Representation of Things and People.” In the Mind’s Eye: Julian Hochberg on

the Perception of Pictures, Films, and the World. Ed. Mary A. Peterson,

Barbara Gillam, and H. A. Sedgwick. New York: Oxford University Press, 2007.

148–185. Print.

Hochberg, Julian, and Virginia Brooks. “Film Cutting and Visual Momentum.” In the

Mind’s Eye: Julian Hochberg on the Perception of Pictures, Films, and the

World. Ed. Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New York:

Oxford University Press, 2007. 206–228. Print.

---. “Movies in the Mind’s Eye.” In the Mind’s Eye: Julian Hochberg on the Perception

of Pictures, Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and

H. A. Sedgwick. New York: Oxford University Press, 2007. 376–395. Print.

---. “The Perception of Motion Pictures.” Cognitive Ecology. Ed. Morton P. Friedman

and Edward C. Carterette. San Diego: Academic Press, 1996. 205–292. Print.

Hochberg, Julian, and Edward McAlister. “A Quantitative Approach to Figural

Goodness.” In the Mind’s Eye: Julian Hochberg on the Perception of Pictures, 267

Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and H. A.

Sedgwick. New York: Oxford University Press, 2007. 11–16. Print.

Hunt, Amelia R., and Fred Halper. “Disorganizing Biological Motion.” Journal of Vision

8.9 (2008): 12. www.journalofvision.org. Web. 7 Oct. 2013.

Huyssen, Andreas. After the Great Divide: Modernism, Mass Culture, Postmodernism.

Bloomington: Indiana University Press, 1986. Print.

Ikeda, Hanako, Randolph Blake, and Katsumi Watanabe. “Eccentric Perception of

Biological Motion Is Unscalably Poor.” Vision Research 45.15 (2005): 1935–

1943. ScienceDirect. Web. 8 Oct. 2013.

Intraub, Helene. “Scene Perception: The World through a Window.” In the Mind’s

Eye: Julian Hochberg on the Perception of Pictures, Films, and the World. Ed.

Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New York: Oxford

University Press, 2007. 454–466. Print.

---. “The Representation of Visual Scenes.” Trends in Cognitive Sciences 1.6 (1997):

217–222. ScienceDirect. Web. 21 Aug. 2013.

Ione, Amy. “The Gift of Seeing: Nineteenth-Century Views from the Field.” Art,

Technology, Consciousness: Mind@large. Ed. Roy Ascott. Bristol: Intellect

Books, 2000. 84–89.

Joyce, Mark. “The Soviet Montage Cinema of the 1920s.” An Introduction to Film

Studies. Ed. Jill Nelmes. New York: Routledge, 1996. 329–364. Print.

Kania, Andrew. “The Illusion of Realism in Film.” British Journal of Aesthetics 42.3

(2002): 243–258. HighWire. Web. 25 Jan. 2010. 268

Kipnis, Laura. “Film and Changing Technologies.” The Oxford Guide to Film Studies.

Ed. W. John Hill and Pamela Church Gibson. New York: Oxford University

Press, 1998. 595–604. Print.

Lefebvre, Martin, and Marc Furstenau. “Digital Editing and Montage: The Vanishing

Celluloid and Beyond.” Cinémas : revue d’études cinématographiques /

Cinémas: Journal of Film Studies 13.1-2 (2002): 69–107. Print.

Lewer, Debbie, ed. Post-Impressionism to World War II. Malden, MA: Blackwell,

2006. Print.

Lichtenstein, Edward H., and William F. Brewer. “Memory for Goal-Directed Events.”

Cognitive Psychology 12.3 (1980): 412–445. ScienceDirect. Web. 30 Nov.

2013.

Lu, Hongjing. “Structural Processing in Biological Motion Perception.” Journal of

Vision 10.12 (2010): 13. www.journalofvision.org. Web. 7 Oct. 2013.

Lunenfeld, Peter. “The Myths of Interactive Cinema.” The New Media Book. Ed. Dan

Harries. London: BFI, 2002. 144–154. Google Books. Web. 22 July 2011.

Malina, Roger F. “Digital Image-Digital Cinema: The Work of Art in the Age of Post-

Mechanical Reproduction.” Leonardo. Supplemental Issue 3 (1990): 33–38.

Print.

Manovich, Lev. “Avant-Garde as Software.” Lev Manovich. N. p., n.d. Web. 19 Jan.

2009.

---. “Image Future.” http://www.manovich.net/. N. p., 2 Nov. 2008. Web. 18 Jan.

2009.

---. The Language of New Media. Cambridge, Mass: MIT Press, 2002. Print. 269

---. “Understanding Hybrid Media.” http://www.manovich.net/. N. p., 2 Nov. 2008.

Web. 18 Jan. 2009.

Mayor, A. Hyatt. “Daguerreotypes and Photographs.” The Metropolitan Museum of

Art Bulletin 34.11 (1939): 240–243. JSTOR. Web. 3 July 2011.

McKoon, Gail, and Roger Ratcliff. “Inference During Reading.” Psychological Review

July 1992 99.3 (1992): 440–466. Print.

Messaris, Paul. “Digital Hollywood: Politics, Economics, Aesthetics.” Annenberg

School of Communication - University of Pennsylvania. N. p., n.d. Web. 21 July

2013.

Metz, Walter. “Hollywood Cinema.” The Cambridge Companion to Modern American

Culture. Ed. Christopher Bigsby. Cambridge: Cambridge University Press,

2006. 374–391. Print.

Meulders, Michel. Helmholtz: From Enlightenment to Neuroscience. Cambridge, MA:

MIT Press, 2010. Print.

Mitchell, William J. The Reconfigured Eye: Visual Truth in the Post-Photographic Era.

Cambridge, Mass: MIT Press, 1994. Print.

Munsterberg, Hugo. The Photoplay: A Psychological Study. New York: Appleton,

1916. Print.

Nelson, Katherine. “The Psychological and Social Origins of Autobiographical

Memory.” Psychological Science 4.1 (1993): 7–14. JSTOR. Web. 6 Jan. 2013.

Neri, Peter, M. Concetta Morrone, and David C. Burr. “Seeing Biological Motion.”

Nature 395.6705 (1998): 894–6. ProQuest. Web. 13 Oct. 2013.

Newcombe, Nora S., Marianne E. Lloyd, and Kristin R. Ratliff. “Development of

Episodic and Autobiographical Memory: A Cognitive Neuroscience 270

Perspective.” Advances in Child Development and Behavior. Ed. Robert V. Kail.

Volume 35. JAI, 2007. 37–85. ScienceDirect. Web. 6 Jan. 2013.

Parks, Theodore E. “Post-Retinal Visual Storage.” The American Journal of Psychology

78.1 (1965): 145–147. JSTOR. Web. 6 Dec. 2014.

Peterson, Mary A. “The Piecemeal, Constructive, and Schematic Nature of

Perception.” In the Mind’s Eye: Julian Hochberg on the Perception of Pictures,

Films, and the World. Ed. Mary A. Peterson, Barbara Gillam, and H. A.

Sedgwick. New York: Oxford University Press, 2007. 419–428. Print.

Pinto, J., and M. Shiffrar. “Subconfigurations of the Human Form in the Perception of

Biological Motion Displays.” Acta Psychologica 102.2–3 (1999): 293–318.

ScienceDirect. Web. 8 Oct. 2013.

Prince, Stephen, and Wayne E. Hensley. “The Kuleshov Effect: Recreating the Classic

Experiment.” Cinema Journal 31.2 (1992): 59–75. JSTOR. Web. 24 Nov. 2012.

Redmond, Sean. “Film since 1980.” The Routledge Companion to Science Fiction. Ed.

Mark Bould et al. Abingdon: Routledge, 2009. Print.

Reisz, Karel, and Gavin Millar. The Technique of Film Editing. 2nd ed. Burlington, MA:

Focal Press, 2010. Print.

Richards, Whitman. “Marr, Gibson, and Gestalt: A Challenge.” Perception 41.9

(2012): 1024 – 1026. Pion Journals. Web. 11 Aug. 2013.

Rock, Irvin. “The Logic of ‘The Logic of Perception.’” Giornale Italiano di Psicologia

20.5 (1993): 841–867. Print.

Rutsky, R. L. High Technē: Art and Technology from the Machine Aesthetic to the

Posthuman. Minneapolis: University of Minnesota Press, 1999. Print. 271

Ryan, Marie-Laure. “From Possible Worlds to Parallel Universes.” Université Paris

Diderot - Paris 7. N. p., 4 May 2006. Web. 21 Nov. 2012.

Schatz, Thomas. “Just When You Thought It Was Safe; Jaws & the New Hollywood;

No Longer Just a Blockbuster, a Movie Is a Marketing Extravanganza Series:

1976-1996: 20 Years of Pop Culture.” Austin American Statesman (1996): 36.

ProQuest. Web. 7 Nov. 2012.

Sedgwick, H. A. “Hochberg on the Perception of Pictures and of the World.” In the

Mind’s Eye: Julian Hochberg on the Perception of Pictures, Films, and the

World. Ed. Mary A. Peterson, Barbara Gillam, and H. A. Sedgwick. New York:

Oxford University Press, 2007. 572–580. Print.

Shaw, Robert, and John Pittenger. “Perceiving Change.” Modes of Perceiving and

Processing Information. Ed. Herbert L. Pick and Elliot Lee Saltzman. Hillsdale,

NJ: Erlbaum, 1978. 187–204. Print.

Shepard, Roger N. “Ecological Constraints on Internal Representation: Resonant

Kinematics of Perceiving, Imagining, Thinking, and Dreaming.” Psychological

Review 91.4 (1984): 417–447. APA PsycNET. Web. 1 Oct. 2013.

Simons, Daniel J., and Daniel T. Levin. “Change Blindness.” Trends in Cognitive

Sciences 1.7 (1997): 261–267. ScienceDirect. Web. 1 May 2013.

Sobchack, Vivian Carol. The Address of the Eye: A Phenomenology of Film Experience.

Princeton, N.J: Princeton University Press, 1992.

Squire, L. R., B. Knowlton, and G. Musen. “The Structure and Organization of

Memory.” Annual Review of Psychology 44 (1993): 453. Print. 272

Steinman, Robert M., Zygmunt Pizlo, and Filip J. Pizlo. “Phi Is Not Beta, and Why

Wertheimer’s Discovery Launched the Gestalt Revolution.” Vision Research

40.17 (2000): 2257–2264. ScienceDirect. Web. 2 Oct. 2013.

Tan, Ed S. “On the Cognitive Ecology of the Cinema.” In the Mind’s Eye: Julian

Hochberg on the Perception of Pictures, Films, and the World. Ed. Mary A.

Peterson, Barbara Gillam, and H. A. Sedgwick. New York: Oxford University

Press, 2007. 562–571. Print.

Thompson, James, and Raja Parasuraman. “Attention, Biological Motion, and Action

Recognition.” NeuroImage 59.1 (2012): 4–13. ScienceDirect. Web. 7 Oct.

2013.

Tillis, Steve. “The Art of Puppetry in the Age of Media Production.” TDR (1988-) 43.3

(1999): 182–195. JSTOR. Web. 27 Aug. 2013.

Travis, David. “Photography as a Print-Making Medium: The Early Successes.”

Bulletin of the Art Institute of Chicago (1973-1982) 69.3 (1975): 15–19. JSTOR.

Web. 3 July 2011.

Troje, Nikolaus. “Biological Motion Perception.” Web. 9 Oct. 2013.

Troje, Nikolaus, and Dorita Chang. “Shape-Independent Processing of Biological

Motion.” Web. 9 Oct. 2013.

Tulving, Endel. “Episodic and Semantic Memory.” Organization of Memory. Ed. Endel

Tulving and Wayne Donaldson. New York: Academic Press, 1972. 381–402.

Print.

Tyler, Christopher W. “Perspective as a Geometric Tool that Launched the

Renaissance.” SKERI – The Smith-Kettlewell Eye Research Institute, n.d. Web.

3 October 2014. 273

Verstraten, Peter. “Between Attraction and Story: Rethinking Narrativity in Cinema.”

Narratology in the Age of Cross-Disciplinary Narrative Research. Ed. Sandra

Heinen and Roy Sommer. Berlin: Walter de Gruyter, 2009. 154–169. Print.

Walters, James. Fantasy Film: A Critical Introduction. Oxford: Berg, 2011. Print.

Walton, Kendall L. “Transparent Pictures: On the Nature of Photographic Realism.”

Marvelous Images: On Values and the Arts. Oxford: Oxford University Press,

2008. 79–109. Print.

Watson, A. B., and A. J. Ahumada. “Model of Human Visual-Motion Sensing.” Optical

Society of America, Journal, A: Optics and Image Science 2 (1985): 322–342.

Print.

“What is the Fantastic?” The University of South Carolina at Chapel Hill. USC-Chapel

Hill, n.d. Web. 4 December 2014.

Young, Kay, and Jeffrey L. Saver. “The Neurology of Narrative.” SubStance 30.1/2

(2001): 72–84. JSTOR. Web. 26 July 2012.

B. Movies

Avatar. Dir. James Cameron. Perf. James Worthington, Zoe Saldana, and Sigourney

Weaver. Twentieth Century Fox, 2009. Film.

Beowulf. Dir. Robert Zemeckis. Perf. Anthony Hopkins, Robin Wright, and Ray

Winstone. Paramount Pictures, 2007. Film.

Die hard. Dir. John McTiernan. Perf. Bruce Willis, Bonnie Bedelia, Alan Rickman, and

Paul Gleason. Twentieth Century Fox, 1998. Film.

District 9. Dir. Neill Blomkamp. Perf. Sharlto Copley, Jason Cope, and Nathalie Boltt.

TriStar Pictures, 2009. Film. 274

Indiana Jones and the Kingdom of the Crystal Skull. Dir. Steven Spielberg. Perf.

Harrison Ford, Cate Blanchett, and Shia LaBeouf. Paramount Pictures, 2008.

Film.

Immortel (Ad Vitam). Dir. Enki Bilal. Perf. Linda Hardy, Thomas Kretschmann, and

Charlotte Rampling. Téléma, 2004. Film.

Intolerance: Love's Struggle Throughout the Ages. Dir. D.W. Griffith. Perf. Lillian Gish,

Spottiswoode Aitken, and Mary Alden. Triangle Film Corporation, 1916. Film.

Jason and the Argonauts. Dir. Don Chaffey. Perf. Todd Armstrong, Nancy Kovack, and

Gary Raymond. Columbia Pictures, 1963. Film.

Jaws. Dir. Steven Spielberg. Perf. Roy Scheider, Robert Shaw, and Richard Dreyfuss.

Zanuck/Brown Productions, 1975. Film.

John Carter. Dir. Andrew Stanton. Perf. Taylor Kitsch, Lynn Collins, and Willem

Dafoe. Walt Disney Pictures, 2012. Film.

Life of an American Fireman. Dir. Edwin S. Porter (uncredited) and George S. Fleming

(uncredited). Perf. Vivian Vaughan (uncredited), Arthur White (uncredited),

and James H. White (uncredited). Edison Manufacturing Company, 1903.

Film.

Man of Steel. Dir. Zack Snyder. Perf. Henry Cavill, Amy Adams, and Michael Shannon.

Warner Bros, 2013. Film.

Nochnoy Dozor. Dir. Timur Bekmambetov. Perf. Konstantin Khabenskiy, Vladimir

Menshov, and Valeriy Zolotukhin. Bazelevs Production, 2004. Film.

Prospero’s Books. Dir. Peter Greenaway. Perf. John Gielgud, Michael Clark, and

Michel Blanc. Allarts, 1991. Film. 275

Sin City. Dir. Frank Miller, Robert Rodriguez, and Quentin Tarantino. Perf. Mickey

Rourke, Benicio Del Toro, and Bruce Willis. Dimension Films, 2005. Film.

Sin City 2: A Dame to Kill For. Dir. Frank Miller, Robert Rodriguez. Perf. Mickey

Rourke, Josh Brolin, and Joseph Gordon-Levitt. Residaco, 2014. Film.

Star Trek XI. Dir. J.J. Abrams. Perf. Chris Pine, Zachary Quinto, and Eric Bana.

Paramount Pictures, 2009. Film.

Star Wars: Episode IV: A New Hope. Dir. George Lucas. Perf. Mark Hamill, Harrison

Ford, and Carrie Fisher. Lucasfilm, 1977. Film.

Star Wars: Episode V: The Empire Strikes Back. Dir. George Lucas. Perf. Mark Hamill,

Harrison Ford, and Carrie Fisher. Lucasfilm, 1980. Film.

Star Wars: Episode VI - Return of the Jedi. Dir. George Lucas. Perf. Mark Hamill,

Harrison Ford, and Carrie Fisher. Lucasfilm, 1983. Film.

Sucker Punch. Dir. Zack Snyder. Perf. Emily Browning, Abbie Cornish, and Jena

Malone. Warner Bros., 2011. Film.

The Birth of a Nation. Dir. D.W. Griffith. Perf. Lillian Gish, Mae Marsh, and Henry B.

Walthall. David W. Griffith Corp., 1915. Film.

The Bourne Ultimatum. Dir. Paul Greengrass. Perf. Matt Damon, Julia Stiles, and

David Strathairn. Universal Pictures, 2007. Film.

The Chronicles of Riddick. Dir. David Twohy. Perf. Vin Diesel, Thandie Newton, and

Karl Urban. Universal Pictures, 2004. Film.

The Fifth Element. Dir. Luc Besson. Bruce Willis, Gary Oldman, and Milla Jovovich.

Gaumont, 1997. Film.

The Fountain. Dir. Darren Aronofsky. Perf. Hugh Jackman, Rachel Weisz, and Ellen

Burstyn. Warner Bros., 2006. Film. 276

The Hobbit: An Unexpected Journey. Dir. Peter Jackson. Perf. Ian McKellen, Martin

Freeman, and Richard Armitage. New Line Cinema, 2012. Film

The Hobbit: The Battle of Five Armies. Dir. Peter Jackson. Perf. Ian McKellen, Martin

Freeman, and Richard Armitage. New Line Cinema, 2014. Film

The Hobbit: The Desolation of Smaug. Dir. Peter Jackson. Perf. Ian McKellen, Martin

Freeman, and Richard Armitage. New Line Cinema, 2013. Film

The Lord of the Rings: The Fellowship of the Ring. Dir. Peter Jackson. Perf. Elijah

Wood, Ian McKellen, and Viggo Mortensen. New Line Cinema, 2001. Film.

The Lord of the Rings: The Return of the King. Dir. Peter Jackson. Perf. Elijah Wood,

Ian McKellen, and Viggo Mortensen. New Line Cinema, 2003. Film.

The Lord of the Rings: The Two Towers. Dir. Peter Jackson. Perf. Elijah Wood, Ian

McKellen, and Viggo Mortensen. New Line Cinema, 2002. Film.

The Matrix. Dir. Andy and Lana Wachowski (as The Wachowski Brothers). Perf. Keanu

Reeves, Laurence Fishburne, and Carrie-Anne Moss. Warner Bros, 1999. Film.

The Matrix Reloaded. Dir. Andy and Lana Wachowski (as The Wachowski Brothers).

Perf. Keanu Reeves, Laurence Fishburne, and Carrie-Anne Moss. Warner Bros,

2003. Film.

The Matrix Revolutions. Dir. Andy and Lana Wachowski (as The Wachowski

Brothers). Perf. Keanu Reeves, Laurence Fishburne, and Carrie-Anne Moss.

Warner Bros, 2003. Film.

The Polar Express. Dir. Robert Zemeckis. Perf. Tom Hanks, Leslie Zemeckis, and Eddie

Deezen. Castle Rock Entertainment, 2004. Film.

The Pillow Book. Dir. Peter Greenaway. Perf. Vivian Wu, Ewan McGregor, and Yoshi

Oida. Kasander & Wigman Productions, 1997. Film. 277

The Tulse Luper Suitcases, Part 1: The Moab Story. Dir. Peter Greenaway. Perf. JJ

Feild, Raymond J. Barry, and Michèle Bernier. ABS Production, 2003. Film.

The Tulse Luper Suitcases, Part 2: Vaux to the Sea. Dir. Peter Greenaway. Perf. JJ

Feild, Raymond J. Barry, and Valentina Cervi. Intuit Pictures, 2004. Film.

The Tulse Luper Suitcases, Part 3: From Sark to the Finish. Dir. Peter Greenaway. Perf.

Roger Rees, Stephen Billington, and Jordi Mollà. Delux Productions, 2004.

Film.

Transformers. Dir. Michael Bay. Perf. Shia LaBeouf, Megan Fox, and Josh Duhamel.

DreamWorks SKG, 2007. Film.

TRON: Legacy. Dir. Joseph Kosinski. Perf. Jeff Bridges, Garrett Hedlund, and Olivia

Wilde. Walt Disney Pictures, 2010. Film.

X-men: Days of Future Past. Dir. Bryan Singer. Perf. Hugh Jackman, James McAvoy,

and Michael Fassbender. Twentieth Century Fox, 2014. Film.

2046. Dir. Kar Wai Wong. Perf. Tony Chiu Wai Leung. Li Gong, and Ziyi Zhang. Jet

Tone Films, 2004. Film.

300. Dir. Zack Snyder. Perf. Gerard Butler, Lena Headey, and Rodrigo Santoro.

Warner Bros., 2006. Film.

300: Rise of an Empire. Dir. Noam Murro. Sullivan Stapleton, Eva Green, and Lena

Headey. Warner Bros., 2014. Film.

278

C. Audiovisual Supplements

Ci. Movie Clips

Aladan57. “Jason and the Argonauts: Battle with the Skeleton Army. (Real Sounds

from the Movie).” Online video clip. YouTube. YouTube, 7 May 2013. Web. 7

Sep. 2014.

AnimationMethods. “What is FK/IK Rigs.” Online video clip. YouTube. YouTube, 15

Oct. 2011. Web. 10 Sep. 2014.

Autochromex. “Life of an American Fireman 1903.” Online video clip. YouTube.

YouTube, 7 Mar. 2007. Web. 24 Aug. 2014.

Beckmsberlin. “Battlefield Earth Official Trailer.” Online video clip. YouTube.

YouTube, 5 Aug. 2008. Web. 6 Sep. 2014.

BioMotionLab. “Johansson: Motion Perception Part 2.” Online video clip. YouTube.

YouTube, 11 Nov. 2011. Web. 24 Aug. 2014.

BiodigitaljazzMann's channel. “Light Cycle Legacy HD.” Online video clip. YouTube.

YouTube, 13 Nov. 2011. Web. 24 Aug. 2014.

Celes Clips. “Transformers (2007) Fighting Scorponok with Sabot Rounds 720p HD.”

Online video clip. YouTube. YouTube, 20 Jun. 2013. Web. 24 Aug. 2014.

EgalmothOfGondolin01. “LOTR The Fellowship of the Ring – Extended Edition – The

Mirror of Galadriel.” Online video clip. YouTube. YouTube, 10 Aug. 2013.

Web. 29 Aug. 2014.

EgalmothOfGondolin01. “LOTR The Fellowship of the Ring – Extended Edition – The

Shire.” Online video clip. YouTube. YouTube, 14 Dec. 2012. Web. 29 Aug.

2014. 279

Ephimetheuss. “Star Trek XI – The New Enterprise.” Online video clip. YouTube.

YouTube, 23 May 2011. Web. 6 Sep. 2014.

Fiske Kraftverk. “The Matrix Reloaded – The Freeway Chase – The Full Scene.” Online

video clip. YouTube. YouTube, 30 Mar. 2012. Web. 23 Aug. 2014.

Gabriel Joseph. “Matrix – Red Pill Blue Pill, Choice [Best].” Online video clip.

YouTube. YouTube, 6 Apr. 2011. Web. 29 Aug. 2014.

HelloGreedo. “Star Wars Changes – Part 1-Episode IV A New Hope.” Online video

clip. YouTube. YouTube, 27 Sep. 2011. Web. 23 Aug. 2014.

Hypnologic. “How It's Made – Film Digitization.” Online video clip. YouTube.

YouTube, 7 Mar. 2013. Web. 23 Aug. 2014.

Канал корисника shickmystic. “À Bout de Souffle (Breathless) – Jean Luc Godard –

Car Scene – Jean Paul Belmondo / Jean Seberg.” Online video clip. YouTube.

YouTube, 3 Dec. 2009. Web. 24 Aug. 2014.

Kenjirou. “Heider and Simmel (1944) Animation.” Online video clip. YouTube.

YouTube, 26 Jul. 2010. Web. 28 Aug. 2014.

Media Magik Entertainment. “Avatar Exclusive – Behind the Scenes (The Art of

Performance Capture).” Online video clip. YouTube. YouTube, 16 Dec. 2009.

Web. 24 Aug. 2014.

Movieclips. “Jurassic Park (1/10) Movie CLIP – Welcome to Jurassic Park (1993) HD.”

Online video clip. YouTube. YouTube, 27 May 2011. Web. 5 Sep. 2014.

Movieclips. “Sin City (6/12) Movie CLIP – Miho vs. Jackie Boy (2005) HD.” Online

video clip. YouTube. YouTube, 27 Sep 2011. Web. 5 Sep. 2014.

Nature video. “Creating Gollum.” Online video clip. YouTube. YouTube, 11 Dec. 2013.

Web. 23 Aug. 2014. 280

Official Avatar. “Pandora Discovered.” Online video clip. YouTube. YouTube, 10 Dec.

2009. Web. 6 Sep. 2014.

Spartacus Leo. “300 First Battle Part 1 Blu-ray.” Online video clip. YouTube. YouTube,

16 Mar 2014. Web. 6 Sep. 2014.

Stefan Vujic. “The Best 3 Scenes in Matrix – Reloaded.” Online video clip. YouTube.

YouTube, 11 Mar. 2013. Web. 6 Sep. 2014.

SuckerPunchFilm2011. “Sucker Punch – Samurai Fight Scene [HD].” Online video clip.

YouTube. YouTube, 19 Jun. 2011. Web. 5 Sep. 2014.

TCG Soundtrack. “1080p Jor-El Krypton Civil War Scene Man of Steel.” Online video

clip. YouTube. YouTube, 13 Nov. 2013. Web. 5 Sep. 2014.

Cii. Images in Appendix II retrieved from

“Amazing Automatons: Ancient Robots & Victorian Androids.” Simon Rose and Avi

Abrams. Dark Roasted Blend: Weird and Wonderful Things. n.d. Web. 23 Jun.

2014.

“Joseph Jastrow and His Duck – Or Is It a Rabbit?” John F. Kihlstrom. University of

California, Berkeley. 20 Jan. 2012. Web. 27 Aug. 2014.

“Joseph Nicéphore Niépce.” Harry Ransom Center – Exhibitions. The University of

Texas at Austin, n.d. Web. 26 Sep. 2014.

“Landing on Mars: Visiting the Set of Andrew Stanton's 'John Carter.'” Alex Billington.

Firstshowing.net. 4 Mar. 2012. Web. 27 Aug. 2014.

“Müller-Lyer Illusion.” Eric W. Weisstein. MathWorld – A Wolfram Web Resource.

Wolfram Research, 5 Dec. 2014. Web. 22 Aug, 2014.

“Na’vi.” Avatar Wiki. N.d. Web. 28 Aug. 2014. 281

“Penrose Stairway.” Eric W. Weisstein. MathWorld – A Wolfram Web Resource.

Wolfram Research, 5 Dec. 2014. Web. 22 Aug, 2014.

“Penrose Triangle.” Eric W. Weisstein. MathWorld – A Wolfram Web Resource.

Wolfram Research, 5 Dec. 2014. Web. 22 Aug. 2014.

“Phi Vs. Beta.” Luke Holzmann. Production Now. n.d. Web. 25 Aug. 2014.

“Star Wars then and now.” S. Simmons. The Editblog. n.d. Web. 22 Jun. 2014.

“‘X-Men: Apocalypse’ Concludes ‘First Class’ Story; May Include Cable & Psylocke.”

Rob Keyes. Screenrant. n.d. Web. 5 Nov. 2014.

“World's First Photograph.” National Geographic. National Geographic, n.d. Web. 22

Jun. 2014. 282

APPENDIX I - GLOSSARY

The following glossary aims to help readers of the present study. It does not intend to be an exhaustive glossary of cinema, cognitive psychology, digital imaging, etc. Some of the terms included in it are common ones used widely in these disciplines; others, on the other hand, are terms that have been introduced in the present study by its author.

Affordance: A term found in ecological theory of perception, most notably in the work of J. J. Gibson and Joseph Anderson. Affordances are specific, useful qualities of things that are perceived in the environment, e.g. the quality of something being able to be thrown, sat upon, grasped, etc. They are picked up by the perceptual system and trigger a certain action on behalf of the observer, which is normally connected with his optimal chances for survival. Anamorphic lens: Anamorphic lenses, as opposed to spherical lenses, produce a compression of the recorded image along the horizontal axis of the frame, causing the objects on screen to look unnaturally thin and stretched vertically. In simple words, the recorded images are squeezed in order for more visual information on the horizontal axis to fit a narrower 35mm frame. These images are subsequently stretched again in post-production or during projection so that they look normal on a wide screen format. Aspect ratio: In imaging the term aspect ratio refers to the analogy between the width of an image, frame, etc. and its height. A normal aspect ratio for television sets used to be 4:3. Today the wide screen format prevails in both television and cinema, with common aspect ratios being 16:9 and 21:9. Bottom-up inferences (see also Top-down inferences): The term refers to perceptual and cognitive processes which the mind uses in order to comprehend a stimulus or phenomenon by using already stored perceptual data. It applies, tests, and verifies perceptual hypotheses to this set of data by starting from them upwards to mind, thus making hypotheses being raised non-consciously or automatically. CGI (Computer Generated Imagery): CGI refers to entirely synthetic images created with the use of computer software, which may include practically anything, from 283

individual characters, objects, fantasy realms, digital reconstructions of real places (contemporary or not), etc. CGI is created without any of the traditional photographic equipment; nevertheless, quite often these images are artificially made to look as if they were actually shot with lenses (e.g. they include lens flares or camera shake) so that they comply to standards of realism as it has been passed down to us by the analog era. Change blindness: The term change blindness refers to the observation that movie spectators are unable to detect minor changes from one view of a scene to the next, unless they attend specifically to those changes. In order for changes to be noticed, they have to affect the center of vision, from which the human visual system normally draws more information. Even so, the visual system is not normally expected to notice changes if it attends to more than one central points at a time, or if those points do not affect objects that are significant for the narrative. Chroma Key technology (aka Green screen or Blue screen technology): The technique by which different images (live footage or CGI) are composed together by the superimposition of one of them on a monochrome background (green or blue), on which the other is added with the use of software during post-production. The result is a composited image that combines both, as if they were originally shot together. Cinematic story (see also Cinematic world): The cinematic story and the cinematic world together form the cinematic universe. Cinematic story refers to the entire set of events, major or minor as well as explicit or implicit, that take place in the universe of the movie. The cinematic story is not equivalent to the concept of narrative, but rather the latter is included in the former. The cinematic story is a wider concept that forms set of events which need to make perfect sense in relation to each other so that they form a solid infrastructure on which the cinematic universe can be established as reliable and credible. Cinematic world (see also Cinematic story): The cinematic world and the cinematic story together form the cinematic universe. The cinematic world refers to the entire world in which a movie narrative takes place, but the concept is wider than simply one of space: apart from locations, it also includes objects, 284

characters, conventions, etc., as well as everything else that is specific and characteristic of this fictional universe, and which contributes to its visual and experiential identity. Closed Films (see also Open Films): A concept by Leo Braudy; it refers to films in which all elements are under the control of the filmmaker and everything plays a specific functional role in the composition of the film; for example, all elements refer back to themselves in a self-reflexive way that is characteristic of, say, the style of a filmmaker. Cognitive processes: As defined by the APA online glossary of Psychological Terms, cognitive processes are higher mental processes, such as perception, memory, language, problem solving, and abstract thinking. Cognitive Search Guidance (CSG) (see also Peripheral Search Guidance (PSG)): CSG is the process by which the viewer, after his visual system having located and recognized landmarks of meaning in an image, analyses those parts and the ones in proximity to them so that he can comprehend the image cognitively. Compositing, Digital: The process by which real-life footage is blended with CGI with the use of computer software, in a way that produces seamless images. Decoupling: The mental capacity that humans have to always keep mental action separate from physical action. Because of decoupling, viewers can emotionally attach themselves to the events of a movie, but under normal conditions do not take the actions that would be appropriate to those events in real life. Episodic memory (see also Semantic memory): The collection and storing of data regarding specific events that took place only once, in clearly marked time and place. Episodic memory is responsible for a person retaining specific spatiotemporal details about events that took place in his past. Establishing shot: A kind of shot in cinematography which establishes the wider situation, location, narrative context, etc. that the viewer will need in order to follow the meaning of the following scenes. It is usually an open kind of shot, like e.g. an extreme long shot. Figural Goodness: see Prägnanz. Figure-ground distinction: One of the cornerstone theoretical propositions of the Gestaltists. It describes the easiness by which a perceiver immediately 285

distinguishes between a figure against the ground it lays, and the other way round. It was considered to be an innate characteristic of perception, with the latter thus being revealed as dynamic and constantly adapting itself to identifying the most normal or familiar percept in each case. Green screen technology: See Chroma Key technology. Gestaltqualität: Probably the most famous dictum of the Gestaltists; it describes their observation that when stimuli are placed in temporal and spatial relationships, a “form quality” arises between them that differs from (is over and above) the sum of the parts acting in isolation. Invariant: A concept on perception in the theory of J.J. Gibson. It describes any characteristic in an object, a “higher-order” pattern or structure, which remains stable during motion, unlike others, and thus provides utilizable spatial information for the moving perceiver. Intensified continuity: A concept introduced by David Bordwell; it describes the tendency that blockbusters have to become increasingly faster in their editing, from the time of the Classical cinema until the present. La Gournerie’s paradox: A term by James Cutting; drawing its name from its first observation by Jules de La Gournerie (1859), it refers to the fact that human perception can tolerate a viewing angle that is significantly wider than the presumed optimal position at the center of a movie theater; thus the shape and identity of 2D objects remains unaffected when watched by spectators on slanted screens. Long shot: A kind of shot in cinematography in which the entire bodies of the characters are shown, and they are also situated in their surrounding environment and proximity of other characters. Masking: The process by which a part of the cinematic image is concealed. For example, the top and bottom, or left and right parts of the projected image are covered to reduce its size, or to give it a shape of some sort (e.g. a circle with a black frame). In the analog days this was done by placing a frame with the desired size or shape in front of the camera. Minimal Departure, the Principle of: A concept by Marie-Laure Ryan in her theoretical model for fictional worlds; it describes the assumption that, in order 286

for a fictional world to be cognitively comprehensible, it has to be accessible by means of concepts from the actual world. These concepts enable the perceiver to fill in the gaps in the fictional world and comprehend it as complete in its own right. The concept is similar to what Kendal Walton’s “Reality Principle.” Mise-en-scène: The selection of everything that is placed within a specific cinematic frame, e.g. actors, props, lighting, costumes, setting, etc., as well as their arrangement by the moviemaker. Morphing: In digital special effects, the term refers to the seamless visual transformation of one thing into another. It has also been used in the present study in a metaphorical sense to refer to the capacity of the digital means to imitate any analog form or practice. Motion Capture (aka MoCap): MoCap involves the use of numerous sensors strapped on the actors’ bodies. A large number of cameras around the actors pick up and record the signal of each sensor, using software to map all signals together to create a synthetic moving image of the actor inside a CGI environment. The structural forms of the actors moving and performing inside a vector space are subsequently used in post-production to create CGI characters with life-like motion. New Hollywood: In Walter Metz’s separation of cinema periods, the New Hollywood period starts in 1975 with the release of Stephen Spielberg’s Jaws and is characterized by a conscious regression to Old Hollywood styles of moviemaking, such as the re-establishment of the “hero” and the prominent use a classical three-act storyline. For Metz, the New Hollywood also marks the birth of the concepts of blockbuster and franchise. Non-overlapping shots: See Overlapping shots. Open Films (see also Closed Films): A concept by Leo Braudy; open films are those that give the sense of an autonomous cinematic reality which is not controlled by the filmmaker, at least not in a way so obviously tight as in closed films. Open films can be understood as peeks into the cinematic world that they depict much like looking through an open window. Overlapping shots (and non-overlapping shots): The terms overlapping and non- overlapping shots are used by Julian Hochberg and Virginia Brooks to denote 287

successive shots which either share or do not share, respectively, any common features inside their frames (objects or parts of the setting, people, etc.). These common features, when they exist, act as landmarks used by human perceptual mechanism as it scans the image for Paramount Decree, the (see also Unit-production system and Unit-package system): The 1948 ruling of the U.S. Supreme Court against the major studios, forcing the latter to move from the “unit-production system” to the “unit- package system” due to US Government accusations of violation of antitrust legislation. Parks effect: An experimental finding on partial viewing by Theodore E. Parks reported in 1965. Parks’ experiment involved a pattern hidden behind a surface that only had a narrow cut on it; the pattern was moving, whereas the surface with the cut remained stationary, thus enabling only a successive view of parts of the pattern as it passed behind the cut. Viewers were nevertheless able to recognize the entire pattern through the narrow cut. Peripheral Search Guidance (PSG) (see also Cognitive Search Guidance (CSG)): PSG is the initial scanning of a newly encountered image, by which the visual system of the viewer searches in the areas of peripheral vision for parts of the image that could function as landmarks of meaning, on which it then attempts to locate and anchor itself for comprehending it. Phi phenomenon (aka Phi movement): Phi movement occurs when two visual stimuli are interchanged in such a way, that, when the intervals between them properly coordinate with the visual system of the perceiver, motion is seen to occur rather than the stimuli themselves. POV (Point-of-View): A kind of camera angle that shows the image as a certain character in a movie supposedly sees it himself in a certain scene. It is used to intensify the action and make the shot more immersive for spectators. Prägnanz, law of: One of the basic assumptions of the Gestalt theory; the phrase expressed the tendency that perception has, wherever possible, towards simplicity, symmetry, and wholeness. 288

Realistic motivation: A term used by David Bordwell to describe everything that seems lifelike to someone under specific conditions and him having accepted a number of conventions as regards what he sees. Retinal inhomogeneity: A limitation of the human visual system, due to which the images that one sees are detailed only at the center of the retina, and becoming increasingly less clear as we move away from the center towards the periphery. Semantic memory (see also Episodic memory): Semantic memory a person’s set of knowledge of symbols, particularly verbal, as well the relations between them and their referents, the code and rules of managing these symbols and associating them with each other, etc. In simpler words, semantic memory refers to one’s factual knowledge of the world, which is general, non-specific, and not linked to individual events; therefore, it is the general and more abstract kind of knowledge about elements in the world that remains after a specific event is detached from its memory. Schematic map: One of the basic notions in Julian Hochberg’s model of perception. It refers to the combination of the sampling of images that come from the human visual system (which are always partial and piecemeal, due to retinal inhomogeneity) with the cognitive processing of those samples, made through expectations, prior experiences, the human tendency towards veridicality of visual stimuli, etc. Instead of perceiving an object in its entirety, which is physically impossible, humans always perceive a schematic formation of it, deriving from the samples that the center of the retinal collects and their cognitive processing towards making a specific meaning. Top-down inferences (see also Bottom-up inferences): The term refers to perceptual and cognitive processes which the mind uses in order to comprehend a stimulus or phenomenon by using already stored perceptual data. It applies, tests, and verifies perceptual hypotheses to this set of data by starting from conscious thoughts downwards to the perceived phenomenon. This top-down process causes the process to be slower in relation to bottom-up hypotheses. Tracking shot: A shot in which a camera records an object in motion by moving next to it, in a way that the object moves in relation to its background, but remains still and in relation to the frame. Some cinema professionals consider it to be the 289

same as the dolly shot, whereas others believe that the tracking shot is made with a movement of the camera parallel to the object, whereas the dolly is made with the camera facing the object as the latter moves forward (hence the camera moves backwards). Transparency: Following Kendal Walton’s original usage of the term, it refers to the characteristic of a medium like e.g. a movie or a painting to become invisible, thus giving the impression to the spectator that he literally sees the object depicted as if through a window. Unconscious Inference: A key term in Hermann von Helmholtz’s theory of perception. It describes the process by which humans perceive visual stimuli (e.g. from an object or scene), break them down to smaller units with specific attributes, and subsequently preform cognitive operations to recognize and combine together those attributes. This combination eventually grants the perceiver with an account of the accumulated characteristics of the object or scene. Unit-package system (see also Unit-production system and Paramount Decree): A system of movie production and distribution that did not involve movies entirely controlled by a single studio, but rather focused on fewer film projects controlled and organized by a producer who hired external crews and personnel specializing in specific tasks. It replaced the Unit-production system after WWII under the effect of the Paramount Decree. Unit-production system (see also Unit-package system and Paramount Decree): A system of movie production and distribution that involved massive film production by a single company which relied on its own technical or other resources. The same company also controlled the distribution and exhibition of films across the country with their privately owned theater chains, as well as with the method of “block booking,” i.e. forcing independent theater owners to prepay studios for the rights to exhibiting a specific number of films from that company. It was replaced by the Unit-package system after WWII under the effect of the Paramount Decree. Visual Atomism: A term used by Lev Manovich. It expresses the possibility to construct a complex message from a collection and assemblage of distinct 290

particles, the function and impact of which is known in advance; this way the response to the complex message can be predicted more easily, as each of the constituting elements will be understood by the receiver, who, however, is required to employ cognitive capacities to re-construct it mentally. Visual Momentum: The motivation and tendency that humans have to always look for new information in the environment and to combine it with existing information, so that they constantly re-assert their cognitive expectations towards making meaning out of what they see. 291

APPENDIX II - ILLUSTRATIONS

(Illustrations and videos are available in the print version.)