Mapping Past and Future Wars in Voice of the People: Experiencing Narrative in the 3D/VR Environment Institution: Florida Atlantic University

Mapping Past and Future Wars in Voice of the People:

Experiencing Narrative in the 3D/VR Environment

Ivan Kalytovskyy

A Thesis Submitted to the Faculty of the

Dorothy F. Schmidt College of Arts and Letters

In Partial Fulfillment of the Requirements for the Degree of

Master of Fine Arts

Florida Atlantic University

Boca Raton, FL

August 2017

Acknowledgements

I would like to express my sincere thanks to Dr. Joey Bargsten for his unrelenting support and encouragement throughout the writing of this manuscript. Additionally, I would like to thank professor Francis X. McAfee for his continued support throughout my undergraduate and graduate studies, his advice and orientation has inspired me to tackle any challenge. Additionally,

Bradley Lewter for his advice in navigating the challenges of this project. I am also grateful to Dr. Eric Freedman and Dr. Noemi Marin for providing me with teaching assistantship that greatly lifted the financial burdens of my education.

ABSTRACT

Author: Ivan Kalytovskyy

Title: Mapping Past and Future Wars in Voice of the People: Experiencing Narrative in the 3D/VR Environment Institution: Florida Atlantic University

Thesis Advisor: Bradley Lewter

Degree: Master of Fine Arts

Year: 2017

We have an opportunity not only to interact with 3D content but also to immerse ourselves in it via Virtual Reality (VR). This work is deeply inspired by my experience as a Ukrainian witnessing the recent turmoil in my homeland. I wanted people around the globe to experience the horrors that are unfolding.

The Voice of the People, explores narrative storytelling through VR. Ultimately, the viewer will be able to put on a VR headset and become deeply immersed in the story. With this technology, the user intimately experiences the war and devastation created by Russian occupation first hand. The end of World War II is a critical time in my country’s history. Included in the 3D environment are video segments of those affected by the current Russian occupation; historical themes underscore the narrative and help the viewer understand the recurring aggression by Russia that is part of Ukraine’s history.

Dedication

This manuscript is dedicated to my mother, my family and my friends.

Thank you for all of your support.

Table of Contents Part 1 Intro

Thesis Advisors and Boards Signatures ...... III

Acknowledgements ...... IV

Abstract ...... VI

List of Figures ...... VIII

Part 2 Critical Overview and Literature Review

I. Introduction ...... 1

II.History of Ukraine ...... 4

III. Treatment of Similar Themes / Ideas in Fiction, Cinema, and New

Media ...... 8

IV. The Hybrid of 3D and the Documentary: History and Fusion ...... 16

Part 3 Materials and Methods of Voice of the People

I. Overview ...... 28

II. Materials ...... 36

III. Methods ...... 42

Part 4 Conclusion

Conclusion ...... 46

Bibliography ...... 47

vii

LIST OF FIGURES

Figure 1: Map of Kievan Rus` ...... 4

Figure 2: Map of Kingdom of Halych-Volynia ...... 4

Figure 3: Map of Ukraine 1919 ...... 5

Figure 4: Map of Ukraine today (with Crimea) ...... 6

Figure 5: Lumieres camera ...... 17

Figure 6: Stereoscope ...... 18

Figure 7: Kinematoscope ...... 19

Figure 8: Sensorama ...... 19

Figure 9: Telesphere Mask ...... 20

Figure 10: Headsight ...... 20

Figure 11: Sega Genesis VR ...... 21

Figure 12: Nintendo’s Virtual Boy ...... 21

Figure 13: StuntMaster VR ...... 22

Figure 14: Cybermaxx VR ...... 22

Figure 15: VFX-1 VR ...... 22

Figure 16: Different virtual reality headsets offered today ...... 23

Figure 17: Game Walk – through Part 1 ...... 28

Figure 18: Game Walk – through Part 2 ...... 29

Figure 19: Game Walk – through Part 3 ...... 30

Figure 20: Game Walk – through Part 4 ...... 31

viii

Figure 21: Game Walk – through Part 5 ...... 31

Figure 22: Game Walk – through Part 6 ...... 32

Figure 23: Game Walk – through Part 7 ...... 32

Figure 24: Game Walk – through Part 8 ...... 33

Figure 25: Game Walk – through Part 9 ...... 34

Figure 26: Game navigation map ...... 35

Figure 27: USSR posters ...... 37

Figure 28: ZIl 130 ...... 38

Figure 29: Tank T-64 ...... 38

Figure 30: Soldier character ...... 40

Figure 31: Texture map of the building. (Displacement, Normal. Color, Gloss) 40

Figure 32: Texture map of a lightpost (Displacement, Gloss, Color, Normal) ... 41

Figure 33: Texture map of Zil-130 firetruck (3 color maps and 1 bump map) ... 41

Figure 34: Interviewed people. (Iruna, Muhailo, Ryslan, Roman) ...... 43

Figure 35: Sample of a JavaScript code...... 43

I. Introduction

My project, Voice of the People, is an interactive documentary. It explores the themes of remembrance, suffering, time, and humanity through a 3D interactive environment and through documentary video embedded in the environment. The user who enters my 3d environment will be immersed in the experiences of civilians who suffered and fought against military aggression from the Soviet Union, the Nazis, as well as Polish forces throughout the 20th century and into the present day. I will create a direct link between the stories of

Ukrainian civilian and the 3D user/viewer, using the Oculus Rift software to enhance the user’s experience of this interactive documentary.

I have always been interested in the newer technologies but have been mesmerized by the well-established media that we experience every day. After spending nearly fourteen years in the USA, I recently went to my native Ukraine, which stirred up in me many nearly-forgotten memories. Growing up in a country which was previously occupied by the Soviet Union, I found that things may not have been easy once the Russians left but at least things were peaceful. As a child I heard the stories of the elders about war, about the suffering and horrors of it, but for me it was always as a horror movie or a very dark tale of the

Brothers Grimm. When I returned to my hometown I noticed a different ambience in the people. Civilians are still going about their business and living casual day

-to-day lives although this conflict has been going on for over two years, but there

was just something “off” in people’s eyes and in their posture. It was as if they were afraid of what is to come. Many people have predicted that Soviet Union or today`s Russia will invade again and that they will never let Ukraine be at peace.

However, no one expected it to happen so soon. It was hard to get anyone to express his/her opinion and thoughts in front of the camera. Based the stories of our parents and grandparents, everyone knows what happens when you speak out against Russia. The options are to lose your life or go to Siberia. People closed themselves off inside their own heads and were afraid to speak up.

Everyone`s nightmares then became reality; everyone’s hopes for peace and prosperity were shattered. History is bound to repeat itself.

Introducing the aspect of interactive documentary - via 3D environments, virtual reality and documentary video - is the best way to realize this piece. I wanted to create a “fake environment” with 3D models using Autodesk Maya that provided me with a good base of models. During the texturing process, I tried to utilize mostly-procedural texturing using Substance Designer instead of creating the textures traditionally in Photoshop. . Implementing all of this into a 3D game engine Unity3D was the last part of the process to set up this reminisced, war- torn, manufactured environment.

While I was visiting Ukraine, I filmed the people and got real stories/experiences and implemented them within my created environment inside

Unity3D. War is fake, at least for the people who declare wars and don’t actually fight wars. They just send people to murder each other and in the end, call them heroes: a small group of people benefiting from the suffering of a vastly larger

group of people. Meanwhile the stories of the people who went through the war, the civilians who suffered through these horrible experiences, are as real as it can get. Implementing a manufactured environment of a war-torn city and displaying documented video of the people’s stories about the Russian invasions, truly displays my vision of the history of Ukraine.

II. History of Ukraine

Ukraine originally was known as

“Kievan Rus`.” Kievan Rus` played a

major role in shaping eastern Europe,

serving not only as cultural center but

also as the major political power. By

the 10th century Kievan Rus` reached

the height of its power and adopted

Byzantine Christianity as its religion

In 1240, the Mongol conquest ended

Figure 1. Map of Kievan Rus`

Kievan Ru`s and it became a Kingdom

of Halych-Volynia. Kingdom of Halych-

Volynia was one of the several most

important powers to emerge from the

collapse of Kievan Rus` in 1240. After

the Mongol invasion, in 1245 Pope

Innocent IV allowed price Danylo to be

crowned king. Under Danylo’s reign,

Kingdom of Halych-Volynia was one of

Figure 2. Map of the Kingdom of Halych- Volynia 4

the most powerful states in east central Europe (Subtelny 55). Over 80 culturally rich towns and cities were located in the Kingdom. From the 13th to the 16th

Century, Kiev was heavily influenced by Poland and Western Europe. In 1654,

Ukraine asked the czar of Moscovy for protection against Poland, and the treaty of Pereyasav was signed that year recognized the suzerainty1 of Moscow. The agreement was interpreted by Moscow as an invitation to take over Kiev, and the

Ukrainian state was

eventually absorbed into the

Russian Empire (Subtelny

123).

Ukraine remained

virtually unchanged until it

declared its independence Figure 3. Map of Ukraine 1919 from Russian Empire on

Jan. 20, 1918, after the Russian Revolution. Several years of warfare ensued with several groups. The Red Army finally was victorious over Kiev, and in 1920

Ukraine became a Soviet Republic. In the 1930s, the Soviet government's enforcement of collectivization met with peasant resistance, which in turn prompted the confiscation of grain from Ukrainian farmers by Soviet authorities; the resulting famine took an estimated 5 million lives. On April 26th, 1986, most likely due in part to the Soviet Union’s neglect of its conquered republics, the world’s worst nuclear accident took place in Chernobyl, the nation’s nuclear

1 Suzerainty - (/ˈsjuːzərənti/ or /ˈsjuːzərɛnti/) is a situation in which a powerful region or people controls the foreign policy and international relations of a tributary vassal state while allowing the subservient nation internal autonomy. 5

power plant. On August 24, 1991, Ukraine declared its independence from USSR

(Subtelny 510).On Oct. 29, 1991, the Ukrainian parliament voted to shut down

the reactor within two years' time and asked for international assistance in

dismantling it. The U.S. announced in Jan. 1994 that an agreement had been

reached with Russian and Ukraine

for destruction of Ukraine`s

nuclear arsenal, and swore to

protect the country under an

invasion, which was violated in

2014 with Russian forces invading

Figure 5. Map of Ukraine 2016 Eastern Ukraine and annexed by

Russian Federation. In November 2004, Ukraine experienced a revolution that is

well-known as the “Orange Revolution.” This revolution involved a powerful

civilian movement, a skilled political opposition group, and a determined middle

class that had come together to stop the ruling elite from falsifying an election

and hijacking Ukraine's presidency, which was ended in January 2005. On

February 18, 2014, Ukrainian people are emerging once more in Kiev to protest

against Ukrainian president, Victor Yanukovych, who believed to be, among the

people, as Russian Federation’s puppet (Sandbox Networks 2000). This resulted

in a series of violent events involving riot police, protesters and unknown

shooters. Ukraine lost control of couple of its own area and one of the biggest of

them was Krimea, which was a much-needed gateway to Black sea to Russia

(Thompson 2017). Today, Ukraine is at war with Russian troops trying to take back control over the occupied areas to restore peace and prosperity.

III. Treatment of Similar Themes / Ideas in Fiction, Cinema, and New Media

I have drawn many inspirations throughout the years from writers and filmmakers all over the world. War, violence, suffering and humanity are not new themes but they are deep topics of research and almost everyone views them in a different light. Some people see war as a necessary evil (“We make war that we may live in peace.” – Aristotle); some people view it as unnecessary (“My first wish is to see this plague of mankind, war, banished from earth.” – George

Washington); some have more poetic viewpoints (“War is peace. Freedom is slavery. Ignorance is strength.” – George Orwell). Perhaps Pope Francis best expressed one of the deeper meanings of war:

“Even today we raise our hand against our brother... We have perfected

our weapons, our conscience has fallen asleep, and we have sharpened

our ideas to justify ourselves as if it were normal we continue to sow

destruction, pain, death. Violence and war lead only to death.”

– Pope Francis.

While was calls forth as many feelings and emotions as those who experience it, it is the people whose voices are silenced, deprived or locked away in fear who can be heard and understood on a new, more intriguing, level through contemporary technology.

I have always been interested in the books and movies on war that spoke

out to me louder than any other genres, but there have been other novels in other genres that also became my favorites. While not unfolding in a war setting, Steinbeck’s The Grapes of Wrath communicates universal themes of dispossession, alienation, violence, extreme suffering and poverty, and ultimately, humanity.

The Grapes of Wrath is a story about farmer Tom Joad and his family during the Depression-era Oklahoma Dust Bowl. Tom’s family is forced to leave their farm and make a long journey to California along with thousands of others in search of jobs, stability and prosperity. Along the way, they face many hardships.

As William Shakespeare wrote in Hamlet, “When sorrows come, they come not single spies, but in battalions!”. In The Grapes of Wrath, a similar fate is awaiting

Tom’s family. Tom an ex-convict who wants to get his life back on track-- encounters reverend Casy and hears his philosophy concerning the holiness of human beings in general. When Casy is murdered by a policeman, Tom kills the policeman and repeats his mistakes with the law. As the family is falling apart and things seem to be at their worst, Tom’s younger sister, Rose of Sharon bears a still-born child. Just as the reader may think all hope is lost, Rose feeds a starving man which represents the full circle of human unity and suggests that there is hope even in a darkest of times and that humanity can be found even in the worst of circumstances.

This novel spoke to me because of its central themes of suffering and violence but also hope and humanity. The Grapes of Wrath was inspired by true events and presents the story of an average Joe and the suffering that he and his

family had to undergo. While it doesn’t take place during war, it is definitely a time of distress for the people. It was a time of hard decisions for them.

Everything they built had to be left behind and they could only look forward. The people were tested, which sometimes only revealed the worst in them. This novel makes us realize how cruel the world can be. It appears to be saying that most of our tragedies are caused by our own hands. The Dust Bowl was a period of severe dust storms that greatly damaged ecology and agriculture. It was caused by severe drought and the failure to apply dryland farming methods to prevent wind erosion. When something terrible like this happens, people lose their better judgment and their instincts for survival take over. People forget everything they have been taught and do not think about each other. Instead of working as a group, they become alienated from each other. Stories like this one have much to say in terms of educating a reader, not only about history but about humanity as well.

Screenwriters Robert Schenkkan and Andrew Knight documented the story of Desmond Doss, a remarkable non-combatant and conscientious objector in World War II. In 2016, Mel Gibson directedHacksaw Ridge, a movie based on

Desmond’s story (with Schenkkan and Knight as screenwriters). As The New

Yorker reviewer, Anthony Lane, said “Mel Gibson uses an implacably violent film to make viewers understand the faith of a Seventh - day Adventist who wants no part of violence at all.” (The Madness and Majesty of "Hacksaw Ridge" 2017).

This film successfully re-creates Desmond Doss’s war stories which concern the brutally horrific, yet inspiring, events of the battle of Okinawa.

The movie begins with a depiction of Desmond’s family who are heavily influenced by war. Desmond’s father, Tom, is a veteran of World War I. He is tormented by this war and the outcome of his fellow soldiers, often coping with his feelings by drowning himself in liquor and beating his sons. Then, when

Desmond and his brother, Hal, get into yet another fight, Desmond picks up a brick and hits Hal over the head. Desmond feels nothing but remorse for his actions and his mother, Bertha, tries to save Hal from dying. Because Desmond’s family are Seventh-Day Adventists, Desmond swears to himself that he will never do anything that might cause another human to die. When Desmond grows up, he falls in love with a nurse who influences him, teaching him the value of caring for people’s health and well-being.

When everyone is recruited to go to war, Desmond feels he cannot stay behind and let someone else protect his country. He enlists in the Army, but as a medic. When he goes through training, he excels in everything; however, he refuses to hold a firearm in rifle training. His personal rule causes him great trouble and he almost gets dishonorably discharged. However, after a long journey and many unpleasant events, Desmond joins a squad and heads off to the Battle of Okinawa that took place in 1945. Then, as soon as he climbs up to

Hacksaw Ridge, the movie starts to take shape. The scene set-up, the shot placement and props for the scenes are brutal and bloody. The viewer experience may be best described as devastating. We see burning flesh, flying body parts, disemboweled soldiers, and the abject fear in soldiers’ eyes as they experience this horrific event.

Director Mel Gibson was able to capture the realistic and horrific events of war, apparently not concerned if he went overboard and got too gory. Mel Gibson appeared dedicated to portraying the real experience of war and Desmond’s true story. As a viewer, I felt uncomfortable watching some of the scenes. At some points, I wanted everything to stop. After the movie, I was not left in a good mood, but this is exactly why I have chosen to include this movie in my work.

This movie dragged me into the story, made me cringe as I emphasized with the soldiers, gaining insight into their intense fear and distress. When Desmond tended to the wounded soldiers left behind--and still vulnerable to attack--in the field, it got me thinking about how good these men must’ve felt when someone came back for them. During his rescue mission, Desmond also encounters some stranded and/or wounded enemy soldiers on the field, and this is where the viewers’ faith in humanity is restored because Desmond decides to help out the enemy and attempt to save them as well as help his own men.

In the end of the movie, we see some of Desmond’s recorded interviews as he talks about these events. The director made some impressive choices and included many shots of events that were portrayed as close as possible to the way Desmond described them. This added depth to the movie and showed exactly how much of the story was based on true events.

This movie is a perfect example of how to make people stop for a second and think about the issues that we face. It shows not only how horrific human nature can be, but also how one man’s decision to help others can have a huge impact. This is a great movie to examine, not only in terms of how well it was

made but also in regard to its effective message.

Akira Kurosawa, born in Tokyo in 1910, was an influential internationally proclaimed filmmaker and director, producing his first film in 1950 (Ebert 2000).

After a long and successful career, he directed the movie Ran (1985), which many people consider to be his last great masterpiece. Ran is a movie about an old man, Lord Hidetora, who has three sons. After ruling over the lands for a long time, he hands down his title to his oldest son, while he banishes his youngest son because he speaks out and offers his father an honest opinion. Then, just as the father betrays the younger son by banishing him, the older son—strongly influenced by his wife, Kaede betrays his father by disrespecting his wishes and making him sign a document stating that the ruler is now is he not Lord Hidetora.

As the movie progresses, the oldest son dies in battle and the middle son rises as his successor. Kaede then immediately betrays her dead husband and transfers all her attention towards the middle brother and starts influencing and poisoning his mind. The old king loses his mind as he realizes then that the only person who actually cared about him, his youngest son, is the person who he has betrayed.

When youngest son shows up to the now new ruler—the middle son - and asks for a permission to retrieve his dad from the lands which are under control of the middle son. The middle son agrees but orders his men to assassinate his family as they reunite. The youngest son and father reunite, express their love, and then the youngest son is killed by the middle son’s assassin, which is then followed by the father suffering a heart attack and dying. In the last scene, Kaede

reveals her true self and confesses that her plan all along was to destroy the whole family and that she has succeeded. She is then killed as the middle son is preparing to die at the hands of the invading generals.

Ran has incredible cinematic effects, which are today mostly replaced by computer graphics, mainly because it is just much cheaper and more efficient; however, for a person who works heavily with 3D, it is astounding. Some of the techniques and other directorial choices are still found in today’s film. The scene with the fiery arrows flying into the room, for example, is very effective and it is incredible to think that it was done on a real stage. Kurosawa definitely set the bar for future directors to follow and to improve upon.

The movie Ran is a great representation of the story line that I prefer to avoid. Cinematically, the movie is great in terms of the choices of scene set ups, cuts, choreography, use of makeup. It is all used to create an enjoyable viewing experience. The story is where it falls flat, in my opinion. In my project, Voice of the People, I try to re-create the experiences of the people who are affected by the harsh decisions of the elite. Thus, in Ran, I would have taken a narrative stand concerning the farmers who had to get up and leave their homes because the powerful elite family was handling their personal disputes, not caring whatsoever about how it affects the people. Instead of dealing with the plight of the common man, the movie appears to want us to relate to the problems of a ruler and a conqueror of lands who then wants to have a peaceful retirement after many years of spilling so much blood. It is also hard to relate to the sons who are under the wings of their father and, when they get the taste of power,

they turn against the person who gave them everything and try to kill him. The viewer might try to imagine how it feels to be in these situations but can never come close because they haven’t walked even a couple of steps in similar shoes.

Again, this movie does not effectively concern itself with how vulnerable and disposable most common people are— nor does it address the damage they suffer at the hands of a small group of egotistic power-hungry rulers….

My interactive documentary with 3D elements can now bring the experience of war and occupation to the next level, not only by the participants’ direct access to the stories of the occupied and war-torn people, but also by the participants’ immersion in a 3D re-creation of the world of those most tragically affected.

IV. The Hybrid of 3D and the Documentary Break down of Voice of the People

Documentary is a nonlinear form of filming in which the primary goal is to document some aspect of reality, mostly for the purpose of education, instruction or maintenance of a historical record. In the mid-1970s, 3D graphics was something close to a miracle, as computer-generated images began to create worlds within a 3D space instead of 2D. Since the development of 3D graphics, the visual capabilities of the technology have only evolved and expanded. This has finally brought us to the current state of virtual reality (VR), which may now become a mainstream technology, offering deeper and richer opportunities to explore the both fiction and non-fiction.

Interactive documentary is a form of media art that is relatively unexplored. Most current examples are hyperlinked text linked to corresponding video. My vision of the interactive documentary merges the worlds of 3D and documentary to create a completely new and totally engrossing experience for the user. A 3D interactive documentary can do more than educate a person. It can also profoundly affect the way a viewer feels about the subject matter by immersing the viewer inside the experience using VR.

The documentary form began in 1895 when the first films were created by the Lumiére brothers. The Lumiéres created a camera that shot short, unedited clips, capturing the environment around them and recording it for about two to two-and-a-half minutes. These short clips were called “Actualites.” One of the

most renowned films “L'Arrivée d'un train en gare de La Ciotat” was made in 1895, and it simply shows a train pulling into the station at

La Ciotat (Cousins 73).

The word “documentary” was invented by John Grierson to describe a film “Nanook of The North”, made by Robert Flaherty in

1922 (Manovich, 78). This film was the first feature-length documentary and was described by Grierson as “the creative interpretation of reality.” Flaherty carefully Image 6. Lumieres camera placed his camera in every shot to make the film exciting and compelling to watch.

Grierson went on to head the GPO film unit in England in the 1930s. He emerged as a major ideologist of the poetic-realist approach to documentary. In

1936, Grierson’s “Nightmail” began as an informational film about the mail delivery train between London and Edinburgh, Scotland. This film emphasized poetic elements such as movement, light, sound and rhythm. Grierson was criticized for neglecting the political and social issues in his films in favor of a modernist approach, implying that he celebrated machinery over human beings

(Foxon 4).

In the 1950s and ‘60s, Direct Cinema took a stand as a movement that began in the United States and was focused on social and political issues. Films

were produced in a direct and immediate way to represent the recorded events exactly as they unfolded, without the involvement of the filmmaker. During that time, smaller and lighter cameras were available that could be hand-held, giving filmmakers more freedom. Meanwhile, a similar movement was emerging in

France called Cinema Verité, which translates to “cinema truth.” Cinema Verite shares with Direct Cinema its emphasis on the use of a hand-held camera, as well as direct sound, natural lighting, and location filming.

The use of Cinema Verite techniques can make the film seem more “real” and “truthful” to the viewer. However, some filmmakers started to use these techniques to create fake content while making it seem real. This form of filmmaking is called mockumentary. While filmmakers continue to use many of techniques and ideas of Direct Cinema and Cinema Verité to create contemporary documentaries, very few documentarians have used 3D technology in a pervasive and all-encompassing manner. 3D technology is ubiquitous in sci-fi and action movies, one reason why it is so interesting and innovative in a documentary setting, giving the 3D interactive documentary its current distinction.

Figure 7. Stereoscope Figure 6 Stereoscope

3D technology has a long and rich history

that can be traced back to the beginnings of

photography. In 1844, David Brewster invented a

stereoscope that could capture stereoscopic 3D

images. At the Great Exhibition of London in

1851, Louis Jules Duboscq used an improved

technology to take a stereoscopic picture of

Queen Victoria. It became one of the highlights of

the show which propelled this new technology to Figure 8. Kinematoscope the worldwide audience (Gunkel 47).

The technology of stereoscopic photography continued to evolve. In 1915,

a stereo animation camera was invented called

Kinematoscope(Figure 7), followed by the first

anaglyph2 movie in 1915. In 1922, the first

public 3D movie came out, “The Power of

Love” and in 1935, the first 3D color movie was

produced. During approximately the same

period, Stanley G. Weinbaum, had the vision of

what virtual reality should be and what its

Figure 9. Sensorama potential might be. He presented the term in

his 1935 short story “Pygmalion`s Spectacles”.

2 Anaglyph 3D is the name given to the stereoscopic 3D effect achieved by means of encoding each eye's image using filters of different (usually chromatically opposite) colors, typically red and cyan. Anaglyph 3D images contain two differently filtered colored images, one for each eye. 19

Technological

discoveries in virtual reality

were stalled until the 1950s.

During this decade, several

3D movies were created with Figure 10. Telesphere Mask Sensorama, the first attempt to create a VR experience. It featured a vibrating chair, speakers, fans, smell generators and, of course, a 3D stereoscopic display, all packaged in an arcade- style theater cabinet (Figure 8). This technology did not last long but many arcade games were inspired by its design and features (Goran 2016).

In the 1960s, the use of two projectors to display a 3D movie was obsolete with the invention of Space-Vision 3D. The first movie to use this technology was

“The Bubble” and it attracted colossal audience.

Many might think that head-mounted displays are a new idea but they have existed since the 1960s. The first of its kind was the Telesphere Mask.

Even though it featured wide vision with stereo sound it was not a precursor to modern virtual reality. In 1961 two Philco Corporation engineers, Charles

Comeau and James Bryan, came up with Headsight. It was a device very similar to a Telesphere mask, but it featured a magnetic motion- tracking technology that was similar to a close-circuit camera.

Unlike technologies developed for Figure 11. Headsight

entertainment purposes, Headsight was

developed for the military to immerse a

person via remote viewing in a

dangerous situation. This technology

was tremendously expensive (Goran

2016). Figure 11. Sega Genesis VR The first commercial VR devices

came out in the late 1980s but still were costing a fortune, EyePhone 1,

manuafactured by VPL, listed for $9,400. The EyePhone HRX which featured

720x480 pixel resolution with PAL and NTSC inputs and included studio-quality

Senheiser headphones and head tracking with optional Polhemus units, listed for

$49,000. Due to its high cost, impracticality and limited graphical experience, the

EyePhone series failed to take off in the market.

By the 1980s many movies in 3D were released, all thanks to the new 3D

technology developed by Stereovision. This new technology used a special

anamorphic lens that would widen the

picture using a series of Polaroid filters.

Then, in 1986, National Film Board of

Canada production Transitions, co-

directed by Colin Low and Tony Ianzelo

and created for Expo 86 in Vancouver,

was the first IMAX presentation using

Figure 12. Nintendo’s Virtual Boy polarized glasses (Grau 212). This new

technology promoted the use of a headset that was capable of color 3D stereoscopic vision and head tracking. This device was developed in the early 1990s, by the company Virtual I/O and was called iGlasses, predating the current naming convention of many products from Apple. Figure 13. StuntMaster VR

During the 1990s several other companies tried producing VR software and equipment, especially gaming companies. The first one was Sega’s VR

project in 1993, which was not a success

(Figure 11). Later, Nintendo joined the

competition with the Virtual Boy 3D

gaming console (Figure 12). These

devices did not take off in the consumer

market. Because they were lacked head- Figure 14. Cybermaxx VR tracking and quality graphics, and they only offered stereoscopic 3D display.

Virtual Reality and IMAX 3D really took off in the 1990s. Many movies were released for IMAX 3D most popular ones were “Into the

Deep” and “Wings of Courage”. While

Figure 15. VFX-1 VR IMAX 3D was a big push toward a

new era, an even greater development came within virtual reality. With the appearance of Cybermaxx by Victormaxx (Figure 14) and the Stuntmaster

(Figure 13), VR was truly immersive for the first time. Both technologies had very similar features, including a colorful stereoscopic 3D display, an LCD screen imbedded in a visor, and a tracking system, and both cost a little under a thousand dollars (Goran 2016). Even though they did not achieve the success many had hoped, they did help to push the research, culminating in the making of the VFX-1, in the mid-1990s (Figure 15).

This system (from Forte Technologies, Inc.) was one of the most capable virtual reality headset entering the market. It featured multi-axis head movement detection and rotation, stereoscopic 3D and the ability to play games that were not truly supported by the system. At a little over $600, the VFX-1 was much more affordable than any other systems, but the technology was ahead of its time and it didn’t really take off. Because of so many devices flooded the market in the 1990s, none of them really succeeded. The research on VR technology

Figure 16. Different virtual reality headsets offered today. (In order: HTC Vive, Facebook Oculus Rift, Sony PlayStation VR, Samsung Gear VR and Google Cardboard)

merely disappeared in 2000s and made space for personal computers, tablets, phones and 3D movies to take over this gap (Gunkel 2000). Finally, the major movie studios started to release movies in 3D, including some of the most known: “Spy Kids 3D: Game Over” and “The Polar Express”, which introduced consumers to the VR experience and resulted in a market-driven momentum toward 3D televisions.

By the 2010, many television channels were displaying animated shows, educational shows, sporting events, musical performances and even documentaries in 3D. This new decade re-booted virtual reality with the Oculus

Rift VR, which started as a Kickstarter project. It quickly grew in popularity and attracted the attention of thousands of consumers attention, this drove many companies like Google, HTC, Steam, Lionsgate and Samsung, to take notice and begin their own VR projects since Oculus VR was still in development. As of today, virtual reality headsets seem to be a huge success and are predicted to only continue growing as technology progresses and the headsets get cheaper.

Oculus Rift, among many, is still considered one of the most developed VR technologies of today but the market is flooded with the cheaper versions of the technology such as HTC Vive, Google Cardboard and the Samsung Gear VR.

As VR and interactive technology is further developed, more new and exciting ways to use it will be discovered, and new “symbolic forms” will emerge for its use. One of these new and creative forms is the interactive documentary.

“Interactive documentary” eludes definition, because the idea is relatively new and experimental. An exact definition has not been fully established, nor has

this new form produced a defining work. Here are some ways in which interactive documentaries have been described:

Any project that starts with an intention to document the ‘real’ and

that does so by using digital interactive technology – i-docs.org

A form of nonfiction storytelling or experience that gives the

audience an active role – Christopher Allen (UnionDocs)

It is not a genre, it is just another way of looking at documentary

storytelling. There’s an evolution of documentary storytelling in which

technology plays a big part – Arnaud Dressen (Honkytonk Films)

The form offers new ways of considering the role of the storyteller

in the world – Katerina Cizek (Highrise)

How digital media looks at the world of reality through an artistic

interpretation – Hugues Sweeney (National Film Board of Canada)

Interactive documentary, at this point of time (2017), can be created in various ways. One of the most common approaches is a website with hyperlinked videos either done in Flash or HTML5. One good example of this approach is “Becoming Human” (Origins 2008). Other documentaries include interaction through the keyboard, where key commands access different videos, images, and texts as in “The Most Northern Place”, which can be found on mostnorthenplace.com.

A more contemporary approach to the interactive documentary, imbeds traditional documentary films within 3D assets, to make the experience more immersive, interactive and entertaining. One of these experiences has been

created by Ubisoft in their game “Valiant Hearts: The Great War”. This puzzle adventure game, a studied depiction of World War I, examines how people coupe and deal with war. It is full of history and takes the player on a journey that is visual interesting, with interactive gameplay that is also educational.

The art direction of the game is also very interesting. While Valiant Hearts is rendered in an almost cartoony, 2D style, the terrors of war are in no way toned down or concealed. Gameplay involves five different people who have their own timelines and their own stories to reveal to the player. Some players might consider the game’s mix of fiction and fact a diminishment of the seriousness of factual events (Ubisoft 2014). Some players might also argue that the experience could be enhanced by using more realistic graphics and pairing them with VR, creating a story that is faithful to actual events, and making missions based on historical accounts. Nevertheless, Valiant Hearts is a great model for demonstrating how game designers can document events and allow players to relive history and connect to it. Games like Valiant Hearts that blend interactive and documentary forms are both entertaining and educational.

An experience that uses virtual reality to develop a new and unique form of storytelling is We Wait, based on BBC News interviews with migrants from

Syria. It opens at a nighttime beach where a group of migrants are about to go on their second journey over the Mediterranean Sea from Turkey to Greece. The characters are very blocky-looking and cartoony, and we don’t see much of the environment since it is almost pitch black. The darkness in this experience makes the viewer uneasy and tense; the quiet and the emptiness is unsettling at

first, but this seems exactly what a virtual reality experience was designed to do.

Overall, We Wait is a short demo (lasting only 8 minutes), but making this experience longer, introducing some backstory to the characters, and allowing the viewer to become attached to them could send a stronger message (BBC

2016). The experience of We Wait could be enhanced by better graphics, more interactivity, and including participatory events, maybe even recreating the exact look of the environment where these particular events happen. In spite of its brevity, We Wait remains a strong opening statement in the interactive 3D documentary form.

The interactive 3D documentary is only in its early stages of existence. Its continued evolution depends on a passionate and talented generation of artists to develop this form into something extraordinary.

I. Overview

The Voice of the People is an interactive 3D documentary, which was filmed in Ukraine with a purpose of telling the story about today’s war from the perspective of regular citizens, and it symbolizes themes of the passage of time, remembrance and suffering. This project was compiled within a 3D environment using Unity Pro for interactivity.

When a player puts on their Oculus Rift virtual reality headset, they will find themselves immersed in the 3D environment and placed inside a municipal fire-house. Within the fire-house there are multiple Zil 130 vehicles, which should indicator to the player that s(he) is located in one of the Soviet Union (USSR) countries. The Zil was a widely used and reused vehicle during the Soviet and post-Soviet areas, throughout the region controlled by the USSR. These trucks were used for anything from farming to government vehicles (with some minor

Figure 17. Game Walk – through Part 1. 28

changes), the biggest differences appearing in the paint and decals.

In the corner there is an old television set which when approached turns on and starts playing a video. In the video, there is a man talking about a displayed barrack in the middle of downtown L’viv. He mentions the war going on in Ukraine, how people donated money and food, and how people survive in the front lines of the conflict.

Once players exit the fire station, if they turn around they will see three words in Cyrillic type, “запобігти” which translates to prevent, “врятувати” which translates to save, and “допомогти” to help. These three word are commonly used in fire stations and they`re even printed on the fire fighters’ uniform. If players recognize the language they will know it’s written in Ukrainian and the experience is taking place in Ukraine.

Continuing the adventure, we can see a Greek Orthodox Church with golden roofs, a replica of a church in L’viv. On the right hand side of the church there are other streets and a road blocked off by tank stoppers, also known as hedgehogs. This should start suggesting this experience is taking place during a

Figure 18. Game Walk – through Part 2. 29

time of war. Next, we see a mini park on the left with a statue to Lenin and an old tram. The small red tram is a major representation of L’viv, as many paintings include this tram and it also serves as an iconic indicator of L’viv’s history. It has been used as a primary means of transportation for the working people and continues to do so even today. As we come closer we see that one window is pitch black and once we approach it the next documentary video starts playing.

In this will video, we meet Mikhailo Krusak, a young man in his early 30s.

He shares his thoughts on today’s Ukrainian war and the recent rebellions. He tells us about his limited involvement, mostly due to his wife and family who would not let him volunteer to fight. He conveys his sorrow for the friends he has lost.

Figure 19. Game Walk – through Part 3.

Figure 20. Game Walk – through Part 4. As we move on, we notice a soccer field on our right hand side. Since

soccer is the most played sport in Ukraine, one can find numerous fields like this

one all over the country. However, the scenery soon includes a little more

destruction over all and more war elements, such as an airplane with its engine

on fire.

The next stop, hidden in a little alley and up the stairs on the last floor will

be a harder to find. It seemed crucial for me to include an apartment building in

this story because I lived in one my whole life, as did everyone I knew. All major

Figure 21. Game Walk – through Part 5. 31

Figure 22. Game Walk – through Part 6. cities are made up of apartment buildings, since cars are now getting more common and people are moving to the villages for quieter life-styles. In this documentary video, we are introduced to an elderly woman, Iruna Radovich, who tells her stories of the USSR occupation during and after World War II. She shares how horrific it is to go through occupation and war again right now, and how to reawakens the horror and terror of her childhood.

As we walk further down the alley, we will encounter two tanks: a German

Panzer II Ausf. F., and T-64BM Bulat, mostly used by USSR. The Panzer is on

Figure 23. Game Walk – through Part 7. 32

fire representing the liberation of Ukraine from the Nazis by the USSR, but since the USSR immediately occupied this newly free country, the T-64 is aiming towards the Panzer to symbolize this historical moment (Figure 22). Behind the

Panzer we find a cemetery with many grave no flowers, no head stones and most importantly no names.

Inside this cemetery and behind the gates is a video of Ruslan Gyshpil, a local businessman who supported the troops by donating money, cars, and food.

He talks about how he sees Russian actions from his perspective as a father, businessman, and patriot. He discusses how he deals with employees who come back from the Anti-Terrorism Operation (ATO), and how war is beneficial to a corrupt government. He describes the horrific events he experienced at the

Maidan Nezalezhnosti (The Square of the Independence ) in Kiev, during both the Orange Revolution of 2004 and the Ukrainian Revolution of 2014.

On the other side of the square past the tanks, we see a war-town grocery store, sand bags, and some overturned light posts (Figure 25).

Figure 24. Game Walk – through Part 8. 33

Following a path on the left side we discover the final video, playing on an old park-movie projector. In this video we meet, Roman Chornobuy, a volunteer in the ATO. This young man tells us his story of joining the fight and volunteering along with his friends to protect what he believes in. He introduces us to first- hand knowledge of what exactly is going on at the enemy lines and he shares some of his views and opinions about the whole situation. One of the most shocking detail is belief that Ukraine is his country and his willingness to give his life for his beliefs, in spite of the fact that he himself is from Moscow and has a

Figure 25. Game Walk – through Part 9. family there.

Figure 26 is a navigational map, with red-dots indicating the location of videos, blue dots for points of interest, and a green line showing the fastest way to navigate this experience. I have chosen this nonlinear, exploratory layout as the best integration between illusory, fictional aspects of the city and the actual elements of L`viv architecture and its iconic transportation.

Figure 26. Game navigation map. I have used many inspirations, from books, movies and my own life, to create this experience. My main idea came to me when I traveled to Ukraine this past year and saw how people are afraid to speak out or to even share their own ideas. Voice of the People project was the best way I could think of making their voices be heard.

II. Materials

Voice of the People uses a 3D engine to re-create the setting of World

War II in Ukraine. The use of a 3D engine allows the viewer to be immersed in the time and place, using interactive virtual reality methods to further enhance the experience of the participant. UnityPro—the 3D engine I selected for this project--is populated with 3D models created in Autodesk Maya and textured using multiple software including Substance Painter, Photoshop and CrazyBump.

This combination of software allowed me to apply my skills to create Voice of the

People and shape it the way I wanted.

This long journey began with some sketches done in my sketchbook and the lengthy process of working through and editing my ideas. I have had the ambition to create some interesting short stories from my many ideas, inspired by

The Brothers Grimm and their tales. Since I was young I have always wondered why so few writers create truly sophisticated stories for children. I created and developed sketches for four stories during my first year of preproduction (i.e., searching for what I really wanted to do). My ideas changed but the central theme remained the same: it was always related to “time”. One of my stories conveyed how we should not fear the passage of time, but enjoy the time we have; another examined our time until death and our mortality; another concerned time repeating

itself; and the last story investigates wasting time. My stories were mostly based

in and around European cities. While working on what I thought would be my last

iteration of the project, I traveled back to Ukraine for the summer. During this

time, news and media has been blowing up about the war in Ukraine with Russia,

an imminent World War III, and other horrifying events. As a Ukrainian, this

worried me deeply but as time passed and I turned my attention to my own

worries and day-to-day issues, I started to worry less about Ukraine, and no

longer thought about it. Once I stepped into the L`viv airport and saw the military

there, I understood things had changed. I traveled to Ukraine almost two years

after the main aggression of 2014 broke out, yet I still noticed that people had

changed and had become distant. Many Ukrainians have lived during the USSR

occupation and during those times if a person spoke out again the USSR or

misbehaved they were sent to a Siberian prison or work camp. Younger people

aren’t old enough to have lived through that era, but they most likely have heard

terrifying stories from their parents or grandparents, and with all that had been

going on, I sensed the people I met were afraid it might happen again, that

history was repeating itself.

Figure 27. USSR posters.

Figure 29. Zil 130 To remind the people who forgot or never knew I have decided to create an interactive 3D documentary which will underline just how recent was Ukraine under USSR (Russian) control. I choose to create a look alike Ukrainian town and what it looked like during USSR occupation and include documentarian videos about the aggression in Ukraine today, tying the two timelines together. I have inserted many clues for viewer to find and identify to place themselves in

Figure 30. Tank T-64 38

space and time of the interactive documentary. Some of these things are acrylic letters, WWII USSR posters, Ukrainian words and letter on the building of the fire house and the store, manhole covers, vehicles, soldier uniform, etc. For almost every model I have created four different maps. These different maps help the light to act naturally upon the 3D object within the game engine, creating the illusion of realism.

Modeling 3D assets, creating UVs, texturing the assets and making them game-ready is only a small part of the process behind this project, but to me as an artist it is definitely the most appealing part. The next parts of the process involve importing the assets into the 3D engine, repositioning them, assigning the appropriate texture to the correct models, programming the interactivity and programming the graphical interface.

Figure 31. Soldier character.

Figure 32. Texture map of the building. (Displacement, Normal, Color, Gloss)

Figure 33. Texture map of a lightpost (Displacement, Gloss, Color, Normal)

Figure 34. Texture map of Zil 13041 firetruck (3 color maps and 1 bump map)

III. Methods

While creating the narrative and the ambience of this interactive 3D documentary, I wanted to stay away from making it feel like a videogame, per se.

I wanted it to feel like an experience, wherein the participant can walk a mile in someone else’s shoes, and experience the stories of real people, up close and personal. I am against video-game-like mentalities that have been glorifying war for a long time, such as games like Call of Duty. In my estimation, they’ve been selling the wrong idea of how conflicts should be handled, misleading players about what really happens in war, and lacking most representation of the genuine feelings people experience in war. The methods of creation of the games and interactive 3D documentaries are almost the same, the intention of creation is where the main difference lies. First, I had to find people to interview about their experiences. It was quite difficult to find people who agreed to be interviewed and to be recorded on camera. Each one had a great story to tell from a very different point of view based on gender, age and experience. These videos then had to be edited, translated and subtitled into English. In order to play these videos in Unity 3D. I had to create a special programming script to manage them, since Unity 3D only allows one video in the scene3 to pay at a time. This slowed down the performance of the executable Unity 3D file and I had to pursue

3 A ‘scene’ in Unity3D denotes a basic level or unit of an interactive experience. In Voice of the People. I was able to author the entire experience in a single ‘scene’. 42

Figure 35. Interviewed people. (Iruna, Muhailo, Ryslan, Roman) another option by playing each video only while the player is next to it (Figure

35). I placed the videos throughout my scene in Unity, trying to embed them as

seamlessly into the scene as I could, through old tv sets, posters, message

Figure 36. Sample of a Javascript code. boards and film projectors.

The next step was to create a player. At first, I created a standard first-

person controller and with the help of that particular controller I was able to do

most of the testing and positioning within my project. Once I started nearing the

completion of the project, I deleted my first-person controller and replaced it with the Oculus Virtual Reality (OVR) controller. The process of adapting to OVR was not easy. I had to watch multiple tutorials and do much troubleshooting for days to get this controller to work properly. Setting up the Oculus headset presented some bumps in the road, but once I started moving around in VR space, there was a major difference I noticed--the controls had changed. I have always noticed that most virtual reality games do not incorporate movement of the player; the player’s motion is usually limited to just looking around, maybe paired with some kind of teleportation system. The system I used featured a headset that was completely separate from trigger manipulation, so once a person turns to the left and presses ‘walk forward’ it will seem that (s)he is moving to the right.

While this creates more freedom for more intuitive controls, it is confusing and hard for players to get used to.

I have tried over and over again to fix the way the Oculus Rift controllers work, because it is nauseating when you are turning your body and head at the same time. For a human body, it is almost impossible to do this it makes the viewer dizzy. If we use the point- of-view as the direction we are walking towards, the dizziness is reduced greatly. Having the feature of in-depth controls is definitely refreshing but does not seem to have completely solved this issue.

The most tweakable and tedious job was to add the finishing touches to the experience, especially its visuals, through Image Effects. Not only must one be careful about how many affects you are applying (since each effect requires more processing power from your computer), but one must also tweak the effect

so it looks correct from every angle. Each one of the effects comes with plenty of settings and options; how one modifies them to get the exact desired look requires much time and experimentation. Since my scene has some indoor parts and some that are outdoors, it compounds the difficulty of applying Image

Effects.

Sometimes when one cannot achieve the look one desires, one will have to resort to programming. Programming is an entirely different skillset on its own, and one that cannot be easily mastered within a year. With my own limited experience in programming and within the timeframe I had to develop this experience, I was able to find much of my code online, or at least enough to identify the parts that I thought would work. Since I did not have a dedicated programmer on my project, I used trial-and-error to ultimately make my code work.

Conclusion

Setting up this Unity 3D scene and working on a project of this scale on my own has been a challenging and many times a seemingly impossible journey.

One of my greatest challenges was to actually come up with the exact idea I wanted to execute, and then modifying it accordingly. I have learned so much while creating this project—such as improving my modeling, manipulating code, learning Unity, its interface, and its limitations--and most importantly, I found what

I really do care about and would like to continue to develop: the interactive 3D documentary. As I neared the completion of this project, I took a step back and realized how much more I can do and how much more I can expand my approach to this form. In my future projects, I will want to make them more interactive and engaging, and to allow the players discover who they are, where they are and what is going on in a much more subtle, nuanced manner.

Like many of my colleagues, I am challenged by new technologies and the forms that are emerging from it, so I am planning to continue to explore and expand my ability to code and program. I know I will either need to expand my team or expand my project timeline in order to fully realize the potentials of this form with current technology, but I am confident that the exploration of the interactive 3D documentary will be my way of contributing to the field of new media.

Bibliography BBC. "We Wait." England: BBC Media Applications Technologies Ltd, 2016. Cousins, Mark. The Story of Film. London: Pavilion, 2013. Book.

Ebert, Roger. "RAN." 1 October 2000. rogerbert. http://www.rogerebert.com/reviews/great-movie-ran-1985. 30 May 2017.

Foxon, Steve. "John grierson." 2007. GriersonTrust. http://www.griersontrust.org/assets/files/articles/john-grierson-s-foxon.pdf. 3 June 2017.

Goran. VIRTUAL REALITY: A BRIEF HISTORY. 9 March 2016. http://www.useoftechnology.com/virtual-reality-history/. 4 June 2017.

Grau, Oliver. Media Art Histories. Massachusetts: The MIT Press, 2006. Book.

Gunkel, David. Rethinking Virtual Reality: Simulation and the Deconstruction of the Image. Gunkwell, 2000. Book.

Hohstaldt, Thomas. The Age of Virtual Reality . lulu.com, 2013. Book.

Interactive Documentary. 8 November 2011. http://www.interactivedocumentary.net/2011/11/08/haw-can-3d-world-be- used-in-documentaries-a-review-of-one-millionth-tower/. 17 April 2017.

Karatnycky, Andrian. "Ukraine`s Orange Revolution." April 2005. Foreign Affairs. https://www.foreignaffairs.com/articles/russia-fsu/2005-03-01/ukraines- orange-revolution. 28 Mach 2017.

Manovich, Lev. The Language of New Media. Cambirdge: MIT Press, 2000.

Origins, Institute of Human. Becoming Human. 22 December 2008. http://www.becominghuman.org/node/interactive-documentary. 12 April 2017.

PBS. The Most Northern Place. 10 February 2014. http://www.themostnorthernplace.com/story4. 27 April 2017.

Sandbox Networks, Inc. "Ukraine." 2000. Inforplease. http://www.infoplease.com/country/ukraine.html. 03 June 2017.

Stanley, David J. Computers, Visualization, and History: How New Technology Will Transform Our Understanding of the Past. Routledge, 2002.

Steinbeck, John. The Grapes of Wrath. New York: Penguin Classics, 1992.

Subtelny, Orest. Ukraine: A History. Toronto: University of Toronto Press, Scholarly Publishing Division; 4 edition, 2009. Book.

"The Madness and Majesty of "Hacksaw Ridge"." 7 November 2016. The New Torker. http://www.newyorker.com/magazine/2016/11/07/the-madness- and-majesty-of-hacksaw-ridge. 30 May 2017.

Thompson, Nick. "Ukraine: Everything you need to know about how we got here." 3 February 2017. CNN. http://www.cnn.com/2015/02/10/europe/ukraine-war-how-we-got-here/. 15 May 2017. Ubisoft. "Valiant Hearts." Ubisoft Montpellier, 24 June 2014.