1 ABSTRACT In this paper, I look at the history of the internet and online advertising. The internet is inextricably linked to capitalism and is fueled by advertising. As a result, companies like Facebook, Apple, Microsoft, Google, and Amazon collect data in large vol- umes to improve targeted advertising. An investigation of new power structures has emerged with the internet and how they dominate its and our future.

My creativity lies between art and technology. By merging new technologies like Artificial Intelligence with humor and graphic design, I try to shine a light on the subject.

Tutors Johanna Lewengard (main tutor), Moa Matthis (text tutor), João Doria (external tutor), Maria Wahlberg (external tutor).

2 INDEX

1. INTRODUCTION 1.1 Intentions 1.2 Research Question 1.3 On a Personal Note

2. BACKGROUND 2.1 The Fall of Soviet Union and the Birth of World Wide Web 2.2 Surveillance Capitalism and How Is the Data Collected? 2.3 Brief History of the Display Banner Ad 2.4 The Illusion of Online, How Our Phone Snitch Us Out 2.5 “’Each of us has heaven and hell in him, Basil,’ cried Dorian with a wild gesture of despair.’” 2.6 FAMGA

3. WORK PROCESS 3.1 Early experiments 3.2 Avatarify and Workflow 3.3 The Final Countdown 3.4 The Title 3.5 Spatial Design for the Exhibition

4. CONCLUSION 4.1 Exhibition 4.2 Examination 4.3 Reflections

3 1. INTRODUCTION 1.1 Intentions “I Want to Breathe You in: Data as Raw Commodity” aims to highlight the extreme 1 Shoshanna Zuboff, ‘The Age of Surveillance Capitalism’, means that companies like Facebook, Google, Amazon, Apple, and Microsoft use to 2019, p.9. collect personal information to predict and modify our behavior — what Shoshana Zuboff calls “surveillance capitalism.”1 The project is an exploration of the conse- quences it has on society today and in the future.

The project has a consistently norm-critical perspective, focusing on the new pow- 2 Yanis Varoufakis, “Yanis Varo- ufakis: Capitalism has become er structures that have emerged with the internet. The new world order that Yanis ‘techno-feudalism’ | UpFront,” Varoufakis calls “techno feudalism.”2 Al Jazeera English, February 19, 2021, 4:10 to 5:14, https://www. youtube.com/watch?v=_ jW0x- The project is designed as a spatial video installation. My aesthetics and methods UmUaUc. operate within “hybrid media” – where different techniques such as animations, 3D graphics, and graphic design together form a whole. I see my work as a part of the post-internet art movement, influenced by online meme culture as a tool for satire.

1.2 Research Question How can I awaken curiosity and interest in the subject of data collection and online surveillance by combining new technology methods such as AI with graphic design and humor?

1.3 On a Personal Note Before I started my master’s degree on Konstfack, I worked at an adtech startup spe- cializing in streaming video on display banners. Me coming from a graphic design- er’s perspective and being a fanatic ad-block user had the naive belief that banner ads were a necessary evil for newspapers to generate revenue from their online publi- cations. But it didn’t take long for me to realize that online advertising played a more significant role than that. The technology’s sophistication was more incredible than I had previously assumed. I felt stupid because I didn’t understand how the internet worked. After all, I spend a significant portion of my day on the internet and have repeatedly defended it as a tool for personal freedom. I can’t believe how clueless I was regarding Google’s position in the online ads infrastructure. Similarly, I felt stu- pid that I didn’t know that Amazon is also the world’s largest server supplier, owning roughly a third of the market share in 2021.

However, as time went on, my interest in and curiosity for banner ads started to grow. I felt glad that I got the opportunity to see how the sausage got made, a peek behind the curtains.

4 2. BACKGROUND 2.1 The Fall of Soviet Union and the Birth of World Wide Web The year 1989 was undoubtedly eventful—the opening of the Berlin Wall, which marked the beginning of the end for the Soviet Union and communism as state practice. Another change was the transition from government-controlled Keynesian economics to neoliberal Monetarist economics implemented by politicians such as Thatcher and Reagan throughout the 1980s.

The significance of the year 1989 was recognized at the time by Francis Fukuyama. 3Francis Fukuyama, ‘The End of History?’, The National Interest, The American scholar published an essay in the journal the National Interest enti- Summer 1989, https://www. tled “The End of History.” The statement of his thesis was: “What we may be witness- embl.de/aboutus/science_so- ing is not just the end of the Cold War or the passing of a particular period of postwar history, but ciety/discussion/discus- sion_2006/ref1-22june06.pdf the end of history as such: that is, the endpoint of mankind’s ideological evolution and the universal- ization of Western liberal democracy as the final form of human government.”3 For the last 150 years, the Marxist movement said the end of history would be communism, but as Fukuyama observed in 88/89 – when Gorbachov was reforming the Soviet Union, communism would not succeed in getting there and would stop at the stage before communism. Fukuyama at the time did not see a higher form of society that would replace liberal democracy.

After the fall of the Berlin wall in 1989, capitalism became the only imaginable mode of production. The concept that it is more manageable to imagine the end of the world than the end of capitalism originally came from American Marxist thinker Fredric Jameson but has been more widely spread by Slovenian philosopher Slavoj Žižek. In this new period, issues like an ecological disaster became much more pres- ent in people’s imaginations. Žižek displays that it is easier to imagine the world’s end than a much more manageable change in the mode of production, however radical it may be. The answer to this conundrum is the topic of the book Capitalist Realism, by Mark Fisher. Žižek and Jameson’s phrase is the opening point of Fisher’s book. The term “capitalist realism” itself is the mechanism by which capitalism embod- ies every aspect of life, making the mere potentiality of a shift to a different system unimaginable. Fisher says that this is especially true for the first generation born after the fall of the Berlin Wall; capitalism as praxis and ideology is enmeshed with every- thing. All of our desires, thoughts, emotions, and actions are capitalistic before we even have them or perform them.

4 “…a whole generation has passed since the collapse of the Berlin Wall. In the 1960s and Mark Fisher, ‘Capitalist Real- ism’, 2009, p.12. 1970s, capitalism had to face the problem of how to contain and absorb energies from outside. It now, in fact, has the opposite problem; having ail-too successfully incorporated externality, how can it function without an outside it can colonize and appropriate? For most people under twenty in Europe and North America, the lack of alternatives to capitalism is no longer even an issue. Capitalism seamlessly occupies the horizons of the thinkable.” 4

5 “What we are dealing with now is not the incorporation of materials that previously seemed Mark Fisher, ‘Capitalist Real- ism’, 2009, p.13. to possess subversive potentials, but instead, their precorporation: the pre-emptive formatting and shaping of desires, aspirations, and hopes by capitalist culture.” 5

5 In addition, Tim Berners-Lee, a British computer scientist working at the CERN laboratory in Geneva, outlined the building blocks for the World Wide Web in 1989. Which, as we know, would become the most valuable information and communica- tion instrument of the digital age.

As mentioned before, Mark Fisher’s thesis in Capitalist Realism says that every- thing after the fall of the Berlin Wall is impregnated by capitalism, which includes the World Wide Web. The internet is or has become a capitalistic product and has become its most excellent tool. It is the world’s biggest mall; it is also the world’s best salesmen who knows what we need when we need it (or rather what we do not need when we are at our lowest). It follows us from store to store; it knows what we have read, what images we have seen, whom we have met, whom we live with, where we live, etc. All this collecting of data to place a product in front of our face at the right moment; this is what Shoshana Zuboff calls “Surveillance Capitalism.”

2.2 Surveillance Capitalism and How Is the Data Collected? From the perspective implied by the concept of surveillance capitalism, people’s lives 6 Shoshanna Zuboff, ‘The Age of Surveillance Capitalism’, are an enormous data resource for creating behavioral archetypes. The so-called 2019, p.14. behavioral surplus gets fed into “machine intelligence” and fabricates into simula- tion tools that predict future behavior. Finally, these prediction resources get traded in what Shoshana Zuboff refers to as “behavioral futures markets,”6 a new type of marketplace for behavioral predictions.

Google was one of the first companies to understand the importance of the data 7 Shoshanna Zuboff, ‘The Age of Surveillance Capitalism’, they had, such as search queries, once regarded as “data exhaust.” Google first used 2019, p.51. the data to develop its product: the search engine. Zuboff refers to this process as a “behavioral value reinvestment cycle.” 7 However, Google did not benefit from their free service. According to Zuboff, the beginning of surveillance capitalism was behavioral data for ad-matching, which meant that the excess data was now analyzed to produce reliable search results and predict the greatness of ads. This practice dif- fers from traditional marketing, centered on educated guesses about what the target audiences might be. Using their users’ behavioral data, Google could align potential customers with relevant ads better and ultimately sell not just the space for adver- tising but also the most prosperous place for the advertisement. What might not be common knowledge is that Google is probably the most significant player globally when it comes to banner ads since they own the data, the platform and the technolo- gy where the transactions occur.

Everything we do is registered and saved. Every like, search, click, and even our be- haviors – for example, if we spend too much time focusing on a picture in our flow. All of this data adds up to form a profile of our behavior, which predicts and mod- ifies our future actions. This ‘data as raw commodity’ of behavior is obtained and claimed as property in the same way that land or natural resources are exploited.

2.3 Brief History of the Display Banner Ad In the mid-90s, advertising split the internet landscape in two. The venture capitalists saw the World Wide Web as a new medium, and advertising had always supported

6 media. If we look at newspapers, radio, and TV for the past hundred years, adver- tising has been the primary income. On the other side, we had the internet utopians, people who saw the web as a place of promised freedom. It was not a commercial environment. Even Google was against the idea of cluttering their product with ban- ner ads when they started. The worlds first banner ad for AT&T from 1994.

The first banner ad appeared on Wired’s online magazine HotWired.com on Octo- ber 27, 1994. It was a small ad that said, “Have you ever clicked your mouse right HERE?” with an arrow pointing to a call-to-action button located to the right of the ad. It was a part of AT&T’s “You Will” campaign, which included a series of televi- sion commercials featuring predicted scenes from an internet-enabled future.

Today’s targeted advertisements uses the HTTP cookies. Cookies are small pieces of data that are saved on a browser when a website is accessed. Programmer Lou Montulli invented the cookie in 1994, and its primary purpose is to store informa- tion. For example, to determine whether we are logged in or not and which account we are using. Without this feature, the site would be unable to determine whether or not it should authenticate itself. Additionally, it can store items added to the shopping cart and our browsing behavior, such as selecting specific items and carry out other valuable tasks that contribute to a positive user experience; that is, the task for the first-party cookies.

Then there are the infamous third-party cookies. Third-party cookies are created by domains other than the one we are currently visiting. This cookie makes it possible to provide advertising, retargeting, analytics and tracking services. Ad retargeting involves following visitors on websites throughout the internet, showing us advertise- ments for products or services we previously interacted with or viewed. Retargeting can take place via social media, display advertising, or email.

When a page loads, website owners place a 1x1 transparent pixel on their site, which sends a request to the ad retargeting server. The server then returns the requested information to assign the user a cookie and retarget them on other websites later.

Today when we visit a site and see a banner ad, that banner ad is a fractured digital version of what we are interested in – that banner ad chose because of our brows- ing history, demographic information, and geographic location. Real-Time Bidding, RTB, is an automated digital auction process that allows advertisers to bid on ad space from publishers. Like an auction, the highest bid from relevant ads will win the ad placement. The RTB process occurs in milliseconds before a website loads, and we will not even notice it happening. The highest bidder wins, and the ad becomes visible on the page.

The third-party cookie was able to carry a burden for which it was not designed, and in 2018, Firefox blocked third-party cookies on their browser. One year later,

7 Apples browser Safari followed the same path. In early 2020, Google revealed that third-party cookies would be phased out of Chrome before 2022. This announce- ment turned the whole industry on its head. How would the future for online adver- tising take form without the ability to track people across platforms and devices?

Google decided to give the long-awaited response in early March 2021; first-party 8David Temkin, ‘Charting a course towards a more priva- data will take the place of third-party cookies. David Temkin, Google’s director of cy-first web’, March 03, 2021, product management for ad privacy and trust, announced the following “Today, we’re https://blog.google/products/ making explicit that once third-party cookies are phased out, we will not build alternate identifiers ads-commerce/a-more-priva- cy-first-web to track individuals as they browse across the web, nor will we use them in our products.” 8 The first-party data will replace third-party cookies, which is perfect for Google; hence, they own and have access to mountains of data. Nevertheless, this solution has raised criticism; smaller publishers are afraid to be left outside of the walled garden of Google and think this will only benefit the big tech behemoths like Google, Amazon, Facebook, Apple, and Microsoft since they already have access to the first-party data. The future of targeted advertising seems to lead to having us logging in to each web- site we visit. Furthermore, we will probably use the accounts that we already have to identify ourselves, such as our Google-, Apple-, Facebook-account. Which will lead to us always being logged in, and every day, we continue to provide them with a tremendous amount of data.

The shape of online advertising to come is still undecided, but the third-party cookie reign is over, and it has been in decline for years, especially after the EU implement- ed the General Data Protection Regulation, GDPR. The GDPR served as a model for many national laws outside of the EU, including Chile, Japan, Brazil, South Korea, Argentina, and Kenya. The California Consumer Privacy Act (CCPA), which went into effect on June 28, 2018, bears many similarities to the GDPR. One of the many things the GDPR made companies do was notify the visitor that the website uses cookies and how these cookies are being used; it also allowed the visitor to opt- out of certain tracking. The GDPR made people aware of the cookie, and to some extent, its use of tracking. It also showcases that regulation does work. The big tech companies want us to believe that regulation against them will not be suitable for anyone; their industry is such a fast-moving industry that regulation would only hurt the economy.

Eric Schmidt, a former CEO of Google, insisted in a 2010 interview with the Wall 9 Lillian Cunningham, “Google’s Eric Schmidt Expounds on His Street Journal that Google did not need oversight because it had good incentives Senate Testimony,” Washing- to “treat its users right.” In 2011 Schmidt said to the Washington Post that he con- ton Post, September 30, 2011, curred with Andy Groves, former CEO of Intel, who had said the following “High http://www.washingtonpost. com/national/on-leadership/ tech runs three-times faster than normal businesses. And the government runs three times slower than googles-eric- schmidt-ex- normal businesses. So we have a nine-time gap... And so what you want to do is you want to make pounds-on-his-senate-testi- mony/2011/09/30/gIQAPyVg- sure that the government does not get in the way and slow things down.” 9

Business Insider covered Schmidt’s remarks at the Mobile World Congress the very 10 Pascal-Emmanuel Gobry, “Eric Schmidt to World Leaders next year. writing, “When asked about government regulation, Schmidt said that technology at EG8: Don’t Regulate Us, or moves so fast that governments really shouldn’t try to regulate it because it will change too fast, and Else,” Business Insider, May 24, any problem will be solved by technology. ‘We’ll move much faster than any government.’” 10 2011, http://www.businessinsid- er.com/eric-schmidt-google- eg8-2011-5. 8 Similarly, Larry Page, founder and former CEO of Google, has said the following 11 Jay Yarow, “Google CEO Larry Page Wants a Totally Separate regarding regulation and technology “Old institutions like the law and so on aren’t keeping World Where Tech Companies up with the rate of change that we’ve caused through technology.... The laws when we went public Can Conduct Experiments on were 50 years old. A law can’t be right if it’s 50 years old, like it’s before the internet.” 11 People,” Business Insider, May 16, 2013, http://www.busines- sinsider.com/google-ceo-larry- Zuboff says the following on big tech and their anti-regulation approach “It is import- page-wants-a-place-for-exper- iments-2013-5. ant to understand that surveillance capitalist are impelled to pursue lawlessness by the logic of their own creation. Google and Facebook vigorously lobby to kill online privacy protection, limit regula- tions, weaken or block privacy-enhancing legislation, and thwart every attempt to circumscribe their 12 Shoshanna Zuboff, ‘The Age of Surveillance Capitalism’, 12 practices because such laws are existential threats to the frictionless flow of behavioral surplus.” 2019, p.74.

2.4 The Illusion of Online, How Our Phone Snitch Us Out There is an illusion that the internet is a part of the computer or smartphone, that to be “online” is when you sit directly in front of your device and that it is an active choice to access it. Data collection has moved from the “virtual world” of embodied human experience into the “real world.” Before, the extraction only occurred online through inputs such as queries, likes, and clicks; now, data gets gathered in ‘offline’ situations as well; this means that the consumer is not to participate actively, that the sensors and devices are supposed to extract whatever data they can. The construc- tion that allows for an interface between the ‘real’ and the ‘digital’ worlds is known as ‘extraction architecture.’ Things such as smart TVs, smartwatches, security alarms, and refrigerators are examples of this; anything connected to the internet can collect data. When combined, devices form “the internet of things” or “the internet of ev- erything.” These devices are linked through the internet, allowing rapid data trans- mission and creating a comprehensive information network in global architecture.

When almost everyone has a smartphone, we are always online, and our personal information is getting gathered all the time. There are a few different ways that our phone keeps tabs on us and our location today; cell tower triangulation, GPS, Blue- Tooth, and Ultra-sonic-cross-device tracking.

If we have Bluetooth activated, our phone communicates identifiable information in all ways, including the model of our phone and our MAC address, which is a unique identifier baked into the Bluetooth chip on our phone. Anyone can pick up these signals that our phones send. Several retailers have begun installing “Blue- tooth beacons” in their locations to identify individuals without us knowing it. There are numerous locations in a society where these beacons can be found, including malls, museums, public transit, and billboards. Bluetooth beacons can not only find our position, but these beacons can also tell us how long we have spent at a specific location. This is invaluable for targeted advertising, allowing them to identify which products attract us and send us targeted advertising based on that.

We can also be tracked by something called “Ultra-Sonic-Cross-Device Tracking,” which uses inaudible, high-frequency tones to link our devices so that advertisers can better track us. The ultrasounds are embedded into television and radio commercials or even hidden in JavaScript in the ads displayed on the browsers. They are inaudi-

9 ble to the human ear, but microphones in our phones and other devices pick up the signals; this is how our phone listens to us, not by what we say, but by what all the electronics around us say to our phone’s microphone. There is currently no way of knowing which apps have ultrasonic tracking capability built-in.

2.5 “’Each of us has heaven and hell in him, Basil,’ cried Dorian with a wild gesture of despair.’” What we consume reflects our personalities; the things we acquire shape our indi- vidualities – the clothes we wear, the furniture we decorate with, the books we read, our music of choice. Nonetheless, these are the conscious choices that we recognize and perfect. On the other hand, a hidden consumer persona lives in a surveillance capitalistic shadow realm, a personality made up of the things we do not want to acknowledge, built on all of our secrets and bad habits. When all of our actions get recorded, so does the “bad” side as well. A sort of Dorian Gray’s portrait, a distorted reflection of ourselves, and if we could access that data in black and white, it would crush even the best of us. Jokes aside, Edward Snowden, the notorious whistleblower, has argued that privacy and anonymity are necessary for freedom of speech because it allows people to experiment with new ideas and opinions. According to a 2013 report by the United Nations Human Rights Council, citizens’ right to privacy is critical for maintaining active and free speech and expression in democracies.

On the subject of “if you have nothing to hide, you have nothing to fear,” Emilio Mordini, 13 Mordini “Nothing to Hide — Biometrics, Privacy and Private a philosopher, and psychoanalyst, argued that people do not need “something to Sphere.” p.257-260 hide” to hide “something.” According to Mordini, what is hidden may or may not be relevant. Instead, he says that an intimate space that can be hidden and restricted in access is required because, psychologically speaking, we become individuals when we discover that we can hide something from others.13

2.6 FAMGA Why have I chosen to work with online surveillance with a focus on corporations rather than governments? When Edward Snowden blew the whistle on NSA in 2013, it was clear that the US intelligence used any means necessary to gather in- formation on its citizens under the PRISM program. The leaked documents iden- tified several tech companies as PRISM program participants, including Microsoft, Yahoo!, Google, Facebook, Paltalk, YouTube, AOL, Skype, and Apple. One hand washes the other.

I have chosen to work with the top five big tech companies, Facebook, Apple, Micro- 14 Yanis Varoufakis, “Yanis Varo- ufakis: Capitalism has become soft, Google, and Amazon, which create the acronym FAMGA. According to Yanis ‘techno-feudalism’ | UpFront,” Varoufakis, an economist and politician, we no longer live in a capitalist society. Al Jazeera English, February 19, Instead, we now live in Techno-Feudalism, which he describes as “something even worse 2021, 4:19 to 6:22, https://www. youtube.com/watch?v=_ jW0x- than capitalism.” He states, “The concept of techno-feudalism refers to the lack of a competitive UmUaUc. capitalist economy. We have a small number of platform companies that effectively own the market; they do not just monopolize it; they own it. We are no longer in capitalism when we visit Amazon. com; instead, we are in a kind of one-person soviet dictatorship”.14 Amazon accounted for 49.1% of all US online retail sales and up to 90% in some categories. Apple has sold more than 1.2 billion iPhones and takes a 30% cut for every purchase made in their

10 app store. Google controls 71.98 percent of the online search engine market. Giving them the power to decide what search comes up for us, de-rank competitors, and promote their services. The more powerful these corporations get to be, the more difficult it becomes for new businesses to compete.

These companies do not just use the data they collect for targeted advertising. They also use it to map out the competition, see trends, and spot potential threats to their business. They either buy them out or steal their concept and use it in their product line. Like how Facebook wanted to buy Snapchat and, when turned down, imple- mented the same feature that made Snapchat famous in their product, Instagram.

These companies are the most influential globally, and they are young; the average age of the corporations is 32 years (Microsoft being the oldest, 1975 and Facebook the youngest, 2004).

This project aims to shine a light on the new power structures that have emerged with the internet. It focuses on the CEO of each company, Mark Zuckerberg (Face- book), Tim Cook (Apple), Satya Nadella (Microsoft), Sundar Pichai (Alphabet/Goo- gle), and Jeff Bezos (Amazon). Why? Because they, in a sense, know everything about us. All our action is being gathered and stored, and as far as we know, forever. They breathe us in.

11 3. WORK PROCESS 3.1 Early Experiments In the beginning, there were experiments; the first was an idea of having Mark Zuck- erberg’s eyes follow the visitor, using a coding language named Processing and an old gadget for the Xbox 360, the Microsoft Kinect camera. This camera, unlike other cameras, also uses infrared light to measure the depth of the images as well. Which means it can recognize and separate objects in the frame. So the idea was to have the camera recognize anything moving in front of it and have Mark Zuckerberg’s eyes follow the person on a TV screen. It worked, but the technology already felt done and overly used within graphic design and the post-internet art. Nevertheless, it was a good starting point. On the left is the eyes that follows the movement of a person moving, on the right is how the Kinect see the depths of the image, the red is me, the grey and black is the ceiling of konstfack.

After that, the plan of turning Jeff Bezos into a 60s cartoon araised after completing research on the world’s richest man. The idea was to create an autobiographical cartoon based on his life. This Napoleon like mad man who wants to take over the world, but in a world without feudalism, how do one rule and conquer all; by becom- ing the biggest capitalist capitalism have ever seen. This short animated story would be a part of several different video installations. My sketches of cartoon Jeff Bezos.

After creating a character design based on the American cartoon “Rocky and Bullwin- kle” and some basic expressions, the realization came that 2D animation and story- telling were not the project I intended to do. I also wanted it to link to data, so at this point, I started to explore an AI named StyleGAN2. By extracting 5000 images from different episodes from “Rocky and Bullwinkle” and “Fractured Fairy Tales” and imported them to the AI, the machine learning mechanisms started their process. By using the app RunwayML, the process was made more accessible. The process of machine learning with StyleGAN2 is that it goes through the process of trying to

12 learn the structures of it by comparing it to a trained model; in this case, faces. The result created a visual nightmare of multiple noses and eyes. The outcome of the styleGAN2 process from en extracted images from “Rocky and Bullwinke” and “Fractured Fairy Tales”.

Next up, deepfakes. The main machine learning methods used to create deepfakes are deep learning-based and involve training generative neural network architec- tures. Inspired by the diabolical Dr. Evil laugh from the Austin Powers movies, the idea was to have the big tech bosses laugh out of control. There is something so vulgar with men in power laughing, and Jeff Bezos laughs a lot. Evil laughter scene from Austin Powers (1997).

Walter Cronkite laughing with President Ronald Reagan and staff.

In the first attempt at creating deepfakes, I chose to put my face on Leonardo Di- Caprios when he won an Oscar for best male actor in 2016. It was a success. The process started with recording a 20 min video of my face. Edit the video that I wanted to become the output; in this case, Leonardos Oscars speech. Feed it into the machine learning application available for Windows; several nights later, the result was ready. However, the process of creating deepfakes is too time-consuming, and the end product can be fun; in early 2021, the creators of south park released a 15 min long video on youtube with a sassy reporter in a small town in, USA. A sort of mockumentary on deepfakes where the reporter has the face of Donald Trump. Nevertheless, in general, all deepfakes all the same; it is faces swapped, and when the result is not perfect, it is unpleasant.

13 My face on Leonardo DiCaprio, made with deepfake technol- ogy..

Sassy Justice with Fred Sassy.

Failed deepfake by me, Jeff Bezos on Lex Luthors body from Superman Returns (2006), a demonstration on how visually unpleasing it gets when it doesn’t work.

3.2 Avatarify and Workflow In the end, I settled for a technique named “Avatarify.” Avatarify is a way to create deepfakes, instead of having an AI learn the structure of the faces in the input- and output video, Avatarify does it in real-time, and it only needs a webcam and an image that we want to manipulate. Avatarify became famous in a recorded zoom meeting. The creator used his technique to prank his friends, pretending to be Elon Musk, who “accidentally” ended up in the wrong Zoom call. Co-founders Ali Aliev and Karim Iskakov wrote the app during the COVID-19 lockdown in April 2020. Ali spent two hours writing a program in Python to transfer his facial expressions to the other person’s face and use a filter in Zoom. The result was a real-time video, which could be streamed to Zoom.

Another thing that differs from deepfakes is that Avatarify rarely looks realistic and is very limited in what one can do. Suppose one moves too much, the illusion breaks.

14 However, when Avatarify breaks, it looks more visually pleasing. It becomes reminis- cent of funhouse mirrors.

For the 30 minutes December presentation, I made a try with this technique and tried to find workflow. Taking a picture off the internet of every CEO of the big five and googled “I want to breathe you in song” and took the first song that came up; it was a country song made by American artist Dierks Bentley and made a karaoke version with these CEOs singing. I added the lyrics to the video as well so the viewers could sing along. Frame from the Avatarify experiment for the 30 minutes December presentation.

The result was fun and seemed promising; however, the output file was small and low quality. The next step was to find a way to upgrade the quality. By using an AI named DFDNet, I could unblur and upscale the image. However, DFDNet is not meant to use for sequences, so I had to export the test clip as a PNG sequence. The result was mixed; on the one hand, the quality of one frame looked incredible; gone was the blurry, pixelated image, now it was clear and sharp. However, DFDNet works because it takes images and replacements every eye, nose, ear, and so on with another ear from its database. So when I put the images back to a sequence, it was almost as of the skin moved on the person portrayed. Nevertheless, I decided that it gave a somewhat pleasing effect and could probably damp it down in After Effects. Example on DFDNet, on the top is the blurry input images. On the bottom is the sharp output images.

Another thing that was not great with the actual output was the lack of frames that Avatarify produced, making the video act choppy. To solve that, I turned to AI once

15 again. With the help of an app named DainApp, With DainApp I was able to add frames. The way DainApp works is that it calculates what would happen between every frame and fills in the blanks.

3.3 The Final Countdown With these experiments, the decision was made; Avatarify was the route for me. I decided to merge my earlier idea of laughter into this technique instead. So I started recording a new video with me laughing, wearing their faces. I had cropped out the background in their pictures to green to easily remove it later like a green screen. And then, I repeated the process that I have had done earlier with my test clip. How- ever, this time, in much larger bulk, every process would take a longer time.

For the background, I choose to have a sky with clouds, and it serves for different metaphors. As a symbol for servers accessed over the internet, also known as “the cloud.” It also helps communicate the feeling that these people are untouchable, “the new gods” or “gods among us” concept with them in the sky. It is also fitting with the title, breathe you in, the sky as a symbol for air. The clip with clouds that I used was small in size, so I used another AI app named Topaz Video Enhancer, which uses AI to upscale the size of a video. I also added frames to it with the DainApp, and lastly, I added more vibrance and saturation to it.

The frame is there to communicate that we are all tangled up in this, the web, the interactivity. The first idea was to have the eyes included on the frame and make the frame in 3D and red. In the end, I thought it became too gruesome. So I decided to do more of a flat approach. I also felt like yellow made it pop more than red. And aslo, in a flat way, symbolize gold. The frame in 3D with eyes. Book cover for George Orwells book 1984 (1948).

The eye, also been, then in some way, from the very beginning. As a symbol, it is very in your face. In this context, it is surveillance, to be watched, the all-seeing eye, “there is evil that does not sleep, and the Great Eye is ever watchful.” The eye is made in the open- source 3D application Blender.

16 The feeling I have aimed for is “eerie”. I want the visitor on the exhibition not to know on whom expense the joke is. Are they laughing because they are happy, or are they laughing at us?

Scene from the film adaptation 1984 (1984).

High chancellor from the film adaptation of V for Vendetta (2003)

There is also a reference to big brother from George Orwell’s book “1984” and how the high chancellor is portrayed in Alan Moore’s graphic novel “V for Vendetta.” Both stories take place in a fascist dystopian surveillance state where you are watched at all times.

3.4 The Title The main title comes from leaked love messages from Jeff Bezos to his mistress, now girlfriend, Lauren Sanchez. After that, the leak happened after Lauren Sanchez gave print screens on the text dialogue to her brother, who later sold the messages to the tabloid newspaper National Enquirer. Print screen of Jeff Bezos leaked message, with the title underlined in red.

I knew early on that I wanted to use humor as a tool. To use humor to transform a relatively dull and complex subject as data collecting, to use satire to mock them. So I took that line from the text early on as a working title; it had that punch that I wanted. But as time went on, I started to like it, even if it is a bit Bezos heavy, so I decided to keep it.

3.5 Spatial Design for the Exhibition I see the spring exhibition as a unique opportunity for me to work spatially with my practice. By letting different components work together in the installation, I want to create a thoughtful overall impression that captures the visitor’s attention.

An early inspiration for the spatial design for this project is Jordan Wolfson’s instal- lation “riverboat song.” It influenced me early on how I imagined the end product would look like on the exhibition. Another inspiration is how the research and design

17 studio Metahaven works with spatial design. Metahaven has worked with similar past subjects, especially with their book “black transparency” from 2015.

An important point for me is to break the illusion that data gathering only occurs when a person is “online.” To do that, I wanted to take some of my work into the physical space. At first, I wanted it to be a rug, with the help of the tactile properties of textiles, to visualize how the abstract digital world is constantly present. I wanted to develop my world of images further and explore my expression when I move my Jordan Wolfson Riverboat song.

Exapmle on how Metahaven works with spatial design.

aesthetics to another medium. However, I could not rationalize a choice in the de- Early render of the design for the exhibition with the ”sic semper tyrannis”-rug.

Different sketches for the rug.

signs I came up with; it felt forced. The design of the carpets stretched from custom lettering “sic semper tyrannis” (“thus always to tyrants”) to trying to visualize and make a comment on data as the oil.

In the end, I decided to ditch the rug and work with the screen and all its sides. I also decided to take the sky and clouds out of the video and use that element to decorate with.

18 4. CONCLUSION 4.1 The Exhibition The exhibition went well, and I am pleased with the result of the spatial design; as I mentioned before, I chose to work with the clouds as a decorative motive on the big box and the smaller box, which had the function of hiding the projector and to be a place to store cards with my contact information. The clouds were created with a sponge and going back and forth with blue and white color. The larger box had the dimension 16:10, and the image covered the surface well.

The response from the visitors was good enough. Some got a crooked smile when they saw the video installation and commented, “it’s a hot topic.” Most went in and then out again. I think one man’s response summarized the general response well; he went in, saw Mark Zuckerberg, and made the acknowledgment “oh yeah, it’s that guy” and went out again.

19 4.2 The Examination I was not as prepared as I wanted to be on my examination; after building the exhibition, I felt like I had fallen out of the theory regarding my project. When the discussion started, I did not feel confident in my answers, and I could not answer my opponent Johanna Burai’s thoughtful questions as well as I wanted.

Johanna Burais read of my installation leaned very heavily on Christian imagery and lore. Which it is, I played with the idea that they see everything, and they are the prophets of our new religion; technology.

Johanna also said that she got scared the first time she saw the video when it transi- tioned from the prelude text to Mark Zuckerberg’s face.

During the discussion, two audience members asked about the effectiveness of hu- mor as a tool, specifically whether it is counterproductive and diverts attention away from the topic. Additionally, how effective are satire and humor at communicating ideas and information? To respond to their questions, there is a time and place for humor depending on the subject, but for me, it is more about time. To my mind, it works extremely well to be introduced to a complex concept or a heavy subject through the use of satire. I attempted to use humor in this project to pique interest in this subject. Additionally, satire and humor are two of the few tools available to us in this hierarchy of power structures between someone like me and Jeff Bezos.

4.3 Reflections A significant portion of this project was spent researching and viewing the internet in a new light to better understand its history and its impact on the world. This has been the most exciting part of the project for me, and I have had some great realiza- tions as a result of it.

20 The first research question I had starting with this project was in the line“is my personal information worth giving up to get access to services on the internet for free?” and to answer that now, I would say yes. However, the means and methods will worsen, and I can see us starting to self censure ourselves. Worth pointing out is that I live in a democratic society with the right to opinions that differ from the state but imagine if someone like Stalin had access to this kind of technology. Alternatively, imagine that our society changes to an authoritarian regime and the state wants to seak out unwanted citizens.

The end product was pretty close to what I had envisioned when I began the project. Some elements were clear from the start, while others took time to develop. I am pleased with the completed installation.

One of my goals for my master’s degree studies at Konstfack was to experiment with new technologies and integrate them into my work as a graphic designer. I gained knowledge of how both software and hardware work as a result of this project.

21