University of Hawai´i at Mānoa BlackBox.Academy: An Educational Website, from Racial Bias and Digital Panopticism to the Technological Open Mari Martinez Performance Studies, MA Dr. Markus Wessendorf (Committee Chair) Dr. Jonathan Goldberg-Hiller Dr. Jason Leigh May 4, 2021 Martinez, Mari Page 2 of 65 Contents Abstract 3 What is the Black Box 3 Racial Bias 5 The Algorithm 11 The Panopticon 17 Mis/Disinfo 29 The Open 43 Martinez, Mari Page 3 of 65 ABSTRACT BlackBox.Academy is an educational website which helps people with minimal or greater technological backgrounds learn important topics in technology, ranging from racial bias to disparity caused by algorithms, from surveillance to digital panopticism, to mis- and disinformation, even ontological questions of technology. The goal is to educate a wide variety of people from multiple backgrounds by breaking down concepts without the use of technological jargon and academic speak. This is done with embedded video clips, hyperlinks, and concepts that gradually build on one another. The website format allows for real time changes and up-to-date information. This PDF version is the written part of a more elaborate, complex, and interactive website that can be found at https://blackbox.academy. What is the Black Box? Technology is this monolithic word that encompasses everything from pencils to medicine, and from toaster ovens to artificial intelligence. So, what do I mean when I say technology? And what's all this talk about a Black Box? The technology that I'm talking about is modern technology, the everyday stuff that we use – computers, smart phones, software, the things we have invited into our home to make our lives easier. I will speak about artificial intelligence and algorithms, but I'll let you know when I do. The black box metaphor started way back in the days of cybernetics and refers to “a system we can only observe the inputs and outputs, but not the internal workings” (Card). Martinez, Mari Page 4 of 65 So, what's this Black Box all about? Think about technology having a background and a foreground. The foreground is what you interact with. The app you click on, the screen that you see, that sort of thing. The background is all the stuff that's working behind the scenes to make that app work, or that screen appear, or your computer work. It's all the sorcery and wizardry that most people don't understand (or care to), that makes all the bleeps and bloops, and all the lights happen and the technology that you're using for work. The Black Box is the background. When I talk about it, if necessary, I will talk about it in those terms: the foreground (the stuff you interact with), and the background (the stuff you don't see that makes it work, everything inside the box) or the black box. This text looks at the black box through a performative lens. What does that mean? Richard Schechner sums it up best in his book Performance Studies: An Introduction: “(DL) Austin notes, "To say something is to do something." In uttering certain sentences people perform acts. Promises, bets, curses, contracts, and judgments do not describe or represent actions: they are actions. Performatives are an integral part of "real life." As many have found out too late, even if the heart says "no," once the tongue says "yes" the performative binds.”1 The same is true with technology; but how does what we say with our technology, what we program our technology to say and do, and what our technology tells others about us, influence and impact our world? It is one thing to think about these things in the foreground of technology, who we text, what we tweet, etc. – but what about the background? How does the information we input into the technological black box impact these actions? How does “to say 1 Richard Schechner. Performance Studies: An Introduction. 4th ed., Routledge, 2020. Martinez, Mari Page 5 of 65 something is to do something” impact us if we can’t see who or what is uttering the words that influence our actions? RACIAL BIAS Why is this important? Why does it matter if we look inside the black box? You may be asking yourself, why does it matter how these things work? Why does what's inside of them matter? Do I really need to know how my TV works in order to watch it? Does it really matter what's inside my smartphone or who programmed TikTok, Instagram, or Twitter as long as I can access them? Many of us have used public restrooms. Many of us have used public restrooms with automatic sensors on the sinks, the ones where you put your hands in front of the light sensors and it activates the water to wash your hands. My partner and I have two completely different experiences when we encounter these, and it is no accident. I also have different experiences depending on the time of year. As a Mexican woman, if it is the middle of summer, and my skin is much darker, it is almost impossible for me to activate one. I ALWAYS have to use the palm of my hands. My partner, who is white, has little difficulty activating the sensors. I know what you're thinking, there's no way that the sink I'm trying to use has any sort of racial bias. This sink can't possibly be racist, it's in inanimate object, a harmless piece of technology, dare I say, neutral. But is it? In 2015, this idea made headlines when a viral video made its rounds on social media after two friends at DragonCon were in a Marriott hotel bathroom (Plenke). One white, who had no problem making the light sensor work and triggering the faucet, and the other, a black man, who could not get the sensor to activate. The video was Martinez, Mari Page 6 of 65 titled "Whites Only," a throw-back to Jim Crow segregation and raised the question of whether or not we need a closer look at who is behind, and what is inside our technology. “Whites Only?” Post by TeejMaximus September 2, 2015 Retrieved from YouTube in December 2020 This may seem frivolous, it's just a soap dispenser, but it becomes even more important when these technologies are unable to recognize people. No longer a thing of the future, every major city in the United States and countries all over the world use biometrics; from tracking citizens on city streets to using it in law enforcement. However, these technologies have been shown, time and again, to not work, and they have high inaccuracies for people of color and women. This technology has been massively rolled out worldwide in countries like the United States, the United Kingdom, and China. Facial Recognition Biometrics: These are physical characteristics like fingerprints, facial recognition, voice, and eye or retinal recognition that can be used to identify who you are. They can be used to unlock your phone, to access bank accounts, open doors, even identify you for crimes (Gillis). Facial recognition is being widely rolled out in major cities across the world with little oversight. Many studies have shown that facial recognition technology is not only biased, but Martinez, Mari Page 7 of 65 that in many circumstances it inaccurately identifies people of color, especially women (Al Jazeera, Rae). (AlJazeeraEnglish, & Rae, A. (2019, July 03). Do Biometrics Protect or Compromise Our Security? | All Hail the Algorithm. Retrieved from YouTube on December 2, 2020) Biometric Technology & Vulnerable Populations In areas where people have minimal rights, like refugee camps, biometric technology is being used to collect data. How should this technology be used in these spaces? Stephanie Hare discusses whether or not it is ethical to use it with groups of people who have limited rights or are not in a position to refuse the use of this technology and collection of such intimate data. Stephanie Hare: “Ethics of Biometrics & Vulnerable Populations” AlJazeeraEnglish, & Rae, A. (2019, July 3). Do Biometrics Protect or Compromise Our Security? | All Hail the Algorithm. Retrieved from YouTube on December 2, 2020. How is this possible? How does technology show racial bias? These are just a few examples of the more obvious racial bias that are present in technology. We can begin to see how these technologies directly impact how we behave (or perform) in the real world, but who or what is dictating our behavior from inside the black box? How did this racial bias get there, or maybe you already know, or have some idea. When thinking about racial bias in technology we need to think about who is making the tech. Martinez, Mari Page 8 of 65 Who makes the technology? The tech industry is predominantly male and white. This isn't to say that the technology is intentionally made not to recognize darker skin, this is to say that the people writing the code, the people making the product, testing the product, and doing quality control are predominantly white men. There aren't people of color to step in, to find racial blind spots, so racial bias gets programmed into the system. When a technology like facial recognition is being tested, if it is only being trained on white, male faces, a bias gets programmed (intentionally or not) into the system. Representation matters, from coding to quality control. Representation matters every step of the way, especially when that technology is helping to put people behind bars. Bias in the Justice System More and more, algorithms are being used in the criminal justice system, from sentencing to parole boards.