Cameras in the Classroom Facial Recognition Technology in Schools
Total Page:16
File Type:pdf, Size:1020Kb
Cameras in the Classroom Facial Recognition Technology in Schools Claire Galligan Hannah Rosenfeld Molly Kleinman Shobita Parthasarathy TECHNOLOGY ASSESSMENT PROJECT REPORT Contents About the Authors 5 Exacerbating Racism 30 FR Accuracy Varies by Race 31 Disproportionately Targeting 31 About the Science, 6 People of Color Technology, and Public Adverse Psychological and Social 35 Policy Program Impacts Acronyms 7 Normalizing 38 Surveillance Technology Expanding the Reach 39 Executive Summary 8 of Surveillance The Implications of FR in Schools 9 Students Feel Powerless 41 National and International Policy 12 Landscape Recommendations 13 Defining the Acceptable 43 Student Reclassifying Expressions of 44 Introduction 17 Individuality How Does the Technology Work? 17 Further Marginalizing Vulnerable 44 FR’s History 19 Groups The FR Landscape Today 20 Altering Behaviors to Avoid 45 Surveillance School Security and FR 22 Analogical Case Comparison: Our 23 Analytic Approach Analogical Case Studies 26 Overview of the Implications 28 Creating New 47 National and 69 Commodities and International Policy Markets Landscape Companies Will Expand 48 Bans and Moratoria 69 Surveillance to Gather Data Consent and Notification Policies 71 Once Collected, Data Can Be 49 Data Security Policies 73 Repurposed Policies to Tailor Use 74 Do Citizens Own Their Data? 50 Oversight, Reporting, and 75 Data Authorization Focuses on 52 Standard-Setting Policies Consent, Not Ownership FR Expansion Without Regulation 76 Limited Data Protections Put 54 Privacy at Risk Regulating FR in Schools 77 Institutionalizing 56 Inaccuracy National and 79 International FR is Inaccurate Among Most 57 Populations, Including Children FR Policy Maps Humans Make Final Matching 58 Determinations Systemic Discrimination Feeds 60 Our Facial Recognition 83 Back Into Surveillance Technology Future Maintaining Accuracy Requires 62 Sustained Resources and Training Difficulty of Assessing Accuracy in 64 Recommendations 85 Preventing Low-Probability Events National Level 86 Local Officials Must Determine 64 State Level 88 Accuracy Among Heterogeneous Products School and School District Level 89 Limited Regulation of Surveillance 65 Technologies Acknowledgements 92 Courts Become Ultimate Arbiters 66 of Accuracy Excitement Over a Technical Fix 67 Leads to Entrenchment References 93 Further Resources 110 Appendix A: 111 Questions for School Administrators and Teachers to Ask Facial Recognition Companies Appendix B: Questions 112 for Parents, Guardians, and Students to Ask Schools and School Districts For Further Information 114 CAMERAS IN THE CLASSROOM: FACIAL RECOGNITION TECHNOLOGY IN SCHOOLS About the Authors to experience in research, regulatory graduated Phi Beta Claire Galligan compliance, and program and technology Kappa from the University of Michigan in May evaluation, she has developed diagnostic 2020 with a BA in Public Policy and a minor in tools that leverage machine learning and Economics. Her research and policy interests computer vision. She has also worked with include technology, trade and economics, university officials and security teams healthcare, and foreign policy. She has held around the country to develop responsive internships in Congressman Dan Kildee’s emergency technology. She is on the (MI-05) Washington, DC office and on TD Foretell Ambassador Board for technology Bank’s Government Relations team. During forecasting with Georgetown’s Center for her time at the University of Michigan, she Security and Emerging Technology (CSET) worked as the President of the Michigan and formerly led the New York City chapter Foreign Policy Council, an on-campus foreign of the LGBTQ+ non-profit Out in Tech before policy think tank, where she helped produce a becoming the Head of Diversity, Inclusion, biannual journal of original student research and Belongingness for the international and pioneered a professional development organization. mentorship program where students were paired with professional researchers at the Brookings Institution to receive research Molly Kleinman is the Program and writing advice. Claire will soon be an Manager of the Science, Technology, and Associate with Kaufman Hall & Associates on Public Policy program at the University of their healthcare consulting team (based in Michigan, and a lecturer at the University Chicago). of Michigan School of Education. She studies higher education policy, access to information, and faculty experiences with is a Hannah Rosenfeld technology. Molly spent several years as Master of Public Policy student at University an academic librarian at the University of of Michigan, where she is in the Science, Michigan, and served as program manager Technology, and Public Policy and Diversity, for clinical and basic science education at Equity, and Inclusion graduate certificate the University of Michigan Medical School. programs. She holds a BA in Biology from Molly received her Ph.D. in Higher Education University of Virginia. Hannah worked Policy from the University of Michigan Center in the tech and tech regulation industry for the Study of Higher and Postsecondary for over 7 years on education technology, Education, with a certificate in Science, medical devices, and personal security, Technology, and Public Policy, her MS in including at Luidia and Oculogica. In addition UNIVERSITY OF MICHIGAN TECHNOLOGY ASSESSMENT PROJECT 2020 5 PDF: Click to return to top CAMERAS IN THE CLASSROOM: FACIAL RECOGNITION TECHNOLOGY IN SCHOOLS Information from the University of Michigan two books: Building Genetic Medicine: Breast School of Information, and her BA in English Cancer, Technology, and the Comparative and Gender Studies from Bryn Mawr College. Politics of Health Care (MIT Press, 2007) and Patent Politics: Life Forms, Markets, and the Public Interest in the United States and Europe Shobita Parthasarathy (University of Chicago Press, 2017). She has is Professor of Public Policy and Women’s advised policymakers in the United States and Studies, and Director of the Science, around the world how to regulate emerging Technology, and Public Policy Program, at the science and technology in the public interest. University of Michigan. She conducts research She is a non-resident fellow of the Center on the ethical, social, and equity dimensions for Democracy and Technology and sits of emerging science and technology and on the advisory board for the Community associated policies, as well as the politics of Technology Collective. She writes frequently evidence and expertise in policymaking, in for the public and co-hosts The Received comparative and international perspective. Wisdom podcast, on the relationships between She is the author of multiple articles and science, technology, policy, and society. About the Science, Technology, and Public Policy Program The University of Michigan’s Science, certificate program, postdoctoral fellowship Technology, and Public Policy (STPP) program, public and policy engagement program is a unique research, education, activities, and a lecture series that brings to and policy engagement center concerned campus experts in science and technology with cutting-edge questions that arise at the policy from around the world. Our affiliated intersection of science, technology, policy, faculty do research and influence policy on and society. Housed in the Ford School of a variety of topics, from national security to Public Policy, STPP has a vibrant graduate energy. UNIVERSITY OF MICHIGAN TECHNOLOGY ASSESSMENT PROJECT 2020 6 PDF: Click to return to top CAMERAS IN THE CLASSROOM: FACIAL RECOGNITION TECHNOLOGY IN SCHOOLS Acronyms APPI Act on the Protection of Personal Information (Japan) BIPA Biometric Information Privacy Act (Illinois) CCTV Closed-Circuit Television DARPA Defense Advanced Research Projects Agency (United States) DPA Data Protection Authority (Sweden) EEA European Economic Authority FBI Federal Bureau of Investigation (United States) FDA Food and Drug Administration (United States) FERET Facial Recognition Technology Program (United States) FERPA Family Educational Rights and Privacy Act (United States) FR Facial Recognition FTC Federal Trade Commission (United States) GDPR General Data Protection Regulation (Europe) K-12 Kindergarten through High School NCES National Center for Education Statistics (NCES) NIH National Institutes of Health (United States) NIST National Institute of Standards and Technology (United States) SHIELD Stop Hacks and Improve Electronic Data Security (New York) SRO School Resource Officer TAP Technology Assessment Project TSA Transportation Security Administration (United States) UNIVERSITY OF MICHIGAN TECHNOLOGY ASSESSMENT PROJECT 2020 7 PDF: Click to return to top CAMERAS IN THE CLASSROOM: FACIAL RECOGNITION TECHNOLOGY IN SCHOOLS Executive Summary Facial recognition (FR) technology was long that they raised. This method enables us considered science fiction, but it is now part to anticipate the consequences of using of everyday life for people all over the world. FR in schools; our analysis reveals that FR FR systems identify or verify an individual’s will likely have five types of implications: identity based on a digitized image alone, and exacerbating racism, normalizing are commonly used for identity verification, surveillance and eroding privacy, narrowing security, and surveillance in a variety the definition of the “acceptable” student, of settings including law enforcement, commodifying data, and institutionalizing commerce, and transportation. Schools have