On the Opportunities and Risks of Foundation Models Rishi Bommasani* Drew A. Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill Erik Brynjolfsson Shyamal Buch Dallas Card Rodrigo Castellon Niladri Chatterji Annie Chen Kathleen Creel Jared Quincy Davis Dorottya Demszky Chris Donahue Moussa Doumbouya Esin Durmus Stefano Ermon John Etchemendy Kawin Ethayarajh Li Fei-Fei Chelsea Finn Trevor Gale Lauren Gillespie Karan Goel Noah Goodman Shelby Grossman Neel Guha Tatsunori Hashimoto Peter Henderson John Hewitt Daniel E. Ho Jenny Hong Kyle Hsu Jing Huang Thomas Icard Saahil Jain Dan Jurafsky Pratyusha Kalluri Siddharth Karamcheti Geoff Keeling Fereshte Khani Omar Khattab Pang Wei Koh Mark Krass Ranjay Krishna Rohith Kuditipudi Ananya Kumar Faisal Ladhak Mina Lee Tony Lee Jure Leskovec Isabelle Levent Xiang Lisa Li Xuechen Li Tengyu Ma Ali Malik Christopher D. Manning Suvir Mirchandani Eric Mitchell Zanele Munyikwa Suraj Nair Avanika Narayan Deepak Narayanan Ben Newman Allen Nie Juan Carlos Niebles Hamed Nilforoshan Julian Nyarko Giray Ogut Laurel Orr Isabel Papadimitriou Joon Sung Park Chris Piech Eva Portelance Christopher Potts Aditi Raghunathan Rob Reich Hongyu Ren Frieda Rong Yusuf Roohani Camilo Ruiz Jack Ryan Christopher Ré Dorsa Sadigh Shiori Sagawa Keshav Santhanam Andy Shih Krishnan Srinivasan Alex Tamkin Rohan Taori Armin W. Thomas Florian Tramèr Rose E. Wang William Wang Bohan Wu Jiajun Wu Yuhuai Wu Sang Michael Xie Michihiro Yasunaga Jiaxuan You Matei Zaharia Michael Zhang Tianyi Zhang Xikun Zhang Yuhui Zhang Lucia Zheng Kaitlyn Zhou Percy Liang*1 Center for Research on Foundation Models (CRFM) Stanford Institute for Human-Centered Artificial Intelligence (HAI) Stanford University AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report arXiv:2108.07258v2 [cs.LG] 18 Aug 2021 provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature. 1Corresponding author: [email protected] *Equal contribution. 1 2 Center for Research on Foundation Models (CRFM) Contents Contents 2 1 Introduction3 1.1 Emergence and homogenization3 1.2 Social impact and the foundation models ecosystem7 1.3 The future of foundation models9 1.4 Overview of this report 12 2 Capabilities 21 2.1 Language 22 2.2 Vision 28 2.3 Robotics 34 2.4 Reasoning and search 40 2.5 Interaction 44 2.6 Philosophy of understanding 48 3 Applications 53 3.1 Healthcare and biomedicine 54 3.2 Law 59 3.3 Education 67 4 Technology 73 4.1 Modeling 74 4.2 Training 81 4.3 Adaptation 85 4.4 Evaluation 91 4.5 Systems 97 4.6 Data 101 4.7 Security and privacy 105 4.8 Robustness to distribution shifts 108 4.9 AI safety and alignment 113 4.10 Theory 117 4.11 Interpretability 122 5 Society 128 5.1 Inequity and fairness 129 5.2 Misuse 135 5.3 Environment 139 5.4 Legality 145 5.5 Economics 148 5.6 Ethics of scale 151 6 Conclusion 160 Acknowledgments 160 References 160 On the Opportunities and Risks of Foundation Models 3 1 INTRODUCTION This report investigates an emerging paradigm for building artificial intelligence (AI) systems based on a general class of models which we term foundation models.2 A foundation model is any model that is trained on broad data at scale and can be adapted (e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al. 2019], GPT-3 [Brown et al. 2020], and CLIP [Radford et al. 2021]. From a technological point of view, foundation models are not new — they are based on deep neural networks and self-supervised learning, both of which have existed for decades. However, the sheer scale and scope of foundation models over the last few years have stretched our imagination of what is possible; for example, GPT-3 has 175 billion parameters and can be adapted via natural language prompts to do a passable job on a wide range of tasks despite not being trained explicitly to do many of those tasks [Brown et al. 2020]. At the same time, existing foundation models have the potential to accentuate harms, and their characteristics are in general poorly understood. Given their impending widespread deployment, they have become a topic of intense scrutiny [Bender et al. 2021]. 1.1 Emergence and homogenization The significance of foundation models can be summarized with two words: emergence and ho- mogenization. Emergence means that the behavior of a system is implicitly induced rather than explicitly constructed; it is both the source of scientific excitement and anxiety about unanticipated consequences. Homogenization indicates the consolidation of methodologies for building machine learning systems across a wide range of applications; it provides strong leverage towards many tasks but also creates single points of failure. To better appreciate emergence and homogenization, let us reflect on their rise in AI research over the last 30 years. Fig. 1. The story of AI has been one of increasing emergence and homogenization. With the introduction of machine learning, how a task is performed emerges (is inferred automatically) from examples; with deep learning, the high-level features used for prediction emerge; and with foundation models, even advanced functionalities such as in-context learning emerge. At the same time, machine learning homogenizes learning algorithms (e.g., logistic regression), deep learning homogenizes model architectures (e.g., Convolutional Neural Networks), and foundation models homogenizes the model itself (e.g., GPT-3). Machine learning. Most AI systems today are powered by machine learning, where predictive models are trained on historical data and used to make future predictions. The rise of machine learning within AI started in the 1990s, representing a marked shift from the way AI systems were built previously: rather than specifying how to solve a task, a learning algorithm would induce it based on data — i.e., the how emerges from the dynamics of learning. Machine learning also 2We chose the term foundation models to capture the unfinished yet important status of these models —see§1.1.1: naming for further discussion of the name. 4 Center for Research on Foundation Models (CRFM) represented a step towards homogenization: a wide range of applications could now be powered by a single generic learning algorithm such as logistic regression. Despite the ubiquity of machine learning within AI, semantically complex tasks in natural lan- guage processing (NLP) and computer vision such as question answering or object recognition, where the inputs are sentences or images, still required domain experts to perform “feature en- gineering” — that is, writing domain-specific logic to convert raw data into higher-level features (e.g., SIFT [Lowe 1999] in computer vision) that were more suitable for popular machine learning methods. Deep learning. Around 2010, a revival of deep neural networks under the moniker of deep learning [LeCun et al. 2015] started gaining traction in the field of machine learning. Deep learning was fueled by larger datasets, more computation (notably, the availability of GPUs), and greater audacity. Deep neural networks would be trained on the raw inputs (e.g., pixels), and higher-level features would emerge through training. This led to massive performance gains on standard benchmarks, for example, in the seminal work of AlexNet [Krizhevsky et al. 2012] on the ImageNet dataset [Deng et al. 2009]. Deep learning also reflected a further shift towards homogenization: rather than having bespoke feature engineering pipelines for each application, the same deep neural network architecture could be used for many applications. Foundation models. Foundation models have taken shape most strongly in NLP, so we focus our story there for the moment. By the end of 2018, the field of NLP was about to undergo another seismic change, marking the beginning of the era of foundation models. On a technical level, foundation models are enabled by transfer learning [Thrun 1998] and scale. The idea of transfer learning is to take the “knowledge” learned from one task (e.g., object recognition in images) and apply it to another task (e.g., activity recognition in videos). Within deep learning, pretraining is the dominant approach to transfer learning: a model is trained on a surrogate task (often just as a means to an end) and then adapted to the downstream task of interest via fine-tuning. Transfer learning is what makes foundation models possible, but scale is what makes them powerful. Scale required three ingredients: (i) improvements in computer hardware — e.g., GPU throughput and memory have increased 10× over the last four years (§4.5: systems); (ii) the development of the Transformer model architecture [Vaswani et al.