Preface Artificial Intelligence: Short History, Present Developments, and Future Outlook Final Report January 2019 Study Committee Dave Martinez, Co-Lead Andre King Nick Malyska,Co-Lead Rich Lippmann Bill Streilein,Co-Lead Benjamin Miller Rajmonda Caceres Doug Reynolds William Campbell Fred Richardson Charlie Dagli Cem Sahin Vijay Gadepally An Tran Kara Greenfield Pierre Trepagnier Robert Hall Joe Zipkin MIT LL Review Team Christopher Roeser, Lead Sanjeev Mohindra Konstantinos Hennighausen Jason Thornton DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the United States Air Force under Air Force Contract No. FA8702- 15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force. © 2019 Massachusetts Institute of Technology. Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227- 7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work. 1 Preface Preface The Director’s Office at MIT Lincoln Laboratory (MIT LL) requested a comprehensive study on artificial intelligence (AI) focusing on present applications and future science and technology (S&T) opportunities in the Cyber Security and Information Sciences Division (Division 5). This report elaborates on the main results from the study. Since the AI field is evolving so rapidly, the study scope was to look at the recent past and ongoing developments to lead to a set of findings and recommendations. It was important to begin with a short AI history and a lay-of-the-land on representative developments across the Department of Defense (DoD), intelligence communities (IC), and Homeland Security. These areas are addressed in more detail within the report. A main deliverable from the study was to formulate an end-to-end AI canonical architecture that was suitable for a range of applications. The AI canonical architecture, formulated in the study, serves as the guiding framework for all the sections in this report. Even though the study primarily focused on cyber security and information sciences, the enabling technologies are broadly applicable to many other areas. Therefore, we dedicate a full section on enabling technologies in Section 3. The discussion on enabling technologies helps the reader clarify the distinction among AI, machine learning algorithms, and specific techniques to make an end-to-end AI system viable. In order to understand what is the lay-of-the-land in AI, study participants performed a fairly wide reach within MIT LL and external to the Laboratory (government, commercial companies, defense industrial base, peers, academia, and AI centers). In addition to the study participants (shown in the next section under acknowledgements), we also assembled an internal review team (IRT). The IRT was extremely helpful in providing feedback and in helping with the formulation of the study briefings, as we transitioned from data- gathering mode to the study synthesis. The format followed throughout the study was to highlight relevant content that substantiates the study findings, and identify a set of recommendations. 2 Preface An important finding is the significant AI investment by the so-called “big 6” commercial companies. These major commercial companies are Google, Amazon, Facebook, Microsoft, Apple, and IBM. They dominate in the AI ecosystem research and development (R&D) investments within the U.S. According to a recent McKinsey Global Institute report, cumulative R&D investment in AI amounts to about $30 billion per year1. This amount is substantially higher than the R&D investment within the DoD, IC, and Homeland Security. Therefore, the DoD will need to be very strategic about investing where needed, while at the same time leveraging the technologies already developed and available from a wide range of commercial applications. As we will discuss in Section 1 as part of the AI history, MIT LL has been instrumental in developing advanced AI capabilities. For example, MIT LL has a long history in the development of human language technologies (HLT) by successfully applying machine learning algorithms to difficult problems in speech recognition, machine translation, and speech understanding. Section 4 elaborates on prior applications of these technologies, as well as newer applications in the context of multi-modalities (e.g., speech, text, images, and video). An end-to- end AI system is very well suited to enhancing the capabilities of human language analysis. Section 5 discusses AI’s nascent role in cyber security. There have been cases where AI has already provided important benefits. However, much more research is needed in both the application of AI to cyber security and the associated vulnerability to the so-called adversarial AI. Adversarial AI is an area very critical to the DoD, IC, and Homeland Security, where malicious adversaries can disrupt AI systems and make them untrusted in operational environments. This report concludes with specific recommendations by formulating the way forward for Division 5 and a discussion of S&T challenges and opportunities. The S&T challenges and opportunities are centered on the key elements of the AI canonical architecture to strengthen the AI capabilities across the DoD, IC, and Homeland Security in support of national security. 1 McKinsey Global Institute, AI The Next Digital Frontier?, June 2017 3 Acknowledgements Acknowledgements The study participants were selected from across different groups within MIT LL’s Division 5. One criterion was that they needed to be practicing researchers in the AI field. This requirement was important for the study to quickly gather inputs inside Division 5 and outside MIT LL, and then formulate a set of important findings. The study participants, as researchers, had a good understanding of the key players in the AI discipline outside of MIT LL. The AI study participants are shown here and spanned expertise in AI applied to HLT and cyber security. The MIT LL review team was chosen from researchers outside Division 5. Dr. Chris Roeser and Dr. Kosti Hennighausen came into the study with a strong background in red teaming critical technologies for national security. Dr. Sanjeev Mohindra has been working on the AI application to the intelligence, surveillance, and reconnaissance (ISR) area. Dr. Jason Thorton has shown successful use of AI in support of homeland defense. Mr. Bob Hall, from the MIT LL Knowledge Services department, was responsible for searching AI literature, notable events, and relevant announcements. He diligently issued a weekly digest containing this information and maintained an archive of all previous literature findings. An example of this digest, which continues today, is provided in Appendix A. MIT LL’s Archives team, also from the Knowledge Services Department, provided a vast number of records with information on the early history of AI at MIT LL. 4 Acknowledgements The study co-leads are also very thankful to the MIT LL Director’s Office for requesting this comprehensive study and their support during the course of the study. We are also thankful to all the study participants for contributing to the successful completion of the study. We are very thankful to the support personnel including Mr. Brad Dillman, Division 5 graphics artist, and Ms. Cynthia Devlin-Brooks and Ms. Kimberly Pitko for their administrative support. Finally, we are also very thankful to the Technical Communications department at MIT Lincoln Laboratory for the dedicated editorial support. 5 Contents Table of Contents Preface ................................................................................................................................... 2 Acknowledgements ................................................................................................................ 4 Artificial Intelligence Study Motivation (D. Martinez) ............................................................. 8 1 History of Artificial Intelligence and Trends (D. Martinez) ............................................... 13 1.1 Notable Events in AI During the Last Seven Decades .................................................................. 13 1.2 AI Global Trends .......................................................................................................................... 18 2 Lay-of-the-Land (D. Martinez) ......................................................................................... 23 2.1 Study Outreach ........................................................................................................................... 23 2.2 AI Canonical Architecture ............................................................................................................ 26 2.3 High-Level Description of Subsystem Components in the AI Canonical Architecture ................. 29 3 Enabling Technologies (V. Gadepally) ............................................................................. 35 3.2 Data Conditioning ....................................................................................................................... 38 3.3 Algorithms ..................................................................................................................................