Artificial Intelligence for Robust Engineering & Science

January 22 – 24, 2020 Organizing Committee

General Chair: Logistics Chair: David Womble Christy Hembree Program Director, Artificial Intelligence Project Support, Artificial Intelligence Oak Ridge National Laboratory Oak Ridge National Laboratory

Committee Members: Jacob Hinkle Justin Newcomer Research Scientist, Computational Sciences Manager, Machine Intelligence and Engineering Sandia National Laboratories Oak Ridge National Laboratory

Frank Liu Clayton Webster Distinguished R&D Staff, Computational Distinguished Professor, Department of Sciences and Mathematics Mathematics Oak Ridge National Laboratory University of Tennessee–Knoxville

Robust engineering is the process of designing, building, and controlling systems to avoid or mitigate failures and everything fails eventually. This workshop will examine the use of artificial intelligence and machine learning to predict failures and to use this capability in the maintenance and operation of robust systems. The workshop comprises four sessions examining the technical foundations of artificial intelligence and machine learning to

1) Examine operational data for failure indicators 2) Understand the causes of the potential failure 3) Deploy these systems “at the edge” with real-time inference and continuous learning, and 4) Incorporate these capabilities into robust system design and operation.

The workshop will also include an additional session open to attendees for “flash” presentations that address the conference theme. AGENDA Wednesday, January 22, 2020

7:30–8:00 a.m. │ Badging, Registration, and Breakfast

8:00–8:45 a.m. │ Welcome and Introduction Jeff Nichols, Oak Ridge National Laboratory David Womble, Oak Ridge National Laboratory

8:45–9:30 a.m. │ Keynote Presentation: Dave Brooks, General Motors Company AI for Automotive Engineering

9:30–10:00 a.m. │ Group Photo and Break

Session 1: Finding the right needles in noisy haystacks Session Chair: Frank Liu, Oak Ridge National Laboratory Identifying early failure indicators from noisy sensor data is a crucial step to ensure robustness and resilience of complex engineering systems. The same methodology can also be applied to identify critical transition points in natural phenomena. Traditionally, time series data analysis methods have been the workhorse. The objective of this session is not only to identify the appropriate engineering/data science solutions, but also to discuss fundamental science questions such as the observability of highly nonlinear systems from noisy and sparse data.

• Can Machine Learning methods be used effectively to indicate or predict failure? • Can Machine Learning fundamentally transform research and practice of early failure detection?

10:00–10:10 a.m. │ Session Introduction

10:10–10:50 a.m. │ Speaker 1–1: Siva Rajamanickam, Sandia National Laboratories

Machine Learning in the presence of noise: Early experiments

10:50–11:20 a.m. │ Speaker 1–2: Peng Li, University of California-Santa Barbara Data-Efficient Robust Anomaly Detection: A Machine Learning Approach

11:20–11:50 a.m. │ Speaker 1–3: Kody Law, University of Manchester

Data Centric (AI for) Science and Engineering in the UK

11:50–12:20 p.m. │ Speaker 1–4: Helen Li, Duke University

Machine Learning in Modern Water Inspection and Chip Design

12:20–12:30 p.m. │ Session Wrap Up

12:30–1:30 p.m. │ Working Lunch with Breakout Discussions

Session 2: Skip the search–from finding needles to understanding needles Session Chair: Justin Newcomer, Sandia National Laboratories

Today, robust engineering of complex systems requires significant investments in design, production, and lifecycle monitoring. Extensive testing is conducted to uncover performance issues, degradations, and precursors to failure - searching for a finite number of needles in an exponentially growing stack of hay - over the life of each system. Identification of anomalies often leads to follow-on investigations to determine the performance impact, root cause, and extent of condition. These reactive investigations are time and labor intensive but are essential for credible decision making. Can AI provide a deep causal understanding of complex systems? Can AI guide collection of the right data at the right time? If so, this will lead to revolutionary advancements such as rapid autonomous failure analyses, learning physical models and design processes to mitigate or eliminate failure mechanisms, anticipating emergent behaviors not traceable to design decisions or requirements, and real-time trusted decision support. This session will explore advancements needed in causal inference, planning under uncertainty, model credibility, reinforcement learning, and human computer interaction to significantly disrupt the current engineering process reliant on an endless search for needles.

• Can artificial intelligence provide a deep causal understanding of complex systems? • Can artificial intelligence guide collection of the right data at the right time?

1:30–1:40 p.m. │ Session Introduction

1:40–2:10 p.m. │ Speaker 2–1: Laura McNamara, Sandia National Laboratories

Adoption Challenges in Artificial Intelligence and Machine Learning: why technology acceptance is so hard (and what we can do about it)

2:10–2:40 p.m. │ Speaker 2–2: Chuck Farrar, Los Alamos National Laboratory

Machine Learning Approaches to Structural Health Monitoring Data Normalization

2:40–3:10 p.m. │ Speaker 2–3: Eli Sherman, Johns Hopkins University

Formal Methods for Addressing Data Complications

3:10–3:50 p.m. │ Networking Break

3:50–4:20 p.m. │ Speaker 2–4: Aurora Schmidt, Johns Hopkins University Applied Physics Laboratory

A Case Study in Safety Constraints to Machine Learning-Based Controllers

4:20–4:30 p.m. │ Session Wrap Up

4:30–5:00 p.m. │ Day 1 Wrap Up

5:00–7:00 p.m. │ Welcome Reception

Thursday, January 23, 2020

7:30–8:15 a.m. │ Breakfast

8:15–8:20 a.m. │ Day 2 Introduction

Session 3: Running in the wild – forget the past and do it fast with online machine learning Session Chair: Clayton Webster, University of Tennessee–Knoxville

Almost all physical systems are instrumented, and data is being generated in huge quantities. Predicting failure in large-scale engineered systems requires the exploitation of such data with tools designed to handle data with high volume, velocity, and variety. Data is not static but typically arrives as a stream of sequentially ordered samples. Due to the high volume, data can be used only once in-situ and cannot be saved for later learning. The objective of this session is to explore machine learning techniques focused on continuous learning using a limited amount of memory in a limited amount of time, while retaining the ability to perform predictions at any point in time.

• How can machine learning enable continuous, dynamic, and short-term learning and prediction for an effective strategy when operating in very fast and dynamic environments?

8:20–8:30 a.m. │ Session Introduction

8:30–9:00 a.m. │ Speaker 3–1: Wilkins Aquino, Duke University

Model-Based Learning of Advection-Diffusion Transport using Mobile Robots

9:00–9:30 a.m. │ Speaker 3–2: Abhinav Saxena, GE Research - AI & Learning Systems

AI Spectrum for Predictive Maintenance 9:30–10:00 a.m. │ Networking Break

10:00–10:30 a.m. │ Speaker 3–3: Nagi Rao, Oak Ridge National Laboratory

Practice of Machine Learning Theory: Case Studies from Nuclear Reactors and Computing Infrastructures

10:30–11:00 a.m. │ Speaker 3–4: Mingzhou Jin, University of Tennessee - Knoxville Geometrical Defect Detection for Additive Manufacturing with Machine Learning Models

11:00–11:10 a.m. │ Session Wrap Up

Session 4: Flash Speaker Presentations Session Chairs: Danny Dunlavy & David Stracuzzi, Sandia National Laboratories

11:10–11:15 a.m. │ Session Introduction

11:15–11:30 a.m. │ Speaker 4–1: Michelle Quirk, DOE/NNSA

AI–Complete Problems

11:30–11:45 a.m. │ Speaker 4–2: Warren Davis, Sandia National Laboratories

In-Situ Anomaly Detection for Intelligent Data Capture in HPC Simulations

11:45–12:00 p.m. │ Speaker 4–3: Iris Bahar, Brown University

A Simulation Framework for Capturing Thermal Noise-Induced Failures in Low-Voltage CMOS SRAM

12:00–1:00 p.m. │ Working Lunch with Presentation by Dave Keim, Oak Ridge National Laboratory

The History of ORNL

1:00–1:15 p.m. │ Speaker 4–4: Shawn Sheng, National Renewable Energy Laboratory

SCADA Data Modeling for Wind Turbine Gearbox Failure Detection using ML and Big Data Technologies

1:15–1:30 p.m. │ Speaker 4–5: Robert Patton, Oak Ridge National Laboratory

Artificial Intelligence for Autonomous Vehicles

1:30–1:45 p.m. │ Speaker 4–6: Ahmedullah Aziz, University of Tennessee–Knoxville

Reliability Concerns in Emerging Neuromorphic Hardware

1:45–2:00 p.m. │ Speaker 4–7: Emily Donahue, Sandia National Laboratories

Identifying Defects in CT Scans without Labelled Data

2:00–2:15 p.m. │ Speaker 4–8: David Mascarenas, National Security Engineering Center

Video-Based, High Resolution, High Sensitivity Structural Health Monitoring

2:15–2:30 p.m. │ Speaker 4–9: Steve Sun, Columbia University

Non-cooperative Game for Learning from Non-Euclidean Microstructural Data for Computational Solid Mechanics

2:30–3:00 p.m. │ Networking Break

3:00–3:15 p.m. │ Speaker 4–10: John Lindberg, Electric Power Research Institute

Data Science in the Nuclear Industry

3:15–3:30 p.m. │ Speaker 4–11: Minsik Cho, IBM

SNOW: Subscribing to Knowledge via Channel Pooling for Transfer & Lifelong/Continual Learning

3:30–3:45 p.m. │ Speaker 4–12: Draguna Vrabie, Pacific Northwest National Laboratory

Learning and Deception – Robust Control

3:45–4:00 p.m. │ Speaker 4–13: Vivek Sarkar, Georgia Institute of Technology

Using AI to Improve Robustness and Productivity of Engineering & Science Software

4:00–4:15 p.m. │ Speaker 4–14: Rick Archibald, Oak Ridge National Laboratory

Machine Learning for Scientific Data

4:15–4:30 p.m. │ Speaker 4–15: Geoffrey Fox, Indiana University

Deep Learning Enhanced Simulation

4:30–4:45 p.m. │ Speaker 4–16: Mariam Kiran, Lawrence Berkeley National Laboratory

Using AI to ESnet, the High-Performance Science Network

4:45–5:00 p.m. │ Day 2 Wrap Up

Friday, January 24, 2020

7:30–8:15 a.m. │ Breakfast

8:15–8:20 a.m. │ Day 3 Introduction

Session 5: Now what? Integrating predictive prognostics into the development and operations of robust systems Session Chair: Jacob Hinkle, Oak Ridge National Laboratory

Much attention has been paid to the development of advanced capabilities for characterizing and detecting eminent failures. In this session, we will explore how to incorporate this capability into an overall system, including system design, combining machine learning with modeling to create digital twins, building adaptive control systems to avoid or mitigate failure, and predicting system life. This session will also discuss deployment of artificial intelligence at the edge for effective monitoring, and modern approaches that can leverage passive prior observations to reduce maintenance and downtime through active monitoring.

• Now that we can use artificial intelligence to predict failures, can we identify the failure modes, use machine learning to update digital twins, or learn control systems to mitigate failures?

8:20–8:30 a.m. │ Session Introduction

8:30–9:00 a.m. │ Speaker 5–1: Sandra Biedron, University of New Mexico; Element Aero

Facilities as Intelligent Systems – today and future wish list

9:30–10:00 a.m. │ Speaker 5–2: Pradeep Ramuhalli, Oak Ridge National Laboratory

Challenges and Solutions for Prognostic Health Management (PHM) in Nuclear Energy

10:00–10:30 a.m. │ Networking Break

10:30–11:00 a.m. │ Speaker 5–3: Jim Tallman, General Electric/GE Research

Exploiting AI for Design Process Improvements at Enterprise Scale

11:00–11:30 a.m. │ Speaker 5–4: Kyriakos Vamvoudakis, Georgia Institute of Technology

Robust and Secure Reinforcement Learning for Prediction and Control

11:30–12:00 p.m. │ Speaker 5–5: Monte Lunacek, National Renewable Energy Laboratory

A Data Driven Operational Model for Traffic at Dallas Fort-Worth Airport

12:00–12:10 p.m. │ Session Wrap Up

12:10–1:00 p.m. │ Working Lunch with Breakout Discussions

1:00–2:00 p.m. │ Wrap Up and Next Steps

PRESENTATION INFO Day 1 Keynote Presentation

- Dave Brooks, General Motors Company Title: AI for Automotive Engineering

Session 1: Finding the right needles in noisy haystacks Session Chair: Frank Liu, Oak Ridge National Laboratory

- Siva Rajamanickam, Sandia National Laboratories Title: Machine Learning in the Presence of Noise: Early Experiments Abstract: We are envisioning a workflow in which ML methods will be applied at the edge ideally using AI/ML accelerators for analyzing data generated as experiments (or simulations) progress. Data for learning and inference at this level tend to be quite noisy. First, I will give an overview of few Sandia projects where we have had preliminary success in using ML methods for "finding the right needle" in noisy data. Second, I will focus on graph neural networks which is gaining popularity in several fields (e.g. material science). I will show examples the show the effects of noise on these ML methods and one method to improve their robustness in the presence of noise.

- Peng Li, University of California–Santa Barbara Title: Data-Efficient Robust Anomaly Detection: A Machine Learning Approach Abstract: Anomaly detection has become an increasingly important research problem and challenge with applications in many science and engineering domains. Advances in anomaly detection can broadly impact a broad range of use cases including verification of mission/safety-critical systems, robust manufacturing, predictive maintenance, and fraud detection.

This talk will present machine learning techniques targeting anomaly detection using severely limited amounts of training data for reasons such as high cost in data collection and/or unavailability of labeled data. As such, the key challenge to be tackled is to enable anomaly detection with desired data efficiency and robustness in potentially high-dimensional input (feature) spaces where complex interactions of such features lead to rareness of anomalies.

First, we will present a data-efficient Bayesian optimization (BO) approach. At the heart of the proposed BO process is a delicate balancing between two competing needs: exploitation of the current statistical model for quick identification of highly likely failures and exploration of undiscovered feature space so as to detect hard-to- find failures over wide ranges of feature values. While one of the key benefits of BO is its generality as a black-box solution, existing BO techniques do not scale well with the dimensionality of the feature space. Dimension reduction, a key enabler for scaling learning in a high-dimensional feature space, will be introduced under the framework of Bayesian optimization. Second, a self-labeling approach for unsupervised learning

and detection of anomalies where no labeled training data is assumed will be presented. Finally, general issues of robust machine learning in terms of model uncertainty and resilience with respect to various local and global attacks will be discussed. The proposed amorally detection techniques will be demonstrated under the context of analog/mixed-signal IC design verification and post-manufacturing test for safety-critical automotive applications with stringent failure rate requirements, e.g. less than one detective parts per million (DPPM).

- Kody Law, University of Manchester Title: Data Centric (AI for) Science and Engineering in the UK Abstract: The past decade has seen an enormous growth in the technological infrastructure and the mathematical/statistical underpinnings of data science. Our ability to collect, store, process, and interpret data has advanced immeasurably, and data-driven methods have had unprecedented impact on business and society at large, in large part because of their success in analyzing and predicting human preferences. Science and engineering stand to benefit from this data revolution in similar ways, but realizing this vision requires thoughtful and concerted effort.

The Lloyd’s Register Foundation Foresight Review of Big Data (UK) calls for putting “data considerations at the core of engineering design. . . [to improve] the performance, safety, reliability and efficiency of assets, infrastructures and complex machines” with data analytics featuring “at all phases of the life-cycle of engineered systems.” Key examples include critical infrastructure networks (transportation, water distribution, power grids, and farm ecosystems) where resilience and real-time adaptability are essential; health monitoring and predictive maintenance of complex assets, ranging from jet engines to the built environment, where data-driven decisions can improve safety and reduce costs; and the engineering design process itself, where data-driven modelling can accelerate the design cycle while yielding more capable and predictable products.

Similarly, the Scientific Machine Learning report from DOE ASCR (USA) stated that "Across the Department of Energy, Scientific Machine Learning has the potential to transform science and energy research by harnessing DOE investments in massive data from scientific user facilities, software for predictive models and algorithms, high- performance computing platforms, and the national workforce. The versatile and crosscutting nature of machine learning provides strong motivation for formulating a prioritized research agenda to maximize its capabilities and scientific benefits for the DOE." While, the National Artificial Intelligence Research and Development Strategic Plan: 2019 Update states that "human-AI collaborations will transform the way science is done."

The Alan Turing Institute, Lloyd's Register Foundation, and MIT recently hosted a workshop on Data Centric Engineering (https://www.dceworkshop.org/) to define, push the boundaries, and chart the future of the field. This talk will summarize the outcomes of the meeting, as well as some other related activities in AI for Science and digital technologies at Turing, Manchester, and around the UK.

- Helen Li, Duke University Title: Machine Learning in Modern Water Inspection and Chip Design

Abstract: The successes of machine learning have spurred interest in its use in the semiconductor industry. This talk will give two examples, representing the uses in the manufacturing stage and chip design, respectively. In wafer detection, companies usually rely on handcrafted recipes and manually detect the differences between normal and defective wafers. We leverage CNNs to automate the wafer inspection process. Several techniques are used to preprocess and augment wafer images for enhancing the model’s generalization on unseen wafers (e.g., from other fabs). The design flow parameters are of utmost importance to chip design quality. Flow parameter tuning is usually performed manually in an ad hoc manner, resulting in a painfully long time to evaluate the effects. We propose a feature-importance sampling and tree-based method, which learns the impact of parameters from previously well-explored designs and fully utilizes such information in its sampling process. Experimental results show that our approach achieves a 25% improvement in design quality or a 37% reduction in sampling cost compared to the state-of-the-art method.

Session 2: Skip the search – from finding needles to understanding needles Session Chair: Justin Newcomer, Sandia National Laboratories

- Laura McNamara, Sandia National Laboratories Title: Adoption Challenges in Artificial Intelligence and Machine Learning: Why technology acceptance is so hard (and what we can do about it) Abstract: Artificial intelligence and machine learning experts see tremendous potential for using algorithms to revolutionize design engineering. However, getting engineering communities to adopt and integrate new methods into existing workflows can be extremely challenging. This talk explores factors that influence technology acceptance at both the individual and organizational levels and describes strategies that AI proponents can use to assess the adoption potential of new methods and techniques in existing engineering workflows.

- Chuck Farrar, Los Alamos National Laboratory Title: Machine Learning Approaches to Structural Health Monitoring Data Normalization Abstract: The process of implementing a damage detection strategy for aerospace, civil and mechanical engineering infrastructure is referred to as structural health monitoring (SHM). The SHM process compliments traditional nondestructive evaluation by extending these concepts to online, in situ system monitoring on a more global scale. It is our belief that the SHM problem is best addressed in terms of a statistical pattern recognition paradigm. In this paradigm, the SHM process can be broken down into four parts: (1) Operational Evaluation, (2) Data Acquisition and Cleansing, (3) Feature Selection and Extraction, and (4) Statistical Model Development for Feature Discrimination. One of the biggest challenges that prevents SHM research from transitioning to practice is that in situ operational and environmental variability can produce changes in the measured system response that can be mistaken for damage. We use the term data normalization to describe the process of separating the changes in sensor reading caused by damage from those caused by operational

and environmental variability. This presentation will begin by describing parametric approaches to data normalization that use direct measurements of the sources of variability followed by a discussion of the limitations of such approaches. Next, unsupervised machine learning approaches to data normalization will be described and four such methods will be applied to data from a laboratory test structure specifically designed with simulated operational and environmental conditions. The relative performance of these machine learning approaches will be quantified, and their relative attributes will be discussed. The presentation will conclude with a summary of some outstanding challenges associated with this data normalization problem.

- Eli Sherman, Johns Hopkins University Title: Formal Methods for Addressing Data Complications Abstract: Recent advances in machine learning techniques, including the rapid development of deep learning methods, have enabled substantial performance improvements in prediction and reinforcement learning. Nevertheless, in complex systems, evidence-based decision-making, troubleshooting, and explanation relies on understanding cause-effect relationships.

In this talk, I will discuss how causal inference and missing data methods can address the many issues inherent in complex systems, such as unobserved confounding, selection effects, missing data, and data dependence. Many of these issues arise (and are ignored) even in classical machine learning settings, such as prediction problems.

- Aurora Schmidt, Johns Hopkins University Applied Physics Laboratory Title: A Case Study in Safety Constraints to Machine Learning-Based Controllers Abstract: There are a variety of emerging applications for advanced machine learning algorithms that include the control of safety-critical systems. Many of these AI systems learn by optimizing a reward function. Unconstrained maximization of a statistical reward leads to a variety of issues in verifying the safe behavior of a machine learning system. As a case study, we present an approach to verifying that such a system for collision-free path planning. We used machine verification to prove formal safety predicates of collision-free flight incorporating flexibility to uncertain parameters and delay. Using these theorems, we were able to (1) check the ML system for safe behavior and (2) build a safe-by-construction fallback control system with formal collision avoidance guarantees. We discuss benefits and limitations of this approach to the incorporation of machine learning controllers in safety-critical contexts. (Coauthors: Yanni Kouskoulas. Daniel Genin, Jean-Baptiste Jeannin, Jessica Lopez)

Session 3: Running in the wild – forget the past and do it fast with online machine learning Session Chair: Clayton Webster, University of Tennessee - Knoxville

- Wilkins Aquino, Duke University Title: Model-Based Learning of Advection-Diffusion Transport using Mobile Robots

Abstract: We will discuss the problem of Active Source Identification (ASI) in steady- state Advection-Diffusion (AD) transport systems using mobile robots. To this end, we developed an integrated active sensing framework in which mobile robots are tasked with obtaining optimal measurements while also estimating the location of contaminant sources. Specifically, we formulate the Source Identification (SI) problem as a PDE-constrained optimization. To collect the measurements, we control a robot sensor through a sequence of waypoints that maximize the minimum- eigenvalue of the Fisher Information Matrix of the unknown source parameters. We will discuss numerical simulations and real-world experiments that show that using the proposed framework, we can efficiently identify sources in complex AD systems and non-convex domains. In addition, will see how our framework is being extended to other applications such as autonomous inspection of nuclear reactors using mobile robots and self-learning ultrasound elastography, among others.

- Abhinav Saxena, GE Research - AI & Learning Systems Title: AI Spectrum for Predictive Maintenance Abstract: Predictive Maintenance (PM) is becoming ubiquitous for improving availability and reliability along with reducing O&M costs in industrial systems. Despite significant research and development investment in the last decade most deployed solutions still tend to be piecemeal (component or failure mode specific) point solutions and generally lack trust with respect to automated decision making. Full end-to-end deployment with system-wide coverage and autonomy still remains an elusive goal in industrial setting. This is primarily due to high cost and limited scalability of conventional modeling approaches for underlying complex systems and processes in large fleets. Specifically, capabilities to safe-guard against unknown-unknowns, lack of explainability and trust tend to be key bottlenecks. Given these systems are heavily instrumented generating large volumes of high-speed data and compute costs continue to go down, recent advancements in data-driven methods using machine learning (ML) and artificial intelligence (AI) have shown promise in a number of areas that previously led to valley of death between PM technology and commercialization. GE’s Digital Twin technology for Predictive Maintenance is leveraging AI to bridge a number of such critical gaps that were otherwise very challenging to tackle through conventional methods. This session will enumerate key challenges in enabling system- wide predictive maintenance and how AI is being used to overcome these. Various applications and use-cases will be shared to show effectiveness of AI and ML using both structured and unstructured data in the context of intelligent PM.

- Nagi Rao, Oak Ridge National Laboratory Title: Practice of Machine Learning Theory: Case Studies from Nuclear Reactors and Computing Infrastructures Abstract: Four practical applications of machine learning are briefly described from the areas of nuclear reactors and computing infrastructures. A detailed solution to a sensor error estimation problem in nuclear power plants based on information fusion method is presented along with generalization equations from statistical theory of machine learning. A generic framework is described for utilizing physical and abstract domain laws to address the limits of computability, learnability and explainability of machine learning solutions.

- Mingzhou Jin, University of Tennessee - Knoxville Title: Geometrical Defect Detection for Additive Manufacturing with Machine Learning Models Abstract: In-situ detection of defects during additive manufacturing may reduce costs and time through early termination or parameter adjustment. This study explores machine-learning (ML) models for detecting geometric defects. The ML models are trained with synthetic 3D objects with defects rather than a large volume of experimental data, which typically are costly to obtain. Numerical comparisons for different shapes show that the ML models outperform classical statistical methods and the Z-difference method regarding detection accuracy because ML models introduce more dimensions to make the classification. ML methods - Bagging of Trees, Gradient Boosting, Random Forest, K-nearest Neighbors and Linear Supported Vector Machine - are compared under various conditions, such as different point cloud density and different defect size. 2D maps of the best algorithm out of the ML methods are plotted. The ML models trained through the proposed training method have the potential to be directly applied to 3D reconstruction during additive manufacturing with the aid of digital cameras or 3D scanners.

Session 4: Flash Speaker Presentations Session Chairs: Danny Dunlavy & David Stracuzzi, Sandia National Laboratories

- Michelle Quirk, DOE/NNSA Title: AI–Complete Problems

- Warren Davis, Sandia National Laboratories Title: In-Situ Anomaly Detection for Intelligent Data Capture in HPC Simulations

- Iris Bahar, Brown University Title: A Simulation Framework for Capturing Thermal Noise-Induced Failures in Low- Voltage CMOS SRAM

- Shawn Sheng, National Renewable Energy Laboratory Title: SCADA Data Modeling for Wind Turbine Gearbox Failure Detection using ML and Big Data Technologies

- Robert Patton, Oak Ridge National Laboratory Title: Artificial Intelligence for Autonomous Vehicles

- Ahmedullah Aziz, University of Tennessee-Knoxville Title: Reliability Concerns in Emerging Neuromorphic Hardware

- Emily Donahue, Sandia National Laboratories Title: Identifying Defects in CT Scans without Labelled Data

- David Mascarenas, National Security Engineering Center Title: Video-Based, High Resolution, High Sensitivity Structural Health Monitoring

- Steve Sun, Columbia University Title: Non-cooperative Game for Learning from Non-Euclidean Microstructural Data for Computational Solid Mechanics

- John Lindberg, Electric Power Research Institute Title: Data Science in the Nuclear Industry

- Minsik Cho, IBM Title: SNOW: Subscribing to Knowledge via Channel Pooling for Transfer & Lifelong/Continual Learning

- Draguna Vrabie, Pacific Northwest National Laboratory Title: Learning and Deception – Robust Control

- Vivek Sarkar, Georgia Institute of Technology Title: Using AI to Improve Robustness and Productivity of Engineering & Science Software

- Rick Archibald, Oak Ridge National Laboratory Title: Machine Learning for Scientific Data

- Geoffrey Fox, Indiana University Title: Deep Learning Enhanced Simulation

- Mariam Kiran, Lawrence Berkeley National Laboratory Title: Using AI to ESnet, the High-Performance Science Network

Session 5: Now what? Integrating predictive prognostics into the development and operations of robust systems Session Chair: Jacob Hinkle, Oak Ridge National Laboratory

- Sandra Biedron, University of New Mexico and Element Aero Title: Facilities as Intelligent Systems – today and the future wish list Abstract: Thoughts on data needs have to start from day 1 no matter how large or small the physical system. What I mean is that data must be weaved into the systems architecture of any physical system design from day 1. The requirements of the data – will data generated be only used for control, for prognostics, for scientific or engineering analysis – need to all be weaved into the systems engineering architecture and are part of the system requirements. Where will the data come from (do we have the right sensors and are we sampling at the right rate to give us the data on the correct time scales? How do these requirements fit into the control system architecture? Do I have the right amount of computational power locally near the sensors and back at the main control center? All the things that used to be afterthoughts (from my experience) need now more than ever to be part of the

physical system’s architecture on day 1. Let us first exploit machine learning and optimization to get us to the right architecture for the what we need the data to do for us. What tools can we build to help us understand what our options are to maximize the limited memory and time and perform fast predictions on the system from its first day of commissioning? Maybe today’s architectures of the backbone for data (design and “experimental”)/control/computing/etc. are all wrong for this next “mission” because we can better exploit machine learning in said physical systems need to be revisited. Then, once we have the right physical system for our needs, (If we are retrofitting a system, similar steps still have to be taken.) How do we know even we are choosing the right machine learning approach for a given system/sub-system. Let’s explore a few examples of how today’s physical system architectures can exploit machine learning for continuous, dynamic, and short-term learning and prediction in a very fast and dynamic environment. We will look at a big scientific system, a system for a first responder, an autonomous satellite, a subsystem for a large facility, and a concept system for a museum for environmental control/exhibit/security. And then, let’s discuss what improvements we need to make in our architecture and in our selection of machine learning approaches (and tools) to make our data sub-systems run even better in our follow-on discussions.

- Pradeep Ramuhalli, Oak Ridge National Laboratory Title: Challenges and Solutions for Prognostic Health Management (PHM) in Nuclear Energy Abstract: Advancements in sensors, communication protocols, data analytics, and visualization technologies are redefining and reshaping the economics of operation, plant performance, and maintenance activities within the energy industry. As an example, in the nuclear energy sector, information on component condition and the failure probability associated with degraded components is considered critical to maintaining adequate safety margins and avoiding unplanned shutdowns, both of which have regulatory and economic consequences. Within this sector, industry is moving towards digital innovation through the deployment of large-scale sensor networks for continuous monitoring plant and component condition. The resulting measurement capabilities can serve to automate several routine functions that are currently performed manually, resulting in cost-effective operation. At the same time, the deployment of continuous monitoring technologies can enable the detection and prediction of degradation in critical components, providing an opportunity for maintaining long-term, reliable, cost-effective operations using predictive operations and maintenance actions. The development and implementation of data analytics techniques for diagnostics and prognostics that can operate on high volume data in near real-time are both critical and enabling for this purpose. This presentation will discuss approaches that leverage machine learning for online monitoring and prognostics for such applications, identify open research questions, and potential solutions. Approaches to utilizing the results from PHM for operations and maintenance decision-making will also be briefly discussed.

- Jim Tallman, General Electric/GE Research Title: Exploiting AI for Design Process Improvements at Enterprise Scale Abstract: Digital Thread for Design (DT4D) is a design community ecosystem built at the GE Research Center around three fundamental “pillars” of digital functionality

relevant to design/modeling/simulation communities. Those pillars are: (1) Workflow Orchestration, (2) Data Management and Socialization, and (3) Surrogate Modeling using Machine Learning and Artificial Intelligence. The intent of DT4D is to greatly accelerate the pace at which engineering models are created, validated, exploited, and socialized throughout a product’s lifecycle. The key ingredient to the DT4D initiative is the surrogate models enabled by modern machine learning techniques, which offer orders of magnitude acceleration in the time requirements needed by an engineering community to perform analyses. As such, DT4D is designed to empower non-AI experts to quickly transform their modeling & simulation workflows into A.I. surrogates, and for non-mod.-sim. experts to quickly find and exploit the surrogates toward solutions. This talk will provide an overview of DT4D and share some of the usage examples where it has been applied to date on design problems of relevance to the General Electric company. The talk will also discuss how DT4D can be used to ensure, and even optimize for, robustness in the design’s performance objective via the readiness of the knowledge contained in the surrogate models. Finally, attention will be given to how an A.I. surrogate model’s response can evolve through a product’s lifecycle: from that based purely on simulated training data at the product’s inception, to a response that accounts for fielded unit data as the product matures in the marketplace.

- Kyriakos Vamvoudakis, Georgia Institute of Technology Title: Robust and Secure Reinforcement Learning for Prediction and Control Abstract: Despite their exposure to the full complexity of a plethora of adversarial human-centric environments, autonomous systems are expected to maintain operation and efficiency at all times. Those requirements span the full spectrum of the different functions that complete a large-scale system – from the low-level control methods to the high-level decision-making mechanisms. In this talk, we will present algorithms that take advantage of the tight interconnection of such systems with their environment in order to optimize their decisions via Reinforcement Learning techniques. Beyond the research that has been conducted on Reinforcement Learning during its construction, current work focuses on allowing learning agents to securely complete their tasks even in the presence of, unpredictable, human attackers. As such, novel solutions and models that combine ideas from psychology, economics and game theory, will be presented. Those models, inspired by experimental results on the bounded rationality of human decision-makers, are brought together with control-theoretic tools in a unified framework that will enable prediction and mitigation of realistic attacks on learning systems. Those “cognitive hierarchy” based approaches, are constructed through iterative model-free processes that approximate the strategies of attackers with limited cognitive abilities, while data- based mechanisms estimate distributions of those abilities in adversarial environments. Further prediction capabilities ae incorporated into large-space graph-embedded decision mechanisms via a predictive Q-learning framework that derives the optimal route in a resilient fashion, in the present of packet drops due to faults and attacks. Humans-in-the-loop are introduced, that offer feedback to the network concerning sensitive node locations. To showcase this method, we utilize anonymized data from Arlington County, Virginia, to compute predictive and resilient scheduling policies for a smart water supply system.

- Monte Lunacek, National Renewable Energy Laboratory Title: A Data Driven Operational Model for Traffic at Dallas Fort-Worth International Airport Abstract: Airports are moving more people and goods faster, cheaper, and with greater convenience than ever before. As air travel continues to grow, airports will face challenges in responding to increasing passenger vehicle traffic, which leads to lower operational efficiency, poor air quality, and security concerns. Our work evaluates methods for traffic demand forecasting, which will allow airport operations staff to accurately forecast traffic and congestion. Using a year of detailed data describing individual vehicle arrivals and departures, aircraft movements and weather at Dallas Fort-Worth (DFW) International Airport, we evaluate multiple prediction methods including seasonal ARIMA, Prophet, modern supervised machine learning algorithms, and modern deep learning models for time series forecasting. We find that machine learning models based on careful feature engineering offer the best prediction for the next 30 minutes, while the deep learning models perform best on the longer prediction periods. Combining these demand forecasts with a traffic microsimulation framework provides a complete picture of traffic and its consequences. The result is an operational intelligence platform for exploring policy changes, as well as infrastructure expansion and disruption scenarios. To demonstrate the value of this approach, we present results from a case study at DFW airport assessing the impact of a policy change for vehicle routing in high demand scenarios. This framework can assist airports like DFW as they tackle daily operational challenges, as well as explore the integration of emerging technology and expansion of their services into long term plans.