Capturing Vortex Dynamics to Predict Acoustic Response using Machine Learning Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University By Ashwati Nair Graduate Program in Computer Science and Engineering The Ohio State University 2019 Dissertation Committee: Dr. Srinivasan Parthasarathy, Advisor Dr. Spyros Blanas Copyright by Ashwati Nair 2019 Abstract Deep learning has become a ubiquitous technology applied widely in scientific as well as non-scientific data domains, alike. Even though, they have shown signif- icant improvements over conventional techniques in various fields of application such as computer vision, not much success has been achieved in dynamic physical systems like fluid flows. One of the prospective domain for application of deep learning techniques is prediction of jet noise that is an active area of research. A significant reason for jet noise is the intermittent events in the acoustic field gener- ated due to interactions between the coherent structures in the flow field. This work introduces a machine learning based technique that extracts temporal sequence of vortex activity and predicts its acoustic emission. Sequence to se- quence learning LSTM model is used for the prediction. Vortex tracking mecha- nism is also developed as part of this work, that traces a vortex through space and time, based on its high vorticity region. The technique is tested on high-fidelity simulation dataset of a Mach 1.3 perfectly expanded jet. The results indicate that ii the model is able to correlate the variation in vorticity and associated acoustic sig- nals of a vortex enabling approximation of acoustic pattern for unseen vortices. iii Dedication To my husband Unni, for believing in my capabilities, and relentlessly supporting me through thick and thin. To my parents, for trusting me and for giving me the freedom to follow my dreams. Last and the best, to my little sister Aatira, whose unconditional love motivates me to be a better person. iv Acknowledgments First and foremost, I would like to thank my advisor, Dr. Srinivasan Parthasarathy, for his constant guidance and motivation throughout this work. Your insights to- wards addressing research problems has inspired me to become a better researcher. Furthermore, your friendly and cheerful demeanour helped me to adjust in the highly competitive environment of The Ohio State University. Next, I would like to thank Dr. Datta V. Gaitonde, John Glenn Chair Professor, Department of Mechanical and Aerospace Engineering, for giving me the oppor- tunity to study and work in OSU. Your trust motivates me to work hard. Finally, I would like to thank Dr. Spyros Blanas for serving on my dissertation committee. Your suggestions helped to set the course for this work. I gratefully acknowledge the support of the Collaborative Center for Aeronautical Sciences and the Office of Naval Research. v Vita 2009 . B.Tech Computer Science and Engineering, Mar Athanasius College of Engineering, Mahatma Gandhi University, Kerala, India 2013 . M.S. Software Systems, Birla Institute of Technology, Pilani, India Fields of Study Major Field: Computer Science and Engineering vi Table of Contents Abstract ...................................... ii Dedication ..................................... iv Acknowledgments ................................ v Vita ......................................... vi List of Figures ................................... ix Chapter 1 Introduction .................................. 1 1.1 Unsupervised Learning . 5 1.1.1 DBSCAN . 6 1.2 Supervised Learning . 8 1.2.1 Recurrent Neural Network or RNN . 10 vii 1.2.2 Long Short-Term Memory or LSTM . 12 2 Objective and Related Work ........................ 15 2.1 Objective . 17 2.2 Related Work . 18 3 Extraction of Vortex Time Series and Prediction of Acoustic Response 21 3.1 Introduction . 21 3.2 Description of the Data . 22 3.3 Technique . 27 3.3.1 Vortex Isolation . 27 3.3.2 Extraction of Vortex Time Series . 31 3.3.3 Training the Sequence to Sequence Model . 33 4 Results and Conclusion ........................... 35 4.1 Results . 35 4.2 Conclusion . 44 4.3 Future Work . 46 Bibliography ................................... 48 viii List of Figures 1.1 Deep Neural Network . 3 1.2 DBSCAN Algorithm . 7 1.3 Single Unit Single Layer Recurrent Neural Network . 10 1.4 Unfolded Recurrent Neural Network . 11 1.5 LSTM Unit . 13 3.1 Computational Grid . 23 3.2 Flow field . 24 3.3 Acoustic Emission from a jet . 25 3.4 2D contour plots showing Q-criterion and Acoustic emission . 26 3.5 Vorticity and Acoustic in an azhimuthal plane . 27 3.6 Flowchart of the Technique . 28 3.7 Sequence to Sequence Neural Network . 34 4.1 Output of DBSCAN on a snapshot . 36 ix 4.2 Evaluation Result using POD Analysis . 37 4.3 Trace of a Vortex . 38 4.4 Vorticity vs Acoustic feature of a vortex time series . 39 4.5 Predictions by the Sequence to sequence learning model . 40 4.6 Training and Testing Accuracy of the model . 43 4.7 Training and Testing loss of the model . 43 4.8 Predictions dissimilar from Actual Time Series . 45 x Chapter 1 Introduction Machine learning (ML) has been omnipresent in all fields of study from scien- tific areas of research such as social sciences and biology to non-scientific domains like business studies and sports. It encompasses a subset of techniques, from a more general field of Artificial Intelligence, that applies statistical or data-driven methods to extract implicit and hidden knowledge from data without any specific programming. The knowledge, thus, obtained is usually interpreted as either clas- sification, prediction, or representation of data. The most widely used machine learning algorithm is principle component analy- sis (PCA) that performs orthogonal transformations on multi-dimensional data to get linearly uncorrelated dimensions such that the majority percentage of statisti- cal variance in data is captured by the initial dimensions or principle components. Its generic application is to handle the curse of dimensionality amongst large scale 1 multi-variate datasets by projecting the data points onto the principle components before data mining. Different variants of PCA are actively used in various applica- tions such as signal analysis and spatio-temporal data analysis. One of the variants called Proper Orthogonal Decomposition (POD) is used in computational fluid dy- namics for reduced order modelling of the flow field such that the dominant modes captures majority of kinetic energy in the system [1]. Deep Neural Networks (DNN) commonly known as Deep Learning is a promising and evolving area in Machine Learning that focuses on solving complex data anal- ysis and representation problems using a multi-layer multi-node framework. The basic concept of neural networks have been around since the 1940s, however, with the exponential growth in computational power especially using GPUs has facili- tated the building of deeper networks to train on multi-dimensional big data. Their successful applications range from the conventional domains such as image clas- sification, speech recognition and natural language processing to the modern and sophisticated areas like self-driving cars and computer games. Figure 1.1 shows a simple fully-connected deep neural network with two layers and nine units per layer. Scientific domains such as bioinformatics [2], computational biology and physical sciences [3], to name a few, are finding wide application of deep learning tech- niques, in their research activities on modelling, classification, anomaly detection and forecasting using enormous datasets. Deep learning models are able to pro- 2 Figure 1.1: An example of a two layer neural network. The first layer is the input and the last is the output. The input can be spatial data or time series. The output can be either discrete or continuous value. Middle two layers are called the hidden layers where activation functions are applied to the data. vide better approximation and accuracy over the conventional machine learning models such as Support Vector Machines and Random Forests. Similarly, active strides are also being made towards their application in the field of computational fluid dynamics. DNN models that explicitly enforce fundamental physical prin- ciples of turbulent flow are being utilized for modelling. In the paper by Kutz [4], the author suggests that DNN models can be applied to turbulent flows either for prediction or for the approximation of reduced order models. However, more focus has been given to approximation models than on forecasting models. This work introduces a Machine Learning based technique that applies forecasting 3 model using deep neural network to predict intermittent acoustic events in a tur- bulent jet flow. Attention is given to the vortical coherent structures formed due to free shear layer interaction and persist for a significant time duration. The tech- nique encompasses a three-stage process for isolating and tracking vortices, and predicting their acoustic response. The formal thesis statement is as follows: jet noise emitted by civil and military aircrafts cause severe hearing related health hazards to ground support staff work- ing in close proximity. Turbulence in jet exhaust encompasses plethora of physi- cal phenomena that attribute to these acoustic emissions. One such phenomenon is the generation of vortices due to free flowing shear layer and their persistent dynamics. We believe that if these vortices are identified and their trajectory are captured, the time series data, thus, sampled can be utilized to train a state-of-the- art deep neural network model for predicting corresponding acoustic radiations. Further description regarding the objective of the dissertation work is in chapter 2. The following subsections present a brief overview of two main categories of ma- chine learning techniques, that are Supervised and Unsupervised Learning and few specific methodologies to enable completeness and better comprehension of the dissertation. 4 1.1 Unsupervised Learning Understanding the underlying implicit knowledge in a given data without addi- tional information such as labels, encompass unsupervised learning.