Capturing Vortex Dynamics to Predict Acoustic Response Using Machine Learning

Capturing Vortex Dynamics to Predict Acoustic Response using Machine Learning

Dissertation

Presented in Partial Fulﬁllment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University

Ashwati Nair

Graduate Program in Computer Science and Engineering

The Ohio State University 2019

Dissertation Committee: Dr. Srinivasan Parthasarathy, Advisor Dr. Spyros Blanas Copyright by

Ashwati Nair

2019 Abstract

Deep learning has become a ubiquitous technology applied widely in scientific as well as non-scientific data domains, alike. Even though, they have shown significant improvements over conventional techniques in various fields of application such as computer vision, not much success has been achieved in dynamic physical systems like fluid flows. One of the prospective domain for application of deep learning techniques is prediction of jet noise that is an active area of research. A significant reason for jet noise is the intermittent events in the acoustic field generated due to interactions between the coherent structures in the flow field.

This work introduces a machine learning based technique that extracts temporal sequence of vortex activity and predicts its acoustic emission. Sequence to sequence learning LSTM model is used for the prediction. Vortex tracking mechanism is also developed as part of this work, that traces a vortex through space and time, based on its high vorticity region. The technique is tested on high-ﬁdelity simulation dataset of a Mach 1.3 perfectly expanded jet. The results indicate that

ii the model is able to correlate the variation in vorticity and associated acoustic signals of a vortex enabling approximation of acoustic pattern for unseen vortices.

iii Dedication

To my husband Unni, for believing in my capabilities, and relentlessly supporting me through thick and thin. To my parents, for trusting me and for giving me the freedom to follow my dreams. Last and the best, to my little sister Aatira, whose

unconditional love motivates me to be a better person.

iv Acknowledgments

First and foremost, I would like to thank my advisor, Dr. Srinivasan Parthasarathy, for his constant guidance and motivation throughout this work. Your insights towards addressing research problems has inspired me to become a better researcher.

Furthermore, your friendly and cheerful demeanour helped me to adjust in the highly competitive environment of The Ohio State University.

Next, I would like to thank Dr. Datta V. Gaitonde, John Glenn Chair Professor,

Department of Mechanical and Aerospace Engineering, for giving me the oppor- tunity to study and work in OSU. Your trust motivates me to work hard.

Finally, I would like to thank Dr. Spyros Blanas for serving on my dissertation committee. Your suggestions helped to set the course for this work.

I gratefully acknowledge the support of the Collaborative Center for Aeronautical

Sciences and the Ofﬁce of Naval Research.

v Vita

2009 ...... B.Tech Computer Science and Engineering, Mar

Athanasius College of Engineering, Mahatma Gandhi

University, Kerala, India

2013 ...... M.S. Software Systems, Birla Institute of Technology,

Pilani, India

Fields of Study

Major Field: Computer Science and Engineering

vi Table of Contents

Abstract ...... ii

Dedication ...... iv

Acknowledgments ...... v

Vita ...... vi

List of Figures ...... ix

Chapter

1 Introduction ...... 1

1.1 Unsupervised Learning ...... 5

1.1.1 DBSCAN ...... 6

1.2 Supervised Learning ...... 8

1.2.1 Recurrent Neural Network or RNN ...... 10

vii 1.2.2 Long Short-Term Memory or LSTM ...... 12

2 Objective and Related Work ...... 15

2.1 Objective ...... 17

2.2 Related Work ...... 18

3 Extraction of Vortex Time Series and Prediction of Acoustic Response 21

3.1 Introduction ...... 21

3.2 Description of the Data ...... 22

3.3 Technique ...... 27

3.3.1 Vortex Isolation ...... 27

3.3.2 Extraction of Vortex Time Series ...... 31

3.3.3 Training the Sequence to Sequence Model ...... 33

4 Results and Conclusion ...... 35

4.1 Results ...... 35

4.2 Conclusion ...... 44

4.3 Future Work ...... 46

Bibliography ...... 48

viii List of Figures

1.1 Deep Neural Network ...... 3

1.2 DBSCAN Algorithm ...... 7

1.3 Single Unit Single Layer Recurrent Neural Network ...... 10

1.4 Unfolded Recurrent Neural Network ...... 11

1.5 LSTM Unit ...... 13

3.1 Computational Grid ...... 23

3.2 Flow ﬁeld ...... 24

3.3 Acoustic Emission from a jet ...... 25

3.4 2D contour plots showing Q-criterion and Acoustic emission . . . 26

3.5 Vorticity and Acoustic in an azhimuthal plane ...... 27

3.6 Flowchart of the Technique ...... 28

3.7 Sequence to Sequence Neural Network ...... 34

4.1 Output of DBSCAN on a snapshot ...... 36

ix 4.2 Evaluation Result using POD Analysis ...... 37

4.3 Trace of a Vortex ...... 38

4.4 Vorticity vs Acoustic feature of a vortex time series ...... 39

4.5 Predictions by the Sequence to sequence learning model . . . . . 40

4.6 Training and Testing Accuracy of the model ...... 43

4.7 Training and Testing loss of the model ...... 43

4.8 Predictions dissimilar from Actual Time Series ...... 45

x Chapter 1

Introduction

Machine learning (ML) has been omnipresent in all fields of study from scientific areas of research such as social sciences and biology to non-scientific domains like business studies and sports. It encompasses a subset of techniques, from a more general field of Artificial Intelligence, that applies statistical or data-driven methods to extract implicit and hidden knowledge from data without any specific programming. The knowledge, thus, obtained is usually interpreted as either classification, prediction, or representation of data.

The most widely used machine learning algorithm is principle component analysis (PCA) that performs orthogonal transformations on multi-dimensional data to get linearly uncorrelated dimensions such that the majority percentage of statistical variance in data is captured by the initial dimensions or principle components.

Its generic application is to handle the curse of dimensionality amongst large scale

1 multi-variate datasets by projecting the data points onto the principle components before data mining. Different variants of PCA are actively used in various applications such as signal analysis and spatio-temporal data analysis. One of the variants called Proper Orthogonal Decomposition (POD) is used in computational fluid dynamics for reduced order modelling of the flow field such that the dominant modes captures majority of kinetic energy in the system [1].

Deep Neural Networks (DNN) commonly known as Deep Learning is a promising and evolving area in Machine Learning that focuses on solving complex data analysis and representation problems using a multi-layer multi-node framework. The basic concept of neural networks have been around since the 1940s, however, with the exponential growth in computational power especially using GPUs has facili- tated the building of deeper networks to train on multi-dimensional big data. Their successful applications range from the conventional domains such as image clas- siﬁcation, speech recognition and natural language processing to the modern and sophisticated areas like self-driving cars and computer games. Figure 1.1 shows a simple fully-connected deep neural network with two layers and nine units per layer.

Scientific domains such as bioinformatics [2], computational biology and physical sciences [3], to name a few, are finding wide application of deep learning techniques, in their research activities on modelling, classification, anomaly detection and forecasting using enormous datasets. Deep learning models are able to pro-

2 Figure 1.1: An example of a two layer neural network. The ﬁrst layer is the input and the last is the output. The input can be spatial data or time series. The output can be either discrete or continuous value. Middle two layers are called the hidden layers where activation functions are applied to the data. vide better approximation and accuracy over the conventional machine learning models such as Support Vector Machines and Random Forests. Similarly, active strides are also being made towards their application in the ﬁeld of computational

ﬂuid dynamics. DNN models that explicitly enforce fundamental physical prin- ciples of turbulent ﬂow are being utilized for modelling. In the paper by Kutz

[4], the author suggests that DNN models can be applied to turbulent ﬂows either for prediction or for the approximation of reduced order models. However, more focus has been given to approximation models than on forecasting models.

This work introduces a Machine Learning based technique that applies forecasting

3 model using deep neural network to predict intermittent acoustic events in a turbulent jet ﬂow. Attention is given to the vortical coherent structures formed due to free shear layer interaction and persist for a signiﬁcant time duration. The technique encompasses a three-stage process for isolating and tracking vortices, and predicting their acoustic response.

The formal thesis statement is as follows: jet noise emitted by civil and military aircrafts cause severe hearing related health hazards to ground support staff work- ing in close proximity. Turbulence in jet exhaust encompasses plethora of physical phenomena that attribute to these acoustic emissions. One such phenomenon is the generation of vortices due to free ﬂowing shear layer and their persistent dynamics. We believe that if these vortices are identiﬁed and their trajectory are captured, the time series data, thus, sampled can be utilized to train a state-of-the- art deep neural network model for predicting corresponding acoustic radiations.

Further description regarding the objective of the dissertation work is in chapter 2.

The following subsections present a brief overview of two main categories of machine learning techniques, that are Supervised and Unsupervised Learning and few speciﬁc methodologies to enable completeness and better comprehension of the dissertation.

4 1.1 Unsupervised Learning

Understanding the underlying implicit knowledge in a given data without additional information such as labels, encompass unsupervised learning. In principle, this is achieved by applying statistical or similarity-based methods to reveal the patterns in data. There are no loop back information to improve the output apart from hyper-parameters associated with a learning model, that could be calibrated to achieve constructive information. Different unsupervised learning techniques include data clustering, dimensionality reduction and anomaly detection.

Data clustering or cluster analysis comprises the labelling of a given set of data points such that those associated to a label are more similar to each other than dif- ferently labelled data points. It can also be imagined as an approach to classify data points where the classes are unknown. Cluster analysis segregates data into sepa- rate groups using various ideas for realization such as the statistical distribution of the objects in a group, minimum multi-dimensional distance between the objects or concentration of objects in data space. Here, the term objects is considered syn- onymous to data points. Signiﬁcant work has been done to introduce advanced clustering techniques accounting these methods of realization. Therefore, multiple clustering algorithms have been developed and are utilized depending on the user’s view of a group or cluster. Some of the popular algorithms are K-means,

DBSCAN and Hierarchical clustering.

5 1.1.1 DBSCAN

Density-based spatial clustering of applications with noise or DBSCAN [5] clus-

tering algorithm segregates data based on the spatial density of the data objects oc-

cupying different regions of the space. Data points that are co-located in a densely

populated manner are grouped together to form a cluster. The density of a region

deﬁned using two parameters and that are, the minimum distance between two

points, and minimum number of points necessary for a cluster minPts.

Different distance metrics can be selected based on the dataset. Some of the popu-

lar metrics are:

Euclidean Distance: It is the straight-line distance between two data points in a euclidean space. It is also known as L2 norm. Its equation is given as: q PN 2 deuclidean(p, q) = i=1(qi − pi)

Manhattan Distance: It is the sum of distance between the data points when pro- jected onto the coordinate axes. It is also known as L1 norm. The equation is given as:

PN dmanhattan(p, q) = i=1 | qi − pi |

Chebyshev Distance: It is calculated by ﬁnding the greatest distance between two data points amongst all the coordinate axes. It is also known as L∞ metric. The equation is given as:

dchebyshev(p, q) = maxi(| qi − pi |))

6 Here p and q represent the data points in N dimensional space.

Based on these hyper-parameters, each data point is identiﬁed as one of three

types, namely, core point, border point or noise point. A data point is labelled

core point if there exists atleast minPts number of neighbors that are at a distance

of utmost . Border Point, although, does not satisfy the conditions to be a core

point but lie within a distance of to one or more core points. All the remaining data points that does not satisfy any of the mentioned conditions are ignored as noise points. This is depicted in Figure 1.2.

Figure 1.2: An example of DBSCAN algorithm labelling the blue data points into core, border and noise points. Here minPts is set as 4. The circle represent the region surrounding a data point and its radius is .

7 The algorithm iteratively identifies and links core points by starting randomly, un- til it can no longer find a neighboring core point. Subsequently, all the linked core points and their neighbors form a cluster or group of data points. The algorithm shows non-deterministic output with respect to the border points that have affilia- tion towards more than one core points. Noise points identified during the process are excluded.

The major advantage of DBSCAN is its ability to give out the clusters without any prior knowledge regarding the number of clusters in the data space. It efﬁciently captures clusters that has random spatial shape and size in an N-dimensional space. And ﬁnally, it detects and rejects outliers from the data automatically based on the setting of hyper-parameters.

1.2 Supervised Learning

Supervised Learning involves inferring of mapping function to relate the input information with the associated output. Such techniques essentially require labelled data composed of data pairs where each pair has a multi-dimensional vector rep- resenting the data point and ground truth information. They are broadly classiﬁed into two categories, Classiﬁcation and Regression. Few of the most commonly used algorithms are linear regression, logistic regression, support vector machine, naive bayes and neural networks.

8 Deep neural network (DNN) have recently gained traction due to their data driven approach and ﬂexibility in application. They are basically neural networks with multiple layers in between input and output layers. Several deep supervised learning architectures like convolutional neural network (CNN) and recurrent neural network (RNN) became popular with their seminal application in computer vision, speed recognition, handwriting recognition, natural language processing and social network representation.

In a DNN, nodes in a layer are the computation unit of the framework and are homogeneous in nature. Whereas layers are the non-homogeneous operatives that process and transform the input, and propagate it forward to the next layer. Every node of a layer solves a linear or non-linear function called the activation function on the input data of the layer and inter-layer weights, and the output, thus, given is combined with output from other similar nodes of the same layer and forwarded as input to the next computation layer that may or may not perform a similar operation on the input data. The inter-layer weights are updated based on the error in output by propagating back the error gradient through the layers. The weights are initialized randomly.

The technique introduced in this dissertation, uses recurrent neural network architecture for the prediction of acoustic signal given the properties of vortical structures in a turbulent flow field. Therefore, the following subsections gives a brief overview on RNN and a specific RNN unit called Long Short Term Memory (LSTM).

9 1.2.1 Recurrent Neural Network or RNN

Recurrent Neural Networks are simple feedforward networks but with an additional self loop called recurrent edge [6]. The self loop adds a temporal perspec- tive to the network enabling retention of temporal dependencies in a sequential data. RNN maintain internal state within each unit to retain historical information with respect to a temporal data. Figure 1.3 demonstrates a simple recurrent neural network. When this network is unrolled, it forms a chain like structure, shown in

Figure 1.3: The ﬁgure depicts a simple recurrent neural network showing the recurrent edge that loops back and facilitates the retention of temporal dynamics of sequential data. xt is the input and ht is the output at time step t.

10 ﬁgure 1.4, where each simple network is for a single time step of the sequence and the hidden state is shared across different time steps.

Figure 1.4: The ﬁgure shows an unfolded simple recurrent neural network showing the recurrent edge that links unit from one timestep to the next timestep for- warding the hidden state.

The compact form of the equations in an RNN unit is as follow:

(t) hx hh (t−1) h = σ(W xt + W h + bh)

(t) yh (t) yˆ = softmax(W h + by)

where W hx refers to the weight matrix between input and hidden layer, W hh is the recurrent weights and W yh is the weight vector between hidden layer and output layer. by and bh are the bias values.

The gradient descent technique used to update the weights of the layers of an

RNN is called Backpropagation Through Time (BPTT). Considering the unfolded form of a simple RNN, the error at each time step is retained. Once the complete

11 sequence is traversed, the cumulative error is propagated back through the unfolded RNN to update the shared weights by calculating the error gradient at each time step.

However, RNN fails to learn long term dependencies due to the vanishing or exploding gradient problem [7]. The vanishing gradient problem occurs when the error gradient, transmitted back through long time sequences becomes insigniﬁcant insofar the weights are not affected. One of the ways to overcome this problem is by using Long Short Term Memory units.

1.2.2 Long Short-Term Memory or LSTM

LSTM is a type of memory units used in recurrent neural network that store information of speciﬁed length in the memory cell [8]. These states are propagated back as input through a self loop, back into the unit. Moreover, the ﬂow of data through the unit is regulated by three sigmoidal gates, namely, input, output and forget gates and the information till the current time step is retained in the memory cell. Due to its capability to store previously seen information, an LSTM network is commonly used in time series prediction [9], speech recognition [10] and handwriting recognition [11].

The gates in an LSTM unit use sigmoid function for activation such that the 1 corresponds to retention of information and 0 means removal. Figure 1.5 depicts

12 the different gates and the memory cell.

Figure 1.5: The ﬁgure shows the set of operations in an LSTM network. In the ﬁgure, the * corresponds to element-wise multiplication (Hadamard Product) whereas + is element-wise addition. The loop-back is a feature of recurrent neural network.

The forget gate ft controls the amount of information to be stored in the memory

cell by deciding which elements in the vector associated with previous cell state

Ct−1 needs to be removed.

ft = σ(Wf Xt + Uf Ht−1)

Introduction of new information into the memory cell in place of the forgotten data

is handled by the input gate. This sigmoidal step determines which all elements

13 are to be updated.

It = σ(WiXt + UiHt−1)

This vector is multiplied with the candidate input values given by the tanh function, using Hadamard Product. Similarly, the previous cell state is multiplied with the vector by forget gate to remove old information. Finally, the output from these two gates are combined to generate the new cell state.

Ct = ft Ct−1 + It tanh(WcXt + UcHt−1)

The ﬁltered version of cell state using tanh function is multiplied with the vector given by sigmoid function of output gate to generate the output.

Ot = σ(WoXt + UoHt−1)

Ht = Ot tanh (Ct)

Here σ is the sigmoid function, tanh is the hyperbolic tangent function, is the

Hadamard product and subscript t corresponds to the time step.

14 Chapter 2

Objective and Related Work

Jet noise poses major challenges in civil and military aviation. Stricter noise lim- its in airports, and hearing-related health hazards in crew members operating in close proximity to aircraft engines, have lead to several efforts aimed at understanding the genesis of acoustic radiation in turbulent jets. Despite this, a fundamental explanation of noise generation from free shear ﬂows remain elusive. There are several factors that make acoustic analysis of jets challenging. A major factor is, although the acoustic waves result in severely detrimental effects on the surrounding, it accounts for only a fractional component of energy in the turbulent

fluctuations. This makes it difficult to predict formation and propagation of sound waves inside the turbulent region of the jet. In addition, noise characteristics of jets are highly sensitive to nozzle-operating conditions, properties of the inlet flow and geometric variations in the nozzle [12].

15 Acoustic radiation is a manifestation of compressibility of fluid flows. A fundamental understanding of fluctuations in such flows can help us better understand the causative mechanisms that are critical to noise generation. The first step towards this is an effective technique to quantify the exact amount of acoustic component that is present in those fluctuations. This is a non-trivial and often challenging task because, fluid fluctuations are usually dominated by hydrodynamic components, which necessarily does not contribute to acoustic phenomena of interest, and thus may lead to incorrect inferences from analyses of measured data.

Doak’s momentum potential theory (MPT) provides a mathematically-rigorous, and physically-consistent procedure, that allows one to split fluid fluctuations into constituent hydrodynamic (or vortical), acoustic and thermal (or entropic) components. Specifically, this decomposition is applied on the momentum fluctuations in the fluid. Such a decomposition essentially acts as a filtering procedure, by gen- erating spatio-temporal data of the pure acoustic mode, which can then be input to further analysis techniques.

Turbulent flows like jets also contain a broad range of spatial and temporal fluctuations. Some of these fluctuations occur across larger scales, and appear as coherent

flow features, that retain their signature over prolonged durations in space and time. An important category of coherent structures in jets is vortices, since they play a crucial role in activating acoustic emissions. The term vortex is qualitatively defined as a region of rotating fluid about an axis line with high vorticity. Vortic-

16 ity is a physical quantity that is mathematically deﬁned as the curl of the velocity vector and represented at a microscopic level as a measure of rotation of any point in the ﬂuid [13].

2.1 Objective

In the current scenario, we are interested in identifying causative mechanisms that result in significant acoustic activity in a supersonic jet. Based on prior works, the vortical structures in this jet, defined by positive Q-criterion, is chosen as the representative field for causative mechanisms. A critical choice here is, how the acoustic activity in the turbulent jet is quantified. For this, we choose the MPT- derived acoustic component, since it is devoid of hydrodynamic or thermal fluctuations. This provides an accurate and relatively low rank representation of the acoustically relevant dynamics in the turbulent region of the jet.

The main goal of this work is to predict intermittent events in the acoustic field of a jet flow based on the dynamics of the coherent structures formed by the free shear layer interaction. The target is to utilize machine learning techniques to isolate vortical structures in the flow to extract their statistically significant features to train a deep neural network model. The training would be focused to translate the vorticity associated to the vortices into acoustic signals. We also attempt to develop a vortex tracking algorithm that would efficiently capture the temporal

17 aspect of a vortex.

The target is to apply this technique to predict acoustic emissions in Mach 1.3 perfectly expanded supersonic jet. Training the machine learning algorithm using this

ﬁltered data will prevent false dependencies of acoustic-predictions, on energeti- cally dominant vortical signatures in the raw data.

2.2 Related Work

Significant amount of research has been conducted for mining vortical structures from fluid flows targeting the improvisation of their visualization. A case study by

Sadarjoen et. al. [14] detailed about the various approaches to visualize a vortex in a two dimensional and three dimensional framework. Their study focused on two important features of vortex that can be effectively harnessed for detection and eventually, visualization. The features include local scalar values such as velocity gradient and global geometric swirling pattern associated with a vortex.

Machine learning techniques have been utilized to facilitate the detection of vortices in large scale turbulent fluid flows. Zhang et. al. [15] apply boosting techniques to augment the weak vortex classification methods. They attempt to develop a robust vortex detection mechanism that combines the features considered by Q-criterion, ∆-criterion, λ2-method and Γ2-function. In addition, they employ

18 expert-in-the-loop to handle ambiguities, and false positives and negatives. Data clustering technique was employed by Padmesh et.al. [16] in their work to estimate threshold values for visualization of isosurfaces of vortical regions in unsteady

ﬂows. They apply k-means clustering to group local differences in λ2-criterion.

Deep learning has also started to mark their way into the ﬁeld of vortex detection.

Deng et. al. [17] show the application of Convolutional Neural Network (CNN) for the identiﬁcation of grid points that are within the vortical regions of a ﬂuid

ﬂow. The authors reduced the vortex detection task into binary classiﬁcation of grid points by training their CNN model on patches of the computational plane.

These works discussed mainly attempt to detect vortices as individual structures and does not look into their dynamics.

Mining interactions between the vortices in a ﬂow ﬁeld was attempted by Yang

[18], in her thesis work. A framework was developed to mine spatio-temporal structures in ﬂuid ﬂow and construct associations amongst these structures. These associations were based on their closeness and were mapped into graphical structures like Clique or Star. It is observed that the interactions amongst the participat- ing vortices of a graphical structure assumes the graph pattern’s characteristics.

The pattern and its variations through time were explored to infer the different events in the ﬂow ﬁeld such as convergence or separation of vortices.

Recent advancements in deep learning have also seen application in other research

19 domains of ﬂuid dynamics. Popular deep learning architecture, Long Short Term

Memory (LSTM) model has been applied for reduced order modelling the temporal dynamics of turbulent ﬂows. In this work by Arvind et. al. [19], the authors train LSTM and Bidirectional LSTM models to predict the time history of POD ba- sis. Zhang et. al. [20] use multilayer perceptron to improve existing equations for Reynolds Averaged NavierStokes (RANS) modelling by introducing new data- driven terms. The authors use perceptron to reconstruct the equation for the new terms and to study their spatio-temporal impact on the computation model. These terms are trained using high ﬁdelity simulation datasets generated using Large

Eddy Simulations (LES) and Direct Numerical Simulations (DNS) models. Ling et. al. [21] propose a novel neural network architecture that enforces a fundamental physical principle by introducing an additional layer in a multilayer perceptron.

The network is utilized to model rotational invariant Reynolds stress anisotropy tensor from high ﬁdelity simulation ﬂow datasets.

20 Chapter 3

Extraction of Vortex Time Series and

Prediction of Acoustic Response

3.1 Introduction

Interactions between the vortices generated by the shear layer is the major source of jet noise. However, capturing their dynamics and associating it with the acoustic activity is a challenging task. Emergence of deep learning techniques such as, recurrent neural networks have enabled data-driven learning of time series variables and predicting unknown future information. Our technique mainly focuses on the extraction of time series corresponding to potential vortices that acts as the source of intermittent events in the acoustic ﬁeld, and then using them to teach a deep neural network model to predict acoustic response given vorticity of a vortex.

21 The technique encompasses three important procedures to predict the acoustic response of a vortex. First process is to isolate vortices at every time snapshot of the data using a state-of-the-art data clustering algorithm. The following process is to capture the temporal evolution of every vortex isolated in each snapshot and extract the associated multi-variate time series. At last, the set of time series is used to train a deep recurrent neural network model designed as an encoder-decoder system to ﬁnally predict the corresponding acoustic response.

The technique is tested on a simulation dataset which is elaborated in the following section.

3.2 Description of the Data

The dataset is generated by a three dimensional simulation of Mach 1.3 perfectly expanded jet with Reynolds number of approximately 1 million [22]. The simulation executes multiple high-ﬁdelity numerical procedures solving governing

Navier-Stokes equations. The computation domain is a cylindrical grid of size 211

X 121 X 53, where 211 is the number streamwise planes, 121 is the number of circumferential plane and 53 corresponds to the number of azimuthal planes. The grid is shown in the Figure 3.1. The data consists of 2040 time-snapshots of the

ﬂow ﬁeld.

22 Figure 3.1: The ﬁgure shows the computational domain for the simulation. It is a cylindrical grid showing the a streamwise (red), circumferential (green) and azimuthal (blue) plane.

Vortices in the flow field are classified using Q-criterion. It is a function of velocity gradients, which highlight regions where fluid-rotation is prominent, and is thus used to identify vortices in the flowfield. The curl in a flow is the vorticity or velocity gradient that is used to characterize vortices. Figure 3.2 shows the different isotropic levels of the velocity gradient.

Subsequently, the acoustic ﬁeld is modelled from the data using Doak’s decomposition [23]. It is a generalized framework to extract the acoustic component

23 Figure 3.2: The figure shows the isotropic levels of magnitude of curl of the velocity. The red colored inner isotropic levels closer to the exhaust corresponds to high velocity region around the potential core. Whereas, the green and blue isotropic levels are of low velocity. of fluctuations in a compressible fluid from the governing Navier-Stokes equations, without any user-defined parameters. When applied to the unsteady flow data obtained from a DNS or LES, this process provides a time-accurate solution of the acoustic field at every spatial location in the computational domain. The acoustic field thus obtained is the isentropic and irrotational part of momentum

ﬂuctuations. This extracted acoustic ﬁeld can be further used for data analysis, since it better highlights the causative relationship between vortical mechanisms in the turbulent jet and its corresponding acoustic response. Figure 3.3 displays the acoustic isolevels of the Mach 1.3 jet extracted using Doak’s decomposition.

24 Figure 3.3: The ﬁgure shows the three dimensional isotropic levels of acoustic value derived using Doak’s Decomposition. The contours represent the value in k=1 azhimuthal plane.

For the simplification of data analysis, we sample the data from a two dimensional azimuthal plane. Specifically from k=1 azhimuthal plane. Figure 3.4(a) show the contours depicting the coherent structures of the flow in the selected azhimuthal plane of the computational grid and 3.4(b) depicts the acoustic emissions in the plane. Since, the three dimensional data is symmetric in nature, we assume that significant insights can be gained by analysis of data at a lower dimension and could be replicated to other planes. Since, majority of acoustic emissions originate

25 (a) 2D contour showing magnitude of curl of the velocity

(b) 2D contour showing acoustic emission

Figure 3.4: The ﬁgures (a) and (b) shows contour representation of velocity gradient and acoustic response, respectively, across an azhimuthal plane. closer to the jet exhaust towards the potential core, data is sampled from the region of interest where range of X coordinate is [2,10] and Y coordinate is [0,2]. Figure 3.5 shows the coherent structures and acoustic emission across in the region of interest of k=1 azhimuthal plane.

26 Figure 3.5: The contour lines show the vortices in the region of interest of the k=1 azhimuthal plane. Vortices with only positive Q-criterion values are shown in the ﬁgure. The black and white smooth contours in the background depicts the acoustic emissions.

3.3 Technique

Figure 3.6 brieﬂy outline the different stages and associated procedures to isolate and extract vortices, and predict their acoustic response. Each procedure is described in the following subsections.

3.3.1 Vortex Isolation

The data set consists of total 2040 time snapshot focused on the region of interest of the ﬂow ﬁeld. Each snapshot consists of 52X160 grid points with the set of features

27 Figure 3.6: The ﬁgure shows a ﬂowchart depicting the various processes involved in the technique along with the sub-tasks.

28 including Q-criterion, X coordinate, Y coordinate, grid indices I & J and acoustic value calculated using Doak’s Decomposition. These grid points are assumed to be the data points that are to be labelled as either a vortex point or otherwise.

Henceforward, the terms grid points and data points will be used interchangeably.

However, the data points are pre-processed to rescale each feature individually within the range of [0,1]. This is achieved using Min-Max Normalization method given by:

0 x−min(X) x = max(X)−min(X) where X is the feature vector, x is the individual element and x0 corresponds rescaled value. Furthermore, all the data points are ﬁltered against their Q-criterion such that grid points with negative value are ignored from the process of vortex isolation.

Vortex can be visualized to be contiguous spatial data points of signiﬁcantly high vorticity. Data clustering techniques can be applied to identify and group such high vorticity points together. Different state-of-the-art algorithms can be used to achieve the same. However, there are few features associated with the data that have to be taken into account while selecting a suitable data clustering algorithm.

First, the vortices acquire non-uniform shape and size in the two dimensional azimuthal plane of the grid. And second, due to mixing in the ﬂow, a considerable number of less signiﬁcant vortex like structures are formed that are short-lived and does not contribute acoustically.

29 Considering the above characteristics of the flow, the technique described here, thus, uses Density-based spatial clustering of applications with noise (DBSCAN) algorithm to isolate the vortices from each snapshot of the flow field. DBSCAN algorithm groups data points that form a non-spherical structure in the multi- dimensional data space. And also, it marks data points that are distant and asso- ciate to low density structure as noise points and ignores them. For further details, refer sub-section 1.1.1 in chapter 1.

The selected features for input to the algorithm comprises the grid indices I & J of the data points rather than the X & Y coordinates since the distribution of the coordinates are not spaced equally such that Y coordinate is divided more closely than the X coordinate. Whereas, I & J indices are uniformly distributed which facilitates the distance calculation. Non-uniform spatial distribution inﬂuence the distance between data points insofar as the distant data points along the X axis is classiﬁed in one vortex. Manhattan distance is used in the algorithm.

Since, the flow field is turbulent and stochastic in nature, the shape and size of a vortex varies through time in addition to its spatial displacement. Thus, the set of contiguous spatial points that constitute a given vortex in consecutive time steps may differ. This renders the task of tracking a vortex through time, based on its comprehensive structure more challenging. Therefore, a point based vortex tracking system is opted in the technique, where a representative point is selected from the set of grid points of a vortex as classified by DBSCAN algorithm.

30 The most common technique is to calculate the centroid of the data points. But, as discussed in chapter 2, vortex is a region of swirling fluid that revolves around an axis and the vorticity is highest for the points closer to the axis line. Considering this, the local point of maximum vorticity is selected to be the representative point for a vortex. Since the set of data points comprising a vortex is convex in nature, a single high vorticity point effectively represents the dynamics of the coherent vortical structure. These points are selected by finding the highest Q-criterion value in a local region within a vortex that is above a given threshold. The local region is set to be a rectangular box of size 6 grid points. Due to turbulent three dimensional nature of the flow, vortical elements often have multiple peaks within a given structure. Therefore, a considerable number of vortices in a snapshot can have more than one representative points.

The next stage is to track vortices through space and time and to extract the associated multivariate time series. This is described in the next section.

3.3.2 Extraction of Vortex Time Series

This section elaborates how the classiﬁcation of grid points as a vortex and the se- lection of its representative points can be used to track it and extract the associated multivariate time series.

Vortices are isolated in every snapshot and consecutively, their representative points

31 are selected. A vortex is tracked by predicting the position of its representative points in the following frame. This is achieved by calculating the maximum pos- sible displacement of the representative points by setting a threshold value, which can be set based on the dataset. In case of high ﬁdelity simulation, the temporal sampling is ﬁne enough such that the displacement of vortices in consecutive snapshot is small thus enabling a smaller threshold displacement value. The displacement is calculated using Chebyshev distance metric since it is empirically established that the representative point appear in either of the consecutive neighboring grid points in following snapshots.

A similar technique is shown in the work by Tissainayagam et. al. [24] where an object is tracked through a sequence of images using key feature points selected from the object contours. However, in their method, a centroid of the key feature points is calculated which is then tracked through consecutive images by predicting the centroid position. There are other methods such as by Mehta et. al. [25], that instead of looking into a limited set of points from a structure, focus on its attributes such as shape and size. In their work, they use angular and linear velocity along with the physical features of tangible structure to represent its trajectory.

Once a vortex is tracked from creation till dissipation through time, the attributes associated with the representative points in every snapshot is extracted to create time series of the vortex. A time series is deﬁned as the series of observation of single or multiple variables sampled over a period of time. Here in case of a vor-

32 tex, the variables consists of the position (X & Y coordinates), Q-criterion value and acoustic value. Hence, we track vortices developed at different timestep and extract corresponding time series to create a time series dataset that can be further applied to recurrent deep learning model for prediction.

3.3.3 Training the Sequence to Sequence Model

As elaborated in chapter 2, the target of this work is to predict the acoustic response generated by a vortex in the ﬂow ﬁeld. The processes in the previous sections, focused on the isolation and extraction of vortex sequences from a turbulent jet

ﬂow. The data, thus, derived is a set of time series that can be used train a recurrent neural network. RNN is a suitable candidate since it retains the implicit temporal dependencies of the data for an arbitrary length along with calculating the output.

Speciﬁcally, a sequence to sequence learning model [26] is used here to translate the vorticity associated with a vortex to the corresponding acoustic response.

The model is designed to contain an encoder and decoder layer. The encoder takes in a multi-dimensional variable length input and generates a ﬁxed-length representation. Whereas, the decoder converts the representation to another variable- length sequence [27]. This is pictorially shown in ﬁgure 3.7. However, in the time series dataset generated in this work, the vorticity input sequence and acoustic output sequence are of equal length. LSTM units are used in the encoder and de-

33 Figure 3.7: The ﬁgure shows the encoding and decoding layer of a sequence to sequence learning model. coder layers owing to its capability to learn long term dependencies and to handle vanishing or exploding gradient problem. Each layer is composed of 256 LSTM units. Adam optimizer is used along with mean squared error as the loss function.

The vortices in a ﬂow prolong for different durations. Thus, the lengths of the time series are non-uniform whereas the model accepts ﬁxed length input. This is handled by slicing every time series sequence into sub-sequences of span 5.

Zero-padding is applied to the last sub-sequence. The input sequence comprises vorticity and position parameters, whereas the output sequence contains only the acoustic value.

The time series dataset contains approximately 50,600 vortex sequences and 90% of the pre-processed sliced data is used for training the model. The data is consumed by the model in batches of size 64.

34 Chapter 4

Results and Conclusion

All the code was written using Python and the deep learning model was developed using keras and tensorﬂow. The model was trained on an NVIDIA GeForceGTX

960M GPU.

4.1 Results

As described in section 3.3, vortices detected using Q-criterion were isolated in every snapshot using DBSCAN algorithm. It was able to classify contiguous spatial grid points as vortices and this can be seen in ﬁgure 4.1. The grid indices were the input features of the data points and the output was set of labels mapping each data point to different vortices. Furthermore, the black circle marked in each color coded vortex in ﬁgure 4.1 are the representative point.

35 Figure 4.1: The figure depicts the different vortices isolated by DBSCAN algorithm. The total number of vortices is given as k=73. The specific time snapshot is given as t=5. Color coding of the contours is used to represent each vortex as identified by DBSCAN. The black circle shows the representative points of the vortices. The black and white contour in the background shows the acoustic field for the snapshot.

The result from DBSCAN is evaluated by extracting modes using Proper Orthog- onal Decomposition (POD) from output of 500 snapshot and comparing them to the modes from actual data. POD provides separation of variables for a spatio- temporal data set. That is, it splits the time-accurate data into a series of spatial modes and associated temporal coefficients. The spatial modes are orthonormal to each other and optimally represents the data in the least-squares sense. When applied to the Q-criterion field it acts as a filter, retaining only the most coherent and energetic vortices. POD is used here as a method-verification tool by applying it to the data before and after the action of the DBSCAN algorithm. It validates whether clustering captures the statistically significant grid points in every snapshot. Also,

36 it facilitates the ﬁne-tuning of hyper-parameters of DBSCAN algorithm.

Figure 4.2: The figure shows initial three POD modes generated from first 500 snapshots of the actual data and the output of DBSCAN. It is applied on the Q- criterion field. Modes on the left hand side are from the actual data and modes shown on the right hand side are from the output of DBSCAN. The black contour in the topmost figure corresponds to the vortices identified by DBSCAN and the color contour represents the actual data in a snapshot.

Figure 4.2 exhibits three initial statistical spatial modes extracted using POD from actual data and output of DBSCAN, respectively. The modes on the left hand side and right hand side are similar highlighting the conﬁdence of DBSCAN in isolating

37 (a) (b)

(e) (f)

(g) (h)

Figure 4.3: The set of ﬁgures show the positioning of a vortex based on its representative point, marked in the ﬁgures as red circle, at different time snapshot. The black and white contour shows acoustic activity.

38 the signiﬁcant vortices from every snapshot.

Following the application of DBSCAN and identifying representative points, the vortices were tracked using these points through consecutive snapshots. Figure 4.3 displays the trace of a vortex through different time snapshot and its footprint in the acoustic ﬁeld. Rate of displacement of the representative point of the vortex is seen to be very small and our technique is able to successfully track the movement of the vortex through grid space. The tracking is leveraged and the time series data featuring vorticity, position and acoustic emission is extracted.

Figure 4.4 exhibits the variation in acoustic emission of a vortex with respect to vorticity. This context matching is leveraged by the LSTM model for prediction.

Figure 4.4: The vortex time series extracted after tracking it consists of vorticity, acoustic and spatial features associated with its representative point. The ﬁgure shows the rate of change of acoustic emission with respect to variation in vorticity of a selected vortex.

39 (a) (b)

(e) (f)

(g) (h)

Figure 4.5: The set of ﬁgures show the translation output from sequence to sequence learning LSTM model trained. Given spatial and vorticity feature of a vortex time series, the model predicts the associated acoustic response. The corresponding similarity between the actual and predicted signals is shown on top of each subﬁgure.

40 The ﬁnal stage of the technique involves training a sequence to sequence learning

LSTM model that inputs vortex time series to learn the acoustic emission by the

vortex. The model is trained using 90 percent of the extracted vortex time series

and the rest is used for validation. The sub-ﬁgures in ﬁgure 4.5 show the actual vs

predicted acoustic signal given vorticity and position of different vortex through

time. The prediction shown in the ﬁgure establishes the capability of the model

to understand the variation in vorticity with respect to the position of a vortex

and to correlate it to its acoustic signature. This association is well captured by the

network in the form of model parameters and utilized in the translation of vorticity

to acoustic signal for unseen vortices of the ﬂow.

The most commonly applied time series similarity measures are euclidean distance

and dynamic time warping (DTW). Euclidean distance is a lock-step distance mea-

sure that performs one-on-one comparison of points at tth time step of both the time series, therefore, it is invariant to the order of the time series. Whereas, DTW is an elastic distance measure that aligns the time series (also called “warping”) such that the accumulated cost of the ﬁnal alignment is minimum [28]. However, the objective of this dissertation work is to see how well the pattern of acoustic emission of a vortex can be approximated by the DNN model, given the vorticity and spatial position. Furthermore, we focus on understanding the capability of DNN model, used in our technique, to translate the variation in vorticity of a vortex into acoustic signals. Therefore, these techniques are not suitable for the

41 effective evaluation of our method.

We use a similarity measure that compares the sign of the variation in acoustic signal between consecutive time steps. As in figure 4.5(a), we calculate the difference between the acoustic value at consecutive time step of both actual and predicted time series and compare their sign. An increase in acoustic value is considered as a positive sign and vice versa. A one-on-one comparison is made between the sign of the differences. If all the variation in actual and predicted time series match, we take the similarity value to be 4. The similarity values are element of Z and range from [0 − 4]. The subfigures in figure 4.5 display the similarity calculated using this technique. In figure 4.5(b), the similarity is shown to be 2 since the variation of acoustic value between time steps 3 & 4, and 4 & 5 are different for the actual and predicted time series.

The model is evaluated using accuracy of the prediction and mean squared error is used as the loss function. The ﬁgures 4.6 and 4.7 shows the rate of change of accuracy and loss, respectively. As can be seen the model shows a relatively infe- rior accuracy and loss values compared to similar models applied to other datasets such as the ones used in language translation. This implies that the model is re- stricted by the amount of training data.

42 Figure 4.6: The plot shows the rate in change in accuracy during training and testing of the learning model, respectively.

Figure 4.7: The plot shows the change in mean squared error based loss during training and testing of the model with learning iterations.

43 4.2 Conclusion

A three-stage technique was designed to isolate and extract vortex time series and predict intermittent acoustic emissions in a jet ﬂow. Based on the results, it could be inferred that the correlation between the vorticity and acoustic signature of a coherent structure can be leveraged to train state-of-the-art sequence translation deep neural network models and to, further, predict intermittent events in the acoustic

field of a jet flow. Such prediction shall facilitate the identification of certain statistically significant vortices that has a higher acoustic impact and therefore, shall enable necessary action for their control.

Reduced Order Modelling methods such as Doak’s decomposition [23] to extract acoustic field from experimental datasets is a challenging task. Our technique can be extended to such experimental datasets, that are generated using PIV techniques, for approximating the acoustic signals in the fluid flow. This can achieved by training the sequence translation model on high-fidelity simulation datasets and utilizing the same for prediction.

However, the technique does show some contrasting results during prediction. In

ﬁgure 4.8, the subﬁgures show predictions that are contrary to pattern of actual time series. The similarity value calculated is low indicating that variation in the predicted time series is different from the actual. The model fails to approximate

44 (a) (b)

Figure 4.8: The ﬁgures show negative results given by the sequence to sequence learning LSTM model used in our technique. The similarity calculated is very low indicating that the prediction does not approximate the pattern of the actual signal. the variations in the acoustic signal with respect to vorticity of vortex, given as input. This is a challenge encountered that needs to be investigated further to see what are the speciﬁc cases where the model fails and how this can be handled in our technique.

45 4.3 Future Work

The technique, introduced in this work, has been applied on the ﬁrst azhimuthal plane of the simulation dataset of a supersonic jet ﬂow. This can be validated using two dimensional data sampled from other planes and replicating the results to en- sure the independence of the technique from the computational grid. Furthermore, it can be tested on subsonic jets and simulation results generated on computational grids different from cylindrical structure.

Experimentation with the design of sequence to sequence learning model can be performed by using other RNN models like Gated Recurrent Unit (GRU) or Bidi- rectional LSTM to understand the effects of models on the prediction.

Spatial and vorticity features were used as input to the DNN model to approximate the acoustic pattern. It is important to understand other factors contributing to the intermittent events in the acoustic ﬁeld of a jet. Therefore, additional tests can be conducted to see the correlation of other physical parameters of a vortex such as velocity, angular velocity and viscosity, with associated acoustic emission and examine the quality of the prediction.

The interactions between the vortices play a major role in the generation of intermittent events and this work lack the capability to capture and map such significant dynamics to acoustic emissions. In the thesis by Yang [18], attention was

46 given to the interactions amongst the spatio-temporal scientiﬁc structures which were mapped into graphical structures for prediction of signiﬁcant events in the physical system. Efforts could be made to utilize this concept in our technique to translate the events predicted into acoustic response.

47 Bibliography

[1] Anupindi, K., “Proper Orthogonal Decomposition,” https://home.iitm. ac.in/kanupindi/course_projects/me611_pod_presentation. pdf, 2010, [Online].

[2] Min, S., Lee, B., and Yoon, S., “Deep learning in bioinformatics,” Brieﬁngs in Bioinformatics, Vol. 18, No. 5, 2017, pp. 851–869.

[3] Hatcher, W. G. and Yu, W., “A Survey of Deep Learning: Platforms, Appli- cations and Emerging Research Trends,” IEEE Access, Vol. 6, 2018, pp. 24411– 24432.

[4] Kutz, J. N., “Deep learning in ﬂuid dynamics,” Journal of Fluid Mechanics, Vol. 814, 2017, pp. 14.

[5] Ester, M., Kriegel, H.-P., Sander, J., and Xu, X., “A Density-based Algorithm for Discovering Clusters a Density-based Algorithm for Discovering Clus- ters in Large Spatial Databases with Noise,” Proceedings of the Second Inter- national Conference on Knowledge Discovery and Data Mining, KDD’96, AAAI Press, 1996, pp. 226–231.

[6] Lipton, Z. C., “A Critical Review of Recurrent Neural Networks for Sequence Learning,” CoRR, Vol. abs/1506.00019, 2015.

[7] Hochreiter, S., Bengio, Y., and Frasconi, P., “Gradient Flow in Recurrent Nets: the Difﬁculty of Learning Long-Term Dependencies,” Field Guide to Dynamical Recurrent Networks, edited by J. Kolen and S. Kremer, IEEE Press, 2001.

[8] Hochreiter, S. and Schmidhuber, J., “Long Short-Term Memory,” Neural Com- put., Vol. 9, No. 8, Nov. 1997, pp. 1735–1780.

48 [9] Schmidhuber, J., Wierstra, D., and Gomez, F., “Evolino: Hybrid neuroevolu- tion / optimal linear search for sequence prediction,” In Proceedings of the 19th International Joint Conference on Artiﬁcial Intelligence (IJCAI, Morgan, 2005, pp. 853–858.

[10] Graves, A., Mohamed, A., and Hinton, G., “Speech recognition with deep recurrent neural networks,” 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, May 2013, pp. 6645–6649.

[11] Graves, A., Fernandez,´ S., Liwicki, M., Bunke, H., and Schmidhuber, J., “Un- constrained Online Handwriting Recognition with Recurrent Neural Net- works,” Proceedings of the 20th International Conference on Neural Information Processing Systems, NIPS’07, Curran Associates Inc., USA, 2007, pp. 577–584.

[12] Sasidharan Nair, U., Jet noise source localization and identiﬁcation, Ph.D. thesis, The Ohio State University, 2017.

[13] Tian, S., Gao, Y., Dong, X., and Liu, C., “Deﬁnitions of vortex vector and vortex,” Journal of Fluid Mechanics, Vol. 849, 2018, pp. 312339.

[14] Sadarjoen, I. A., Post, F. H., Ma, B., Banks, D. C., and Pagendarm, H. ., “Se- lective visualization of vortices in hydrodynamic ﬂows,” Proceedings Visual- ization ’98 (Cat. No.98CB36276), Oct 1998, pp. 419–422.

[15] Zhang, L., Deng, Q., Machiraju, R., Rangarajan, A., Thompson, D., Walters, D. K., and Shen, H.-W., “Boosting Techniques for Physics-Based Vortex De- tection,” Comput. Graph. Forum, Vol. 33, No. 1, Feb. 2014, pp. 282–293.

[16] Padmesh, K., Ferrari, S., Hu, Y., and Martinuzzi, R. J., “Clustering-based threshold estimation for vortex extraction and visualization,” 2017 IEEE In- ternational Conference on Systems, Man, and Cybernetics (SMC), Oct 2017, pp. 677–682.

[17] Deng, L., Wang, Y., Liu, Y., Wang, F., Li, S., and Liu, J., “A CNN-based vortex identiﬁcation method,” Journal of Visualization, Oct 2018.

[18] Yang, H., A general framework for mining spatial and spatio-temporal object association patterns in scientiﬁc data, Ph.D. thesis, The Ohio State University, 2006.

[19] Mohan, A. T. and Gaitonde, D. V., “A Deep Learning based Approach to Re- duced Order Modeling for Turbulent Flow Control using LSTM Neural Net- works,” ArXiv e-prints, April 2018.

[20] Zhang, Z. J. and Duraisamy, K., chap. Machine Learning Methods for Data- Driven Turbulence Modeling, AIAA AVIATION Forum, American Institute of Aeronautics and Astronautics, Jun 2015, 0.

49 [21] Ling, J., Kurzawski, A., and Templeton, J., “Reynolds averaged turbulence modelling using deep neural networks with embedded invariance,” Journal of Fluid Mechanics, Vol. 807, 2016, pp. 155166.

[22] Gaitonde, D. V. and Samimy, M., “Coherent structures in plasma-actuator controlled supersonic jets: Axisymmetric and mixed azimuthal modes,” Physics of Fluids, Vol. 23, No. 9, 2011, pp. 095104.

[23] Unnikrishnan, S. and Gaitonde, D. V., “Acoustic, hydrodynamic and thermal modes in a supersonic cold jet,” Journal of Fluid Mechanics, Vol. 800, 2016, pp. 387432.

[24] Tissainayagam, P. and Suter, D., “Object tracking in image sequences using point features,” Pattern Recognition, Vol. 38, No. 1, 2005, pp. 105 – 113.

[25] Mehta, S., Parthasarathy, S., and Machiraju, R., “On Trajectory Represen- tation for Scientiﬁc Features,” Sixth International Conference on Data Mining (ICDM’06), Dec 2006, pp. 997–1001.

[26] Sutskever, I., Vinyals, O., and Le, Q. V., “Sequence to Sequence Learning with Neural Networks,” Proceedings of the 27th International Conference on Neural In- formation Processing Systems - Volume 2, NIPS’14, MIT Press, Cambridge, MA, USA, 2014, pp. 3104–3112.

[27] Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y., “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), As- sociation for Computational Linguistics, 2014, pp. 1724–1734.

[28] Serr, J. and Arcos, J. L., “An empirical evaluation of similarity measures for time series classiﬁcation,” Knowledge-Based Systems, Vol. 67, 2014, pp. 305 – 314.