Differential Equations & Functional Data Analysis Parameter Estimation for Differential Equations Whitney Huang Department of Statistics Purdue University April 21, 2014 Whitney Huang (Purdue University) DE's & FDA April 21, 2014 1 / 27 Outline 1 Motivation 2 The estimation procedure 3 Example: Groundwater Levels Whitney Huang (Purdue University) DE's & FDA April 21, 2014 2 / 27 Differential Equations A differential equation is an equation that relates some function of one or more variables with its derivatives. Ordinary differential equation: mx¨ + bx_ + kx = Fext Partial differential equation: @u @2u @2u @2u − α + + = 0 @t @x2 @y 2 @z2 Whitney Huang (Purdue University) DE's & FDA April 21, 2014 3 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations I Widely used to model dynamic systems I Maxwell's equations in electromagnetism I Navier{Stokes equations in fluid dynamics in fluid dynamics I The Black{Scholes PDE in Economics I Forward problem (i.e. solving differential equations) has been studied extensively by mathematicians Inverse Problem Suppose a given data set can be reasonably model by a differential equation but with unknown coefficients. Can we make a statistical inference? Whitney Huang (Purdue University) DE's & FDA April 21, 2014 4 / 27 Differential Equations & Functional Data Analysis Most dynamic systems defined by the solutions of their differential equations are not fit to data, they intend to capture gross shape features in the specified context. However, ··· I Solutions of differential equations are functions I We can treat the data as an approximated solution of the corresponding differential equation ) FDA framework can help for the inverse problem Whitney Huang (Purdue University) DE's & FDA April 21, 2014 5 / 27 Differential Equations & Functional Data Analysis Most dynamic systems defined by the solutions of their differential equations are not fit to data, they intend to capture gross shape features in the specified context. However, ··· I Solutions of differential equations are functions I We can treat the data as an approximated solution of the corresponding differential equation ) FDA framework can help for the inverse problem Whitney Huang (Purdue University) DE's & FDA April 21, 2014 5 / 27 Differential Equations & Functional Data Analysis Most dynamic systems defined by the solutions of their differential equations are not fit to data, they intend to capture gross shape features in the specified context. However, ··· I Solutions of differential equations are functions I We can treat the data as an approximated solution of the corresponding differential equation ) FDA framework can help for the inverse problem Whitney Huang (Purdue University) DE's & FDA April 21, 2014 5 / 27 Differential Equations & Functional Data Analysis Most dynamic systems defined by the solutions of their differential equations are not fit to data, they intend to capture gross shape features in the specified context. However, ··· I Solutions of differential equations are functions I We can treat the data as an approximated solution of the corresponding differential equation ) FDA framework can help for the inverse problem Whitney Huang (Purdue University) DE's & FDA April 21, 2014 5 / 27 Differential Equations & Functional Data Analysis Most dynamic systems defined by the solutions of their differential equations are not fit to data, they intend to capture gross shape features in the specified context. However, ··· I Solutions of differential equations are functions I We can treat the data as an approximated solution of the corresponding differential equation ) FDA framework can help for the inverse problem Whitney Huang (Purdue University) DE's & FDA April 21, 2014 5 / 27 Outline 1 Motivation 2 The estimation procedure 3 Example: Groundwater Levels Whitney Huang (Purdue University) DE's & FDA April 21, 2014 6 / 27 Set-up Differential equation: f (x; x_; x¨; ··· ; θ) = 0 e.g.x _ = −βx + µ ) f =x _ + βx − µ = 0 Observed data: y(ti ) = x(ti ) + "i ; i = 1; ··· ; n i.i.d. 2 "i ∼ N(0; σ ) Goal: to estimate the unknown θ in the differential equation from the data with measurement error and to quantify the uncertainty of the estimates Whitney Huang (Purdue University) DE's & FDA April 21, 2014 7 / 27 Statistical Challenge Most differential equations are not solvable analytically. In order to carry out the estimation I it requires repeatedly solving differential equation numerically I the initial value of the differential equation need to be known Whitney Huang (Purdue University) DE's & FDA April 21, 2014 8 / 27 Statistical Challenge Most differential equations are not solvable analytically. In order to carry out the estimation I it requires repeatedly solving differential equation numerically I the initial value of the differential equation need to be known Whitney Huang (Purdue University) DE's & FDA April 21, 2014 8 / 27 Estimation procedure (Ramsay et al. 2007): basic idea 1 Use basis function expansion to approximate x(t), i.e^x(t) = cT φ(t) 2 Estimate the coefficients c of the chosen basis functions by incorporating differential equation defined penalty 3 Estimate the parameters θ in the differential equation 4 Choosing the amount of smoothing λ Whitney Huang (Purdue University) DE's & FDA April 21, 2014 9 / 27 Estimation procedure (Ramsay et al. 2007): basic idea 1 Use basis function expansion to approximate x(t), i.e^x(t) = cT φ(t) 2 Estimate the coefficients c of the chosen basis functions by incorporating differential equation defined penalty 3 Estimate the parameters θ in the differential equation 4 Choosing the amount of smoothing λ Whitney Huang (Purdue University) DE's & FDA April 21, 2014 9 / 27 Estimation procedure (Ramsay et al. 2007): basic idea 1 Use basis function expansion to approximate x(t), i.e^x(t) = cT φ(t) 2 Estimate the coefficients c of the chosen basis functions by incorporating differential equation defined penalty 3 Estimate the parameters θ in the differential equation 4 Choosing the amount of smoothing λ Whitney Huang (Purdue University) DE's & FDA April 21, 2014 9 / 27 Estimation procedure (Ramsay et al. 2007): basic idea 1 Use basis function expansion to approximate x(t), i.e^x(t) = cT φ(t) 2 Estimate the coefficients c of the chosen basis functions by incorporating differential equation defined penalty 3 Estimate the parameters θ in the differential equation 4 Choosing the amount of smoothing λ Whitney Huang (Purdue University) DE's & FDA April 21, 2014 9 / 27 Basic function expansion K