Numerical methods in MATLAB

Mariusz Janiak p. 331 C-3, 71 320 26 44

c 2015 Mariusz Janiak

All Rights Reserved Contents

1 Introduction

2 Ordinary Differential Equations

3 Optimization Introduction

Using numeric approximations to solve continuous problems

Numerical analysis is a branch of mathematics that solves continuous problems using numeric approximation. It involves designing methods that give approximate but accurate numeric solutions, which is useful in cases where the exact solution is impossible or prohibitively expensive to calculate. Numerical analysis also involves characterizing the convergence, accuracy, stability, and computational complexity of these methods.a

aThe MathWorks, Inc. Introduction

Numerical methods in Matlab Interpolation, extrapolation, and regression Differentiation and integration Linear systems of equations Eigenvalues and singular values Ordinary differential equations (ODEs) Partial differential equations (PDEs) Optimization ... Introduction

Numerical Integration and Differential Equations Ordinary Differential Equations – ordinary differential equation initial value problem solvers Boundary Value Problems – boundary value problem solvers for ordinary differential equations Delay Differential Equations – delay differential equation initial value problem solvers Partial Differential Equations – 1-D Parabolic-elliptic PDEs, initial-boundary value problem solver Numerical Integration and Differentiation – quadratures, double and triple integrals, and multidimensional derivatives Ordinary Differential Equations

An ordinary differential equation (ODE) contains one or more derivatives of a dependent variable y with respect to a single independent variable t, usually referred to as time. The notation used here for representing derivatives of y with respect to t is y 0 for a first derivative, y 00 for a second derivative, and so on. The order of the ODE is equal to the highest-order derivative of y that appears in the equation. Example y 00 = 2y Ordinary Differential Equations

In an Initial Value Problem, the ODE is solved by starting from an initial state. Using the initial condition y0 as well as a period of time over which the answer is to be obtained (t0, tf ), the solution is obtained iteratively. At each step the solver applies a particular algorithm to the results of previous steps. At the first such step, the initial condition provides the necessary information that allows the integration to proceed. The final result is that the ODE solver returns a vector of time steps t=[t0,t1,t2,...,tf] as well as the corresponding solution at each step y=[y0,y1,y2,...,yf]. Ordinary Differential Equations

Solvers in Matlab solve these types of first-order ODEs (1) Explicit ODEs of the form

y 0 = f (t, y)

Linearly implicit ODEs of the form

M(t, y)y 0 = f (t, y),

where M(t, y) is a nonsingular mass matrix. The mass matrix can be time- or state-dependent, or it can be a constant matrix. Linearly implicit ODEs can always be transformed to an explicit form y 0 = M−1(t, y)f (t, y) Ordinary Differential Equations

Solvers in Matlab solve these types of first-order ODEs (2) If some components of y 0 are missing, then the equations are called Differential Algebraic Equations (DAE)

y 0 = f (t, y, z), 0 = g(t, y, z),

and the system of DAEs contains some algebraic variables. Algebraic variables are dependent variables whose derivatives do not appear in the equations. The number of derivatives needed to rewrite a DAE as an ODE is called the differential index Ordinary Differential Equations

Solvers in Matlab solve these types of first-order ODEs (3) Fully implicit ODEs of the form

f (t, y, y 0) = 0

Fully implicit ODEs cannot be rewritten in an explicit form, and might also contain some algebraic variables Systems of ODEs

 0    y1 f1(t, y1, y2,..., yn)  0    y2 f2(t, y1, y2,..., yn)  .  =  .   .   .   .   .  0 yn fn(t, y1, y2,..., yn) Ordinary Differential Equations

Higher-Order ODEs (1) The Matlab ODE solvers only solve first-order equations The higher-order ODEs have to be rewritten as an equivalent system of first-order equations using the generic substitutions

  0 y1 = y y1 = y2   y = y 0 y 0 = y  2  2 3  00  0 y3 = y =⇒ y3 = y4 . . . .    (n−1)  0 yn = y yn = f (t, y1, y2, y3,..., yn) Ordinary Differential Equations

Higher-Order ODEs (2) Example: y 000 − y 00y + 1 = 0

  0 y1 = y y1 = y2  0  0 y2 = y =⇒ y2 = y3  00  0 y3 = y y3 = y1y3 − 1 Ordinary Differential Equations

Stiffness Lack of a precise definition In general, stiffness occurs when there is a difference in scaling somewhere in the problem, eg. two solution components that vary on drastically different time scales The step size taken by the solver is forced down to an unreasonably small level in comparison to the interval of integration, even in a region where the solution curve is smooth. Nonstiff solvers are unable to solve the problem or are extremely slow Stiff solvers reliability and efficiency can be improved by supplying the Jacobian matrix or its sparsity pattern Ordinary Differential Equations

Basic Solver Selection

Solver Problem Type Accuracy When to use ode45 Nonstiff Medium Should be the first solver you try ode23 Nonstiff Low Can be more efficient than ode45 at problems with crude tolerances, or in the presence of moderate stiffness ode113 Nonstiff Low to High Can be more efficient than ode45 at problems with stringent error tolerances, or when the ODE function is expensive to evaluate ode15s Stiff Low to Medium Use when ode45 fails or is inefficient and you suspect that the problem is stiff. Also use ode15s when solving differential algebraic equations (DAEs). ode23s Stiff Low Can be more efficient than ode15s at problems with crude error tolerances. It can solve some stiff problems for which ode15s is not effective. ode23t Stiff Low Use if the problem is only moderately stiff and you need a solution without numerical damping. Can solve differential algebraic equations (DAEs) ode23tb Stiff Low Solver might be more efficient than ode15s at problems with crude error tolerances ode15i Fully implicit Low Use for fully implicit problems f (t, y, y 0) = 0 and for diffe- rential algebraic equations (DAEs) of index 1. Ordinary Differential Equations

Solver ode45 (1)

[t,y] = ode45(odefun ,tspan ,y0) [t,y] = ode45(odefun ,tspan ,y0,options) [t,y,te,ye,ie] = ode45(odefun,tspan ,y0,options) s o l = ode45 ( )

Integrates the system of differential equations y 0 = f (t, y) on time horizon tspan with initial conditions y0 odefun — Functions to solve, specified as a function handle which defines the functions to be integrated. The function dydt = odefun(t,y), for a scalar t and a column vector y, must return a column vector dydt of data type single or double that corresponds to f (t, y). odefun must accept both input arguments, t and y, even if one of the arguments is not used in the function Ordinary Differential Equations

Solver ode45 (2) tspan – Interval of integration, specified as a vector. At minimum, tspan must be a two element vector [t0 tf] specifying the initial and final times. To obtain solutions at specific times between t0 and tf, use a longer vector of the form [t0,t1,t2,...,tf]. The elements in tspan must be all increasing or all decreasing. The solver imposes the initial conditions, y0, at tspan(1), then integrates from tspan(1) to tspan(end) y0 – Initial conditions, specified as a vector. y0 must be the same length as the vector output of odefun, so that y0 contains an initial condition for each equation defined in odefun Ordinary Differential Equations

Solver ode45 (3) options – Option structure, specified as a structure array. Use the odeset function to create or modify the options structure t – Evaluation points, returned as a column vector y – Solutions, returned as an array. Each row in y corresponds to the solution at the value returned in the corresponding row of t. te – Time of events, returned as a column vector ye – Solution at time of events (te), returned as an array ie – Index of vanishing event function, returned as a column vector sol – Structure for evaluation, returned as a structure array. Use this structure with the deval function to evaluate the solution at any point in the interval [t0 tf] Ordinary Differential Equations

ODE Event Location (1) Use event functions to detect when certain events occur during the solution of an ODE Event functions take an expression that user specify, and detect an event when that expression is equal to zero They can also signal the ODE solver to halt integration when they detect an event Use the ’Events’ option of the odeset function to specify an event function Ordinary Differential Equations

ODE Event Location (2) The event function must have the general form

[value ,isterminal ,direction] = myEventsFcn(t,y)

The output arguments are vectors whose ith element corresponds to the ith event value(i) is a mathematical expression describing the ith event. An event occurs when value(i) is equal to zero isterminal(i) = 1 if the integration is to terminate when the ith event occurs. Otherwise, it is 0. direction(i) = 0 if all zeros are to be located (the default). A value of +1 locates only zeros where the event function is increasing, and -1 locates only zeros where the event function is decreasing. Specify direction = [] to use the default value of 0 for all events. Ordinary Differential Equations

Unicycle example (1)

Z (q 1 ,q 2 ) Y

q3 X  q˙ 1 = u1 cos q3,  q˙ 2 = u1 sin q3, t = [0, 2π], q(0) = (0, 0, 0).  q˙ 3 = u2,

Assuming control u1 = 1, u2 = 0.1 sin t, we are looking for solution

q(t) = ϕq0,t (u). Ordinary Differential Equations

Unicycle example (2) File unicycle.m

f u n c t i o n Dq = unicycle(t, q, u) % unicycle(t,q,u) −− system definition % %t −− time %q −− s t a t e vector %u −− c o n t r o l

Dq = zeros(3, 1); Dq( 1 ) = u ( 1 ) * cos ( q ( 3 ) ) ; Dq( 2 ) = u ( 1 ) * s i n ( q ( 3 ) ) ; Dq(3) = u(2); Ordinary Differential Equations

Unicycle example (3) File sym1.m

% Set symulaiton parameters q0 =[0; 0; 0]; t s p a n = [ 0 2* p i ] ; u = [ 1 ; 0 . 1 ] ; % Define odefun ans solve fun = @(t, y)(unicycle(t, y, [1, 0.1 * s i n ( t ) ] ) ) ; [t, q] = ode45(fun, tspan, q0); % Plot rezults figure; ax1 = subplot(1,2,1); ax2 = subplot(1,2,2); plot(ax1, q(:,1), q(:,2)) x l a b e l ( ' q 1 ' ) ; y l a b e l ( ' q 2 ' ); t i t l e ( ax1 , ' Path ' ) plot(ax2, t, q) x l a b e l ( ' t ' ) ; y l a b e l ( ' q ' ); t i t l e ( ax2 , ' T r a j e c t o r y ' ) l e g e n d ( ' q 1 ' , ' q 2 ' , ' q 3 ' , 2 ) Ordinary Differential Equations

Unicycle example (4)

Path Trajectory 0.7 7 q 1 q 2 q 0.6 6 3

0.5 5

0.4 4 2 q q

0.3 3

0.2 2

0.1 1

0 0 0 2 4 6 8 0 2 4 6 8 q 1 t Optimization

Optimization techniques are used to find a set of design parameters, x = x1, x2, ..., xn, that can in some way be defined as optimal. In a simple case this might be the minimization or maximization of some system characteristic that is dependent on x. In a more advanced formulation the objective function, f (x), to be minimized or maximized, might be subject to constraints in the form of equality constraints, Gi (x) = 0, (i = 1, ..., me ); inequality constraints, 1 Gi (x) ¬ 0, (i = me + 1, ..., m); and/or parameter bounds, xl , xu.

1MathWorks Inc. Optimization

A General Problem (GP) description is stated as

min f (x) x subject to

Gi (x) = 0 i = 1,..., me

Gi (x) ¬ 0 i = me + 1,..., m where x is the vector of length n design parameters, f (x) is the objective function, which returns a scalar value, and the vector function G(x) returns a vector of length m containing the values of the equality and inequality constraints evaluated at x. Optimization

Optimization Toolbox Solvers linear programming (LP) mixed-integer linear programming (MILP) quadratic programming (QP) nonlinear programming (NLP) constrained linear least squares nonlinear least squares nonlinear equations optimtool – Select solver and optimization options, run problems Optimization

Optimization Decision Table

Objective Type Constraint Type Linear Quadratic Last Squares Smooth Nonlinear Nonsmooth mldivide, n/a (f = const, fminsearch, None quadprog lsqcurvefit, fminsearch,* or min = −∞) fminunc lsqnonlin lsqcurvefit, lsqlin, fminbnd, Bound linprog quadprog fminbnd,* lsqnonlin, fmincon, fseminf lsqnonneg Linear linprog quadprog lsqlin fmincon, fseminf * General Smooth fmincon fmincon fmincon fmincon, fseminf * Discrete, with intlinprog * * * * Bound or Linear

* means relevant solvers are found in Global Optimization Toolbox Optimization fmincon – Nonlinear programming solver (1) Find minimum of constrained nonlinear multivariable function Finds the minimum of a problem specified by

min f (x) x subject to

c(x) ¬ 0

ceq(x) = 0 A · x ¬ b

Aeq · x = beq lb ¬ x ¬ ub Optimization fmincon – Nonlinear programming solver (2) x = fmincon(fun,x0,A,b) x = fmincon(fun ,x0,A,b,Aeq,beq) x = fmincon(fun ,x0,A,b,Aeq,beq,lb ,ub) x = fmincon(fun ,x0,A,b,Aeq,beq,lb ,ub,nonlcon) x = fmincon(fun ,x0,A,b,Aeq,beq,lb ,ub,nonlcon ,options) x = fmincon(problem) [x,fval] = fmincon( ) [x,fval ,exitflag ,output] = fmincon( ) [x,fval ,exitflag ,output ,lambda,grad,hessian] = fmincon( )

fun – Function to minimize

f u n c t i o n f = myfun(x) f = ... %Compute function value at x

x0 – Initial point, specified as a real vector or real array Optimization fmincon – Nonlinear programming solver (3) A – Linear inequality constraints, matrix b – Linear inequality constraints, vector Aeq – Linear quality constraints, matrix beq – Linear equality constraints, vector lb – Lower bounds, specified as a real vector or real array ub – Upper bounds, specified as a real vector or real array nonlcon – Nonlinear constraints, a function handle or name

f u n c t i o n [c,ceq] = mycon(x) c = ... % Compute nonlinear inequalities at x. ceq = ... % Compute nonlinear equalities at x.

... Optimization

Example (1) Find the point where Rosenbrock’s function

2 2 f (x) = 100 ∗ (x2 − x1) + (1 − x1)

1 1 1 is minimized within a circle centered at [ 3 , 3 ] with radius 3

 12  12 12 c(x) = x − + x − − 1 3 2 3 3 subject to bound constraints 0 ¬ x1 ¬ 0.5, 0.2 ¬ x2 ¬ 0.8. File circlecon.m

f u n c t i o n [c,ceq] = circlecon(x) c = ( x ( 1 ) −1/3)ˆ2 + (x(2) −1/3)ˆ2 − (1/3) ˆ 2 ; ceq = [ ] ; Optimization

Example (2) File sym2.m % Define problem fun =@(x)100*( x ( 2 )−x(1)ˆ2)ˆ2 + (1−x ( 1 ) ) ˆ 2 ; A = [ ] ; b = [ ] ; Aeq = [ ] ; beq = [ ] ; lb =[0, 0.2]; ub =[0.5, 0.8]; x0 =[1/4, 1/4]; nonlcon = @circlecon; % Solve x = fmincon(fun ,x0,A,b,Aeq,beq,lb ,ub,nonlcon)

Solution x = 0.5000 0.2500 The End

Thank you for your kind attention.