Outline What is non-linear programming?
Math 408A: Non-Linear Optimization
Introduction Professor James Burke
Math Dept, University of Washington
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
What is non-linear programming?
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I The function to be minimized or maximized is called the objective function.
I The set of alternatives is called the constraint region (or feasible region).
I In this course, the feasible region is always taken to be a n subset of R (real n-dimensional space) and the objective n function is a function from R to R.
Outline What is non-linear programming?
What is optimization?
A mathematical optimization problem is one in which a given function is either maximized or minimized relative to a given set of alternatives.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I The set of alternatives is called the constraint region (or feasible region).
I In this course, the feasible region is always taken to be a n subset of R (real n-dimensional space) and the objective n function is a function from R to R.
Outline What is non-linear programming?
What is optimization?
A mathematical optimization problem is one in which a given function is either maximized or minimized relative to a given set of alternatives.
I The function to be minimized or maximized is called the objective function.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I In this course, the feasible region is always taken to be a n subset of R (real n-dimensional space) and the objective n function is a function from R to R.
Outline What is non-linear programming?
What is optimization?
A mathematical optimization problem is one in which a given function is either maximized or minimized relative to a given set of alternatives.
I The function to be minimized or maximized is called the objective function.
I The set of alternatives is called the constraint region (or feasible region).
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
What is optimization?
A mathematical optimization problem is one in which a given function is either maximized or minimized relative to a given set of alternatives.
I The function to be minimized or maximized is called the objective function.
I The set of alternatives is called the constraint region (or feasible region).
I In this course, the feasible region is always taken to be a n subset of R (real n-dimensional space) and the objective n function is a function from R to R.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization X is the variable space (or decision space),
f0 : X → R ∪ {±∞} is the objective function, and Ω ⊂ X is the constraint region.
Outline What is non-linear programming?
What is optimization?
P : minimize f0(x) x∈X subject to x ∈ Ω.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization f0 : X → R ∪ {±∞} is the objective function, and Ω ⊂ X is the constraint region.
Outline What is non-linear programming?
What is optimization?
P : minimize f0(x) x∈X subject to x ∈ Ω.
X is the variable space (or decision space),
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Ω ⊂ X is the constraint region.
Outline What is non-linear programming?
What is optimization?
P : minimize f0(x) x∈X subject to x ∈ Ω.
X is the variable space (or decision space),
f0 : X → R ∪ {±∞} is the objective function, and
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
What is optimization?
P : minimize f0(x) x∈X subject to x ∈ Ω.
X is the variable space (or decision space),
f0 : X → R ∪ {±∞} is the objective function, and Ω ⊂ X is the constraint region.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z
I unconstrained: Ω = X I constrained: Ω 6= X
I Variable Type:
I Constraint Type:
Outline What is non-linear programming?
What is optimization?
Problem Categories:
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I unconstrained: Ω = X I constrained: Ω 6= X
n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z I Constraint Type:
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type:
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I unconstrained: Ω = X I constrained: Ω 6= X
n I discrete variable: X = Z s t I mixed variable: X = R × Z I Constraint Type:
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I unconstrained: Ω = X I constrained: Ω 6= X
s t I mixed variable: X = R × Z I Constraint Type:
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R n I discrete variable: X = Z
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I unconstrained: Ω = X I constrained: Ω 6= X
I Constraint Type:
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I unconstrained: Ω = X I constrained: Ω 6= X
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z I Constraint Type:
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I constrained: Ω 6= X
Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z I Constraint Type:
I unconstrained: Ω = X
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
What is optimization?
Problem Categories:
I Variable Type: n I continuous variable: X = R n I discrete variable: X = Z s t I mixed variable: X = R × Z I Constraint Type:
I unconstrained: Ω = X I constrained: Ω 6= X
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Convex Programming: f0 a convex function, Ω a convex set n Definition: Ω ⊂ R is convex if for every x, y ∈ Ω one has [x, y] ⊂ Ω where [x, y] is the line segment connecting x and y:
[x, y] = {λx + (1 − λ)y : 0 ≤ λ ≤ 1}.
n Definition: f : R → R ∪ {±∞} is convex if n+1 epi(f ) = {(x, µ): f (x) ≤ µ} is a convex set in R . In particular,
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y)
for all 0 ≤ λ ≤ 1 and points x, y for which not both f (x) and f (y) are infinite.
Outline What is non-linear programming?
Problem Types
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization f0 a convex function, Ω a convex set n Definition: Ω ⊂ R is convex if for every x, y ∈ Ω one has [x, y] ⊂ Ω where [x, y] is the line segment connecting x and y:
[x, y] = {λx + (1 − λ)y : 0 ≤ λ ≤ 1}.
n Definition: f : R → R ∪ {±∞} is convex if n+1 epi(f ) = {(x, µ): f (x) ≤ µ} is a convex set in R . In particular,
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y)
for all 0 ≤ λ ≤ 1 and points x, y for which not both f (x) and f (y) are infinite.
Outline What is non-linear programming?
Problem Types
Convex Programming:
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization n Definition: Ω ⊂ R is convex if for every x, y ∈ Ω one has [x, y] ⊂ Ω where [x, y] is the line segment connecting x and y:
[x, y] = {λx + (1 − λ)y : 0 ≤ λ ≤ 1}.
n Definition: f : R → R ∪ {±∞} is convex if n+1 epi(f ) = {(x, µ): f (x) ≤ µ} is a convex set in R . In particular,
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y)
for all 0 ≤ λ ≤ 1 and points x, y for which not both f (x) and f (y) are infinite.
Outline What is non-linear programming?
Problem Types
Convex Programming: f0 a convex function, Ω a convex set
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization n Definition: f : R → R ∪ {±∞} is convex if n+1 epi(f ) = {(x, µ): f (x) ≤ µ} is a convex set in R . In particular,
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y)
for all 0 ≤ λ ≤ 1 and points x, y for which not both f (x) and f (y) are infinite.
Outline What is non-linear programming?
Problem Types
Convex Programming: f0 a convex function, Ω a convex set n Definition: Ω ⊂ R is convex if for every x, y ∈ Ω one has [x, y] ⊂ Ω where [x, y] is the line segment connecting x and y:
[x, y] = {λx + (1 − λ)y : 0 ≤ λ ≤ 1}.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Problem Types
Convex Programming: f0 a convex function, Ω a convex set n Definition: Ω ⊂ R is convex if for every x, y ∈ Ω one has [x, y] ⊂ Ω where [x, y] is the line segment connecting x and y:
[x, y] = {λx + (1 − λ)y : 0 ≤ λ ≤ 1}.
n Definition: f : R → R ∪ {±∞} is convex if n+1 epi(f ) = {(x, µ): f (x) ≤ µ} is a convex set in R . In particular,
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y)
for all 0 ≤ λ ≤ 1 and points x, y for which not both f (x) and f (y) are infinite.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Linear programming is a special case of convex programming. In this case the constraint region Ω is called a polyhedral convex set. Polyhedra have a very special geometric structure.
Outline What is non-linear programming?
Problem Types
Linear Programming: The minimization or maximization of a linear functional subject to a finite number of linear inequality and/or equality constraints. T n f0(x) := c x for some c ∈ R and T ≤ bi i = 1,..., s Ω := x : ai x . = bi i = s + 1,..., m
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Problem Types
Linear Programming: The minimization or maximization of a linear functional subject to a finite number of linear inequality and/or equality constraints. T n f0(x) := c x for some c ∈ R and T ≤ bi i = 1,..., s Ω := x : ai x . = bi i = s + 1,..., m
Linear programming is a special case of convex programming. In this case the constraint region Ω is called a polyhedral convex set. Polyhedra have a very special geometric structure.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization The minimization or maximization of a quadratic objective functions over a convex polyhedron: 1 f (x) = xT Qx + bT x + α 0 2
Fact: f0 is convex if and only if Q is positive semi-definite.
Outline What is non-linear programming?
Problem Types
Quadratic Programming:
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Fact: f0 is convex if and only if Q is positive semi-definite.
Outline What is non-linear programming?
Problem Types
Quadratic Programming: The minimization or maximization of a quadratic objective functions over a convex polyhedron: 1 f (x) = xT Qx + bT x + α 0 2
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Problem Types
Quadratic Programming: The minimization or maximization of a quadratic objective functions over a convex polyhedron: 1 f (x) = xT Qx + bT x + α 0 2
Fact: f0 is convex if and only if Q is positive semi-definite.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
I Differentiable: fi is smooth i = 0,..., m I Nonsmooth: at least one fi is not smooth I Semi-Infinite: m = +∞. Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I Differentiable: fi is smooth i = 0,..., m I Nonsmooth: at least one fi is not smooth I Semi-Infinite: m = +∞. Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}. Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I Nonsmooth: at least one fi is not smooth I Semi-Infinite: m = +∞. Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}. Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
I Differentiable: fi is smooth i = 0,..., m
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I Semi-Infinite: m = +∞. Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}. Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
I Differentiable: fi is smooth i = 0,..., m I Nonsmooth: at least one fi is not smooth
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}. Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
I Differentiable: fi is smooth i = 0,..., m I Nonsmooth: at least one fi is not smooth I Semi-Infinite: m = +∞.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Problem Types
Mini-Max: f0(x) = max{fi (x): i = 1,..., s}. Nonlinear Programming:
Ω := {x ∈ X : fi (x) ≤ 0, i = 1,..., s, fi (x) = 0, i = j + 1,... m}
I Differentiable: fi is smooth i = 0,..., m I Nonsmooth: at least one fi is not smooth I Semi-Infinite: m = +∞. Box Constraints: n Ω := {x ∈ R : li ≤ xi ≤ ui , i = 1,..., n} li ∈ R ∪ {−∞}, ui ∈ R ∪ {+∞}, li ≤ ui
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Problem Types
Parameter Identification: m t s Given data points {(xi , yi )}i=1 ⊂ R × R , find the function f (x) := m(p, x) from a parametrized class of functions
n M := {m(p, ·) |p ∈ Γ ⊂ R }
that “best” fits the data.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Parameter Identification
Polynomial Least-Squares: The function class is the set of polynomials of degree n or less.
n Pn = {m(p, x) = p0 + p1x + ··· + pnx |pi ∈ R, i = 0, 1,..., n} .
A ”best” fit can be found by minimizing the sum of squares
n X 2 f0(p) = (m(p, xi ) − yi ) i=1
n+1 over all choices of p ∈ R .
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization This is a matrix equation in the unknowns p0,..., pn.
2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 = . . . . . . 2 n 1 xm xm ··· xm pn ym
Outline What is non-linear programming?
Polynomial Least-Squares
We would like to satisfy the following equations.
2 n p0 + p1x1 + p2x1 + ··· + pnx1 = y1 2 n p0 + p1x2 + p2x2 + ··· + pnx2 = y2 . . . . 2 n p0 + p1xm + p2xm + ··· + pnxm = ym
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization 2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 = . . . . . . 2 n 1 xm xm ··· xm pn ym
Outline What is non-linear programming?
Polynomial Least-Squares
We would like to satisfy the following equations.
2 n p0 + p1x1 + p2x1 + ··· + pnx1 = y1 2 n p0 + p1x2 + p2x2 + ··· + pnx2 = y2 . . . . 2 n p0 + p1xm + p2xm + ··· + pnxm = ym
This is a matrix equation in the unknowns p0,..., pn.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Polynomial Least-Squares
We would like to satisfy the following equations.
2 n p0 + p1x1 + p2x1 + ··· + pnx1 = y1 2 n p0 + p1x2 + p2x2 + ··· + pnx2 = y2 . . . . 2 n p0 + p1xm + p2xm + ··· + pnxm = ym
This is a matrix equation in the unknowns p0,..., pn.
2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 = . . . . . . 2 n 1 xm xm ··· xm pn ym
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Typically, n << m so this equation has no solution. To get the ”best” solution we solve
1 2 min kXp − yk2 p∈Rn+1 2
This is an example of a linear least squares problem.
Outline What is non-linear programming?
Polynomial Least-Squares
Set 2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 X = , p = , and y = . . . . . . 2 n 1 xm xm ··· xm pn ym Then the matrix equation becomes Xp = y.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization To get the ”best” solution we solve
1 2 min kXp − yk2 p∈Rn+1 2
This is an example of a linear least squares problem.
Outline What is non-linear programming?
Polynomial Least-Squares
Set 2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 X = , p = , and y = . . . . . . 2 n 1 xm xm ··· xm pn ym Then the matrix equation becomes Xp = y. Typically, n << m so this equation has no solution.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization This is an example of a linear least squares problem.
Outline What is non-linear programming?
Polynomial Least-Squares
Set 2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 X = , p = , and y = . . . . . . 2 n 1 xm xm ··· xm pn ym Then the matrix equation becomes Xp = y. Typically, n << m so this equation has no solution. To get the ”best” solution we solve
1 2 min kXp − yk2 p∈Rn+1 2
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Polynomial Least-Squares
Set 2 n 1 x1 x1 ··· x1 p0 y1 2 n 1 x2 x2 ··· x2 p1 y2 X = , p = , and y = . . . . . . 2 n 1 xm xm ··· xm pn ym Then the matrix equation becomes Xp = y. Typically, n << m so this equation has no solution. To get the ”best” solution we solve
1 2 min kXp − yk2 p∈Rn+1 2
This is an example of a linear least squares problem.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Linear least squares problems are convex quadratic programs: 1 1 1 kAx − bk2 = xT AT Ax − (AT b)T x + bT b. 2 2 2 2
Outline What is non-linear programming?
Linear Least-Squares
A linear least squares problem is any optimization problem of the form 1 2 min kAx − bk2, x∈Rn 2 m×n m for some A ∈ R and b ∈ R .
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Linear Least-Squares
A linear least squares problem is any optimization problem of the form 1 2 min kAx − bk2, x∈Rn 2 m×n m for some A ∈ R and b ∈ R .
Linear least squares problems are convex quadratic programs: 1 1 1 kAx − bk2 = xT AT Ax − (AT b)T x + bT b. 2 2 2 2
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Nonlinear Programming
minimize f0(x)
subject to fj (x) ≤ 0, j = 1, 2,..., s
fj (x) = 0, j = s = 1,..., m .
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω. I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution. I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution. I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution. I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution. I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution. I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω. I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution. I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω. I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution. I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω. I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution. I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
n n f0 : R → R and Ω ⊂ R
I x ∈ Ω is said to be a global solution to the problem
min{f0(x): x ∈ Ω}
if f0(x) ≤ f0(x) for all x ∈ Ω. I If in fact f0(x) < f0(x) for all x ∈ Ω, then x is said to be a strict global solution. I x ∈ Ω is said to be a local solution to the problem min{f0(x): x ∈ Ω} if there is an > 0 such that
f0(x) ≤ f0(x) for all x ∈ Ω satisfying kx − xk ≤ .
I If f0(x) < f0(x) for all x ∈ Ω with kx − xk ≤ , then x is called a strict local solution. I The solution x is said to be isolated if x is the only local solution in the set {x ∈ Ω: kx − xk ≤ }.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization An optimality condition is a simple practical test for optimality.
Theorem: If f0 is differentiable at x and x is a local solution to the n problem min{f0(x): x ∈ R }, then ∇f0(x) = 0.
The condition ∇f0(¯x) = 0 is necessary, but clearly not sufficient for optimality.
Outline What is non-linear programming?
Optimality Conditions
These definitions, although sensible, are not practical since they require one to check infinitely many points to determine either local or global optimality.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Theorem: If f0 is differentiable at x and x is a local solution to the n problem min{f0(x): x ∈ R }, then ∇f0(x) = 0.
The condition ∇f0(¯x) = 0 is necessary, but clearly not sufficient for optimality.
Outline What is non-linear programming?
Optimality Conditions
These definitions, although sensible, are not practical since they require one to check infinitely many points to determine either local or global optimality.
An optimality condition is a simple practical test for optimality.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization The condition ∇f0(¯x) = 0 is necessary, but clearly not sufficient for optimality.
Outline What is non-linear programming?
Optimality Conditions
These definitions, although sensible, are not practical since they require one to check infinitely many points to determine either local or global optimality.
An optimality condition is a simple practical test for optimality.
Theorem: If f0 is differentiable at x and x is a local solution to the n problem min{f0(x): x ∈ R }, then ∇f0(x) = 0.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Optimality Conditions
These definitions, although sensible, are not practical since they require one to check infinitely many points to determine either local or global optimality.
An optimality condition is a simple practical test for optimality.
Theorem: If f0 is differentiable at x and x is a local solution to the n problem min{f0(x): x ∈ R }, then ∇f0(x) = 0.
The condition ∇f0(¯x) = 0 is necessary, but clearly not sufficient for optimality.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization We design our algorithms to locate points that satisfy some testable, or constructive, optimality conditions and then terminate when the procedure “nearly” solves these conditions.
Our first order of business is to derive workable optimality conditions. In order to do this we must first review some facts from linear algebra and multi-variable calculus.
Outline What is non-linear programming?
Optimality Conditions
Optimality conditions play a key role in both the design of our algorithms and our tests for termination.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Our first order of business is to derive workable optimality conditions. In order to do this we must first review some facts from linear algebra and multi-variable calculus.
Outline What is non-linear programming?
Optimality Conditions
Optimality conditions play a key role in both the design of our algorithms and our tests for termination.
We design our algorithms to locate points that satisfy some testable, or constructive, optimality conditions and then terminate when the procedure “nearly” solves these conditions.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization Outline What is non-linear programming?
Optimality Conditions
Optimality conditions play a key role in both the design of our algorithms and our tests for termination.
We design our algorithms to locate points that satisfy some testable, or constructive, optimality conditions and then terminate when the procedure “nearly” solves these conditions.
Our first order of business is to derive workable optimality conditions. In order to do this we must first review some facts from linear algebra and multi-variable calculus.
Introduction Professor James Burke Math Dept, University of Washington Math 408A: Non-Linear Optimization