SOME CHARACTERIZATIONS AND PROPERTIES OF

THE DISTANCE TO ILLPOSEDNESS AND

THE CONDITION MEASURE OF A CONIC LINEAR SYSTEM

Rob ert M Freund

MIT

Jorge R Vera

Catholic University of Chile

Octob er Revised June Revised June

Abstract

A conic linear system is a system of the form

P d nd x that solves b Ax C x C

Y X

where C and C are closed convex and the data for the system is d A b This system

X Y

iswellposed to the extent that small changes in the data A b do not alter the status of the

system the system remains solvable or not Renegar dened the distance to illp osedness

d to b e the smallest change in the data d A b for which the system P d d is

illp osed ie d d is in the intersection of the of feasible and infeasible instances

d A b of P Renegar also dened the condition measure of the data instance d as

C d kdkd and showed that this measure is a natural extension of the familiar condition

measure asso ciated with systems of linear equations This study presents two categories of

results related to d the distance to illp osedness and C d the condition measure of d

The rst category of results involves the approximation of d as the optimal value of certain

mathematical programs We present ten dierent mathematical programs each of whose optimal

values provides an approximation of d to within certain constants dep ending on whether P d

is feasible or not and where the constants dep end on prop erties of the cones and the norms

used The second category of results involves the existence of certain inscrib ed and intersecting

balls involving the feasible region of P d or the feasible region of its alternative system in the

spirit of the ellipsoid algorithm These results roughly state that the feasible region of P d or

its alternative system when P d is not feasible will contain a of radius r that is itself no

more than a distance R from the origin where the ratio R r satises R r c O C d and

1

1

such that r c and R c O C d where c c c are constants that dep end only on

2 3 1 2 3

C (d)

prop erties of the cones and the norms used Therefore the condition measure C d is a relevant

to ol in proving the existence of an inscrib ed ball in the feasible region of P d that is not to o

far from the origin and whose radius is not to o small

AMS Sub ject Classication C C C

Keywords Complexity of Innite Programming Point Metho ds

Conditioning Error Analysis

1

This research has b een partially supp orted through a grant from FONDECYT pro ject number by

NSF Grant INT and by a research fellowship from CORE Catholic University of Louvain

2

MIT OR Center Massachusetts Ave Cambridge MA USA email rfreundmitedu

3

Department of Industrial and System Engineering Catholic University of Chile Campus San Joaquin Vicuna

Mackenna Santiago CHILE email jveraingpuccl

“Some Characterizations and Properties of the ‘Distance to Ill-Posedness’ and the Condition Measure of a Conic Linear System.” Freund, Robert M., and Jorge R. Vera. Mathematical Programming Vol. 86, No. 2 (1999): 225-260. © Springer-Verlag Berlin Heidelberg 1999 http://doi.org/10.1007/s10107990063a

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Intro duction

This pap er is concerned with characterizations and prop erties of the distance to illp osedness

and of the condition measure of a conic linear system ie a system of the form

P d nd x that solves b Ax C x C

Y X

where C X and C Y are each a closed convex in the nite ndimensional normed

X Y

linear X with kxk for x X and in the nite mdimensional linear vector

space Y with norm ky k for y Y resp ectively Here b Y and A LX Y where LX Y

denotes the set of all linear op erators A X Y At the moment we make no assumptions on

C and C except that each is a closed The reader will recognize immediately that

X Y

n m n m

when X R and Y R and either i C fx R j x g and C fy R j y g

X Y

n m n m

ii C fx R j x g and C fg R or iii C R and C fy R j y g

X Y X Y

then P d is a linear inequality system of the format i Ax b x ii Ax b x or iii

Ax b resp ectively

The problem P d is a very general format for studying the feasible region of a mathematical

program and even lends itself to analysis by interiorp oint metho ds see Nesterov and Nemirovskii

and Renegar and

The concept of the distance to illp osedness and a closely related condition measure for

problems such as P d was intro duced by Renegar in in a more sp ecic setting but then

generalized more fully in and in We now describ e these two concepts in detail

We denote by d A b the data for the problem P d That is we regard the cones C

X

and C as xed and given and the data for the problem is the linear op erator A together with the

Y

vector b We denote the set of solutions of P d as X to emphasize the dep endence on the data d

d

ie

X fx X j b Ax C x C g

Y X

d

We dene

F fA b LX Y Y j there exists x satisying b Ax C x C g

Y X

Then F corresp onds to those data instances A b for which P d is consistent ie P d has a

solution

For d A b LX Y Y we dene the pro duct norm on the cartesian pro duct LX Y

Y as

kdk kA bk maxfkAk kbkg

where kbk is the norm sp ecied for Y and kAk is the op erator norm namely

kAk maxfkAxk j kxk g

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

C C

We denote the of F by F Then F consists precisely of those data instances

d A b for which P d is inconsistent

C

The b oundary of F and of F is precisely the set

C C

B F F cl F cl F

where S denotes the b oundary of a set S and cl S is the closure of a set S Note that if

d A b B then P d is illp osed in the sense that arbitrary small changes in the data

d A b will yield consistent instances of P d as well as inconsistent instances of P d

For any d A b LX Y Y we dene

d inf kdk inf kA bk

d A b

C

st d d B st A A b b cl F cl F

Then d is the distance to illp osedness of the data d ie d is the distance of d

to the set B of illp osedness instances In addition to the work of Renegar cited earlier further

analysis of the distance to illp osedness has b een studied by Vera Filip owski

and Nunez and Freund

In addition to the general case P d we will also b e interested in two sp ecial cases when

one of the cones is either the entire space or only the zerovector When C fg then P d

Y

sp ecializes to

Ax b x C

X

When C X then P d sp ecializes to

X

b Ax C x X

Y

One of the purp oses of this pap er is to explore approximate characterizations of the distance

to illp osedness d as the optimal value of a mathematical program whose solution is relatively

easy to obtain By relatively easy we roughly mean that such a program is either a convex

program or is solvable through O m or O n convex programs Vera and explored such

characterizations for linear programming problems and the results herein expand the scop e of this

of research in two ways rst by expanding the problem context from linear equations and

linear inequalities to conic linear systems and second by developing more ecient mathematical

programs that characterize d Renegar presents a characterization of the distance to ill

p osedness as the solution of a certain mathematical program but this characterization is not in

general easy to solve

There are a numb er of reasons for exploring various characterizations of d not the least

of which is to b etter understand the underlying nature of d First we anticipate that such

characterization results for d will b e useful in the complexity analysis of a variety of algorithms

for of problems in conic There is also the intellectual issue of the

complexity of computing d or an approximation thereof and there is the prosp ect of using such

characterizations to further understand the b ehavior of the underlying problem P d Furthermore

when an approximation of d can b e computed eciently then there is promise that the problem

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

of deciding the feasibility of P d or the infeasibility of P d can b e pro cessed eciently say in

p olynomial time as shown in In Section of this pap er we present ten dierent mathematical

programs each of whose optimal values provides an approximation of d to within certain constant

factors dep ending on whether P d is feasible or not and where the constants dep end only on the

structure of the cones C and C and not on the dimension or on the data d A b

X Y

The second purp ose of this pap er is to prove the existence of certain inscrib ed and inter

secting balls involving the feasible region of P d or the feasible region of the alternative system

of P d if P d is infeasible in the spirit of the ellipsoid algorithm and in order to set the stage

for an analysis of the ellipsoid algorithm hop efully in a subsequent pap er Recall that when P d

is sp ecialized to the case of nondegenerate linear inequalities and the data d A b is an array

of rational numb ers of bitlength L then the feasible region of P d will intersect a ball of radius

L L

R centered at the origin and will contain a ball of radius r where r n and R n

Furthermore the ratio R r is of critical imp ortance in the analysis of the complexity of using the

ellipsoid algorithm to solve the system P d in this particular case For the general case of P d

the Turing machine mo del of computation is not very appropriate for analyzing issues of complex

ity and indeed other mo dels of computation have b een prop osed see Blum et al also Smale

By analogy to the prop erties of rational nondegenerate linear inequalities mentioned ab ove

Renegar has shown that the feasible region X if nonempty must intersect a ball of radius R

d

centered at the origin where R kdkd Renegar denes the condition measure of the data

d A b to b e C d

kdk

C d

d

L

and so R C d Here we see the value n has b een replaced by the condition measure C d

For the problem P d considered herein in the feasible region is the set X In Sections

d

and of this pap er we utilize the characterization results of Section to prove that the feasible

region X or the feasible region of the alternative system when P d is infeasible must contain an

d

inscrib ed ball of radius r that is no more than a distance R from the origin and where the ratio R r

and R c O C d where satises R r c O C d Furthermore we prove that r c

C d

the constants c c c dep end on prop erties of the cones and the norms used and c c c

if the norms of the spaces are chosen in a particular way Note that by analogy to rational

L

nondegenerate linear inequalities the quantity n is replaced by C d Therefore the condition

measure C d is a very relevant to ol in proving the existence of an inscrib ed ball in the feasible

region of P d that is not to o far from the origin and whose radius is not to o small This should

prove eective in the analysis of the ellipsoid algorithm as applied to solving P d

The pap er is organized as follows Section contains preliminary results denitions and

analysis Section contains the ten dierent mathematical programs each of whose optimal values

provides approximations of d to within certain constant factors as discussed earlier Section

contains four lemmas that give partial or full characterizations of certain inscrib ed and intersecting

balls related to the feasible region of P d or its alternative region in the case when P d is

infeasible Section presents a synthesis of all of the results in the previous two sections into

theorems that give a complete treatment b oth of the characterization results and of the inscrib ed

and intersecting ball results

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Acknowledgment We gratefully acknowledge comments from James Renegar on the rst draft

of this pap er which have contributed to a restatement and simplication of the pro ofs of Lemma

and Lemma We also gratefully acknowledge the comments of the asso ciate editor which

have contributed to improved exp osition in the pap er We are also grateful for the fellowship and

research environment at CORE Catholic University of Louvain where this work was initiated

Preliminaries and Some More Notation

We will work in the setup of nite dimensional normed linear vector spaces Both X and

Y are normed linear spaces of nite dimension n and m resp ectively endowed with norms kxk for

x X and ky k for y Y For x X let B x r denote the ball centered at x with radius r ie

B x r fx X j kx x k r g

and dene B y r analogously for y Y

For d A b LX Y Y we dene the ball

B d r fd A b LX Y Y jkd d k r g

With this additional notation it is easy to see that the denition of d given in is

equivalent to

sup f j B d F g if d F

d

n o

C C

sup j B d F if d F

We asso ciate with X and Y the dual spaces X and Y of linear functionals dened on X

and Y resp ectively and whose induced dual norms are denoted by kuk for u X and kw k

for w Y Let c X In order to maintain consistency with standard linear algebra notation

in mathematical programming we will consider c to b e a column vector in the space X and will

T

denote the linear function cx by c x Similarly for A LX Y and f Y we denote Ax by

T T

Ax and f y by f y We denote the adjoint of A by A

If C is a convex cone in X C will denote the dual convex cone dened by

T

C fz X j z x for any x C g

Remark If we identify X with X then C C whenever C is a closed convex cone

Remark If C X then C fg If C fg then C X

X X

X X

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

We denote the set of real numb ers by R and the set of nonnegative real numb ers by R

Regarding the consistency of P d we have the following partial theorem of the alterna

tive the pro of of which is a straightforward exercise using a separating hyp erplane argument

Prop osition If P d has no solution then the system has a solution

T

A y C

X

y C

Y

T

y b

y

If the system has a solution

T

A y C

X

y C

Y

T

y b

then P d has no solution

Using Prop osition it is elementary to prove the following

Lemma Consider the set of il lposed instances B Then B can be characterized as

B fd A b LX Y Y j there exists x r X R with

x r and y Y with y satisfying br Ax C x C r

Y X

T T

y C A y C and y b g

Y X

We now recall some facts ab out norms Given a nite dimensional linear vector space X

endowed with a norm kxk for x X the dual norm induced on the space X is denoted by kz k

for z X and is dened as

T

kz k maxfz x j kxk g

If we denote the unit balls in X and X by B and B then it is straightforward to verify that

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

T

B fx X j kxk g fx X j z x for all z with kz k g

and

T

B fz X j kz k g fz X j z x for all x with kxk g

Furthermore

T

z x kz k kxk for any x X and z X

T

which is the Holder inequality Finally note that if A uv then it is easy to derive that

kAk kv k kuk using and

If X and V are nitedimensional normed linear vector spaces with norm kxk for x X

and norm kv k for v V then for x v X V the function f x v dened by

f x v kx v k kxk kv k

denes a norm on X V whose dual norm is given by

kw uk max fkw k kuk g for w u X V X V

The following result which is a sp ecial case of the HahnBanach Theorem see eg

will b e used extensively in our analysis We include a short pro of based on the sub dierential

op erator of a

Prop osition For every x X there exists z X with the property that kz k and

T

kxk z x

Pro of If x then any z X with kz k will satisfy the statement of the prop osition

Therefore we supp ose that x Consider kxk as a function of x ie f x kxk Then f is a

realvalued convex function and so the sub dierential op erator f x is nonempty for all x X

see Consider any x X and let z f x Then

T

f w f x z w x for any w X

T

Substituting w we obtain kxk f x z x Substituting w x we obtain f x f x

T T T

f x z x x and so f x z x whereby f x z x From it then follows that kz k

Now if we let u X and set w x u we obtain from that f u f x f u x f w

T T T T

f x z w x f x z u x x f x z u Therefore z u f u kuk and so from

we obtain kz k Therefore kz k

Because X and Y are normed linear vector spaces of nite dimension all norms on each

space are equivalent and one can sp ecify a particular norm for X and a particular norm for Y if

n

so desired If X R the L norm is given by

p

p

n

X

p

A

jx j kxk

j p

j

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

for p The norm dual to kxk is kz k kz k where q satises p q with appropriate

p q

limits as p and p

We will say that a cone C is regular if C is a closed convex cone has a nonempty interior

and is p ointed ie contains no line

Remark If C is a closed convex cone then C is regular if and only if C is regular

Let C b e a regular cone in the normed linear vector space X A critical comp onent of our

analysis concerns the extent to which the norm function kxk can b e approximated by some linear

T

function u x over the cone C for some particularly go o d choice of u X Let u intC b e

T

given and supp ose that u has b een normalized so that kuk Let f u minimumfu x j x

T

C kxk g Then it is elementary to see that f u and also that f ukxk u x kxk

T

for any x C Therefore the linear function u x approximates kxk over all x C to within the

T

factor f u Put another way the larger f u is the closer u x approximates kxk over all x C

Maximizing the value of f u over all u X satisfying kuk we are led to the following

denition

Denition If C is a regular cone in the normed linear vector space X the coecient of

linearity for the cone C is given by

T

sup inf u x

u X x C

kuk kxk

Let u denote that value of u X that achieves the supremum in We refer to u

T

generically as the norm approximation vector for the cone C Then for all x C kxk u x

T

kxk and so kxk is approximated by the linear function u x to within the factor over the cone

T

C Therefore measures the extent to which kxk can b e approximated by a linear function u x

T

on the cone C Also u x is the b est such linear approximation of kxk over this cone It is easy

T

to see that since u x kuk kxk for u and x as in The larger the value of the

T

more closely kxk is approximated by a linear function u x over x C For this reason we refer to

as the co ecient of linearity for the cone C

We have the following prop erties of the co ecient of linearity

Prop osition Suppose that C is a regular cone in the normed linear vector space X and let

denote the coecient of linearity for C Then Furthermore the norm approximation

vector u exists and is unique and satises the fol lowing propertiesi u intC

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

ii ku k

T

iii minfu x j x C kxk g and

T

iv kxk u x kxk for any x C

The pro of of Prop osition follows easily from the following observation

T

Remark Suppose C is a closed convex cone Then u intC if and only if u x for al l

T

x C fg Also if u intC the set fx C j u x g is a closed and bounded convex set

We illustrate the construction of the co ecient of linearity on two families of cones the

nn

n

and the p ositive semidenite cone S We rst consider the nonnegative nonnegative orthant R

n n n

f x R j x g Then we can identify X with X and in orthant Let X R and C R

n n

it is straightforward to show that as well If kxk kxk then for x R so doing C R

p

1

T T

p

u n e where e ie the linear function given by u x is the b est linear

n

approximation of the function kxk on the set R Furthermore straightforward calculation yields

1

p

that n Then if p but if p then

Now consider the p ositive semidenite cone which has b een shown to b e of enormous

imp ortance in mathematical programming see Alizadeh and Nesterov and Nemirovskii Let

nn

nn nn

X S denote the set of real n n symmetric matrices and let C S fx S j x g

where is the Lowner partial ordering ie x w if x w is a p ositive semidenite symmetric

matrix Then C is a closed convex cone We can identify X with X and in so doing it is

nn nn

elementary to derive that C S ie C S is selfdual For x X let x denote the

T

th

nvector of ordered eigenvalues of x That is x x x where x is the i

n i

largest eigenvalue of X For any p let the norm of x b e dened by

1

p

n

X

p

A

kxk kxk j xj

p j

j

ie kxk is the L norm of the vector of eigenvalues of x see eg for a pro of that kxk is a

p p p

norm

When p kxk corresp onds precisely to the Frob enius norm of x When p kxk is the

n

P

nn

x sum of the absolute values of the eigenvalues of x Therefore when x S kxk tr x

ii

i

nn

th

where x is the i diagonal entry of the real matrix x and so is linear on C S It is easy to

ii

1 1

nn

p p

I has ku k ku k and that n that u n show for the norm kxk over S

q p

p

Thus for the Frob enius norm we have and for the L norm we have

n

The co ecient of linearity for the regular cone C is essentially the same as the scalar

dened in Renegar on page In is referred to as a measure of p ointedness of the

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

cone C In fact one can dene p ointedness in a geometrically intuitive way and it can b e shown

that corresp onds precisely to the p ointedness of the cone C However this result is b eyond the

scop e of this pap er

The co ecients of linearity for the cones C andor C play a role in virtually all of the

X Y

results in this pap er Generally the results in Section and Section will b e stronger to the extent

that these co ecients of linearity are large The following remark shows that by a judicious choice

of the norm on the vector space X one can ensure that the co ecient of linearity for a cone C or

the co ecient of linearity for the dual cone C are equal to but not b oth

Remark If C is a regular cone then it is possible to choose the norm on X in such a way

that the coecient of linearity for C is Alternatively it is possible to choose the norm on

X in such a way that the coecient of linearity for C is

To see why this remark is true recall that for nite dimensional linear vector spaces that

all norms are equivalent Now supp ose that C is a regular cone Pick any u intC Let the unit

ball for X denoted as B b e dened as

T T

B conv fx C j u x g fx C j u x g

where convS T denotes the of the sets S and T It can then easily b e veried

that this ball induces a norm k k on X Furthermore it is easy to see that for all x C that

T

kxk u x whereby Alternatively a similar typ e of construction can b e applied to the dual

cone C to ensure that the co ecient of linearity for C satises However b ecause the

norm on X or on X induces the dual norm on the it is not generally p ossible to

construct the dually paired norms k k and k k in such a way that b oth and

Characterization Results for (d)

Given a data instance d A b LX Y Y we now present characterizations of the

distance to illp osedness d for the feasibility problem P d given in

C

The characterizations of d will dep end on whether d F or d F recall ie

whether P d is consistent or not We rst study the case when d F P d is consistent

C

followed by the case when d F P d is not consistent Before pro ceeding we adopt the

following notational conventions

For the remainder of this study we make the following mo dication of our notation

Denition Whenever the cone C is regular the coecient of linearity for C is denoted by

X X

and the coecient of linearity for C is denoted by Whenever the cone C is regular the

Y X

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

coecient of linearity for C is denoted by and the coecient of linearity for C is denoted by

Y

Y

Furthermore when the cone C is regular we denote the norm approximation vector for

X

the cone C by u Also when the cone C is regular we denote the norm approximation vector

X Y

for the cone C by z

Y

In particular then we have the following partial restatement of Prop osition

Corollary If C is regular then u intC and ku k and

X

X

T

kxk u x kxk for any x C

X

If C is regular then z intC and kzk and

Y Y

T

ky k z y ky k for any y C

Y

We emphasize that the four co ecient of linearity constants and dep end

only on the norms kxk and ky k and the cones C and C and are indep endent of the data A b

X Y

dening the problem P d

Characterization Results when P (d) is consistent

The starting p oint of our analysis is the following result of Renegar which we motivate

as follows Consider the following homogenization and normalization of P d

H

br Ax C

Y

x C

X

r

jr j kxk

Recall that d measures the extent to which the data d A b can b e altered and yet

will still b e feasible for the new system A mo dication of this view is that d measures the

extent to which the system can b e mo died while ensuring its feasibility Consider the following program

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

P d

r

r d minimum maximum

v Y r x

kv k st br Ax v C

Y

x C

X

r

jr j kxk

Then r d is the largest scaling factor such that for any v with kv k v can b e added to

the rst inclusion of H without aecting the feasibility of the system The following is a slightly

altered restatement of a result due to Renegar

Theorem Theorem of Suppose that d F Then

r d d

Now note that the inner maximization program of is a convex program If we replace

this inner maximization program by an appropriately constructed Lagrange dual we obtain the

following mo dication of program

n o

T T

minimum minimum max kA y q k jb y g j

v Y y q g

kv k

T

st y v

y C

Y

q C

X

g

By combining the inner and outer minimizations in and using the prop erties

of norms see we obtain the following program

P d

j

n o

T T

j d minimum max kA y q k jb y g j

y q g

st y C

Y

q C

X

g

ky k

Now note that program P d is a measure of how close the system is to b eing infeasible

j

C

To see this note that if d A b were in F then from Prop osition it would b e true that

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

j d The nonnegative quantity j d measures the extent to which the alternative system is

not feasible The smaller the value of j d is the closer the conditions are to b eing satised and

so the smaller the value of d should b e These arguments are imprecise but the next theorem

validates the intuition of this line of thinking

Theorem Suppose that d F Then

j d d

One way to prove would b e to prove that the duality constructs employed in mo difying

to to are indeed valid thereby showing that j d r d d and establishing that

program P d is just a partial dualization of P d Instead we oer the following pro of which is

j r

more direct and do es not rely explicitly on

Pro of Supp ose that j d d Then there exists d A b such that kA Ak j d and

C

kb bk j d and d F From Prop osition there exists y C with kyk that satises

Y

T T T T

A y C b y Let q A y and g b y Then

Y

T T T

kA y qk kA y q A A yk kA Akkyk j d

and

T T T T T

jb y gj jb y b b y b yj jb b yj kb bkkyk j d

and so y q g is feasible for P d with ob jective value less than j d which is a contradiction

j

Therefore j d d

Now supp ose that j d d for some Then there exists y q g such that y C

Y

T T T

q C g and kA y qk jb yg j and kyk Let y satisfy kyk y y kyk

X

T T T

see Prop osition and let A A y y A q b b y b y g for all Then

C T T T T

y g for all Therefore d A b F A y A y A y q q C and b

X

T T T

for all However kA Ak kA y qk and kb bk jb y g j jb y gj

for and suciently small and so d kd dk maxfkA Ak kb bkg for all

and suciently small This to o is a contradiction and so j d d whereby j d d

Remark P d is not in general a convex program due to the nonconvex constraint ky k

j

m m

However in the case when Y R then Y can also be identied with R if we choose

the norm on Y to be the L norm so the norm on Y is the L norm then P d can be

j

solved by solving m convex programs To see this observe that when ky k ky k in

P d then the constraint ky k can be replaced by the constraint y for some

j i

i f mg without changing the optimal objective value of P d This implies that j d j

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

min fj d j d j d j dg where

m m

n o

T T

j d minimum max kA y q k jb y g j

i

y q g

st y C

Y

q C

X

g

y

i

and j d is the optimal objective value of a convex program i m

i

We now pro ceed to present ve dierent mathematical programs each of whose optimal

values provides an approximation of the value of the distance to illp osedness d in the case when

P d is consistent For each of these ve mathematical programs the nature of the approximation

of d is sp ecied in a theorem stating the result For the rst program supp ose that C is a

X

regular cone and consider

P d

d minimum

y

T

st A y u C

X

T

b y

ky k

y C

Y

Theorem If d F and C is regular then

X

d d d

C

Pro of Recall from Corollary that u intC Therefore d since otherwise d F via

X

Prop osition which would violate the supp osition of the theorem Supp ose that y is feasible

T T T

for P d Let q A y u and notice that kA y q k ku k Also if we let g b y

n o

T T T

then g and jb y g j j j Therefore max kA y q k jb y g j and y q g is

feasible for P d with ob jective value It then follows that d j d d from

j

On the other hand supp ose that y q g is feasible for P d and let

j

n o

T T

max kA y q k jb y g j

Then it must b e true that

T

A y u C X

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

To demonstrate the validity of supp ose the contrary Then there exists x C with kxk

X

T T

for which x A y u But then

n o

T T T T T T T

max kA y q k jb y g j kA y q k kq A y k kxk q x x A y

T T T

x A y u x kxk

where the last inequality is from Corollary a contradiction Therefore is true Then

T T

also jb y g j b y and so y y is feasible for P d It then follows that

d j d d from completing the pro of

Similar to P d P d is generally a nonconvex program due to the constraint ky k

j

T

When C is also regular then from Corollary the linear function z y is a b est linear

Y

T

approximation of ky k on C and if we replace ky k by z y in P d we obtain the

Y

following convex program

P d

d minimum

y

T

st A y u C

X

T

b y

T

z y

y C

Y

Replacing the norm constraint by its linear approximation will reduce by a constant the

extent to which the program computes an approximation of d and the analog of Theorem

b ecomes

Theorem If d F and both C and C are regular then

X Y

d d d

T T

Pro of Supp ose that y is a feasible solution of P d Then y z y z y is a feasible

T

solution of P d with ob jective function value z y ky k from Corollary

It then follows that d d Applying Theorem we obtain d d

Next supp ose that y is a feasible solution of P d Then y ky k ky k is a feasible

T

solution of P d with ob jective function value ky k z y from Corollary It then

follows that d d Applying Theorem we obtain d d

The next mathematical program supp oses that the cone C is regular Consider the fol

X

lowing program

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

P d

w

w d minimum maximum

v Y r x

kv k st br Ax v C

Y

x C

X

T

r u x

r

Notice that P d is identical to P d except that the norm constraint jr j kxk in

w r

T

P d is replaced by the linearized version r u x We have

r

Theorem If d F and C is regular then

X

w d d w d

T

Pro of The pro of follows from using the inequalities kxk u x kxk of Corollary

using the same logic as in the pro of of Theorem

The fourth mathematical program supp oses that the cone C is regular Consider the

Y

following convex program

P d

u

n o

T T

ud minimum max kA y q k jb y g j

y q g

st y C

Y

q C

X

g

T

z y

Notice that P d is identical to P d except that the norm constraint ky k in P d

u j j

T

is replaced by the linearized version z y We have

Theorem If d F and C is regular then

Y

ud d ud

T

Pro of The pro of follows from using the inequalities ky k z y ky k of Corollary

using the same logic as in the pro of of Theorem

Notice that the feasible region of P d is a convex set and that the ob jective function is a

u

gauge function ie a nonnegative convex function that is p ositively homogeneous of degree see

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

A mathematical program that minimizes a gauge function over a convex set is called a gauge

program and corresp onding to every gauge program is a dual gauge program that also minimizes

a dual gauge function over a dual convex set see For the program P d its dual gauge

u

program is given by the following convex program

P d

v

v d minimum kxk jr j

x r

br Ax z C

Y

x C

X

r

One can interpret P d as measuring the extent to which P d has a solution x for which

v

b Ax is in the interior of the cone C To see this note from Corollary that z intC and

Y Y

so P d will only b e feasible if P d has a solution x for which b Ax is in the interior to C The

v Y

more interior a solution there is the smaller r x can b e scaled and still satisfy br Ax z C

Y

One would then exp ect v d to b e inversely prop ortional to d and to ud as the next theorem

indicates Indeed the theorem states that ud v d where we employ the convention that

when fud v dg f g

Theorem Suppose that d F and C is regular Then ud v d and

Y

d

v d v d

Pro of Supp ose that d Then ud from Theorem and from and there

T T

exists y C satisfying A y C b y and kyk which in turn implies that P d

v

Y X

T

cannot have a feasible solution for if x r is feasible for P d then y br Ax z a

v



d contradiction Thus v d and so ud v d by convention and also

v d

v d

Therefore supp ose that d Then ud from Theorem and also it is straight

forward to show that b oth P d and P d are feasible and attain their optima Note that for any

u v

y q g and x r feasible for P d and P d resp ectively we have

u v

T T T T T T

z y y br y Ax y br g r y Ax q x

T T

kxk jr j max fkA q k jb y g jg

whereby ud v d and so in particular v d We now will show that ud v d which

will complete the pro of

Dene the following set

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

S f w v Y X j there exists r x X s C and p C which

Y X

satisfy kxk jr j br Ax z w s x v pg

Then S is a nonempty convex set and basic limit arguments easily establish that S is also a

For any given and xed v d the p oint v d S for otherwise the

optimal value of P d would b e less than or equal to v d a contradiction Since S is a closed

v

nonempty convex set v d can b e strictly separated from S by a hyp erplane ie there

exists y q and such that

i v d

T T T

ii y w q w q v for any w v S

In particular ii implies that

T T

kxk jr j y br Ax z s q x p

for any x X r s C and p C

Y X

This implies that y C q C and

Y X

Supp ose rst that Then we can rescale y q and so that Then notice

T T

that implies that b y Also we claim that implies that kA y q k To

T

see this supp ose instead that kA y q k Then there exists x X such that kx k and

T T

x q A y and then setting x x for and suciently large we can drive the

lefthandside of to a negative numb er which would yield a contradiction Also from and

T

0 0 0

b y

q y

T

and g Then y C i note that y z v d Dene y q

T T T

Y

y z y z y z

0 0 0 0 0 0 0 0 0 0

T T T

q C g y z and maxfkA y q k jb y g jg and so y q g

T

X

v d

y z

is feasible for P d with ob jective value at most Therefore ud Since this is true

u

v d v d

for any v d then ud and then the second assertion of the theorem follows from

v d

Theorem

T

It remains to consider the case where Then and implies that y b

T T T

A y q C and y z Then we can rescale y so that y z and if we dene

X

T

g b y then y q g is feasible for P d with an ob jective value of zero Therefore ud

u

which implies via Theorem that d which contradicts the supp osition Therefore

is an imp ossibility and the theorem is proved

A simplifying p ersp ective on the results in this subsection is that all ve characterization

theorems of this subsection are either directly or indirectly derived from Theorem of To

see this rst recall that Theorem of shows that d is obtained as the optimal value of

the program P d Theorem was obtained by linearizing the norm constraint jr j kxk

r

in P d Theorem was obtained by linearizing the norm constraint ky k of P d but

r j

P d was itself constructed from P d via two partial duality derivations Also Theorem

j r

and Theorem were obtained by taking particular advantage of prop erties of the co ecients of

linearity and as they p ertain to mo dications of P d as well Finally Theorem was j

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

obtained by applying gauge duality to P d which itself was obtained from P d by linearization

u j

of the norm constraint ky k of P d

j

We conclude this subsection with the following comment The ve characterization theorems

in this subsection provide approximations of d but are exact characterizations when

andor However from Remark we can cho ose the norms on X and on Y in such a way

as to guarantee that and If the norms are so chosen then all ve theorems provide

exact characterizations of d

Characterization Results when P (d) is not consistent

In this subsection we parallel the results of the previous subsection for the case when P d

is not consistent That is we present ve dierent mathematical programs and we prove that the

optimal value of each of these mathematical programs provides an approximation of the value of

d in the case when P d is not consistent For each of these ve mathematical programs the

nature of the approximation of d is sp ecied in a theorem stating the result

As in the previous subsection the starting p oint of our analysis is an application of Theorem

of Renegar which we motivate as follows Consider the following normalization of the

alternative system

HD

T

A y C

X

T

b y

y C

Y

ky k

Consider the following program based on HD

P d

d minimum maximum

v X y

T

kv k st A y v C

X

T

b y

y C

Y

ky k

Then d is the largest scaling factor such that for any v with kv k v can b e added to

the rst inclusion of HD and can b e added to the second inclusion of HD without aecting the

feasibility of the system HD The following is also a slightly altered restatement of a result due to

Renegar

C

Theorem Theorem of Suppose that d F Then

d d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Exactly as in the previous subsection we can use partial duality constructs to create the

following program from P d

P d

k

k d minimum kbr Ax w k

x r w

st x C

X

r

w C

Y

kxk r

Note that program P d is a measure of how close the system P d is to b eing feasible To

k

see this note that if d A b were in F then it would b e true that k d The nonnegative

quantity k d measures the extent to which is not feasible The smaller the value of k d is

the closer the conditions are to b eing satised and so the smaller the value of d should b e

These arguments are validated in the following theorem

C

Theorem Suppose that d F Then

k d d

Pro of Supp ose that k d d Then there exists d A b such that kA Ak k d and

kb bk k d and d F Therefore there exists x r with r x C br Ax C and

X Y

jrj kx k Let w br Ax Then

kbr Ax w k kbr Ax w b b r A A x k

kb bkjrj kA Akkx k k d

But then x r w is feasible for P d with ob jective value less than k d which is a contradiction

k

Therefore k d d

Now supp ose that k d d for some Then there exists x r w such that x C

X

T

r w C and kbr Ax w k and jrj kx k Let x satisfy kx k and x x kx k

Y

see Prop osition For let

T

A A br Ax w x

Then br A x w C and r and x C Therefore d A b F However

Y X

kA Ak kbr Ax w k kbk kbk

d

C

we have kA Ak d whereby d A b F a contradiction Therefore For

kbk

k d d and so k d d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

We p oint out that b ecause P d was constructed by using partial duality constructs applied

k

to P d as illustrated ab ove one can also view as an application of Theorem of

Remark P d is not in general a convex program due to the nonconvex constraint r kxk

k

n

However in the case when X R if we choose the norm on X to be the L norm then P d

k

can be solved by solving n convex programs where the construction exactly paral lels that given for

P d earlier in this section One can easily show that k d min fk k k k g

j m m

where

k d minimum kbr Ax w k

j

x r w

st x C

X

r

w C

Y

x r

j

We now pro ceed to present ve dierent mathematical programs each of whose optimal

values provides an approximation of the value of the distance to illp osedness d when P d is

not consistent For the rst program supp ose that C is a regular cone and consider

Y

P d

d minimum

r x

st br Ax z C

Y

r kxk

r

x C

X

C

Theorem If d F and C is regular then

Y

d d d

Pro of Recall from Corollary that z intC Therefore d since otherwise there

Y

would exist x r satisfying br Ax intC r x C contradicting the hyp othesis that

Y X

C

d F Supp ose that r x is feasible for P d and let w br Ax z Then r x w is

feasible for P d with ob jective value kbr Ax w k k zk kzk It then follows that

k

k d d and so d d from

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

On the other hand supp ose that x r w is feasible for P d and let kbr Ax w k

k

Then

br Ax z C

Y

To demonstrate the validity of supp ose the contrary Then there exists y C with ky k

Y

T

z But then and y br Ax



T T T

kbr Ax w k y w Ax br y Ax br y z



ky k



where the last inequality is from Corollary As this is a contradiction is true Therefore

k d

is a feasible ob jective value of P d and so d whereby d k d d

 

from

Similar to P d P d is generally a nonconvex program due to the constraint r kxk

k

T

When C is also regular if we replace r kxk by r u x in P d we obtain the

X

following convex program

P d

d minimum

r x

st br Ax z C

Y

T

r u x

r

x C

X

The analog of Theorem b ecomes

C

Theorem If d F and both C and C are regular then

X Y

d d d

T T

Pro of Supp ose that r x is a feasible solution of P d Then r r u x xr u x r

T T

u x is a feasible solution of P d with ob jective function value r u x r kxk

from Corollary It then follows that d d Applying Theorem we obtain

d d

Next supp ose that r x is a feasible solution of P d Then r r kxk xr

kxk r kxk is a feasible solution of P d with ob jective function value r kxk

T

r u x from Corollary It then follows that d d Applying Theorem we

obtain d d

For the next mathematical program supp ose that the cone C is regular and consider Y

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

P d

d minimum maximum

v X y

T

kv k st A y v C

X

T

b y

y C

Y

T

z y

Notice that P d is identical to P d except that the norm constraint ky k in P d

T

is replaced by the linearized version z y We have

C

Theorem If d F and C is regular then

Y

d d d

T

Pro of The pro of follows from using the inequalities ky k z y ky k of Corollary

using the same logic as in the pro of of Theorem

The fourth mathematical program supp oses that the cone C is regular Consider the

X

following convex program

P d

g

g d minimum kbr Ax w k

x r w

st x C

X

r

w C

Y

T

u x r

Notice that P d is identical to P d except that the norm constraint r kxk in

g

k

T

P d is replaced by the linearized version r u x We have

k

C

Theorem If d F and C is regular then

X

g d d g d

T

Pro of The pro of follows from using the inequalities kxk u x kxk of Corollary

using the same logic as in the pro of of Theorem

Notice that P d is a gauge program its dual gauge program is given by g

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

P d

h

hd minimum ky k

y

T

st A y u C

X

T

b y

y C

Y

Note that P d is also a convex program One can interpret P d as measuring the extent

h h

T T

to which has a solution y for which A y intC and that satises b y To see this note

X

from Corollary that u intC and so P d will only b e feasible if the rst and the third

h

X

conditions of are satised in their interior The more interior a solution there is the smaller y

T T

can b e scaled and still satisfy A y u C and b y One would then exp ect hd to

Y

b e inversely prop ortional to d and to g d as Theorem indicates Just as in the case of

Theorem we employ the convention that when fg d hdg f g

C

Theorem Suppose that d F and C is regular Then g d hd and

X

d

hd hd

Pro of This pro of parallels that of Theorem Supp ose rst that d Then g d

from Theorem And from and there exists x r w satisfying br Ax w

r x C w C kx k r which in turn implies that P d cannot have a feasible

X Y

h

T T T T

solution for if y is feasible for P d then x A y u rb y w y and so

h

T T

y br Ax w u x r a contradiction Thus hd and so g d hd by

d convention and also

hd hd

Therefore supp ose that d Then g d from Theorem and also it is straight

forward to show that b oth P d and P d are feasible and attain their optima Note that for any

g

h

x r w and y feasible for P d and P d resp ectively that

g

h

t T T T T T

u x r y Ax y br y Ax y br w y k y k k br Ax w k

whereby g d hd and so in particular hd We now will show g d hd which

will complete the pro of

Dene the following set

S f q s p X Y j there exists y Y v C u C

X Y

T T

which satisfy ky k A y u s v b y q y p ug

Then S is a nonempty convex set and basic limit arguments easily establish that S is also a closed

set For any hd the p oint hd S for otherwise the optimal value of P d

h

would b e no greater than hd a contradiction Since S is a closed nonempty convex set

hd can b e strictly separated from S by a hyp erplane ie there exists r x w

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

and such that

i hd

T T

ii r q x s w p for any q s p S

In particular ii implies that

T T T T

ky k r b y x A y u v w y u

for any y Y v C and u C

X Y

T

This implies that r x C w C and r u x

X Y

Supp ose rst that and so by rescaling r x w and we can presume that

Then implies that kbr Ax w k To see this note that if kbr Ax w k then

T

there exists y Y for which kyk and y w Ax br and then setting y y for

and suciently large we can drive the lefthandside of to a negative numb er which

T

is a contradiction Also not that implies that r u x hd from i Dene

0 0 0 0 0 0

x r w x r w Then x r w is feasible for P d and g d kbr Ax w k

g

T

r u x

kbr Axw k

Since this is true for any hd then g d and then

T T

hd hd

r u x r u x

the second assertion of the theorem follows from Theorem

It only remains to consider the case when Then and implies that r

T

x C br Ax w w C and r u x We can rescale r x w and so

X Y

T

that r u x and then r x w is feasible for P d with an ob jective value of zero Therefore

g

g d which implies via Theorem that d which contradicts the supp osition that

d Therefore is an imp ossibility and the theorem is proved

The comments at the end of the previous subsection apply to this subsection as well all ve

characterization theorems of this subsections are either directly or indirectly derived from Theorem

of Also by appropriate choice of norms on X andor Y all ve characterization theorems

provide exact characterizations of d

Bounds on Radii of Contained and Intersecting Balls

In this section we develop four results concerning the radii of certain inscrib ed balls in the feasible

region of the system or in the case when P d is not consistent of the alternative system

These results are stated as Lemmas and of this section While these results are of

an intermediate nature it is nevertheless useful to motivate them which we do now by thinking

in terms of the ellipsoid algorithm for nding a p oint in a convex set

Consider the ellipsoid algorithm for nding a feasible p oint in a convex set S Roughly

sp eaking the main ingredients that are needed to apply the ellipsoid algorithm and to pro duce a

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

complexity b ound on the numb er of iterations of the ellipsoid algorithm are the existence of

i a ball B x r with the prop erty that B x r S

ii a ball B R with the prop erty that B x r B R and

iii an upp er b ound on the ratio R r

With these three ingredients it is then p ossible to pro duce a complexity b ound on the

numb er of iterations of the ellipsoid algorithm which will b e O n lnR r In addition it is also

convenient to have the following

iv a lower b ound on the radius r of the contained ball B x r and

v an upp er b ound on the radius R of the initial ball B R

In the bit mo del of complexity as applied to linear inequality systems one is usually able

L L

to set r n and R n where L is the numb er of bits needed to represent the system

Of course these values of r and R break down when the system is degenerate in our parlance

illp osed in which case the system must b e p erturb ed rst

By analogy for the problem P d considered herein in the convex set in mind is the set

L

X which is the feasible region of the problem P d and n is generally replaced by the condition

d

measure of d A b denoted C d which is dened to b e

kdk

C d

d

see Renegar The value of C d is a measure of the relative conditioning of the data instance

d The condition measure C d can b e viewed as a scaleinvariant recipro cal of the distance to

illp osedness d as it is elementary to demonstrate that C d C d for any p ositive scalar

The results in this section will b e used in Section to demonstrate in general that we can

nd a p oint x X and radii r and R with the ve prop erties b elow that are analogs of the ve

d

prop erties listed ab ove

i B x r X

d

ii B x r B R

iii R r c O C d

iv r c and

C d

v R c O C d

where the constants c c and c dep end only on the co ecients of linearity for the cones C C C

X Y

X

L

and C and are indep endent of the data d A b of the problem P d Here the quantity n is

Y

roughly replaced by C d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

The ab ove remarks p ertain to the the case when P d is consistent ie when P d has a

solution When P d is not consistent then the convex set in mind is the feasible region for the

alternative system denoted by Y The results in this section will also b e used in Section to

d

demonstrate in general that we can nd a p oint y in Y and radii r and R with the three prop erties

d

b elow that are analogs of the rst three prop erties listed ab ove

i B y r Y

d

ii B y r B R

iii R r c O C d

where again the constant c dep ends on the co ecients of linearity for the cones C C C and

X Y

X

C and is indep endent of the problem data d Because the system is homogeneous it makes

Y

little sense to b ound r from b elow or R from ab ove as all constructions can b e scaled by any

p ositive quantity Therefore prop erties iv and v ab ove are not relevant

The results in this section are rather technical and the reader may rst want to read Section

b efore p ondering the results in this section in detail

We rst examine the case when P d is consistent in which case the feasible region X

d

fx X jb Ax C x C g is nonempty

Y X

Lemma Suppose that d F and C is regular If d then there exists x X and

Y

d

positive scalars r and R satisfying

i B x r fx X jb Ax C g

Y

ii B x r B R

kdk

R

1

iii



r

d

1



d

and iv r

kdk

kdk

v R



d

In order to prove Lemma we rst prove

Prop osition d kdk

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

C C

Pro of If d F resp ectively F then d F resp ectively F for all Therefore

C

d A b B cl F cl F and so d kd dk kd k kdk

Pro of of Lemma For any w C with kw k we have

Y

T

kdk kw k b w

T

z w

kdk kdk



so that z b C Now let x r solve P d see and let

Y v

kdk

x x

x



r

kdk

 

Let q br Ax z Then q C and we have br Ax b where r z

Y

kdk kdk

 

b b q C so that b Ax z q z z C whereby b Ax z C z

Y Y Y

kdk kdk



Then if kx x k r we have Thus x C and b Ax C so x X Let r

X Y

d

kdk

b Ax b Ax Ax x z z Ax x b Ax

z Ax x y

where y C Thus for any w C with kw k

Y

Y



T T T

w b Ax z w w Ax x kw k kAk kx xk



kdkr

Therefore b Ax C proving i

Y

Next let R kx k r and so ii is satised

To prove iii observe that

kx k kx k v d

R

1

from

r r r r

1 1 1 1

from Theorem

r d

1

kdk



d

 

so that proving iii To compute the b ounds in iv and v notice rst that r

kdk kdk

r Therefore from iii we have

kdk R

R

r

d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

proving v We also have

 

r v d from

kdk kdk

v d from Prop osition

d

from Theorem

d





d

proving iv Therefore r

kdk kdk

Remark The proof of Lemma can be modied to yield dierent constants on the bounds on

R

1

r and R By changing the constant in the third line of the proof to for and

r

1

kdk

R

1

r making suitable changes in the proof one can obtain the fol lowing bounds



r

d

1



d kdk

One can then choose to optimize one of these bounds for example and R



kdk

d

We next have

Lemma Suppose that d F and C is regular If d then there exists x X and

X

d

positive scalars r and R satisfying

i B x r C

X

ii B x r B R

kdk

R

2

iii



r d

2



d

iv r

kdk

kdk

and v R



d

Pro of Let x denote the norm approximation vector for the cone C Consider the following

X

optimization problem

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Q maximize

r x

st br Ax b Ax C

Y

x C

X

r

jr j kxk

d

From and the optimal value of Q is at least and so there exits r x feasible

kbAx k

d

for Q with and so

kbAx k

d d d

kbAx k kbkkAk kdk

Dene

x x

r and R kx k r x

r r

Then the feasibility r x in Q ensures that b Ax C x C so that x is feasible for P d

Y X

For any v X satisfying kv k r and for any u C we have

X

T T

u x u x

T T

u x v u v

r



kuk



kv kkuk from Prop osition

r



r kuk

r

and so B x r C which shows i and ii follows from the denition of R

X

To prove iii observe that

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

kx k kx k

R

2



r r

2 2



kdk

from



d

kdk

from Propsition



d



To prove iv and v note rst that r since and r Then from iii we

r

have that

kdk R

R

r d

which proves v To prove iv observe that

kdk

r r

from

  



  

r

d

2

kdk

from Prop osition



d



d

and so r completing the pro of

kdk

Remark Similarly to Remark the proof of Lemma can be modied to yield dierent

R

2

constants on the bounds on r and R By changing the rst inclusion in the feasibility condi

r

2

tions of program Q above to br Ax b Ax C for and making suitable changes

Y



kdk d

R

2

in the proof one can obtain the fol lowing bounds r and





r

d kdk

2

kdk

One can then choose to optimize one of these bounds as wel l R

d

We now turn to the case when P d is inconsistent ie P d has no solution In this case

from Prop osition the system has a solution and let us then examine the set of all solutions

to which we denote by Y to emphasize the dep endence on the data d A b

d

T T

y b g y C Y fy Y jA y C

d

Y X

C

Lemma Suppose that d F and C is regular If d then there exists y Y and

X

d

kdk

R

3

positive scalars r and R satisfying and that satisfy

r

d

3

T T

B y r fy Y jA y C b y g

X

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

and

kyk R

T T

Pro of of Lemma Let y solve P d Then A y u C b y and y C Then

h

X Y

y

since y otherwise u C and so C is not regular via Prop osition let y Let

X

X

kyk



g d

r and let R

kdk

T T

and y b We To prove it suces to show that if ky yk r then A y C

X

u

T

have that q A y C For any x C with kxk

X

X

kyk



T T

u x u x

T T T T T T

x A y x A y y x A y

kyk kyk

 

T

u x

T

kxk kAk ky yk x q

kyk



T

u x

T

since x q kAkr

kyk



T

u x

since kdk kAk g d

kyk



g d from Corollary

kyk



g d

hd

from Theorem

T

Therefore A y C Similarly

X

T T T

b y b y y b y

T

b y

kdk ky yk

kyk



g d

kyk



g d g d from Theorem

hd

from Prop osition

T T

Therefore A y C and b y which proves

X

To prove note that kyk R which demonstrates Finally note that

kdk kdk

R

3

from Theorem

r r g d d

3 3

We next prove

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

C

Lemma Suppose that d F and C is regular If d then there exists y Y and

Y

d

kdk

R

4

positive scalars r and R satisfying and that satisfy

r

d

4

B y r C

Y

and

kyk R

Pro of Let y denote the norm approximation vector for the cone C Then kyk and

Y

T

y y ky k for any y C see Prop osition Supp ose that d satises kdk Then for any

Y

T

Therefore y C we have y d y ky k kdk ky k ky k ky k and so y d C

Y

Y

B y C and recall that kyk

Y

Consider now the following system in the variable y

T T

A y A y C

X

C d

T T

b y b y

C d

y C

Y

ky k

kAkkyk d kbkkyk d

 

T T

Then k A yk d and j b yj d Then from

C d kdk C d kdk

and it follows that has a solution y

Dene

C d

y y y

C d C d

T

Then kyk since y is a of y and y Also y C and A y C and

Y X

T

and R Then b y from convexity and from Therefore y Y Let r

d

C d

C Also kyk R Finally B y C and y C imply that B y r B y

Y Y Y

C d

C d kdk C d

R

4

note that

r r

d

4 4

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Synthesis of Results

In this section we synthesize the results of the previous two sections into theorems that

characterize asp ects of the distance to illp osedness for the three particular cases of problem P d

of namely

i Case C and C are b oth regular

X Y

ii Case C is regular and C fg

X Y

iii Case C X and C is regular

X Y

and for the status of solvability of P d of namely

a P d is consistent ie has a solution and

b P d is inconsistent ie has a solution

Each of the six theorems of this section synthesizes our results of the previous two sections

as applied to the one of the three cases ab ove and one of the two status of the solvability of P d

Each theorem summarizes the applicable approximation characterizations of d of Section and

also synthesizes the appropriate b ounds on radii of contained and intersecting balls develop ed in

Section For a motivation of the imp ortance of these b ounds on radii of contained and intersecting

balls contained herein the reader is referred to the op ening discussion at the b eginning of Section

Each case is treated as a separate subsection and all pro ofs are deferred to the end of the

section

Case C and C are b oth regular

X Y

Theorem Suppose that C and C are both regular If P d is consistent ie d F then

X Y

i d d d

ii d d d

iii w d d w d

iv ud d ud



v d

v d v d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

vi If d then there exists x X and positive scalars r and R satisfying

d

a B x r X

d

b B x r B R

kdk

R

c

 

r

minf gd

 

minf gd

d r

kdk

kdk

e R

 

minf gd

C

Theorem Suppose that C and C are both regular If P d is not consistent ie d F

X Y

then

i d d d

ii d d d

iii d d d

iv g d d g d

v d

hd hd

vi If d then there exists y Y and positive scalars r and R satisfying

d

a B y r Y

d

b B y r B R

kdk

R

c

r

minf gd

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Case C is regular and C = f0g

X Y

Theorem Suppose that C is regular and C fg If P d is consistent ie d F then

X Y

i d d d

ii w d d w d

iii If d then there exists x X and positive scalars r and R satisfying

d

a fx X j jkx x k r Ax bg X

d

b B x r B R

kdk

R

c



r

d



d

d r

kdk

kdk

e R



d

C

Theorem Suppose that C is regular and C fg If P d is not consistent ie d F

X Y

then

i g d d g d

ii d

hd hd

iii If d then there exists y Y and positive scalars r and R satisfying

d

a B y r Y

d

b B y r B R

kdk

R

c

r d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Case C = X and C is regular

X Y

Theorem Suppose that C X and C is regular If P d is consistent ie d F then

X Y

i ud d ud



ii d

v d v d

iii If d then there exists x X and positive scalars r and R satisfying

d

a B x r X

d

b B x r B R

kdk

R

c



r

d



d

d r

kdk

kdk

e R



d

C

Theorem Suppose that C X and C is regular If P d is not consistent ie d F

X Y

then

i d d d

ii d d d

iii If d then there exists y Y and positive scalars r and R satisfying

d

T

a fy Y jky yk r A y g Y

d

b B y r B R

kdk

R

c



r

minf gd

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Pro of of Theorem Parts i ii iii iv and v follow directly from Theorems

and resp ectively It remains to prove part vi

Let S fx X jb Ax C g and T C Then S T X From Lemma there

Y X

d

exists x X and r R satisfying conditions i v of Lemma From Lemma there

d

exists x X and r R satisfying conditions i v of Lemma Then the conditions of

d

Prop osition A of the App endix are satised and so there exists x and r R satisfying the ve

conditions of Prop osition A Therefore i B x r S T X which is a Also from ii

d

B x r B R which is b From iii we have

R R R kdk

r r r

minf gd

invoking Lemma iii and Lemma iii which is c Similarly applying Lemma and

and Prop osition A in parts iv and v yields

minf gd

min fr r g r

kdk

and

kdk

R max f R R g

minf gd

Pro of of Theorem Parts i ii iii iv and v follow directly from Theorems

and resp ectively It remains to prove part vi

T T

Let S fy Y jA y C b y g and T C Then S T Y From Lemma

d

X Y

kdk

R

3

there exists y r R satisfying y S T B y r S and ky k R and

r

d

3

From Lemma there exists y r R satisfying y S T B y r T and ky k R

kdk

R

4

Then from Prop osition A of the App endix there exists y and r R satisfying and

r

d

4

B y r S T Y and kyk R and

d

kdk

R R

R

3 4

r r r

minf gd

3 4

Now let R R r Then for any y B y r ky k kyk r R r R and

kdk

R R

r r

minf gd

Pro of of Theorem Parts i and ii follow directly from Theorems and resp ectively

To prove iii we apply Lemma there exists x X and r R satisfying the ve conditions

d

of Lemma Let r r and R R Then b c d and e follow directly To prove a

observe that from Lemma i that

fx X j kx x k r g C

X

and intersecting b oth sides with the ane set fx X jAx bg gives

fx X j kx x k r Ax bg C fx X jAx bg X

X d

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

Pro of of Theorem Parts i and ii follow directly from Theorems and resp ectively

kdk

R

3

To prove iii we apply Lemma there exists y Y and r R satisfying and

d

r

d

3

and Let r r and R R r Then from we obtain

T T

fy Y j ky yk r g fy Y jA y C b y g Y

d

X

Also for any y satisfying ky yk r ky k kyk r R r R Finally note that

kdk

R

R

3

r r

d

3

Pro of of Theorem Parts i and ii follow directly from Theorems and resp ectively

To prove iii we apply Lemma there exists x X and r R satisfying the ve conditions of

d

Lemma Let r r and R R Then b c d and e following directly To prove a

observe from Lemma i that

fx X j kx x k r g f x X jb Ax C g X

Y

d

Pro of of Theorem Parts i and ii follow from Theorems and resp ectively It

remains to prove iii

T

Let S fy Y jb y g If we let v in we see that there exists y C

Y

T T T

satisfying A y z y and b y d d from Theorem Therefore if we

T

d

z y

1

we have ky k R from Corollary and for any y and R set r

  

kdk

T T T

satisfying ky y k r we have b y b y y b y kbk ky y k d kdkr d

T

and so B y r S If we let T C we have S T fy Y jA y g Y and y Y

d d

Y

kdk

R

4

and and Then from From Lemma there exists y and r R satisfying

r

d

4

Prop osition A of the App endix there exists y and r R satisfying B y r S T and kyk R

and

R R R kdk

r r r

minf gd

Note also that

T T

fy Y j ky yk r A y g S T fy Y jA y g Y

d

Let R R r Then for any y B y r ky k kyk r R r R and

kdk

R R



r r

minf gd

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

APPENDIX

This app endix contains two simple constructions with balls on the intersection of two convex

sets

Prop osition A Let X be a nitedimensional normed linear vector space with norm k k and

let S and T be convex of X Suppose that

i x S T B x r S where r and kx k R and

ii x S T B x r T where r and kx k R

r r r

2 1 2

Let and r and R R R

r r r r

1 2 1 2

Then the point x x x wil l satisfy

i B x r S T

ii kx k R

R R

R

1 2

and iii

r r r

1 2

Pro of First note that Because B x r S and x S B x x r S

Similarly b ecause B x r T and x T B x x r T Noticing

that r r r we have B x r B x x r S T Also kx k

kx k kx k R R R Finally to show iii we have

R R R R R

r r r r

Prop osition A Let X be a nitedimensional normed linear vector space with norm k k and

let S and T be convex subsets of X Suppose that

i x S T B x r S where r and B x r B R and

ii x S T B x r T where r and B x r B R

r r r

2 1 2

Let and r and R R R Then the point x x x

r r r r

1 2 1 2

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

wil l satisfy

i B x r S T

ii B x r B R

R R

R

1 2

iii

r r r

1 2

iv r minfr r g

and v R maxfR R g

Pro of Parts i and iii follow identically the pro of of Prop osition A To see iv note

minfr r g maxf r r g

1 2 1 2

min fr r g Part v follows from the fact that by denition of r r

maxfr r g

1 2

that R is a convex combination of R and R To prove ii note that for any x B x r we

have kxk kx k r kx k kx k r However kx k r R i so that

i i i

kxk R r R r r R r R which completes the pro of

References

F Alizadeh Interior p oint metho ds in semidenite programming with applications to combi

natorial optimization SIAM Journal on Optimization

Mokhtar S Bazaraa Hanif D Sherali and C M Shetty Theory and

Algorithms John Wiley Sons Inc New York second edition

L Blum M Shub and S Smale On a theory of computation and complexity over the real

numb ers NPcompleteness recursive functions and universal machines Bul l Amer Math

Soc

Sharon Filip owski On the complexity of solving linear programs sp ecied with approximate

data and known to b e feasible Technical Rep ort Dept of Industrial and Manufacturing

Systems Engineering Iowa State University May

Sharon Filip owski On the complexity of solving sparse symmetric linear programs sp ecied

with approximate data Mathematics of Operations Research

Rob ert M Freund Dual gauge programs with applications to quadratic programming and

the minimumnorm problem Mathematical Programming

Roger A Horn and Charles R Johnson Matrix Analysis Cambridge University Press New

York

Yurii Nesterov and Arkadii Nemirovskii InteriorPoint Polynomial Algorithms in Convex

Programming So ciety for Industrial and Applied Mathematics SIAM Philadelphia

Manuel A Nunez and Rob ert M Freund Condition measures and prop erties of the cen

tral tra jectory of a linear program Technical Rep ort WP MSA Sloan Scho ol of

Management Massachusetts Institute of Technology

PROPERTIES OF THE DISTANCE TO ILLPOSEDNESS

James Renegar Some p erturbation theory for linear programming Mathematical Program

ming

James Renegar Incorp orating condition measures into the complexity theory of linear pro

gramming SIAM Journal on Optimization

James Renegar Linear programming complexity theory and elementary

Mathematical Programming

James Renegar Condition numb ers the barrier metho d and the conjugate gradient metho d

SIAM Journal on Optimization

RT Ro ckafellar Princeton University Press Princeton NJ

S Smale Some remarks on the foundations of numerical analysis SIAM Review

Jorge R Vera Il lPosedness in Mathematical Programming and Problem Solving with Approx

imate Data PhD thesis Cornell University

Jorge R Vera Illp osedness and the complexity of deciding existence of solutions to linear

programs Technical Rep ort Cornell University June To app ear in SIAM Journal on

Optimization

Jorge R Vera On the complexity of linear programming under nite precision arithmetic

Technical Rep ort Dept of Industrial Engineering University of Chile August

K Yosida Functional Analysis SpringerVerlag Berlin