Error Bounds for Regularized

ERROR BOUNDS FOR REGULARIZED

COMPLEMENTARITY PROBLEMS

Paul Tseng

July revised Septemb er

Abstract

An approach to solving a complementarity problem entails regular

izingp erturb in g the problem by adding to the given mapping another

mapping multiplied by a small p ositive parameter We study prop erties

of the limit p oint of the solution to the regularized problem We also

derive lo cal error b ounds on the distance from the solution to its limit

p oint expressed in terms of the regularizatio n parameter

Keywords Regularizati on complementarity problem optimization error b ound

AMS sub ject classication M C C C C

Intro duction

Consider the complementarity problem CP of nding an x satisfying

x F x F x x

n n

where F is a given continuous mapping This is a wellknown

problem in optimization with many applications In various regulariza

tioncontinuationsmoothing approaches to solving this problem one adds to

n n

the mapping F another mapping G multiplied by a small p ositive

scalar and computes p ossibly inexactly an x satisfying

x F x Gx F x Gx x

Then one may decrease and up date x accordingly Our interests are in

prop erties of any limit p oint of x along some sequence of and the

distance from x to this limit p oint There are also the related issues of exis

tenceuniquenessb oundednes s of x as which we will not fo cus on

This research is supp orted by National Science Foundation Grant CCR

Department of Mathematics University of Washington Seattle Washington

USA tsengmathwashingtonedu

Paul Tseng

The regularized CP is closely linked to a regularized smo oth optimization

problem of the form

minimize f u g u sub ject to u f u g u i m

i i

where f f f are continuously dierentiable functions dened on some

op en set containing and g g g are continuously dierentiable func

tions dened on m l In particular it is well known that the

asso ciated KarushKuhnTucker condition is exactly with x and

P P

m m

rg u v rg u rf u v rf u

i i i i

i i

Gx F x

m m

g u f u

i i

Moreover if f f f are convex resp ectively quadratic on their resp ective

domains then this F is monotone resp ectively ane and continuous on

Example and similarly for G A well studied case in this optimization

setting is when g for i m ie constraint functions are unregularized

For the regularized CP one p opular choice of G is the identity mapping

Gx x

corresp onding to the wellknown Tikhonov regularization technique This choice

has b een much studied including in the general

setting of nding a zero of a maximal monotone op erator page Chapter

I I The analogous choice of

g u kuk

for has b een considered by Karlin Mangasarian and others

in the context of linear programs LP and by Tikhonov and various

others in the general optimization setting see and references therein

It was shown in Prop osition iii also see Theorem b

Prop osition Theorem that if F is monotone then each limit

p oint of x as is the least norm solution of CP Analogous results

were obtained by Mangasarian in the context of LP also see for

extensions to other choices of g in this context and by Levitin and Polyak

and others in the general optimization setting see pages

Prop osition and references therein If F is only a P function Sznader

and Gowda Theorem showed that any limit p oint is weakParetominimal

in the sense that no other solution is comp onentwise strictly less so any non

p ositive solution is weakParetominimal A second p opular choice of G is the

inverse function

Gx x

corresp onding to logbarrier metho ds and interiorp oint metho ds This choice

has b een considered Ko jima et al and Guler and in the

Illp osed Variatonal Problems and Regularization Techniques

general setting of nding a zero of a maximal monotone op erator by McLinden

The analogous choice of

lnu g u

for has b een much studied in the context of LP see and

references therein It was shown by McLinden Corollary that if F is

monotone and CP has a strictly complementary solution then any limit p oint

of x is a least weighted lnu solution of CP Analogus results were obtained

by McLinden Theorem and Megiddo in the context of LP A third

choice is the logarithm function

Gx lnx

The analogous choice of

u lnu g u

j j

for was considered in the context of LP by Fang et al and

from a dual exp onential p enalty view by Cominetti et al It was

shown in that any limit p oint of the solution of the regularized LP is the

least u lnuentropy solution of the LP This result was generalized recently by

Auslender et al to convex programs with g b eing a certain kind of separable

strictly convex essentially smo oth function A similar result was shown in

for the LP case without the convexity and smo othness assumption Related

results in the general optimization setting are given in and references

therein These results do not assume g to b e separable or even continuous but

they do need g to b e lower semicontinuous and realvalued at the limit p oint

to b e meaningful

As the preceding discussion shows there have b een many studies of the

prop erties of a limit p ointx of x with particular fo cus on the cases of G given

by or or However there have b een relatively few studies of the

distance from x to x estimated in terms of In the context of LP with g

given by this distance is known to b e in the order of e for some constant

Theorem The same reference also gives distance estimates for the

dual LP If g is more generally a separable strictly convex essentially smo oth

function this distance can b e estimated in terms of rg and If the

LP has a multicommo dity network ow structure and g is a weighted inverse

barrier function whose weights are ane functions of a nonnegative variable

this distance is known to b e in the order of

In this pap er we study the ab ove questions in the context of CP and its

regularization In particular we show that if F is pseudomonotone on

and G is continuous atx thenx solves the variational inequality problem with

Paul Tseng

mapping G over the solution set Moreover if F is analytic on an op en set

n T

containing then the generalized distance Gx Gx x x is in the

order of for some with if F is ane see Alternatively if

Gx G x

j j

where each G is strictly increasing and continuous on but may tend to

at eg G given by or we show that in each co ordinate subspace over

which F is pseudomonotone x solves the variational inequality problem with

mapping G over the solution set see Prop osition a Moreover under the

assumption that either i F is pseudomonotone on and lim tG t

t j

for j J eg G given by or ii F is ane with certain principal

submatrices of its Jacobian p ositive semidenite and spanning the corresp onding

rows or iii F is ane with certain principal submatrix of its Jacobian p ositive

semidenite and lim sup tG t for j J eg G given by we

estimate jx x j in terms of x and in the case where F is ane we

j j J j J

j j

estimate the latter in terms of where J is the set of indices j with G x

j j

see Prop osition cc Thus our results may b e applied to analyze

regularization of a convex quadratic program of the form Our study is

motivated by a related work in the context of LP although our results and

our pro ofs are quite dierent from those in due to the dierent problem

structure and regularization

In our notation denotes the space of ndimensional real column vectors

n n n

and denote the nonnegative orthant and the p ositive orthant in

T n

resp ectively and denotes transp ose For any x we denote by x the ith

comp onent of x and for any I N f ng by x the vector obtained

by removing from x those x with i I and by x the vector in whose

i I

ith comp onent is x if i I and is zero otherwise Here and throughout

means dene We denote by jI j the cardinality of I and denote I N nI

x x kxk max jx j For any M and any I J N kxk

iN i

we denote by M the submatrix of M obtained by removing all rows of M with

indices outside of I and by M the submatrix of M obtained by removing

IJ I

n n

all columns of M with indices outside of J For any F and any

nonempty closed convex set we denote

VI F fx F x y x y g

Thus x satises if and only if x VI F and x satsies if and only

if x VI F G We denote by F the ith comp onent of F and for any

I N by F the mapping obtained by removing from F those F with i I

I i

We say F is pseudomonotone on page if

T T n

and F y x y F x x y x y

n n

For any x and any nonempty closed set we denote dist x

min ky xk

Illp osed Variatonal Problems and Regularization Techniques

Error Bounds on Distance to Limiting Solu

tion

First we have the following b ound on the distance from x to the solution set

of in terms of the regularization G x This is a simple consequence of an

error b ound result for analytic systems

n n

Prop osition Consider an open set containing and an analytic

n n n

F with VI F nonempty Then for every bounded

there exist and such that

distx k maxf G x gk jG x x j

n n n

for al l G and al l x VI F G

Pro of We have that an x satises

x F x F x x

and that an x VI F G satises

T T

x F x G x F x x G x x

So if x is also in the b ounded set then since F is analytic on an op en set

containing an error b ound result of Lo jasiewicz as extended by Luo and

Pang for analytic systems Theorem yields with and

some constants

Note Prop osition do es not say anything ab out the existence or unique

ness or b oundedness of x VI F G In the case where F is mono

tone and ane and G x M x q Robinson Theorem showed

that b eing nonempty and b ounded is b oth necessary and sucient for the

existence of x satisfying distx as kM k kq k If F is a

continuously dierentiable P function a result of Facchinei Theorem

implies that b eing nonempty and b ounded is sucient for the existence of

x satisfying distx as where G is continuous and satises

lim sup kG xk for some If F is a continuously dif

xdist x

ferentiable P function and G x x Facchinei and Kanzow Theorem

showed existence and uniqueness of x for all and if in addition

is nonempty and b ounded then x is b ounded and distx as

Ravindran and Gowda extended the preceding two results to CP with b ound

constraints and they weakened the dierentiability assumption on F to continu

ity In the case where F is a p olynomial P function and G x x Szna jder

and Gowda Theorem showed that x either converges or diverges in norm

Paul Tseng

Prop osition do es not give an estimate of the Holder constant In the

case where F is ane and monotone a b ound with can b e shown

Theorem By adding a mild assumption on G we derive b elow a second

distance b ound in terms of G x with when F is ane Moreover in

and G x converges we derive the case where F is pseudomonotone on

a b ound on the distance from x to its limit p oint in terms of

n n

Prop osition Consider a continuous F a sequence of positive

n n

scalars f g tending to zero and for each a G

F G such that x converges to some x and and an x VI

G x converges to some g as Then x VI F and

the fol lowing hold

a If F is pseudomonotone on then x VI g

b If F is analytic on an open set containing then there exist and

such that

distx kG x k

for al l with whenever F is ane

c If F is pseudomonotone on and is analytic on an open set containing

then there exist and such that

G x g x x kg kkG x k

for al l x VI g and al l with whenever F is ane

Pro of Since x so that x F x G x F x G x x

for all we have in the limit also using G x thatx F x

F x x Thus x

a Assume F is pseudomonotone on Then is closed convex page

Moreover for any y and the fact that F y x y imply

T T

F x x y G x y x

where the second inequality uses x and y Dividing b oth sides by

yields in the limit that g y x

b For each since x we have

c c

x x G x F x G x F x

I I I I

I I

for some I N Let f holdsg Consider any I N such

that j j Since G x as then any cluster p oint x of x

I I

satises

c c

x x F x F x

I I I I

Illp osed Variatonal Problems and Regularization Techniques

Assume F is analytic on an op en set containing Then an error b ound result

of Lo jasiewicz as extended by Luo and Pang to analytic systems Theorem

implies the nonlinear system has a solution y satisfying

kx y k kG x k

where and are constants dep ending on F and I and and

I I

kx k only Thus y and moreover in the case where F is ane sup

a lemma of Homan implies that For any I with j j

I I

let y b e any xed element of for all and then for any

I I

would hold for a suitable since its lefthand side is b ounded and its

righthand side is b ounded away from zero Taking min and

I I

yields for all with kG x k sup max

I I I I

whenever F is ane

c Assume F is pseudomonotone on and is analytic on an op en set

containing Fix any x VI g and any Letting y satisfy

kx y k distx we have together with in part b that

T T T

g y x g y x g x x

kg kkG x k g x x

for some constants and Also since x we have from and

x that

T T

F x x x F x G x x x

Adding the ab ove two inequalities to the previous inequality multiplied by we

obtain

kg kkG x k g G x x x

Rearranging terms yields

Note Notice that Prop osition is stated in the setting of along a sequence

rather than in a continuum as in Prop osition Although for practical pur

p oses such as analyzing the convergence of an iterative metho d the former

setting is sucient it is nevertheless p ossible to extend Prop osition to the

latter setting provided kG x k is b ounded away from zero whenever in the

continuum is b ounded away from zero Also Prop ositions and may p ossibly

b e extended to F b eing piecewiseanalytic and more generally subanalytic

Note In the case where F is an analytic P function and the solution set

is nonempty and b ounded Theorem implies that x is dened and

b ounded as and so Prop osition b yields that for any sequence of

along which x converges holds for all in this sequence where

dep end on the limit p oint if F is ane A similar result was shown

Paul Tseng

earlier by Robinson Theorem in the case of F b eing monotone and ane

If in addition F is p olynomial and G x x then x converges Bounds

of the typ e were also derived by Fischer Section under similar

though not identical assumptions on F and G Fischer derived his b ounds

by applying a stability result of Klatte for parametric optimization In the

F has the Aubin property VI case where the setvalued mapping F

n n

relative to fF G G sup n kGxk g at F for

x kxxk

x a b ound similar to with holds F However verifying the

Aubin prop erty may b e dicult In the optimization setting an analogous

Lipschitzian prop erty can b e shown under very mild assumptions for the set

of approximate solutions Theorem

Note In the case where F is pseudomonotone on and G x Gx

with G continuous atx Prop osition ac imply g Gx andx VI G

This extends previous results Theorem Theorem b for

the case of F b eing monotone or pseudomonotone and Gx x also see

page Prop osition iii Prop osition for analogous results in

the context of maximal monotone op erators in an innitedimensional space

If F is also analytic on an op en set containing taking x x in yields

Gx Gx x x kGx kkGx k

Thus if in addition G is strictly monotone at x in the sense that there exist

such that

Gx Gx x x kx xk x with kx xk

then would yield the error b ound that kx xk is in the order of

whenever kx xk Notice that G essentially needs to b e continuous at x

in order to satisfy the assumption that G x converges as x x

In deriving the error b ound in Prop osition c we have required G x

as This rules out the imp ortant case of G x Gx where G is given

by or or more generally with p ossibly lim G t In

t j

Prop osition b elow we consider this case and we study prop erties of any limit

p ointx of x see part a and derive error b ounds on the distance from x to

x see parts cc In particular parts c c c of this prop osition

estimate under various assumptions on F and G G the distance kx x k

n J

k and parts c and c estimate in the case where F is ane in terms of kx

the latter in terms of with J b eing the set of indices j with G x

j j

While these error b ounds may b e complex Example b elow suggests that

this complexity is needed to account for the dierent relative growth rates of

G G near zero and the linkage among the comp onents of x as imp osed by

the complementarity condition

Illp osed Variatonal Problems and Regularization Techniques

n n

Prop osition Consider a continuous F and a continuous G

n n

given by where lim G t G for al l j N

t j j

Consider a sequence of positive scalars f g tending to zero and

for each an x VI F G converging to some x as

F and the fol lowing hold with J fj Then x VI

N G x g being the pathconnected component of containing

j j

x J fj N G x for some x g and for each I J

j j

f F x Gx x g

I I I

J nI

a For each J H J such that x F x is pseudomonotone on

H H

jH j

and fx x g is convex and has an element y

H H H H

y with G y for al l j H we have x VI p

j j H j j H H H

k respectively H J and x VI p G if kq

H H H H

k some as along some subsequence respectively kq

of where p is any cluster point of q kq k along this subsequence

H H

and q F x F x

b If F is analytic on an open set containing then there exist and

such that

k distx kGx k kx

for al l with whenever F is ane

n c

c If F is pseudomonotone on and lim tG t for al l j J

t j

then J J and x VI G If in addition F is analytic on an open set

containing then there exist and such that

T T

Gx Gx x x kGx k kx k Gx x

c c c

J J

J J J J

for al l with whenever F is ane

nn n jJ jjJ j

c If F x M xq for some M q with M M

J J J J

T jJ jjJ j

M and M and if is convex then J J and x

J J

VI G Moreover for each I J with j j and M positive

I II

jI jjJ j

c c c

there exists for some N M N semidenite and M

IJ II IJ IJ

such that

T T

x x x kGx k kx Gx Gx N k Gx

c c

J I J IJ

J J J I

for al l

nn n

c If F x M x q for some M q and if is convex and

G is strictly increasing for al l j J then there exist and

such that

k x G j K kx kx k h

j K

LnJ

j L

Paul Tseng

for al l suciently smal l where K J K may depend on

L J nK and h is the unique satisfying G for

c T

j J If M M then remains true without the h terms

nn n

positive c If F x M x q for some M q with M

J J

semidenite and if G is strictly increasing with lim sup tG t for

j j

al l j J then is the union of a countable col lection of subsequences

for each of which there exist properly nested J H

H H J and depending on M and x

r r

N H

only l r such that as and for each

l r

or x j H nH either x G

l l

j l j l

for al l suciently smal l And if in addition G is local ly Lips

chitzian at x with constant and satises with

then there exist independent of such that

kx x k

for al l suciently smal l

Pro of For each j J we have G x G x as so the

j j j

fact x satises for all yields in the limit that x F x

j j

F x x For each j J we have x and F x G x for

j j j j j

all suciently small The latter yields in the limit F x Thus x

satises and hence x

a Consider any J H J such that x F x is pseudomonotone

H H

jH j

on and is convex and contains an element y y with G y

H H j j H j j

for all j H Then we have from y and x that

T T

F y x y F x q Gx y x

H H H H H

jH j

with q F x F x Since x the rst inequality and the

H H

jH j

pseudomonotonicity of x F x on imply F x x y

H H H

which when added to the second inequality yields

q Gx y x

H H

Consider any subsequence of along which either i kq k or ii

kq k and let p b e any cluster p oint of q kq k along this

H H H

subsequence In case i dividing b oth sides of by kq k and using y

x and G x for all j J yield in the limit that

j j

y x p

H H H

Illp osed Variatonal Problems and Regularization Techniques

Since is convex this holds for all y so x VI p In case

H H H H H H

ii dividing b oth sides of by and arguing as in case i yield in the limit

that H J and

p Gx y x

H H H H

Since is convex this holds for all y sox VI p G

H H H H H H H

b For each j J we have x for all and G x for all

j j

b elow some Consider any I J such that j j For each

I I

b elow since x we have

F x Gx F x Gx F x

I I J

J nI J nI

x x x x

c c

J I J

J nI

Since F x F x and Gx Gx as yields in the

J J I

limit thatx satises

c c

x x F x F x

I I I I

Assume F is analytic on an op en set containing Then an error b ound result

of Lo jasiewicz as extended by Luo and Pang to analytic systems Theorem

implies the nonlinear system has a solution y satisfying

ky x k kGx k kx k

I J

kx k where and are constants dep ending on F and I and sup

I I

only Thus y and in the case where F is ane a lemma of Homan

implies If is nite but nonempty let y b e any xed element of

I I

for all and then would hold for any and a suciently large

I I

Taking min and

I I I

k kGx k kx sup max

I J

yields for all

I I

c Assume F is pseudomonotone on and lim tG t for all

t j

j J Then is closed convex page so and there exists y

such that G y for all j J For each since y and

j j

x imply

T T

F x x y Gx y x

Also by a x so J J For each j J we have G x y x

j j

converges as For each j J nJ we have y and G x

j j

so G x y x as For each j J we have x y

j j j

so our assumption on G yields G x y x as Hence

j j j

j j

implies J nJ ie J J Now for any y and x imply

Paul Tseng

holds Dividing b oth sides of by and using y and G x x for

j j

c c

all j J J yields in the limit that

T T

Gx y x Gx y x

here so x VI G

Assume in addition F is analytic on an op en set containing so that by

part a there exist and such that holds for all Let

y satisfy kx y k distx Now for each since x

and x imply

T T

F x x x F x Gx x x

Adding these two inequalities and dividing by gives Gx x x Also

y x since J J Adding x VI G and y imply Gx

gives these two inequalities and usingx

T T T

x Gx y x Gx Gx x x Gx

c c

J J

J J J J

Combining this with and renaming kGx k as yields

nn n

c Assume F x M x q for some M q and assume

is convex Consider any I J such that j j and M N M and

c c

J J J J J J

jJ jjJ j T jJ jjJ j

for some N First we N M and some N M

J J J J J J J J

J J

for all y If M y q for some i N and some y q have M

i i

J J

y then the convexity of would imply x for all x so i J

d Fix any y Since x then d y x satises d and M

J J J J

Moreover for each i J with x we have d Thus there exists

i i

such that z x d for all suciently small Then x and

x imply

M x q Gx z x

d Gx x q M

J J J J

d x x Gx M

J J J

d N x x Gx x x M

c c

J J J J J J J J

Gx d

T T

where the last equality uses M N d This shows that d M

J J J J

Gx Since x so that J J if J J then the convexity d

of would imply the existence of y with y Using this y in the

ab ove argument would yield d and hence Gx a contra d

J J

diction Thus J J Then G is continuous at x for all j J and the ab ove

j j

inequality yields in the limit as that

T T

Gx y x Gx d

J J

Illp osed Variatonal Problems and Regularization Techniques

here sox VI G

Consider any I J with j j M p ositive semidenite and M

I II IJ

jI jjJ j

c c

Then x satises For each for some N M N

I IJ II IJ

Homans lemma implies has a solution y satisfying with and

Then y since the line segment joiningx and y lies in so the

factx VI G implies

Gx y x

T T

Gx y x Gx x x

I I

kGx kkGx k kx k Gx x x

I I J I

J I

Also we have from and x satisfying that

M x q M x q Gx

I I

yields M x and x which when subtracted and using x

II I

J nI

Gx This and the p ositive semidenite prop erty of M x x N

I II I IJ

yield

c c

x M x x N x x x N

c c

II I IJ I IJ

J J

x x x N Gx

I IJ I

Dividing the ab ove inequality by and adding it to the inequality we

obtain

T T

kGx kkGx k kx x k Gx Gx x x Gx N

c c

I I J I IJ

J J I I

Using x x and renaming kGx k as yield

I I I

J nI

nn n

c Assume F x M x q for some M q and assume

is convex and each G is strictly increasing Fix any I J such that j j

j I

For each Homans lemma implies has a solution y satisfying

with and Let denote the set of x satisfying and

I I

ky k We claim that there exists a scalar such that for kxk sup

every y there exists a K J such that

M y q j K and M q for some

j j L L

where L J nK cf pro of of Prop osition If not then for every

sequence of scalars k tending to zero there would exist a

k c

such that for every K J we have

k k

M q for some j K or M q

j j L L

c k

where L J nK Since is b ounded and closed then k has a

cluster p oint such that for every K J we have

M q for some j K or M q

j j L L

Paul Tseng

where L J nK However this cannot b e true since the ab ove relations fail

to hold for K fj J M q g and

j j

For each we have y and hence there exists a K J such that

holds with y y and L J nK Since the numb er of such subset K is

nite by passing into a subsequence if necessary we can assume it is the same

K for all Since and x x imply x y as we

I I

have M x q for all j K and suciently small in which case

j j I

and the strictly increasing prop erty of G would imply M x q G x

j j j j

x G M x q G j K

j j

For each let L fj L M x q h g and let L LnL

I j j j

Since there is only a nite numb er of dierent L and L by passing to a

subsequence if necessary we can assume that L and L are the same for all

Then we have as argued ab ove that

h h j L x G

j j

We claim there exists constant such that

k kx h k kx

K L

1 L nJ

j L

for all If not then there would exist a subsequence of along

I I

which kx k h kx k By there exists

K L

j L

1 L nJ

satisfying M q Then would satisfy which together with

L L

implies

x M x Gx

I I I

0 0

x Gx M

J J J nI

x x

x M

K L

x q x M M

L L L

2 2 2

where I fi I g J fi J nI M q g and K fi K

i i i

k and using M x q h M q g Dividing b oth sides by kx

j j j i i

L nJ

for j L would yield in the limit that

u M u

I I

u u M

J nI

u u M

K L K L

1 1

u u M

L L

2 2

for some u with u Then since and M q the

L L

L nJ

vector u would b e in for all suciently small Since and x

satisfy so that this vector is also in Since u this would

L nJ

contradict the fact that x for all x

c J

Illp osed Variatonal Problems and Regularization Techniques

Assume M M We claim that there exists constant such that

k kx k kx

LnJ

for all If not then there would exist a subsequence of along which

I I

kx k kx k By there exists satisfying M q

L L

LnJ

Then would satisfy which together with implies

x M x Gx

I I I

0 0

x x Gx M

J J

J nI

x x x M

K K

x M x

L L

where I fi I g J fi J nI M q g and K fi

i i i

K M q g Dividing b oth sides by kx k would yield in the limit that

i i

LnJ

u M u

I I

u u M

J nI

u u M

K K

u M u

L L

n T T

for some u with u Since M M so that u M u the

LnJ

ab ove implies u M u for all j L Then since and M q

j j L L

the vector u would b e in for all suciently small and hence as

argued earlier would b e in Since u this would contradict the fact

LnJ

that x for all x

nn n

p osi c Assume F x M x q for some M q with M

J J

tive semidenite Also assume G is strictly increasing with lim sup tG t

j j

for all j J Fix any I J such that j j Let H J Initialize

to comprise all suciently small so that G x for all j J

I j

Given H for some l we construct b elow by passing to a subsequence of

if necessary a prop er subset H of H having the desired prop erties

l l

until H J For notational simplicity we will write H and H as H and

l l l

new

H resp ectively dropping the subscript l First by passing to a subsequence

if necessary we assume there exist q and h n

satisfying

x M q as

NH h

H h

x To see that such a decomp osition exists let q M If q for

all small then cho ose q and for h n Otherwise

take any subsequence of along which q and q kq k converges Let q

b e its limit and let q b e the orthogonal pro jection of q onto the subspace

Paul Tseng

orthogonal to q Thenq q q for some satisfying kq k along

the subsequence Apply the ab ove construction inductively to q restricted to

the ab ove subspace yields For each we have from and

G x for all j J and J H that x satises cf

M x x M q Gx

LH LH L L

H H

q Gx M x M c x

J nI J nI J nIH J nIH

J nI

H H

c c c c

x q x M x M

H H H H H

H H

H nJ

where for convenience we let L I H nJ Letting q q I J nI

K I and L H and y x we see from the ab ove relations and

that the following holds with k

M y y q Gx

H LH K h L L

k k h

y M y Gx q

H I H I h I

J nI

k k k

k k h

y q M y

h L H L H

H nJ

k k

k h hk k

u I J y x

h k

for all Now supp ose that hold for some k for all By

further passing to a subsequence if necessary we can assume one of the following

two cases o ccurs

Case for all There exist j H nJ and such that G x

new new

In this case let H H nfj g and we have that H is a prop er subset of

H and contains J Moreover the strictly increasing prop erty of G implies

x G

k j

as For all j H nJ G x Case

In this case by further passing to a subsequence we can assume either

or converges as

Supp ose as Since holds for all dividing

all sides by and using L I H nJ and the fact we are in Case yield in

the limit

u M u q

k K LH k H k L

u M u q

k I H k H k I

J nI

k k

u M u q

k L H k H k L

H nJ

k k

By further passing for some u Notice that u x so u

k J

to a subsequence if necessary we can assume one of the following two sub cases o ccurs

Illp osed Variatonal Problems and Regularization Techniques

Sub case a There exist j H nJ and such that y u for

k j

k k

all

new new

In this sub case let H H nfj g and we have that H is a prop er subset

of H and contains J Moreover u x so yields

j j

u y u x

h j k j

h k k

Sub case b For all j H nJ y u as

k j

k k

In this sub case let I fi I M u q g and K fi

k k iH k H k i k

K u g and L fi L M u q g Then and

k k i k k iH k H k i

yield

M y u q Gx y u

LH k H h L L k K

k+1

k k h k k

u M y u y Gx q

k H I H k I h I

J nI

k+1 k+1 k+1

k k k k h

u y q u M y

k h L k H L H

H nJ

k+1 k+1

k k h hk k k

u and we see that y for all suciently small Letting y

k k k

hold with k replaced by k Below we show that k n so that

we can rep eat the ab ove construction with k replaced by k Supp ose not

so that k n Then dividing all sides of by min y and using

j H nJ

the fact that we are in Sub case b so that G x for all j L and

y for all j H nJ as yield in the limit that

M u u u j H nJ

LH n H n n j

J nI

for some u Then using this and and H L J nI we see that

y y ky k u satises

H nJ

n n

M y Gx

LL L

u so that y Multiplying the ab ove Also y x x

h h

equation on the left by y and using the p ositive semidenite prop erty of

Gx is p ositive semidenite and L J yields y M since M

L LL

J J

Dividing b oth sides by and using L I H nJ gives

X X

G x y G x y

j j

j j j j

j I

j H nJ

For each j H nJ we have y y ky k u y

j n j j

H nJ

j n n n

x u so that also using G x

h j j

j j

G x x u x G x y

j h j j

j j j h j j

Paul Tseng

Since we are in Sub case b with k n then for each j H nJ we have

y as so that y x u yields

j j h j

n n

h h

x This together with the ab ove inequality and lim sup tG t

implies the lefthand side of is p ositive and b ounded away from zero On

the other hand we have Gx Gx and y as so the

I I

righthand side of tends to zero a contradiction

Supp ose instead converges to some c as Then k

for j H nJ it must b e that and since we are in Case and G x

H nJ ie H J The rst equation in can then b e written using

y as

J nI

M y q Gx

II I h I I

k h

Dividing this by yields in the limit

M u cq Gx

II k I k I I

for some u Combining the ab ove two equations yields

q Gx Gx M cq

h I I I II I k I

h k

u u Multiplying the lefthand x x where u y

h k k

h k

is and using the p ositive semidenite prop erty of M since M side by

J J

p ositive semidenite and I J yields

T T

Gx Gx cq q

I I k I h I

I I k h

Thus dividing b oth sides by and expanding yields

x x Gx Gx

I I

Gx Gx u u

I I h I k I

n k

X X

h k

c q q x x u u

k I h I I h I k I

hk h

u u kGx Gx k

h I k I I I

k n

X X

k h

u u kx x k c q q

h I k I I k I h I

h hk

Illp osed Variatonal Problems and Regularization Techniques

Supp ose in addition G is lo cally Lipschitzian at x with constant and

J J

satises with Then for all suciently small

so that kx x k and kGx Gx k kx x k the preceding

I I I I

inequality yields

r s r s

P P

k n

k h

where r k u u k s c q q

h I k I k I h I

h hk

and kx x k Solving this using the quadratic formula yields

r s r s r s r s r s r s s

a where the last two inequalities use the identities a b b and ab

a b This and x and by taking suciently large so that

J nI

r for all yields

k h

kx x k c kq k kq k

J h J k J

Thus letting in case and letting u in

h j

k h

case yield Similarly yields for suitable choice of

By rep eating the ab ove argument with n in place of we can extract

I I

another subsequence of having the same prop erties as and so on We

do this for all I J with j j thus yielding a countable collection of

subsequences whose union is

Note A few words ab out the assumptions in Prop osition are in or

der First the assumptions on F x and in part a are satised

H H

by H J if F is pseudomonotone on since is convex in this

case Second the assumption of lim tG t in part c is equiv

t j

if G is strictly increasing This is b e alent to lim G

cause G c implies by G b eing strictly increasing that

c cG c Conversely tG t c for some c implies by G

j j

b eing strictly increasing that c G cttc Third the assumptions

on M in part c are satised by any I J N if M is symmetric p osi

tive semidenite see eg Lemma or if M is symmetric nondegenerate

ie M is nonsingular for all I N It is also satised by any I J N

which is neither symmetric nor nondegenerate Fourth if M

for the h dened in part c direct calculation nds that for G given by

and for G given by h is the uniqe satisfying ln h

j j

Paul Tseng

so that h o for any xed To see that the b ound is

reasonable notice that for n and F x and Gx x we have

Gx so that x G Similarly for n and

Notice F x x and Gx x we have x Gx so that x

that the skew symmetry assumption M M is satised when an LP is for

mulated as a CP The dep endence of K on cannot b e removed as is shown by

an example in in the context of LP Fifth the nesting of index sets in part

c reects a nested dep endence of the convergence rate of some comp onents

of x indexed by H on the remaining comp onents Intuitively if x converges

more slowly than x then the term M x can inuence what the limitx will

ij i

i j

b e and the rate at which x converges to this limit

Note Ifx in Prop osition satises strict complementarity iex F x

then parts c and c of this prop osition simplify considerably In par

ticular we have F x as well as F x G x for all j

j j j

J so that F x F x and the strictly increasing prop erty of G yield

j j j

for all suciently small where F x G x G

j j

min F x

j J j

Note If F is ane and pseudomonotone on and G is strictly increasing

with lim tG t for all j J then and Prop osition cc

t j

yield the error b ound and for all suciently small with J J

and some constants and with K J dep ending on

c n

and L J nK Similarly if F is ane and monotone on and G is

strictly increasing with lim sup tG t for all j J and G is Lipschitz

j j

continuous and strongly monotone near x for all j J then and

Prop osition cc yield the error b ound for all suciently

small along some subsequence etc Moreover there exists a c such that

tG t c for all j J and all t suciently small implying t G ct

j j

Thus the second case in implies the rst case

Note Prop osition do es not say anything ab out existence or uniqueness or

b oundedness of x VI F G In the case where G is given by it

was shown by Ko jima et al Theorem that F b eing a continuous P

function and satisfying strict feasibility ie x F x has a solution and

a b oundedness condition implies the existence and uniqueness of x for all also

see Theorem for the case of ane F and see for extensions to other

typ es of F Analogous results were shown earlier by McLinden in the context

of convex programs and more generally when F is a maximal monotone

op erator These results were further improved and extended by Ko jima et

al and Guler Recently Chen et al Corollary showed that

F b eing a continuously dierentiable P function and b eing nonempty and

b ounded is sucient for the existence and uniqueness of x for all suciently

small Subsequently Gowda and Tawhid Theorems and weakened the

dierentiability assumption on F to continuity and considered more general

regularizations on F

Illp osed Variatonal Problems and Regularization Techniques

We illustrate Prop osition with the following example with n variables

Example Consider

G x x

F x M xq M q Gx G x x

G x x

with Notice that M is p ositive semidenite and it can b e checked that

ft t t g

i Supp ose so that lim tG t Then either by direct calculation

or by using Prop osition c we nd that x x VI G as

Thus J J f g and for suciently small we have x and

hence F x Gx Then direct calculation yields

x x G x G

and x x x O This illustrates

parts c and c of Prop osition

ii Supp ose so that lim tG t Then direct calculation nds

that for all suciently small we have x and x x

with x satisfying x G x Thus x x and

J J f g Moreover x VI p with ft t t g

J J J J

Lastly we have and p

x G x x

and hence x x This illustrates parts a and c of Prop o

sition

iii Supp ose and G G are changed to G t G t t First

we claim that for each x exists and is unique To see this let I f g

is p ositive semidenite and and note that M

b b y M x for some x y M x q

I I I II I I I I I

for all x so a result of Ko jima et al Corollary Theorem

implies that for each x the equation M x M x q Gx

II I I I I

has a unique solution x x which is continuous in x and is b ounded as

x Then the equation

M x x M x q G x x x x x

I I

has a solution x since the lefthand side is continuous in x and tends

to as x and tends to as x Then x x x x

Paul Tseng

satises

x x x G x

F x Gx

x x G x

Uniqueness of x follows from F G b eing strictly monotone on Now

for all x x x x x and x imply x

so x is b ounded as Then x has a cluster p oint x which by

Prop osition is in Since M is p ositive semidenite so that is convex

or p VI and J f g Prop osition a with H J implies either x

J J J

ft t t g for some where G p VI x

J J J J J

In either case we havex so that the third equation in and p

yields

x x x x x x G

p VI Since this shows x so we are in the case of x

J J J

yielding x and J fg Thus J nJ fg Now subtracting the

second equation in from the rst equation and using G t t yields

x x G x so that cf

x G x x G x x

Finally the second equation in implies

min f g

x x x x x

Notice that G is lo cally Lipschitzian at x and satises with some

This illustrates parts a c and c of Prop osition

For part c we have H f g H fg Corresp ondingly for l the

decomp osition with the subscript l restored holds with q M

x yielding x For l holds with q M

T T

x if or q M x if

etc

iv Supp ose and G G are changed to G t G t t as in

iii Supp ose we also change M to M It can b e seen that this

do es not change Moreover M is p ositive semidenite so is convex

and J f g Using an argument analogous to that used in iii we have

that x exists and is unique for all and x is b ounded as Also M

satises the assumptions in Prop osition c for any I N so it follows that

any cluster p oint of x is in VI G f g Thus x x

with J J Moreover F x Gx yields

h x

Illp osed Variatonal Problems and Regularization Techniques

recall h is the unique satisfying G and using

symmetry x x O x O This illustrates parts c

and c of Prop osition Compared to iii we see that changing M changes

b oth the limit p ointx and the convergence rate even when G and the solution

set are unchanged

Summary and Op en Questions

We have considered regularizing the mapping F in a complementarity problem

by another mapping and we studied prop erties of any limit p oint of the so

lution of the regularized problem We have also derived error b ounds on the

distance from the solution of the regularized problem to its limit p oint These

error b ounds are fairly complex reecting b oth the lo cal growth rate of the reg

ularization mapping and the linkage among solution comp onents through the

complementarity condition

There remain many op en questions to b e answered We list a few b elow

Q Can parts c and c of Prop osition b e simpliedstrengthened in the

case of G G

Q For the G given by the convergence result of McLinden requires

F to b e monotone and continuous whereas our error b ound result requires

F to b e ane and satisfying the assumptions of either part c or part c

of Prop osition For this particular choice of G can an error b ound result

analogous to Prop osition cc b e obtained for nonane F

Q Consider higherorder regularization of the form F x F x G x

p p p

G x G x where p and G G are suitable mappings

What can we say ab out any limit p oint of x VI F as See

Section for discussions in the optimization setting What kind of error

b ounds can b e derived

Q Here we have considered the CP where the feasible set is Can our

results b e extended to variational inequality problems where the feasible set is

a p olyhedral set or more generally a nonempty closed convex set of How

ab out extension to spaces other than such as the space of n n symmetric

matrices with replaced by the convex cone of n n symmetric p ositive

semidenite matrices or an innitedimensional space

References

Attouch H Viscosity Solutions of Minimization Problems SIAM Journal

on Optimization Vol pp

Paul Tseng

Attouch H Chbani Z and Mouda A Recession Op erators and Solv

ability of Variational Problems in Reexive Banach Spaces in Calculus of

Variations Homogenization and Continuum Mechanics edited by G Bou

chitte G Buttazzo and P Suquet pp World Scientic Teaneck

Attouch H and Wets R JB Quantitative Stability of Variational Sys

tems I I I approximate Solutions Mathematical Programming Vol pp

Auslender A Cominetti R and Haddou M Asymptotic Analysis for

Penalty and Barrier Metho ds in Convex and Linear Programming Mathe

matics of Operations Research Vol pp

Auslender A Optimisation Methodes Numeriques Masson Paris

Billups S C and Ferris M C QPCOMP A Quadratic Program Based

Solver for Mixed Complementarity Problems Mathematical Programming

Vol pp

Bonnans J F and Haddou M Asymptotic Analysis of Congested Com

munication Networks Research Rep ort No Institut National de

Recherche en Informatique et en Automatique Ro cquencourt March

Brezis H Operateurs Maximaux Monotones et SemiGroupes de Contrac

tions dans les Espaces de Hilbert NorthHolland Amsterdam

Chen B Chen XJ and Kanzow C A Penalized FischerBurmeister

NCPFunction Theoretical Investigation and Numerical Results Depart

ment of Management and Systems Washington State University Pullman

Septemb er

Cominetti R and Dussault J P Stable Exp onentialPenalty Algorithm

with Sup erlinear Convergence Journal of Optimization Theory and Appli

cations Vol pp

Cominetti R and San Martin J Asymptotic Analysis of the Exp onen

tial Penalty Tra jectory in Linear Programming Mathematical Programming

Vol pp

Cottle R W Pang JS and Stone R E The Linear Complementarity

Problem Academic Press New York New York

Dontchev A L and Zolezzi T Wel lPosed Optimization Problems

SpringerVerlag Berlin

Facchinei F Structural and Stability Prop erties of P Nonlinear Comple

mentarity Problems Dipartimento di Informatica e Sistemistica Universita di Roma La Sapienza Rome April

Illp osed Variatonal Problems and Regularization Techniques

Facchinei F and Kanzow C Beyond Monotonicity in Regularization

Metho ds for Nonlinear Complementarity Problems Dipartimento di Infor

matica e Sistemistica Universita di Roma La Sapienza Rome May

Fang S C An Unconstrained Convex Programming View of Linear Pro

gramming Zeitschrift fur Operations Research Methods and Models of

Operations Research Vol pp

Fang S C Ra jasekera J R and Tsao HS J Entropy Optimization

and Mathematical Programming Kluwer Norwell

Fang S C and Tsao HS J Linear Programming with Entropic Per

turbation Zeitschrift fur Operations Research Methods and Models of Op

erations Research Vol pp

Ferris M C and Mangasarian O L Finite Perturbation of Convex Pro

grams Applied Mathematics and Optimization Vol pp

Fiacco A V and McCormick G P Nonlinear Programming Sequential

Unconstrained Minimization Techniques Wiley New York reprinted

by SIAM Philadelphia

Fischer A Merit Functions and Stability for Complementarity Problems

Institute for Numerical Mathematics Technical University of Dresden Dres

den Decemb er

Gowda M S and Tawhid M A Existence and Limiting Behavior of

Tra jectories Asso ciated with P Equations Department of Mathematics and

Statistics University of Maryland Baltimore Novemb er

Guler O Existence of Interior Points and Interior Paths in Nonlinear

Monotone Complementarity Problems Mathematics of Operations Research

Vol pp

Harker P and Pang JS FiniteDimensional Variational Inequality and

Nonlinear Complementarity Problems a Survey of Theory Algorithms and

Applications Mathematical Programming Vol pp

Homan A J On Approximate Solutions of Systems of Linear Inequali

ties Journal of Research of the National Bureau of Standards Vol pp

Isac G Tikhonovs Regularization and the Complementarity Problem in

Hilb ert Space Journal of Mathematical Analysis and Applications Vol

Karlin S Mathematical Methods and Theory in Games Programming

and Economics Vol I AddisonWesley Reading

Paul Tseng

Ko jima M Megiddo N and Mizuno S A General Framework of Contin

uation Metho ds for Complementarity Problems Mathematics of Operations

Research Vol

Ko jima M Megiddo N and Noma T Homotopy Continuation Meth

o ds for Nonlinear Complementarity Problems Mathematics of Operations

Research Vol pp

Ko jima M Megiddo N Noma T and Yoshise A A Unied Approach

to Interior Point Algorithms for Linear Complementarity Problems Lecture

Notes in Computer Science SpringerVerlag Berlin

Ko jima M Mizuno M and Noma T Limiting Behavior of Tra jecto

ries Generated by a Continuation Metho ds for Monotone Complementarity

Problems Mathematics of Operations Research Vol pp

Lemaire B WellPosedness Conditioning and Regularization of Minimiza

tion Inclusion and FixedPoint Problems Rep ort Department of Mathe

matics University of Montp ellier Montp ellier Novemb er to app ear

in Pliska Studia Mathematica

Luo ZQ and Pang JS Error Bounds for Analytic Systems and Their

Applications Mathematical Programming Vol pp

Luo ZQ and Sturm J F Error Bounds for Quadratic Systems De

partment of Electrical and Computer Engineering McMaster University

Hamilton January

Luo ZQ and Tseng P On the Convergence of a Matrix Splitting Algo

rithm for the Symmetric Monotone Linear Complementarity Problem SIAM

Journal on Control and Optimization Vol pp

Madsen K Nielsen H B and Pinar M C A New Finite Continuation

Algorithm for Linear Programming SIAM Journal on Optimization Vol

Mangasarian O L Iterative Solution of Linear Programs SIAM Journal

on Numerical Analysis Vol pp

Mangasarian O L Normal Solution of Linear Programs Mathematical

Programming Study Vol pp

Mangasarian O L Some Applications of Penalty Functions in Mathemat

ical Programming in Optimization and Related Fields edited by R Conti

E De Giorgi and F Giannessi pp SpringerVerlag Heidelb erg

Illp osed Variatonal Problems and Regularization Techniques

Mangasarian O L and Meyer R R Nonlinear Perturbation of Linear

Programs SIAM Journal on Control and Optimization Vol pp

McLinden L An Analogue of Moreaus Proximation Theorem Pacic

Journal of Mathematics Vol pp

McLinden L The Complementarity Problem for Maximal Monotone Mul

tifunctions in Variational Inequalities and Complementarity Problems The

ory and Applications edited by R W Cottle F Giannessi and JL Lions

pp John Wiley Sons New York

Megiddo N Pathways to the Optimal Set in Linear Programming in

Progress in Mathematical Programming InteriorPoint and Related Meth

ods edited by N Megiddo pp SpringerVerlag

Pinar M C PiecewiseLinear Pathways to the Optimal Solution Set in

Linear Programming Journal of Optimization Theory and Applications to

app ear

Ra jasekera J R and Fang S C Deriving an Unconstrained Convex

Program for Linear Programming Journal of Optimization Theory and Ap

plications Vol pp

Ravindran G and Gowda M S Regularization of P Functions in Box

Variational Inequality Problems Department of Mathematics and Statistics

University of Maryland Baltimore August

Robinson S M Generalized Equations and Their Solutions Part I Basic

Theory Mathematical Programming Study Vol pp

Ro ckafellar R T and Wets R JB Variational Analysis Springer

Verlag Berlin

Subramanian P K A Note on Least Two Norm Solutions of Monotone

Complementarity Problems Applied Mathematics Letters Vol pp

Sun D A Regularization Newton Metho d for Solving Nonlinear Comple

mentarity Problems Applied Mathematics Rep ort AMR Scho ol of

Mathematics the University of New South Wales Sydney July re

vised February to app ear in Applied Mathematics and Optimization

Szna jder R and Gowda M S On the Limiting Behavior of the Tra jectory

of Regularized Solutions of a P Complementarity Problem Department of

Mathematics and Statistics University of Maryland Baltimore Octob er

Paul Tseng

Tseng P Convergence and Error Bound for Perturbation of Linear Pro

grams Department of Mathematics University of Washington Seattle June

revised May to app ear in Computational Optimization and Ap

plications

Venkateswaran V An Algorithm for the Linear Complementarity Problem

with a P Matrix SIAM Journal on Matrix Analysis and Applications Vol

Wright S J PrimalDual InteriorPoint Methods SIAM Publications Philadelphia