Definitions and Examples of Inverse and Ill-Posed Problems

c de Gruyter 2008 J. Inv. Ill-Posed Problems 16 (2008), 317–357 DOI 10.1515 / JIIP.2008.069

Deﬁnitions and examples of inverse and ill-posed problems

S. I. Kabanikhin

Survey paper

Abstract. The terms “inverse problems” and “ill-posed problems” have been steadily and surely gaining popularity in modern science since the middle of the 20th century. A little more than fifty years of studying problems of this kind have shown that a great number of problems from various branches of classical mathematics (computational algebra, differential and integral equations, partial differential equations, functional analysis) can be classified as inverse or ill-posed, and they are among the most complicated ones (since they are unstable and usually nonlinear). At the same time, inverse and ill-posed problems began to be studied and applied systematically in physics, geophysics, medicine, astronomy, and all other areas of knowledge where mathematical methods are used. The reason is that solutions to inverse problems describe important properties of media under study, such as density and velocity of wave propagation, elasticity parameters, conductivity, dielectric permittivity and magnetic permeability, and properties and location of inhomogeneities in inaccessible areas, etc. In this paper we consider definitions and classification of inverse and ill-posed problems and describe some approaches which have been proposed by outstanding Russian mathematicians A. N. Tikhonov, V. K. Ivanov and M. M. Lavrentiev.

Key words. Inverse and ill-posed problems, regularization.

AMS classification. 65J20, 65J10, 65M32. 1. Introduction. 2. Classification of inverse problems. 3. Examples of inverse and ill-posed problems. 4. Some first results in ill-posed problems theory. 5. Brief survey of definitions of inverse problems.

1. Introduction

Everything has been said before, but since nobody listens we have to keep going back and beginning all over again. Andre Gide

First publications on inverse and ill-posed problems date back to the first half of the 20th century. Their subjects were related to physics (inverse problems of quantum was partly supported by Russian Foundation for Basic Research (grants 08-01-00312-a, 08-01-90260-Uzb). 318 S. I. Kabanikhin scattering theory), geophysics (inverse problems of electrical prospecting, seismology, and potential theory), astronomy, and other areas of science. Since the advent of powerful computers, the area of application for the theory of inverse and ill-posed problems has extended to almost all fields of science that use mathematical methods. In direct problems of mathematical physics, researchers try to find exact or approximate functions that describe various physical phenomena such as the propagation of sound, heat, seismic waves, electromagnetic waves, etc. In these problems, the media properties (expressed by the equation coefficients) and the initial state of the process under study (in the nonstationary case) or its properties on the boundary (in the case of a bounded domain and/or in the stationary case) are assumed to be known. However, it is precisely the media properties that are often unknown. This leads to inverse problems, in which it is required to determine the equation coefficients from the information about the solution of the direct problem. Most of these problems are ill-posed (unstable with respect to measurement errors). At the same time, the unknown equation coefficients usually represent important media properties such as density, electrical conductivity, heat conductivity, etc. Solving inverse problems can also help to determine the location, shape, and structure of intrusions, defects, sources (of heat, waves, potential difference, pollution), and so on. Given such a wide variety of applications, it is no surprise that the theory of inverse and ill-posed problems has become one of the most rapidly developing areas of modern science since its emergence. Today it is almost impossible to estimate the total number of scientific publications that directly or indirectly deal with inverse and ill-posed problems. However, since the theory is relatively young, there is a shortage of textbooks on the subject. This is under- standable, since many terms are still not well-established, many important results are still being discussed and attempts are being made to improve them. New approaches, concepts, and theorems are constantly emerging.

2. Classiﬁcation of inverse problems One calls two problems inverse to each other if the formulation of one problem involves the other one. J. B. Keller

In our everyday life we are constantly dealing with inverse and ill-posed problems and, given good mental and physical health, we are usually quick and effective in solving them. For example, consider our visual perception. It is known that our eyes are able to perceive visual information from only a limited number of points in the world around us at any given moment. Then why do we have an impression that we are able to see everything around? The reason is that our brain, like a personal computer, completes the perceived image by interpolating and extrapolating the data received from the identified points. Clearly, the true image of a scene (generally, a three-dimensional color scene) can be adequately reconstructed from several points only if the image is familiar to us, i.e., if we previously saw and sometimes even touched most of the objects in it. Thus, although the problem of reconstructing the image of an object and its surroundings is ill-posed (i.e., there is no uniqueness or stability of solutions), our Inverse and ill-posed problems 319 brain is capable of solving it rather quickly. This is due to the brain’s ability to use its extensive previous experience (a priori information). A quick glance at a person is enough to determine if he or she is a child or a senior, but it is usually not enough to determine the person’s age with an error of at most five years. Attempting to understand a substantially complex phenomenon and solve a problem such that the probability of error is high, we usually arrive at an unstable (ill-posed) problem. Ill-posed problems are ubiquitous in our daily lives. Indeed, everyone real- izes how easy it is to make a mistake when reconstructing the events of the past from a number of facts of the present (for example, to reconstruct a crime scene based on the existing direct and indirect evidence, determine the cause of a disease based on the results of a medical examination, and so on). The same is true for tasks that involve predicting the future (predicting a natural disaster or simply producing a one week weather forecast) or “reaching into” inaccessible zones to explore their structure (subsurface exploration in geophysics or examining a patient’s brain using NMR tomography). Almost every attempt to expand the boundaries of visual, aural, and other types of perception leads to ill-posed problems. What are inverse and ill-posed problems? While there is no universal formal definition for inverse problems, an “ill-posed problem” is a problem that either has no solutions in the desired class, or has many (two or more) solutions, or the solution procedure is unstable (i.e., arbitrarily small errors in the measurement data may lead to indefinitely large errors in the solutions). Most difficulties in solving ill-posed problems are caused by the solution instability. Therefore, the term “ill-posed problems” is often used for unstable problems. To define various classes of inverse problems, we should first define a direct (forward) problem. Indeed, something “inverse” must be the opposite of something “direct”. For example, consider problems of mathematical physics. In mathematical physics, a direct problem is usually a problem of modeling some physical fields, processes, or phenomena (electromagnetic, acoustic, seismic, heat, etc.). The purpose of solving a direct problem is to find a function that describes a physical field or process at any point of a given domain at any instant of time (if the field is nonstationary). The formulation of a direct problem includes

• the domain in which the process is studied; • the equation that describes the process; • the initial conditions (if the process is nonstationary); • the conditions on the boundary of the domain. For example, we can formulate the following direct initial-boundary value problem for the acoustic equation: In the domain

n Ω ⊂ R with boundary Γ = ∂Ω, (2.1) it is required to ﬁnd a solution u(x, t) to the acoustic equation

−2 c (x)utt = ∆u − ∇ ln ρ(x) · ∇u + h(x, t) (2.2) 320 S. I. Kabanikhin that satisﬁes the initial conditions

u(x, 0) = ϕ(x), ut(x, 0) = ψ(x) (2.3) and the boundary conditions ∂u = g(x, t). (2.4) ∂n Γ Here u(x, t) is the acoustic (exceeded) pressure, c(x) is the speed of sound in the medium, ρ(x) is the density of the medium, and h(x, t) is the source function. Like most direct problems of mathematical physics, this problem is well-posed, which means that it has a unique solution and is stable with respect to small perturbations in the data. The following is given in the direct problem (2.1)–(2.4): the domain Ω, the coefﬁcients c(x) and ρ(x), the source function h(x, t) in the equation (2.2), the initial conditions ϕ(x) and ψ(x) in (2.3), and the boundary conditions g(x, t) in (2.4). In the inverse problem, aside from u(x, t), the unknown functions include some of the functions occurring in the formulation of the direct problem. These unknowns are called the solution to the inverse problem. In order to ﬁnd the unknowns, the equations (2.2)–(2.4) are supplied with some additional information about the solution to the direct problem. This information represents the data of the inverse problem. For example, let the additional information be represented by the values of the solution to the direct problem (2.2)–(2.4) on the boundary:

u|Γ = f(x, t). (2.5) In the inverse problem, it is required to determine the unknown functions occurring in the formulation of the direct problem from the data f(x, t). Inverse problems of mathematical physics can be classiﬁed into groups depending on which functions are unknown or some other criteria. We will use our example to describe this classiﬁcation.

2.1. Classification based on the unknown functions The inverse problem (2.2)–(2.5) is said to be retrospective (Alifanov et al, 1995; Engl et al, 1996; Kabanikhin et al, 2006) if it is required to determine the initial conditions, i.e., the functions ϕ(x) and ψ(x) in (2.3). The inverse problem (2.2)–(2.5) is called a boundary problem, if it is required to determine the function in the boundary condition (the function g(x, t)). The inverse problem (2.2)–(2.5) is called a continuation problem if the initial conditions (2.3) are unknown and the additional information (2.5) and the boundary conditions (2.4) are specified only on a certain part Γ1 ⊆ Γ of the boundary of the domain Ω, and it is required to find the solution u(x, t) of equation (2.2) (ex- tend the solution to the interior of the domain; Lavrentiev, 1967). The inverse problem (2.2)–(2.5) is called a source problem if it is required to determine the source, i.e., the function h(x, t) in equation (2.2). The inverse problem (2.2)–(2.5) is called a coefficient inverse problem or inverse medium problem if it is required to reconstruct the coefficients (c(x) and ρ(x)) in the main equation (Romanov, 1987, 2002; Isakov, 1998; Kabanikhin and Lorenzi, 1999). Inverse and ill-posed problems 321

It should be noted that this classiﬁcation is still incomplete. There are cases where both initial and boundary conditions are unknown, and cases where the domain Ω (or a part of its boundary) is unknown.

2.2. Classification based on the additional information Aside from the initial-boundary value problem (2.2)–(2.4), it is possible to give other formulations of the direct problem of acoustics (such as spectral, scattering, kinematic formulation, etc.) in which it is required to find the corresponding parameters of the acoustic process (eigenfrequencies, reflected waves, wave travel times, and so on). Measuring these parameters in experiments leads to new classes of inverse problems of acoustics. In practice, the data (2.5) on the boundary of the domain under study are the most accessible for measurement, but sometimes measuring devices can be placed inside the object under study:

u(xm, t) = fm(t), m = 1, 2,..., (2.6) which leads us to interior problems (Lavrentiev and Saveliev, 2006). Much like problems of optimal control, retrospective inverse problems include the so-called “ﬁnal” observations

u(x, T ) = fˆ(x) (2.7)

(Alifanov et al, 1995; Maksimov, 2002). Inverse scattering problems are formulated in the case of harmonic oscillations u(x, t) = eiwtu¯(x, ω), the additional information being speciﬁed, for example, in the form

u¯(x, ωα) = f¯(x, α), x ∈ X1, α ∈ Ω, (2.8) where X1 is the set of observation points and {ωα}α∈Ω is the set of observation frequencies. A lot of theoretical results and applications can be found in (Chadan and Sabatier, 1989; Colton and Kress, 1992; Kress, 2007). In some cases, the eigenvalues of the corresponding differential operator

∆U − ∇ ln ρ · ∇U = λU and various eigenfunction characteristics are known (inverse spectral problems) (Lev- itan, 1987; Yurko, 2002). It is sometimes possible to register the arrival time at points {xk} for the waves generated by local sources concentrated at points {xm}:

m m m τ(x , xk) = f˜(x , xk), xk ∈ X1, x ∈ X2.

In this case, the problem of reconstructing the speed c(x) is called an inverse kinematic problem (see Lavrentiev et al, 1986; Romanov, 1987; Uhlmann, 1999). 322 S. I. Kabanikhin

2.3. Classiﬁcation based on equations

As shown above, the acoustic equation alone yields as many as M1 different inverse problems depending on the number and the form of the unknown functions. On the other hand, one can obtain M2 different variants of inverse problems depending on the number and the type of the measured parameters (additional information), i.e., the data of the inverse problem. Let q represent the unknown functions, and let f denote the data of the inverse problem. Then the inverse problem can be written in the form of an operator equation

Aq = f, (2.9) where A is an operator acting the space Q of the unknown elements to the space F of the measured parameters. It should be noted in conclusion that, instead of the acoustic equation, we could take the heat conduction equation (Alifanov et al, 1995; Lattes and Lions, 1967; Kabanikhin et al, 2006), the transport equation, Laplace or Poisson equation (Anger, 1990; Chered- nichenko, 1996; Vasin and Ageev, 1995) or the systems of Lame´ or Maxwell equations (Romanov, 1987; Romanov and Kabanikhin, 1994), and the like. Suppose this could yield M3 different variants. Then, for equations of mathematical physics alone, it is possible to formulate about M1M2M3 different inverse problems. Many of these inverse problems became the subjects of monographs and the great number of papers.

2.4. The structure of the operator A The direct problem (2.1)–(2.4) can be written in the operator form

A1(Γ, c, ρ, h, ϕ, ψ, g) = u.

This means that the operator A1 maps the data of the direct problem (namely, boundary Γ of the domain, coefficients c and ρ, source function h, initial data ϕ and ψ, boundary data g) to the solution of the direct problem, u(x, t). We call A1 the operator of the direct problem. Some data of the direct problem are unknown in the inverse problem. We denote these unknowns by q, and the restriction of A1 to q by A1. For example, in coefficient inverse problem two coefficients c(x) and ρ(x) are unknown, therefore q = (c, ρ) and then A1(q) = u(x, t; q). The measurements operator A2 maps the solution u(x, t) of the direct problem to the additional information f, for example, A2u = u|Γ or A2u = u(xk, t), k = 1, 2,... , etc. Then equation (2.9) becomes as follows: Aq ≡ A2A1q = f, i.e., the operator A is the result of the consecutive application (composition) of the operators A1 and A2. For example, in the retrospective inverse problem, we have q = (ϕ, ψ), f = u(x, T ), A1q = u(x, t), and A2u = u(x, T ); in the coefficient inverse problem, we have q = (c, ρ), f = u|Γ, A1q = u(x, t), and A2u = u|Γ, etc. The operator A1 of the direct problem is usually continuous (the direct problem is well- posed), and so is the measurements operator (normally, one chooses to measure stable Inverse and ill-posed problems 323

parameters of the process under study). The properties of the composition A2A1 are usually even too good (in ill-posed problems, A = A2A1 is often a compact operator). This complicates ﬁnding the inverse of A, i.e., solving the inverse problem Aq = f. In a simple example where A is a constant, the smaller the number A being multiplied by q, generally speaking, the smaller the error (if q is approximate):

A(q + δq) = f˜ = Aq + A δq, i.e., the operator A of the direct problem has good properties. However, when solving the inverse problem with approximate data f˜ = f + δf, we have

f˜ f δf q˜ = q + δq = = + , A A A δf δq = , A and the error δq = q˜ − q tends to infinity as A → 0 if the measurement error is fixed. Note that in one of the most difficult ill-posed problem, namely, in operator equation Aq = f with compact operator A : D(A) ⊂ Q → R(A) ⊂ F acting between separable Hilbert spaces Q and F , we again meet the division by small (singular) numbers

∞ X hf, uji q = vj, (2.10) σj j=1 because singular values 0 ≤ ... ≤ σn ≤ σn−1 ≤ . . . σ1 of the compact operator A tend to zero while n → ∞.

2.5. Inverse problem is often (almost always) ill-posed The study of inverse and ill-posed problems began in the early 20th century. In 1902, J. Hadamard formulated the concept of the well-posedness (properness) of problems for differential equations. A problem is called well-posed in the sense of Hadamard if there exists a unique solution to this problem that continuously depends on its data (see Definition 4.1). Hadamard also gave an example of an ill-posed problem (see Exam- ple 3.12), namely, the Cauchy problem for Laplace equation. In 1943, A. N. Tikhonov pointed out the practical importance of such problems and the possibility of finding stable solutions to them. In the late 1950’s and especially early 1960’s there appeared a series of new approaches that became fundamental for the theory of ill-posed problems and attracted the attention of many mathematicians to this theory. With the advent of powerful computers, inverse and ill-posed problems started to gain popularity very rapidly. By the present day, the theory of inverse and ill-posed problems has developed into a new powerful and dynamic field of science that has an impact on almost every area of mathematics, including algebra, calculus, geometry, differential equations, mathematical physics, functional analysis, computational mathematics, etc. Some examples of well-posed and ill-posed problems are presented in the table below. It should be emphasized that, one way or the other, each ill-posed problem in the right column 324 S. I. Kabanikhin

Well-posed problems Ill-posed problems Arithmetic Multiplication by a small number A Division by a small number Aq = f q = A−1f (A 1)

Algebra q = A−1f, Multiplication by a matrix A is an ill-conditioned, Aq = f degenerate or rectangular m × n-matrix

Calculus Integration Differentiation Z x f(x) = f(0) + q(ξ) dξ q(x) = f 0(x) 0 Differential equations The Sturm–Liouville problem The inverse Sturm–Liouville problem. u00(x) − q(x)u(x) = λu(x), Find q(x) using spectral data u(0) − hu0(0) = 0, {λ , ku k} u(1) − Hu0(1) = 0 n n

Integral geometry Z Z Find integrals q(x, y) ds Find q from q(x, y) ds = f(ξ, η) Γ(ξ,η) Γ(ξ,η) Integral equations Volterra equations Volterra equations and Fredholm equations and Fredholm equations of the second kind of the ﬁrst kind Z x Z x q(x) + K(x, ξ)q(ξ) dξ = f(x) K(x, ξ)q(ξ) dξ = f(x) 0 0 Z b Z b q(x) + K(x, ξ)q(ξ) dξ = f(x) K(x, ξ)q(ξ) dξ = f(x) a a Operator equations Aq = f A : D(A) ⊂ Q → R(A) ⊂ F ∃m > 0: ∀q ∈ Q A is a compact linear operator mhq, qi ≤ hAq, qi with singular values σn & 0, n → ∞ Inverse and ill-posed problems 325

Elliptic equations ∆u = 0, x ⊂ Ω ∂u ∆u = 0, x ∈ Ω u|Γ = g or = f, ∂n Γ Cauchy problem ∂u Initial-boundary value problem or αu + β = h ∂n Γ with data given on

Dirichlet or Neumann problem, a part of the boundary Γ1 ⊂ Γ = ∂Ω Robin problem (mixed)

Parabolic equations Cauchy problem in reversed time (backwards parabolic equation) ut = ∆u, t > 0, x ∈ Ω −ut = ∆u, t > 0, x ∈ Ω, Cauchy problem u|t=0 = f u|t=0 = f(x) Initial-boundary value problem with Initial-boundary value problem data given on a part of the boundary (sideways heat equation) u|t=0 = 0 u = ∆u, x ∈ Ω u|Γ = g(x, t) t ∂u u = f1, = f2 Γ1 ∂n Γ1

Hyperbolic equations Dirichlet and Neumann problems Cauchy problem Cauchy problem with data on utt = ∆u, t > 0, a time-like surface u|t=0 = ϕ(x), ut|t=0 = ψ(x) utt = ∆u, x ∈ Ω Initial-boundary value problem ∂u u|Γ = g u = f1, = f2 Γ1 ∂n Γ1

Direct problems Coefﬁcient inverse problems (inverse medium problem)

utt = ∆u − q(x)u ( utt = ∆u − q(x)u u|t=0 = ϕ(x), ut|t=0 = ψ(x) u|t=0 = ϕ(x), ut|t=0 = ψ(x) ∂u u|Γ = f, = g ∂n Γ ut = ∆u − q(x)u ut = ∆u − q(x)u u| = 0 t=0 ∂u u|t=0 = 0, u|Γ = f, = g ∂n Γ ∇(q(x)∇u) = 0, x ∈ Ω ∂u ∇(q(x)∇u) = 0 u|Γ = g or = f Γ ∂u ∂n u|Γ = g and = f ∂n Γ 326 S. I. Kabanikhin of the table can be formulated as an inverse problem with respect to the corresponding direct problem from the left column. The theory of inverse and ill-posed problems is also widely used in solving applied problems in almost all ﬁelds of science, in particular:

• physics (astronomy, quantum mechanics, acoustics, electrodynamics, etc.); • geophysics (seismic exploration, electrical, magnetic and gravimetric prospecting, logging, magnetotelluric sounding, etc.); • medicine (X-ray and NMR tomography, ultrasound testing, etc.); • ecology (air and water quality control, space monitoring, etc.); • economics (optimal control theory, ﬁnancial mathematics, etc.). (More detailed examples of applications of inverse and ill-posed problems are given in the next section.) Without going into further details of mathematical deﬁnitions, we note that, in most cases, inverse and ill-posed problems have one important property in common: instability. In the majority of cases that are of interest, inverse problems turn out to be ill-posed and, conversely, an ill-posed problem can usually be reduced to a problem that is inverse to some direct (well-posed) problem. To sum up, it can be said that specialists in inverse and ill-posed problems study the properties of and regularization methods for unstable problems. In other words, they develop and study stable methods for approximating unstable mappings, trans- formations, or operations. From the point of view of information theory, the theory of inverse and ill-posed problems deals with maps from data tables with very small epsilon-entropy to tables with large epsilon-entropy (Babenko, 2002). Many problems of mathematical statistics in a sense may be considered as inverse to some problems of probability theory (Bertero and Boccacci, 1998; Engl et al, 1999; Tarantola, 2005).

3. Examples of inverse and ill-posed problems

Example 3.1 (algebra, systems of linear algebraic equations). Consider the system of linear algebraic equations Aq = f, (3.1) where A is an m×n matrix, q and f are n- and m-dimensional vectors, respectively. Let the rank of A be equal to min (m, n). For m < n the system may have many solutions. For m > n there may be no solutions. For m = n the system has a unique solution for any right-hand side. In this case, there exists an inverse operator (matrix) A−1. It is bounded, since it is a linear operator in a ﬁnite-dimensional space. Thus, all three conditions of well-posedness in the sense of Hadamard are satisﬁed. We now analyze in detail the dependence of the solution on the perturbations of the right-hand side f in the case where the matrix A is nondegenerate. Subtracting the original equation (3.1) from the perturbed equation

A(q + δq) = f + δf, (3.2) Inverse and ill-posed problems 327 we obtain Aδq = δf, which implies δq = A−1δf and kδqk ≤ kA−1kkδfk. We also have kAkkqk ≥ kfk. As a result, we have the best estimate for the relative error of the solution: kδqk kδfk ≤ kAkkA−1k , (3.3) kqk kfk which shows that the error is determined by the constant µ(A) = kAkkA−1k called the condition number of the system (matrix). Systems with relatively large condition number are said to be ill-conditioned. For normalized matrices (kAk = 1), it means that there are relatively large elements in the inverse matrix and, consequently, small variations in the right-hand side may lead to relatively large (although ﬁnite) variations in the solution. Therefore, systems with ill-conditioned matrices can be considered practically unstable, although formally the problem is well-posed and the stability condition kA−1k < ∞ holds. For example, the matrix  1 a 0 ... 0   0 1 a . . . 0     . . . . .   ......   . . . .     0 0 0 . . . a  0 0 0 ... 1 is ill-conditioned for sufﬁciently large n and |a| > 1 because there are elements of the form an−1 in the inverse matrix. In the case of perturbations in the elements of the matrix, the estimate (3.3) becomes as follows: kδqk kδAk . kδAk ≤ µ(A) 1 − µ(A) kqk kAk kAk (for kA−1kkδAk < 1). Let m = n and the determinant of A be zero. Then the system (3.1) may either have no solutions or more than one solution. It follows that the problem Aq = f is ill-posed for degenerate matrices A (det A = 0).

0 Example 3.2 (calculus, differentiation). Let√f (x) = q(x) and suppose√ that instead of f(x) we know fn(x) = f(x)+sin (nx)/ n. Then qn(x) = q(x)+ n cos (nx) and we conclude that kq − qnkC → ∞ while kf − fnkC → 0 for n → ∞.

Example 3.3 (calculus, summing a Fourier series). The problem of summing a Fou- rier series consists in finding a function q(x) from its Fourier coefficients. We show that the problem of summation of a Fourier series is unstable with respect to small variations in the Fourier coefficients in the l2 metric if the variations of the sum are estimated in the C metric. Let ∞ X q(x) = fk cos kx, k=1 328 S. I. Kabanikhin

and let the Fourier coefﬁcients fk of the function q(x) have small perturbations: f˜k = fk + ε/k. Set ∞ X q˜(x) = f˜k cos kx. k=1

The coefﬁcients of these series in the l2 metric differ by

∞ ∞ r n X o1/2 n X 1 o1/2 π2 (f − f˜ )2 = ε = ε , k k k2 6 k=1 k=1 which vanishes as ε → 0. However, the difference

∞ X 1 q(x) − q˜(x) = ε cos kx k k=1 can be as large as desired because the series diverges for x = 0. Thus, if the C metric is used to estimate variations in the sum of the series, then summation of the Fourier series is not stable.

Example 3.4 (geometry). Consider a body in space that can be illuminated from various directions. If the shape of the body is known, the problem of determining the shape of its shadow is well-posed. The inverse problem of reconstructing the shape of the body from its projections (shadows) onto various planes is well-posed only for convex bodies, since a concave area obviously cannot be detected this way. Note that it was Aristotle who first formulated and solved a problem of this type. Observing the shadow cast by the Earth on the moon, he concluded that the Earth is spherical: “Some think the Earth is spherical, others that it is flat and drum-shaped. For evidence they bring the fact that, as the sun rises and sets, the part concealed by the Earth shows a straight and not a curved edge, whereas if the Earth were spherical the line of section would have to be circular...... but in eclipses of the moon the outline is always curved: and, since it is the interposition of the Earth that makes the eclipse, the form of this line will be caused by the form of the Earth’s surface, which is therefore spherical”. (Aristotle. “On the Heavens”. Written 350 B.C.E.). It is clear that considering only the shape of the Earth’s shadow on the surface of the moon is not sufficient for the unique solution of the inverse problem of projective geometry (reconstructing the shape of the Earth). Aristotle writes that some think the Earth is drum-shaped based on the fact that the horizon line at sunset is straight. And he provides two more observations as evidence that the Earth is spherical (uses additional information concerning the solution of inverse problem): objects fall vertically (towards the center of gravity) at any point of the Earth’s surface, and the celestial map changes as the observer moves on the Earth’s surface (which contradicts to the drum-shaped Earth). Example 3.5 (integral geometry on straight lines, Radon inversion, X-ray tomography). One of the problems arising in computerized tomography is to determine a func- Inverse and ill-posed problems 329 tion of two variables q(x, y) from the collection of integrals Z q(x, y) dl = f(p, ϕ) L(p,ϕ) along various straight lines L(p, ϕ) in the plane (x, y), where p and ϕ are the line parameters.

Example 3.6 (integral geometry on circles). Consider the problem of determining a function of two variables q(x, y) from the integral of this function over a collection of circles whose centers lie on a fixed line. Assume that q(x, y) is continuous for all (x, y) ∈ R2. Consider a collection of circles whose centers lie on a fixed line (for definiteness, let this line be the coordinate axis y = 0). Let L(a, r) denote the circle (x − a)2 + y2 = r2, which belongs to this collection. It is required to determine q(x, y) from the function f(x, r) such that Z q(ξ, τ) dl = f(x, r), (3.4) L(x,r) and f(x, r) is defined for all x ∈ (−∞, ∞) and r > 0. The solution of this problem is not unique in the class of continuous functions, since for any continuous functionq ˜(x, y) such thatq ˜(x, y) = −q˜(x, −y) the integrals Z q˜(ξ, τ) dl L(x,r) vanish for all x ∈ R and r > 0. Indeed, using the change of variables ξ = x + r cos ϕ, τ = r sin ϕ, we obtain Z Z 2π q˜(ξ, τ) dl = q˜(x + r cos ϕ, r sin ϕ)r dϕ = (3.5) L(x,r) 0 Z π Z 2π = q˜(x + r cos ϕ, r sin ϕ)r dϕ + q˜(x + r cos ϕ, r sin ϕ)r dϕ. 0 π Puttingϕ ¯ = 2π − ϕ and using the conditionq ˜(x, y) = −q˜(x, −y), we transform the last integral in the previous formula: Z 2π Z 0 q˜(x + r cos ϕ, r sin ϕ)r dϕ = q˜(x + r cosϕ, ¯ −r sinϕ ¯)r dϕ¯ = π π Z π = − q˜(x + r cosϕ, ¯ r sinϕ ¯)r dϕ.¯ 0 Substituting the result into (3.5), we have Z q˜(ξ, τ) dl = 0 L(x,r) for x ∈ R, r > 0. 330 S. I. Kabanikhin

Thus, if q(x, y) is a solution to the problem (3.4), then q(x, y)+q˜(x, y), whereq ˜(x, y) is any continuous function such thatq ˜(x, y) = −q˜(x, −y) is also a solution to (3.4). For this reason, the problem can be reformulated as the problem of determining the even component of q(x, y) with respect to y. The ﬁrst well-posedness condition is not satisﬁed in the above problem: solutions may not exist for some f(x, r). However, the uniqueness of solutions in the class of even functions with respect to y can be established (Denisov, 1999).

Example 3.7 (differential equation). The rate of radioactive decay is proportional to the amount of the radioactive substance. The proportionality constant q1 is called the decay constant. The process of radioactive decay is described by the solution of the Cauchy problem for an ordinary differential equation du = −q u(t), t ≥ 0, (3.6) dt 1

u(0) = q0, (3.7) where u(t) is the amount of the substance at a given instant of time and q0 is the amount of the radioactive substance at the initial instant of time. The direct problem: Given the constants q0 and q1, determine how the amount of the substance u(t) changes with time. This problem is obviously well-posed. Moreover, its solution can be written explicitly as follows:

−q1t u(t) = q0e , t ≥ 0.

Assume now that the decay constant q1 and the initial amount of the radioactive substance q0 are unknown, but we can measure the amount of the radioactive substance u(t) for certain values of t. The inverse problem consists in determining the coefﬁcient q1 in equation (3.6) and the initial condition q0 from the additional information about the solution to the direct problem u(tk) = fk, k = 1, 2,...,N.

Example 3.8 (system of differential equations). A chemical kinetic process is described by the solution of the Cauchy problem for the system of linear ordinary differential equations du i = q u (t) + q u (t) + ... + q u (t), (3.8) dt i1 1 i2 2 in n

ui(0) = q¯i, i = 1,...,N. (3.9)

Here ui(t) is the concentration of the ith substance at the instant t. The constant parameters qij characterize the dependence of the change rate of the concentration of the ith substance on the concentration of the substances involved in the process. The direct problem: Given the parameters qij and the concentrationq ¯i at the initial instant of time, determine ui(t). The following inverse problem can be formulated for the system of differential equations (3.8). Given that the concentrations of substances ui(t), i = 1,...,N, are measured over a period of time t ∈ [t1, t2], it is required to determine the values of the Inverse and ill-posed problems 331

parameters qij, i.e., to find the coefficients of the system (3.8) from a solution of this system. Two versions of this inverse problem can be considered. In the first version, the initial conditions (3.9) are known, i.e.,q ¯i are given and the corresponding solutions ui(t) are measured. In the second version,q ¯i are unknown and must be determined together with qij (Denisov, 1995).

Example 3.9 (differential equation of the second order). Suppose that a particle of unit mass is moving along a straight line. The motion is caused by a force q(t) that depends on time. If the particle is at the origin x = 0 and has zero velocity at the initial instant t = 0, then, according to Newton’s laws, the motion of the particle is described by a function u(t) satisfying the Cauchy problem

d2u = q(t), t ∈ [0,T ], (3.10) dt2 du u(0) = 0, (0) = 0, (3.11) dt where u(t) is the coordinate of the particle at the instant t. Assume now that the force q(t) is unknown, but the coordinate of the particle u(t) can be measured at any instant of time (or at certain points of the interval [0,T ]). It is required to reconstruct q(t) from u(t). Thus, we have the following inverse problem: determine the right-hand side of equation (3.10) (the function q(t)) from the known solution u(t) of the problem (3.10), (3.11). We now prove that the inverse problem is unstable. Let u(t) be a solution to the direct problem for some q(t). Consider the following perturbations of the solution to the direct problem: 1 u (t) = u(t) + cos (nt). n n These perturbations correspond to the right-hand sides

qn(t) = q(t) − n cos (nt).

Obviously, ku − unkC[0,T ] → 0 as n → ∞, and kq − qnkc[0,T ] → ∞ as n → ∞. Thus, the problem of determining the right-hand side of the linear differential equation (3.10), (3.11) from its right-hand side is unstable. Note that the inverse problem is reduced to double differentiation if the values of u(t) are given for all t ∈ [0,T ].

Example 3.10 (Fredholm integral equation of the first kind). Consider the problem of solving a Fredholm integral equation of the first kind Z b K(x, s)q(s) ds = f(x), c ≤ x ≤ d, (3.12) a where the kernel K(x, s) and the function f(x) are given and it is required to find q(s). It is assumed that f(x) ∈ C[c, d], q(s) ∈ C[a, b], and K(x, s), Kx(x, s), and Ks(x, s) 332 S. I. Kabanikhin are continuous in the rectangle c ≤ x ≤ d, a ≤ s ≤ b. The problem of solving equation (3.12) is ill-posed because solutions may not exist for some functions f(x) ∈ C[c, d]. For example, take a function f(x) that is continuous but not differentiable on [c, d]. With such a right-hand side, the equation cannot have a continuous solution q(s) since the conditions for the kernel K(x, s) imply that the integral in the left-hand side of (3.12) is differentiable with respect to the parameter x for any continuous function q(s). The condition of continuous dependence of solutions on the initial data is also not satisfied for equation (3.12).

Example 3.11 (Volterra integral equations of the ﬁrst kind). Consider the problem of solving a Volterra integral equation of the ﬁrst kind Z x K(x, s) q(s) ds = f(x), 0 ≤ x ≤ 1. (3.13) 0 0 For K ≡ 1 the problem√ (3.13) is equivalent to differentiation f (x) = q(x). The sequence fn = cos (nx)/ n demonstrates the instability of the problem.

Example 3.12 (Cauchy problem for Laplace equation). Let u = u(x, y) be a solution to the following problem:

∆u = 0, x > 0, y ∈ R, (3.14) u(0, y) = f(y), y ∈ R, (3.15) ux(0, y) = 0, y ∈ R. (3.16) Let the data f(y) be chosen as follows: 1 f(y) = u(0, y) = sin (ny). n Then the solution to the problem (3.14)–(3.16) is given by 1 u(x, y) = sin (ny)(enx + e−nx) ∀n ∈ . (3.17) n N For any fixed x > 0 and sufficiently large n, the value of the solution (3.17) can be as large as desired, while f(y) tends to zero as n → ∞. Therefore, small variations 1 l in the data in C or W2 (for any l < ∞) may lead to indefinitely large variation in the solution, which means that the problem (3.14)–(3.16) is ill-posed.

Example 3.13 (inverse problem for a partial differential equation of the ﬁrst order). Let q(x) be continuous and ϕ(x) be continuously differentiable for all x ∈ R. Then the following Cauchy problem is well-posed: 2 ux − uy + q(x)u = 0, (x, y) ∈ R , (3.18) u(x, 0) = ϕ(x), x ∈ R. (3.19) Inverse and ill-posed problems 333

Consider the inverse problem of reconstructing q(x) from the additional information about the solution to the problem (3.18), (3.19)

u(0, y) = ψ(y), y ∈ R. (3.20) The solution to (3.18), (3.19) is given by the formula

x Z 2 u(x, y) = ϕ(x + y) exp q(ξ) dξ , (x, y) ∈ R . x+y

The condition (3.20) implies

0 Z ψ(y) = ϕ(y) exp q(ξ) dξ , y ∈ R. (3.21) y

Thus, the conditions

1) ϕ(y) and ψ(y) are continuously differentiable for y ∈ R;

2) ψ(y)/ϕ(y) > 0, y ∈ R; ψ(0) = ϕ(0) are necessary and sufﬁcient for the existence of a solution to the inverse problem, which is determined by the formula

d h ψ(x)i q(x) = − ln , x ∈ . (3.22) dx ϕ(x) R

If ϕ(y) and ψ(y) are only continuous, then the problem is ill-posed. Example 3.14 (Cauchy problem for the heat equation in reversed time — backward parabolic equation). The Cauchy problem in reversed time is formulated as follows: Let a function u(x, t) satisfy the equation

ut = uxx, 0 < x < π, t > 0 (3.23) and the boundary conditions

u(0, t) = u(π, t) = 0, t > 0. (3.24)

It is required to determine the values of u(x, t) at the initial instant t = 0:

u(x, 0) = q(x), 0 ≤ x ≤ π, (3.25) given the values of u(x, t) at a ﬁxed instant of time t = T > 0

u(x, T ) = f(x), 0 ≤ x ≤ π. (3.26) 334 S. I. Kabanikhin

This problem is inverse to the problem of ﬁnding a function u(x, t) satisfying (3.23)– (3.25), provided that the function q(x) is given. The solution to the direct problem (3.23)–(3.25) is given by the formula

∞ X −n2t u(x, t) = e qn sin nx, (3.27) n=1 where {qn} are the Fourier coefﬁcients of q(x):

π 2 Z q (x) = q(x) sin (nx) dx. n π 0 Setting t = T in (3.27), we get

∞ X −n2T f(x) = e qn sin nx, x ∈ [0, π], (3.28) n=1 which implies n2T qn = fn e , n = 1, 2,..., where {fn} are the Fourier coefﬁcients of f(x). Since the function q(x) ∈ L2(0, π) is uniquely determined by its Fourier coefﬁ- 2 cients {qn}, the solution of the inverse problem is unique in L (0, π). Note that the condition (3.25) holds as a limit condition:

π Z 2 lim u(x, t) − q(x) dx = 0. t→+0 0 The inverse problem (3.23)–(3.26) has a solution if and only if

∞ X 2 2n2T fn e < ∞, n=1 which obviously cannot hold for all functions f ∈ L2(0, π).

Example 3.15 (coefﬁcient inverse problem for the heat conduction equation). The solution u(x, t) to the boundary value problem for the heat conduction equation

cρut = (kux)x − αu + f, 0 < x < l, 0 < t < T, (3.29)

u(0, t) − λ1ux(0, t) = µ1(t), 0 ≤ t ≤ T, (3.30)

u(`, t) − λ2ux(l, t) = µ2(t), 0 ≤ t ≤ T, (3.31) u(x, 0) = ϕ(x), 0 ≤ x ≤ l, (3.32) Inverse and ill-posed problems 335 describes many physical processes such as heat distribution in a bar, diffusion in a hollow tube, etc. The equation coefficients and the boundary conditions represent the parameters of the process under study. If the problem (3.29)–(3.32) describes the process of heat distribution in a bar, then c and k are the heat capacity coefficient and the heat conduction coefficient, respectively, which characterize the material of the bar. In this case, the direct problem consists in determining the temperature of the bar at a point x at an instant t (i.e., determining the function u(x, t)) from the known parameters c, ρ, k, α, f, λ1, λ2, µ1, µ2, and ϕ. Now suppose that all coefficients and functions that determine the solution u(x, t) except the heat conduction coefficient k = k(x) are known, and the temperature of the bar can be measured at a certain interior point x0, i.e., u(x0, t) = f(t), 0 ≤ t ≤ T . The following inverse problem arises: determine the heat conduction coefficient k(x), provided that the function f(t) and all other functions in (3.29)–(3.32) are given. Other inverse problems can be formulated in a similar way, including the cases where C = C(u), k = k(u), etc. (Alifanov, Artyukhin, and Rumyantsev, 1988).

Example 3.16 (interpretation of measurement data). The operation of many measurement devices that register nonstationary ﬁelds can be described as follows: a signal q(t) arrives at the input of the device, and a function f(t) is registered at the output. In the simplest case, the functions q(t) and f(t) are related by the formula

Z t g(t − τ)q(τ) dτ = f(t). (3.33) 0 In this case, g(t) is called the impulse response function of the device. In theory, g(t) is the output of the device in the case where the input is the generalized function δ(t), R t i.e., Dirac’s delta function: a g(t − τ)δ(t)dτ = g(t). In practice, in order to obtain g(t), a sufﬁciently short and powerful impulse is provided as an input. The resulting output function represents the impulse response function. Thus, the problem of interpreting measurement data, i.e., determining the form of the input signal q(t) is reduced to solving the integral equation of the ﬁrst kind (3.33). The relationship between the input signal q(t) and the output function f(t) can be more complicated. For a “linear” device, this relationship has the form

Z t K(t, τ)q(τ) dτ = f(t). 0 The relationship between q(t) and f(t) can be nonlinear:

Z t K(t, τ, q) dτ = f(t). 0 This model describes the operation of devices that register alternate electromagnetic ﬁelds, pressure and tension modes in a continuous medium, seismographs, which record vibrations of the Earth’s surface, and many other kinds of devices. 336 S. I. Kabanikhin

Example 3.17. The most general and important example of linear ill-posed problem is the operator equation Aq = f. Here A : D(A) ⊂ Q → R(A) ⊂ F is the compact operator, Q and F are separable Hilbert spaces. Using singular system {σn, un, vn} one can construct pseudo-solution with minimal norm (qnp — normal pseudo-solution)

† X hf, uni qnp = A f = vn, σn σn=6 0 where A† is the Moore-Penronse inverse of a compact linear operator A. The Picard criterion (Engl et al, 1996)

2 † X |hf, uni| f ∈ D(A ) ⇐⇒ 2 < ∞ σn σn=6 0 says that the best approximate solution qnp exists only if the (generalized) Fourier co- efﬁcients hf, uni with respect to singular functions un decay fast enough relative to the singular values. Therefore the solutions of Aq = f may exist only for a special √ √ data f. Moreover if we choose fn = σn un then qn = vn/ σn and Aqn = fn. So fn → 0 but qn → ∞ for n → ∞. The degree of ill-posedness (Hofmann, 1995) is described by the convergence rate of σn & 0, conditional stability function (Ka- banikhin and Schieck, 2008; Lavrentiev and Saveliev, 2006), modulus of continuity ω(M, δ) := sup {kqk : q ∈ M, kAqk ≤ δ} or index function.

Remark 3.18. To solve the equation (3.33), one can use Fourier or Laplace transforms. Letg ˜(λ),q ˜(λ), and f˜(λ) be the Fourier transforms of the functions g(t), q(t), and f(t), respectively: Z ∞ Z ∞ Z ∞ g˜(λ) = eiλt g(t) dt, q˜(λ) = eiλt q(t) dt, f˜(λ) = eiλt f(t) dt. 0 0 0 Then, by the convolution theorem, g˜(λ)q˜(λ) = f˜(λ), and, consequently, the inversion of the Fourier transform yields the formula for the solution of (3.33): 1 Z ∞ 1 Z ∞ f˜(λ) q(t) = e−iλt q˜(λ) dλ = e−iλt dλ. (3.34) 2π −∞ 2π −∞ g˜(λ) The calculation method provided by formula (3.34) is unstable because the func- tiong ˜(λ), which is the Fourier transform of the impulse response function of the device, tends to zero as λ → ∞ in real-life devices. This means that arbitrarily small variations in the measured value of f˜(λ) can lead to very large variations in the solution q(t) for sufﬁciently large λ.

Remark 3.19. If g(t) is a constant, then the problem of solving (3.33) is reduced to differentiation. Inverse and ill-posed problems 337

4. Some first results in ill-posed problems theory All happy families resemble one another, each unhappy family is unhappy in its own way. Leo Tolstoy There is no universal method for solving ill-posed problems. In every specific case, the main “trouble” — instability — has to be tackled in its own way. This section is a brief introduction to the foundations of the theory of ill-posed problems. We begin with abstract definitions of well-posedness (Section 4.1), ill-posedness, and conditional well-posedness. Paying tribute to the founders of the theory of ill- posed problems, we devote the next three sections 4.2–4.4 to the theorems and methods formulated by A. N. Tikhonov, V. K. Ivanov, and M. M. Lavrentiev. The Tikhonov’s version of the theorem on the continuity of the inverse mapping A−1 : A(M) → M for any compact set M and continuous one-to-one operator A provides the basis for the theory of conditionally well-posed problems and for constructing a variety of effective numerical algorithms for solving them (Section 4.2). The concept of a quasi- solution introduced by V.K. Ivanov (Section 4.3) serves three purposes: it generalizes the concept of a solution, restores all the well-posedness conditions, and yields new algorithms for the approximate solution of ill-posed problems. The Lavrentiev method (Section 4.4) is used to construct numerical algorithms for solving a wide class of systems of algebraic equations, linear and nonlinear integral and operator equations of the first kind. The Tikhonov concept of a regularizing family of operators (Section 4.5) is fundamental to the theory of ill-posed problems. In brief, a regularizing family {Rα}α>0 (or a regularizing sequence {Rn}) consists of operators Rα such that there exists a stable procedure for constructing approximate solutions qα = Rαf to the equation Aq = f which converge to the exact solution qe as α → +0. Another important property of regularizing operators consists in providing a way to construct an approximate solution qαδ = Rαfδ from the approximate data fδ for which the solution to the equation Aq = fδ may not exist. In many cases, it can be proved that qαδ converges to the exact solution qe if the regularization parameter α and the error δ in the right-hand side f tend to zero being connected by some special rule.

4.1. Well-posed and ill-posed problems Let an operator A map a topological space Q into a topological space F (A : Q → F ). For any topological space Q, let O(q) denote a neighbourhood of an element q ∈ Q. Throughout what follows, D(A) is the domain of deﬁnition and R(A) is the range of A. Deﬁnition 4.1 (well-posedness of a problem; well-posedness in the sense of Hadamard) The problem Aq = f is well-posed on the pair of topological spaces Q and F if the following three conditions hold:

(1) for any f ∈ F there exists a solution qe ∈ Q to the equation Aq = f (the existence condition), i.e., R(A) = F ;

(2) the solution qe to the equation Aq = f is unique in Q (the uniqueness condition), i.e., there exists an inverse operator A−1 : F → Q; 338 S. I. Kabanikhin

(3) for any neighbourhood O(qe) ⊂ Q of the solution qe to the equation Aq = f, there is a neighbourhood O(f) ⊂ F of the right-hand side f such that for all −1 fδ ∈ O(f) the element A fδ = qδ belongs to the neighbourhood O(qe), i.e., the operator A−1 is continuous (the stability condition).

Note that we use the notation qe for the exact solution in opposite to qk which is quasi-solution. Definition 4.1 can be made more specific by replacing the topological spaces Q and F by metric, Banach, Hilbert, or Euclidean spaces. In some cases, it makes more sense to take a topological space for Q and a Euclidean space for F , and so on. It is only the requirements of the existence, uniqueness, and stability of the solution that are fixed in the definition.

Deﬁnition 4.2. The problem Aq = f is ill-posed on the pair of spaces Q and F if at least one of the three well-posedness conditions does not hold. M. M. Lavrentiev proposed to distinguish the class of conditionally well-posed problems as a subclass of ill-posed problems. Let Q and F be topological spaces and let M ⊂ Q be a ﬁxed set. We denote by A(M) the image of M under the map A : Q → F , i.e., A(M) = {f ∈ F : ∃q ∈ M such that Aq = f}. It is obvious that A(M) ⊂ F .

Deﬁnition 4.3 (conditional/Tikhonov well-posedness). The problem Aq = f is said to be conditionally well-posed on the set M if f ∈ A(M) and the following conditions hold:

(1) a solution qe to the equation Aq = f, f ∈ A(M), is unique on M;

(2) for any neighbourhood O(qe) of a solution to the equation Aq = f, there exists a neighbourhood O(f) such that for any fδ ∈ O(f) ∩ A(M) the solution to the equation Aq = fδ belongs to O(qe)(conditional stability).

It should be emphasized that the variations fδ of the data f in the second condition are assumed to lie within the class A(M), which ensures the existence of solutions.

Deﬁnition 4.4. The set M in Deﬁnition 4.3 is called the well-posedness set for the problem Aq = f.

Remark 4.5. To prove that the problem Aq = f is well-posed, it is necessary to prove the existence, uniqueness and stability theorems for its solutions. To prove that the problem Aq = f is conditionally well-posed, it is necessary to choose a well-posedness set M, prove that the solution of the problem is unique in M and that q is conditionally stable with respect to small variations in the data (the right-hand side) f that keep the solutions within the well-posedness set M.

Remark 4.6. Stability depends on the choice of topologies on Q and F . Formally, the continuity of the operator A−1 can be ensured, for example, by endowing F with Inverse and ill-posed problems 339 the strongest topology. If A is a linear one-to-one operator and Q and F are normed spaces, then the following norm can be introduced in F :

−1 kfkA = kA fk.

In this case kA−1fk kA−1k = sup = 1 f=6 0 kfkA and therefore A−1 is continuous. In practice, however, the most commonly used spaces are Cm and Hn, where m and n are not very large.

Remark 4.7. It should be emphasized that proving the well-posedness of a problem necessarily involves proving the existence of a solution, whereas in the case of conditional well-posedness the existence of a solution is assumed. It certainly does not mean that the existence theorem is of no importance or that one should not aim to prove it. We only mean to emphasize that in the most interesting and important cases, the necessary conditions for the existence of a solution qe to a conditionally well-posed problem Aq = f which should hold for the data f turn out to be too complicated to verify and directly apply in numerical algorithms (see the Picard criterion, the solvability conditions for the inverse Sturm-Liouville problem, etc.). In this sense, the title of V.P. Maslov’s paper “The existence of a solution of an ill-posed problem is equivalent to the convergence of a regularization process” (Uspekhi Mat. Nauk, 1968) is very characteristic. It reveals one of the main problems in the study of strongly ill-posed problems. For this reason, the introduction of the concept of conditional well-posedness shifts the focus to the search for stable methods for approximate solution of ill-posed problems. How- ever, this does not make the task of detailed mathematical analysis of the solvability conditions for any speciﬁc problem less interesting or important! In particular, one can construct regularizing algorithms converging to a pseudo-solution (of linear algebraic systems) or a quasi-solution when the exact solution does not exist (Tikhonov et al, 1983). In the following section we will show that the uniqueness of a solution implies its conditional stability if the well-posedness set M is compact.

4.2. The Tikhonov approach The trial-and-error method (Tikhonov and Arsenin, 1974) was one of the ﬁrst methods for the approximate solution of conditionally well-posed problems. In the description of this method, we will assume that Q and F are metric spaces and M is a compact set. For an approximate solution, we choose an element qK from the well-posedness set M such that the residual ρF (Aq, f) attains its minimum at qK , i.e.,

ρF (AqK , f) = inf ρF (Aq, f) q∈M

(see Deﬁnition 4.15 in Section 4.3 for the deﬁnition of a quasi-solution). 340 S. I. Kabanikhin

Let {qn} be a sequence of elements in Q such that lim ρF (Aqn, f) = 0. We n→∞ denote by qe the exact solution to the problem Aq = f. If M is compact, then from lim ρF (Aqn, f) = 0 it follows that lim ρQ(qn, qe) = 0, which is proved by the n→∞ n→∞ following theorem.

Theorem 4.8. Let Q and F be metric spaces, and let M ⊂ Q be compact. Assume that A is a one-to-one map from M onto A(M) ⊂ F . Then, if A is continuous, then so is the inverse map A−1 : A(M) → M.

−1 Proof. Let f ∈ A(M) and Aqe = f. We will prove that A is continuous at f. Assume the contrary. Then there exists an ε∗ > 0 such that for any δ > 0 there is an element fδ ∈ A(M) for which

−1 −1 ρF (fδ, f) < δ and ρQ(A (fδ),A (f)) ≥ ε∗.

Consequently, for any n ∈ N there is an element fn ∈ A(M) such that

−1 −1 ρF (fn, f) < 1/n and ρQ(A (fn),A (f)) ≥ ε∗.

Therefore, lim fn = f. n→∞ −1 −1 Since A (fn) ∈ M and M is compact, the sequence {A (fn)} has a converging −1 subsequence: lim A (fn ) = q¯. k→∞ k −1 −1 Since ρQ(q,¯ A (f)) ≥ ε∗, it follows thatq ¯ =6 A (f) = qe. On the other hand, −1 from the continuity of the operator A it follows that the subsequence A(A (fnk )) −1 converges and lim A(A (fn )) = lim fn = f = Aqe. Hence,q ¯ = qe. We have k→∞ k k→∞ k arrived at a contradiction, which proves the theorem. 2 Theorem 4.8 provides a way of minimizing the residual. Let A be a continuous one-to-one operator from M to A(M), where M is compact. Let {δn} be a decreasing sequence of positive numbers such that δn & 0 as n → ∞. Instead of the exact right- hand side f we take an element fδk ∈ A(M) such that ρF (f, fδk ) ≤ δk. For any n, using the trial-and-error method we can ﬁnd an element qn such that ρF (Aqn, fδ) ≤ δn. The elements qn approximate the exact solution qe to the equation Aq = f. Indeed, since A is a continuous operator, the image A(M) of the compact set M is compact. Consequently, by Theorem 4.8, the inverse map A−1 is continuous on A(M). Since

ρF (Aqn, f) ≤ ρF (Aqn, fδ) + ρF (fδ, f), the following inequality holds:

δ ρF (Aqn, f) ≤ δn + δ = γn.

−1 Hence, by the continuity of the inverse map A : A(M) → M we have ρQ(qn, qe) ≤ δ δ δ ε(γn), where ε(γn) → 0 as γn → 0. Inverse and ill-posed problems 341

Note that Theorem 4.8 implies the existence of a function ω(δ) such that 1) lim ω(δ) = 0; δ→0

2) for all q1, q2 ∈ M the inequality

ρF (Aq1, Aq2) ≤ δ

implies ρQ(q1, q2) ≤ ω(δ).

Deﬁnition 4.9. Assume that Q and F are metric spaces, M ⊂ Q is a compact set, and A : Q → F is a continuous one-to-one operator that maps M onto A(M) ⊂ F . The function ω(δ) = sup ρQ(q1, q2) q1,q2∈M ρF (Aq1,Aq2)≤δ is called the modulus of continuity of the operator A−1 on the set A(M). Given the modulus of continuity ω(δ) or a function that majorizes it, one can estimate the norm of the deviation of the exact solution from the solution corresponding to the approximate data. Indeed, let qe ∈ M be an exact solution to the problem Aq = f and qδ ∈ M be a solution to the problem Aq = fδ, where kf − fδk ≤ δ. Then kq −qδk ≤ ω(δ) if M is compact. Therefore, after proving the uniqueness theorem, the most important stage in the study of a conditionally well-posed problem is obtaining a conditional stability estimate. We conclude this section with several results of a more general character.

Theorem 4.10 (stability in topological spaces). Let Q and F be topological spaces, F be a Hausdorff space, and M ⊂ Q. Let A be a continuous one-to-one operator that maps M onto A(M). If M is a compact set, then A−1 is continuous on A(M) in the relative topology.

Deﬁnition 4.11. An operator A : Q → F is called a closed operator if its graph

G(A) = {(q, Aq), q ∈ D(A)} is closed in the product of topological spaces Q × F .

Theorem 4.12. Let Q and F be Hausdorff topological spaces satisfying the ﬁrst axiom of countability. Let A : Q → F be a closed one-to-one operator with domain of deﬁnition D(A), K ⊂ Q be a compact set, and M = D(A) ∩ K. Then A(M) is closed in F and the operator A−1 is continuous on A(M) in the relative topology.

Remark 4.13. The assertion of the theorem still holds without the assumption that Q satisﬁes the ﬁrst axiom of countability (Ivanov, 1969). 342 S. I. Kabanikhin

Remark 4.14. Theorem 4.12 generalizes Theorem 4.10 by relaxing the conditions imposed on the operator A: it is assumed to be closed, but not necessarily continuous.

Ill-posed problems can be formulated as problems of determining the value of a (generally, unbounded) operator at a point

T f = q, f ∈ F, q ∈ Q. (4.1)

If the operator A−1 exists for the problem Aq = f, then this problem is equivalent to problem (4.1). The operator A−1, however, may not exist. Moreover, in many applied problems (for example, differentiating a function, summing a series, etc.) the representation Aq = f may be inconvenient or even unachievable, although both problems can theoretically be studied using the same scheme (Ivanov et al, 2002). We now reformulate the conditions for Hadamard well-posedness for problem (4.1) as follows:

1) the operator T is deﬁned on the entire space F : D(T ) = F ;

2) T is a one-to-one map;

3) T is continuous.

Problem (4.1) is said to be ill-posed if at least one of the well-posedness conditions does not hold. In particular, the case where the third condition does not hold (the case of instability) is the most important and substantial. In this case, problem (4.1) is reduced to the problem of approximating an unbounded operator with a bounded one.

4.3. The Ivanov theorem. Quasi-solution There are other approaches to ill-posed problems that involve a generalization of the concept of a solution. Let A be a compact operator. If q is assumed to belong to a compact set M ⊂ Q and f ∈ A(M) ⊂ F , then the formula

q = A−1f can be used to construct an approximate solution that is stable with respect to small variations in f. It should be noted that the condition that f belong to A(M) is essential for the applicability of the formula q = A−1f for finding an approximate solution, since the expression A−1f may be meaningless if the said condition does not hold. However, finding out whether f belongs to A(M) is a complicated problem. Furthermore, even if f ∈ A(M), measurement errors may cause this condition to fail, i.e., fδ may not belong to A(M). To avoid the difficulties arising when the equation Aq = f has no solutions, the notion of a quasi-solution to Aq = f is introduced as a generalization of the concept of a solution to this equation. Inverse and ill-posed problems 343

Deﬁnition 4.15 (Ivanov, 1962b). A quasi-solution to the equation Aq = f on a set M ⊂ Q is an element qK ∈ M that minimizes the residual:

ρF (AqK , f) = inf ρF (Aq, f). q∈M

If M is compact, then there exists a quasi-solution for any f ∈ F . If, in addition, f ∈ A(M), then quasi-solutions qK coincide with the exact solution (qK is not necessarily unique!). We will give a sufﬁcient condition for a quasi-solution to be unique and continuously depend on the right-hand side f.

Deﬁnition 4.16. Let h be an element of a space F , and let G ⊂ F . An element g ∈ G is called a projection of h onto the set G if

ρF (h, g) = ρF (h, G) := inf ρF (h, p). p∈G

The projection g of h onto G is written as g = PGh.

Theorem 4.17. Assume that the equation Aq = f has at most one solution on a compact set M and for any f ∈ F its projection onto A(M) is unique. Then a quasi- solution to the equation Aq = f is unique and continuously depends on f. The proof of this theorem and Theorem 4.19 can be found in (Ivanov et al, 2002; Tikhonov and Arsenin, 1974). Note that all well-posedness conditions are restored when passing to quasi-solutions if the assumptions of Theorem 4.17 hold. Consequently, the problem of ﬁnding a quasi-solution on a compact set is well-posed.

Remark 4.18. If the solution of the equation Aq = f is not unique, then its quasi- solutions form a subset D of the compact set M. In this case, even without the conditions imposed on A(M) in the assumption of the theorem, the set D continuously depends on f in the sense of the continuity of multi-valued maps (Ivanov, 1963). If the operator A is linear, the theorem can be stated in the following speciﬁc form.

Theorem 4.19. (Ivanov, 1963) Let A : Q → F be a linear operator and assume that the homogeneous equation Aq = 0 has only one solution q = 0. Furthermore, assume that M is a convex and compact set and any sphere in F is strictly convex. Then a quasi-solution to the equation Aq = f on M is unique and continuously depends on f. We now consider the case where Q and F are separable Hilbert spaces. Let A : Q → F be a compact operator and

M = B(0, r) := {q ∈ Q: kqk ≤ r}.

By A∗ we denote the adjoint of the operator A. 344 S. I. Kabanikhin

It is known that A∗A is a self-adjoint positive compact operator from F into Q (positivity means that hA∗Aq, qi > 0 for all q =6 0). ∗ Let {λn} be the sequence of eigenvalues of the operator A A (in descending order), and let {ϕn} be the corresponding complete orthonormal sequence of eigenfunctions (vectors). The element A∗f can be represented as a series:

∞ ∗ X ∗ A f = fnϕn, fn = hA f, ϕ∗i. (4.2) n=1 Under these conditions, the following theorem holds (Ivanov, 1963).

Theorem 4.20. A quasi-solution to the equation Aq = f on the set B(0, r) is given by the formula  ∞ ∞ 2  X fn X fn 2  ϕn if 2 < r ,  λn λn q = n=1 n=1 K ∞ ∞ 2 X fn X f  ϕ n ≥ r2,  n if 2  λn + β λ n=1 n=1 n where β satisﬁes the equation

∞ 2 X fn 2 2 = r . (λn + β) n=1

∞ P 2 2 2 Proof. If fn/λn < r , then a quasi-solution qK that minimizes the functional n=1 2 ρF (Aq, f) := J(q) = hAq − f, Aq − fi on B(0, r) can be obtained by solving the Euler-Lagrange equation A∗Aq = A∗f. (4.3) We will seek a solution to this equation in the form of a series:

∞ X qK = qnϕn. n=1 Substituting this series into equation (4.3) and using the expansion (4.2) for A∗f, we obtain ∞ ∞ ∞ ∗ X ∗ X X A AqK = qnA Aϕn = qnλnϕn = fnϕn. n=1 n=1 n=1 ∞ ∞ X 2 2 2 X Hence, qn = fn/λn. Since fn/λn < r , qK = (fn/λn)ϕn ∈ B(0, r) minimizes n=1 n=1 the functional J(q) on B(0, r). Inverse and ill-posed problems 345

∞ X 2 2 2 On the other hand, if fn/λn ≥ r , then, taking into account that qK must belong n=1 to B(0, r), it is required to minimize the functional J(q) = hAq − f, Aq − fi on the sphere kqk2 = r2. Applying the method of Lagrange multipliers, this problem is reduced to ﬁnding the global extremum of the functional

Jα(q) = hAq − f, Aq − fi + αhq, qi.

To ﬁnd the minimum of the functional Jα, it is required to solve the corresponding Euler equation αq + A∗Aq = A∗f. (4.4)

∞ ∞ P ∗ P Substituting qK = qnϕn and A f = fnϕn into this equation, we obtain n=1 n=1 2 2 qn = fn/(α + λn). The parameter α is determined from the condition kqk = r , ∞ P 2 2 2 which is equivalent to the condition w(α) := fn/(α + λn) = r . We now take the n=1 root of the equation ω(α) = r2 for β, which completes the proof of the theorem. 2

Remark 4.21. The equation w(α) = r2 is solvable because w(0) ≥ r2 and w(α) monotonically decreases with the increase of α and vanishes as α → ∞.

4.4. The Lavrentiev method

If the approximate right hand side fδ of the equation Aq = f does not belong to A(M), then one can try to replace this equation with a similar equation

αq + Aq = fδ, α > 0 (4.5) for which the problem becomes well-posed. In what follows, we prove that in many cases this equation has a solution qαδ that tends to the exact solution qe of the equation Aq = f as α and the error δ in the approximation of f tend to zero with δ/α → 0 (Lavrentiev, 1959). Assume that Q = F , Q is separable Hilbert space, and A is a linear, compact, self- adjoint and positive (i. e., all eigenvalues of A are positive) operator. Assume that for f ∈ F there exists a qe such that Aqe = f. Then take the solution −1 qα = (αE + A) f to the equation αq + Aq = f as an approximate solution to the equation Aq = f (the existence of qα will be proved below). If the data f is approximate, i.e., instead of f we have fδ such that kf − fδk ≤ δ, then we set −1 qαδ = (αE + A) fδ. 346 S. I. Kabanikhin

−1 Equation (4.5) deﬁnes a family of regularizing operators Rα = (αE + A) , α > 0. Consider this matter in more detail. Let {ϕk} be a complete orthonormal sequence of eigenfunctions and {λk} (0 < · · · ≤ λk+1 ≤ λk ≤ · · · ≤ λ1) be the corresponding sequence of eigenvalues of the operator A. Assume that the equation

Aq = f (4.6) has a solution qe. Substituting the expansions

∞ X qe = qkϕk, qk = hqe, ϕki, k=1 (4.7) ∞ X f = fkϕk, fk = hf, ϕki, k=1 into (4.6), we conclude that qk = fk/λk and therefore

∞ X fk qe = ϕk. λk k=1

Since qe ∈ Q, the series ∞ 2 X fk (4.8) λk k=1 converges. Consider the auxiliary equation

αq + Aq = f. (4.9)

As before, a solution qα to equation (4.9) can be represented in the form

∞ X fk qα = ϕk. (4.10) α + λk k=1

Taking into account that fk = λkqk, we estimate the difference

∞ ∞ X X λkqk qe − qα = qkϕk − ϕk α + λk k=1 k=1 ∞ ∞ X λkqk X qk = qk − ϕk = α ϕk. α + λk α + λk k=1 k=1 Consequently, ∞ 2 2 2 X qk kqe − qαk = α 2 . (4.11) (α + λk) k=1 Inverse and ill-posed problems 347

It is now easy to show that lim kqe − qαk = 0. Let ε be an arbitrary positive α→+0 number. The series (4.11) is estimated from above as follows:

∞ 2 n 2 ∞ 2 2 X qk 2 X qk 2 X qk α 2 = α 2 + α 2 (α + λk) (α + λk) (α + λk) k=1 k=1 k=n+1 ∞ α2 X ≤ kqk2 + q2. (4.12) λ2 k n k=1

∞ X 2 Since the series qk converges, there is a number n such that the second term in k=1 the right-hand side of (4.12) is less than ε/2. Then we can choose α > 0 such that the ﬁrst term is also less than ε/2. It follows that lim kqe − qαk = 0. α→+0 We now consider the problem with approximate data

Aq = fδ, (4.13)

(where kf − fδk < δ) and the regularized problem (4.5):

αq + Aq = fδ.

Put fδ,k = hfδ, ϕki. Then the solution qαδ to (4.5) can be represented as a series

∞ X fδ,k qαδ = ϕk. α + λk k=1 We now estimate the difference

kqe − qαδk ≤ kqe − qαk + kqα − qαδk. (4.14)

The ﬁrst term in the right-hand side of (4.14) vanishes as α → 0. The second term can is estimated as follows:

∞ f f 2 ∞ (f − f )2 2 X k δ,k X k δ,k kqα − qαδk = − ϕk = 2 α + λk α + λk (α + λk) k=1 k=1 ∞ 1 X 1 δ2 ≤ (f − f )2 = kf − f k2 ≤ . (4.15) α2 k δ,k α2 δ α2 k=1

We now prove that qαδ → qe as α and δ tend to zero and δ/α → 0. Take an arbitrary positive number ε. Choose α such that kqe − qαk < ε/2. Then ﬁnd δ > 0 such that δ/α < ε/2. Then from (4.14) and (4.15) it follows that

kqe − qαδk ≤ kqe − qαk + kqα − qαδk ≤ ε/2 + ε/2 = ε. 348 S. I. Kabanikhin

In conclusion, we note that if A is not positive and self-adjoint, then the equation Aq = f can be reduced to an equation with a positive self-adjoint operator by applying the operator A∗: A∗Aq = A∗f. Then the regularizing operator is written as

∗ −1 ∗ Rα = (αE + A A) A .

4.5. The Tikhonov regularization method In many ill-posed problems Aq = f, the class M ⊂ Q of possible solutions is not a compact set, and measurement errors in the data f may result in the right-hand side not belonging to the class A(M) for which solutions exist. The regularization method developed by A. N. Tikhonov (see Tikhonov, 1963a, 1963b, 1964) can be used to construct approximate solutions to such problems. First, we give the general deﬁnition of a regularizing algorithm for the problem Aq = f (Vasin and Ageev, 1995). Let A be a bounded linear operator. Suppose that, instead of the operator A and the right-hand side f, we have their approximations Ah and fδ that satisfy the conditions

kA − Ahk ≤ h, kf − fδk ≤ δ

(Q and F are normed spaces). Let A be the set of admissible perturbations of A.

Deﬁnition 4.22. A family of mappings Rδh : f × A → Q is called a regularizing algorithm for the problem Aq = f if

−1 sup kRδh(fδ,Ah) − A fk → 0 kf−fδ k≤δ, kA−Ahk≤h fδ ∈F,Ah∈A as δ → 0 and h → 0 for all f ∈ R(A) = A(Q). The set {Rδh(fδ,Ah)}, δ ∈ (0, δ0], h ∈ (0, h0] is called a regularized family of approximate solutions to the problem Aq = f.

If there is any a priori information about a solution qe to the equation Aq = f, such −1 as the condition qe ∈ M ⊂ Q, then the set A f in the above deﬁnition can be replaced with A−1f ∩ M. In the sequel, in most cases we assume that the representation of the operator A is exact. Now assume that Q and F are metric spaces, A : Q → F , and qe is an exact solution to the ill-posed problem Aq = f for some f ∈ F .

Deﬁnition 4.23 (regularizing family of operators). A family of operators {Rα}α>0 is said to be regularizing for the problem Aq = f if

1) for any α > 0 the operator Rα : F → Q is continuous, Inverse and ill-posed problems 349

2) for any ε > 0 there exists an α∗ > 0 such that

ρQ(Rαf, qe) < ε

for all α ∈ (0, α∗), in other words,

lim Rαf = qe. (4.16) α→+0

If the right-hand side of the equation Aq = f is approximate and the error ρF (fδ, f) ≤ δ in the initial data is known, then the regularizing family {Rα}α>0 allows us not only to construct an approximate solution qαδ = Rαfδ, but also to estimate the deviation of the approximate solution qαδ from the exact solution qe. Indeed, using the triangle inequality, we have

ρQ(qαδ, qe) ≤ ρQ(qαδ,Rαf) + ρQ(Rαf, qe). (4.17) The second term in the right-hand side of (4.17) vanishes as α → +0. Since the problem is ill-posed, the estimation of the first term as α → +0 and δ → +0 is a difficult task which is usually solved depending on the specific character of the problem under study and on the a priori and/or a posteriori information about the exact solution. For example, consider the case where Q and F are Banach spaces, A : Q → F is a completely continuous linear operator, and Rα is a linear operator for any α > 0. As- sume that there exists a unique solution qe for f ∈ F , and fδ ∈ F is an approximation of f such that kf − fδk ≤ δ. (4.18)

We now estimate the norm of the difference between the exact solution qe and the regularized solution qαδ = Rαfδ:

kqe − qαδk ≤ kqe − Rαfk + kRαf − Rαfδk. (4.19)

Set kqe −Rαfk = γ(qe, α). The property (4.16) of a regularizing family implies that the ﬁrst term in the right-hand side of (4.19) vanishes as α → 0, i.e., lim γ(qe, α) = 0. α→0 Since Rα is linear, from the condition (4.18) it follows that

kRαf − Rαfδk ≤ kRαk δ. Recall that the norm of an operator A : Q → F for Banach spaces Q and F is determined by the formula kAqk kAk = sup . q∈Q kqk q=6 0

The norm kRαk cannot be uniformly bounded because the problem is ill-posed. Indeed, −1 otherwise we would have lim Rα = A and the problem Aq = f would be well- α→+0 posed in the classical sense. However, if α and δ tend to zero being connected in some way, then the right-hand side of the obtained estimate

kqe − qαδk ≤ γ(qe, α) + kRαkδ (4.20) 350 S. I. Kabanikhin

tends to zero. Indeed, setting ω(qe, δ) = inf {γ(qe, α) + kRαkδ}, we will show that α>0

lim ω(qe, δ) = 0. δ→0

Take an arbitrary ε > 0. Since lim γ(qe, α) = 0, there exists an α0(ε) such that for α→0 all α ∈ (0, α0(ε)) we have γ(qe, α) < ε/2.

Put µ0(ε) = inf kRαk and take δ0(ε) = ε/(2µ0(ε)). Then α∈(0,α0(ε))

inf {kRαk δ} ≤ δ inf {kRαk} ≤ ε/2. α>0 α∈(0,α0(ε)) for all δ ∈ (0, δ0(ε)). Thus, for any ε > 0 there exist α0(ε) and δ0(ε) such that

kqe − qαδk < ε for all α ∈ (0, α0(ε)) and δ ∈ (0, δ0(ε)). For speciﬁc operators A and families {Rα}α>0, an explicit formula for the relationship between the regularization parameter α and the data error δ can be derived. One of the well-known methods for constructing a regularizing family is the mini- mization of the Tikhonov functional

2 0 T (q, fδ, α) = kAq − fδk + αΩ(q − q ), where q0 is a test solution, α is the regularization parameter, and Ω is a stabilizing functional, which is usually represented by the norm (or a seminorm), for example, Ω(q) = kqk2. The stabilizing functional Ω uses the a priori information about the degree of smoothness of the exact solution (or about the solution structure) and de- termines the type of the convergence of approximate solutions to the exact one for a given relationship between α(δ) → 0 and δ → 0. For example, the Tikhonov method is Ω 2 effective for (q) = kqk 1 in the numerical solution of integral equations of the ﬁrst W2 kind that have a unique solution which is sufﬁciently smooth (see Vasin and Ageev, 1995) and the bibliography therein). First, consider a simple example T (q, f, α) = kAq − fk2 + αkqk2, α > 0. (4.21)

Theorem 4.24. Let Q and F be Hilbert spaces and A be a compact linear operator. Then for any f ∈ F and α > 0, the functional T (q, f, α) attains its greatest lower bound at a unique element qα.

Applying Theorem 4.24, we can construct an approximate solution qαδ from the approximate data fδ ∈ F satisfying the condition kf − fδk ≤ δ and prove that qαδ converges to the exact solution qe to the equation Aq = f as the parameters α and δ tend to zero at the same rate. Indeed, let the functional T (q, fδ, α) attain its greatest lower bound at a point qαδ, whose existence and uniqueness is guaranteed by Theorem 4.24. Inverse and ill-posed problems 351

Theorem 4.25. Let the assumptions of Theorem 4.24 hold. Assume that there exists a unique solution qe to the equation Aq = f for some f ∈ F . Let {fδ}δ>0 be a family of approximate data such that kf − fδk < δ for each of its elements. Then, if the regularization parameter α = α(δ) is chosen so that lim α(δ) = 0 and lim δ2/α(δ) = δ→0 δ→0 0, then the element qα(δ),δ which minimizes the regularizing functional T (q, fδ, α) tends to the exact solution qe to the equation Aq = f, i.e., lim kqα(δ),δ − qek = 0. δ→0

We now return to a more general case, where the problem Ahq = fδ is being solved instead of Aq = f.

Theorem 4.26. Let the operator Ah and the right-hand side fδ satisfy the approximation conditions kAh − Ak ≤ h, kf − fδk ≤ δ, (4.22) where h ∈ (0, h0), δ ∈ (0, δ0). Assume that A and Ah are bounded linear operators 0 from Q into F , where Q and F are Hilbert spaces. Let qN be a solution to the problem Aq = f that is normal with respect to q0 (i.e., an element that minimizes the functional 0 kq − q k on the set Qf of all solutions to the problem Aq = f). Then for all α > 0, α q0 ∈ Q, h ∈ (0, h0), and δ ∈ (δ0) there exists a unique solution qhδ to the problem 2 0 2 min {kAhq − fδk + αkq − q k , q ∈ Q}. (4.23) Furthermore, if the regularization parameter α satisﬁes the conditions (h + δ)2 lim α(∆) = 0, lim = 0, ∆→0 ∆→0 α(∆) √ where ∆ = h2 + δ2, then α 0 lim kqhδ − qN k = 0. ∆→0

5. Brief survey of deﬁnitions of inverse problems Although inverse problems are very diverse, attempts to give a universal deﬁnition of an inverse problem are of interest. We consider the several approaches.

Example 5.1 (H. W. Engl et al, 1996). When using the term inverse problem, one im- mediately is tempted to ask “inverse to what?”. Following J. B. Keller (1976), one calls two problems inverse to each other if the formulation of one problem involves the other one. For mostly historic reasons, one might call one of these problems (usually the simpler one or the one which was studied earlier) the direct problem, the other one the inverse problem. However, if there is a real-world problem behind the mathematical problem studied, there is, in most cases, a quite natural distinction between the direct and the inverse problem. E.g., if one wants to predict the future behaviour of a physical system from knowledge of its present state and the physical laws (including concrete values of all relevant physical parameters), one will call this the direct problem. Possible inverse problems are the determination of the present state of the system 352 S. I. Kabanikhin from future observations (i.e., the calculation of the evolution of the system backwards in time) or the identification of physical parameters from observations of the evolution of the system (parameter identification). There are, from the applications point of view, two different motivations for studying such inverse problems: first, one wants to know past states or parameters of a physical system. Second, one wants to find out how to influence a system via its present state or via parameters in order to steer it to a desired state in the future. Thus, one might say the inverse problems are concerned with determining causes for a desired or an observed effect.

Example 5.2 (H. Moritz, 1993). Consider the following symbolic expression

f = Aq.

It may be interpreted in several ways, for instance by the diagram

q A f input output

We have a “black box” in which for every input q (from a certain class) we get a uniquely defined output. Mathematically we say that f is a (generalized) function of q, or that A denotes the operator which acts on q and produces f. This operator may be linear or nonlinear. The principle of deterministic evolution has been given a classical formulation by Laplace in 1814 (Laplace’s demon): “An intelligent being which, for some given moment of time, knew all the forces by which nature is driven, and the relative position of the objects of which it is composed (provided the being’s intelligence were so vast as to be able to analyze all the data), would be able to comprise, in a single formula, the movements of the largest bodies in the universe and those of the lightest atom: nothing would be uncertain to it, and both the future and the past would be present to its eyes. The human mind offers in the perfection which it has been able to give to astronomy, a feeble inkling of such an intelligence.” We cannot here offer anything like a philosophical discussion of determinism or causality. We only mention that quantum theory has thoroughly shaken the foundations on which it is based, introducing a basic background of randomness. But determinism has also been under attack from its very stronghold, classical mechanics, as the following example shows. Deterministic chaos (nonlinear dynamics). It is difficult to extract one’s mind from the persuasive power of Laplacian determinism (also called the principle of causality), but the modern theory of nonlinear dynamic systems has shown that it is wrong: even classical dynamic systems are frequently unstable. This means that two trajectories which were very close together at time t0, need not be close all the time (stability) but Inverse and ill-posed problems 353 may drift apart considerably (instability). Since the initial conditions x0 and x00 can be given only subject to an error (however small) because of measurement inaccuracies, instability means that it is impossible to produce the state of the system: small causes may cause large effects, and the system may become unpredictable after a sufficiently large time t − t0. Everyone knows the case of weather prediction, and the most famous example of chaos, Edward Lorenz’ strange attractor, has been discovered by studying a simplified model of a weather forecast.

Example 5.3 (K. Chadan and P. C. Sabatier, 1989). The mathematical expression of a physical law is a rule which defines a mapping M of a set of functions S, called the parameters, into a set of functions E called the results. This rule is usually a set E of equations, in which the parameters are elements of S, the solutions are the corresponding elements of E, so the definition of M is quite involved. However, from the very definition of a mapping, the solution of E must exist and be unique in E for any element of S. This is the only constraint one has to put on E. We called “computed results” the elements of E which are thus obtained from those of S. Deriving the “computed result” for a given element of S is called “solving the direct problem”. Conversely, obtaining the subset of S that corresponds to a given element of E is called “solving the inverse problem”. To give a physical meaning to M, E must be such that its elements can be compared to the experimental results. In the following, we assume that the result of any relevant measurement is an element of a subset, E0, of E, called the set of experimental results. E therefore contains the union of the experimental and the computed results. It is also assumed that E can be given the structure of a metric space. The comparison of a given computed result, e0, and a given experimental result, e00, is then measured by the distance d(e0, e00). The set S was defined as the set of functions for which E can be solved. Stronger limitations usually come in when “physical properties” are taken into account. In other words, S could be the set of all the functions F for which E can be solved and which are consistent with all the “physical information” coming either from general principles or from previous measurements. However, the definition of F is, in most cases, indirect and difficult to make precise, and so one is led to choose S for a convenient subset of F, with a clear definition. On the other hand, attempt to enlarge the definition of sometimes gives access to new classes of “parameters” for which the direct and inverse problems can be solved. With these definitions, it may seem that all physical problems are inverse problems. Actually, one usually reserves this name for the problems where precise mathematical forms are sought for the generalized inverse mappings of E into S. This excludes the so-called “fitting procedure” in which models depending on a few parameters and giving a good fit of the experimental results are obtained by trial and error or any other technique. (Note that P. C. Sabatier gave above definitions first in 1971).

Example 5.4 (A. Tarantola, 2005). Let S be the physical system under study. For instance, S can be a galaxy for an astrophysicist, Earth for geophysicist, or a quantum particle for a quantum physicist. 354 S. I. Kabanikhin

The scientiﬁc procedure for the study of a physical system can be (rather arbitrarily) divided into the following three steps.

i) Parameterization of the system: discovery of a minimal set of model parameters whose values completely characterize the system (from a given point of view).

ii) Forward modeling: discovery of the physical laws allowing us, for given values of model parameters, to make predictions on the results of measurements on some observable parameters.

iii) Inverse modeling: use of actual results of some measurements of the observable parameters to infer the actual values of the model parameters.

String feedback exists between these steps, and a dramatic advance in one of them is usually followed by advances in the other two.

Example 5.5 (R. G. Newton — Foreword from the book by K. Chadan and P. Sabatier, 1989). The normal business of physicists may be schematically thought of as predicting the motions of particles on the basis of known forces, or the propagation of radiation on the basis of a known constitution of matter. The inverse problem is to conclude what the forces or constitutions are on the basis of the observed motion. A large part of our sensory contact with the world around us depends on an intuitive solution of such an inverse problem: We infer the shape, size, and surface texture of external objects from their scattering and absorption of light as detected by our eyes. When we use scattering experiments to learn the size or shape of particles, or the forces they exert upon each other, the nature of the problem is similar, if more refined. The kinematics, the equations of motion, are usually assumed to be known. It is the forces that are sought, and how they vary from point to point. As with so many other physical ideas, the first one we know of to have touched upon the kind of inverse problem was Lord Rayleigh (1877). In the course of describing the vibrations of strings of variable density he briefly discussed the possibility of infer- ring the density distribution from the frequencies of vibration. This passage may be regarded as a precursor of the mathematical study of inverse spectral problems some seventy years later. Its modern analogue and generalization was given in a famous lecture be Marc Kac (1966), entitled “Can one hear the shape of a drum?”

Remark 5.6. It is certainly possible to make the deﬁnition of an inverse problem even more general, since even the law described by the main equation itself is sometimes unknown. In this case, it is required to determine the law (equation) from the measurement results. The discoveries of new laws expressed in the mathematical form (equations) are brought about by a lot of experiments, reasoning, and discussions. This complex process is not exactly a process of solving an inverse problem. However, the term “inverse problems” is gaining popularity in the scientiﬁc literature on a wide variety of subjects. For example, there have been attempts to view mathematical statistics as an inverse Inverse and ill-posed problems 355 problem with respect to probability theory. There is a lot in common between the theory of inverse problems and control theory, the theory of pattern recognition, and many other areas. This is easy to see, for example, from the results of a search for “inverse problems books” on the web site www.amazon.com. At the time when this part of the paper was being written (in May, 2007) the search produced 107 titles containing the words “inverse problems”, and the reference to each of these books contained an advice to “see more references in this book”, i.e., each of the 107 books had a list of references to the relevant literature. Furthermore, a less restrictive search “inverse problems” produced 6687 titles! Most probably nobody can pretend to make the complete survey concerning inverse and ill-posed problems. Therefore the list of references is far from being complete.

Acknowledgments. The author thanks B. Hofmann, M. Kokurin, R. Kress, P. Sabatier, A. Yagola, M. Yamamoto for very fruitful discussion and remarks.

References

1. O. M. Alifanov, E. A. Artyukhin, and S. V. Rumyantsev, Extreme Methods for Solving Ill-Posed Problems with Applications to Inverse Heat Transfer Problems. Begell House Inc., New York, 1995. 2. G. Anger, Inverse Problems in Differential Equations. Akademie-Verlag, Berlin; Plenum, Lon- don, 1990. 3. K. I. Babenko, Foundations of Numerical Analysis. Regular and Chaotic Dynamics, Moscow- Izhevsk, 2002 (in Russian). 4. M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging. Institute of Physics Publishing, 1998. 5. A. S. Blagoveshchenskii, Inverse Problems of Wave Processes. VSP, Utrecht, 2001 6. K. Chadan and P. Sabatier, Inverse Problem in Quantum Scattering Theory. Springer-Verlag, 1989. 7. V. G. Chedrednichenko, Inverse Logaritlimic Potential Problem. VSP, Utrecht, 1996. 8. D. Colton and R. Kress, Inverse Acoustic and Electromagnetic Scattering Theory. Springer, Berlin, 1992. 9. A. M. Denisov, Elements of the Theory of Inverse Problems. VSP, Utrecht, 1999. 10. H. W. Engl, Inverse problems and their regularization. In: Computational Mathematics Driven by Industrial Problems. Springer, Berlin–New York, 1999, 126–150. 11. H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems. Kluwer Academic Publishers, The Netherlands, 1996. 12. C. W. Groetsch, Inverse Problems in the Mathematical Sciences. Vieweg, Braunschweig, 1993. 13. J. Hadamard, Sur les problemes` aux deriv´ ees´ partielles et leur signiﬁcation physique. In: Prince- ton University Bulletin, 1902, 49–52. 14. , Le probleme` de Cauchy et les Equations´ aux Deriv´ ees´ Partielles Lineaires´ Hyper- boliques, Lec¸ons Professees´ a` l’Universite´ Yale. Hermann, Paris, 1932. 356 S. I. Kabanikhin

15. B. Hofmann, Ill-posedness and regularization of inverse problems — a review on mathematical methods. In: The Inverse Problem. Symposium ad memoriam H. v. Helmholtz, H. Lubbig¨ (Ed). Akademie-Verlag, Berlin; VCH, Weinheim, 1995, 45–66. 16. U. M. Isakov, Inverse Problems in Partial Differential Equations. Springer, Berline–New York, 1998. 17. V. K. Ivanov, On linear problems which are not well-posed. Dokl. Akad. Nauk SSSR 145, 2 (1962), 270–272 (in Russian). 18. , On ill-posed problems. Mat. Sb. 61, 2 (1963), 211–223 (in Russian). 19. , Ill-posed problems in topological spaces. Sibirsk. Mat. Zh. 10 (1969), 1065–1074 (in Russian). 20. V. K. Ivanov, V. V. Vasin, and V. P. Tanana, Theory of Linear Ill-Posed Problems and its Appli- cations. VSP, Utrecht, 2002. 21. S. I. Kabanikhin and M. Schieck, Impact of conditional stability: Convergence rates for general linear regularization methods. J. Inv. Ill-Posed Problems 16 3 (2008), 267–282. 22. S. I. Kabanikhin and M. M. Lavrentiev (Eds), Ill-Posed and Non-Classical Problems of Mathe- matical Physics and Analysis. VSP, Utrecht, 2003. 23. S. I. Kabanikhin and A. Lorenzi, Identification Problems of Wave Phenomena. VSP, Utrecht, 1999. 24. S. I. Kabanikhin, A. D. Satybaev, and M. A. Shishlenin, Direct Methods of Solving Multidimen- sional Inverse Hyperbolic Problems. VSP, Utrecht, 2004. 25. S. I. Kabanikhin, M. A. Bektemesov, and A. T. Nurseitova, Iterative Methods of Solving Inverse and Ill-Posed Problems with Data Given on the Part of the Boundary. International Foundation of Inverse Problems, Almaty, 2006 (in Russian). 26. M. Kac, Can one hear the shape of the drum? Amer. Math. Monthly 73, 2, part II, 1–23. 27. J. Keller, Inverse problems. Amer. Math. Monthly 83 (1976), 107–118. 28. M. V. Klibanov and A. A. Timonov, Carleman Estimates for Coefficient Inverse Problems and Numerical Applications. VSP, Utrecht, 2004. 29. R. Kress, Uniqueness and numerical methods in inverse obstacle scattering. Inverse Problems in Applied Sciences — towards breakthrough. J. Physics: Conference Series 73 (2007) 012003. 30. R. Lattes and J.-L. Lions, Methode´ de Quasi Reversibilite´ et Applications. Dunod, Paris, 1967. 31. M. M. Lavrentiev, Integral equations of the first kind. Dokl. Akad. Nauk SSSR 127, 1 (1959), 31–33 (in Russian). 32. , Some Improperly Posed Problems of Mathematical Physics. SB RAS, Novosibirsk, 1962 (in Russian); Springer, Berlin, 1967 (in English). 33. M. M. Lavrentiev, V. G. Romanov, and S. P. Shishatskii, Ill-Posed Problems of Mathematical Problems. Vol. 64. AMS, Providence, 1986. 34. M. M. Lavrentiev and L. Ya. Saveliev, Operator Theory and Ill-Posed Problems. VSP, Utrecht, 2006. 35. B. M. Levitan, Inverse Sturm–Liouville Problem. VSP, Utrecht, 1987. 36. A. Lorenzi and S. I. Kabanikhin, Identification Problems of Wave Phenomena. VSP, Utrecht, 1999. 37. V. P. Maslov, The existence of a solution of an ill-posed problem is equivalent to the convergence of a regularization process. Uspekhi Mat. Nauk 23, 3 (1968), 183–184 (in Russian). 38. V. I. Maksimov. Dynamical Inverse Problems of Distributed Systems. VSP, Utrecht, 2002. Inverse and ill-posed problems 357

39. H. Moritz, General considerations regarding inverse and related problems. In: Inverse Prob- lems: Principles and Applications in Geophysics, Technology and Medicine. Akademie-Verlag, Berlin, 1993, 11–23. 40. F. Natterer, The Mathematics of Computerized Tomography. New York, Wiley & Sons, 1986. 41. V. G. Romanov, Inverse Problems of Mathematical Physics. VSP, Utrecht, 1987. 42. , Investigation Methods for Inverse Problems. VSP, Utrecht, 2002. 43. , Stability estimates in inverse problems for hyperbolic equations. Milan J. Mathematics 74 (2006), 357–385. 44. V. G. Romanov and S. I. Kabanikhin, Inverse Problems for Maxwell’s Equations. VSP, Utrecht, 1994. 45. A. A. Samarskii and P. N. Vabishchevich, Numerical Methods for Solving Inverse Problems of Mathematical Physics. VSP, Utrecht, 2007. 46. J. Sylvester and G. Uhlmann, A uniqueness theorem for an inverse boundary value problem in electrical prospecting. Comm. Pure and Appl. Math. 39 (1986), 92–112. 47. A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, 2005. 48. A. N. Tikhonov, On the stability of inverse problems. Doklady Acad. Sci. USSR 39 (1943), 176–179. 49. , On the solution of ill-posed problems and the method of regularization. Dokl. Akad. Nauk SSSR 151, 3 (1963), 501–504 (in Russian). 50. , On the regularization of ill-posed problems. Dokl. Akad. Nauk SSSR 153, 1 (1963), 49–52 (in Russian). 51. A. N. Tikhonov and V. Ya. Arsenin, Solutions of Ill-Posed Problems. Wiley, New York, 1977. 52. A. N. Tikhonov, A. V. Goncharskii, V. V. Stepanov, and A. G. Yagola, Regularizing Algorithms and A Priori Information. Nauka, Moscow, 1983 (in Russian). 53. G. Uhlmann, Developments in inverse problems since Calderon’s foundational paper. Chap- ter 19 in “Harmonic Analysis and Partial Differential Equations”. University of Chicago Press, 1999, 295–345. 54. V. V. Vasin and A. L. Ageev, Ill-posed Problems with A Priori Information. VSP, Utrecht, 1995. 55. A. G. Yagola, I. V. Kochikov, and G. M. Kuramshina, Inverse Problems of Vibrational Spec- troscopy. VSP, Utrecht, 1999. 56. V. Yurko, Method of Spectral Mappings in the Inverse Problem Theory. VSP, Utrecht, 2002.

Received March 3, 2007; revised April 3, 2008

Author information

S. I. Kabanikhin, Sobolev Institute of Mathematics, Siberian Branch of Russian Academy of Sci- ences, Acad. Koptyug prosp., 4, 630090 Novosibirsk, Russia. Email: [email protected]