Note on Mathematics of Imaging Haocheng Dai 1 Vector Space Definition 1. Vector space is a set of elements called vectors together with two operations: addition and scalar multiplication, which are assumed to satisfy the following axioms: 1. x + y = y + x 2.( x + y) + z = x + (y + z) 3. There is a null vector θ 2 X such that x + θ = x for every x 2 X 4. α(x + y) = αx + αy;(α + β)x = αx + βx 5.( αβ)x = α(βx) 6.0 x = θ; 1x = x Definition 2. Inner product [1] h·; ·i : X × X ! R is a mapping that satisfies the following axioms: 1. hx; yi = hy; xi 2. hx + y; zi = hx; zi + hy; zi 3. hλx, yi = λhx; yi 4. hx; xi ≥ 0 and hx; xi = 0 if and only if x = θ where X is a vector space. Example 1. For A; B 2 GL(n), the inner product hA; Bi = Tr(AT B) and the associated norm T 1=2 kAk2 = Tr(A A) . Definition 3. Norm k · k : X × X ! R is a mapping that satisfies the following axioms: 1. kxk ≥ 0 for all x 2 X, kxk = 0 if and only in x = θ 2. kx + yk ≤ kxk + kyk for each x; y 2 X. Triangle inequality 3. kαxk = jαj · kxk for all scalars α and each x 2 X where X is a vector space. 1 Remark 1. Norm and inner product are two independent concepts. Norm is not necessarily defined by inner product. But when the Banach space's norm is defined by inner product, then it is called Hilbert space. Example 2. The norm is clearly an abstraction of our usual concept of length. For continuous situation, the supremum norm kfk1 is the supremum (lowest upper bound) of all elements of its domain evaluated in f. For discrete situation, the sup norm equals to the maximum of absolute values of its components, namely kfk1 = max jfij. n Remark 2. If f : R ! R; f(x) = kxkp; p ≥ 1, then f is convex. Figure 1: Image of 2D norm p Definition 4. l space consists of all sequences of scalars fξ1; ξ2; · · · g for which 1 X p jξij < 1 i=1 where 1 ≤ p < 1. p The norm of an element x = fξig in l is defined as 1 !1=p X p kxkp = jξij i=1 Definition 5. Lp[a; b] space (Lebesgue sapce) consists of all functions f(u) for which Z b jf(u)jpdu < 1 a where 1 ≤ p < 1. The norm of an element f(u) in Lp is defined as 1=p Z b ! p kfkp = jf(u)j du a The Lp-functions are the functions for which this integral converges. For p 6= 2, the space of Lp-functions is a Banach space which is not a Hilbert space. Remark 3. Always remember the absolute value sign in norm calculation. Remark 4. Lp space is a space of measurable functions for which the p-th power of the absolute value is Lebesgue integrable. 2 Figure 2: Visualization of kxkp = 1, which are the cross sections of Figure 1. Definition 6. Lebesgue integral of a function f over a measure space X is written Z fdµ X to emphasize that the integral is taken with respect to the measure µ. Remark 5. In mathematics, the integral of a non-negative function of a single variable can be regarded, in the simplest case, as the area between the graph of that function and the x-axis. The Lebesgue integral extends the integral to a larger class of functions. Figure 3: Riemann-Darboux's integration (in blue) and Lebesgue integration (in red). Definition 7. Normed linear vector space is a vector space X on which there is defined a real- valued function which maps each element x in X into a real number kxk. Definition 8. Pre-Hilbert space is a linear vector space X together with an inner product defined on X × X. Definition 9. Cauchy sequence is a sequence fxng in a normed space such that kxn − xmk ! 0 as n; m ! 1. Remark 6. In a normed space, every convergent sequence is a Cauchy sequence, however, a Cauchy sequence may not be convergent. 3 Figure 4: Example of Cauchy sequence Definition 10. A normed linear vector space X is complete is every Cauchy sequence from X has a limit in X. The limit is also a vector. Definition 11. Banach space is a complete normed linear vector space. Definition 12. Hilbert space is a complete pre-Hilbert space or a Banach space whose norm is defined by inner product. Remark 7. Actually, the hypothesis of completness is weak, it is always possible to complete a space X endowed with a inner product. The \completed" norm is then associated with the inner product. Remark 8. Hilbert spaces are complete infinitedimensional spaces in which distances and angles can be measured. These spaces have a major impact in analysis and topology and will provide a convenient and proper setting for the functional analysis of partial differential equations. Theorem 1. Holder Inequality. If p and q are positive numbers 1 ≤ p ≤ 1; 1 ≤ q ≤ 1, such that 1=p + 1=q = 1 and if x = fξ1; ξ2; · · · g 2 lp; y = fη1; η2; · · · g 2 lq, then 1 X jξiηij ≤ kxkp · kykq i=1 1=q 1=p Equality holds if and only if jξij = jηij for each i. kxkp kykq Theorem 2. Cauchy-Schwarz Inequality. If p = 2 and q = 2 and if x = fξ1; ξ2; · · · g 2 l2; y = fη1; η2; · · · g 2 l2, then 1 X jξiηij ≤ kxk2 · kyk2 i=1 Theorem 3. Minkowski Inequality. If x and y are in lp, 1 ≤ p ≤ 1, then kx + ykp ≤ kxkp + kykp Equality holds if and only if k1x = k2y for some positive constants k1 and k2. Theorem 4. Divergence Theorem. Letting ' be a C1 vector field, defined on Ω, which is a region in the plane with boundary @Ω, then Z Z div'dx = h'; Nidl Ω @Ω where N is the outward normal to Ω and div(') = trace(D'). 4 Definition 13. Sobolev space W k;p(R) for 1 ≤ p ≤ 1 in one-dimensional case is defined as the subset of functions f in Lp(R) such that f and its weak derivatives1 up to order k have a finite Lp norm. 1 k ! p Z p X (i) kfkk;p = f (t) dt : i=0 Remark 9. In the one-dimensional problem it is enough to assume that the (k−1)-th derivative f (k−1) is differentiable almost everywhere and is equal almost everywhere to the Lebesgue integral of its derivative (this excludes irrelevant examples such as Cantor's function). Example 3. Sobolev spaces with p = 2 are especially important because of their connection with Fourier series and because they form a Hilbert space. A special notation has arisen to cover this case, since the space is a Hilbert space: Hk = W k;2: Thereby, the frequently occurring H1 denotes the Sobolev space is constituted by the functions f such that its first derivative have a finite L2 norm. Example 4. The space Hk can be defined naturally in terms of Fourier series whose coefficients decay sufficiently rapidly, namely, ( 1 ) 2 k 2 X 2 4 2k H (T) = f 2 L (T): 1 + n + n + ··· + n fb(n) < 1 n=−∞ where fb is the Fourier series of f, and T denotes the 1-torus. As above, one can use the equivalent norm 1 2 2 X 2k kfkk;2 = 1 + jnj fb(n) : n=−∞ Remark 10. Overview of several spaces: Spaces Elements Operations Equivalents Vector vectors x + y; αx Pre-Hilbert vectors x + y; αx, h·; ·i vector space + h·; ·i Banach vectors x + y; αx; k · k vector space(complete) +k · k x + y; αx, Hilbert vectors vector space(complete)+ h·; ·i + k · k k · k defined by h·; ·i functions(vectors) Lebesgue x + y; αx; k · k Banach space s.t. kfkp < 1, p 2 [1; 1) functions(vectors) H1 Sobolev x + y; αx; k · k Banach space s.t. f 0 has L2 norm 1A weak derivative is a generalization of the concept of the derivative of a function (strong derivative) for functions not assumed differentiable, but only integrable to lie in the Lp space. 5 Definition 14. Euler-Lagrange equation is defined as @F @ @F − = 0: @y @x @y0 which is used to find a y = f(x) making this integral Z x2 L(y) = F (x; y; y0)dx x1 stationary. Example 5. Suppose A and B are two points in a Euclidean space. We want to find the geodesic between A and B. Solution. We would like to minimize Z B L = 1ds, where ds = p(dx)2 + (dy)2 = p1 + (y0)2dx A which can be written in another form Z B L = p1 + (y0)2dx A We need to find a y(x) which minimize L, where F = p1 + (y0)2 Substituting it into the Euler-Lagrange equation, we have @F d @F − = 0 @y dx @y0 ! d y0 − = 0 dx p1 + (y0)2 y0 = c p1 + (y0)2 c2 (y0)2 = 1 − c2 0 y = c1 y = c1x + c2 Theorem 5. The property of a Green's function can be exploited to solve differential equations of the form Lu(x) = f(x); where L and f(x) are given. If the kernel of L is non-trivial, then the Green's function is not unique. A Green's function, G(x; s) of a linear differential operator L = L(x) at point s, is any solution of LG(x; s) = δ(s − x); 6 Operator L(f + g) = L(f) + L(g) L(tf) = tL(f) d(f+g) df dg d(tf) df Differential dx = dx + dx dx = t dx Integral R (f + g)dx = R fdx + R gdx R (tf)dx = t R fdx Gradient r(f + g) = rf + rg r(tf) = trf Fourier F(f + g) = Ff + Fg F(tf) = tFf Laplacian ∆(f + g) = ∆f + ∆g ∆(tf) = t∆f Expectation E(f + g) = E(f) + E(g) E(tf) = tE(f) where δ is the Dirac delta function.