Appendix A Vector and Matrix Norms The lengths of n-element vectors can be characterized by the introduction of their norms. A vector-variable, real valued function x →||x|| is called a norm of vector x, if it satisfies the following properties: (a) ||x|| 0 and ||x|| = 0 if and only if x is the zero vector; (b) ||αx|| = |α|·||x|| for all vectors x and real numbers α; (c) ||x + y|| ||x|| + ||y|| for all vectors x and y. The last property is known as the triangle inequality. The most frequently used vector norms are given as follows: n ||x||1 = |xi |, i=1 where the ith element of vector x is xi ; n || || = 2 x 2 xi i=1 which is called the Euclidean norm; and ||x||∞ = max |xi | i which is called the maximum norm. The topology of n-element vector spaces can be easily developed based on vector norms. The distance of vectors x and y is the norm of their difference, ||x − y||. ∗ A sequence of vectors xk converges to a vector x if with some vector norm || − ∗|| → xk x 0 © Springer Japan 2016 231 A. Matsumoto and F. Szidarovszky, Game Theory and Its Applications, DOI 10.1007/978-4-431-54786-0 232 Appendix A: Vector and Matrix Norms as k →∞. This definition does not depend on the choice of a particular norm, since the norms of n-element vectors are equivalent to each other, that is, if ||·||a and ||·||b are two vector norms then there are positive constants K1 and K2 such that for all n-element vectors x, K1||x||a ||x||b K2||x||a. Similar to the usual notation R of the set of real numbers, we use the notation Rn for the set of all n-element real vectors. An open ball with center x ∈ Rn and radius r is defined as B(x, r) = y|y ∈ Rn, ||x − y|| < r , and the corresponding closed ball is given similarly as B(x, r) = y|y ∈ Rn, ||x − y|| r . Let D ⊆ Rn be any set of n-element vectors. A point x ∈ D is called interior,if there is an r > 0 such that B(x; r) ⊆ D that is, set D contains point x and an open ball centered at x. A point x ∈ Rn is a boundary point of D if every ball B(x, r) with r > 0 contains infinitely many points of D and also infinitely many points that do not belong to D. Set D is called open if its every point is interior. Set D is called closed if it contains its all boundary points. Clearly, the complement of an open set is closed, while the complement of a closed set is open. AsetD ⊆ Rn is called bounded, if there is a constant K such that ||x|| K for all x ∈ D. Since the norms are equivalent, this definition does not depend on the norm selection. The closed and bounded subsets of Rn are called compact. An important property of finite dimensional vector spaces is the following, which is known as the Bolzano-Weierstrass theorem: { } Let xk be a bounded infinite sequence of n-element vectors. Then it has a con- vergent subsequence. In one dimension for sequences of real numbers this statement can be proved easily. Its n-dimensional extension can be shown by selecting a subse- quence where the first component is convergent. Then taking a subsequence of this where the second component converges, do the same with the third component, and so on. This very important property of finite dimensional vector spaces is the basis for proving many other results. Assume next that D is compact in Rn and let f : D → R be a vector-variable, real-valued continuous function. Then there is a point x∗ ∈ D such that ∗ f (x ) = max f (x)|x ∈ D . Appendix A: Vector and Matrix Norms 233 In other words, a continuous function on a compact set D reaches its maximal value on D. Since − f is also continuous, function f also reaches its minimal value on D. This property is known as the Weierstrass theorem. Assume again that D is compact in Rn, and a vector x ∈ Rn does not belong to D. Then there is a point y ∈ D such that ||x − y|| = min ||x − z|| | z ∈ D , that is, there is a point y ∈ D with minimum distance from x. In examining the structure of vector spaces linear mappings x → A x are often examined where x ∈ Rn, and A is an nth order square matrix. We are often interested in how large the image of a given vector can be. For this reason we introduce matrix norms in the following way. Let || · || be a given vector norm in Rn, and compute the quantity max ||Ax|||||x|| = 1 which shows the largest norm of the images of the points from the unit ball with respect to the selected vector norm. This quantity is considered as the norm of matrix A, ||A||, generated from (or associated to) the vector norm || · ||. It can be shown that any such matrix norm satisfies the following properties: (a) ||A|| 0 and ||A|| = 0 if and only if A =0; (b) ||α A|| = |α|·||A|| for all n × n matrices and real numbers α; (c) ||A + B|| ||A|| + ||B|| for all n × n matrices A and B; (d) ||A B|| ||A|| · ||B|| for all n × n matrices A and B; (e) ||A x|| ||A|| · ||x|| for all n × n matrices A and n-element vectors x, where the matrix norm is generated from the vector norm being on both sides of the inequality. Let aij denote the (i, j) element of matrix A, then the matrix norms generated by vector norms || · ||1, || · ||2 and || · ||∞ are as follows: n ||A||1 = max |aij| (column norm) i=1 || || = λ A 2 max AT A (Euclidean norm|) λ T where AT A denotes the eigenvalues of matrix A A. Notice that this matrix is positive semidefinite with nonnegative eigenvalues, and n ||A||∞ = max |aij| (row norm). j=1 234 Appendix A: Vector and Matrix Norms In addition to these matrix norms, the Frobenius matrix norm has a certain impor- tance in applications: n n || || = 2. A F aij i=1 j=1 It is easy to show that it satisfies properties (a)–(d) of matrix norms and satisfies property (e) with the vector norm ||. ||2, that is, for all n×n matrices A and n-element vectors x, ||Ax||2 ||A||F ·||x||2. We mention here√ that the Frobenius norm cannot be generated from any vector norm, since ||I||F = n and with any matrix norm generated from a vector norm, ||I|| = max ||I x|| | ||x|| = 1 = 1. Let A be an n × n real matrix and λ one of its eigenvalues. Then |λ| || A || with any matrix norm. In examining the stability of discrete dynamic systems the order of magnitude of the eigenvalues plays an important role. Matrix norms can provide a simple bound. More details and proofs of the facts discussed above can be found, for example, in Szidarovszky and Molnàr (2002). Appendix B Convexity, Concavity Let D ⊆ Rn be an arbitrary set. We say that D is convex, if for all x, y ∈ D and 0 α 1, the point αx + (1 − α)y also belongs to D. That is, with any two points a convex set also contains the linear segment between the two points. Clearly, any intersection of convex sets is also convex, but the union of convex sets is not necessarily convex. Assume next that D ⊆ Rn is a convex set. A real-valued function f : D → R defined on D is called convex (Fig. B.1), if for all x, y ∈ D and 0 α 1, f αx + (1 − α)y α f (x) + (1 − α) f (y). (B.1) Function f : D → R with D being a convex set is called strictly convex,iffor all x, y ∈ D, x = y and 0 <α<1, f αx + (1 − α)y <αf (x) + (1 − α) f (y). (B.2) A function is called (strictly) concave if— f is (strictly) convex, that is, in (B.1) or (B.2) the opposite inequality direction holds. Assume next that f : D → R is convex and differentiable. Then from (B.1), f y + α(x − y) − f (y) f (x) − f (y) α and if α → 0, then the left-hand side converges to the derivative of f x + α(x − y) with respect to α, therefore f (y)(x − y) f (x) − f (y) (B.3) and by interchanging x and y, f (x)(y − x) f (y) − f (x) (B.4) © Springer Japan 2016 235 A. Matsumoto and F. Szidarovszky, Game Theory and Its Applications, DOI 10.1007/978-4-431-54786-0 236 Appendix B: Convexity, Concavity Fig. B.1 Convex function α f (x)+(1 − α) f (y) f (αx +(1 − α)y) x z = αx +(1 − α)y y where f is the gradient vector of f the components of which are the partial deriv- atives of f with respect to its variables. Note that for concave functions (B.3) and (B.4) hold with opposite inequality directions. In minimizing convex functions an important fact is the following. Let D ⊆ Rn be a closed convex set and function f : D → R a continuous convex function. Define the minimum set of f as follows: fmin = x|x ∈ D, f (x) f (y) for all y ∈ D , then fmin is either empty or a closed, convex set.