Linear Vector Spaces
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Orthogonal Functions, Radial Basis Functions (RBF) and Cross-Validation (CV)
Exercise 6: Orthogonal Functions, Radial Basis Functions (RBF) and Cross-Validation (CV) CSElab Computational Science & Engineering Laboratory Outline 1. Goals 2. Theory/ Examples 3. Questions Goals ⚫ Orthonormal Basis Functions ⚫ How to construct an orthonormal basis ? ⚫ Benefit of using an orthonormal basis? ⚫ Radial Basis Functions (RBF) ⚫ Cross Validation (CV) Motivation: Orthonormal Basis Functions ⚫ Computation of interpolation coefficients can be compute intensive ⚫ Adding or removing basis functions normally requires a re-computation of the coefficients ⚫ Overcome by orthonormal basis functions Gram-Schmidt Orthonormalization 푛 ⚫ Given: A set of vectors 푣1, … , 푣푘 ⊂ 푅 푛 ⚫ Goal: Generate a set of vectors 푢1, … , 푢푘 ⊂ 푅 such that 푢1, … , 푢푘 is orthonormal and spans the same subspace as 푣1, … , 푣푘 ⚫ Click here if the animation is not playing Recap: Projection of Vectors ⚫ Notation: Denote dot product as 푎Ԧ ⋅ 푏 = 푎Ԧ, 푏 1 (bilinear form). This implies the norm 푢 = 푎Ԧ, 푎Ԧ 2 ⚫ Define the scalar projection of v onto u as 푃푢 푣 = 푢,푣 | 푢 | 푢,푣 푢 ⚫ Hence, is the part of v pointing in direction of u | 푢 | | 푢 | 푢 푢 and 푣∗ = 푣 − , 푣 is orthogonal to 푢 푢 | 푢 | Gram-Schmidt for Vectors 푛 ⚫ Given: A set of vectors 푣1, … , 푣푘 ⊂ 푅 푣1 푢1 = | 푣1 | 푣2,푢1 푢1 푢2 = 푣2 − = 푣2 − 푣2, 푢1 푢1 as 푢1 normalized 푢1 푢1 푢3 = 푣3 − 푣3, 푢1 푢1 What’s missing? Gram-Schmidt for Vectors 푛 ⚫ Given: A set of vectors 푣1, … , 푣푘 ⊂ 푅 푣1 푢1 = | 푣1 | 푣2,푢1 푢1 푢2 = 푣2 − = 푣2 − 푣2, 푢1 푢1 as 푢1 normalized 푢1 푢1 푢3 = 푣3 − 푣3, 푢1 푢1 What’s missing? 푢3is orthogonal to 푢1, but not yet to 푢2 푢3 = 푢3 − 푢3, 푢2 푢2 If the animation is not playing, please click here Gram-Schmidt for Functions ⚫ Given: A set of functions 푔1, … , 푔푘 .We want 휙1, … , 휙푘 ⚫ Bridge to Gram-Schmidt for vectors: ∞ ⚫ Define 푔푖, 푔푗 = −∞ 푔푖 푥 푔푗 푥 푑푥 ⚫ What’s the corresponding norm? Gram-Schmidt for Functions ⚫ Given: A set of functions 푔1, … , 푔푘 .We want 휙1, … , 휙푘 ⚫ Bridge to Gram-Schmidt for vectors: ∞ ⚫ 푔 , 푔 = 푔 푥 푔 푥 푑푥 . -
Anatomically Informed Basis Functions — Anatomisch Informierte Basisfunktionen
Anatomically Informed Basis Functions | Anatomisch Informierte Basisfunktionen Dissertation zur Erlangung des akademischen Grades doctor rerum naturalium (Dr.rer.nat.), genehmigt durch die Fakult¨atf¨urNaturwissenschaften der Otto{von{Guericke{Universit¨atMagdeburg von Diplom-Informatiker Stefan Kiebel geb. am 10. Juni 1968 in Wadern/Saar Gutachter: Prof. Dr. Lutz J¨ancke Prof. Dr. Hermann Hinrichs Prof. Dr. Cornelius Weiller Eingereicht am: 27. Februar 2001 Verteidigung am: 9. August 2001 Abstract In this thesis, a method is presented that incorporates anatomical information into the statistical analysis of functional neuroimaging data. Available anatomical informa- tion is used to explicitly specify spatial components within a functional volume that are assumed to carry evidence of functional activation. After estimating the activity by fitting the same spatial model to each functional volume and projecting the estimates back into voxel-space, one can proceed with a conventional time-series analysis such as statistical parametric mapping (SPM). The anatomical information used in this work comprised the reconstructed grey matter surface, derived from high-resolution T1-weighted magnetic resonance images (MRI). The spatial components specified in the model were of low spatial frequency and confined to the grey matter surface. By explaining the observed activity in terms of these components, one efficiently captures spatially smooth response components induced by underlying neuronal activations lo- calised close to or within the grey matter sheet. Effectively, the method implements a spatially variable anatomically informed deconvolution and consequently the method was named anatomically informed basis functions (AIBF). AIBF can be used for the analysis of any functional imaging modality. In this thesis it was applied to simu- lated and real functional MRI (fMRI) and positron emission tomography (PET) data. -
A Local Radial Basis Function Method for the Numerical Solution of Partial Differential Equations Maggie Elizabeth Chenoweth [email protected]
Marshall University Marshall Digital Scholar Theses, Dissertations and Capstones 1-1-2012 A Local Radial Basis Function Method for the Numerical Solution of Partial Differential Equations Maggie Elizabeth Chenoweth [email protected] Follow this and additional works at: http://mds.marshall.edu/etd Part of the Numerical Analysis and Computation Commons Recommended Citation Chenoweth, Maggie Elizabeth, "A Local Radial Basis Function Method for the Numerical Solution of Partial Differential Equations" (2012). Theses, Dissertations and Capstones. Paper 243. This Thesis is brought to you for free and open access by Marshall Digital Scholar. It has been accepted for inclusion in Theses, Dissertations and Capstones by an authorized administrator of Marshall Digital Scholar. For more information, please contact [email protected]. A LOCAL RADIAL BASIS FUNCTION METHOD FOR THE NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS Athesissubmittedto the Graduate College of Marshall University In partial fulfillment of the requirements for the degree of Master of Arts in Mathematics by Maggie Elizabeth Chenoweth Approved by Dr. Scott Sarra, Committee Chairperson Dr. Anna Mummert Dr. Carl Mummert Marshall University May 2012 Copyright by Maggie Elizabeth Chenoweth 2012 ii ACKNOWLEDGMENTS I would like to begin by expressing my sincerest appreciation to my thesis advisor, Dr. Scott Sarra. His knowledge and expertise have guided me during my research endeavors and the process of writing this thesis. Dr. Sarra has also served as my teaching mentor, and I am grateful for all of his encouragement and advice. It has been an honor to work with him. I would also like to thank the other members of my thesis committee, Dr. -
Linear Basis Function Models
Linear basis function models Linear basis function models Linear models for regression Francesco Corona FC - Fortaleza Linear basis function models Linear models for regression Linear basis function models Linear models for regression The focus so far on unsupervised learning, we turn now to supervised learning ◮ Regression The goal of regression is to predict the value of one or more continuous target variables t, given the value of a D-dimensional vector x of input variables ◮ e.g., polynomial curve fitting FC - Fortaleza Linear basis function models Linear basis function models Linear models for regression (cont.) Training data of N = 10 points, blue circles ◮ each comprising an observation of the input variable x along with the 1 corresponding target variable t t 0 The unknown function sin(2πx) is used to generate the data, green curve ◮ −1 Goal: Predict the value of t for some new value of x ◮ 0 x 1 without knowledge of the green curve The input training data x was generated by choosing values of xn, for n = 1,..., N, that are spaced uniformly in the range [0, 1] The target training data t was obtained by computing values sin(2πxn) of the function and adding a small level of Gaussian noise FC - Fortaleza Linear basis function models Linear basis function models Linear models for regression (cont.) ◮ We shall fit the data using a polynomial function of the form M 2 M j y(x, w)= w0 + w1x + w2x + · · · + wM x = wj x (1) Xj=0 ◮ M is the polynomial order, x j is x raised to the power of j ◮ Polynomial coefficients w0,..., wM are collected -
Introduction to Radial Basis Function Networks
Intro duction to Radial Basis Function Networks Mark J L Orr Centre for Cognitive Science University of Edinburgh Buccleuch Place Edinburgh EH LW Scotland April Abstract This do cumentisanintro duction to radial basis function RBF networks a typ e of articial neural network for application to problems of sup ervised learning eg regression classication and time series prediction It is now only available in PostScript an older and now unsupp orted hyp ertext ver sion maybeavailable for a while longer The do cumentwas rst published in along with a package of Matlab functions implementing the metho ds describ ed In a new do cument Recent Advances in Radial Basis Function Networks b ecame available with a second and improved version of the Matlab package mjoancedacuk wwwancedacukmjopapersintrops wwwancedacukmjointrointrohtml wwwancedacukmjosoftwarerbfzip wwwancedacukmjopapersrecadps wwwancedacukmjosoftwarerbfzip Contents Intro duction Sup ervised Learning Nonparametric Regression Classication and Time Series Prediction Linear Mo dels Radial Functions Radial Basis Function Networks Least Squares The Optimal WeightVector The Pro jection Matrix Incremental Op erations The Eective NumberofParameters Example Mo del Selection Criteria CrossValidation Generalised -
Linear Models for Regression
Machine Learning Srihari Linear Models for Regression Sargur Srihari [email protected] 1 Machine Learning Srihari Topics in Linear Regression • What is regression? – Polynomial Curve Fitting with Scalar input – Linear Basis Function Models • Maximum Likelihood and Least Squares • Stochastic Gradient Descent • Regularized Least Squares 2 Machine Learning Srihari The regression task • It is a supervised learning task • Goal of regression: – predict value of one or more target variables t – given d-dimensional vector x of input variables – With dataset of known inputs and outputs • (x1,t1), ..(xN,tN) • Where xi is an input (possibly a vector) known as the predictor • ti is the target output (or response) for case i which is real-valued – Goal is to predict t from x for some future test case • We are not trying to model the distribution of x – We dont expect predictor to be a linear function of x • So ordinary linear regression of inputs will not work • We need to allow for a nonlinear function of x • We don’t have a theory of what form this function to take 3 Machine Learning Srihari An example problem • Fifty points generated (one-dimensional problem) – With x uniform from (0,1) – y generated from formula y=sin(1+x2)+noise • Where noise has N(0,0.032) distribution • Noise-free true function and data points are as shown 4 Machine Learning Srihari Applications of Regression 1. Expected claim amount an insured person will make (used to set insurance premiums) or prediction of future prices of securities 2. Also used for algorithmic -
Linear Operators
C H A P T E R 10 Linear Operators Recall that a linear transformation T ∞ L(V) of a vector space into itself is called a (linear) operator. In this chapter we shall elaborate somewhat on the theory of operators. In so doing, we will define several important types of operators, and we will also prove some important diagonalization theorems. Much of this material is directly useful in physics and engineering as well as in mathematics. While some of this chapter overlaps with Chapter 8, we assume that the reader has studied at least Section 8.1. 10.1 LINEAR FUNCTIONALS AND ADJOINTS Recall that in Theorem 9.3 we showed that for a finite-dimensional real inner product space V, the mapping u ’ Lu = Óu, Ô was an isomorphism of V onto V*. This mapping had the property that Lauv = Óau, vÔ = aÓu, vÔ = aLuv, and hence Lau = aLu for all u ∞ V and a ∞ ®. However, if V is a complex space with a Hermitian inner product, then Lauv = Óau, vÔ = a*Óu, vÔ = a*Luv, and hence Lau = a*Lu which is not even linear (this was the definition of an anti- linear (or conjugate linear) transformation given in Section 9.2). Fortunately, there is a closely related result that holds even for complex vector spaces. Let V be finite-dimensional over ç, and assume that V has an inner prod- uct Ó , Ô defined on it (this is just a positive definite Hermitian form on V). Thus for any X, Y ∞ V we have ÓX, YÔ ∞ ç. -
Radial Basis Functions
Acta Numerica (2000), pp. 1–38 c Cambridge University Press, 2000 Radial basis functions M. D. Buhmann Mathematical Institute, Justus Liebig University, 35392 Giessen, Germany E-mail: [email protected] Radial basis function methods are modern ways to approximate multivariate functions, especially in the absence of grid data. They have been known, tested and analysed for several years now and many positive properties have been identified. This paper gives a selective but up-to-date survey of several recent developments that explains their usefulness from the theoretical point of view and contributes useful new classes of radial basis function. We consider particularly the new results on convergence rates of interpolation with radial basis functions, as well as some of the various achievements on approximation on spheres, and the efficient numerical computation of interpolants for very large sets of data. Several examples of useful applications are stated at the end of the paper. CONTENTS 1 Introduction 1 2 Convergence rates 5 3 Compact support 11 4 Iterative methods for implementation 16 5 Interpolation on spheres 25 6 Applications 28 References 34 1. Introduction There is a multitude of ways to approximate a function of many variables: multivariate polynomials, splines, tensor product methods, local methods and global methods. All of these approaches have many advantages and some disadvantages, but if the dimensionality of the problem (the number of variables) is large, which is often the case in many applications from stat- istics to neural networks, our choice of methods is greatly reduced, unless 2 M. D. Buhmann we resort solely to tensor product methods. -
On Some Extensions of Bernstein's Inequality for Self-Adjoint Operators
On Some Extensions of Bernstein's Inequality for Self-Adjoint Operators Stanislav Minsker∗ e-mail: [email protected] Abstract: We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statis- tics, signal processing and theoretical computer science. The main feature of our bounds is that, unlike the majority of previous related results, they do not depend on the dimension d of the ambient space. Instead, the dimension factor is replaced by the “effective rank" associated with the underlying distribution that is bounded from above by d. In particular, this makes an extension to the infinite-dimensional setting possible. Our inequalities refine earlier results in this direction obtained by D. Hsu, S. M. Kakade and T. Zhang. Keywords and phrases: Bernstein's inequality, Random Matrix, Effective rank, Concen- tration inequality, Large deviations. 1. Introduction Theoretical analysis of many problems in applied mathematics can often be reduced to problems about random matrices. Examples include numerical linear algebra (randomized matrix decom- positions, approximate matrix multiplication), statistics and machine learning (low-rank matrix recovery, covariance estimation), mathematical signal processing, among others. n P Often, resulting questions can be reduced to estimates for the expectation E Xi − EXi or i=1 n P n probability P Xi − EXi > t , where fXigi=1 is a finite sequence of random matrices and i=1 k · k is the operator norm. Some of the early work on the subject includes the papers on noncom- mutative Khintchine inequalities [LPP91, LP86]; these tools were used by M. -
MATRICES WHOSE HERMITIAN PART IS POSITIVE DEFINITE Thesis by Charles Royal Johnson in Partial Fulfillment of the Requirements Fo
MATRICES WHOSE HERMITIAN PART IS POSITIVE DEFINITE Thesis by Charles Royal Johnson In Partial Fulfillment of the Requirements For the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 1972 (Submitted March 31, 1972) ii ACKNOWLEDGMENTS I am most thankful to my adviser Professor Olga Taus sky Todd for the inspiration she gave me during my graduate career as well as the painstaking time and effort she lent to this thesis. I am also particularly grateful to Professor Charles De Prima and Professor Ky Fan for the helpful discussions of my work which I had with them at various times. For their financial support of my graduate tenure I wish to thank the National Science Foundation and Ford Foundation as well as the California Institute of Technology. It has been important to me that Caltech has been a most pleasant place to work. I have enjoyed living with the men of Fleming House for two years, and in the Department of Mathematics the faculty members have always been generous with their time and the secretaries pleasant to work around. iii ABSTRACT We are concerned with the class Iln of n><n complex matrices A for which the Hermitian part H(A) = A2A * is positive definite. Various connections are established with other classes such as the stable, D-stable and dominant diagonal matrices. For instance it is proved that if there exist positive diagonal matrices D, E such that DAE is either row dominant or column dominant and has positive diag onal entries, then there is a positive diagonal F such that FA E Iln. -
ECE 487 Lecture 3 : Foundations of Quantum Mechanics II
ECE 487 Lecture 12 : Tools to Understand Nanotechnology III Class Outline: •Inverse Operator •Unitary Operators •Hermitian Operators Things you should know when you leave… Key Questions • What is an inverse operator? • What is the definition of a unitary operator? • What is a Hermetian operator and what do we use them for? M. J. Gilbert ECE 487 – Lecture 1 2 02/24/1 1 Tools to Understand Nanotechnology- III Let’s begin with a problem… Prove that the sum of the modulus squared of the matrix elements of a linear operator  is independent of the complete orthonormal basis used to represent the operator. M. J. Gilbert ECE 487 – Lecture 1 2 02/24/1 1 Tools to Understand Nanotechnology- III Last time we considered operators and how to represent them… •Most of the time we spent trying to understand the most simple of the operator classes we consider…the identity operator. •But he was only one of four operator classes to consider. 1. The identity operator…important for operator algebra. 2. Inverse operators…finding these often solves a physical problem mathematically and the are also important in operator algebra. 3. Unitary operators…these are very useful for changing the basis for representing the vectors and describing the evolution of quantum mechanical systems. 4. Hermitian operators…these are used to represent measurable quantities in quantum mechanics and they have some very powerful mathematical properties. M. J. Gilbert ECE 487 – Lecture 1 2 02/24/1 1 Tools to Understand Nanotechnology- III So, let’s move on to start discussing the inverse operator… Consider an operator  operating on some arbitrary function |f>. -
An Introduction to Functional Analysis for Science and Engineering David A
An introduction to functional analysis for science and engineering David A. B. Miller Stanford University This article is a tutorial introduction to the functional analysis mathematics needed in many physical problems, such as in waves in continuous media. This mathematics takes us beyond that of finite matrices, allowing us to work meaningfully with infinite sets of continuous functions. It resolves important issues, such as whether, why and how we can practically reduce such problems to finite matrix approximations. This branch of mathematics is well developed and its results are widely used in many areas of science and engineering. It is, however, difficult to find a readable introduction that both is deep enough to give a solid and reliable grounding but yet is efficient and comprehensible. To keep this introduction accessible and compact, I have selected only the topics necessary for the most important results, but the argument is mathematically complete and self-contained. Specifically, the article starts from elementary ideas of sets and sequences of real numbers. It then develops spaces of vectors or functions, introducing the concepts of norms and metrics that allow us to consider the idea of convergence of vectors and of functions. Adding the inner product, it introduces Hilbert spaces, and proceeds to the key forms of operators that map vectors or functions to other vectors or functions in the same or a different Hilbert space. This leads to the central concept of compact operators, which allows us to resolve many difficulties of working with infinite sets of vectors or functions. We then introduce Hilbert-Schmidt operators, which are compact operators encountered extensively in physical problems, such as those involving waves.