DOCSLIB.ORG

A1 Time-Frequency Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A1 Time-Frequency Analysis A1 2011 1 / 1 A1 Time-Frequency Analysis David Murray [email protected] www.robots.ox.ac.uk/∼dwm/Courses/2TF Hilary 2011 A1 2011 2 / 1 Course Content 1 From Signals to Complex Fourier Series 2 From Complex Fourier Series to the Fourier Transform 3 Convolution. The impulse response and transfer functions 4 Sampling, Aliasing 5 Power & Energy Spectra, Autocorrelation, and Spectral Densities 6 Random Processes and Signals A1 2011 3 / 1 Introduction In this lecture we will described the mathematic operation of the convolution of two continuous functions f (t) ∗ g(t) Two functions are blended or folded together. We then discuss the impulse response h(t) of a system, and show how it is related to the transfer function H(s) of the system. First we define a the δ-function or unit impulse. Like the Heaviside step function u(t), it is a generalized function or “distribution”, and is best defined by considering another function in conjunction with it. A1 2011 4 / 1 The δ-function (1) Consider a function g(t) 1/ 1=w 0− < t < w w g(t) = 0 otherwise 0 w + As w becomes very small, w ! 0 , δ (t) the function g(t) turns into a δ-function δ(t) indicated by the arrowed spike. 0 0− is a infinitesimally small amount less than zero, and 0+ is a infinitesimally small amount more than zero. A1 2011 5 / 1 The δ-function (2) One thing of note about g(t) is that g(t) Z w 1/ w g(t)dt = 1: 0− This property is retained as w ! 0+, 0 w so that δ Z 1 Z 0+ (t) δ(t)dt = δ(t)dt = 1 −∞ 0− Akin to the derivative of the Heaviside unit step 0 Z t δ(t)dt = u(t) : −∞ A1 2011 6 / 1 The δ-function (3) More formally the δ−function is δ defined in association with another f(t) (t) arbitrary function f (t), as Z 1 f (t)δ(t)dt = f (0) : 0 −∞ Picking out values of a function is δ (t−τ ) called sifting of f (t) by δ(t). We can also see that f(t) Z 1 f (t)δ(t − τ)dt = f (τ) t −∞ 0 τ How? Subst p = t − τ: then Z 1 f (p + τ)δ(p)dp = f (0 + τ) = f (τ) −∞ A1 2011 7 / 1 The δ-function (4) Although the δ-function is infinitely A δ (t−τ ) high, very often you will see a described as the unit δ-function This is to denote a delta-function t where τ Z 1 0 δ(t)dt = 1 −∞ If you see a δ-function spike with an amplitude A by it this denotes Z 1 Aδ(t)dt = A −∞ A1 2011 8 / 1 Properties of the δ-function #1 Fourier transform of the delta function: FT [δ(t)] = 1 Proof: Use the definition of the δ-function and sift the function f (t) = e−i!t : Z 1 δ(t)e−i!t dt = e−i!0 = 1 : −∞ NB!! Check the minus sign is in the notes ... This make some sense. We know from FS that describing abrupt changes requires many terms in the series — i.e. the high frequencies are important. The δ-function is the mother of all abrupt changes. A1 2011 9 / 1 Properties of the δ-function #2 Symmetry: The δ-function has even symmetry, δ(t) = δ(−t) Proof: Consider sifting an even function E(t) with E(0)=0 and substitute p = (−t) Z 1 E(t)δ(t)dt = E(0) −∞ Z −∞ ) − E(−p)δ(−p)dp = E(0) 1 Z 1 ) E(p)δ(−p)dp = E(0) −∞ Z 1 ) E(t)δ(−t)dt = E(0) −∞ So δ(−t) = δ(t) and δ(t) is even. A1 2011 10 / 1 Properties of the δ-function #3 Parameter Scaling: 1 δ(at) = δ(t) jaj R 1 Proof: Use the fundamental sifting definition −∞ f (t)δ(t)dt = f (0) The jaj again hints dealing with things in two stages ... If a ≥ 0 substitute (at) for t (no change in limits) Z 1 Z 1 f (at)δ(at)d(at) = a f (at)δ(at)dt = f (0) −∞ −∞ Z 1 But f (at)δ(t)dt = f (0) )a δ(at) = δ(t): −∞ A1 2011 11 / 1 Properties of the δ-function #3 /ctd Proof: continued Now if a < 0 substitute (at) for t (but need to change limits as a negative) Z −∞ Z 1 f (at)δ(at)d(at) = −a f (at)δ(at)dt = f (0) 1 −∞ Z 1 But f (at)δ(t)dt = f (0) ) − a δ(at) = δ(t): −∞ So to cover both cases: 1 δ(at) = δ(t) jaj A1 2011 12 / 1 Fourier Transforms that involve the δ-function Fourier Transform of ei!0t Z 1 FT ei!0t = ei(!0−!)t dt −∞ = 2πδ(! − !0) : How could you prove this? Fourier Transform of ein!0t Z 1 FT ein!0t = ei(n!0−!)t dt −∞ = 2πδ(! − n!0) : A1 2011 13 / 1 Fourier Transforms that involve the δ-function Fourier Transform of 1 FT [1] = 2πδ(!) : Proof Either i! t put !0 = 0 into FT e 0 = 2πδ(! − !0) or use the dual property FT [δ(t)] = 1 )FT [1] = 2πδ(−!) )FT [1] = 2πδ(!) as δ is an even function. A1 2011 14 / 1 Fourier Transforms that involve the δ-function Fourier Transform of cos !0t 1 FT [cos ! t] = FT ei!0t + e−i!0t 0 2 = π (δ(! − !0) + δ(! + !0)) : Fourier Transform of sin !0t 1 FT [sin ! t] = FT ei!0t − e−i!0t 0 2i = −iπ (δ(! − !0) − δ(! + !0)) : A1 2011 15 / 1 Fourier Transforms that involve the δ-function Fourier Transform of Complex Fourier Series " n=1 # n=1 X in!0t X FT Cne = 2π Cnδ(! − n!0) : n=−∞ n=−∞ If you are wondering ... ... yes, this will prove very useful! A1 2011 16 / 1 Convolution (1) This is a key technique in signal processing The convolution of continuous functions f (t) and g(t) is defined by an integral The convolution integral. The value of f ∗ g at t is Z 1 f ∗ g (t) = f (τ)g(t − τ)dτ −∞ where τ is a dummy variable. Note that one function is “reversed” ... But the process is commutative, so it doesn’t matter which: Z 1 Z 1 (f ∗g)(t) ≡ (g ∗f )(t) ) f (τ)g(t −τ)dτ ≡ f (t −τ)g(τ)dτ −∞ −∞ A1 2011 17 / 1 | Convolution Example #1 [Q] Find and sketch the convolution of f (t) = u(t)e−at with g(t) = u(t)e−bt , where both a and b are positive. [A] The “unhelpful” answer must be R 1 −aτ −b(t−τ) f ∗ g = −∞ u(τ)e u(t − τ)e dτ : f(τ ) g(τ ) To make sense of this... Make sketches of the functions f (τ) and g(t − τ) as τ τ τ varies. 0 0 Function f (τ) looks just like g(−τ ) g(t−τ ) f (t) of course. But g(t − τ) is a reflected (“time reversed”) and shifted version of g(t). τ τ 0 0 t A1 2011 18 / 1 | Convolution Example #1 Now multiply the two τ ) τ ) functions g(t−τ ) f( g(t−τ ) f( BUT we must worry τ τ about the fact that t is t 0 0 t a variable. f(τ )g(t−τ ) f(τ )g(t−τ ) t <0 t >0 In this case there are two different regimes, τ τ one when t < 0 and 0 t 0 t the other when t ≥ 0. A1 2011 19 / 1 | Convolution Example #1 For t < 0, the function to be integrated is everywhere zero, and the result is zero. g(t−τ ) f(τ ) For t ≥ 0 1 Z τ u(τ)e−aτ u(t − τ)e−b(t−τ)dτ 0 −∞ t t Z τ ) = e−bt e(b−a)τ dτ f(τ )g(t− 0 t >0 e−bt = e(b−a)t − 1 b − a τ 0 t So e−at − e−bt =(b − a) for t ≥ 0 f ∗g(t) = 0 for t < 0 A1 2011 20 / 1 | Convolution Example #1 Important to realize that the convolution function is the integral evaluated for the complete range of t values. e−at − e−bt =(b − a) for t ≥ 0 f ∗ g(t) = 0 for t < 0 This shows the t > 0 part of the convolution for b = 2 and a = 1. 0.25 (exp(-x)-exp(-2*x)) 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 A1 2011 21 / 1 | Convolution Example #2 [Q] Derive an expression for the convolution of an arbitrary signal f with the function g shown in the figure. Determine the convolution when f (t) = A, a constant, and when f (t) = A + (B − A) u(t). g(t) f(t) g(t− τ ) f(t) 1 a t 1 −a t t−a t τ −1 t t+a −1 [A] Function f (τ) looks exactly like f (t), but g(t − τ) is reflected and shifted. Multiply and integrate over τ from −∞ to 1. Because g only has finite range, we can pinch in the limits of integration, and the convolution becomes Z t Z t+a f ∗ g(t) = − f (τ)dτ + f (τ)dτ t−a t A1 2011 22 / 1 | Convolution Example #2 ctd Z t Z t+a f ∗ g(t) = − f (τ)dτ + f (τ)dτ t−a t When f (t) = A, f ∗ g(t) = 0 for all t.
Load more

Recommended publications
  • Discontinuous Forcing Functions
    Math 135A, Winter 2012 Discontinuous forcing functions 1 Preliminaries If f(t) is defined on the interval [0; 1), then its Laplace transform is defined to be Z 1 F (s) = L (f(t)) = e−stf(t) dt; 0 as long as this integral is defined and converges. In particular, if f is of exponential order and is piecewise continuous, the Laplace transform of f(t) will be defined. • f is of exponential order if there are constants M and c so that jf(t)j ≤ Mect: Z 1 Since the integral e−stMect dt converges if s > c, then by a comparison test (like (11.7.2) 0 in Salas-Hille-Etgen), the integral defining the Laplace transform of f(t) will converge. • f is piecewise continuous if over each interval [0; b], f(t) has only finitely many discontinuities, and at each point a in [0; b], both of the limits lim f(t) and lim f(t) t!a− t!a+ exist { they need not be equal, but they must exist. (At the endpoints 0 and b, the appropriate one-sided limits must exist.) 2 Step functions Define u(t) to be the function ( 0 if t < 0; u(t) = 1 if t ≥ 0: Then u(t) is called the step function, or sometimes the Heaviside step function: it jumps from 0 to 1 at t = 0. Note that for any number a > 0, the graph of the function u(t − a) is the same as the graph of u(t), but translated right by a: u(t − a) jumps from 0 to 1 at t = a.
    [Show full text]
  • Laplace Transforms: Theory, Problems, and Solutions
    Laplace Transforms: Theory, Problems, and Solutions Marcel B. Finan Arkansas Tech University c All Rights Reserved 1 Contents 43 The Laplace Transform: Basic Definitions and Results 3 44 Further Studies of Laplace Transform 15 45 The Laplace Transform and the Method of Partial Fractions 28 46 Laplace Transforms of Periodic Functions 35 47 Convolution Integrals 45 48 The Dirac Delta Function and Impulse Response 53 49 Solving Systems of Differential Equations Using Laplace Trans- form 61 50 Solutions to Problems 68 2 43 The Laplace Transform: Basic Definitions and Results Laplace transform is yet another operational tool for solving constant coeffi- cients linear differential equations. The process of solution consists of three main steps: • The given \hard" problem is transformed into a \simple" equation. • This simple equation is solved by purely algebraic manipulations. • The solution of the simple equation is transformed back to obtain the so- lution of the given problem. In this way the Laplace transformation reduces the problem of solving a dif- ferential equation to an algebraic problem. The third step is made easier by tables, whose role is similar to that of integral tables in integration. The above procedure can be summarized by Figure 43.1 Figure 43.1 In this section we introduce the concept of Laplace transform and discuss some of its properties. The Laplace transform is defined in the following way. Let f(t) be defined for t ≥ 0: Then the Laplace transform of f; which is denoted by L[f(t)] or by F (s), is defined by the following equation Z T Z 1 L[f(t)] = F (s) = lim f(t)e−stdt = f(t)e−stdt T !1 0 0 The integral which defined a Laplace transform is an improper integral.
    [Show full text]
  • 10 Heat Equation: Interpretation of the Solution
    Math 124A { October 26, 2011 «Viktor Grigoryan 10 Heat equation: interpretation of the solution Last time we considered the IVP for the heat equation on the whole line u − ku = 0 (−∞ < x < 1; 0 < t < 1); t xx (1) u(x; 0) = φ(x); and derived the solution formula Z 1 u(x; t) = S(x − y; t)φ(y) dy; for t > 0; (2) −∞ where S(x; t) is the heat kernel, 1 2 S(x; t) = p e−x =4kt: (3) 4πkt Substituting this expression into (2), we can rewrite the solution as 1 1 Z 2 u(x; t) = p e−(x−y) =4ktφ(y) dy; for t > 0: (4) 4πkt −∞ Recall that to derive the solution formula we first considered the heat IVP with the following particular initial data 1; x > 0; Q(x; 0) = H(x) = (5) 0; x < 0: Then using dilation invariance of the Heaviside step function H(x), and the uniquenessp of solutions to the heat IVP on the whole line, we deduced that Q depends only on the ratio x= t, which lead to a reduction of the heat equation to an ODE. Solving the ODE and checking the initial condition (5), we arrived at the following explicit solution p x= 4kt 1 1 Z 2 Q(x; t) = + p e−p dp; for t > 0: (6) 2 π 0 The heat kernel S(x; t) was then defined as the spatial derivative of this particular solution Q(x; t), i.e. @Q S(x; t) = (x; t); (7) @x and hence it also solves the heat equation by the differentiation property.
    [Show full text]
  • Delta Functions and Distributions
    When functions have no value(s): Delta functions and distributions Steven G. Johnson, MIT course 18.303 notes Created October 2010, updated March 8, 2017. Abstract x = 0. That is, one would like the function δ(x) = 0 for all x 6= 0, but with R δ(x)dx = 1 for any in- These notes give a brief introduction to the mo- tegration region that includes x = 0; this concept tivations, concepts, and properties of distributions, is called a “Dirac delta function” or simply a “delta which generalize the notion of functions f(x) to al- function.” δ(x) is usually the simplest right-hand- low derivatives of discontinuities, “delta” functions, side for which to solve differential equations, yielding and other nice things. This generalization is in- a Green’s function. It is also the simplest way to creasingly important the more you work with linear consider physical effects that are concentrated within PDEs, as we do in 18.303. For example, Green’s func- very small volumes or times, for which you don’t ac- tions are extremely cumbersome if one does not al- tually want to worry about the microscopic details low delta functions. Moreover, solving PDEs with in this volume—for example, think of the concepts of functions that are not classically differentiable is of a “point charge,” a “point mass,” a force plucking a great practical importance (e.g. a plucked string with string at “one point,” a “kick” that “suddenly” imparts a triangle shape is not twice differentiable, making some momentum to an object, and so on.
    [Show full text]
  • An Analytic Exact Form of the Unit Step Function
    Mathematics and Statistics 2(7): 235-237, 2014 http://www.hrpub.org DOI: 10.13189/ms.2014.020702 An Analytic Exact Form of the Unit Step Function J. Venetis Section of Mechanics, Faculty of Applied Mathematics and Physical Sciences, National Technical University of Athens *Corresponding Author: [email protected] Copyright © 2014 Horizon Research Publishing All rights reserved. Abstract In this paper, the author obtains an analytic Meanwhile, there are many smooth analytic exact form of the unit step function, which is also known as approximations to the unit step function as it can be seen in Heaviside function and constitutes a fundamental concept of the literature [4,5,6]. Besides, Sullivan et al [7] obtained a the Operational Calculus. Particularly, this function is linear algebraic approximation to this function by means of a equivalently expressed in a closed form as the summation of linear combination of exponential functions. two inverse trigonometric functions. The novelty of this However, the majority of all these approaches lead to work is that the exact representation which is proposed here closed – form representations consisting of non - elementary is not performed in terms of non – elementary special special functions, e.g. Logistic function, Hyperfunction, or functions, e.g. Dirac delta function or Error function and Error function and also most of its algebraic exact forms are also is neither the limit of a function, nor the limit of a expressed in terms generalized integrals or infinitesimal sequence of functions with point wise or uniform terms, something that complicates the related computational convergence. Therefore it may be much more appropriate in procedures.
    [Show full text]
  • On the Approximation of the Step Function by Raised-Cosine and Laplace Cumulative Distribution Functions
    European International Journal of Science and Technology Vol. 4 No. 9 December, 2015 On the Approximation of the Step function by Raised-Cosine and Laplace Cumulative Distribution Functions Vesselin Kyurkchiev 1 Faculty of Mathematics and Informatics Institute of Mathematics and Informatics, Paisii Hilendarski University of Plovdiv, Bulgarian Academy of Sciences, 24 Tsar Assen Str., 4000 Plovdiv, Bulgaria Email: [email protected] Nikolay Kyurkchiev Faculty of Mathematics and Informatics Institute of Mathematics and Informatics, Paisii Hilendarski University of Plovdiv, Bulgarian Academy of Sciences, Acad.G.Bonchev Str.,Bl.8 1113Sofia, Bulgaria Email: [email protected] 1 Corresponding Author Abstract In this note the Hausdorff approximation of the Heaviside step function by Raised-Cosine and Laplace Cumulative Distribution Functions arising from lifetime analysis, financial mathematics, signal theory and communications systems is considered and precise upper and lower bounds for the Hausdorff distance are obtained. Numerical examples, that illustrate our results are given, too. Key words: Heaviside step function, Raised-Cosine Cumulative Distribution Function, Laplace Cumulative Distribution Function, Alpha–Skew–Laplace cumulative distribution function, Hausdorff distance, upper and lower bounds 2010 Mathematics Subject Classification: 41A46 75 European International Journal of Science and Technology ISSN: 2304-9693 www.eijst.org.uk 1. Introduction The Cosine Distribution is sometimes used as a simple, and more computationally tractable, approximation to the Normal Distribution. The Raised-Cosine Distribution Function (RC.pdf) and Raised-Cosine Cumulative Distribution Function (RC.cdf) are functions commonly used to avoid inter symbol interference in communications systems [1], [2], [3]. The Laplace distribution function (L.pdf) and Laplace Cumulative Distribution function (L.cdf) is used for modeling in signal processing, various biological processes, finance, and economics.
    [Show full text]
  • On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions
    www.biomathforum.org/biomath/index.php/biomath ORIGINAL ARTICLE On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions Anton Iliev∗y, Nikolay Kyurkchievy, Svetoslav Markovy ∗Faculty of Mathematics and Informatics Paisii Hilendarski University of Plovdiv, Plovdiv, Bulgaria Email: [email protected] yInstitute of Mathematics and Informatics Bulgarian Academy of Sciences, Sofia, Bulgaria Emails: [email protected], [email protected] Received: , accepted: , published: will be added later Abstract—We study the uniform approximation practically important classes of sigmoid functions. of the sigmoid cut function by smooth sigmoid Two of them are the cut (or ramp) functions and functions such as the logistic and the Gompertz the step functions. Cut functions are continuous functions. The limiting case of the interval-valued but they are not smooth (differentiable) at the two step function is discussed using Hausdorff metric. endpoints of the interval where they increase. Step Various expressions for the error estimates of the corresponding uniform and Hausdorff approxima- functions can be viewed as limiting case of cut tions are obtained. Numerical examples are pre- functions; they are not continuous but they are sented using CAS MATHEMATICA. Hausdorff continuous (H-continuous) [4], [43]. In some applications smooth sigmoid functions are Keywords-cut function; step function; sigmoid preferred, some authors even require smoothness function; logistic function; Gompertz function; squashing function; Hausdorff approximation. in the definition of sigmoid functions. Two famil- iar classes of smooth sigmoid functions are the logistic and the Gompertz functions. There are I. INTRODUCTION situations when one needs to pass from nonsmooth In this paper we discuss some computational, sigmoid functions (e.
    [Show full text]
  • Maximizing Acquisition Functions for Bayesian Optimization
    Maximizing acquisition functions for Bayesian optimization James T. Wilson∗ Frank Hutter Marc Peter Deisenroth Imperial College London University of Freiburg Imperial College London PROWLER.io Abstract Bayesian optimization is a sample-efficient approach to global optimization that relies on theoretically motivated value heuristics (acquisition functions) to guide its search process. Fully maximizing acquisition functions produces the Bayes’ decision rule, but this ideal is difficult to achieve since these functions are fre- quently non-trivial to optimize. This statement is especially true when evaluating queries in parallel, where acquisition functions are routinely non-convex, high- dimensional, and intractable. We first show that acquisition functions estimated via Monte Carlo integration are consistently amenable to gradient-based optimiza- tion. Subsequently, we identify a common family of acquisition functions, includ- ing EI and UCB, whose properties not only facilitate but justify use of greedy approaches for their maximization. 1 Introduction Bayesian optimization (BO) is a powerful framework for tackling complicated global optimization problems [32, 40, 44]. Given a black-box function f : , BO seeks to identify a maximizer X!Y x∗ arg maxx f(x) while simultaneously minimizing incurred costs. Recently, these strategies have2 demonstrated2X state-of-the-art results on many important, real-world problems ranging from material sciences [17, 57], to robotics [3, 7], to algorithm tuning and configuration [16, 29, 53, 56]. From a high-level perspective, BO can be understood as the application of Bayesian decision theory to optimization problems [11, 14, 45]. One first specifies a belief over possible explanations for f using a probabilistic surrogate model and then combines this belief with an acquisition function to convey the expected utility for evaluating a set of queries X.
    [Show full text]
  • Expanded Use of Discontinuity and Singularity Functions in Mechanics
    AC 2010-1666: EXPANDED USE OF DISCONTINUITY AND SINGULARITY FUNCTIONS IN MECHANICS Robert Soutas-Little, Michigan State University Professor Soutas-Little received his B.S. in Mechanical Engineering 1955 from Duke University and his MS 1959 and PhD 1962 in Mechanics from University of Wisconsin. His name until 1982 when he married Patricia Soutas was Robert William Little. Under this name, he wrote an Elasticity book and published over 20 articles. Since 1982, he has written over 100 papers for journals and conference proceedings and written books in Statics and Dynamics. He has taught courses in Statics, Dynamics, Mechanics of Materials, Elasticity, Continuum Mechanics, Viscoelasticity, Advanced Dynamics, Advanced Elasticity, Tissue Biomechanics and Biodynamics. He has won teaching excellence awards and the Distinguished Faculty Award. During his tenure at Michigan State University, he chaired the Department of Mechanical Engineering for 5 years and the Department of Biomechanics for 13 years. He directed the Biomechanics Evaluation Laboratory from 1990 until he retired in 2002. He served as Major Professor for 22 PhD students and over 100 MS students. He has received numerous research grants and consulted with engineering companies. He now is Professor Emeritus of Mechanical Engineering at Michigan State University. Page 15.549.1 Page © American Society for Engineering Education, 2010 Expanded Use of Discontinuity and Singularity Functions in Mechanics Abstract W. H. Macauley published Notes on the Deflection of Beams 1 in 1919 introducing the use of discontinuity functions into the calculation of the deflection of beams. In particular, he introduced the singularity functions, the unit doublet to model a concentrated moment, the Dirac delta function to model a concentrated load and the Heaviside step function to start a uniform load at any point on the beam.
    [Show full text]
  • Dirac Delta Function of Matrix Argument
    Dirac Delta Function of Matrix Argument ∗ Lin Zhang Institute of Mathematics, Hangzhou Dianzi University, Hangzhou 310018, PR China Abstract Dirac delta function of matrix argument is employed frequently in the development of di- verse fields such as Random Matrix Theory, Quantum Information Theory, etc. The purpose of the article is pedagogical, it begins by recalling detailed knowledge about Heaviside unit step function and Dirac delta function. Then its extensions of Dirac delta function to vector spaces and matrix spaces are discussed systematically, respectively. The detailed and elemen- tary proofs of these results are provided. Though we have not seen these results formulated in the literature, there certainly are predecessors. Applications are also mentioned. 1 Heaviside unit step function H and Dirac delta function δ The materials in this section are essential from Hoskins’ Book [4]. There are also no new results in this section. In order to be in a systematic way, it is reproduced here. The Heaviside unit step function H is defined as 1, x > 0, H(x) := (1.1) < 0, x 0. arXiv:1607.02871v2 [quant-ph] 31 Oct 2018 That is, this function is equal to 1 over (0, +∞) and equal to 0 over ( ∞, 0). This function can − equally well have been defined in terms of a specific expression, for instance 1 x H(x)= 1 + . (1.2) 2 x | | The value H(0) is left undefined here. For all x = 0, 6 dH(x) H′(x)= = 0 dx ∗ E-mail: [email protected]; [email protected] 1 corresponding to the obvious fact that the graph of the function y = H(x) has zero slope for all x = 0.
    [Show full text]
  • A Heaviside Function Approximation for Neural Network Binary Classification
    A Heaviside Function Approximation for Neural Network Binary Classification Nathan Tsoi, Yofti Milkessa, Marynel Vazquez´ Yale University, New Haven, CT 06511 [email protected] Abstract Neural network binary classifiers are often evaluated on met- rics like accuracy and F1-Score, which are based on con- fusion matrix values (True Positives, False Positives, False Negatives, and True Negatives). However, these classifiers are commonly trained with a different loss, e.g. log loss. While it is preferable to perform training on the same loss as the eval- uation metric, this is difficult in the case of confusion matrix based metrics because set membership is a step function with- out a derivative useful for backpropagation. To address this Figure 1: Heaviside function H (left) and (Kyurkchiev and challenge, we propose an approximation of the step function Markov 2015) approximation (right) at τ = 0:7. that adheres to the properties necessary for effective train- ing of binary networks using confusion matrix based met- rics. This approach allows for end-to-end training of binary deep neural classifiers via batch gradient descent. We demon- strate the flexibility of this approach in several applications with varying levels of class imbalance. We also demonstrate how the approximation allows balancing between precision and recall in the appropriate ratio for the task at hand. 1 Introduction Figure 2: Proposed Heaviside function approximation H In the process of creating a neural network binary classi- (left) and a sigmoid fit to our proposed Heaviside approx- fier, the network is often trained via the binary cross-entropy imation (right) at τ = 0:7.
    [Show full text]
  • Spectral Density Periodogram Spectral Density (Chapter 12) Outline
    Spectral Density Periodogram Spectral Density (Chapter 12) Outline 1 Spectral Density 2 Periodogram Arthur Berg Spectral Density (Chapter 12) 2/ 19 Spectral Density Periodogram Spectral Density Periodogram Spectral Approximation Motivating Example Any stationary time series has the approximation Let’s consider the stationary time series q X xt = Uk cos(2π!k t) + Vk sin(2π!k t) xt = U cos(2π!0t) + V sin(2π!0t) k=1 where Uk and Vk are independent zero-mean random variables with where U; V are independent zero-mean random variables with equal 2 2 variances σk at distinct frequencies !k . variances σ . We shall assume !0 > 0, and from the Nyquist rate, we The autocovariance function of xt is (HW 4c Exercise) can further assume !0 < 1=2. q The “period” of xt is 1=!0, i.e. the process makes !0 cycles going from xt X 2 γ(h) = σk cos(2π!k h) to xt+1. k=1 From the formula q e−iα + e−iα In particular X 2 cos(α) = γ(0) = σk 2 |{z} k=1 we have var(xt ) | {z } sum of variances σ2 γ( ) 2 −2π!0h 2πi!0h Note: h is not absolutely summable,1 i.e. γ(h) = σ cos(2π!0h) = e + 2 X 2 jγ(h)j = 1 h=−∞ Arthur Berg Spectral Density (Chapter 12) 4/ 19 Arthur Berg Spectral Density (Chapter 12) 5/ 19 Spectral Density Periodogram Spectral Density Periodogram Riemann-Stieltjes Integration Integration w.r.t. a Step Function Riemann-Stieltjes integration enables one to integrate with respect to a general nondecreasing function.
    [Show full text]