www.biomathforum.org/biomath/index.php/biomath ORIGINAL ARTICLE On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions Anton Iliev∗y, Nikolay Kyurkchievy, Svetoslav Markovy ∗Faculty of Mathematics and Informatics Paisii Hilendarski University of Plovdiv, Plovdiv, Bulgaria Email: [email protected] yInstitute of Mathematics and Informatics Bulgarian Academy of Sciences, Sofia, Bulgaria Emails: [email protected], [email protected] Received: , accepted: , published: will be added later Abstract—We study the uniform approximation practically important classes of sigmoid functions. of the sigmoid cut function by smooth sigmoid Two of them are the cut (or ramp) functions and functions such as the logistic and the Gompertz the step functions. Cut functions are continuous functions. The limiting case of the interval-valued but they are not smooth (differentiable) at the two step function is discussed using Hausdorff metric. endpoints of the interval where they increase. Step Various expressions for the error estimates of the corresponding uniform and Hausdorff approxima- functions can be viewed as limiting case of cut tions are obtained. Numerical examples are pre- functions; they are not continuous but they are sented using CAS MATHEMATICA. Hausdorff continuous (H-continuous) [4], [43]. In some applications smooth sigmoid functions are Keywords-cut function; step function; sigmoid preferred, some authors even require smoothness function; logistic function; Gompertz function; squashing function; Hausdorff approximation. in the definition of sigmoid functions. Two famil- iar classes of smooth sigmoid functions are the logistic and the Gompertz functions. There are I. INTRODUCTION situations when one needs to pass from nonsmooth In this paper we discuss some computational, sigmoid functions (e. g. cut functions) to smooth modelling and approximation issues related to sigmoid functions, and vice versa. Such a neces- several classes of sigmoid functions. Sigmoid sity rises the issue of approximating nonsmooth functions find numerous applications in various sigmoid functions by smooth sigmoid functions. fields related to life sciences, chemistry, physics, artificial intelligence, etc. In fields such as signal One can encounter similar approximation prob- processing, pattern recognition, machine learning, lems when looking for appropriate models for artificial neural networks, sigmoid functions are fitting time course measurement data coming e. g. also known as “activation” and “squashing” func- from cellular growth experiments. Depending on tions. In this work we concentrate on several the general view of the data one can decide to use Citation: Anton Iliev, Nikolay Kyurkchiev, Svetoslav Markov, On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions, Biomath 4 (2015), 15, Page 1 of 12 http://dx.doi.org/10.11145/j.biomath.2015.. A. Iliev et al., On the Approximation of the Cut and Step Functions by Logistic ... initially a cut function in order to obtain rough the real line, that is functions s of the form initial values for certain parameters, such as the s : R −! R. Sigmoid functions can be defined maximum growth rate. Then one can use a more as bounded monotone non-decreasing functions on sophisticate model (logistic or Gompertz) to obtain R. One usually makes use of normalized sigmoid a better fit to the measurement data. The presented functions defined as monotone non-decreasing results may be used to indicate to what extend and functions s(t); t 2 R, such that lim s(t)t→−∞ = 0 in what sense a model can be improved by another and lim s(t)t!1 = 1. In the fields of neural one and how the two models can be compared. networks and machine learning sigmoid-like func- tions of many variables are used, familiar under the Section 2 contains preliminary definitions and name activation functions. (In some applications motivations. In Section 3 we study the uniform and the sigmoid functions are normalised so that the Hausdorff approximation of the cut functions by lower asymptote is assumed −1: lim s(t) = logistic functions. Curiously, the uniform distance t→−∞ −1.) between a cut function and the logistic function of best uniform approximation is an absolute constant Cut (ramp) functions. Let ∆ = [γ − δ; γ + δ] be not depending on the slope of the functions, a an interval on the real line R with centre γ 2 R result observed in [18]. By contrast, it turns out and radius δ 2 R. A cut function (on ∆) is defined that the Hausdorff distance (H-distance) depends as follows: on the slope and tends to zero when increasing the slope. Showing that the family of logistic functions Definition 1. The cut function cγ,δ on ∆ is defined cannot approximate the cut function arbitrary well, for t 2 R by we then consider the limiting case when the cut 8 > 0, if t < ∆, > function tends to the step function (in Hausdorff < t − γ + δ sense). In this way we obtain an extension of a cγ,δ(t) = , if t 2 ∆, (1) 2δ previous result on the Hausdorff approximation > :> 1, if ∆ < t. of the step function by logistic functions [4]. In Section 4 we discuss the approximation of the Note that the slope of function cγ,δ(t) on the cut function by a family of squashing functions interval ∆ is 1=(2δ) (the slope is constant in the induced by the logistic function. It has been shown whole interval ∆). Two special cases are of interest in [18] that the latter family approximates uni- for our discussion in the sequel. formly the cut function arbitrary well. We propose Special case 1. For γ = 0 we obtain a cut a new estimate for the H-distance between the function on the interval ∆ = [−δ; δ]: cut function and its best approximating squashing function. Our estimate is then extended to cover 8 > 0, if t < −δ, the limiting case of the step function. In Section 5 > < t + δ the approximation of the cut function by Gompertz c0,δ(t) = , if −δ ≤ t ≤ δ, (2) functions is considered using similar techniques > 2δ > as in the previous sections. The application of the : 1, if δ < t. logistic and Gompertz functions in life sciences Special case 2 γ = δ is briefly discussed. Numerical examples are pre- . For we obtain the cut ∆ = [0; 2δ] sented throughout the paper using the computer function on : algebra system MATHEMATICA. 8 0, if t < 0, > II. PRELIMINARIES <> t cδ,δ(t) = , if 0 ≤ t ≤ 2δ, (3) 2δ Sigmoid functions. In this work we consider > sigmoid functions of a single variable defined on :> 1, if 2δ < t. Biomath 4 (2015), 15, http://dx.doi.org/10.11145/j.biomath.2015.. Page 2 of 12 A. Iliev et al., On the Approximation of the Cut and Step Functions by Logistic ... Step functions. The step function (with “jump” at For the sake of simplicity throughout the pa- γ 2 R) can be defined by per we shall work with some of the special cut 8 0, if t < γ, functions (2), (3), instead of the more general <> (arbitrary shifted) cut function (1); these special hγ(t) = cγ;0(t) = [0; 1], if t = γ, (4) > choices will not lead to any loss of generality : 1, if t > γ; concerning the results obtained. Moreover, for all which is an interval-valued function (or just in- sigmoid functions considered in the sequel we terval function) [4], [43]. In the literature various shall define a “basic” sigmoid function such that point values, such as 0; 1=2 or 1, are prescribed any member of the corresponding class is obtained to the step function (4) at the point γ; we prefer by replacing the argument t by t − γ, that is by the interval value [0; 1]. When the jump is at the shifting the basic function by some γ 2 R. origin, that is γ = 0, then the step function is Logistic and Gompertz functions: applications known as the Heaviside step function; its “inter- to life-sciences. In this work we focus on two val” formulation is: 8 familiar smooth sigmoid functions, namely the 0, if t < 0, <> Gompertz function and the Verhulst logistic func- h0(t) = c0;0(t) = [0; 1], if t = 0, (5) tion. Both their inventors, B. Gompertz and P.- :> 1, if t > 0: F. Verhulst, have been motivated by the famous demographic studies of Thomas Malthus. H-distance. The step function can be perceived The Gompertz function was introduced by as a limiting case of the cut function. Namely, Benjamin Gompertz [22] for the study of de- for δ ! 0, the cut function cδ,δ tends in “Haus- mographic phenomena, more specifically human dorff sense” to the step function. Here “Haus- aging [38], [39], [47]. Gompertz functions find dorff sense” means Hausdorff distance, briefly numerous applications in biology, ecology and H-distance. The H-distance ρ(f; g) between two medicine. A. K. Laird successfully used the Gom- interval functions f; g on Ω ⊆ R, is the distance pertz curve to fit data of growth of tumors [32]; between their completed graphs F (f) and F (g) tumors are cellular populations growing in a con- considered as closed subsets of Ω × R [24], [41]. fined space where the availability of nutrients is More precisely, limited [1], [2], [15], [19]. ρ(f; g) = maxf sup inf jjA − Bjj; (6) A number of experimental scientists apply A2F (f) B2F (g) Gompertz models in bacterial cell growth, more sup inf jjA − Bjjg; specifically in food control [10], [31], [42], [48], B2F (g) A2F (f) [49], [50]. Gompertz models prove to be useful in 2 wherein jj:jj is any norm in R , e. g. the maximum animal and agro-sciences as well [8], [21], [27], norm jj(t; x)jj = max jtj; jxj. [48].