' $ ' Beyond linearity $ Part IV: • So far, the majority of methods/models we have seen are linear • In general, it is highly unlikely that the true underlying function f(·) is Basis expansion methods linear ! either on the original scale, or some transformed scale Reading • Linear models can serve as good approximations • HTF: Chapters 5, 8 ! locally, any function can be well approximated by a line • RWC: Chapters 3, 4, 5 ! convenient analytically ! interpretable • Here we would like to move beyond linearity &188 S. Haneuse; Biostat/Stat 572 % &189 S. Haneuse; Biostat/Stat 572 % ' Hormone replacement therapy over time $ ' $ 550 BCSC data 450 • Rates of hormone replacement therapy (HRT) use and cancer, between 350 HRT use 1997 and 2004 250 Invasive cancer DCIS • Data obtained from the Breast Cancer Surveillance Consortium 150 ! on-going study of mammography in the US ! collected from 7 sites around the country 6 4 Cancer 2 • Rates based on data from women aged 50-69 0 ! age- and site-adjusted 1997 1998 1999 2000 2001 2002 2003 2004 ! n = 84, month-specific values Calendar date &190 S. Haneuse; Biostat/Stat 572 % &191 S. Haneuse; Biostat/Stat 572 % 'Linear regression $ ' $ Linear regression • Consider the task of describing the trend in HRT use over time 550 • Although the structure is evidently non-linear, we could start by considering a linear model 450 th • Let xi denote the date of the i observation, and yi the corresponding observed rate of HRT use 350 HRT use (per 1,000) • The form of the linear regression model is 250 yi = β0 + β1xi + #i, 150 with the corresponding fit shown next 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date &192 S. Haneuse; Biostat/Stat 572 % &193 S. Haneuse; Biostat/Stat 572 % '• The systematic component of this model consists of a linear combination $ '• After standardizing, so that x ∈ [0, 1], the basis can be displayed as $ of two functions: 1.0 h0(x) = 1 0.8 h1(x) = x 0.6 • These two functions are referred to as basis functions and form the basis 0.4 of the model 0.2 • The basis corresponds to the columns of the design matrix 0.0 0.0 0.2 0.4 0.6 0.8 1.0 1 x1 . X = . • The core idea of Part IV is to augment/replace the input vector by considering transformations of the X-space via families of basis functions 1 xn &194 S. Haneuse; Biostat/Stat 572 % &195 S. Haneuse; Biostat/Stat 572 % ' $ ' $ Polynomial basis expansion Quadratic regression model: p = 2 550 • A straightforward way of augmenting the design matrix is to consider higher-degree polynomials 450 • A pth-degree polynomial basis corresponds to the basis functions 350 j hj(x) = x j = 0, . , p HRT use (per 1,000) and yield the design matrix 250 p 1 x1 . x1 . 150 X = . . 1997 1998 1999 2000 2001 2002 2003 2004 p 1 xn . xn Calendar date &196 S. Haneuse; Biostat/Stat 572 % &197 S. Haneuse; Biostat/Stat 572 % ' $ ' $ Cubic regression model: p = 3 10^th!degree polynomial regression 550 550 450 450 350 350 HRT use (per 1,000) HRT use (per 1,000) 250 250 150 150 1997 1998 1999 2000 2001 2002 2003 2004 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date Calendar date &198 S. Haneuse; Biostat/Stat 572 % &199 S. Haneuse; Biostat/Stat 572 % '• Basis representations: $ ' Piecewise polynomials $ Quadratic regression basis Cubic regression basis 1.0 1.0 • The polynomial basis imposes a single structure throughout the X-space 0.8 0.8 • We could relax this by partitioning the X-space into a series of disjoint intervals, and adopt a polynomial structure within each 0.6 0.6 0.4 0.4 • For the HRT data, we could partition the [1997, 2004] interval into three sub-intervals by defining two split points 0.2 0.2 ξ = (ξ1, ξ2) = (2000, 2002.5) 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 &200 S. Haneuse; Biostat/Stat 572 % &201 S. Haneuse; Biostat/Stat 572 % 'Discrete piecewise polynomials $ ' $ Piecewise constant • Given ξ, the simplest model would assume f(X) is piecewise constant 550 ! estimation reduces to calculating the sample mean within the three intervals 450 • This model can be represented with three basis functions: ∗ 350 h0(x) = 1[x<ξ1] ∗ h1(x) = 1[ξ1≤x<ξ2] HRT use (per 1,000) ∗ 250 h2(x) = 1[x≥ξ2] so that 150 !1 !2 3 ∗ 1997 1998 1999 2000 2001 2002 2003 2004 f(x) = βj hj (x) j=0 Calendar date ' &202 S. Haneuse; Biostat/Stat 572 % &203 S. Haneuse; Biostat/Stat 572 % '• An alternative representation is given by $ ' $ Piecewise linear ∗ h0(x) = 1 550 ∗ h1(x) = 1[ξ1≤x<ξ2] ∗ h2(x) = 1[x≥ξ2] 450 • We can relax this model by allowing the piecewise components to be linear 350 ! require three additional basis functions HRT use (per 1,000) 250 h00(x) = 1, h10(x) = 1[ξ1≤x<ξ2], h20(x) = 1[x≥ξ2] and 150 !1 !2 1997 1998 1999 2000 2001 2002 2003 2004 h01(x) = x, h11(x) = 1[ξ1≤x<ξ2]x, h21(x) = 1[x≥ξ2]x Calendar date &204 S. Haneuse; Biostat/Stat 572 % &205 S. Haneuse; Biostat/Stat 572 % ' $ '• Can extend this to arbitrary degrees, where we define for j = 0, . ., p $ Piecewise cubic j h0j(x) = x 550 j h1j(x) = 1[ξ1≤x<ξ2]x j h2j(x) = 1[x≥ξ2]x 450 • Given K interior split points, or knots, this results in 350 (K + 1) × (p + 1) HRT use (per 1,000) 250 parameters 150 !1 !2 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date &206 S. Haneuse; Biostat/Stat 572 % &207 S. Haneuse; Biostat/Stat 572 % 'Continuous piecewise polynomials $ ' $ Continuous piecewise linear • Typically, we don’t expect the true underlying function to be disjoint at a 550 set of (essentially) arbitrary knots • Ensure continuity by imposing constraints at the knot locations 450 • For the piecewise linear model there are two constraints: 350 ξ1 : β00 + ξ1β01 = β10 + ξ1β11 HRT use (per 1,000) 250 ξ2 : β10 + ξ2β11 = β12 + ξ1β12 150 ! 6 parameters + 2 constraints = 4 free parameters !1 !2 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date &208 S. Haneuse; Biostat/Stat 572 % &209 S. Haneuse; Biostat/Stat 572 % '• Alternatively, we can use a basis which directly incorporates the $ 'Truncated-power basis $ constraints: • We might also be interested in imposing a certain degree of smoothness at h0(x) = 1, h1(x) = x the knot locations h2(x) = (x − ξ1)+, h3(x) = (x − ξ2)+ ! t+ denotes the positive part • Achieved by incorporating additional constraints at the knot locations ! 1st constraint ensures continuity of f(·) nd $ • After standardizing x, we 1.0 ! 2 constraint ensures continuity of the first derivative f (·) can represent this basis ! 3rd constraint ensures continuity of the second derivative f $$(·) 0.8 graphically as: ! etc. 0.6 • Q: Can we intuitively ex- 0.4 • For a polynomial of degree p, imposing K × p constraints yields a plain how this representa- function, f(·), which has continuous derivatives, wrt x, up to order p − 1 tion achieves the goal? 0.2 • Total degrees of freedom are: 0.0 0.0 0.2 0.4 0.6 0.8 1.0 (K + 1) × (p + 1) − K × p = K + p + 1 &210 S. Haneuse; Biostat/Stat 572 % &211 S. Haneuse; Biostat/Stat 572 % '• Build on the representation for continuous piecewise linear model, and $ ' $ increase the order of the local polynomial Piecewise linear: df = 6 550 • General form of the truncated-power basis: ! degree-p spline, with knots ξk, k = 1, . , K 450 j hj (x) = x , j = 0, . , p p hp+k(x) = (x − ξk)+, k = 1, . , K 350 HRT use (per 1,000) ! K + p + 1 basis functions 250 150 !1 !2 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date &212 S. Haneuse; Biostat/Stat 572 % &213 S. Haneuse; Biostat/Stat 572 % ' $ ' $ Linear spline: df = 4 Quadratic spline: df = 5 550 550 450 450 350 350 HRT use (per 1,000) HRT use (per 1,000) 250 250 150 150 !1 !2 !1 !2 1997 1998 1999 2000 2001 2002 2003 2004 1997 1998 1999 2000 2001 2002 2003 2004 Calendar date Calendar date &214 S. Haneuse; Biostat/Stat 572 % &215 S. Haneuse; Biostat/Stat 572 % ' $ 'B-splines $ Cubic spline: df = 6 • The form of the truncated power basis leads to numerical instability 550 ! powers of large numbers lead to rounding problems 450 • B-splines provide an alternative, computationally convenient, and equivalent representation of the truncated power basis ! equivalent in that the two bases span the same set of functions 350 • B-spline representation is via a series of polynomial basis functions which HRT use (per 1,000) 250 have local support • Two key aspects of their definition are 150 !1 !2 ! they are locally defined, and hence require an augmentation of the knot 1997 1998 1999 2000 2001 2002 2003 2004 sequence Calendar date ! a recursive relation used to build up the degree of the polynomial functions &216 S. Haneuse; Biostat/Stat 572 % &217 S. Haneuse; Biostat/Stat 572 % '• To help understand the of the representation, we can consider the cubic $ ' $ spline (p = 3) on the unit interval, x ∈ [0, 1] with K = 3 interior knots B!spline basis functions of degree 0: df = 4 ξ = (0.25, 0.50, 0.75) 1 • Initially, consider truncated power basis of degree 0 ! i.e., a piecewise constant function ) x ( 0 i • The degree 0 B-spline representation is a series of locally constant B functions over the support of X ! requires augmenting ξ with boundary knots, say, ξ0 = 0 and ξK+1 = ξ4 = 1, to give 0 !0 !1 !2 !3 !4 ∗ ξ = (0, 0.25, 0.50, 0.75, 1) 0.00 0.25 0.50 0.75 1.00 x ! boundary knots define the support over which the spline is evaluated &218 S.