DOCSLIB.ORG

Simple Linear Regression Models: Only One Predictor

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Simple Linear Regression Models: Only One Predictor SimpleSimple LinearLinear RegressionRegression ModelsModels ©2010 Raj Jain www.rajjain.com 14-1 OverviewOverview 1. Definition of a Good Model 2. Estimation of Model parameters 3. Allocation of Variation 4. Standard deviation of Errors 5. Confidence Intervals for Regression Parameters 6. Confidence Intervals for Predictions 7. Visual Tests for verifying Regression Assumption ©2010 Raj Jain www.rajjain.com 14-2 SimpleSimple LinearLinear RegressionRegression ModelsModels Regression Model: Predict a response for a given set of predictor variables. Response Variable: Estimated variable Predictor Variables: Variables used to predict the response. predictors or factors Linear Regression Models: Response is a linear function of predictors. Simple Linear Regression Models: Only one predictor ©2010 Raj Jain www.rajjain.com 14-3 DefinitionDefinition ofof aa GoodGood ModelModel y y y x x x Good Good Bad ©2010 Raj Jain www.rajjain.com 14-4 GoodGood ModelModel (Cont)(Cont) Regression models attempt to minimize the distance measured vertically between the observation point and the model line (or curve). The length of the line segment is called residual, modeling error, or simply error. The negative and positive errors should cancel out ⇒ Zero overall error Many lines will satisfy this criterion. ©2010 Raj Jain www.rajjain.com 14-5 GoodGood ModelModel (Cont)(Cont) Choose the line that minimizes the sum of squares of the errors. where, is the predicted response when the predictor variable is x. The parameter b0 and b1 are fixed regression parameters to be determined from the data. Given n observation pairs {(x1, y1), …, (xn, yn)}, the estimated response for the ith observation is: The error is: ©2010 Raj Jain www.rajjain.com 14-6 GoodGood ModelModel (Cont)(Cont) The best linear model minimizes the sum of squared errors (SSE): subject to the constraint that the mean error is zero: This is equivalent to minimizing the variance of errors (see Exercise). ©2010 Raj Jain www.rajjain.com 14-7 EstimationEstimation ofof ModelModel ParametersParameters Regression parameters that give minimum error variance are: and where, ©2010 Raj Jain www.rajjain.com 14-8 ExampleExample 14.114.1 The number of disk I/O's and processor times of seven programs were measured as: (14, 2), (16, 5), (27, 7), (42, 9), (39, 10), (50, 13), (83, 20) For this data: n=7, Σ xy=3375, Σ x=271, Σ x2=13,855, Σ y=66, Σ y2=828, = 38.71, = 9.43. Therefore, The desired linear model is: ©2010 Raj Jain www.rajjain.com 14-9 ExampleExample 14.114.1 (Cont)(Cont) ©2010 Raj Jain www.rajjain.com 14-10 ExampleExample 14.14. (Cont)(Cont) Error Computation ©2010 Raj Jain www.rajjain.com 14-11 DerivationDerivation ofof RegressionRegression ParametersParameters The error in the ith observation is: For a sample of n observations, the mean error is: Setting mean error to zero, we obtain: Substituting b0 in the error expression, we get: ©2010 Raj Jain www.rajjain.com 14-12 DerivationDerivation ofof RegressionRegression ParametersParameters (Cont)(Cont) The sum of squared errors SSE is: ©2010 Raj Jain www.rajjain.com 14-13 DerivationDerivation (Cont)(Cont) Differentiating this equation with respect to b1 and equating the result to zero: That is, ©2010 Raj Jain www.rajjain.com 14-14 AllocationAllocation ofof VariationVariation Error variance without Regression = Variance of the response and ©2010 Raj Jain www.rajjain.com 14-15 AllocationAllocation ofof VariationVariation (Cont)(Cont) The sum of squared errors without regression would be: This is called total sum of squares or (SST). It is a measure of y's variability and is called variation of y. SST can be computed as follows: Where, SSY is the sum of squares of y (or Σ y2). SS0 is the sum of squares of and is equal to . ©2010 Raj Jain www.rajjain.com 14-16 AllocationAllocation ofof VariationVariation (Cont)(Cont) The difference between SST and SSE is the sum of squares explained by the regression. It is called SSR: or The fraction of the variation that is explained determines the goodness of the regression and is called the coefficient of determination, R2: ©2010 Raj Jain www.rajjain.com 14-17 AllocationAllocation ofof VariationVariation (Cont)(Cont) The higher the value of R2, the better the regression. R2=1 ⇒ Perfect fit R2=0 ⇒ No fit Coefficient of Determination = {Correlation Coefficient (x,y)}2 Shortcut formula for SSE: ©2010 Raj Jain www.rajjain.com 14-18 ExampleExample 14.214.2 For the disk I/O-CPU time data of Example 14.1: The regression explains 97% of CPU time's variation. ©2010 Raj Jain www.rajjain.com 14-19 StandardStandard DeviationDeviation ofof ErrorsErrors Since errors are obtained after calculating two regression parameters from the data, errors have n-2 degrees of freedom SSE/(n-2) is called mean squared errors or (MSE). Standard deviation of errors = square root of MSE. SSY has n degrees of freedom since it is obtained from n independent observations without estimating any parameters. SS0 has just one degree of freedom since it can be computed simply from SST has n-1 degrees of freedom, since one parameter must be calculated from the data before SST can be computed. ©2010 Raj Jain www.rajjain.com 14-20 StandardStandard DeviationDeviation ofof ErrorsErrors (Cont)(Cont) SSR, which is the difference between SST and SSE, has the remaining one degree of freedom. Overall, Notice that the degrees of freedom add just the way the sums of squares do. ©2010 Raj Jain www.rajjain.com 14-21 ExampleExample 14.314.3 For the disk I/O-CPU data of Example 14.1, the degrees of freedom of the sums are: The mean squared error is: The standard deviation of errors is: ©2010 Raj Jain www.rajjain.com 14-22 ConfidenceConfidence IntervalsIntervals forfor RegressionRegression ParamsParams Regression coefficients b0 and b1 are estimates from a single sample of size n ⇒ Random ⇒ Using another sample, the estimates may be different. If β0 and β1 are true parameters of the population. That is, Computed coefficients b0 and b1 are estimates of β0 and β1, respectively. ©2010 Raj Jain www.rajjain.com 14-23 ConfidenceConfidence IntervalsIntervals (Cont)(Cont) The 100(1-α)% confidence intervals for b0 and b1 can be be computed using t[1-α/2; n-2] --- the 1-α/2 quantile of a t variate with n-2 degrees of freedom. The confidence intervals are: And If a confidence interval includes zero, then the regression parameter cannot be considered different from zero at the at 100(1-α)% confidence level. ©2010 Raj Jain www.rajjain.com 14-24 ExampleExample 14.414.4 For the disk I/O and CPU data of Example 14.1, we have n=7, =38.71, =13,855, and se=1.0834. Standard deviations of b0 and b1 are: ©2010 Raj Jain www.rajjain.com 14-25 ExampleExample 14.414.4 (Cont)(Cont) From Appendix Table A.4, the 0.95-quantile of a t-variate with 5 degrees of freedom is 2.015. ⇒ 90% confidence interval for b0 is: Since, the confidence interval includes zero, the hypothesis that this parameter is zero cannot be rejected at 0.10 significance level. ⇒ b0 is essentially zero. 90% Confidence Interval for b1 is: Since the confidence interval does not include zero, the slope b1 is significantly different from zero at this confidence level. ©2010 Raj Jain www.rajjain.com 14-26 CaseCase StudyStudy 14.1:14.1: RemoteRemote ProcedureProcedure CallCall ©2010 Raj Jain www.rajjain.com 14-27 CaseCase StudyStudy 14.114.1 (Cont)(Cont) UNIX: ©2010 Raj Jain www.rajjain.com 14-28 CaseCase StudyStudy 14.114.1 (Cont)(Cont) ARGUS: ©2010 Raj Jain www.rajjain.com 14-29 CaseCase StudyStudy 14.114.1 (Cont)(Cont) Best linear models are: The regressions explain 81% and 75% of the variation, respectively. Does ARGUS takes larger time per byte as well as a larger set up time per call than UNIX? ©2010 Raj Jain www.rajjain.com 14-30 CaseCase StudyStudy 14.114.1 (Cont)(Cont) Intervals for intercepts overlap while those of the slopes do not. ⇒ Set up times are not significantly different in the two systems while the per byte times (slopes) are different. ©2010 Raj Jain www.rajjain.com 14-31 ConfidenceConfidence IntervalsIntervals forfor PredictionsPredictions This is only the mean value of the predicted response. Standard deviation of the mean of a future sample of m observations is: m =1 ⇒ Standard deviation of a single future observation: ©2010 Raj Jain www.rajjain.com 14-32 CICI forfor PredictionsPredictions (Cont)(Cont) m = ∞ ⇒ Standard deviation of the mean of a large number of future observations at xp: 100(1-α)% confidence interval for the mean can be constructed using a t quantile read at n-2 degrees of freedom. ©2010 Raj Jain www.rajjain.com 14-33 CICI forfor PredictionsPredictions (Cont)(Cont) Goodness of the prediction decreases as we move away from the center. ©2010 Raj Jain www.rajjain.com 14-34 ExampleExample 14.514.5 Using the disk I/O and CPU time data of Example 14.1, let us estimate the CPU time for a program with 100 disk I/O's. For a program with 100 disk I/O's, the mean CPU time is: ©2010 Raj Jain www.rajjain.com 14-35 ExampleExample 14.514.5 (Cont)(Cont) The standard deviation of the predicted mean of a large number of observations is: From Table A.4, the 0.95-quantile of the t-variate with 5 degrees of freedom is 2.015. ⇒ 90% CI for the predicted mean ©2010 Raj Jain www.rajjain.com 14-36 ExampleExample 14.514.5 (Cont)(Cont) CPU time of a single future program with 100 disk I/O's: 90% CI for a single prediction: ©2010 Raj Jain www.rajjain.com 14-37 VisualVisual TestsTests forfor RegressionRegression AssumptionsAssumptions Regression assumptions: 1.
Load more

Recommended publications
  • Lecture 22: Bivariate Normal Distribution Distribution
    6.5 Conditional Distributions General Bivariate Normal Let Z1; Z2 ∼ N (0; 1), which we will use to build a general bivariate normal Lecture 22: Bivariate Normal Distribution distribution. 1 1 2 2 f (z1; z2) = exp − (z1 + z2 ) Statistics 104 2π 2 We want to transform these unit normal distributions to have the follow Colin Rundel arbitrary parameters: µX ; µY ; σX ; σY ; ρ April 11, 2012 X = σX Z1 + µX p 2 Y = σY [ρZ1 + 1 − ρ Z2] + µY Statistics 104 (Colin Rundel) Lecture 22 April 11, 2012 1 / 22 6.5 Conditional Distributions 6.5 Conditional Distributions General Bivariate Normal - Marginals General Bivariate Normal - Cov/Corr First, lets examine the marginal distributions of X and Y , Second, we can find Cov(X ; Y ) and ρ(X ; Y ) Cov(X ; Y ) = E [(X − E(X ))(Y − E(Y ))] X = σX Z1 + µX h p i = E (σ Z + µ − µ )(σ [ρZ + 1 − ρ2Z ] + µ − µ ) = σX N (0; 1) + µX X 1 X X Y 1 2 Y Y 2 h p 2 i = N (µX ; σX ) = E (σX Z1)(σY [ρZ1 + 1 − ρ Z2]) h 2 p 2 i = σX σY E ρZ1 + 1 − ρ Z1Z2 p 2 2 Y = σY [ρZ1 + 1 − ρ Z2] + µY = σX σY ρE[Z1 ] p 2 = σX σY ρ = σY [ρN (0; 1) + 1 − ρ N (0; 1)] + µY = σ [N (0; ρ2) + N (0; 1 − ρ2)] + µ Y Y Cov(X ; Y ) ρ(X ; Y ) = = ρ = σY N (0; 1) + µY σX σY 2 = N (µY ; σY ) Statistics 104 (Colin Rundel) Lecture 22 April 11, 2012 2 / 22 Statistics 104 (Colin Rundel) Lecture 22 April 11, 2012 3 / 22 6.5 Conditional Distributions 6.5 Conditional Distributions General Bivariate Normal - RNG Multivariate Change of Variables Consequently, if we want to generate a Bivariate Normal random variable Let X1;:::; Xn have a continuous joint distribution with pdf f defined of S.
    [Show full text]
  • A Generalized Linear Model for Principal Component Analysis of Binary Data
    A Generalized Linear Model for Principal Component Analysis of Binary Data Andrew I. Schein Lawrence K. Saul Lyle H. Ungar Department of Computer and Information Science University of Pennsylvania Moore School Building 200 South 33rd Street Philadelphia, PA 19104-6389 {ais,lsaul,ungar}@cis.upenn.edu Abstract they are not generally appropriate for other data types. Recently, Collins et al.[5] derived generalized criteria We investigate a generalized linear model for for dimensionality reduction by appealing to proper- dimensionality reduction of binary data. The ties of distributions in the exponential family. In their model is related to principal component anal- framework, the conventional PCA of real-valued data ysis (PCA) in the same way that logistic re- emerges naturally from assuming a Gaussian distribu- gression is related to linear regression. Thus tion over a set of observations, while generalized ver- we refer to the model as logistic PCA. In this sions of PCA for binary and nonnegative data emerge paper, we derive an alternating least squares respectively by substituting the Bernoulli and Pois- method to estimate the basis vectors and gen- son distributions for the Gaussian. For binary data, eralized linear coefficients of the logistic PCA the generalized model's relationship to PCA is anal- model. The resulting updates have a simple ogous to the relationship between logistic and linear closed form and are guaranteed at each iter- regression[12]. In particular, the model exploits the ation to improve the model's likelihood. We log-odds as the natural parameter of the Bernoulli dis- evaluate the performance of logistic PCA|as tribution and the logistic function as its canonical link.
    [Show full text]
  • Applied Biostatistics Mean and Standard Deviation the Mean the Median Is Not the Only Measure of Central Value for a Distribution
    Health Sciences M.Sc. Programme Applied Biostatistics Mean and Standard Deviation The mean The median is not the only measure of central value for a distribution. Another is the arithmetic mean or average, usually referred to simply as the mean. This is found by taking the sum of the observations and dividing by their number. The mean is often denoted by a little bar over the symbol for the variable, e.g. x . The sample mean has much nicer mathematical properties than the median and is thus more useful for the comparison methods described later. The median is a very useful descriptive statistic, but not much used for other purposes. Median, mean and skewness The sum of the 57 FEV1s is 231.51 and hence the mean is 231.51/57 = 4.06. This is very close to the median, 4.1, so the median is within 1% of the mean. This is not so for the triglyceride data. The median triglyceride is 0.46 but the mean is 0.51, which is higher. The median is 10% away from the mean. If the distribution is symmetrical the sample mean and median will be about the same, but in a skew distribution they will not. If the distribution is skew to the right, as for serum triglyceride, the mean will be greater, if it is skew to the left the median will be greater. This is because the values in the tails affect the mean but not the median. Figure 1 shows the positions of the mean and median on the histogram of triglyceride.
    [Show full text]
  • 1. How Different Is the T Distribution from the Normal?
    Statistics 101–106 Lecture 7 (20 October 98) c David Pollard Page 1 Read M&M §7.1 and §7.2, ignoring starred parts. Reread M&M §3.2. The eects of estimated variances on normal approximations. t-distributions. Comparison of two means: pooling of estimates of variances, or paired observations. In Lecture 6, when discussing comparison of two Binomial proportions, I was content to estimate unknown variances when calculating statistics that were to be treated as approximately normally distributed. You might have worried about the effect of variability of the estimate. W. S. Gosset (“Student”) considered a similar problem in a very famous 1908 paper, where the role of Student’s t-distribution was first recognized. Gosset discovered that the effect of estimated variances could be described exactly in a simplified problem where n independent observations X1,...,Xn are taken from (, ) = ( + ...+ )/ a normal√ distribution, N . The sample mean, X X1 Xn n has a N(, / n) distribution. The random variable X Z = √ / n 2 2 Phas a standard normal distribution. If we estimate by the sample variance, s = ( )2/( ) i Xi X n 1 , then the resulting statistic, X T = √ s/ n no longer has a normal distribution. It has a t-distribution on n 1 degrees of freedom. Remark. I have written T , instead of the t used by M&M page 505. I find it causes confusion that t refers to both the name of the statistic and the name of its distribution. As you will soon see, the estimation of the variance has the effect of spreading out the distribution a little beyond what it would be if were used.
    [Show full text]
  • Lecture 13: Simple Linear Regression in Matrix Format
    11:55 Wednesday 14th October, 2015 See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 13: Simple Linear Regression in Matrix Format 36-401, Section B, Fall 2015 13 October 2015 Contents 1 Least Squares in Matrix Form 2 1.1 The Basic Matrices . .2 1.2 Mean Squared Error . .3 1.3 Minimizing the MSE . .4 2 Fitted Values and Residuals 5 2.1 Residuals . .7 2.2 Expectations and Covariances . .7 3 Sampling Distribution of Estimators 8 4 Derivatives with Respect to Vectors 9 4.1 Second Derivatives . 11 4.2 Maxima and Minima . 11 5 Expectations and Variances with Vectors and Matrices 12 6 Further Reading 13 1 2 So far, we have not used any notions, or notation, that goes beyond basic algebra and calculus (and probability). This has forced us to do a fair amount of book-keeping, as it were by hand. This is just about tolerable for the simple linear model, with one predictor variable. It will get intolerable if we have multiple predictor variables. Fortunately, a little application of linear algebra will let us abstract away from a lot of the book-keeping details, and make multiple linear regression hardly more complicated than the simple version1. These notes will not remind you of how matrix algebra works. However, they will review some results about calculus with matrices, and about expectations and variances with vectors and matrices. Throughout, bold-faced letters will denote matrices, as a as opposed to a scalar a. 1 Least Squares in Matrix Form Our data consists of n paired observations of the predictor variable X and the response variable Y , i.e., (x1; y1);::: (xn; yn).
    [Show full text]
  • Understanding Linear and Logistic Regression Analyses
    EDUCATION • ÉDUCATION METHODOLOGY Understanding linear and logistic regression analyses Andrew Worster, MD, MSc;*† Jerome Fan, MD;* Afisi Ismaila, MSc† SEE RELATED ARTICLE PAGE 105 egression analysis, also termed regression modeling, come). For example, a researcher could evaluate the poten- Ris an increasingly common statistical method used to tial for injury severity score (ISS) to predict ED length-of- describe and quantify the relation between a clinical out- stay by first producing a scatter plot of ISS graphed against come of interest and one or more other variables. In this ED length-of-stay to determine whether an apparent linear issue of CJEM, Cummings and Mayes used linear and lo- relation exists, and then by deriving the best fit straight line gistic regression to determine whether the type of trauma for the data set using linear regression carried out by statis- team leader (TTL) impacts emergency department (ED) tical software. The mathematical formula for this relation length-of-stay or survival.1 The purpose of this educa- would be: ED length-of-stay = k(ISS) + c. In this equation, tional primer is to provide an easily understood overview k (the slope of the line) indicates the factor by which of these methods of statistical analysis. We hope that this length-of-stay changes as ISS changes and c (the “con- primer will not only help readers interpret the Cummings stant”) is the value of length-of-stay when ISS equals zero and Mayes study, but also other research that uses similar and crosses the vertical axis.2 In this hypothetical scenario, methodology.
    [Show full text]
  • Linear, Ridge Regression, and Principal Component Analysis
    Linear, Ridge Regression, and Principal Component Analysis Linear, Ridge Regression, and Principal Component Analysis Jia Li Department of Statistics The Pennsylvania State University Email: [email protected] http://www.stat.psu.edu/∼jiali Jia Li http://www.stat.psu.edu/∼jiali Linear, Ridge Regression, and Principal Component Analysis Introduction to Regression I Input vector: X = (X1, X2, ..., Xp). I Output Y is real-valued. I Predict Y from X by f (X ) so that the expected loss function E(L(Y , f (X ))) is minimized. I Square loss: L(Y , f (X )) = (Y − f (X ))2 . I The optimal predictor ∗ 2 f (X ) = argminf (X )E(Y − f (X )) = E(Y | X ) . I The function E(Y | X ) is the regression function. Jia Li http://www.stat.psu.edu/∼jiali Linear, Ridge Regression, and Principal Component Analysis Example The number of active physicians in a Standard Metropolitan Statistical Area (SMSA), denoted by Y , is expected to be related to total population (X1, measured in thousands), land area (X2, measured in square miles), and total personal income (X3, measured in millions of dollars). Data are collected for 141 SMSAs, as shown in the following table. i : 1 2 3 ... 139 140 141 X1 9387 7031 7017 ... 233 232 231 X2 1348 4069 3719 ... 1011 813 654 X3 72100 52737 54542 ... 1337 1589 1148 Y 25627 15389 13326 ... 264 371 140 Goal: Predict Y from X1, X2, and X3. Jia Li http://www.stat.psu.edu/∼jiali Linear, Ridge Regression, and Principal Component Analysis Linear Methods I The linear regression model p X f (X ) = β0 + Xj βj .
    [Show full text]
  • In This Segment, We Discuss a Little More the Mean Squared Error
    MITOCW | MITRES6_012S18_L20-04_300k In this segment, we discuss a little more the mean squared error. Consider some estimator. It can be any estimator, not just the sample mean. We can decompose the mean squared error as a sum of two terms. Where does this formula come from? Well, we know that for any random variable Z, this formula is valid. And if we let Z be equal to the difference between the estimator and the value that we're trying to estimate, then we obtain this formula here. The expected value of our random variable Z squared is equal to the variance of that random variable plus the square of its mean. Let us now rewrite these two terms in a more suggestive way. We first notice that theta is a constant. When you add or subtract the constant from a random variable, the variance does not change. So this term is the same as the variance of theta hat. This quantity here, we will call it the bias of the estimator. It tells us whether theta hat is systematically above or below than the unknown parameter theta that we're trying to estimate. And using this terminology, this term here is just equal to the square of the bias. So the mean squared error consists of two components, and these capture different aspects of an estimator's performance. Let us see what they are in a concrete setting. Suppose that we're estimating the unknown mean of some distribution, and that our estimator is the sample mean. In this case, the mean squared error is the variance, which we know to be sigma squared over n, plus the bias term.
    [Show full text]
  • Simple Linear Regression with Least Square Estimation: an Overview
    Aditya N More et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 7 (6) , 2016, 2394-2396 Simple Linear Regression with Least Square Estimation: An Overview Aditya N More#1, Puneet S Kohli*2, Kshitija H Kulkarni#3 #1-2Information Technology Department,#3 Electronics and Communication Department College of Engineering Pune Shivajinagar, Pune – 411005, Maharashtra, India Abstract— Linear Regression involves modelling a relationship amongst dependent and independent variables in the form of a (2.1) linear equation. Least Square Estimation is a method to determine the constants in a Linear model in the most accurate way without much complexity of solving. Metrics where such as Coefficient of Determination and Mean Square Error is the ith value of the sample data point determine how good the estimation is. Statistical Packages is the ith value of y on the predicted regression such as R and Microsoft Excel have built in tools to perform Least Square Estimation over a given data set. line The above equation can be geometrically depicted by Keywords— Linear Regression, Machine Learning, Least Squares Estimation, R programming figure 2.1. If we draw a square at each point whose length is equal to the absolute difference between the sample data point and the predicted value as shown, each of the square would then represent the residual error in placing the I. INTRODUCTION regression line. The aim of the least square method would Linear Regression involves establishing linear be to place the regression line so as to minimize the sum of relationships between dependent and independent variables.
    [Show full text]
  • Application of General Linear Models (GLM) to Assess Nodule Abundance Based on a Photographic Survey (Case Study from IOM Area, Pacific Ocean)
    minerals Article Application of General Linear Models (GLM) to Assess Nodule Abundance Based on a Photographic Survey (Case Study from IOM Area, Pacific Ocean) Monika Wasilewska-Błaszczyk * and Jacek Mucha Department of Geology of Mineral Deposits and Mining Geology, Faculty of Geology, Geophysics and Environmental Protection, AGH University of Science and Technology, 30-059 Cracow, Poland; [email protected] * Correspondence: [email protected] Abstract: The success of the future exploitation of the Pacific polymetallic nodule deposits depends on an accurate estimation of their resources, especially in small batches, scheduled for extraction in the short term. The estimation based only on the results of direct seafloor sampling using box corers is burdened with a large error due to the long sampling interval and high variability of the nodule abundance. Therefore, estimations should take into account the results of bottom photograph analyses performed systematically and in large numbers along the course of a research vessel. For photographs taken at the direct sampling sites, the relationship linking the nodule abundance with the independent variables (the percentage of seafloor nodule coverage, the genetic types of nodules in the context of their fraction distribution, and the degree of sediment coverage of nodules) was determined using the general linear model (GLM). Compared to the estimates obtained with a simple Citation: Wasilewska-Błaszczyk, M.; linear model linking this parameter only with the seafloor nodule coverage, a significant decrease Mucha, J. Application of General in the standard prediction error, from 4.2 to 2.5 kg/m2, was found. The use of the GLM for the Linear Models (GLM) to Assess assessment of nodule abundance in individual sites covered by bottom photographs, outside of Nodule Abundance Based on a direct sampling sites, should contribute to a significant increase in the accuracy of the estimation of Photographic Survey (Case Study nodule resources.
    [Show full text]
  • The Simple Linear Regression Model
    The Simple Linear Regression Model Suppose we have a data set consisting of n bivariate observations {(x1, y1),..., (xn, yn)}. Response variable y and predictor variable x satisfy the simple linear model if they obey the model yi = β0 + β1xi + ǫi, i = 1,...,n, (1) where the intercept and slope coefficients β0 and β1 are unknown constants and the random errors {ǫi} satisfy the following conditions: 1. The errors ǫ1, . , ǫn all have mean 0, i.e., µǫi = 0 for all i. 2 2 2 2. The errors ǫ1, . , ǫn all have the same variance σ , i.e., σǫi = σ for all i. 3. The errors ǫ1, . , ǫn are independent random variables. 4. The errors ǫ1, . , ǫn are normally distributed. Note that the author provides these assumptions on page 564 BUT ORDERS THEM DIFFERENTLY. Fitting the Simple Linear Model: Estimating β0 and β1 Suppose we believe our data obey the simple linear model. The next step is to fit the model by estimating the unknown intercept and slope coefficients β0 and β1. There are various ways of estimating these from the data but we will use the Least Squares Criterion invented by Gauss. The least squares estimates of β0 and β1, which we will denote by βˆ0 and βˆ1 respectively, are the values of β0 and β1 which minimize the sum of errors squared S(β0, β1): n 2 S(β0, β1) = X ei i=1 n 2 = X[yi − yˆi] i=1 n 2 = X[yi − (β0 + β1xi)] i=1 where the ith modeling error ei is simply the difference between the ith value of the response variable yi and the fitted/predicted valuey ˆi.
    [Show full text]
  • Minimum Mean Squared Error Model Averaging in Likelihood Models
    Statistica Sinica 26 (2016), 809-840 doi:http://dx.doi.org/10.5705/ss.202014.0067 MINIMUM MEAN SQUARED ERROR MODEL AVERAGING IN LIKELIHOOD MODELS Ali Charkhi1, Gerda Claeskens1 and Bruce E. Hansen2 1KU Leuven and 2University of Wisconsin, Madison Abstract: A data-driven method for frequentist model averaging weight choice is developed for general likelihood models. We propose to estimate the weights which minimize an estimator of the mean squared error of a weighted estimator in a local misspecification framework. We find that in general there is not a unique set of such weights, meaning that predictions from multiple model averaging estimators might not be identical. This holds in both the univariate and multivariate case. However, we show that a unique set of empirical weights is obtained if the candidate models are appropriately restricted. In particular a suitable class of models are the so-called singleton models where each model only includes one parameter from the candidate set. This restriction results in a drastic reduction in the computational cost of model averaging weight selection relative to methods which include weights for all possible parameter subsets. We investigate the performance of our methods in both linear models and generalized linear models, and illustrate the methods in two empirical applications. Key words and phrases: Frequentist model averaging, likelihood regression, local misspecification, mean squared error, weight choice. 1. Introduction We study a focused version of frequentist model averaging where the mean squared error plays a central role. Suppose we have a collection of models S 2 S to estimate a population quantity µ, this is the focus, leading to a set of estimators fµ^S : S 2 Sg.
    [Show full text]