DOCSLIB.ORG

A New Approach of Principal Component Regression Estimator with Applications to Collinear Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A New Approach of Principal Component Regression Estimator with Applications to Collinear Data International Journal of Engineering Research and Technology. ISSN 0974-3154, Volume 13, Number 7 (2020), pp. 1616-1622 © International Research Publication House. http://www.irphouse.com A New Approach of Principal Component Regression Estimator with Applications to Collinear Data 1Kayode Ayinde, 2Adewale F. Lukman, 1Olusegun. O. Alabi and 1Hamidu A. Bello 1Department of Statistics, Federal University of Technology, Akure, Nigeria. 2Department of Physical Sciences, Landmark University, Omu-Aran, Nigeria. Corresponding Author Email: [email protected] ORCID: 0000-0002-2993-6203 (Hamidu A. Bello), ORCID: 0000-0003-2881-1297 (Lukman) Abstract components (PCs) (Naes and Marten, 1988; Chatterjee and Hadi, 2006). Other methods range from this method to more In this paper, a new approach to estimating the model specialized techniques for regularization (Naes, and Marten, parameters in Principal Components (PCs) is developed. 1988). The PCR method has been proposed as alternatives to Principal Component Analysis (PCA) is a method of variable the OLS estimator when the independent assumption has not reduction that has found relevance in combating been satisfied in the analysis (Massy, 1965; Lee et al., 2015). multicollinearity problem in Multiple Linear Regression Models. In this paper, a new approach to estimating the model parameters in Principal Components (PCs) is developed The 2. LITERATURE REVIEW method requires using the PCs as regressors to predict the dependent variable and further utilizing the predicted variable Consider the standard linear regression model defined as: as a dependent variable on the original explanatory variables y = β + β x + β x + ...+ β x + u i = 1,2,...,n (2.1) in an Ordinary Least Square (OLS) regression. The resulting i 0 1 i1 2 i2 p ip i parameter estimates are the same as the estimates from the and defined in matrix form as usual Principal Components Regression Estimator. The sampling properties of the new estimator are proved to be y = Xβ + u (2.2) same as the existing ones. The new approach is simpler and easier to use. Real-life data sets are used to illustrate both the where y is (n ×1) vector for the observations of dependent conventional and the developed method. variable; X is (n × p) matrix of the observations of Keywords: Multiple Linear Regression Models, explanatory variables, β is ( p ×1) vector of regression Multicollinearity, Principal Components, Ordinary Least Square coefficients u is (n ×1) vector of the error terms assumed to multivariate normal distribution with mean zero and variance, 2 σ I n . Thus, the parameters of the model can be estimated 1. INTRODUCTION using the use of Ordinary Least Square (OLS) estimator Violation of independence of explanatory variables in obtained as: multiple linear regression models results in multicollinearity −1 problem (Gujarati, 2005). Literature has reported the use of βˆ = X 1 X X 1 y (2.3) OLS ( ) the Ordinary Least Square (OLS) Estimator in the presence of multicollinearity with consequences. It does produce unbiased The unbiased variance–covariance matrices of the OLS but inefficient estimates of the model parameters, estimates of estimator can also be given as: regression coefficients having wrong signs and high standard −1 Var (βˆ ) = σˆ 2 X 1 X (2.4) deviation, and the parameter estimates and their standard OLS ( ) errors become extremely sensitive to slight changes in the data n points (Chatterjee and Hadi, 2006; Lukman et al., 2018; ()y − yˆ 2 Ayinde et al., 2018). This consequence tends to inflate the ∑ i i where σˆ 2 = i=1 , (Maddalla, 1988; Kibra, 2003). estimated variance of the predicted values (Fomby et al., n − p 1984; Maddalla, 1988; Ayinde et al., 2015). Several other techniques are in use to combat this problem, Lukman et al. (2020) reported the OLS estimator as Best and Principal Components Analysis (PCA) is one. It is a Linear Unbiased Estimator (BLUE) provided there is no multivariate technique developed by Hotelling (1933) for multicollinearity among the explanatory variables is not explaining a set of correlated variables by a reduced number perfect. However, the estimator is still not efficient when there of uncorrelated ones with maximum variances called Principal is multicollinearity (Chatterjee and Price, 1977; Gujarati, 1616 International Journal of Engineering Research and Technology. ISSN 0974-3154, Volume 13, Number 7 (2020), pp. 1616-1622 © International Research Publication House. http://www.irphouse.com 2005; Lukman et al., 2019; Lukman et al., 2020). Among the n ⎛ _ ⎞⎛ _ ⎞ estimators commonly used to address this problem are the ∑⎜ xij − x j ⎟⎜ xik − x k ⎟ where i=1 ⎝ ⎠⎝ ⎠ , estimator based on Principal Component Regression rjk = 2 2 developed by Massy (1965), the Ridge Regression (RRE) ⎛ n ⎛ _ ⎞ ⎞⎛ n ⎛ _ ⎞ ⎞ ⎜ x − x j ⎟⎜ x − x k ⎟ Estimator (Hoerl and Kennard, 1970), estimator based on ⎜∑⎜ ij ⎟ ⎟⎜∑⎜ ik ⎟ ⎟ i=1 ⎝ ⎠ i=1 ⎝ ⎠ Partial Least Square (PLS) Regression (Wold, 1985), and Liu ⎝ ⎠⎝ ⎠ estimator (Liu, 1993) and recently, the KL estimator by Kibria j=1, 2,3, …, p; k= 1, 2,3, …, p and Lukman (2020). Estimator based on Principal Component Regression requires iii. Calculate the Eigenvalues and hence the Eigenvectors of reducing the explanatory variables into a smaller number of the correlation matrices, say W. These become the summary variables often called Principal Components (or coefficients (weights) that are attached to the standardized factors) that explain most of the variations in the data. explanatory variables. Thus, According to Chatterjee and Price (1977), they summarized the algorithm of the procedures using the correlation matrices ⎡w11 w12 w13 w14 . w1p ⎤ as follows: ⎢w w w w . w ⎥ ⎢ 21 22 23 24 2 p ⎥ i. Standardize both the response and the explanatory ⎢w31 w32 w33 w34 . w3 p ⎥ variables: The standardized variables respectively become: ⎢ ⎥ ⎢w41 w42 w43 w44 . w4 p ⎥ − W = (2.8) 1 yi − y ⎢ . ⎥ yi = (2.5) ⎢ ⎥ s y ⎢ . ⎥ 2 ⎢ . ⎥ n n ⎛ _ ⎞ ⎢ ⎥ y ⎜ y − y⎟ _ ∑ i ∑ i ⎢w1p w2 p w3 p w4 p . wpp ⎥ i=1 i=1 ⎝ ⎠ ⎣ ⎦ where y = and sy = n n =1 − iv. Obtain the Principal Components (PCs) using the xij − x j x1 = (2.6) linear equation given as: ij s x j p PC = w x1 , i = 1, 2,…,n; j=1,2,…,p (2.9) 2 ij ∑ kj ik n n ⎛ _ ⎞ k=1 x x − x j _ ∑ ij ∑⎜ ij ⎟ i=1 i=1 ⎝ ⎠ These PCs are p orthogonal variables. where x j = and s = x j n n =1 for j = 1,2, 3,…, p v. Determine the number of PCs to use: There is a need to determine the number of PCs to use. Since there are p of them ii. Obtain the correlation matrices of the explanatory and using all of them will produce the same results with OLS. variables, say R. Thus, Hence a particular number of them has to be chosen. Kaiser (1960) suggested the use of PCs whose Eigenvalue is more ⎡ 1 r r r . r ⎤ 12 13 14 1p than one. Constantin (2014) suggested using any component ⎢r 1 r r . r ⎥ ⎢ 12 23 24 2 p ⎥ that accounts for at least 5% or 10% of the total variance. Some other authors emphasized that components should be ⎢r13 r23 1 r34 . r3 p ⎥ ⎢ ⎥ selected enough so that the cumulative per cent of variance r r r 1 . r R = ⎢ 14 24 34 4 p ⎥ (2.7) accounted for is so high, between 80% and 90% (Naes and ⎢ . ⎥ Marten, 1988; Ali and Robert, 1998; Abdul–Wahab et al., ⎢ ⎥ 2005). ⎢ . ⎥ ⎢ . ⎥ vi. Run the OLS regression (without intercept) of the ⎢ ⎥ standardized dependent variable on the selected m PCs. Thus, ⎢r1p r2 p r3 p r4 p . 1 ⎥ ⎣ ⎦ 1 yi = f (PC1 , PC 2 ,..., PC m ) (2.10) and obtain the estimates of the regression coefficients ) ) ) as γ 1 , γ 2 ,…,γ m . 1617 International Journal of Engineering Research and Technology. ISSN 0974-3154, Volume 13, Number 7 (2020), pp. 1616-1622 © International Research Publication House. http://www.irphouse.com vii. Obtain the true regression coefficients of the linear Theorem 1: The Principal Component Regression Estimator model (2.1 or 2.2) based on Principal Component Regression of a Linear Regression Model y = Xβ + u , where y is method using: (n ×1) vector for the observations of dependent variable; X m ) s y ⎡ ⎤ is (n × p) matrix of the observations of explanatory β = w γ) j= 1,2, …,p (2.11) PCR j ⎢∑ jt t ⎥ s variables, β is ( p ×1) vector of regression coefficients u is x j ⎣ t=1 ⎦ (n ×1) vector of the error terms assumed to multivariate Also, the intercept can be obtained using: 2 normal distribution with mean zero and variance,σ I n which ) _ p ) − β = y− β x j (2.12) uses r Principal Components, r ≤ p , can be obtained as, PCR0 ∑ PCRj j=1 ˆ 1 −1 1 β PCR = (X X ) X yˆ r , where yˆ r is the predicted variable 1 Alternatively, re-consider model (2.2) again. Then, X X when y is regressed on the r principal components. becomes a ( p × p) matrix whose Eigenvectors result into Proof: ( p × p) orthogonal matrixT = (t1 ,t2 ,...t p ) = [Tr ,T p−r ]. Recall that the estimator of the parameters of the Principal Utilizing this and following the study of Ozkale (2009), Component Regression that uses r Principal Components is Batah et al. (2009), and Chang and Yang (2012), (2.2) can be given in (2.14) as: written as: ˆ 1 −1 1 1 1 −1 1 1 1 1 1 α PCR = (Z r Z r )(Z r y = Tr X XTr )Tr X y (3.1) y = XTT β + u = XTrTr β + XT p−rT p−=r β + u (2.13) = Z rα r + Z p−rα p−r + u Then, the predicted value of dependent variable becomes: 1 1 −1 1 1 1 yˆ r = Z r αˆ PCR = XTr (Tr X XTr ) Tr X y (3.2 where Z r = XTr , α r = Tr β , Z p−r = XTp−r , 1 α p −r = T p −r β .
Load more

Recommended publications
  • Multiple Discriminant Analysis
    1 MULTIPLE DISCRIMINANT ANALYSIS DR. HEMAL PANDYA 2 INTRODUCTION • The original dichotomous discriminant analysis was developed by Sir Ronald Fisher in 1936. • Discriminant Analysis is a Dependence technique. • Discriminant Analysis is used to predict group membership. • This technique is used to classify individuals/objects into one of alternative groups on the basis of a set of predictor variables (Independent variables) . • The dependent variable in discriminant analysis is categorical and on a nominal scale, whereas the independent variables are either interval or ratio scale in nature. • When there are two groups (categories) of dependent variable, it is a case of two group discriminant analysis. • When there are more than two groups (categories) of dependent variable, it is a case of multiple discriminant analysis. 3 INTRODUCTION • Discriminant Analysis is applicable in situations in which the total sample can be divided into groups based on a non-metric dependent variable. • Example:- male-female - high-medium-low • The primary objective of multiple discriminant analysis are to understand group differences and to predict the likelihood that an entity (individual or object) will belong to a particular class or group based on several independent variables. 4 ASSUMPTIONS OF DISCRIMINANT ANALYSIS • NO Multicollinearity • Multivariate Normality • Independence of Observations • Homoscedasticity • No Outliers • Adequate Sample Size • Linearity (for LDA) 5 ASSUMPTIONS OF DISCRIMINANT ANALYSIS • The assumptions of discriminant analysis are as under. The analysis is quite sensitive to outliers and the size of the smallest group must be larger than the number of predictor variables. • Non-Multicollinearity: If one of the independent variables is very highly correlated with another, or one is a function (e.g., the sum) of other independents, then the tolerance value for that variable will approach 0 and the matrix will not have a unique discriminant solution.
    [Show full text]
  • An Overview and Application of Discriminant Analysis in Data Analysis
    IOSR Journal of Mathematics (IOSR-JM) e-ISSN: 2278-5728, p-ISSN: 2319-765X. Volume 11, Issue 1 Ver. V (Jan - Feb. 2015), PP 12-15 www.iosrjournals.org An Overview and Application of Discriminant Analysis in Data Analysis Alayande, S. Ayinla1 Bashiru Kehinde Adekunle2 Department of Mathematical Sciences College of Natural Sciences Redeemer’s University Mowe, Redemption City Ogun state, Nigeria Department of Mathematical and Physical science College of Science, Engineering &Technology Osun State University Abstract: The paper shows that Discriminant analysis as a general research technique can be very useful in the investigation of various aspects of a multi-variate research problem. It is sometimes preferable than logistic regression especially when the sample size is very small and the assumptions are met. Application of it to the failed industry in Nigeria shows that the derived model appeared outperform previous model build since the model can exhibit true ex ante predictive ability for a period of about 3 years subsequent. Keywords: Factor Analysis, Multiple Discriminant Analysis, Multicollinearity I. Introduction In different areas of applications the term "discriminant analysis" has come to imply distinct meanings, uses, roles, etc. In the fields of learning, psychology, guidance, and others, it has been used for prediction (e.g., Alexakos, 1966; Chastian, 1969; Stahmann, 1969); in the study of classroom instruction it has been used as a variable reduction technique (e.g., Anderson, Walberg, & Welch, 1969); and in various fields it has been used as an adjunct to MANOVA (e.g., Saupe, 1965; Spain & D'Costa, Note 2). The term is now beginning to be interpreted as a unified approach in the solution of a research problem involving a com-parison of two or more populations characterized by multi-response data.
    [Show full text]
  • S41598-018-25035-1.Pdf
    www.nature.com/scientificreports OPEN An Innovative Approach for The Integration of Proteomics and Metabolomics Data In Severe Received: 23 October 2017 Accepted: 9 April 2018 Septic Shock Patients Stratifed for Published: xx xx xxxx Mortality Alice Cambiaghi1, Ramón Díaz2, Julia Bauzá Martinez2, Antonia Odena2, Laura Brunelli3, Pietro Caironi4,5, Serge Masson3, Giuseppe Baselli1, Giuseppe Ristagno 3, Luciano Gattinoni6, Eliandre de Oliveira2, Roberta Pastorelli3 & Manuela Ferrario 1 In this work, we examined plasma metabolome, proteome and clinical features in patients with severe septic shock enrolled in the multicenter ALBIOS study. The objective was to identify changes in the levels of metabolites involved in septic shock progression and to integrate this information with the variation occurring in proteins and clinical data. Mass spectrometry-based targeted metabolomics and untargeted proteomics allowed us to quantify absolute metabolites concentration and relative proteins abundance. We computed the ratio D7/D1 to take into account their variation from day 1 (D1) to day 7 (D7) after shock diagnosis. Patients were divided into two groups according to 28-day mortality. Three diferent elastic net logistic regression models were built: one on metabolites only, one on metabolites and proteins and one to integrate metabolomics and proteomics data with clinical parameters. Linear discriminant analysis and Partial least squares Discriminant Analysis were also implemented. All the obtained models correctly classifed the observations in the testing set. By looking at the variable importance (VIP) and the selected features, the integration of metabolomics with proteomics data showed the importance of circulating lipids and coagulation cascade in septic shock progression, thus capturing a further layer of biological information complementary to metabolomics information.
    [Show full text]
  • Multicollinearity Diagnostics in Statistical Modeling and Remedies to Deal with It Using SAS
    Multicollinearity Diagnostics in Statistical Modeling and Remedies to deal with it using SAS Harshada Joshi Session SP07 - PhUSE 2012 www.cytel.com 1 Agenda • What is Multicollinearity? • How to Detect Multicollinearity? Examination of Correlation Matrix Variance Inflation Factor Eigensystem Analysis of Correlation Matrix • Remedial Measures Ridge Regression Principal Component Regression www.cytel.com 2 What is Multicollinearity? • Multicollinearity is a statistical phenomenon in which there exists a perfect or exact relationship between the predictor variables. • When there is a perfect or exact relationship between the predictor variables, it is difficult to come up with reliable estimates of their individual coefficients. • It will result in incorrect conclusions about the relationship between outcome variable and predictor variables. www.cytel.com 3 What is Multicollinearity? Recall that the multiple linear regression is Y = Xβ + ϵ; Where, Y is an n x 1 vector of responses, X is an n x p matrix of the predictor variables, β is a p x 1 vector of unknown constants, ϵ is an n x 1 vector of random errors, 2 with ϵi ~ NID (0, σ ) www.cytel.com 4 What is Multicollinearity? • Multicollinearity inflates the variances of the parameter estimates and hence this may lead to lack of statistical significance of individual predictor variables even though the overall model may be significant. • The presence of multicollinearity can cause serious problems with the estimation of β and the interpretation. www.cytel.com 5 How to detect Multicollinearity? 1. Examination of Correlation Matrix 2. Variance Inflation Factor (VIF) 3. Eigensystem Analysis of Correlation Matrix www.cytel.com 6 How to detect Multicollinearity? 1.
    [Show full text]
  • Best Subset Selection for Eliminating Multicollinearity
    Journal of the Operations Research Society of Japan ⃝c The Operations Research Society of Japan Vol. 60, No. 3, July 2017, pp. 321{336 BEST SUBSET SELECTION FOR ELIMINATING MULTICOLLINEARITY Ryuta Tamura Ken Kobayashi Yuichi Takano Tokyo University of Fujitsu Laboratories Ltd. Senshu University Agriculture and Technology Ryuhei Miyashiro Kazuhide Nakata Tomomi Matsui Tokyo University of Tokyo Institute of Tokyo Institute of Agriculture and Technology Technology Technology (Received July 27, 2016; Revised December 2, 2016) Abstract This paper proposes a method for eliminating multicollinearity from linear regression models. Specifically, we select the best subset of explanatory variables subject to the upper bound on the condition number of the correlation matrix of selected variables. We first develop a cutting plane algorithm that, to approximate the condition number constraint, iteratively appends valid inequalities to the mixed integer quadratic optimization problem. We also devise a mixed integer semidefinite optimization formulation for best subset selection under the condition number constraint. Computational results demonstrate that our cutting plane algorithm frequently provides solutions of better quality than those obtained using local search algorithms for subset selection. Additionally, subset selection by means of our optimization formulation succeeds when the number of candidate explanatory variables is small. Keywords: Optimization, statistics, subset selection, multicollinearity, linear regression, mixed integer semidefinite optimization 1. Introduction Multicollinearity, which exists when two or more explanatory variables in a regression model are highly correlated, is a frequently encountered problem in multiple regression analysis [11, 14, 24]. Such an interrelationship among explanatory variables obscures their relationship with the explained variable, leading to computational instability in model es- timation.
    [Show full text]
  • On the Determination of Proper Regularization Parameter
    Universität Stuttgart Geodätisches Institut On the determination of proper regularization parameter α-weighted BLE via A-optimal design and its comparison with the results derived by numerical methods and ridge regression Bachelorarbeit im Studiengang Geodäsie und Geoinformatik an der Universität Stuttgart Kun Qian Stuttgart, Februar 2017 Betreuer: Dr.-Ing. Jianqing Cai Universität Stuttgart Prof. Dr.-Ing. Nico Sneeuw Universität Stuttgart Erklärung der Urheberschaft Ich erkläre hiermit an Eides statt, dass ich die vorliegende Arbeit ohne Hilfe Dritter und ohne Benutzung anderer als der angegebenen Hilfsmittel angefertigt habe; die aus fremden Quellen direkt oder indirekt übernommenen Gedanken sind als solche kenntlich gemacht. Die Arbeit wurde bisher in gleicher oder ähnlicher Form in keiner anderen Prüfungsbehörde vorgelegt und auch noch nicht veröffentlicht. Ort, Datum Unterschrift III Abstract In this thesis, several numerical regularization methods and the ridge regression, which can help improve improper conditions and solve ill-posed problems, are reviewed. The deter- mination of the optimal regularization parameter via A-optimal design, the optimal uniform Tikhonov-Phillips regularization (a-weighted biased linear estimation), which minimizes the trace of the mean square error matrix MSE(ˆx), is also introduced. Moreover, the comparison of the results derived by A-optimal design and results derived by numerical heuristic methods, such as L-curve, Generalized Cross Validation and the method of dichotomy is demonstrated. According to the comparison, the A-optimal design regularization parameter has been shown to have minimum trace of MSE(ˆx) and its determination has better efficiency. VII Contents List of Figures IX List of Tables XI 1 Introduction 1 2 Least Squares Method and Ill-Posed Problems 3 2.1 The Special Gauss-Markov medel .
    [Show full text]
  • Baseline and Interferent Correction by the Tikhonov Regularization Framework for Linear Least Squares Modeling
    Baseline and interferent correction by the Tikhonov Regularization framework for linear least squares modeling Joakim Skogholt1, Kristian Hovde Liland1 and Ulf Geir Indahl1 1Faculty of Science and Technology, Norwegian University of Life Sciences,˚As, Norway Abstract Spectroscopic data is usually perturbed by noise from various sources that should be re- moved prior to model calibration. After conducting a pre-processing step to eliminate un- wanted multiplicative effects (effects that scale the pure signal in a multiplicative manner), we discuss how to correct a model for unwanted additive effects in the spectra. Our approach is described within the Tikhonov Regularization (TR) framework for linear regression model building, and our focus is on ignoring the influence of non-informative polynomial trends. This is obtained by including an additional criterion in the TR problem penalizing the resulting regression coefficients away from a selected set of possibly disturbing directions in the sam- ple space. The presented method builds on the Extended Multiplicative Signal Correction (EMSC), and we compare the two approaches on several real data sets showing that the sug- gested TR-based method may improve the predictive power of the resulting model. We discuss the possibilities of imposing smoothness in the calculation of regression coefficients as well as imposing selection of wavelength regions within the TR framework. To implement TR effi- ciently in the model building, we use an algorithm that is heavily based on the singular value decomposition (SVD). Due to some favourable properties of the SVD it is possible to explore the models (including their generalized cross-validation (GCV) error estimates) associated with a large number of regularization parameter values at low computational cost.
    [Show full text]
  • Application of Principal Component Analysis (PCA) to Reduce Multicollinearity Exchange Rate Currency of Some Countries in Asia Period 2004-2014
    International Journal of Educational Methodology Volume 3, Issue 2, 75 - 83. ISSN: 2469-9632 http://www.ijem.com/ Application of Principal Component Analysis (PCA) to Reduce Multicollinearity Exchange Rate Currency of Some Countries in Asia Period 2004-2014 Sri Rahayu TeguhSugiarto* Diponegoro University, INDONESIA Brawijaya University, INDONESIA LudiroMadu Holiawati Ahmad Subagyo UPN ’Veteran’ Yogyakarta, INDONESIA UniversitasPamulang, INDONESIA GICI Business School, INDONESIA Abstract:This study aims to apply the model Principal component Analysis to reduce multicollinearity on variable currency exchange rate in eight countries in Asia against US Dollar including the Yen (Japan), Won (South Korea), Dollar (Hongkong), Yuan (China), Bath (Thailand), Rupiah (Indonesia), Ringgit (Malaysia), Dollar (Singapore). It looks at yield levels of multicolinierity which is smaller in comparison with PCA applications using multiple regression. This study used multiple regression test and PCA application to investigate the differences in multicollinearity at yield. From this research, it can be concluded that the use of PCA analysis applications can reduce multicollinearity in variables in doing research. Keywords:Principal component analysis (PCA), multiple regression, matrix var-cov, exchange rate. To cite this article: Rahayu, S.,Sugiarto, T., Madu, L., Holiawati, &Subagyo,A. (2017). Application of principal component analysis (PCA) to reduce multicollinearity exchange rate currency of some countries in asia period 2004-2014. International Journal of Educational Methodology, 3(2), 75-83.doi: 10.12973/ijem.3.2.75 Introduction In Indonesia's economy in particular, Bank of Indonesia (BI) predicts the exchange rate is in the range of Rp 13,500 to Rp 13,800. However, when seen as a point averages, the central bank predicts the rupiah will be at Rp 13,600 per US dollar by the end of 2016.
    [Show full text]
  • Year Seminars on College Students' Life-Long Learning Orientations
    Padgett, R. D., Keup, J. R., & Pascarella, E. T. (2013). The impact of first-year seminars on college students’ life-long learning orientations. Journal of Student Affairs Research and Practice, 50(2), 133–151. doi:10.1515/jsarp-2013-0011 Available at http://dx.doi.org/10.1515/jsarp-2013-0011 Innovations in Research and Scholarship Feature The Impact of First- Year Seminars on College Students’ Life-long Learning Orientations Ryan D. Padgett, Northern Kentucky University Jennifer R. Keup, University of South Carolina Ernest T. Pascarella, The University of Iowa Using longitudinal data from the Wabash National Study of Liberal Arts Education, this study measured the impact of first-year seminars on college students’ life-long learning orientations. The findings suggest that first-year seminars enhance students’ life-long learning orientations and that the effect of first-year seminars is mediated through vetted good practices. These findings provide empirical evidence that first-year sem- inars can positively impact complex learning in ways that were previously untested. Findings from the Association of American Colleges and Universities’ (AAC&U) Greater Expectations Forum on Twenty-First-Century Liberal Arts Education Practice (see Leskes & Miller, 2006) identified four learning outcomes that students should be developing through college: integrative Ryan D. Padgett, Northern Kentucky University, Senior Analyst for Co-Curricular Assessment and Research. Jennifer R. Keup, University of South Carolina, Director, National Resource Center for The First-Year Experience and Students in Transition. Ernest T. Pascarella, The University of Iowa, Mary Louise Petersen Professor of Higher Education. Correspondence concerning this article should be addressed to Padgett at [email protected].
    [Show full text]
  • Ocrep: an Optimally Conditioned Regularization for Pseudoinversion
    OCReP: An Optimally Conditioned Regularization for Pseudoinversion Based Neural Training Rossella Cancellierea, Mario Gaib, Patrick Gallinaric, Luca Rubinia aUniversity of Turin, Dep. of Computer Sciences, C.so Svizzera 185, 10149 Torino, Italy bNational Institute of Astrophysics, Astrophys. Observ. of Torino, Pino T.se (TO), Italy cLaboratory of Computer Sciences, LIP6, Univ. Pierre et Marie Curie, Paris, France Abstract In this paper we consider the training of single hidden layer neural net- works by pseudoinversion, which, in spite of its popularity, is sometimes affected by numerical instability issues. Regularization is known to be ef- fective in such cases, so that we introduce, in the framework of Tikhonov regularization, a matricial reformulation of the problem which allows us to use the condition number as a diagnostic tool for identification of instabil- ity. By imposing well-conditioning requirements on the relevant matrices, our theoretical analysis allows the identification of an optimal value for the regularization parameter from the standpoint of stability. We compare with the value derived by cross-validation for overfitting control and optimisation of the generalization performance. We test our method for both regression and classification tasks. The proposed method is quite effective in terms of predictivity, often with some improvement on performance with respect to the reference cases considered. This approach, due to analytical determina- tion of the regularization parameter, dramatically reduces the computational load required by many other techniques. arXiv:1508.06095v1 [cs.NE] 25 Aug 2015 Keywords: Regularization parameter, Condition number, Pseudoinversion, Numerical instability 1. Introduction In past decades Single Layer Feedforward Neural Networks (SLFN) train- ing was mainly accomplished by iterative algorithms involving the repetition of learning steps aimed at minimising the error functional over the space of Preprint submitted to Neural Networks August 14, 2021 network parameters.
    [Show full text]
  • DIAGNOSTICS in MULTIPLE REGRESSION the Data Are in the Form
    PubH 7405: REGRESSION ANALYSIS DIAGNOSTICS IN MULTIPLE REGRESSION The data are in the form : {( yi ; x1i , x2i ,, xki )}i=1,,n Multiple Regression Model : Y = β0 + β1x1 + β2 x2 +βk xk + ε ε ∈ N(0,σ 2 ) The error terms are identically and independently distributed as normal with a constant variance. A simple strategy for the building of a regression model consists of some, most, or all of the following five steps or phases: (1) Data collection and preparation, (2) Preliminary model investigation, (3) Reduction of the predictor variables, (4) Model refinement and selection, and (5) Model validation Let say we finished step #3; we tentatively have a good model: it’s time for some fine tuning! Diagnostics to identify possible violations of model’s assumptions are often focused on these “major issues”: (1) Non-linearity, (2) Outlying & influential cases, (3) Multi-collinearity, (4) Non-constant variance & (5) Non-independent errors. DETECTION OF NON-LINEARITY A limitation of the usual residual plots (say, residuals against values of a predictor variable): they may not show the nature of the “additional contribution” of a predictor variable to those by other variables already in the model. For example, we consider a multiple regression model with 2 independent variables X1 and X2; is the relationship between Y and X1 linear? “Added-variable plots” (also called “partial regression plots” or “adjusted variable plots”) are refined residual plots that provide graphic information about the marginal importance of a predictor variable given the other variables already in the model. ADDED-VARIABLE PLOTS In an added-variable plot, both the response variable Y and the predictor variable under investigation (say, X1) are both regressed against the other predictor variables already in the regression model and the residuals are obtained for each.
    [Show full text]
  • SUSAM : a Teaching Tool for Multicollinearity Analysis
    ICOTS 3, 1990: Gianfranco Galmacci and Maria Pannone SUSAM : A Teaching Tool for Multicollinearity Analysis Gianhco Galmacci and Maria A Pannone - Perugia, Italy 1. Introduction The teaching of statistics requires constant integration of theoretical and practical work. In the teaching of advanced statistical methodology, this asks for strong commit- ment on the part of the teacher, who is expected not only to make students acquire knowledge of methods, but also to generate afeel for data and the competence to choose the most suitable techniques for obtaining the information needed, which are the essential abilities of an expert statistician. Technological development makes it possible nowadays to delegate part of this work to specialised software systems to be used as tutorial aids in the teaching of specific subjects. To date, much research has been carried out in this direction, some of which has achieved extremely successful results. Because of the problems involved, however, the computer assisted teaching is still in an experimental phase and it is necessary to identify the most correct didactic approaches for devising the aids. Our study is meant as a contribution in this direction. It aims at verifying whether a software system which operates as a tutor for practical work in a course on statistics can be used as a valid aid to the theoretical work carried out by the teacher. In developing a system with the given tutorial features, we referred to expert system techniques to obtain a versatile environment. But the theoretical subject, i.e. the statistical domain, had to be quite limited, so that the expert system would have no difficulty in dealing with it; at the same time, it had to be complex enough from a statistical point of view to serve as a good test for checking the validity of this teaching approach.
    [Show full text]