Factor Analysis
Total Page:16
File Type:pdf, Size:1020Kb
NCSS Statistical Software NCSS.com Chapter 420 Factor Analysis Introduction Factor Analysis (FA) is an exploratory technique applied to a set of observed variables that seeks to find underlying factors (subsets of variables) from which the observed variables were generated. For example, an individual’s response to the questions on a college entrance test is influenced by underlying variables such as intelligence, years in school, age, emotional state on the day of the test, amount of practice taking tests, and so on. The answers to the questions are the observed variables. The underlying, influential variables are the factors. Factor analysis is carried out on the correlation matrix of the observed variables. A factor is a weighted average of the original variables. The factor analyst hopes to find a few factors from which the original correlation matrix may be generated. Usually, the goal of factor analysis is to aid data interpretation. The factor analyst hopes to identify each factor as representing a specific theoretical factor. Therefore, many of the reports from factor analysis are designed to aid in the interpretation of the factors. Another goal of factor analysis is to reduce the number of variables. The analyst hopes to reduce the interpretation of a 200-question test to the study of 4 or 5 factors. One of the most subtle tasks in factor analysis is determining the appropriate number of factors. Factor analysis has an infinite number of solutions. If a solution contains two factors, these may be rotated to form a new solution that does just as good a job at reproducing the correlation matrix. Hence, one of the biggest complaints of factor analysis is that the solution is not unique. Two researchers can find two different sets of factors that are interpreted quite differently yet fit the original data equally well. NCSS provides the principal axis method of factor analysis. The results may be rotated using varimax or quartimax rotation. The factor scores may be stored for further analysis. Many books are devoted to factor analysis. We suggest you obtain a book on the subject from an author in your own field. An excellent introduction to the subject is provided by Tabachnick (1989). 420-1 © NCSS, LLC. All Rights Reserved. NCSS Statistical Software NCSS.com Factor Analysis Technical Details Mathematical Development This section will document the basic formulas used by NCSS in performing a factor analysis. The following table lists many of the matrices that are used in the discussion to follow. Label Matrix Name Size Description R Correlation pxp Matrix of correlations between each pair of variables. X Data Nxp Observed data matrix with N rows (observations) and p columns (variables). Z Standardized data Nxp Matrix of standardized data. The standardization of each variable is made by subtracting its mean and dividing by its standard deviation. A Factor loading pxm Matrix of correlations between the original variables and the factors. Also represents the contribution of each factor in estimating the original variables. L Eigenvalue mxm Diagonal matrix of eigenvalues. Only the first m eigenvalues are considered. V Eigenvector pxm Matrix of eigenvectors. Only the first m columns of this matrix are used. B Factor-score coefficients pxm Matrix of regression weights used to construct the factor scores from the original variables. U Uniqueness pxp Matrix of uniqueness values. F Factor score Nxm Matrix of factor scores. For each observation in the original data, the values of each of the retained factors are estimated. These are the factor scores. The principal-axis method is used by NCSS to solve the factor analysis problem. Factor analysis assumes the following partition of the correlation matrix, R: R = AA ′+U The principal-axis method proceeds according to the following steps: 1. Estimate U from the communalities as discussed below. 2. Find L and V, the eigenvalues and eigenvectors of R-U using standard eigenvalue analysis. 3. Calculate the loading matrix as follows: 1 A =VL 2 4. Calculate the score matrix as follows: − 1 B =VL 2 5. Calculate the factor scores as follows: F = ZB Steps 1-3 may be iterated since a new U matrix may be estimated from the current loading matrix. 420-2 © NCSS, LLC. All Rights Reserved. NCSS Statistical Software NCSS.com Factor Analysis Initial Communality Estimation We close this section with a discussion of obtaining an initial value of U. NCSS uses the initial estimation of Cureton (1983), which will be outlined here. The initial communality estimates, cii, are calculated from the correlation and inverse correlation matrices as follows: p ∑ max ( r jk ) over j ≠k 1 k =1 c ii = 1 - R ii p 1 1 - ∑ kk k =1 R th where Rii is the i diagonal element of R-1 and rjk is an element of R. The value of U is then estimated by 1-cii. Missing Values and Robust Estimation Missing values and robust estimation are done the same way as in principal components analysis. Refer to that chapter for details. How Many Factors Several methods have been proposed for determining the number of factors that should be kept for further analysis. Several of these methods will now be discussed. However, remember that important information about possible outliers and linear dependencies may be determined from the factors associated with the relatively small eigenvalues, so these should be investigated as well. Kaiser (1960) proposed dropping factors whose eigenvalues are less than one since these provide less information than is provided by a single variable. Jolliffe (1972) feels that Kaiser’s criterion is too large. He suggests using a cutoff on the eigenvalues of 0.7 when correlation matrices are analyzed. Other authors note that if the largest eigenvalue is close to one, then holding to a cutoff of one may cause useful factors to be dropped. However, if the largest factors are several times larger than one, then those near one may be reasonably dropped. Cattell (1966) documented the scree graph, which will be described later in this chapter. Studying this chart is probably the most popular method for determining the number of factors, but it is subjective, causing different people to analyze the same data with different results. Another criterion is to preset a certain percentage of the variation that must be accounted for and then keep enough factors so that this variation is achieved. Usually, however, this cutoff percentage is used as a lower limit. That is, if the designated number of factors do not account for at least 50% of the variance, then the whole analysis is aborted. Varimax and Quartimax Rotation Factor analysis finds a set of dimensions (or coordinates) in a subspace of the space defined by the set of variables. These coordinates are represented as axes. They are orthogonal (perpendicular) to one another. For example, suppose you analyze three variables that are represented in three-dimensional space. Each variable becomes one axis. Now suppose that the data lie near a two-dimensional plane within the three dimensions. A factor analysis of this data should uncover two factors that would account for the two dimensions. You may rotate the axes of this two-dimensional plane while keeping the 90-degree angle between them, just as the blades of a helicopter propeller rotate yet maintain the same angles among themselves. The hope is that rotating the axes will improve your ability to interpret the “meaning” of each factor. Many different types of rotation have been suggested. Most of them were developed for use in factor analysis. NCSS provides two orthogonal rotation options: varimax and quartimax. 420-3 © NCSS, LLC. All Rights Reserved. NCSS Statistical Software NCSS.com Factor Analysis Varimax Rotation Varimax rotation is the most popular orthogonal rotation technique. In this technique, the axes are rotated to maximize the sum of the variances of the squared loadings within each column of the loadings matrix. Maximizing according to this criterion forces the loadings to be either large or small. The hope is that by rotating the factors, you will obtain new factors that are each highly correlated with only a few of the original variables. This simplifies the interpretation of the factor to a consideration of these two or three variables. Another way of stating the goal of varimax rotation is that it clusters the variables into groups; each “group” is actually a new factor. Since varimax seeks to maximize a specific criterion, it produces a unique solution (except for differences in sign). This has added to its popularity. Let the matrix G = {gij} represent the rotated factors. The goal of varimax rotation is to maximize the quantity: p p p ∑ g 4 - ∑ g 2 k ij ij i = 1 i = 1 Q1 = ∑ j = 1 p This equation gives the raw varimax rotation. This rotation has the disadvantage of not spreading the variance very evenly among the new factors. Instead, it tends to form one large factor followed by many small ones. To correct this, NCSS uses the normalized-varimax rotation. The quantity maximized in this case is: p g 4 p g 2 ij ij k p ∑ - ∑ i =1 c i i =1 c i Q N = ∑ j =1 p 2 where ci is the square root of the communality of variable i. Quartimax Rotation Quartimax rotation is similar to varimax rotation except that the rows of G are maximized rather than the columns of G. This rotation is more likely to produce a “general” factor than will varimax. Often, the results are quite similar. The quantity maximized for the quartimax is: p g 4 ij ∑ k i = 1 c i Q N = ∑ j = 1 p 420-4 © NCSS, LLC.