Factor Analysis: illustrations/graphics/tables using painters One way to do pairwise PLOTs for the painters4a data is to use ‘splom’ painters4a is just the first four columns of painters (MASS lib.) plus a binary variable that distinguishes ‘School D’ from the other Schools; see painter documentation. Where the commands used here were: > splom(~painters4a[painters4a[,5]==1,1:4]) > par(new=T) > splom(~painters4a[painters4a[,5]==1,1:4]) > mtext("Splom of painters4a data, school D points as '+'",line=3)

Splom plot of painters4a data, school D points as '+' + + ++ + 6 310 4 155 6 + + + 155 + + + + + + + + + 4 10 + + + 3 Expression 3 2 5 1

+++ + + + + + 00 1 52 3 0 + + + + 18 + + 16 10 17 15 18 15 + + + + 17 + +

10 + ++ + + 16 Colour 16 + + +

5 + + 15 +

+ + + + 14 0 15 5 16 14 0 + + + 18 + + 1012 1412 1614 18 + + 14 + + + + 16

1412

12 12 + 10 Drawing 10 + + + + 10 + + + + + 8 + + + + 8

++ 6 8 8 10 1012 6 + + + + + + + + 14 1010 12 1514 15 12 + ++ + +

10 10 Composition 8 + + + + + + 5 6 + + + + + + + + 04 6 5 8 04 + + +

St udy this plot carefully in relation to the correlation matrix for these data: rnd2(cor(painters4a)) Composition Drawing Colour Expression Schl.nD Composition 1.00 .42 -.10 .66 -.29 Drawing .42 1.00 -.52 .57 -.36 Colour -.10 -.52 1.00 -.20 .53 Expression .66 .57 -.20 1.00 -.45 Schl.nD -.29 -.36 .53 -.45 1.00 On the following page, principal components analysis (unweighted) is compared w/ common for these data; be sure you follow all steps carefully.

PCA results for Painters4a data The Unrotated coefficients matrix for 2 components is: pca.coef2_eigen(cor(painters4a))$ve[,1:2]%*%diag( + sqrt(eigen(cor(painters4a))$va[1:2])) rnd2(pca.coef2) [,1] [,2] [1,] -.68 -.58 [2,] -.80 .07 [3,] .62 -.70 [4,] -.81 -.41 [5,] .71 -.35 And its ROTATED counterpart is: > -rnd2(rotate(pca.coef2)$rma) [,1] [,2] with T matrix: [1,] .90 -.04 rotate(pca.coef2)$tm [2,] .54 -.59 [,1] [,2] [3,] .02 .93 [1,] .732 -.681 [4,] .87 -.26 [2,] .681 .732 [5,] -.28 .74 REASONS for choosing TWO components/factors are noted below ------COMMON FACTOR ANALYSIS NEXT------Compare with the COMMON FACTOR coefficients matrix: >cfa.coef2_cf.wad(cor(painters4a),2,54,3)$f 2 1/2 >cfa.coef2  [[cf.wad uses computation: S Q2 (Dlambda – arr I) ]] 1 2 Composition .76 -.09 Note that magnitudes of pca Drawing .50 -.56 coefficients are generally (but Colour -.03 .80 not always) larger than their Expression .81 -.26 common factor coef. counterparts. Schl.nD -.30 .62 Also, compare the LACK of FIT matrices for PCA and CFA: Separately examine diagonals and off-diagonal values >rnd2(cor(painters4a)-pca.coef2%*%t(pca.coef2)) Composition Drawing Colour Expression Schl.nD Composition .19 -.09 -.08 -.14 -.01 Drawing -.09 .36 .02 -.05 .23 Colour -.08 .02 .14 .02 -.15 Expression -.14 -.05 .02 .17 -.01 Schl.nD -.01 .23 -.15 -.01 .37 >cfa.coef2_cf.wad(cor(painters4a),2,54,3)$f >rnd2(cor(painters4a)-cfa.coef2%*%t(cfa.coef2)) Composition Drawing Colour Expression Schl.nD Composition .41 -.01 .00 .02 -.01 Drawing -.01 .44 -.05 .02 .14 Colour .00 -.05 .36 .03 .03 Expression .02 .02 .03 .28 -.04 Schl.nD -.01 .14 .03 -.04 .53

An example of a 'scree' plot based on eigenvalues of R 3.0 2.5 2.0 1.5 1.0 eigen(cor(painters4a))$va 0.5 0.0

1 2 3 4 5 1:5

> plot(1:5,eigen(cor(painters4a))$va,ylim=c(0,3),type='b') > abline(h=(eigen(cor(painters4a))$va[3:5]),lty=3) > title("An example of a 'scree' plot based on eigenvalues of R") Compare with Common factor counterpart:

A 'scree' plot based on eigenvalues of rescaled R: S.m1 R S.m1 6 5 4 3 2 svd.da(painters4a)$rts.r 1 0

1 2 3 4 5

1:5

abline(h=mean(svd.da(painters4a)$rts.r[3:5]),lty=3) Next, examine the ‘biplot’ display of the painters4a data: (done simply w/ principal components analysis, as shown) Display based on 'biplot(princomp(painters4a))'

-30 -20 -10 0 10 20 30 30

0.3 Testa Fr. Penni

Michelangelo Pourbus 20 0.2 Josepin Parmigiano Van Leyden Volterra Perugino 10 0.1 Bourdon Perino del Vaga GuilioDa RomanoVinci Barocci F. Zucarro Le Suer Drawing Fr.Del Salviata Sarto Primaticcio L. JordaensDurer Bellini Poussin

AlbaniLanfranco 0 0.0 T. Zucarro Otho Venius

Comp. 2 CaravaggioPalma Vecchio GuercinoSchl.nD Murillo Le Brun Bassano Cortona Domenichino Diepenbeck TeniersDel Piombo PordenoneDa Udine -0.1 Palma Giovane -10 Vanius Expression J. Jordaens The Carraci Tintoretto Giorgione Composition Veronese Raphael CorregioHolbeinTitian -0.2 -20

Van Dyck Rembrandt

-0.3 Colour -30 Rubens

-0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 Comp. 1

The special virtue of this type of display is that it shows simultaneously both data points, in this case for the painters with their names identified, and the variable as vectors. In this case the biplot has been done w/out rotation of component axes, but that could be done rather easily if desired. A biplot can also be done for a common factor solution…and there would be some advantage in doing that for some data sets (but it makes relatively little difference in this case). Note that any interpretation of such data can take account of a great deal of information. To give an example, we see that Expression and Composition, the two highest loading variables on factor or component I (after rotation) are variables that largely define the first component, the horizontal axis here; and painters such as LeBrun, Demenichino, and Raphael differ especially from painters Bellini, Murillo, Calavaggio (to name a few) at the opposite side of the display…that is, they differ especially with respect to the Expression and Composition ratings. Painters whose names are in close proximity to one another tend to be similar with respect to the information summarized by both these components, and painters far from one another may differ with respect to both or just one component (would also be true for common factors). Biplot displays like this are not confined to two factor solutions as the term biplot does not refer to two-dimensional displays, but rather to two kinds of information, coefficients and scores. Take some time to relate what you see summarized here, and what you can see in the data matrix itself for the painters4a data. That matrix is given at the end of this handout ON THE WEB ONLY.

Summary: The comparisons of results for common factor analysis (CFA) and principal component analysis (PCA) generally shows similar findings to those shown here: better fits to the OFF- DIAGONAL entries of the correlation matrix when using CFA; worse fits to the DIAGONALS (ones) w/ CFA (but we almost never care about fitting the diagonals, so CFA may be preferable). The coefficients matrix generated using CFA has more clearly interpretable values as each coefficient represents a simple correlation between a (row) variable and a (column) common factor, with little bias; the PCA coefficients are spurious (often larger in magnitude) in the sense that these correlations are between variables and derived components where the components are (merely) linear combinations of the variables themselves. Common factors are ‘latent’ or ‘unobservable’ variates, postulated in a common factor model; components are not latent, so they are computable. To elaborate further, the common factor model postulates that individual scores (ratings in this example) are individually composed of two (orthogonal) parts, the common factor parts (thought to be ‘less error-full’) and ‘uniqueness parts’, essentially the error parts. When one entertains a common factor model in the context of an analysis of a matrix of data, one supposes that individual scores (such as ratings) tend to be ‘fallible’, that in isolation such scores may well misrepresent the ‘true’ rating that might be conceived of as the ‘true common part’ of the data; when the number of common factors is chosen, the value m, one makes a choice for the dimensionality of the common factor space…which we might hope will have relatively low dimensionality so as to enhance the interpretability and reduce the complexity of the account given of the data by way of common factors. Principal component analysis is better thought of as just a data reduction method, as there is no model for it, just a way algebraically to decompose the data (think of the svd) into separable parts. With PCA one has little guidance of the kind available in CFA for choosing the number of components to use in data reduction. Nevertheless, PCA is used more often than CFA (which has MANY different forms and algorithms, and is in some ways controversial in its assumption of underlying or latent factors) and PCA does often produce certain results (see the biplot) that may be very close to common factor counterparts…so it almost inevitable that there will be a good deal of confusion in the minds of those not highly knowledgeable about common factor analysis between PCA and CFA. Factor and component rotation (see function ‘rotate’ in Splus, or ‘varimax’ in R) is easily accomplished and is worth further study if you aim to analyze real data with two or more factors, especially if the ‘model’ for CFA seems credible for your data.

Homework: Carry out CFA starting from either a rectangular data matrix (such as painters4a [below], or ?? [see MASS library or do help.search("datasets") in R ) or for a correlation matrix (see Harman74.cor in R, for which you will find that you will need to see its documentation [and use >data(Harman74.cov), then Harman74.cor$cov]) to get started. First produce a as I have demonstrated, and make a decision about the number of common factors; then proceed to use the functions I have provided (csmooth, fmore.rt, or svd.da, etc.) OR use your own more basic approach (and your own functions if you’d like) to analyze these or other data. (the UScereal data, MASS library, excluding first and last columns, is worth your attention. Compute common factor coefficient matrices, both unrotated and rotated, and try to INTERPRET your results, including the derived communality estimates. Note that if you begin from a rect. data matrix you may be able to generate a biplot display, but not if you begin from a correlation matrix. Make careful decisions about what information to include in what you hand it; and summarize in English what your analysis shows. Finally, if you have time, try a principal component analysis of the same data, and then compare your two sets of results. painters4a Composition Drawing Colour Expression Schl.nD Da Udine 10 8 16 3 1 Da Vinci 15 16 4 14 1 Del Piombo 8 13 16 7 1 Del Sarto 12 16 9 8 1 Fr. Penni 0 15 8 0 1 Guilio Romano 15 16 4 14 1 Michelangelo 8 17 4 8 1 Perino del Vaga 15 16 7 6 1 Perugino 4 12 10 4 1 Raphael 17 18 12 18 1 F. Zucarro 10 13 8 8 1 Fr. Salviata 13 15 8 8 1 Parmigiano 10 15 6 6 1 Primaticcio 15 14 7 10 1 T. Zucarro 13 14 10 9 1 Volterra 12 15 5 8 1 Barocci 14 15 6 10 1 Cortona 16 14 12 6 1 Josepin 10 10 6 2 1 L. Jordaens 13 12 9 6 1 Testa 11 15 0 6 1 Vanius 15 15 12 13 1 Bassano 6 8 17 0 4 Bellini 4 6 14 0 4 Giorgione 8 9 18 4 4 Murillo 6 8 15 4 4 Palma Giovane 12 9 14 6 4 Palma Vecchio 5 6 16 0 4 Pordenone 8 14 17 5 4 Tintoretto 15 14 16 4 4 Titian 12 15 18 6 4 Veronese 15 10 16 3 4 Albani 14 14 10 6 1 The rest are all school 1; you have the data

Caravaggio 6 6 16 0 1 Corregio 13 13 15 12 1 Domenichino 15 17 9 17 1 Guercino 18 10 10 4 1 Lanfranco 14 13 10 5 1 The Carraci 15 17 13 13 1 Durer 8 10 10 8 1 Holbein 9 10 16 13 1 Pourbus 4 15 6 6 1 Van Leyden 8 6 6 4 1 Diepenbeck 11 10 14 6 1 J. Jordaens 10 8 16 6 1 Otho Venius 13 14 10 10 1 Rembrandt 15 6 17 12 1 Rubens 18 13 17 17 1 Teniers 15 12 13 6 1 Van Dyck 15 10 17 13 1 Bourdon 10 8 8 4 1 Le Brun 16 16 8 16 1 Le Suer 15 15 4 15 1 Poussin 15 17 6 15 1