Discriminant Analysis (DA)

1. Basic Concept a) Dependent Variable: Categorical (nominal). If the dependent variable is ordinal or metric, convert it to mutually exclusive and exhaustive categories.  If the number of categories is large, one may choose the polar extremes approach, where only extreme categories are analyzed, and the middle ones excluded from the analysis.  The number of observations in each category (the group size) should be at least 20.  If the group sizes vary significantly, select randomly observations from the larger groups to make the resulting samples comparable to the smaller groups.

(i) Two-group DA - the dependent variable has two categories, e.g. Male = 1, Female = 2 (ii) Multiple Discriminant Analysis (MDA) – the dependent variable has more than two categories

b) Independent Variables: Metric (required ratio of observations to the number of independent variables: at least 20; the minimum ratio: 5). This ratio applies to all variables initially included in the analysis (even if some would later be eliminated by the stepwise procedure)

2. Randomly split (e.g., 50-50, or 60-40, or 75-25) the sample into two parts (use a proportionately stratified sampling procedure): a) Analysis Sample: for estimation of the discriminant function b) Validation (Holdout) Sample: for validating the discriminant function.  Ex. Split 60-40: Total sample: 80 males and 50 females Analysis Sample: 48 males and 30 females; Holdout Sample: 32 males and 20 females.

3. Assumptions of DA a) No outliers. Verify their presence with both the SRE (Studentized Residuals) and the Mahalanobis distance. Remove the outliers from further analysis. b) Linear relationships between the dependent variable and the independent variables. Check the linearity with the procedures explained in the previous sessions. Correct any nonlinear relationships with specific variable transformations. c) No multicollinearity among the independent variables. Check the Pooled Within- Groups Correlation Matrix. Analyze the collinearity diagnostics. d) Equal covariance matrices for the groups as defined by the dependent variable. Use Box’s M test. If Sig. > 0.05, cannot reject Ho: The population covariance matrices are equal. Note: In cases when some of the assumptions of DA are not clearly met (e.g. the normality is dubious), a researcher is allowed to use a threshold value less than 0.05, e.g. 0.03. e) The independent variables are normally distributed. Use appropriate tests explained in the previous sessions.

1 4. Apply DA to the Analysis Sample a) Direct (Simultaneous) Method – when the discrimination is to be based on ALL independent variables b) Stepwise (Elimination) DA – when only a subset of the most discriminatory independent variables is to be included in the discriminant function

5. Example: Discriminant Analysis (DA)  Convert file ResortsVisitsDA.xls to ResortsVisitsDA.sav (the first 30 rows are Analysis Sample, and the last 12 rows are Holdout Sample)  TransformRecodeInto Different VariablesPaste the variable “no” as Numeric Variable into Output Variable (“no1”)Old and New Values: Range 1 thru 30 1; Range 31 thru 42 2; ContinueChange  AnalyzeClassifyDiscriminantGrouping Variable (visit)Define Range (Minimum 1; Maximum 2)Independents (income, travel, vacation, hsize, age)Selection Variable: no1 = 1 Statistics (check all boxes)Classify (Limit cases to first 30)Save (check all boxes)Method (Enter independents together)OK  Interpretation of the results based on the Analysis Sample: n = 30 (going from the top of the computer printout) (i) Group Means and Standard Deviations:  Income (Large difference in means; large standard deviations in each group)  Age (Small difference in means; large standard deviations in each group)  Travel, vacation, hsize (small difference in means; small standard deviations in each group)  Test of Equality of Group Means: Wilks’ Lambda  Income: Sig. = 0.000  Travel: Sig. = 0.143  Vacation: Sig. = 0.021  Hsize: Sig. = 0.001  Age: Sig. = 0.257  Income, Vacation, and Hsize show significant (p = 0.05) univariate differences between the two groups (Group 1: Those who visited the resort during the last 2 years; Group 2: Those who not)  There is no difference in resorts visits based on Attitude toward travel and Age.  It is therefore obvious that Income, Vacation, and Hsize may best discriminate between the two groups of visitors. If you are interested in the efficiency of only these three variables in discriminating between the two groupsUse the Stepwise procedure. If you are interested in the

2 efficiency of each of the five independent variablesUse the Direct procedure (which we are going to follow below)

(ii) Pooled within-groups correlation matrix: low correlations  lack of multicollinearity among the independent variables (rule of thumb: if a correlation coefficient is < 0.30)

(iii) Box’s M Test of Equality of Covariance Matrices

 Sig. = 0.141 > 0.05: Cannot reject Ho: The population covariance matrices are equal

(iv) Summary of Canonical Discriminant Functions  Canonical Correlation = 0.801(0.801)2 = 64.1% of the variance in the dependent variable (Resort Visit) can be accounted for by the model (all the five independent variables).  The Wilks’ Lambda = 0.359 (which is equivalent to Chi-square = 26.130 with 5 d.f.) is significant at the 0.000 level. This means that the discriminant function computed in this procedure is statistically significant at the 0.000 level. Only then, one can proceed to interpret the results.

(v) Standarized Canonical Discriminant Function Coefficients Income: 0.743 Travel: 0.096 Vacation: 0.233 Hsize: 0.469 Age: 0.209

Note: the signs (+ or -) indicate a positive or a negative relationship with the dependent variable.  The discriminant function (based on standarized discriminant coefficients):

Z = 0.743Income + 0.096Travel + 0.233Vacation + 0.469Hsize + 0.209Age

(vi) Structure Matrix (Discriminant Loadings – order from highest to lowest by the absolute size of the loading, the sign + or – indicates only a positive or a negative relationship with the dependent variable)

 The discriminant function (based on discriminant loadings)

Z = 0.822Income + 0.541Hsize + 0.346Vacation + 0.213Travel + 0.164Age

(vii) (Unstandarized) Canonical Discriminant Function Coefficients

3  The discriminant function (based on unstandarized discriminant coefficients)

Z = -7.975 + 0.085Income + 0.050Travel + 0.120Vacation + 0.427Hsize + 0.025 Age

 Which discriminant function to use and when?  For interpretation purposes, use discriminant loadings. The standarized discriminant coefficients can also be used, although in the literature they are less preferred than the loadings. Any variable exhibiting a loading of more than +0.30 or less than –0.30 is considered a substantive discriminator (i.e., Income, Hsize, and Vacation)

 To calculate the discriminant Z scores for the classification purposes, use the unstandarized discriminant coefficients.

(viii) Functions at Group Centroids: The value of the discriminant function (with the unstandarized coefficients) at the group means.

 Ex. Gc1 = -7.975 + 0.085*60.520 + 0.05*5.4 + 0.120*5.8 + 0.427*4.333 + 0.025*53.733 = 1.291

 So, the group centroid for the visitors to the resort (Group 1) is +1.291  The group centroid for non-visitors (Group 2) is –1.291 (they do not have to be equal in absolute terms)

 The optimum cutting score is based on the Group Centroids:

 For two groups of equal size: Gc = (Gc1 + Gc2)/2 = (1.291 – 1.291)/2 = 0  Thus, if Zi > 0  assign case i to Group1; if Zi < 0  assign case i to Group 2 (see the additional columns dis_1 and dis1_1 saved by the DA in the SPSS Input file:

 For example, for Case number 1: dis1_1 = -0.17214 was calculated as follows (based on the unstandarized discriminant coefficients) dis1_1 = -7.975476 + 0.0847671*50.2 + 0.04964455*5 + … + 0.0245438*43 = -0.17214 < 0  Case 1 is assigned to Group 2, which is a mistake, because we know from the sample that Case 1 belongs to Group 1(visitors). There are altogether 3* such mistakes made, i.e. when a Case from Group 1 has been assigned to Group 2. However, there were 0* such mistakes made when assigning members from Group 2 – all of them were correctly assigned to Group 2.

4 Classification Results (Original)

Group 1 Group 2 Group 1 12 3* Group 2 0* 15

 For two groups of different sizes n1 and n2:  Zcs = (n1*Gc1 + n2*Gc2)/(n1 + n2) (ix) Classification Function Coefficients (Fisher’s Linear Discriminant Functions) – can also be used for classification purposes.

Z1 = -57.532 + 0.678Income + 1.509Travel + 0.938Vacation + 3.322Hsize + 0.832Age Z2 = -36.936 + 0.459Income + 1.381Travel + 0.628Vacation + 2.218Hsize + 0.768Age

Ex. Consider again Case 1. Calculate: Z1(Case 1) = -57.532 + 0.678*50.2 + .. + 0.832*43 = 37.295 Calculate: Z2(Case 1) = -36.936 + 0.459*50.2 + … + 0.768*43 = 37.7128 Because 37.7128 > 37.295, assign Case 1 to Group 2

(x) Classification Results

 Based on the Analysis Sample (called in SPSS: Original)  The Hit Ratio = % of correctly classified cases = (12 + 15)/30 = 90%

 Based on “leave-one-out” principle (called in SPSS: Cross-validated)  The Hit Ratio = % of correctly classified cases = (11 + 13)/30 = 80%  Based on the Holdout Sample (n = 12) (called in SPSS: Cases Not Selected – Original)  The Hit Ratio = % of correctly classified cases = (4 + 6)/12 = 83.3%

 In either case, compare the Hit Ratio with the Chance Ratio:  If the group sizes are equal: Chance Ratio = 1/(number of groups)  In our example, Chance Ratio = ½ = 0.5. (The Hit Ratio should be 1.25 times greater than the Chance Ratio in order for the Validity of the DA to be satisfactory). The lowest of the three Hit Ratios = 80% > 1.25*50% = 62.5% The validity of our DA is satisfactory.

 If the group sizes are different, two Chance Ratios are possible:  Maximum Chance Criterion = The percentage of the total sample represented by the largest of the groups

5  Ex. Group 1 = 65, Group 2 = 25, Group 3 = 10  The MCC = 0.65 = 65%

 Proportional Chance Criterion = p12 + p22 + .. + pk2  Ex. The PCC = 0.652 + 0.252 + 0.12 = 0.495 = 49.5%

 If the Hit Ratio > 1.25*max(MCC, PCC)  The validity of DA is satisfactory

 Another statistical test for the discriminatory power of the classification matrix:

 Press’s Q statistic = [N – (Ncorrect*c)]2/N(c-1)

 Where: c = number of groups  N = sample size  Ncorrect = number of observations correctly classified

 Ex. c = 2, N = 30, Ncorrect = 24 (based on the cross-validated procedure)  Press’s Q = [30 – 24*2]2/30*(2-1) = 10.8.

 The critical level at 0.05 is Q = 6.63  Q = 10.8>Qcritical, hence the classification matrix can be deemed significantly (p = 0.05) statistically better than chance.