Hogan Development Survey Manual Second Edition Robert Hogan, Ph.D. Joyce Hogan, Ph.D. Hogan Assessment Systems Tulsa, OK 74114, USA 2009 © Hogan Assessment Systems 2009 Robert Hogan, Ph.D. Joyce Hogan, Ph.D. No part of this work may be copied or transferred to any other form or expression without the expressed written consent from Hogan Assessment Systems, Inc. Hogan Personality Inventory ™ Hogan Development Survey ™ Motives, Values, Preferences Inventory ™ are the exclusive registered trademarks of Hogan Assessment Systems, Inc. www.hoganpress.com First printing 05-09 ISBN 978-0-9816457-7-3 CONTENTS Contents 1. Conceptual Background 1 1.1 Introduction 1 1.2 The Problem 1 1.3 The Theoretical Context: Socioanalytic Theory and Organizational Dynamics 2 1.4 The Nature of Dysfunctional Dispositions 3 1.5 Development Guidelines 6 2. Inventory Construction, Reliability, and Structural Psychometrics 9 2.1 Development 9 2.2 Definitions of the Scales 13 2.3 Item Composition of the Scales 14 2.4 The 2008 US Normative Sample Dataset 14 2.4.1 Selection Versus Development Group Comparability Analysis 15 2.4.2 HDS Scale Score Mean Differences between the Development and Selection Groups 15 2.4.3 HDS Coefficient Alpha Reliabilities for the Development and Selection Groups 16 2.4.4 Correlation Pattern Hypothesis Test – Comparing Development and Selection Group Correlation Matrices 17 2.4.5 Comparison of Development and Selection Group Component Factor Analysis Solutions 18 2.4.6 Comparison of Development and Selection Group Nonmetric Smallest Space Analysis Solutions 19 2.4.7 Summary 21 2.5 Structural Psychometrics of the Normative Dataset (N = 109,103) 21 2.6 Descriptive Statistics and Reliabilities for the 2008 Normative Dataset (N =109,103) 22 2.7 Test-Retest Reliability 25 2.7.1 Short-Term Stability (less than 3 months) 25 2.7.2 Long-Term Stability (between 9 to 12 months) 25 3. Validity 31 3.1 Construct Validity 31 3.2 Correlations with Other Assessments 31 3.2.1 Procedure 31 3.2.2 Samples and Instruments 32 3.2.3 Results of Scale to Scale Correlates 42 3.3 Correlations between the HDS and Others’ Descriptions 58 3.3.1 Procedure 59 3.3.2 Samples and Instruments 59 3.3.3 Results of Scale and Observer Description Correlates 67 3.4 HDS Correlates of Organizational Behavior in Managerial and Professional Jobs 76 3.4.1 Competency Domains of Job Performance 77 3.4.2 Methods Used for Meta-Analysis 80 3.4.3 Curvilinear Relations 84 III CONTENTS HDS TECHNICAL MANUAL 3.5 HDS Relations with the Leadership Versatility Index (LVI): A Case Study 85 3.5.1 Leadership Versatility Index 85 3.5.2 HDS and LVI Relations 87 3.5.3 Summary 97 4. Interpretation and Uses 99 4.1 Introduction 99 4.2 Scale-by-Scale Interpretation 100 4.2.1 Excitable 101 4.2.2 Skeptical 103 4.2.3 Cautious 104 4.2.4 Reserved 106 4.2.5 Leisurely 107 4.2.6 Bold 109 4.2.7 Mischievous 111 4.2.8 Colorful 112 4.2.9 Imaginative 114 4.2.10 Diligent 116 4.2.11 Dutiful 117 4.3 An Interpretive Strategy 119 4.3.1 The Value of Multiple Data Sources 119 4.3.2 Configural Analysis 120 4.3.3 Conflict Analysis 120 4.4 Sample HDS Profile Interpretations 122 4.4.1 The “Moving Away” Profile 122 4.4.2 The “Moving Against” Profile 124 4.4.3 The “Moving Toward” Profile 126 4.4.4 The “Corporate Guerilla” Profile 127 4.4.5 The “Insecure Showboat” Profile 129 4.4.6 The “Litigious” Profile 131 4.4.7 The “Fear-Driven Salesman” Profile 133 4.5 Uses 135 5. Administering the HDS 139 5.1 Key Features of the Web-Based Platform 139 5.2 Completing the HDS Using the Online Internet System 139 5.3 Participant’s Informed Consent 142 5.4 Using International Translations of the HDS 144 5.5 Accommodating Individuals with Disabilities 148 5.6 Frequently Asked Questions 148 5.7 Alternative Testing Solutions 149 IV CONTENTS 6. Compilation of Norms 151 6.1 Importance of Norms for Interpretation and Decision-Making 151 6.1.1 Presentation of Normative Data 151 6.1.2 Professional Standards for Norm Development 152 6.2 Updating Norms for the 2008 HDS 152 6.2.1 Job Families 154 6.2.2 Application of Data 156 6.2.3 Ethnicity 157 6.2.4 Gender 157 6.2.5 Age 157 6.3 Stratified Sampling of the Norming Population 158 6.3.1 Initial Population 158 6.3.2 Elimination of Cases 159 6.4 Demographics of the Norming Sample 161 6.5 Descriptive Statistics of the Norming Sample 163 7. References 173 Appendix A: Correlations Between the HDS Scales and the HPI Scales and Homogenous Item Composites 185 Appendix B: References for Studies Included in HDS Meta-Analysis 187 Appendix C: Complete Meta-Analysis Results for the HDS Scales Aligned with Competency Domains and Criteria 189 Appendix D: Norms for the Total Sample (N = 109,103) 199 V TABLES & FIGURES Tables & Figures Table 1.1 HDS Scales: Themes and Implications 7 Table 2.1 HDS Scale Score Means, Standard Deviations, and Comparative Effect Sizes Between Development and Selection Groups 16 Table 2.2 HDS Coefficient Alphas for the Development and Selection Groups 16 Figure 2.1 Selection Group Standardized Residual Histogram 17 Figure 2.2 Development Group Standardized Residual Histogram 18 Table 2.3 1997 Component Factor Solution with the Development Target and Maximally Congruent Selection Group Matrices 19 Figure 2.3 Nonmetric Smallest Space Analysis and Configural Matching of Development and Selection Group HDS Scores 21 Figure 2.4 A Nonmetric Smallest Space Analysis Visualization of the HDS Global Components 22 Table 2.4 1997 Component Factor Solution With the 2008 Normative Sample Results (N=109,036) Using Both Varimax and Direct Oblimin Rotations 23 Table 2.5 Descriptive Statistics, Reliabilities, and Standard Errors of Measurement for the HDS scales 24 Table 2.6 Pearson Correlations Between the 11 HDS Scale Scores Using Complete-Case Data Within the Normative Sample (N=109,036) 26 Table 2.7 Short- and Long-Term Stability Coefficients for the HDS 28 Table 2.8 Signed-Difference Descriptive Statistics for Short- and Longer Term Stability Coefficients 29 Table 3.1 Correlations Between HDS Scales and HPI Scales 37 Table 3.2 Correlations Between HDS Scales and CPI Scales 37 Table 3.3 Correlations Between the HDS Scales and NEO PI-R Scales/Facets 38 Table 3.4 Correlations Between the HDS Scales and the IPIP Big 5 20-Item Scales 39 Table 3.5 Correlations Between the HDS Scales and the 16PF Scales 39 Table 3.6 Correlations Between HDS Scales and MVPI Scales 40 Table 3.7 Correlations Between the HDS Scales and the CISS Interest and Skill Scales 40 Table 3.8 Correlations Between the HDS Scales and the JPI-R Scales 41 Table 3.9 Correlations Between the HDS Scales and the HBRI Scales 41 Table 3.10 Correlations between the HDS Scales and the Watson-Glaser Scales 42 Table 3.11 HDS Correlations with Observer Ratings for Big-Five Mini-Marker Adjectives 61 Table 3.12 HDS Correlations with Observer Ratings for Big-Five Inventory Phrases 62 Table 3.13 HDS Correlations with Co-Worker Description Ratings 63 Table 3.14 HDS Correlations with Spouse and Coach Description Ratings 65 Figure 3.1 Domain Model of Organizational Performance 77 Table 3.15 Meta-Analysis Results for HDS Scales Aligned with Competency Domains and Criteria 82 Table 3.16 The Leadership Versatility Index (LVI) Content Model 86 VII HDS TECHNICAL MANUAL Figure 3.2 The Leadership Versatility Index (LVI) Rating Scale 86 Figure 3.3 HDS Excitable Scale Plotted Against the LVI Forceful Declares Leadership Ratings 88 Figure 3.4 HDS Skeptical Scale Plotted Against the LVI Operational Efficiency Leadership Ratings 88 Figure 3.5 HDS Cautious Scale Plotted Against the LVI Enabling Leadership Ratings 89 Figure 3.6 HDS Reserved Scale Plotted Against the LVI Enabling Supports Leadership Ratings 90 Figure 3.7 HDS Leisurely Scale Plotted Against the LVI Strategic Innovation Leadership Ratings 91 Figure 3.8 HDS Bold Scale Plotted Against the LVI Enabling Leadership Ratings 92 Figure 3.9 HDS Mischievous Scale Plotted Against the LVI Operational Efficiency Leadership Ratings 93 Figure 3.10 HDS Colorful Scale Plotted Against the LVI Forceful Takes Charge Leadership Ratings 94 Figure 3.11 HDS Imaginative Scale Plotted Against the LVI Strategic Innovation Leadership Ratings 95 Figure 3.12 HDS Diligent Scale Plotted Against the LVI Operational Execution Leadership Ratings 96 Figure 3.13 HDS Dutiful Scale Plotted Against the LVI Forceful Declares Leadership Ratings 97 Table 4.1 A Quick Guide for Interpreting the HDS 119 Figure 4.1 HDS “Moving Away” Profile 122 Figure 4.2 HPI “Moving Away” Profile 123 Figure 4.3 HDS “Moving Against” Profile 124 Figure 4.4 HPI “Moving Against” Profile 125 Figure 4.5 HDS “Moving Toward” Profile 126 Figure 4.6 HPI “Moving Toward” Profile 127 Figure 4.7 HDS “Corporate Guerilla” Profile 128 Figure 4.8 HPI “Corporate Guerilla” Profile 129 Figure 4.9 HDS “Insecure Showboat” Profile 130 Figure 4.10 HPI “Insecure Showboat” Profile 131 Figure 4.11 HDS “Litigious” Profile 132 Figure 4.12 HPI “Litigious” Profile 133 Figure 4.13 HDS “Fear-Driven Salesman” Profile 134 Figure 4.14 HPI “Fear-Driven Salesman” Profile 135 Figure 5.1 Hogan Assessment Systems Participant Login Web Page 140 Figure 5.2 Hogan Assessment Systems Participant Information Web Page 141 Figure 5.3 Hogan Assessment Systems Participant Menu Web Page 142 Figure 5.4 Hogan Assessment Systems Questionnaire Assessment Web Page 143 Table 5.1 HDS Language Translations 144 Figure 5.5 Hogan Assessment Systems Report Language Selection 145 Figure 5.6 Hogan Assessment Systems Language Translation Flags 146 Figure 5.7 Hogan Assessment Systems Participant Information Page 146 Figure 5.8 Hogan Assessment