+DQGERRNRI (WKLFVLQ 4XDQWLWDWLYH 0HWKRGRORJ\ TAF-Y101790-10-0602-C000.indd i 12/4/10 9:45:02 AM Multivariate Applications Series Sponsored by the Society of Multivariate Experimental Psychology, the goal of this series is to apply complex statistical methods to sig- nifi cant social or behavioral issues, in such a way so as to be accessible to a nontechnical readership (e.g., non-methodological researchers, teach- ers, students, government personnel, practitioners, and other profession- als). Applications from a variety of disciplines such as psychology, public health, sociology, education, and business are welcome. Books can be single- or multiple-authored or edited volumes that (1) demonstrate the application of a variety of multivariate methods to a single, major area of research; (2) describe a multivariate procedure or framework that could be applied to a number of research areas; or (3) present a variety of perspec- tives on a topic of interest to applied multivariate researchers. There are currently 17 books in the series: • What if there were no signifi cance tests? co-edited by Lisa L. Harlow, Stanley A. Mulaik, and James H. Steiger (1997) • Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming, written by Barbara M. Byrne (1998) • Multivariate Applications in Substance Use Research: New Methods for New Questions, co-edited by Jennifer S. Rose, Laurie Chassin, Clark C. Presson, and Steven J. Sherman (2000) • Item Response Theory for Psychologists, co-authored by Susan E. Embretson and Steven P. Reise (2000) • Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, written by Barbara M. Byrne (2001) • Conducting Meta-Analysis Using SAS, written by Winfred Arthur, Jr., Winston Bennett, Jr., and Allen I. Huffcutt (2001) • Modeling Intraindividual Variability with Repeated Measures Data: Methods and Applications, co-edited by D. S. Moskowitz and Scott L. Hershberger (2002) • Multilevel Modeling: Methodological Advances, Issues, and Applications, co-edited by Steven P. Reise and Naihua Duan (2003) • The Essence of Multivariate Thinking: Basic Themes and Methods, written by Lisa Harlow (2005) • Contemporary Psychometrics: A Festschrift for Roderick P. McDonald, co-edited by Albert Maydeu-Olivares and John J. McArdle (2005) • Structural Equation Modeling with EQS: Basic Concepts, Applications, and Programming, Second Edition, written by Barbara M. Byrne (2006) TAF-Y101790-10-0602-C000.indd ii 12/4/10 9:45:02 AM • A Paul Meehl Reader: Essays on the Practice of Scientifi c Psychology, co-edited by Niels G. Waller, Leslie J. Yonce, William M. Grove, David Faust, and Mark F. Lenzenweger (2006) • Introduction to Statistical Mediation Analysis, written by David P. MacKinnon (2008) • Applied Data Analytic Techniques for Turning Points Research, edited by Patricia Cohen (2008) • Cognitive Assessment: An Introduction to the Rule Space Method, writ- ten by Kikumi K. Tatsuoka (2009) • Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, Second Edition, written by Barbara M. Byrne (2010) • Handbook of Ethics in Quantitative Methodology, co-edited by A. T. Panter and Sonya K. Sterba (2011) Anyone wishing to submit a book proposal should send the following: (1) author/title; (2) timeline including completion date; (3) brief overview of the book’s focus, including table of contents and, ideally, a sample chap- ter (or chapters); (4) a brief description of competing publications; and (5) targeted audiences. For more information, please contact the series editor, Lisa Harlow, at Department of Psychology, University of Rhode Island, 10 Chafee Road, Suite 8, Kingston, RI 02881-0808; phone (401) 874-4242; fax (401) 874-5562; or e-mail [email protected]. Information may also be obtained from members of the editorial/advisory board: Leona Aiken (Arizona State University), Daniel Bauer (University of North Carolina), Jeremy Biesanz (University of British Columbia), Gwyneth Boodoo (Educational Testing Services), Barbara M. Byrne (University of Ottawa), Scott E. Maxwell (University of Notre Dame), Liora Schmelkin (Hofstra University), and Stephen West (Arizona State University). TAF-Y101790-10-0602-C000.indd iii 12/4/10 9:45:02 AM TAF-Y101790-10-0602-C000.indd iv 12/4/10 9:45:02 AM +DQGERRNRI (WKLFVLQ 4XDQWLWDWLYH 0HWKRGRORJ\ $73DQWHU 7KH8QLYHUVLW\RI1RUWK&DUROLQD&KDSHO+LOO 6RQ\D.6WHUED 9DQGHUELOW8QLYHUVLW\ TAF-Y101790-10-0602-C000.indd v 12/4/10 9:45:02 AM MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. Routledge Routledge Taylor & Francis Group Taylor & Francis Group 270 Madison Avenue 27 Church Road New York, NY 10016 Hove, East Sussex BN3 2FA © 2011 by Taylor and Francis Group, LLC Routledge is an imprint of Taylor & Francis Group, an Informa business Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number: 978-1-84872-854-7 (Hardback) 978-1-84872-855-4 (Paperback) For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organiza- tion that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Panter, A. T. Handbook of ethics in quantitative methodology / A.T. Panter, Sonya K. Sterba. p. cm. -- (Multivariate applications series) Includes bibliographical references and index. ISBN 978-1-84872-854-7 (hbk. : alk. paper) -- ISBN 978-1-84872-855-4 (pbk. : alk. paper) 1. Quantitative research--Moral and ethical aspects. 2. Social sciences--Methodology--Moral and ethical aspects. I. Sterba, Sonya K. II. Title. H62.P276 2011 174’.900142--dc22 2010045883 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the Psychology Press Web site at http://www.psypress.com TAF-Y101790-10-0602-C000.indd vi 12/4/10 9:45:02 AM In memory of J. S. Tanaka – ATP To my parents, Jim and Janet – SKS TAF-Y101790-10-0602-C000e.indd vii 12/4/10 9:45:17 AM TAF-Y101790-10-0602-C000e.indd viii 12/4/10 9:45:17 AM Contents Preface ............................................................................................................. xiii Editors ...............................................................................................................xv Contributors .................................................................................................. xvii Software Notice ...............................................................................................xix 1 Ethics in Quantitative Methodology: An Introduction ..................... 1 A. T. Panter and Sonya K. Sterba Section I Developing an Ethical Framework for Methodologists 2 Ethics in Quantitative Professional Practice ...................................... 15 John S. Gardenier 3 Ethical Principles in Data Analysis: An Overview .......................... 37 Ralph L. Rosnow and Robert Rosenthal Section II Teaching Quantitative Ethics 4 A Statistical Guide for the Ethically Perplexed ................................. 61 Lawrence Hubert and Howard Wainer Section III Ethics and Research Design Issues 5 Measurement Choices: Reliability, Validity, and Generalizability ..................................................................................... 127 Madeline M. Carrig and Rick H. Hoyle 6 Ethics and Sample Size Planning....................................................... 159 Scott E. Maxwell and Ken Kelley 7 Ethics and the Conduct of Randomized Experiments and Quasi-Experiments in Field Settings ................................................ 185 Melvin M. Mark and Aurora L. Lenz-Watson ix TAF-Y101790-10-0602-C000toc.indd ix 12/4/10 9:46:14 AM x Contents 8 Psychometric Methods and High-Stakes Assessment: Contexts and Methods for Ethical Testing Practice ........................ 211 Gregory J. Cizek and Sharyn L. Rosenberg 9 Ethics in Program Evaluation.............................................................. 241 Laura C. Leviton Section IV Ethics and Data Analysis Issues 10 Beyond Treating Complex Sampling Designs as Simple Random Samples: Data Analysis and Reporting ............................ 267 Sonya K. Sterba, Sharon L. Christ, Mitchell J. Prinstein, and Matthew K. Nock 11 From Hypothesis Testing to Parameter Estimation: An Example of Evidence-Based Practice in Statistics .................... 293 Geoff Cumming and Fiona Fidler 12 Some Ethical Issues in Factor Analysis ............................................. 313 John J. McArdle 13 Ethical Aspects of Multilevel Modeling ...........................................341 Harvey Goldstein 14 The Impact of Missing Data on the Ethical Quality of a Research Study ....................................................................................... 357 Craig K. Enders