Construct Validity in Psychological Tests
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities— a Foundation for Research
The Journal of Technology, Learning, and Assessment Volume 10, Number 5 · November 2010 Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities— A Foundation for Research Patricia Almond, Phoebe Winter, Renée Cameto, Michael Russell, Edynn Sato, Jody Clarke-Midura, Chloe Torres, Geneva Haertel, Robert Dolan, Peter Beddow, & Sheryl Lazarus www.jtla.org A publication of the Technology and Assessment Study Collaborative Caroline A. & Peter S. Lynch School of Education, Boston College Volume 10, Number 5 Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities—A Foundation for Research Patricia Almond, Phoebe Winter, Renée Cameto, Michael Russell, Edynn Sato, Jody Clarke-Midura, Chloe Torres, Geneva Haertel, Robert Dolan, Peter Beddow, & Sheryl Lazarus Editor: Michael Russell [email protected] Technology and Assessment Study Collaborative Lynch School of Education, Boston College Chestnut Hill, MA 02467 Copy Editor: Jennifer Higgins Design: omas Ho mann Layout: Aimee Levy JTLA is a free online journal, published by the Technology and Assessment Study Collaborative, Caroline A. & Peter S. Lynch School of Education, Boston College. Copyright ©2010 by the Journal of Technology, Learning, and Assessment (ISSN 1540-2525). Permission is hereby granted to copy any article provided that the Journal of Technology, Learning, and Assessment is credited and copies are not sold. Preferred citation: Almond, P., Winter, P., Cameto, R., Russell, M., Sato, E., Clarke-Midura, J., Torres, C., Haertel, G., Dolan, R., Beddow, P., & Lazarus, S. (2010). Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities—A Foundation for Research. -
MORALITY in IRAN 1 in Press, Evolution and Human Behavior
MORALITY IN IRAN 1 In press, Evolution and Human Behavior https://doi.org/10.1016/j.evolhumbehav.2020.07.014 Foundations of Morality in Iran Mohammad Atari1, Jesse Graham2, Morteza Dehghani1,3 1Department of Psychology, University of Southern California 2Department of Management, David Eccles School of Business, University of Utah 3Department of Computer Science, University of Southern California Author Note Correspondence regarding this article should be addressed to Mohammad Atari, [email protected], 362 S. McClintock Ave, Los Angeles, CA 90089-161 MORALITY IN IRAN 2 Abstract Most moral psychology research has been conducted in Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. As such, moral judgment, as a psychological phenomenon, might be known to researchers only by its WEIRD manifestations. Here, we start with evaluating Moral Foundations Theory (MFT) using the Moral Foundations Questionnaire, and follow up by building a bottom-up model of moral values, in Iran, a non-WEIRD, Muslim-majority, understudied cultural setting. In six studies (N = 1,945) we examine the structural validity of the Persian translation of the Moral Foundations Questionnaire, compare moral foundations between Iran and the US, conduct qualitative interviews regarding moral values, expand the nomological network of “Qeirat” as a culture-specific set of moral values, and investigate the pragmatic validity of “Qeirat” in Iranian culture. Our findings suggest an additional moral foundation in Iran, above and beyond the five foundations identified by MFT. Specifically, qualitative studies highlighted the role of “Qeirat” values in Iranian culture, which are comprised of guarding and protectiveness of female kin, romantic partners, broader family, and country. -
What Teachers Know About Validity of Classroom Tests: Evidence from a University in Nigeria
IOSR Journal of Research & Method in Education (IOSR-JRME) e-ISSN: 2320–7388,p-ISSN: 2320–737X Volume 6, Issue 3 Ver. I (May. - Jun. 2016), PP 14-19 www.iosrjournals.org What Teachers Know About Validity of Classroom Tests: Evidence from a University in Nigeria Christiana Amaechi Ugodulunwa1,Sayita G. Wakjissa2 1,2Department of Educational Foundations, Faculty of Education, University of Jos, Nigeria Abstract: This study investigated university teachers’ knowledge of content and predictive validity of classroom tests. A sample of 89 teachers was randomly selected from five departments in the Faculty of Education of a university in Nigeria for the study. A 41-item Teachers’ Validity Knowledge Questionnaire (TVK-Q) was developed and validated. The data collectedwere analysed using descriptive statistics, t-test and ANOVA techniques. The results of the analysis show that the teachers have some knowledge of content-related evidence, procedure for ensuring coverage and adequate sampling of content and objectives, as well as correlating students’ scores in two measures for predictive validation. The study also reveals that the teachers lack adequate knowledge of criterion-related evidence of validity, concept of face validity and sources of invalidity of test scores irrespective of their gender, academic disciplines, years of teaching experience and ranks. The implications of the results were discussed and recommendations made for capacity building. Keywords:validity, teachers’ knowledge, classroom tests, evidence, university, Nigeria I. Introduction There is increased desire for school effectiveness and improvement at all levels of education in Nigeria. The central role of assessment in the school system for improving teaching and learning therefore demands that classroom tests should be valid and reliable measure of students’ real knowledge and skills and not testwiseness or test taking abilities. -
Assessment Center Structure and Construct Validity: a New Hope
University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2015 Assessment Center Structure and Construct Validity: A New Hope Christopher Wiese University of Central Florida Part of the Psychology Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Wiese, Christopher, "Assessment Center Structure and Construct Validity: A New Hope" (2015). Electronic Theses and Dissertations, 2004-2019. 733. https://stars.library.ucf.edu/etd/733 ASSESSMENT CENTER STRUCTURE AND CONSTRUCT VALIDITY: A NEW HOPE by CHRISTOPHER W. WIESE B.S., University of Central Florida, 2008 A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Psychology in the College of Sciences at the University of Central Florida Orlando, Florida Summer Term 2015 Major Professor: Kimberly Smith-Jentsch © 2015 Christopher Wiese ii ABSTRACT Assessment Centers (ACs) are a fantastic method to measure behavioral indicators of job performance in multiple diverse scenarios. Based upon a thorough job analysis, ACs have traditionally demonstrated very strong content and criterion-related validity. However, researchers have been puzzled for over three decades with the lack of evidence concerning construct validity. ACs are designed to measure critical job dimensions throughout multiple situational exercises. However, research has consistently revealed that different behavioral ratings within these scenarios are more strongly related to one another (exercise effects) than the same dimension rating across scenarios (dimension effects). -
Construct Validity and Reliability of the Work Environment Assessment Instrument WE-10
International Journal of Environmental Research and Public Health Article Construct Validity and Reliability of the Work Environment Assessment Instrument WE-10 Rudy de Barros Ahrens 1,*, Luciana da Silva Lirani 2 and Antonio Carlos de Francisco 3 1 Department of Business, Faculty Sagrada Família (FASF), Ponta Grossa, PR 84010-760, Brazil 2 Department of Health Sciences Center, State University Northern of Paraná (UENP), Jacarezinho, PR 86400-000, Brazil; [email protected] 3 Department of Industrial Engineering and Post-Graduation in Production Engineering, Federal University of Technology—Paraná (UTFPR), Ponta Grossa, PR 84017-220, Brazil; [email protected] * Correspondence: [email protected] Received: 1 September 2020; Accepted: 29 September 2020; Published: 9 October 2020 Abstract: The purpose of this study was to validate the construct and reliability of an instrument to assess the work environment as a single tool based on quality of life (QL), quality of work life (QWL), and organizational climate (OC). The methodology tested the construct validity through Exploratory Factor Analysis (EFA) and reliability through Cronbach’s alpha. The EFA returned a Kaiser–Meyer–Olkin (KMO) value of 0.917; which demonstrated that the data were adequate for the factor analysis; and a significant Bartlett’s test of sphericity (χ2 = 7465.349; Df = 1225; p 0.000). ≤ After the EFA; the varimax rotation method was employed for a factor through commonality analysis; reducing the 14 initial factors to 10. Only question 30 presented commonality lower than 0.5; and the other questions returned values higher than 0.5 in the commonality analysis. Regarding the reliability of the instrument; all of the questions presented reliability as the values varied between 0.953 and 0.956. -
Curriculum B.Sc. Programme in Clinical Genetics 2020-21
BLDE (DEEMED TO BE UNIVERSITY) Choice Based Credit System (CBCS) Curriculum B.Sc. Programme in Clinical Genetics 2020-21 Published by BLDE (DEEMED TO BE UNIVERSITY) Declared as Deemed to be University u/s 3 of UGC Act, 1956 The Constituent College SHRI B. M. PATIL MEDICAL COLLEGE, HOSPITAL & RESEARCH CENTRE, VIJAYAPURA Smt. Bangaramma Sajjan Campus, B. M. Patil Road (Sholapur Road), Vijayapura - 586103, Karnataka, India. BLDE (DU): Phone: +918352-262770, Fax: +918352-263303 , Website: www.bldedu.ac.in, E-mail:[email protected] College: Phone: +918352-262770, Fax: +918352-263019, E-mail: [email protected] BLDE (Deemed to be University) Vision: To be a leader in providing quality medical education, healthcare & to become an Institution of eminence involved in multidisciplinary and translational research, the outcome of which can impact the health & the quality of life of people of this region. Mission: To be committed to promoting sustainable development of higher education, including health science education consistent with statutory and regulatory requirements. To reflect the needs of changing technology Make use of academic autonomy to identify dynamic educational programs To adopt the global concepts of education in the health care sector Clinical Genetics 1 BLDE (Deemed to be University) Type Duration Course Hours/ IA Exam Total of of Exam Code Course Name Week Marks Marks Marks Credits paper (Hours) SEMESTER-I BCT 1.1 Fundamentals of T 4 3 30 70 100 2 Cell Biology BCT 1.1P Fundamentals of P 3 3 15 35 50 1 Cell Biology -
Reliability and Validity: How Do These Concepts Influence Accurate Student Assessment?
LEARNING POINT Reliability and validity: How do these concepts influence accurate student assessment? There are two concepts central to ac- scale reads 50 pounds heavier than Internal consistency reliability can curate student assessment: reliability expected. A second reading shows a be thought of as the degree to which and validity. Very briefly explained, weight that is 60 pounds lighter than responses to all the items on a test reliability refers to the consistency of expected. Clearly, there is something “hang together.” This is expressed as test scores, where validity refers to wrong with this scale! It is not mea- a correlation between 0 and 1, with the degree that a test measures what suring the person’s weight accurately values closer to 1 indicating higher re- or consistently. This is liability or internal consistency within the basic idea behind the measure. Internal consistency reliability for education- values are impacted by the structure al measurements. of the test. Tests that consist of more homogeneous items (i.e., all mea- In practice, education- sure the same thing) will have higher al assessment is not internal consistency values than tests quite as straightfor- that have more divergent content. For ward as the bathroom example, we would expect a test that scale example. There contains only American history items are different ways to to have higher reliability (internal think of, and express, consistency) values than a test that the reliability of educa- contains American history, African tional tests. Depending history, and political science ques- on the format and purpose of the it purports to measure. -
On the Validity of Reading Assessments GOTHENBURG STUDIES in EDUCATIONAL SCIENCES 328
On the Validity of Reading Assessments GOTHENBURG STUDIES IN EDUCATIONAL SCIENCES 328 On the Validity of Reading Assessments Relationships Between Teacher Judgements, External Tests and Pupil Self-assessments Stefan Johansson ACTA UNIVERSITATIS GOTHOBURGENSIS LOGO GOTHENBURG STUDIES IN EDUCATIONAL SCIENCES 328 On the Validity of Reading Assessments Relationships Between Teacher Judgements, External Tests and Pupil Self-assessments Stefan Johansson ACTA UNIVERSITATIS GOTHOBURGENSIS LOGO © STEFAN JOHANSSON, 2013 ISBN 978-91-7346-736-0 ISSN 0436-1121 ISSN 1653-0101 Thesis in Education at the Department of Education and Special Education The thesis is also available in full text on http://hdl.handle.net/2077/32012 Photographer cover: Rebecka Karlsson Distribution: ACTA UNIVERSITATIS GOTHOBURGENSIS Box 222 SE-405 30 Göteborg, Sweden Print: Ale Tryckteam, Bohus 2013 Abstract Title: On the Validity of Reading Assessments: Relationships Between Teacher Judgements, External Tests and Pupil Self-assessments Language: English with a Swedish summary Keywords: Validity; Validation; Assessment; Teacher judgements; External tests; PIRLS 2001; Self-assessment; Multilevel models; Structural Equation Modeling; Socioeconomic status; Gender ISBN: 978-91-7346-736-0 The purpose of this thesis is to examine validity issues in different forms of assessments; teacher judgements, external tests, and pupil self-assessment in Swedish primary schools. The data used were selected from a large-scale study––PIRLS 2001––in which more than 11000 pupils and some 700 teachers from grades 3 and 4 participated. The primary method used in the secondary analyses to investigate validity issues of the assessment forms is multilevel Structural Equation Modeling (SEM) with latent variables. An argument-based approach to validity was adopted, where possible weaknesses in assessment forms were addressed. -
EDS 245 Psychology in the Schools Stephen E. Brock, Ph.D., NCSP 1
EDS 245 Psychology in the Schools EDS 245: Psychology in the Schools Stephen E. Brock, Ph.D., NCSP California State University, Sacramento Introduction to Psychological Testing IMPORTANT!!! The final exam will emphasize the material covered in this second half of the course. Psychological Assessment: A Credentialing Standard NASP Practice Model: Domains of Practice Stephen E. Brock, Ph.D., NCSP 1 EDS 245 Psychology in the Schools Psychological Assessment: A Credentialing Standard NASP Practice Model: Domains of Practice Domain 1: Data-Based Decision-Making and Accountability: School psychologists have knowledge of varied models and methods of assessment and data collection for identifying strengths and needs, developing effective services and programs, and measuring progress and outcomes. Psychological Assessment: A Credentialing Standard NASP Practice Model: Domains of Practice Domain 1: Data-Based Decision-Making and Accountability: As part of a systematic and comprehensive process of effective decision making and problem solving that permeates all aspects of service delivery, school psychologists demonstrate skills to use psychological and educational assessment, data collection strategies, and technology resources and apply results to design, implement, and evaluate response to services and programs. Psychological Assessment: A Credentialing Standard NASP Practice Model: Domains of Practice Domain 1: Data-Based Decision-Making and Accountability: Examples of professional practices associated with data-based decision making and accountability include the following: School psychologists use a problem-solving frame-work as the basis for all professional activities. School psychologists systematically collect data from multiple sources as a foundation for decision-making and consider ecological factors (e.g., classroom, family, community characteristics) as a context for assessment and intervention in general and special education settings. -
Questionnaire Validity and Reliability
Questionnaire Validity and reliability Department of Social and Preventive Medicine, Faculty of Medicine Outlines Introduction What is validity and reliability? Types of validity and reliability. How do you measure them? Types of Sampling Methods Sample size calculation G-Power ( Power Analysis) Research • The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions • In the broadest sense of the word, the research includes gathering of data in order to generate information and establish the facts for the advancement of knowledge. ● Step I: Define the research problem ● Step 2: Developing a research plan & research Design ● Step 3: Define the Variables & Instrument (validity & Reliability) ● Step 4: Sampling & Collecting data ● Step 5: Analysing data ● Step 6: Presenting the findings A questionnaire is • A technique for collecting data in which a respondent provides answers to a series of questions. • The vehicle used to pose the questions that the researcher wants respondents to answer. • The validity of the results depends on the quality of these instruments. • Good questionnaires are difficult to construct; • Bad questionnaires are difficult to analyze. •Identify the goal of your questionnaire •What kind of information do you want to gather with your questionnaire? • What is your main objective? • Is a questionnaire the best way to go about collecting this information? Ch 11 6 How To Obtain Valid Information • Ask purposeful questions • Ask concrete questions • Use time periods -
How to Create Scientifically Valid Social and Behavioral Measures on Gender Equality and Empowerment
April 2020 EMERGE MEASUREMENT GUIDELINES REPORT 2: How to Create Scientifically Valid Social and Behavioral Measures on Gender Equality and Empowerment BACKGROUND ON THE EMERGE PROJECT EMERGE (Evidence-based Measures of Empowerment for Research on Box 1. Defining Gender Equality and Gender Equality), created by the Center on Gender Equity and Health at UC San Diego, is a project focused on the quantitative measurement of gender Empowerment (GE/E) equality and empowerment (GE/E) to monitor and/or evaluate health and Gender equality is a form of social development programs, and national or subnational progress on UN equality in which one’s rights, Sustainable Development Goal (SDG) 5: Achieve Gender Equality and responsibilities, and opportunities are Empower All Girls. For the EMERGE project, we aim to identify, adapt, and not affected by gender considerations. develop reliable and valid quantitative social and behavioral measures of Gender empowerment is a type of social GE/E based on established principles and methodologies of measurement, empowerment geared at improving one’s autonomy and self-determination with a focus on 9 dimensions of GE/E- psychological, social, economic, legal, 1 political, health, household and intra-familial, environment and sustainability, within a particular culture or context. and time use/time poverty.1 Social and behavioral measures across these dimensions largely come from the fields of public health, psychology, economics, political science, and sociology. OBJECTIVE OF THIS REPORT The objective of this report is to provide guidance on the creation and psychometric testing of GE/E scales. There are many good GE/E measures, and we highly recommend using previously validated measures when possible. -
Reliability & Validity
Another bit of jargon… Reliability & Validity In sampling… In measurement… • More is Better We are trying to represent a We are trying to represent a • Properties of a “good measure” population of individuals domain of behaviors – Standardization – Reliability – Validity • Reliability We select participants We select items – Inter-rater – Internal – External • Validity The resulting sample of The resulting scale/test of – Criterion-related participants is intended to items is intended to –Face – Content represent the population represent the domain – Construct For both “more is better” “more give greater representation” Whenever we’ve considered research designs and statistical conclusions, we’ve always been concerned with “sample size” We know that larger samples (more participants) leads to ... • more reliable estimates of mean and std, r, F & X2 • more reliable statistical conclusions • quantified as fewer Type I and II errors The same principle applies to scale construction - “more is better” • but now it applies to the number of items comprising the scale • more (good) items leads to a better scale… • more adequately represent the content/construct domain • provide a more consistent total score (respondent can change more items before total is changed much) Desirable Properties of Psychological Measures Reliability (Agreement or Consistency) Interpretability of Individual and Group Scores Inter-rater or Inter-observers reliability • do multiple observers/coders score an item the same way ? • critical whenever using subjective measures