University of Education, Lahore
Total Page:16
File Type:pdf, Size:1020Kb
DEVELOPMENT OF MATH PROFICIENCY TEST USING ITEM RESPONSE THEORY (IRT) MUHAMMAD AZEEM Reg. No. 0499086 DIVISION OF EDUCATION UNIVERSITY OF EDUCATION, LAHORE 2012 DEVELOPMENT OF MATH PROFICIENCY TEST USING ITEM RESPONSE THEORY (IRT) MUHAMMAD AZEEM Reg. No. 0499086 Submitted in partial fulfillment of the requirement for the degree of Doctor of Philosophy in Education at Division of Education, University of Education, Lahore 2012 ii IN THE NAME OF ALLAH, WHO IS MOST MERCIFUL AND BENEFICENT iii This Work is dedicated to: MUHAMMAD (PEACE BE UPON HIM) iv PhD Thesis Certificate of Acceptance Name of Student Student No. Programme Muhammad Azeem 04-PhD-001 PhD (Education) TOPIC: DEVELOPMENT OF MATH PROFICIENCY TEST BASED ON ITEM RESPONSE THEORY (IRT) AT SECONDARY LEVEL Date of Acceptance: ___________________ THESIS COMMITTEE SR. # Name Position Signature 1. 2. 3. 4. 5. v ACNOWLEDGEMENTS Firstly, I owe to the almighty ALLAH, The Most Merciful and The Most Beneficent, who give me courage to accomplish this research work. The PhD scholar is deeply indebted to Dr. Muhammad Bashir Gondal, supervisor of this doctoral research work, for his affection, guidance, interest, encouragement and supportive attitude, and constructive advice which enlightened my ways in completion of my research project. Besides it, researcher is also indebted to him for providing necessary facilities and feedback during this research project that helped me a lot in accomplishment of my research project. Special bundle of thanks go to Dr Izhar-ul-Haq (Late) for encouragement and supportive attitude, and constructive advice for completion of my doctoral research at University of Education. May ALLAH rest his soul in paradise (Ameen). Special bundle of thanks go to Dr. Muhammad Khalid Mehmood, Assisstant professor, University of Education, Lahore, for his encouragement and supportive attitude for data analysis by using IRT software. Thanks go to Dr. Glenn Fulcher, Dr Chris Comber, Dr Janet Ainley, and Dr Saeeda Shah, faculty members of School of Education, University of Leicester, UK for their encouragement and help in research work. Thanks go to all other faculty members especially Dr Muhammad Khalid Ibrahim and Dr Shahbaz Khan of Division of Education, University of Education, Lahore, for their helping and supportive attitude. Thanks go to Dr Muhammad Saeed, Madam Bushra and Muhammad Tanveer Iqbal professionals working in PITE, Lahore for their help in Item development and test construction. Thanks go to Ch. Munir Ahmed, principal, GHSS Chak Jhumra, for granting study leave and supporting in this research work. Thanks go to Muhammad Tanveer Afzal, PhD Scholar, UE, Lahore, Muhammad Idrees Asad, PhD Scholar, UE, Lahore, Naseer Ahmad, PhD Scholar, UE, Lahore, Khalil-ur-Rehman, PhD Scholar, UE, Lahore, Noor Muhammad Asif, PhD Scholar, UE, Lahore for their help in construction & validation of math proficiency test and data collection. Thanks go to all prospective teachers of Division of Education, University of Education, Lahore, and all teacher educators (T.E.) working in all district of PUNJAB province especially Saadia Hadiat, Sahiwal, Nasrullah Bhatti, Bhakkar, Amjad Mahmood, Attock, Asmat Ullah, DG Khan, Muhammad Ishfaq Ahmad, Sargodha, Muhammad Akram Junaid, Lodhran, Muhammad Arshad Javid, Bahawalnagar, Malik Muhammad Ahmed, Bahawalpur, Mr Amjad Ali Bhatti, Hafizabad, for their sincere, honest, and swift participation as test administrators. Special thanks go to Syed Afaq Assisstant professor Math, divisional educational technology resource centre, Madam Shafia Assisstant Professor Math, federal college of education, Islamabad, Qasimabad, Karachi, Shabana Haider Assisstant professor Math, Govt. Elementry College, Hussainabad, Karachi, Irfan Ullah Quraishi Assisstant professor Math, Govt. F.B. area Karachi, Nadeem Sultan SSS Math PEACE, NWFP, Naveed Alam Math teacher, Leicester Islamic Academy, UK, Thanks go to Muhammad Nusrullah, Assistant Education Officer, Chishtian, Munir Hussain PhD Scholar, IER, PU, Lahore, Shafqat Hussain SSS statistics, GCET, vi Mianwali, Muhammad Akram SS economics, GHSS Jalalpur Pirwala, Multan, Rai Manzoor Ahmad, SSS biology GCET Muzaffargarh, Zulfiqar Ali, Executive District Officer EDO, Nankana Sahib, Bakar Hussain Tariq SST, GHS 203RB Faisalabad, Imran Ashraf SST, GHS 493 JB, Jhang, Omer Draz, SSS Math GHSS Lalian, Muhammad Ajmal Headmaster Govt. Model H/S Khanewal, Munir Ahmad, Senior Headmaster GHS 202RB Faisalabad, Rasid Ahmad Khawer SST, GHS Pakpatan, Muhammad Khalid SST, GHS Salarwala, Faisalabad, Nadeem AT, GES 190RB Faisalabad, Muhammad Afzal HM, GHS Dijkot, Faisalabad, Bakar Hussain Tariq SST GHS Manawala, Faisalabad, for their sincere, honest, and swift participation as test administrators. Thanks go to Abdul Ghaffar, SST, GHSS Chak Jhumra, Zulfiqar Ali, SV, GHSS Chak Jhumra, Amjad Naseem Ansari DTE, GHS Salarwala for their sincere, honest, and swift participation as test administrators and translation of English version of Math Proficiency Test in to Urdu version. Thanks go to David J. Weiss, Professor of Psychology at Applied Psychological Measurement Inc. member of APM Graduate Student Grants committee, N660 Elliott Hall, University of Minnesota Minneapolis MN 55455-0344, U.S.A. for providing IRT Software Multilog 7 and IRT related books. Thanks go to Susan Cooper Loomis, Assistant Director, Psychometrics, National Assessment Governing Board, USA, for granting permission to adapt NAEP’s Math assessment framework in this research study. Special thanks go to higher education commission HEC of Pakistan for providing funds for doctoral fellowship at Division of Education, University of Education, Lahore and funds for foreign visit of School of Education, University of Leicester, UK, under International Research Support Initiatives Program IRSIP to complete this thesis. Thanks go to Shah Nawaz IT specialist of PEAS for his support in editing and designing thesis. Thanks go to all those who helped me in my research work but I have forgot because of deeply engagement with research activities. Special thanks go to all my family members for suffering a lot of due to my busyness and friends for supporting in this research work. Muhammad Azeem vii DEVELOPMENT OF MATH PROFICIENCY TEST USING ITEM RESPONSE THEORY (IRT) ABSTRACT This study was conducted to develop a math proficiency test using IRT, based on NAEP’s adapted Mathematics Framework, and aligned with the newly developed standards, and benchmarks of National Mathematics Curriculum. Mathematical Proficiency is the ability to use the mathematical power for conceptual understanding, procedural knowledge, and problem-solving in the real world using appropriate strategies. One hundred and ninety six multiple choice and short constructed response items were developed. Due to limitations of the study only, multiple choice and short constructed response, dichotomously scored items were developed. Items were spot tested on 200 students and piloted on 550 students. A final 60-item math proficiency test was constructed. Stratified cluster sampling technique was used for administration of final math proficiency test. All the 9th grade students studying in public high and higher secondary schools in the province of Punjab comprised the population of the study. Sample of 2680 students and 134 schools was selected using sample design tables and IRT based analysis requirements. Final math proficiency test was administered and data was received from 2617 students. Data was analyzed by using SPSS, Conquest, and Multilog software. Data was well fitted with Rasch Model. The infit and outfit means square statistics of items was within 0.8 to 1.30. The reliability and discrimination index of final math proficiency test was above 0.90 and 0.45 respectively. The IRT based person-item map showed that test items covered ±3 range of abilities. Test characteristics and item characteristics curves, factor analysis, and dimensionality analysis also supported the reliably and validity of the test. Different viii estimations-WLE, EAP, and MLE also supported reliability and validity of test items. Estimations WLE, EAP, and MLE have advantages and disadvantages over each other. For this study the average of these estimations was used as proficiency score for each student. It explored that math proficiency test is appropriately constructed. Replication of this study is recommended by using 2PLM and 3PLM of Item Response Theory (IRT). ix DECLARATION I, Muhammad Azeem S/O Muhammad Yaqub Registration No. 0499086, a student of PhD at the University of Education, do hereby solemnly declare that the thesis entitled “Development of Math Proficiency Test Using Item Response Theory (IRT)” submitted by me in partial fulfillment of PhD degree (Education) is my original work, except where otherwise acknowledge in the text, and has not been submitted or published earlier and shall not, in future, be submitted by me for obtaining any degree from this or any other University or institution. Signature_______________________ Muhammad Azeem x TABLE OF CONTENTS Sr. No. Contents Page No. i Title i, ii ii In the name of Allah, Who is most merciful and beneficent iii iii Dedication to Muhammad (P.B.U.H) iv iv Approval Sheet v v Acknowledgments vi vi Abstract viii vii Declaration x viii Table of Contents xi viii List of Appendices xviii ix List of Tables xix x List of Figures xxi xi Acronyms xxiii CHAPTER NO. 1 : INTRODUCTION 1.1 Introduction 1 1.2 Statement of Problem 4 1.3 Significance of the study 5 1.4 Objectives of Study 7 1.5