New Assessment Tools for Cross-Curricular Competencies in the Domain of Problem Solving
Total Page:16
File Type:pdf, Size:1020Kb
New Assessment Tools for Cross-Curricular Competencies in the Domain of Problem Solving Final report of project ERB-SOE2-CT98-2042 Funded under the Targeted Socio-Economic Research (TSER) Programme - Directorate General XII Science, Research and Development/ Directorate F EUROPEAN Commission Project coordinator: Dr. Jean-Paul Reeff Ministère de l´Education Nationale, de la Formation Professionnelle et des Sports 29, rue Aldringen L-2926 Luxembourg Partners: Institut für Bildungsforschung (IBF), Germany Max Planck Institute for Human Development, Germany Center for School Development, Austria University of Groningen, Netherlands University of Heidelberg, Germany 1 Executive Summary.......................................................................................................3 2 Individual network members..........................................................................................4 3 Background and objectives of the project .......................................................................5 4 Network activities ..........................................................................................................7 5 Main Results................................................................................................................11 5.1 Problem Solving as a Cross-Curricular Competence.................................................11 5.1.1 Read instruction ...............................................................................................11 5.1.2 Method.............................................................................................................12 5.1.3 Introduction into Problem Solving as a Cross Curricular Competence...............12 5.1.4 Theoretical approaches towards Problem Solving .............................................15 5.1.5 Available instruments to measure problem solving competence ........................22 5.1.6 Conclusions......................................................................................................27 5.1.7 References........................................................................................................29 5.2 Problem Solving in Vocational Training and Education............................................31 5.2.1 Point of departure.............................................................................................31 5.2.2 Method.............................................................................................................31 5.2.3 Results .............................................................................................................32 5.2.4 Conclusion .......................................................................................................36 5.2.5 References........................................................................................................38 5.3 Alternative Concepts and Tools................................................................................41 5.3.1 Introduction......................................................................................................41 5.3.2 Method.............................................................................................................41 5.3.3 New assessment tools .......................................................................................41 5.3.4 Next steps.........................................................................................................43 5.3.5 Conclusion .......................................................................................................45 5.3.6 References........................................................................................................46 5.4 Assessment of Cross-Curricular Problem-Solving Competencies in Large-Scale Surveys. ..........................................................................................................................47 5.4.1 Method.............................................................................................................47 5.4.2 The concept of Problem Solving.......................................................................48 5.4.3 The assessment of Problem Solving across and cross domains in PISA.............54 5.4.4 Indicators of problem solving competencies in large scale assessments.............56 5.4.5 Conclusions......................................................................................................63 5.4.6 References........................................................................................................65 6 Conclusions and policy implications ............................................................................70 7 Annex..........................................................................................................................72 7.1 Problem Solving Competency of Students: Ad-hoc Questionnaire for Education Policy Makers..................................................................................................................73 2 1 Executive Summary The NATCCC-PS network has been created as a small and short-term co-operation network of European researchers to improve the visibility and to increase the impact of European research in the field of Problem Solving, with special emphasis given to large scale international comparative studies. The main objectives of the network were - to connect European policy demands with European experts and experience in the field of Problem Solving; - to bring together a range of scientific experts working in the field of Problem Solving to develop new concepts and tools, and to create synergies; - to produce a research and development plan directed to educational indicators in the field of Problem Solving. While weak on the first objective, the network proved to be extremely successful on the two remaining goals. The networks' activities had significant impact both on the OECD PISA study and the Adult Literacy and Life Skills Survey (ALL). The activities went far beyond the originally defined work plan and generated both new research projects and applied research transfer contracts. 3 2 Individual network members who actively participated in the preparation of this report Judith Ebach (IBF) Birgit Eschmann (MENFPS) Joachim Funke (Univ. Heidelberg) Eckhard Klieme (MPI) Iris Langer (MENFPS) Jules Peschar (Univ. Groningen) Jean-Paul Reeff (MENFPS) Astrid Schorn-Buchner (MENFPS) Erich Svecnik (ZFS) Hendrik Vogt (IBF) Marieke van der Waal (Univ. Groningen) Sietske Waslander (Univ. Groningen) Joachim Wirth (MPI) 4 3 Background and objectives of the project One of the biggest challenges to educational research and policy is providing relevant information regarding the education system's outcome at different levels. Student outcome indicators are of special interest as obviously this information is a primary criterion for different activities such as teaching, assessment, quality improvement programs, evaluation studies, and steering (as expressed by the French word "pilotage") the educational systems. In the last few decades a major effort was exerted at the international level (e.g. IEA, OECD, EU) to develop student outcome indicators for comparative purposes. The most recent of these enterprises, the Third International Mathematics and Science Study (TIMSS) conducted by IEA between 1994 and 1996, exemplifies the progress that has been made. In the next decade, a major source of information about education will be provided by a recently launched OECD-study in 26 countries (comprising all EU-countries). While the main focus of the three three-year surveys will be on Reading Literacy, Mathematics and Science, all three domains will be included in all of the surveys. The effort required to carry out this study is substantial. It requires well-tuned and well co-ordinated actions within and between countries. This is especially true for European countries, where the need for co-ordinated actions and intra-European comparative studies has been stressed in all major EU-programs during the last several years. Substantial scientific improvements have been made in the field of student outcome indicators over the years. The focus of these studies is, however, limited to the traditional assessment domains of Reading, Mathematics and Science. Due to rapid technological and social changes there is a growing demand for competencies beyond these traditional domains, such as problem-solving, communication skills, and learning to learn. There is thus a widely acknowledged need for broader indicators and the ability to monitor these competencies that lie outside traditional domains. To date, the most successful and innovative initiative to develop additional educational indicators was conducted by the OECD. A small component on cross-curricular competencies has been included in the first cycle of the PISA study, results will be available in autumn 2001. The DeSeCo (Definition and Selection of Competencies) initiative, studies the theoretical foundations of Cross-Curricular Competencies and key qualifications and will propose relevant competencies to be assessed in the futures. The CCC-initiative will have a major impact on advancements made in the field of student outcomes in the years ahead. For education policy and education science alike, it is therefore highly important that countries of the European Union ensure the contributions of their scientific expertise and ensure incorporation of a European perspective into these new developments. Beyond the initiatives