DOCSLIB.ORG

Pisa-Based Test for Schools

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Pisa-Based Test for Schools PISA-BASED TEST FOR SCHOOLS TECHNICAL REPORT 2016 PISA-BASED TEST FOR SCHOOLS TECHNICAL REPORT 2016 This work is published under the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of OECD member countries. This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. Photo credits: © Khoa Vu/Flickr/Getty Images © Shutterstock/Kzenon © Simon Jarratt/Corbis © Stocklib Image Bank/No. 13883137 © OECD 2017 This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO (CC BY-NC-SA 3.0 IGO). For specific information regarding the scope and terms of the licence as well as possible commercial use of this work or the use of PISA data please consult Terms and Conditions on www.oecd.org. FOREWORD – 3 Foreword The OECD PISA-based Test for Schools assessment is geared for use by schools and networks of schools around the world to support research, international benchmarking and school improvement efforts. It collects information about 15-year-old students’ applied knowledge and competencies in reading, mathematics and science as well as their attitudes toward learning and school. PISA-based Test for Schools examines how students in the participating schools are prepared to meet the challenges of the future. The data collected by the assessment are an extremely valuable source of information for school principals, educators, parents and students. The methodology of the PISA-based Test for Schools is complex and demanding. The PISA-based Test for Schools Report describes those procedures and methodologies along with other features that enable the PISA-based Test for Schools to provide high-quality data to schools and local school administrations wanting to go further in understanding how their own individual schools perform compared with the world’s leading school systems. The descriptions are provided at a level that will enable replication of all implementation procedures. This Technical Report concerns the paper-and-pencil version of the PISA-based Test for Schools. The first edition of this report was drafted by Noémie Le Donné, Tue Halgreen and Kelly Makowiecki. The 2016 edition of the report was prepared by Javier Suárez-Álvarez with advice from Francesco Avvisati, Isabel Benitez, Richard Li, François Seyler and Tse Chi Sum, and under the supervision of Tue Halgreen. Administrative support for the 2016 edition was provided by Florence Bernard and Sophie Limoges co-ordinated production. PISA-BASED TEST FOR SCHOOLS TECHNICAL REPORT 2016 © OECD 2017 FOREWORD – 4 PISA-BASED TEST FOR SCHOOLS TECHNICAL REPORT 2016 © OECD 2017 TABLE OF CONTENTS – 5 Table of Contents 1. Introduction ........................................................................................................................................ 9 a. Features of the PISA-based Test for Schools .................................................................................. 9 b. Managing and implementing the PFS: Overview of roles and responsibilities ............................ 10 c. Structure of this report .................................................................................................................. 11 d. Additional information .................................................................................................................. 12 Chapter 1 Instrument design.................................................................................................................... 13 2. Test design, scope and development ................................................................................................ 13 a. Test design .................................................................................................................................... 13 b. Test scope ...................................................................................................................................... 14 c. Test development .......................................................................................................................... 15 d. Reporting PFS results on PISA scales .......................................................................................... 16 3. Context Questionnaires ..................................................................................................................... 17 a. Student Questionnaire ................................................................................................................... 17 b. School Questionnaire .................................................................................................................... 19 References ............................................................................................................................................... 21 Chapter 2 Operations ................................................................................................................................ 23 4. Overview of the roles and responsibilities ........................................................................................ 23 a. School Coordinators ...................................................................................................................... 23 b. Test Administrators ....................................................................................................................... 24 5. Preparing for implementation ........................................................................................................... 25 a. Translation, adaptation and verification of the test and survey material....................................... 25 b. Field trial participation and outcomes ........................................................................................... 27 6. Field operations ................................................................................................................................ 28 a. School and student eligibility to participate in the survey ............................................................ 28 b. Sampling ....................................................................................................................................... 29 c. Packaging and shipping materials ................................................................................................. 32 d. Receipt of materials by the National Service Provider after testing ............................................. 32 e. Coding of the cognitive test and of the context questionnaires ..................................................... 33 f. Coding the test booklets ................................................................................................................ 33 g. Student questionnaire coding ........................................................................................................ 34 h. School questionnaire coding ......................................................................................................... 35 Chapter 3 Data Processing ....................................................................................................................... 37 7. Constructing the initial PFS database ............................................................................................... 37 a. Files in the initial PFS database .................................................................................................... 37 b. Records in the database ................................................................................................................. 39 c. Representing missing data ............................................................................................................ 40 d. Merging the data files ................................................................................................................... 41 8. Procedures for scaling cognitive data ............................................................................................... 41 PISA-BASED TEST FOR SCHOOLS TECHNICAL REPORT 2016 © OECD 2017 6 – TABLE OF CONTENTS a. The mixed coefficients multinomial logit model .......................................................................... 41 b. Application to PFS ........................................................................................................................ 44 9. Procedures for constructing contextual variables ............................................................................. 52 a. Overview ....................................................................................................................................... 52 b. Simple questionnaire variables ..................................................................................................... 53 c. Questionnaire scale indices ........................................................................................................... 54 d. Other complex questionnaire variables ........................................................................................
Load more

Recommended publications
  • Understanding the Value of Arts & Culture | the AHRC Cultural Value
    Understanding the value of arts & culture The AHRC Cultural Value Project Geoffrey Crossick & Patrycja Kaszynska 2 Understanding the value of arts & culture The AHRC Cultural Value Project Geoffrey Crossick & Patrycja Kaszynska THE AHRC CULTURAL VALUE PROJECT CONTENTS Foreword 3 4. The engaged citizen: civic agency 58 & civic engagement Executive summary 6 Preconditions for political engagement 59 Civic space and civic engagement: three case studies 61 Part 1 Introduction Creative challenge: cultural industries, digging 63 and climate change 1. Rethinking the terms of the cultural 12 Culture, conflict and post-conflict: 66 value debate a double-edged sword? The Cultural Value Project 12 Culture and art: a brief intellectual history 14 5. Communities, Regeneration and Space 71 Cultural policy and the many lives of cultural value 16 Place, identity and public art 71 Beyond dichotomies: the view from 19 Urban regeneration 74 Cultural Value Project awards Creative places, creative quarters 77 Prioritising experience and methodological diversity 21 Community arts 81 Coda: arts, culture and rural communities 83 2. Cross-cutting themes 25 Modes of cultural engagement 25 6. Economy: impact, innovation and ecology 86 Arts and culture in an unequal society 29 The economic benefits of what? 87 Digital transformations 34 Ways of counting 89 Wellbeing and capabilities 37 Agglomeration and attractiveness 91 The innovation economy 92 Part 2 Components of Cultural Value Ecologies of culture 95 3. The reflective individual 42 7. Health, ageing and wellbeing 100 Cultural engagement and the self 43 Therapeutic, clinical and environmental 101 Case study: arts, culture and the criminal 47 interventions justice system Community-based arts and health 104 Cultural engagement and the other 49 Longer-term health benefits and subjective 106 Case study: professional and informal carers 51 wellbeing Culture and international influence 54 Ageing and dementia 108 Two cultures? 110 8.
    [Show full text]
  • A Comprehensive Framework to Reinforce Evidence Synthesis Features in Cloud-Based Systematic Review Tools
    applied sciences Article A Comprehensive Framework to Reinforce Evidence Synthesis Features in Cloud-Based Systematic Review Tools Tatiana Person 1,* , Iván Ruiz-Rube 1 , José Miguel Mota 1 , Manuel Jesús Cobo 1 , Alexey Tselykh 2 and Juan Manuel Dodero 1 1 Department of Informatics Engineering, University of Cadiz, 11519 Puerto Real, Spain; [email protected] (I.R.-R.); [email protected] (J.M.M.); [email protected] (M.J.C.); [email protected] (J.M.D.) 2 Department of Information and Analytical Security Systems, Institute of Computer Technologies and Information Security, Southern Federal University, 347922 Taganrog, Russia; [email protected] * Correspondence: [email protected] Abstract: Systematic reviews are powerful methods used to determine the state-of-the-art in a given field from existing studies and literature. They are critical but time-consuming in research and decision making for various disciplines. When conducting a review, a large volume of data is usually generated from relevant studies. Computer-based tools are often used to manage such data and to support the systematic review process. This paper describes a comprehensive analysis to gather the required features of a systematic review tool, in order to support the complete evidence synthesis process. We propose a framework, elaborated by consulting experts in different knowledge areas, to evaluate significant features and thus reinforce existing tool capabilities. The framework will be used to enhance the currently available functionality of CloudSERA, a cloud-based systematic review Citation: Person, T.; Ruiz-Rube, I.; Mota, J.M.; Cobo, M.J.; Tselykh, A.; tool focused on Computer Science, to implement evidence-based systematic review processes in Dodero, J.M.
    [Show full text]
  • Technical Writing
    Technical Writing Engineers and scientists perform many complex and intricate tasks using the world's most sophisticated equipment. However, their performance as engineers and scientists is almost always related to their use of one of the oldest tools - the pen. In academia, the saying "publish or perish" often describes the process of acquiring tenure as well as credibility. In industry, both large and small organizations communicate everything through memos, reports, and short presentations. Product development decisions are often made by a committee of people far removed from the actual technology. The saying "he who has the most convincing viewgraphs and reports, wins..." can sometimes apply to industry. Therefore, it should be clear that an ability to concisely and efficiently prepare technical reports, research papers, and or viewgraph presentations can have a profound positive impact on an individual's career. Consider the following statement by anonymous Fortune 500 corporate vice president: "... in any large organization, the person who decides whether you get a promotion, or who determines the size of a pay raise, does not know you personally. The only thing they have to go on is what other people write about you and what you write about you ..." It can be seen that if one should write a lot of material to get ahead in one's career, it makes sense to write as objectively and concisely as possible. Objective writing is essential because good technical writing should not be seen as erroneous after new discoveries are made. A good technical report should present a clear milestone of what was done and understood at the time of the writing.
    [Show full text]
  • Bibliometric Impact Measures Leveraging Topic Analysis
    Bibliometric Impact Measures Leveraging Topic Analysis Gideon S. Mann, David Mimno, Andrew McCallum Department of Computer Science University of Massachusetts Amherst Amherst MA 01003 {gmann,mimno,mccallum}@cs.umass.edu ABSTRACT tical methods that leverage the availability of large digital Measurements of the impact and history of research liter- document collections can enhance bibliometric analysis. ature provide a useful complement to scientific digital li- Discovering topical affinities between documents is an ac- brary collections. Bibliometric indicators have been exten- tive area of research in bibliometrics and scientometrics. In sively studied, mostly in the context of journals. However, particular, the use of journals as a representation of topics journal-based metrics poorly capture topical distinctions in is problematic for a variety of reasons. Journals generally fast-moving fields, and are increasingly problematic with the represent a variety of sub-areas and publications often com- rise of open-access publishing. Recent developments in la- bine multiple topical facets. Additionally, with the growth tent topic models have produced promising results for au- of open-access publishing, publication venue information is tomatic sub-field discovery. The fine-grained, faceted top- becoming increasingly dispersed and frequently simply un- ics produced by such models provide a clearer view of the available or undefined. topical divisions of a body of research literature and the There has been much work recently in machine learning interactions between those divisions. We demonstrate the on latent topic models, such as Latent Dirichlet Allocation usefulness of topic models in measuring impact by applying (LDA) [2], the Author-Topic model [22], and the Author- a new phrase-based topic discovery model to a collection Recipient-Topic model [19].
    [Show full text]
  • Benchmarking of Control Strategies for Wastewater Treatment Plants Layout, Controllers, Sensors, Performance Criteria and Test Procedures, I.E
    ©2020 The Author(s) This is an Open Access book distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying and redistribution provided the original work is properly cited. (http://creativecommons.org/licenses/by/4.0/). This does not affect the rights licensed or assigned from any third party in this book. This title was made available Open Access through a partnership with Knowledge Unlatched. IWA Publishing would like to thank all of the libraries for pledging to support the transition of this title to Open Access through the KU Select 2019 program. Downloaded from https://iwaponline.com/ebooks/book-pdf/650794/wio9781780401171.pdf by guest on 26 February 2020 Wastewater treatment plants are large non-linear systems subject to large 23 No. Report Technical and Scientific perturbations in wastewater flow rate, load and composition. Nevertheless these Scientific and Technical Report No. 23 plants have to be operated continuously, meeting stricter and stricter regulations. Many control strategies have been proposed in the literature for improved and more efficient operation of wastewater treatment plants. Unfortunately, their evaluation and comparison – either practical or based on simulation – are difficult. This is partly due to the variability of the influent, to the complexity of the biological and physico-chemical phenomena and to the large range of time constants (from a few seconds to several weeks). The lack of standard evaluation criteria is also a tremendous disadvantage. To really enhance the acceptance of innovative control strategies, such an evaluation Benchmarking of needs to be based on a rigorous methodology including a simulation model, plant Plants Treatment Strategies for Wastewater Benchmarking of Control layout, controllers, sensors, performance criteria and test procedures, i.e.
    [Show full text]
  • Technical Reports, Working Papers, and Preprints
    LIBRARY OF CONGRESS COLLECTIONS POLICY STATEMENTS Technical Reports, Working Papers, and Preprints Contents I. Scope II. Research Strengths III. Collecting Policy IV.Best Editions and Preferred Formats V. Acquisition Sources I. Scope This statement describes the Library's collection policies for technical reports, working papers, and preprints, in all subjects. These formats for publication are used by researchers or contractors to inform sponsoring agencies, peers, or others of the progress of research. This policy statement covers these formats when issued either by government or non-government publishers, or from both domestic and foreign sources. This statement is further limited to technical reports, working papers, and preprints that are issued in numbered or otherwise clearly identifiable series. Some technical reports, working papers, and preprints may be collected on a case-by-case basis rather than as a series, in which case they will fall under the Library's other Collections Policy Statements by subject rather than this Collections Policy Statement. Although the Library of Congress has a separate custodial Technical Reports collection, all Recommending Officers in appropriate fields are responsible for identifying series of technical reports, working papers, and preprints that are of interest to the Library's legislative, federal, and research clientele. The custodial location of reports acquired by the Library may include the Science, Technology & Business Division’s Automation, Collections Support and Technical Reports Section, the Serial and Government Publications Division, the Collections Management Division or any other appropriate custodial divisions, including Law and custodial area studies divisions. Format characteristics of technical reports, working papers, and preprints: The names given to these publication series vary.
    [Show full text]
  • Technical Report Writing
    Technical Report Writing CHAPTER 1 Introduction A. Course Overview An overview of this course is presented below as Figure 1-1. Take a moment to glance through the topics to be covered and then consider this one central idea. The premise of this course is that good technical reports don't just happen. They are "designed". Their design may be learned and that knowledge can be applied in your everyday work in the same way as you apply your other technical skills. We cannot all be William Shakespeare or Ernest Hemingway, but it is possible to acquire enough writing skill to write simple, declarative sentences and readable reports. Subjects you know about, you can also learn to write about. Figure 1-1 B. Types of Reports The suggestions presented in this course may be employed whether you are writing a single page letter report or a multi-volume research study. That is true because technical reports do, or at least should, have an explicit purpose and an identifiable audience. If you know clearly "why" you are writing and "who" will read your report, you have the basis for the report design. Your audience, and their needs, must remain at the forefront of your thinking as you write. In most organizations, the audience is a multi-layered collection of people. Some of your intended readers may be known to you while others may not. Once you have determined the purpose of your writing, you need to analyze your audience to the best of your ability. This course will suggest some ways to do that in systematic fashion.
    [Show full text]
  • Campus Climate Survey Validation Study Final Technical Report
    Bureau of Justice Statistics Research and Development Series Campus Climate Survey Validation Study Final Technical Report Christopher Krebs, Christine Lindquist, Marcus Berzofsky, Bonnie Shook-Sa, and Kimberly Peterson RTI International 3040 Cornwallis Road, P.O. Box 12194 Research Triangle Park, NC 27709-2194 Michael Planty, Lynn Langton, and Jessica Stroop Bureau of Justice Statistics 810 Seventh Street, NW Washington, DC 20531 R&DP-2015:04 NCJ 249545 January 2016 Research papers have been reviewed by BJS to ensure the accuracy of information presented and adherence to confidentiality and disclosure standards. This paper is released to inform interested parties about research and methodologies sponsored by BJS. Any opinions and conclusions expressed herein are those of the author(s) and do not necessarily represent the views of BJS and the U.S. Department of Justice. This report was prepared using federal funds provided by the Bureau of Justice Statistics, U.S. Department of Justice, under award number 2011-NV-CX-K068. The BJS project managers were Michael Planty, Victimization Unit Chief, and Lynn Langton, Senior Statistician. Campus Climate Survey Validation Study Abstract Presents the results of a nine-school pilot test that was conducted to develop a campus climate survey that collects school-level data on sexual victimization of undergraduate students. The report describes the development of the survey instrument and procedures for data collection, nonresponse bias analysis, weighting, and validity assessment. It presents estimates for each school on the prevalence and incidence of sexual assault, rape, and sexual battery during the 2014–15 academic year, as well as characteristics of the victims and incidents.
    [Show full text]
  • Guidelines for the Production of Scientific and Technical Reports: How to Write and Distribute Grey Literature
    Guidelines for the production of scientific and technical reports: how to write and distribute grey literature Version 1.0 Grey Literature International Steering Committee March 2006 Guidelines for the production of scientific and technical reports: how to write and distribute grey literature Version 1.0 Grey Literature International Steering Committee March 2006 This document was prepared by: Paola De Castro and Sandra Salinetti Istituto Superiore di Sanità, Rome – Italy and critically revised by: Joachim Schöpfel and Christiane Stock Institut de l’Information Scientifique et Technique (INIST-CNRS), Nancy – France Dominic Farace Grey Literature Network Service (GreyNet), Amsterdam – The Netherlands Catherine Candea and Toby Green Organization for Economic Co-operation and Development (OECD), Paris – France Keith G. Jeffery Council for the Central Laboratory of the Research Councils (CCLRC), Chilton Didcot – UK Acknowledgements to: Marcus A. Banks (Memorial Sloan-Kettering Cancer Center, New York – USA) Stefania Biagioni (Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, ISTI-CNR, Pisa – Italy) June Crowe (Information International Associates Inc., IIA, Oak Ridge – USA) Markus Weber (Swiss Federal Office of Public Health, SFOPH, Berne – Switzerland) This document is available from: www.glisc.info Any request shall be addressed to [email protected] © Grey Literature International Steering Committee (GLISC) 2006 TABLE OF CONTENTS 1. Statement of purpose..............................................................................................................................1
    [Show full text]
  • A Framework to Monitor Open Science Trends in the EU
    OECD Blue Sky III Forum, Informing science and innovation policies: Towards the next generation of data and indicators. 19 – 21 September 2016, Ghent, Belgium A framework to monitor open science trends in the EU Elta Smith ([email protected]), Salil Gunashekar ([email protected]), Catherine Lichten, Sarah Parks, Joanna Chataway RAND Europe, Cambridge, United Kingdom Abstract Open science refers to ongoing changes in the way research is conducted, with a move towards increased transparency, collaboration, communication and participation. Open science involves changes for scientists themselves, such as open access to scientific papers and open research data, as well as increased understanding of and participation in science by citizens. These changes are expected to enable greater transparency, collaboration and research integrity in the short term and improve scientific quality in the long term. Open science is one of three priority areas for European research, science and innovation policy. A challenge for policymaking in this emerging area, however, is a lack of evidence regarding open science. In this context, RAND Europe and our partners—Deloitte, Observatoire des Sciences et des Technologies, Altmetric and Digital Science— are developing a monitor on behalf of the European Commission, Directorate-General for Research and Innovation. The monitor will track open science trends in Europe and identify the main drivers, incentives and constraints on its evolution. RAND Europe will also develop a website to host the monitor that will provide policymakers and stakeholders with access to data on open science. Introduction In the past two centuries, a picture has emerged of scientists toiling away in labs and offices.
    [Show full text]
  • Guide for Writing Technical Reports
    GUIDE FOR WRITING TECHNICAL REPORTS DISCIPLINE OF MECHANICAL ENGINEERING May 2015 Acknowledgement Adapted from the Guide for Writing Technical Reports (Third Edition, 2007) by AH Basson and TW von Backström of the Faculty of Engineering, University of Stellenbosch, by Michael Brooks and Fran Saunders. Our thanks go to Professor Basson for generously allowing us to use their material. Abstract This guide provides guidelines to engineering students for writing technical reports, theses, and dissertations. It provides a discussion of the content of the various main elements of a technical report and gives the recommended format. Important elements of good microstructure and style plus practical guidelines for the compilation of a technical report are also provided. The guide concludes with a checklist that can be used to eliminate general mistakes. The appendices provide guidelines for the content of reports on experiments, designs and calculations, guidelines for thesis proposals, use of SI units, and use of the Harvard referencing method. 2 TABLE OF CONTENTS Page 1. INTRODUCTION 5 2. THE PROCESS OF COMPILING A TECHNICAL REPORT 5 3. EXTERNAL STRUCTURE 5 3.1 Sequence of Main Sections 5 3.2 Cover Page 6 3.3 Title Page 6 3.4 Abstract 6 3.5 Dedication 7 3.6 Acknowledgements 7 3.7 Table of Contents 7 3.8 List of Tables and List of Figures 7 3.9 Nomenclature 7 3.10 Introduction 7 3.11 Central Chapters 8 3.12 Conclusions 8 3.13 Tables and Figures 9 3.14 Appendices 10 3.15 References 10 3.15.1 The alphabetical system 10 3.15.2 The numerical system 10 3.16 Bibliography 10 4.
    [Show full text]
  • How to Write a Good Technical Paper
    Point of view How to Write a Good Technical Paper Learn how to write easy-to-read technical papers BY RACHEL J. DETWILER AND DAVID DARWIN he ability to communicate is a Because a technical paper should abstract. The abstract condenses T critical skill that not only reconstruct the process of the the main ideas of the paper into a affects our success as profession- investigation, the generic outline of single paragraph (or several als, but also determines how a paper closely resembles the steps paragraphs for a long report). An effectively lessons from the labora- of the scientific method: informative abstract includes a tory and field are transferred into 1. Identification of the problem summary of the investigation and general practice. As members of or question to be investigated; the main findings. ACI’s Publications Committee, and 2. Evaluation of previous work in The first sentence or two of the as authors ourselves, we offer some the field; abstract should give the context of guidelines on one form of communi- 3. Formulation of the hypothesis the investigation and the nature of cation: writing technical papers. or definition of the key aspects of the work. In other words, it tells You can apply the principles we the investigation; why the study is relevant and of discuss to formal technical reports, 4. Design of experiments, obser- interest and what you investigated. reports to clients, conference vations, or both; The next two or three sentences papers, and journal articles. Pub- 5. Gathering of data; should describe the scope of the lishing organizations prescribe the 6.
    [Show full text]