New Culture of Checking and Assessing the Knowledge

Assist. Prof. Dr. Milena Ivanuš Grmek, University of Maribor, Faculty of Education Teaching Assist. Marija Javornik, University of Maribor, Faculty of EducationNew Culture of Checking and Assessing the KnowledgePaper presented at the European Conference on Educational Research, University of Hamburg, 17-20 September 2003 SummaryIn the first part, the article is talking about the impacts of external examination on learning and school lessons. Numerous researches and analyses are showing that students from specialized high schools retain incomplete, shallow and often completely incorrect ideas about some natural and social phenomena. They don’t know how to connect the knowledge that they have gained. The reason for the low applicability of knowledge and lack of retention is mainly due to the domination of shallow and reproductive studying, which is even encouraged by the external Matura. Multiple choice questions and tasks of low taxonomic level, which can be objectively measured or graded, predominate the external examination.The results in the second part of the article indicate that the knowledge gained in high schools is narrowing throughout the years of schooling. This mean, that with the approach of the final examination, exact summaries of learned material, definitions and information, are demanded from students more often at the end than at the beginning of high school. Students’ comprehension, linking the knowledge with other subjects and topics, critical thinking and applying the knowledge to new examples, is demanded more often at the beginning of schooling (in the first year) than in the forth year of high school (just before the Matura). Also, the democratisation of the grading process is decreasing throughout the schooling. Students are debating with the teachers about form and methods and contents of grading examinations more frequently in the first year than in the forth year of their schooling.Eg “conference paper”: presented in Network 9 (Student Assessment) at the ECER – 2003, Hamburg 1. IntroductionLately we have been paying more and more attention to consequential validity, which, together with consequential philosophy and ethics, is developing into a special research field of determining the consequences of examination and grading on studying and school lessons. These are consequences which ascertain and evaluate good and bad, are intended and unintended, and which can be short or long-term. It also examines the impact of (external) examination on knowledge quality, motivation, the independence of high school students, comprehension, and the treatment of teachers.Also, in Slovenia, consequential validity is being talked about more and more (Šimenc, 2000, Marentič Požarnik, 2000), especially because of the researches and analyses (Šteh Kure, 2000, Ilc, 2000, Javornik, 2002) about the occurrences in gymnasiums after 1995, when the external final examination (external Matura) was initiated. Since 1995, gymnasium students (gymnasium is school after obligatory primary school, which prepares students for study at university) in Slovenia finish their schooling with the external Matura, which contains 5 subjects (3 are obligatory: math, Slovene and a foreign language plus two optional subjects). The Matura is the last exam and it is on a national level. External means that the examination questions, standards and grading methods are not within the jurisdiction of the schools, but the National Examination Centre. All the candidates take the test under the same, pre- defined conditions. The importance of the Matura, which is the ending of high school and also the entrance ticket to University, is showing an impact on teachers, students and school lessons, which is decreasing the quality of students’ knowledge. The quality of knowledge is actually hard to define. The most recognized definition was from the view of durability and the applicability of the knowledge in new situations, or its transferable value. (Biggs, Moore, 1993, Biggs, 1999, Entwistle, 1988)The above-mentioned research and analyses are showing that students are only directed towards grades and passing the Matura. Too little attention is devoted to the meta-cognitive processes. While transferring the finished knowledge and memorized information, students don’t have enough time to discover and construct their own knowledge and achieve different views about things. They don’t have enough time for a deeper understanding of natural and social phenomena, which gymnasium students, as future students and representatives of the intellectual elite, should get a chance to do. In conclusion, the impacts of the Matura are shown in three directions:1. Breaking up of teaching material Students are stating, that they know how to find exactly defined information that will answer the test question.2. Deviation from teaching plan Teachers are leaving out material, which will not be included on the test.3. Trivialization of school lessons Students are taught to achieve only lower goals (understanding, application), and not the higher goals (analysis, evaluation), which are impossible to gain by means of tests. Activities, which require divergent and creative thinking, project work, cooperative studying, reading books, creative writing, etc., are reduced or even dismissed. Teachers are noticing that the integral approaches are being abandoned and that a behavioural model of teaching is becoming more and more dominant.2 Research Ascertainment 2.1 Purpose of the Research With the research accomplished in March 2003, we wanted to ascertain: - How are the teachers in gymnasium changing their grading requirements as the students approach the Matura (what knowledge are they actually assessing the best?)? - How is the democratisation of grading changing during the approach of the Matura?2.2 Methodology 2.2.1 The Method of the Research The Research is based on descriptive and causal – not experimental method of empirical pedagogical research (Sagadin, 1993).2.2.2 Sample In the research, we included first and fourth year gymnasium students in the school year 2002/2003. At the level of the inferential statistic, the captured sample represented a simple random sample from the hypothetical statistical mass. It consisted of 360 gymnasium students: 180 first year students and 180 fourth year students.2.2.3 Procedure of Collecting the Data The data was collected in March 2003 with a group poll, not a conducted poll. The instrument for collecting data was a questionnaire for high-school students. The questionnaire was structured from a complex (17) with four-degree descriptive grading scales. Students were grading the frequency of particular grading characteristics in two subjects: math and Slovene, which are obligatory subjects on the Matura.2.2.4 Data Processing We first carefully considered the data: descriptive grading scales were arranged using numerical values: always = 4, often = 3, sometimes = 2, never = 1. Then we statistically processed the data in the program, SPSS (version 11). For assessing the differences between the groups’ (between the first and the fourth years students) answers on each grading scale, we used the Mann-Whitney test.2.3 Results Review and Interpretation The results, arranged in two charts, are introduced in two complexes: - Results referring to democratisation of grading process - Results showing the content requirements for grading as well as openness of knowledge Results with statistically important differences between the years of schooling are in bold print. 2.3.1 Democratisation of the Grading ProcessThe democratisation rate of the examination and grading process was determined by the frequency of the seven categories written in the following charts.Chart 1: Analyses of the role that the schooling year plays on the democratisation of the grading process.Slovene Language Math Average Average Year N Z P Z P rank rank Teachers and first 180 189,20 189,98 students discuss -1,648 0, 099 -1,787 0,074 the grading fourth 180 171,80 171,02 content Teachers and first 180 189,30 192,35 students discuss -1,723 0,085 -2,328 0,020 the grading methods fourth 180 171,70 168,65 Teachers and students form the First 180 178,64 186,88 -0,430 0,667 -1,362 0,173 grading criteria together fourth 180 182,36 174,12 Teacher first 180 174,03 185,30 announces -1,247 0,212 -0,925 0,355 fouth 180 186,97 175,70 grading Teacher supports first 180 178,04 190,99 -0,465 0,642 -1,990 0,047 the grade fourth 180 182,96 170,01 Possibility for first 180 174,23 183,05 student -1,182 0,237 -0,482 0,630 complaints about fourth 180 186,77 177,95 the grade Possibility to first 180 175,14 189,86 -1,082 0,279 -1,928 0,064 rectify the grade fourth 180 185,86 171,14Analyses of the differences in the Slovene subject do not show any statistically important differences between the first and fourth year of schooling. The tendency is shown in the following categories: - Teachers and students discuss the grading content (Z = -1,648, P = 0,099). - Teachers and students discuss the grading methods (Z = -1,723, P = 0,095). This tendency indicates that teacher and student discussion about the grading content and methods is more frequent in the first year than in the fourth year of schooling.The results in math are a bit different. Statistically important differences are shown in two items: - Teachers and students discuss the grading methods (Z = -2,328, P = 0,020). The same tendency is shown also in Slovene. - Teacher supports the grade (Z = -1,928, P = 0,064).In both examples, these characteristics are more frequent in the first year of schooling.The tendency in math, and also in Slovene, is shown in the category, Teacher and Students Discuss the Grading Content (Z = -1,787, P = 0,074). These discussions are more frequent in the first year of schooling than in the fourth year of schooling. The tendency is shown also in the students’ possibility to rectify the grade (Z = -1,928, P = 0,064), which is becoming less possible as the school years progress toward the Matura.According to the results, the rate of democratisation of the grading process lowers as the school years progress towards the Matura.2. 3. 2 Grading Content Requirements We approached the phase of grading from a content level. We were ascertaining what teachers grade the best, which is also showing their content requirements for grading and, therefore, the openness of knowledge. The results are shown in the following chart. Chart 2: Analyses or the role that the school year plays on the grading content requirements Slovene Language Math Teacher grades Average Average Year n z P z P the highest: rank rank Exact summary of first 180 157,45 154,73 material from the -4,412 0,000 -4,489 0,000 text book fourth 180 203,55 206,27 Exact summary of first 180 166,09 167,99 notes taken from -2,811 0,005 -2,416 0,016 teacher’s fourth 180 194,91 193,01 explanation Amount of memorized first 180 167,69 161,20 information -2,474 0,013 -3,690 0,000 (names, numbers, fourth 180 193,31 199,80 dates, signs..) Word for word first 180 161,98 165,42 memorized -3,522 0,000 -2,892 0,004 fourth 180 199,02 195,58 definitions Subject first 180 181,66 186,42 -0,227 0,820 -1,175 0,240 comprehension fourth 180 179,34 174,58 Use of gained first 180 171,92 181,75 knowledge on -1,641 0,101 -0,237 0,813 fourth 180 189,08 179,25 new examples Student’s critical first 180 183,87 192,72 relation to school -0,667 0,505 -2,421 0,015 fourth 180 177,13 168,28 material Student’s first 180 192,82 193,67 personal views -2,364 0,018 -2,618 0,009 fourth 180 168,18 167,33 and deliberation Linking the first 180 181,54 196,60 knowledge with -0,206 0,837 -3,204 0,001 other subject and fourth 180 179,46 164,40 themes Student’s knew first 180 189,55 192,94 examples for the -1,774 0,076 -2,481 0,013 fourth 180 171,45 168,06 school materialFrom the chart, it is clear that there are statistically important differences between the first and fourth year of schooling. In Math and Slovene, as shown in the following categories, the teacher graded the highest. - Exact summary of material from the text book (in Slovene: Z = -4,412, P = 0,000; in Math: z = -4,489, P = 0,000),- Exact summary of notes taken from teacher’s explanation (in Slovene: Z = -2,811, P = 0,005; in Math: z = -2,416, P = 0,016), - Amount of memorized information (names, numbers, dates, signs..) (in Slovene Z = -2,474, P = 0,013; in Math: z = -3,690, P = 0,000) - Word for word memorized definitions (in Slovene Z = -3,522, P = 0,000; in Math: z = -2,892, P = 0,004).All listed demands are intensifying as the school years progress; therefore, the demands for exact information are more frequent in the fourth year than in the first year of schooling. The rest of the categories also display this tendency. The rest of the characteristics in the chart (without the once listed above) show an openness of the knowledge gained and the results are the opposite. The following statistically important characteristics (demands) appear more frequently in the first year than in the fourth year. - Teachers require students’ personal views and thinking while grading (in Slovene: z = -2,364, P = 0,018; in Math: z = -2,618, P = 0,009) - In Math (but not at Slovene) teachers require: students’ critical relation to school material (z = -2,421, P = 0,015), linking the knowledge with other subject and themes (z = -3,204, P = 0,001) new examples for the school material (z = -2,481, P = 0,013). In Slovene, the tendency of difference is shown as: (z = -1,774, P = 0,076). Statistically important differences between the two school years are not shown in subject comprehension, or in the use of gained knowledge in new material. In Slovene there are no statistically important differences in the following requirements: student’s critical relation to school material, linking the knowledge with other subject and themes, and students’ new examples for the school material.If we conclude: the results are showing that the students’ knowledge is “closing” as the school years progress. According to this, students’ personal views and their critical relations are less frequently demanded and wished for in the fourth year as they are in the first year of schooling. We can conclude that memorizing facts and formulas still have not been substituted for stimulation of comprehension, which also presents criticism and deliberation. Even though we cannot deny the need for objectivity and original dimensions of knowledge, the results can be alarming. We can look for the causes in the external final examination, which awaits gymnasium students at the end of high-school education and demands exactly defined and prescribed knowledge. Because of its objectivity and reliability, it gives priority to exams, which are simple and, especially, straight forward to grade. On the basis of our results, we can conclude that the teacher, above all, demands lower goals in their lessons. They do not demand higher goals, because they are harder to grade objectively and they are impossible to include on the Matura tests. This is confirming the results of other researches about the impact of the Matura on school lessons (Slivar 1997, Lipužič 1998, The second conference of the final exam 1998).3 Findings and Conclusion The purpose of the research was to reveal grading characteristics in gymnasium. With this, we answered two questions: how great is the rate of the democratisation of the grading process and to what extent can we speak about the openness of the knowledge when grading (about inter-subject connections, students’ personal views, critical thinking…) We presumed that as the school years progress (approach of the Matura), the characteristics, which are the democratisation of grading and openness of knowledge, will become less frequent than they were at the beginning of gymnasium schooling. Our presumptions were confirmed.While analysing the role of each year of schooling, we ascertained that all the characteristics, which are showing the democratisation of the grading process and the tendency of the differences, are more frequent in the first than in the fourth year of schooling. These characteristics are: - Teachers and students discuss the grading content - Teachers and students discuss the grading methods - Teacher supports the grade - Possibility to rectify the gradeIndividual characteristics that show the frequency of the rate of the democratisation of the grading process decline as the school years progress. Characteristic of the grading content and teachers’ demands while grading are showing that the characteristics, which are showing the openness of the knowledge, are more frequent in the first than in the fourth year of schooling. These are: - Students’ critical relation to school material - Students’ personal views and deliberation - Linking the knowledge with other subject and themes - Students’ new examples for the school materialTeachers demanded exactly memorized information more frequently in the fourth year than in the first year of schooling. - Exact summary of material from the text book - Exact summary of notes taken from teacher’s explanation - Amount of memorized information (names, numbers, dates, signs.) - Word for word memorized definitionsThe results proved our presumptions that the rate of democratisation of the grading process and the openness of the knowledge are lessening as the school years progress. According to some other research (Kompare, 1997, The Second Conference of the Final Exam, 1998, Šteh Kure, 1998, Brinovec, Juvan, 1999), we can state that the final external examination (the Matura), which strongly dictates the direction of school lessons in gymnasiums and has an impact on professors, students, studying and school lessons, intensifies productivity pressure and, therefore, hinders the achievement of goals like: increasing of self-initiative, autonomy, criticism and inventiveness by students and teachers.We can not deny the need for external examination as an important element to gain public trust in school institutions (Black, William, 1998), but it is necessary to evaluate the tests all the time. At the same time, we cannot forget that the Matura is not the only culprit for the existing situation. The Matura cannot be treated as being isolated from the school system, which (unfortunately) still relies on memorizing and transferring “finished” knowledge. On the other hand, the Matura does not lead to a better quality of teaching, studying and knowledge. References Bečaj, J, (2002). Šolska avtonomija: nedosegljiva nujnost? Sodobna pedagogika, 53(1), pp. 78-96. (School Autonomy: An Unachievable Necessity?), (Contemporary Pedagogy) Biggs, J. & Moore, P. J. (1993) Process of Learning, New Jersey, Prentice Hall.Biggs, J. (1999) Teaching for Quality Learning at University (Buckingham, Open University Press).Black, P. & Wiliam, D. (1998) Inside the Black Box. Raising Standards through Classroom Assessment (London, King's College).Bloom, B. (1956) Taxonomy of Educational Objectives. Classification of Educational Goals. Handbook I: Cognitive Domain (New York, David McKay Company).Boud, D. (1995) Assessment and Learning: Contradictory or Complementary? Knight, P. (Eds) Assessment for Learning in Higher Education (London, Kogan Page), pp. 35-48.Brinovec, D. & Juvan, R. (1999) Vplivi zunanjega preverjanja na pouk slovenskega jezika (The Impact of the External Matura on Slovene lessons) (Ljubljana, University of Ljubljana, Faculty of Arts).Drugi strokovni posvet o maturi. (The Second Conference on the Final Exam) (1998) (Ljubljana, DIC).Entwistle, N. (1988) Styles of Learning and Teaching (London, David Fulton Publ.).Ilc, Z. (2000) Poročilo o spremljavi vplivov mature na pouk (Ljubljana, Zavod Republike Slovenije za šolstvo in šport). (A Report on the Impact of the Matura on school lessons, Ljubljana, Institution for Sport and Education of The Republic of Slovenia)Javornik, M. (2002) Trajnost gimnnazijskega znanja iz slovenskega jezika in književnosti, Sodobna pedagogika, 53(2), pp.192-209. (Endurance of Secondary School Knowledge of the Slovenian Language and Literature, Contemporary Pedagogy) Kompare, A. (1997) Gospostvo na maturi. Didakta, 6(34-35), pp. 8-14. (Domination in the Matura, Didactics)Lipužič, B. (1998) Nevšečne primerjave. Šolski razgledi, XLIX, pp. 4. (Disagreeable Comparisons, School Views XLIX, pp. 4.)Marentič Požarnik, B. (1999a) Kakovost (gimnazijskega) znanja s konstruktivistične perspektive, Vzgoja in izobraževanje, 30(6), pp. 8-12. (Quality of Gymnasium Knowledge from a Constructivist Perspective, Education and Training, 30(6), pp. 8-12.),Marentič Požarnik, B. (1999b) Matura - dobra popotnica za prihodnje stoletje? Šolski razgledi, 50, pp. 5. (The Matura – Good Ration for the Future Century?, School Views 50, pp. 5.)Marentič Požarnik, B. (2000) Ocenjevanje učenja ali ocenjevanje za uspešno učenje? Vzgoja in izobraževanje, 31(2-3), pp. 4-9. (Assessing Learning or Assessing for (Effective) Learning? How to Reduce Inconsistency between the Purposes and Effects? Education and Training),Maturitetni izpitni katalog (1994) (Ljubljana, Republiški izpitni center). (The Matura Catalog, 1994, Ljubljana, National Examination Center)Polak, A. (1996) Subjektivne teorije učiteljev in študentov pedagoških smeri glede na smer izobrazbe in pedagoške izkušnje (Subjective Theories by Teachers and Students of Pedagogy According to the Direction of Education and Pedagogical Experience, Ljubljana, University of Ljubljana, Faculty of Arts).Rutar Ilc, Z. (1995) Zunanje zaključno preverjanje: neproduktivnost odločanja v opoziciji produktno – procesno. Didakta, 2(22-23), pp. 3-4. (External final examination: unproductiveness of decision-making in opposition of productive – process, Didactics)Rutar Ilc, Z. & Šteh Kure, B. (1999) Model evalvacije učnih aktivnosti v okviru gimnazijskega izobraževanja. Sodobna pedagogika, 50(4), pp. 70-87. (Model of Evaluation Study of the Remodeled Gymnasium Education, Contemporary Pedagogy)Saljő, R. (1979) Learning about Learning, Higher Education, 8, pp. 443-451.Sagadin, J. (1993) Poglavja iz metodologije pedagoškega raziskovanja (Ljubljana, Zavod Republike Slovenije za šolstvo in šport). (Chapters from the Pedagogical Research Methodology, Ljubljana, Institution for Sport and Education of The Republic of Slovenia) Slivar, B. (1997) Študija o nekaterih psiholoških in pedagoških vidikih vpliva mature na pouk (Ljubljana, Zavod republike Slovenije za šolstvo) (A Study of Some Pedagogical and Psychological Viewpoints on the Impact of the Matura on School Lessons, Ljubljana, Institution for Sport and Education of The Republic of Slovenia)Strmčnik, F. (2001) Didaktika (Didactics) (Ljubljana, University of Ljubljana, Faculty of Arts).Šimenc, M. (2000) K razmerju med zunanjim in notranjim preverjanjem znanja, Vzgoja in izobraževanje, 31(2-3), pp. 20-22. (Towards the Relation between External and Internal Examination of Knowledge, Education and Training) Šteh Kure, B. (1998) Sovplivanje pojmovanja znanja in učenja pri učiteljih in učencih (Co Influence of Knowledge Comprehension and Learning by Teachers and Students) (Ljubljana, University of Ljubljana, Faculty of Arts).Šteh Kure, B. (2000) Kakovost učenja in poučevanja v okviru gimnazijskega programa (The Quality of Learning and Teaching within the General Secondary School Program) (Ljubljana, University of Ljubljana, Faculty of Arts).

New Culture of Checking and Assessing the Knowledge

Details

Download

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

Support