The Chair of Pedagogy and Language
Susanna Asatryan
PhD, professor assistant
The Chair of pedagogy and language
teaching methodology
Testing
The aim of this lecture
§ To think about language testing as a tool
§ To provide information about language testing
§ To reflect on testing language competence and communicative skills[1].
A definition of a “language test” is a tool for measuring language performance in learners (Bachman, L., and Palmer, A. 1997. Language Testing in Practice. p. 8) A fundamental principle in language testing is “correspondence between language test performance and real world language use”.
It is important to understand the difference between testing and teaching. In some ways the two are so interwoven and interdependent that it is difficult to tease them apart.
Every instructional sequence, has a testing component to it, whether the tests themselves are formal or informal. That is, teachers measure or judge learners' competence all the time and, ideally, learners measure and judge themselves[2].
Whenever a student responds to a question or tries out a new word or structure, you might be testing that student.
Written work is a test. Oral work is a test. Reading and writing performance are tests.
How, then, are testing and teaching different?
The difference lies in what we'll call formal and informal testing.
Pedagogically, it is very important to make the distinction between teaching and formal testing, especially from the point of view of principles of intrinsic motivation. For optimal learning to take place, students must have the freedom in the classroom to experiment, to try things out, to "test" their own hypotheses about language without feeling that their overall competence is being "judged" in terms of these trials and errors.
The Principles Towards Creating Intrinsically Motivating Tests
1. The principle of giving students advance preparation
This may sound simple, but much too often teachers do little to help students to prepare for a test. Tests, by their very nature, are anxiety-raising experiences. Students don't know what to expect. And they may not be aware of test-taking strategies that could help them. So, the teacher's task in creating intrinsically motivating tests is to be an ally in the preparation process.
He can do even more:
§ Provide information about the general format of a test
§ Provide information about types of items that will appear
§ Give students opportunities to practice certain item types
§ Encourage a thorough review of material to be covered
§ Offer advice on strategies for test preparation
§ Offer advice on strategies to use during the test itself
§ Give anxiety-lowering reassurance.
2. The principle of face validity
Sometimes students don't know what is being tested when they tackle a test. Sometimes they feel, for a variety of possible reasons, that a test isn't testing what it is "supposed" to test. Face validity means that the students, as they perceive the test, feel that it is valid. You can help to foster that perception with:
1. a carefully constructed, well thought-out format
2. items that are clear and uncomplicated
3. directions that are crystal clear
4. tasks that are familiar, that relate to their course work
5. a difficulty level that is appropriate for your students
6. test conditions that are biased for best-that bring out students' best performance.
3. The principle of authenticity
Make sure that the language in your test is as natural and authentic as possible. Also, try to give language some context so that items aren't just a string of unrelated language samples. Thematic organization of items may help in this regard.
4. The principle of "washback"
"Washback" is the benefit that tests offer to learning. When students take a test, they should be able within a reasonably short period of time to utilize the information about their competence that test feedback offers. Formal tests must therefore be learning devices through which students can receive a diagnosis of areas of strength and weakness. Their incorrect responses can become windows of insight about further work. Your prompt return of written tests with your feedback is therefore very important to intrinsic motivation.
Some Practical Steps to Test Construction
1. Test toward clear, unambiguous objectives
You need to know as specifically as possible what it is you want to test. Sometimes teachers give tests simply because it's Friday or it's the third week of the course. This is no way to approach a test. Carefully list everything that you think your students should "know" or be able to "do," based on the material that students are responsible for.
2. Draft your test
A first draft will give you a good idea of what the test will look like, how students will perceive it (face validity), the extent to which authentic language and contexts are present, the length of the listening stimuli, how well a story line comes across, how things like the cloze testing format will work, and other practicalities.
4.Revise your test
At this stage, you will work through all the items you have devised and ask a number of important questions:
§ Are the directions to each section absolutely clear?
§ Is there an example item for each section?
§ Does each item measure a specified objective?
§ Is each item stated in clear, simple language?
§ Does each multiple choice item have appropriate distracters, that is, are the wrong items clearly wrong and yet sufficiently "alluring" that they aren't ridiculously easy?
§ Does the difficulty of each item seem to be appropriate for your students?
§ Does the sum of the items and and test as a whole adequately reflect the learning objectives?
5. Utilize your feedback after administering the test.
After you give the test, you will have some "information about how easy or difficult it was, about the time limits, and about your students' affective reaction to it and their general performance. Take note of these forms of feedback and use them for making your next test.
6. Work for washback.
As you evaluate the test and return it to your students, your feedback should reflect the principles of washback discussed earlier. Use the information from the test performance as a springboard for review and/or for moving on to the next unit.
Three Criteria of a Good Test
1. A good test will seem fair and appropriate to the students (and to anyone who needs to know the results, e.g. head teacher, other teachers, employers, parents, etc.).
2. It will not be too troublesome to mark.
3. It will provide clear results that serve the purpose for which it was set.
Types of Tests and Testing
Generally we distinguish two main categories of tests:
You could test:
§ the students' progress over the course so far (a progress test /achievement test);
§ their general level of English, without reference to any course (a proficiency test) [3].
Most internal school tests tend to be progress tests; most external ones (e.g. state or international exams) are usually proficiency tests.
You can test anything that has been studied; this usually means the four language systems and the four language skills. Remember your students' course has probably included not only reading and writing, grammar and lexis, but also speaking, listening, phonology and function. Somehow tests often seem to focus far more on the first four than the last four.
Traditional 'pen-and-paper' tests are usually made up of two types of questions:
§ discrete item tasks (i.e. testing specific individual language points);
§ integrative tasks (i.e. a number of items or skills tested in the same question).
These can be marked in two ways:
§ objectively (i.e. there is a clear correct answer, and every marker would give the same marks to the same question);
§ subjectively (i.e. the marking depends largely on the personal decision of the marker; different markers might give different marks for the same question).
Language systems are easier to test objectively; language skills tend to be tested subjectively.
Let’s consider proficiency tests, progress or achievement tests, diagnostic tests and placement tests.
Proficiency tests
Proficiency tests measure learners’ language ability regardless of the training they may have had or the vocabulary and topics they may have studied. Proficiency tests are not based on the contents of a language course but rather on the general knowledge of the target language and culture.
Achievement tests
Achievement tests are directly related to the language courses taught to the examinees. The purpose of achievement tests is to judge upon the success of individual learners or groups in achieving the objectives of the language course. Achievement tests are always “course related” meaning course contents and objectives.
Diagnostic tests
Diagnostic tests identify students strengths and weaknesses. They provide the teachers with the information on what further teaching is necessary and what problems the students might have in coping with the instruction demands.
Placement tests
Placement tests provide information that helps to place the students at the most suitable stage of the teaching curriculum, bearing in mind their level of the language achieved so far. (Adapted from Hughes, A., 1996. Testing for Language Teachers. p. 9-21).
Performance-based testing
Instead of just offering paper-and-pen-cil single-answer tests of possibly hundreds of discrete items, performance-based testing of typical school subjects involves:
§ open-ended problems
§ hands-on projects
§ student portfolios
§ experiments
§ labs
§ essay writing.
Interactive language tests
The language version of performance-based testing comes in the form of various interactive language tests. Students are assessed in the process of creatively interacting with people. This means that tests have to involve people in actually performing the behavior that we want to measure.
Paper and pencil multiple choice tests certainly do not involve test takers in speaking, requesting, responding, interacting, and in combining listening and speaking, or reading and writing. Interactive testing involves them in all of the above rather than relying on the assumption that a good paper and pencil test taker is a good overall language performer.
Testing Techniques
Gap-fill[4]
Gap filling refers to tasks where the test-takers are given separate sentences with some words or phrases deleted. The task is to restore the missing words. In these tests answer keys can sometimes have more than one answer for a space. Some missing words can have a structural value for the sentence (e.g. prepositions or conjunctions). Other words can have full lexical meaning. Sometime a list of the words can be given to the test-takers to be used in filling in the gaps. In such cases the number of words in the list is usually larger than the number of gaps in the text.
§ Single sentence
Fill in the blanks. Use only one word in each space.
I'd ……. go to the cafe than the pub.
Answer: rather
(If answers of more than one word are allowed, then other answers are
possible; instructions need to be clear!).
§ Using given words
Put one word from the list below in each gap. thought switched unlocked arrived
He (1) ………. home late that night. As he (2) ………..the front door, he (3)…….
he heard a noise in the sitting room. He tiptoed carefully into the room and
(4)….. on the light.
Answers: 1 arrived 2 unlocked 3 thought 4 switched
§ Using other clues (e.g. pictures, anagrams, first letters, lines indicating how
many letters in word, etc.)
He looked through the …………… and was amazed to see that she had finally come ……….. .(Students have pictures of a window and a house.)
Answers: window, home.
§ Transformation of a given word
He could produce no ……………. evidence to support his argument, (photograph)
Answer: photographic
Cloze
A cloze test is a gap-fill exercise using a longer text and with a consistent number of words between gaps (e.g. every ninth word). The word 'cloze' is often incorrectly used to describe any gap-filling task.
Cloze procedure involves deleting a number of words in the whole text, requiring the test-takers to restore the original words. The procedure of based on the assumption of gestalt psychology that human psyche tends to “complete the closures” in the elements of the surrounding world that is being perceived. This makes the cloze procedure authentic to cognitive processes. The first short passage of the text is usually left not mutilated. After this brief “lead-in” approximately every seventh word is deleted. Disadvantages of the test are that some words prove more difficult to restore than others. There could be more than one answer for any one gap. It is not always easy to say what language area (grammar, lexis, prepositions etc) or skills (receptive or productive) are actually measured by the cloze procedure.
Exploratory task:
Do the cloze procedure and comment on the difficulties that you have experienced
The amount of physical education taught in primary schools is decreasing because of the government's emphasis on the "three R’s", according to a survey. It shows that half a million hours of …lessons were lost in the past year because more …was spent on literacy and numeracy. Teachers have warned that more PE time could be… from September, as schools concentrate more on … numeracy.Multiple choice
Choose the word or phrase which best completes each sentence.
If I went to Jakarta, ………. buy some jewellery.
a I'll bl c I will dl'd
Answer: d
(Multiple choice is, of course, a very widely used testing technique and can be
used for more than simple gap-filling tests.)
Multiple-choice tests have a “stem” (the basic and unaltered part of the sentence) and a number of “options” , only one of which is correct. The other options are wrong in the particular context and are called “distracters”.
The advantages of the multiple-choice test are that they can produce a reliable and economical scoring. A test can include a fairly long list of items and increase the reliability of procedure, thus decreasing the randomness of the results.
The disadvantages of the multiple-choice test are that it checks only recognition knowledge. Guessing can have an effect on the scores. The plausible (looking correct) distracters are not always available and this makes test writers include “fool-proof” distracters. The “correct answer” can in quite a few cases be questioned.