11/2008 p. 1 of 7

Scanner Scoring Instruction Manual

DO NOT REMOVE FROM THIS ROOM

If you are having a problem with the scanner (hardware or software) or reports, please contact Dale, 5-6828, David, 5-6773, or Jim 5-4219.

This is a step by step process, follow instructions carefully.

You are encouraged to scan a "practice" run before your first actual exam. There are two envelopes in the scanning area. One is marked "Single Version" - it contains an answer sheet and about 30-40 completed forms. A second envelope is marked "Multiple Version." It has three answer forms (A, B, and C) and about 70 completed student forms that are marked Form "A," "B," or "C." Scanning multiple forms is much more complicated than scanning a single test form. Get comfortable scanning a one-form test before trying a multiple-version test.

When scanning an actual exam, be sure you have o Your class roster, o your exam and answer key, o the students answer forms, and a o data storage device (floppy, CD-R, or flash drive). o It is also now possible to e-mail yourself your saved files.

Remember to shut down everything when you finish. See Sec VII 11/2008 p. 2 of 7

Scanner Scoring Instruction Manual November 2008

ESSENTIAL – EVERYONE READ

The Department uses the Remark Classic (RC) software. 1. This manual is a step by step process how to use the scantron machine to score tests. No training or tutoring sessions are being planned. If you have difficulty, please contact another TA or ask Dale 5-6828, David 5-6773 or Jim 5-4219. 2. You must use the large, red scantron forms. The ID # is entirely optional. Be sure students bubble in their name and, if more than one test form (version) is used, the correct test form (A or B). If you have students supply ID #, note that the scanner is set only to read the first six boxes (in the 10 box ID grid). 3. CRITICALLY IMPORTANT! Paper jams are a “potentially destructive” problem. It is very important that you remove a jam properly. You must pull the form FIRMLY to the right (back toward the bed), away from you (toward the wall), or both. DO NOT pull a jammed sheet forward, because this can damage the rollers. Note: Glossary items are Italicized and marked with an “*” asterisk throughout the manual.

Table of Contents

I. STAGE 1: POWERING ON, TEMPLATE, SCANNING FORMS, EDITING ...... p. 3

II. STAGE 2A: SCORING THE DATA GRID...... p. 3

III. STAGE 2B: SCORING WITH MULTIPLE EXAM VERSIONS (FORMS) ...... p. 5

IV. STAGE 3: TEST STATISTICS, ITEM ANALYSIS, REPORTS, ETC...... p. 5

V. STAGE 4: GRADE REPORTS (INDIVIDUAL RESPONSE REPORTS) ...... p. 6

VI. STAGE 5: EXPORTING GRADES (SCORES) TO EXCEL ...... p. 6

VII. SHUTTING DOWN ...... p. 6

VIII. GLOSSARY...... p. 7 11/2008 p. 3 of 7

I. STAGE ONE: POWERING ON, SELECTING TEMPLATE, SCANNING FORMS, EDITING

A. Power everything on (computer, monitor, scanner, and printer) via the toggle switch (located between the computer and the scanner). You will also have to turn the computer on and wait for the sign-on screen. The password is rm102a B. From the Desktop screen Click the Remark Classic (OMR) icon. C. Open the Template*: File-Open-Template. Then select: red_200b_ses D. Place all of your forms in the scanner bed, with your Answer Key on top. The timing marks* will be on the edge closest to you, and the skunk marks* (near the name grid) will be the “leading edge.” E. The answer key: Simply code in A_KEY in the name grid and code in all of your answers (ordinarily only one answer per question). If you have more than one test version (form), prepare a separate key of each version (A_KEY, B_KEY, etc.), indicating the form (A, B, C, or D) in “Test Form.” [This convention for the key is not mandatory but it allows you to readily identify your key. Also, if you sort your data file, it will maintain your key at the top of the datafile.) See below for an alternate method for entering your answer key*. F. Scan (“read”) the answer sheet(s). a) CNTL-R, OK [Alternatively: Tools-Read, OK or Click small scanner icon on the menu bar (4th icon from right)]. b) Whenever you are prompted “End Session?” or “Continue?” choose Continue, until you have scanned all of your sheets. c) After the key is scanned, make sure that every intended answer appears in the data grid.* For example, if your exam has 47 items, make sure each answer appears in the grid. If any are missing, simply edit them in by hand. G. Scan (“read”) the student exam forms (same as F above). H. Editing: Any omits* or multiple marks* will be highlighted in the data grid. You will want to inspect these marks, to check for light marks or incomplete erasures, respectively. Locate the student’s sheet from the pile of scanned forms. To make this easier, you might want to scan a few sheets (20-30), check for errors, and look up sheets. Then scan another 20-30 forms, etc.

II. STAGE TWO-A: SCORING THE DATA GRID. Instructions A – F apply to a single-form exam. If you have more than one form, find further instructions below (Stage Two-B).

A. The first line in the data grid* is automatically considered to be the Answer Key. B. Score the grid by: Analysis-Grade-Easy; CNTL-G; or clicking the 3rd icon from the right. C. Scoring is immediately, but you will not see any immediate results, and you get various reports on a new screen. D. Save the data grid into a data file*. This will permit you to return later and rescore an exam (perhaps after discussing the results with an instructor). Be sure to choose the file format 11/2008 p. 4 of 7

(Excel?) and location. It is encouraged to save your data file either on your own floppy disk (a :). File – Save Data As – File Name (and directory). E. To rescore the current data grid, just return to C. above. You can always modify the Answer Key (line 1 in the grid) and then rescore. For example, you can change an answer; omit an item (by removing the answer from the item on the key line), etc. Procedure for Scoring a Test with More than One Correct Answer (e.g., A or E are correct) A. You cannot use Easy Grade; instead you must use Grade Wizard. B. Scan your key and all of the student forms in the usual way. a) Bubble in the two (or more) correct answers on the answer key before scanning it. b) If you have already scanned the key, just modify the key within the data grid. Do this: c) Go to the item with multiple answers. E.g., click on item 15. d) After you click on that item, click the arrow on the right side of the box. Click each correct alternative, until the box looks like this “(A, C)” (assuming A & C are both correct). C. When you are ready to score the test: Click Analysis – Grade – Wizard. You may append the new report to the existing one, or write over it. Welcome Screen - click Next. a) Step 1 – click Next. b) Step 2 – Get Key and the Next c) Step 3 – Click any fields (e.g., Name, Test Form, ID #) that you want to appear on the student Grade Report. Then click Next d) Steps 4, 5 & 6 - click Next. e) Step 7 – This is where you will indicate whether the multiple item is ALL (both A & C must be marked) or ONE (either A or C may be marked). Click on the highlighted box for the options. When done, click Finish. f) You will be prompted: Save the Grade Report Template? Ordinarily, you can answer No. If you anticipate repeating this process (with the same data set and key), then you could save the Grade Report Template. That will save you having to repeat steps 1-7 moments later. g) This will complete the scoring process, and Quick Stats will then appear as usual.

Procedure for Differential Weighting of Items (e.g., items 4 and 7 count double) 1. Follow all of the above procedures, except:

2. At 3h: On Step 7, locate each item that you wanted weighted. For example, if items 4 and 7 count\2 pts instead of one, just change “Correct Points” from 1 to 2 for each of these items. 3. Then follow Steps 3i and 3 j. above 11/2008 p. 5 of 7

III. STAGE TWO-B: SCORING WITH MULTIPLE EXAM VERSIONS (FORMS) Follow steps A through E above (section IV). Be sure to save the full data grid into a data file. Suggested filename is: yourname_300_x1tot. The “tot” is important.

A. After saving the grid into a data file (which you will call up later), sort the data grid: B. Edit – Sort – Sort by test form. C. Delete all lines from your data grid except those (key plus student forms) with Form A. D. Score Form A in the usual way. E. Save this data grid into a data file (suggested: dodd_300_x1A). F. Inspect the Item Analysis. If necessary, modify the exam key and rescore (and resave). G. Recall the entire data file (File-Data-Filename) and sort as above. H. Delete all lines of the data grid except for those (key and students) for Form B. I. Score Form B (see above); inspect the Item Analysis, etc. J. Repeat for Form C and Form D, if necessary.

IV. STAGE THREE: TEST STATISTICS, ITEM ANALYSIS, REPORTS, ETC.

A. Test results will appear in a new window entitled: “Quick Stats. To print any of these results, use an approach similar to that of SPSS (File – Print – etc.). B Test statistics: gives an overview of your exam, including number of items, number of respondents, score average and range, etc. C. Respondent statistics: Shows each student’s exam results; respondents are listed by name but not in alphabetical order. (If you want this report alphabetized, you will have to sort the data grid by name and then rescore: Edit – Sort – Sort by name.) This display is different from the Grade Report* that you will generate. The Grade Report will be in Excel format (usually) and you will probably want to print your roster from Excel. D. Frequency Distribution: self-explanatory. E. Item Statistics: This is not the item analysis of responses. What is most useful about this display is the “P value” (percent correctly answering the item) and the Point Biserial value, which reflects the discrimination index of each item (roughly, item-total r). Also, this screen indicates how many are “missing”* (omits plus multiples) per item. F. Item Analysis: Provides for each item the response breakdown. “Missing” will tally both omits and multiples. G. Evaluating your test items: Use Item Statistics (E above) and Item Analysis (F) to decide whether to omit items or change the answer key. H. Modifying the answer key and rescoring. 1. Go back to the Data Grid (click at bottom of screen). 2. Go to your Key line (first line of file). Change or delete the answer. 3. Rescore. (See Stage II.) 11/2008 p. 6 of 7

V. STAGE FOUR: GRADE REPORTS

A. You will probably want the Grade Reports printed in alphabetical order (especially for large classes). To do this, you will first have to sort the data grid by name. Return to the data grid, and then: (Edit – Sort – Sort by test form.) B. If you want to give students item feedback (similar to the Individual Response Reports in ESS), you process Grade Reports* (from Quick Stats: Statistics – Grade Reports). Make sure you are in Quick Stats (not the data grid). C. If you are prompted “Append to Existing Report?” generally you can answer “yes.” This just stacks new reports (or analyses) to the end of previous ones. D. The printing default should be set on one student per page. If you are getting several students per page (and don’t want to clip), then do this: View-Preferences-Grade Reports (then click the box for “Page Break After”).

VI. STAGE FIVE: EXPORTING GRADES (SCORES) TO EXCEL

A. After you are completely finished scanning answer sheets, scoring, evaluating your test items, and printing reports, you will want to export your students’ grades (scores) and then enter them into your own gradebook (usually Excel). B. From the Quick Stats screen: 1. File-Export Grades – Numeric Data and Results 2. Save your file on your own diskette: “Look In a: “ 3. Enter your file name, select the file type (usually Excel 97/2000). 4. You must also enter a sheet name. Keep it short; it will produce a name for your sheet in Excel. This is not too important, unless you have more than one test version. Then you will want to keep the same filename for all forms, but change the sheet name (“Form A,” “Form B,” etc.) 5. You can open this saved file in Excel. Note that the number correct and % correct will be at the far right for each student (past Q200). You can just delete all of the columns for items (Q1 through Q200) for your convenience.).

VII. SHUTTING DOWN

It is important to properly shut down the system, in order to make Windows easily available for the next user and to reduce wear on the read heads of the scanner. When you are finished scanning, printing, exporting files, etc., do this: 1. Delete any files created in C: /that you no longer need 2. Close all programs and 3. shut down the computer (Start – Turn Off Computer). 4. Turn off the power strip (located behind the scanner). 5. Turn off lights. 6. Be sure door is locked and return key to Jim 11/2008 p. 7 of 7

VIII. GLOSSARY

Answer key – the sheet that identifies the correct answers for your exam. Ordinarily, you code your answers onto a form and scan that sheet first. Alternatively, you can go to the data grid and code in A_KEY (under “Name”) and the answers directly onto the first row of the grid. You may or may not find this to be more convenient.

Data File - Saving the data grid to a data file gives you “permanence” so you can go back later and make corrections or rescore. It eliminates the need to ever rescan the sheets. If you save the data without sorting the grid, the data file will list students in the order scanned. If you first sort the data file by name, you can save the student data alphabetically. This might be neater but will make it much more difficult to locate a student’s actual sheet from a pile of unsorted sheets.

Data Grid - The key and student data are shown on a grid similar to Excel. You will score your rest from this grid. You can easily modify the data (key or student responses) and rescore the test.

Data grid – The spreadsheet within RC that shows your answer key and each student sheet scanned. Rows represent students and columns represent Name, items, etc.

Grade Report – This output will provide you with an individual feedback form for each student. It shows the students name (and form, if applicable), test result, and items missed (with correct answers provided). The grade report will not be in alphabetical order, unless you first sort the data grid.

Missing – RC considers any value (an omit or multiple) as missing. On reports, “missing” equals the number of students who omitted and item plus the number who marked multiple responses.

Multiple mark – The scanner “sees” two responses where there should be one. This can be due to a “true multiple” (two responses marked) or a “false multiple” (an incompletely erased response.

Omit – The scanner “sees” no response where there should be one. This can be due to a “true omit” (no response) or a “false omit” (a very light answer).

Skunk marks – marks at the top of the sheet that identifies (for the scanner) the layout of the sheet.

Template - the representation of the form (sheet) for the software. The template defines where the items are, what responses are acceptable, etc. You must select the correct template before you begin scanning.

Timing marks – marks on the long-side of the sheet that identify (for the scanner) what part of the sheet is passing over the reader head.