Use of Language in Ballot Instructions
Total Page:16
File Type:pdf, Size:1020Kb
NISTIR 7556 Report of Findings: Use of Language in Ballot Instructions Janice (Ginny) Redish, Ph.D. Redish & Associates Dana E. Chisnell UsabilityWorks Ethan Newby Newby Research Sharon J. Laskowski Information Access Division Information Technology Laboratory National Institute of Standards and Technology Svetlana Z. Lowry Information Access Division Information Technology Laboratory National Institute of Standards and Technology May 2009 NISTIR 7556 Report of Findings: Use of Language in Ballot Instructions Janice (Ginny) Redish, Ph.D. Redish & Associates Dana E. Chisnell UsabilityWorks Ethan Newby Newby Research Sharon J. Laskowski Information Access Division Information Technology Laboratory National Institute of Standards and Technology Svetlana Z. Lowry Information Access Division Information Technology Laboratory National Institute of Standards and Technology May 2009 U.S. Department of Commerce Gary Locke, Secretary National Institute of Standards and Technology Patrick D. Gallagher, Deputy Director NISTIR 7556 Report of Findings: Use of Language in Ballot Instructions Janice (Ginny) Redish, Ph.D. Redish & Associates Dana Chisnell UsabilityWorks Ethan Newby Newby Research Sharon J. Laskowski Information Access Division Information Technology Laboratory National Institute of Standards and Technology Svetlana Z. Lowry Information Access Division Information Technology Laboratory National Institute of Standards and Technology May 2009 This document has been prepared by the National Institute of Standards and Technology (NIST) and describes research in support of test methods and materials for the Election Assistance Commission's next iteration of the Voluntary Voting System Guidelines (VVSG). It does not represent a consensus view or recommendation from NIST, nor does it represent any policy positions of NIST. Certain commercial entities, equipment, or material may be identified in the document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that these entities, materials, or equipment are necessarily the best available for the purpose. Report of Findings: Use of Language in Ballot Instructions Table of Contents Highlights and Summary ..................................................................................... 1 What we learned ..................................................................................... 2 Where (and why) participants had problems .......................................... 3 Recommendations .................................................................................. 6 Suggestions for future research .............................................................. 7 What we did ............................................................................................ 11 Part 1: Description of the Study ........................................................................ 13 What did we study? ................................................................................ 13 What is plain language? ......................................................................... 13 What were the goals of the study? ......................................................... 14 Where did the traditional and plain language instructions come from?............................................................................................. 14 How did we conduct the study? .............................................................. 15 How many people participated in the study? .......................................... 16 When and where did we conduct the test sessions? .............................. 16 How did we recruit our participants? ....................................................... 17 We recruited with only two screening criteria ...................... 17 We strove for diversity ........................................................ 18 We focused on people with a high school education ........... 18 We hoped for a range of voting experience ........................ 19 A professional recruiter helped us ....................................... 20 What were the ballots like for this study? ............................................... 21 What did we do in each session? ........................................................... 22 How were participants compensated? .................................................... 24 What was the setup like in each location? .............................................. 24 How were the ballots presented? ........................................................... 25 What materials did we use? ................................................................... 26 How did we control for bias in the study? ............................................... 26 What tasks and directions did we give the participants as voters? ......... 27 What data did we collect while participants voted? ................................. 29 What data did we collect in the forced choice comparison interview? .... 29 May 2009 page i of 198 Report of Findings: Use of Language in Ballot Instructions Part 2: Results ..................................................................................................... 30 Participants voted more accurately on the ballot with plain language instructions .............................................................. 30 What did we count as accuracy? ........................................ 30 How did accuracy compare between the two ballots? ......... 31 Participants who voted B first did better on A than participants who voted A first ...................................... 32 Only education level made a difference in how accurately different groups of participants voted .................................. 33 Education made a slightly greater difference for Ballot A than for Ballot B .................................................................. 34 Participants recognized the difference in language ................................ 34 Participants overwhelmingly preferred the plain language instructions ............................................................................................. 36 In the page-by-page comparison, participants preferred the page from Ballot B most of the time .............................. 36 A large majority (82%) of participants chose Ballot B for their overall preference .................................................. 36 Although Ballot B was better than Ballot A, it was not perfect ................ 38 Part 3 Discussion: Where did participants have problems? ......................... 40 Other studies looked at error rates after ballots were cast ..................... 40 We observed voting, giving us a better understanding of why voters make mistakes.................................................................. 41 In this part, we use mistakes that participants recovered from as well as “official” errors ........................................................................ 41 How we organized Part 3 ....................................................................... 41 How participants performed .................................................................... 42 Participants across the spectrum voted perfect ballots ....... 42 Most participants made relatively few errors ....................... 44 A few participants are responsible for a sizable portion of the total error count ......................................................... 44 How pages performed: Six pages had very high error rates .................. 44 Why did these six pages have such high error rates? ............................ 45 Voters must know a lot about how elections work to follow a ballot .................................................................. 46 Experience with other technology does not necessarily give voters a good mental model of using an electronic voting system ...................................................................... 46 May 2009 page ii Report of Findings: Use of Language in Ballot Instructions Many voters do not understand different levels of offices .... 47 Electronic voting systems compound the problem because voters do not know what is yet to come on the ballot .......... 47 Did plain language make a difference? .................................................. 48 Straight Party Vote/Straight Party Voting ................................................ 49 US Senate, State Senator ...................................................................... 52 Registrar of Deeds .................................................................................. 56 County Commissioners .......................................................................... 60 City Council ............................................................................................ 62 Ballot Summary/Review Your Choices ................................................... 65 Part 4 Discussion: Which ballot did participants prefer in a page-by-page comparison? ............................................................ 72 Thank you (last page in each ballot) ....................................................... 73 Instructions to Voters/How to Vote ......................................................... 74 Ballot Summary/Review Your Choices ................................................... 76 Confirmation ..........................................................................................