234 Tips and Tricks for Recruiting Users as Participants in Usability Studies By Deborah Hinderer Sova and Jakob Nielsen 48105 WARM SPRINGS BLVD. FREMONT, CA 94539–7498 USA WWW.NNGROUP.COM Copyright © Nielsen Norman Group. All Rights Reserved. Get your own copy from: www.nngroup.com/reports/how-to-recruit-participants-usability-studies About This Free Report This report is a gift for our loyal audience of UX enthusiasts. Thank you for your support over the years. We hope this information will aid your efforts to improve user experiences for everyone. The research for this report was done in 2003, but the majority of the advice may still be applicable today, because people and principles of good design change much more slowly than computer technology does. We sometimes make older report editions available to our audience at no cost, because they still provide interesting insights. Even though these reports discuss older designs, it’s still worth remembering the lessons from mistakes made in the past. If you don’t remember history, you’ll be doomed to repeat it. We regularly publish new research reports that span a variety of web and UX related topics. These reports include thousands of actionable, illustrated user experience guidelines for creating and improving your web, mobile, and intranet sites. We sell our new reports to fund independent, unbiased usability research; we do not have investors, government funding or research grants that pay for this work. Visit our reports page at https://www.nngroup.com/reports/ to see a complete list of these reports. HOW TO SHARE Do not link directly to the PDF file (the hosted address could change). Instead, we encourage you to distribute the following link to the report’s page on our website to allow people to decide whether to download it themselves: www.nngroup.com/reports/how-to-recruit-participants-usability-studies © NIELSEN NORMAN GROUP WWW.NNGROUP.COM 2 Copyright Notice Please do not post this document to the internet or to publicly available file-sharing services. This report is free, but it is still copyrighted information that may be updated from time to time, so please don’t distribute this file or host it elsewhere. Even when people post documents with a private URL to share only with a few colleagues or clients, search engines often index the copy anyway. Indexing means that thousands of people will find the secret copy through searches. 48105 WARM SPRINGS BLVD. FREMONT, CA 94539-7498 USA [email protected] 3 Contents Executive Summary ........................................................................... 6 Introduction ...................................................................................... 9 Tips and Tricks Summary ................................................................ 11 Planning Your Recruiting Needs ...................................................... 20 Developing Recruiting Criteria ........................................................... 21 Learning About the Users ................................................................................... 21 Deciding How Many Participants to Recruit ........................................................ 27 Drafting the Screening Criteria ........................................................................... 29 Determining the Appropriate Incentives .............................................. 31 Monetary Incentives .......................................................................................... 33 Non-monetary Incentives ................................................................................... 38 How Incentives May Affect Participant Behavior ................................................ 43 How and When to Provide Incentives ................................................................. 44 Considering the Study Locale ............................................................ 46 When Participants Come to You ......................................................................... 47 When You Must Go to the Participants ............................................................... 50 Planning for Training and Orientation .................................................. 53 Preparing the Screening Script and Questionnaire ................................ 55 Cover Page and Opening Script .......................................................................... 56 Screening Questions .......................................................................................... 58 Invitation and Schedule ..................................................................................... 64 Sample Participant Screening Script and Questionnaire ..................................... 65 Screening and Scheduling Participants ............................................ 72 Working With an Outside Recruiting Agency ........................................ 73 When to Outsource Recruiting ........................................................................... 73 Finding a Recruiting Agency ............................................................................... 74 What to Expect from a Recruiting Agency .......................................................... 76 Choosing a Recruiting Agency ............................................................................ 79 Managing a Recruiting Agency to Your Best Benefit ........................................... 82 Doing Your Own Recruiting ................................................................ 86 Internal vs. External Studies .............................................................................. 86 Finding Participants for Internal Studies ............................................................ 90 Finding Participants for External Studies ........................................................... 91 Making the Calls and Tracking Recruitment Progress ......................................... 96 Reusing Participants ....................................................................... 104 When It Is Okay to Reuse Participants ............................................................. 104 When to Avoid Reusing Participants ................................................................. 104 © NIELSEN NORMAN GROUP WWW.NNGROUP.COM 4 Preparing Participant Session Forms ............................................. 106 Honoring the “Participant Bill of Rights” ....................................... 117 Treating Participants as Human Beings ............................................................ 117 Attending to Participants’ Physical Comfort ..................................................... 121 Ensuring Participants’ Safety ........................................................................... 123 Ensuring Participants’ Privacy .......................................................................... 124 Dealing with Unqualified Participants .............................................................. 126 Future Planning ............................................................................. 129 Building and Maintaining a Participant Database ................................. 129 Building and Managing a Recruiting Staff .......................................... 130 Appendix A: Participant Recruiting Survey .................................... 134 About the Survey ........................................................................... 134 Survey Results .............................................................................. 136 Geographical Areas from Which Participants are Recruited.............................. 136 Who Recruits Participants ................................................................................ 143 Internal Time Spent Recruiting Participants .................................................... 148 Agency Lead Time Required and Fees Charged ................................................ 148 Participant User Groups, Incentives, and Session Length ................................ 150 No-Show Rates ................................................................................................. 158 Pre-Session Interaction ................................................................................... 159 Participant Session Documents ........................................................................ 161 Participant Follow-Up ....................................................................................... 162 Appendix B: Sample Scripts and Forms ......................................... 164 Background Questionnaire ............................................................................... 164 Confirmation Message ...................................................................................... 166 Dealing with Unqualified Participants .............................................................. 167 Eliciting Sensitive Information – Sample Questions ......................................... 167 Eliciting Web Experience - Sample Questions ................................................... 168 Interview Topics for Meeting with the System Team ........................................ 169 Interview Questions for the System Team ....................................................... 170 Minor Consent Form ......................................................................................... 172 Nondisclosure Agreement ................................................................................ 174 Participation Consent Form .............................................................................