Behavior Research Methods, Instruments, & Computers 1999,31 (4), 572-577 Web-based administration ofa personality questionnaire: Comparison with traditional methods ROBERT N. DAVIS University ofMichigan, Ann Arbor, Michigan The World-Wide Web holds great promise as a meehanism for questionnaire-based research. But are data from Web-based questionnaires eomparable to data from standard paper-and-peneil question­ naires? This study assessed the equivalenee ofthe Ruminative Responses Seale in a Web-based format and in a paper-and-peneil format among introductory psyehology, upper-level psyehology, and non­ psyehology students. Internal eonsisteney eoefficients were eomparable aeross the groups. The par­ ticipants in the Web sample reported higherlevels of self-foeused rumination than did the othergroups. Women in the Web sample reported more self-foeused rumination than did women in the other groups. In the Websample, results did not eovary with aeeess loeation. These results suggest thatfindings from Web-based questionnaire research are eomparable with results obtained using standard proeedures. The eomputerized Web interface may also facilitate self-disclosure among research participants. The use ofcomputers in psychological assessment has have reported favorable results (e.g., Krantz, Ballard, & increased steadily over the last 30 years (Burke, 1992). Scher, 1997). For example, the results from one experi­ The advent ofthe World-Wide Web otTers unprecedented ment that was conducted both on the Web and in a labo­ opportunities for researchers to benefit from the use of ratory were nearly perfectly correlated (Krantz et al., computers in data collection (Schmidt, 1997; Smith & 1997). Although only a few published studies have ex­ Leigh, 1997). As a research medium, the Web offers re­ amined the equivalence question with respect to Web­ searchers at least four major benefits. First, the Web per­ based versus paper-and-pencil administration ofperson­ mits individuals to send data to a researcher at their con­ ality questionnaires, nascent studies in this area have venience, in terms oftime and location, which increases reported comparable results across domains (Buchanan the potential number of eligible research participants & Smith, in press; Pasveer & Ellard, 1998; Smith & Leigh, (Schmidt, 1997; Smith & Leigh, 1997). Second, the Web 1997; Stanton, 1998). For example, Buchanan and Smith permits automatie transformation ofraw data into an an­ administered the revised version ofthe Self-Monitoring alyzable format, such as an SPSS data file, using proce­ Scale (SMS-R; Gangestad & Snyder, 1985) to Web users dures such as CGI scripts (see Kieley, 1996; Schmidt, on line and also to a local student sampie in paper-and­ Hoffman, & MacDonald, 1997). Third, Web-based re­ pencil format. The factor structure ofthe SMS-R and co­ search is etTicient in terms oftime and the resources it re­ etTicient alphas were found to be comparable across the quires (Schmidt, 1997). Fourth, the Web provides a degree groups, with the Web version yielding some superior re­ of anonymity to research participants, which decreases sults (e.g., coefficient alpha), as compared with previ­ the influence of demand characteristics by facilitating ously reported studies (Buchanan & Smith, 1999). An­ self-disclosure and eliminating observer bias (Esposito, other study that compared Web-based and paper-and­ Agard, & Rosnow, 1984; Hewson, Laurent, & Vogel, pencil vers ions ofan organizationaljustice scale found a 1996; Meszaros, Engelsmann, Meterissian, & Kusalic, similar factor structure across the two domains, as weil 1995; Smith & Leigh, 1997). as more item variability and less missing data in the Web­ Is research conducted over the Web equivalent to re­ based condition (Stanton, 1998). search conducted in more traditional settings, such as a The advent ofWeb-based data collection is a direct ex­ psychology laboratory? Validity studies ofWeb-based re­ tension ofdata collection on stand-alone computers (Bu­ search have recently begun to examine this question and chanan & Smith, in press). Given the current dearth of findings pertaining to Web-based data collection, studies of computerized (non-Web-based) versus paper-and­ This research was supported by U.S. Public Health Grant 51817 to pencil data collection procedures may provide useful in­ Susan Nolen-Hoeksema. I thank Michael Bishop, Jamie Polito, and An­ formation. Research comparing computerized and paper­ drew Ward for their assistance with data collection. I also thank Tom and-pencil assessments has generally found that research Buchanan for his helpful comments on an earlier draft of this manu­ script. Correspondence concerning this article should be addressed to participants prefer computerized assessments and are R. N, Davis, who is now at the University ofHouston, Department ofPsy­ equally, ifnot more, willing to report sensitive information chology, Houston, TX 77204-5341 (e-mail: [email protected]). to a computer than to a paper-and-pencil questionnaire Copyright 1999 Psychonomic Society, Inc. 572 WEB-BASED ADMINISTRATION OF A PERSONALITY QUESTIONNAIRE 573 or to a human interviewer (Booth-Kewley, Edwards, & Participants scoring at the upper and/or the lower end of Rosenfeld, 1992; Evan & Miller, 1969; Finegan & Allen, an overall distribution on variables of interest are fre­ 1994; Hile & Adkins, 1997; King & Miles, 1995; Mill­ quently selected for follow-up studies on the basis ofre­ stein, 1987; Petrie & Abell, 1994; Wilson, Genco, & Yager, sults from an initial screening session-for example, a 1986). In one study, recently admitted parasuicide pa­ tendency to engage in self-focused rumination when ex­ tients preferred a computerized interview to an interview periencing dysphoric affect (cf. Davis & Nolen-Hoeksema, with a clinician (Petrie & Abell, 1994). In another study, in press). These procedures facilitate research on inter­ recent suicide attempters disclosed more information to actions between subject variables, such as personality computerized depression and suicidal ideation question­ traits, and experimental treatments (cf. Lyubomirsky & naires than to a clinician interviewer (Levine, Ancill, & Nolen-Hoeksema, 1993, 1995), as weil as cross-sectional Roberts, 1989). Moreover, the computerized measures studies of variables on which an experimenter wants to were better predictors of suicidality than was the clini­ select participants (e.g., age or gender). cian interviewer (Levine et al., 1989). People also appear The measure used in the present study assesses the ex­ to be more willing to report behaviors such as substance tent to which people tend to respond to feelings of sad­ use to a computer than to a clinician or a paper-and-pencil ness or depression with self-focused rumination. Self­ questionnaire (Lucas, 1977; Lucas, Mullin, Luna, & Mc­ focused rumination describes a tendency to engage in Inroy, 1977; Skinner & Allen, 1983). Although these passive thoughts and behaviors that focus one's attention findings are informative, the extent to which they gener­ on one's depressed mood and on the implications ofthese alize to Web-based research is an open question. symptoms,rather than taking action to alleviate one's Despite the initial promising evidence in support of symptoms (see Nolen-Hoeksema, 1990, 1991,for reviews). Web-based psychological research (e.g., Buchanan & People who ruminate may tend to sit alone and think Srnith, in press; Krantz et al., 1997; Michalak, 1998; about how tired and unmotivated they feel or passively Pasveer & Ellard, 1998; Smith & Leigh, 1997; Stanton, review all the negative happenings in their lives, without 1998), it is necessary to demonstrate the equivalence of taking action to change their situation. The RSQ (Nolen­ Web-based and traditional data collection methods em­ Hoeksema & Morrow, 1991) was developed to assess peo­ pirically be fore adopting Web-based methods exten­ ple's habitual responses to depressed mood, and the RRS sively (Krantz et al., 1997; Smith & Leigh, 1997). There consists of aseries of items on the RSQ that assess the remain numerous potential threats to the reliability and tendency to engage in self-focused rumination. A short validity ofWeb-based questionnaires (see Hewson et al., (IO-item) form of the RRS was recently developed and 1996; Schrnidt, 1997, for reviews). For example, partic­ was used in the present study (cf. Davis & Nolen-Hoek­ ipants recruited from the Web are likely to be more het­ sema, in press). erogeneous on demographie and other variables than par­ In the present study, the RRS was administered to four ticipants recruited by traditional means (Buchanan & sampies ofparticipants: (I) introductory psychology stu­ Smith, in press). Moreover, researchers lose consider­ dents; (2) upper-Ievel psychology students; (3) nonpsy­ able control over the testing environment when conduct­ chology students; and (4) a Web-based studentsampie. ing Web-based research.Some participants may send The nonpsychology and upper-Ievel psychology student data from a quiet roorn, whereas others might send it groups were included in order to determine whether they from a noisy computer laboratory (Reips, 1996). In ad­ differed from introductory psychology students even dition, Web-based research is open to exploitation by peo­ when using the same methods (i.e., paper and pencil) for pie who participate numerous times and/or provide dis­ filling out the questionnaire. The Web sampie was de­ ingenuous data (Schmidt, 1997; Smith & Leigh, 1997).