Evaluating the Success of Telecommuting at the Census Bureau1
Total Page:16
File Type:pdf, Size:1020Kb
Joint Statistical Meetings - Section on Survey Research Methods EVALUATING THE SUCCESS OF TELECOMMUTING AT THE CENSUS BUREAU1 Jennifer A. Guarino and Julie A. Bouffard U.S. Census Bureau, 4700 Silver Hill Road, Rm 3087/3, Washington, DC 20233 Keywords: web surveys, logistic regression In the spring of 2000, the Census Bureau’s Labor Management Partnership Council chartered a workgroup Abstract to develop recommendations for a pilot telecommuting program. The workgroup recommended testing a six- Many administrative programs have been launched at month telecommuting pilot, involving all interested the Census Bureau with little prior testing and employees who receive supervisory approval within evaluation. In an effort to create programs that are three areas at Census: Services Sector Statistics mutually desirable for both employees and the agency, Division (SSSD), Demographic Surveys Division the Census Bureau has started to thoroughly evaluate (DSD), and the Commerce Administrative Management such programs before full implementation. The first Systems (CAMS) office of the Financial and program to undergo a large-scale, comprehensive Administrative Systems Division (FASD). The evaluation was a telecommuting pilot that involved telecommuting pilot permitted participants to work from employees working in three areas at Census. The home or a telecommuting center a maximum of two purpose of the evaluation was to measure the effect of days in a two-week period. telecommuting on key factors such as performance, productivity, and job satisfaction. In an attempt to The Partnership Council recommended a formal attribute any differences in these factors to evaluation of the pilot to identify any critical issues telecommuting, the evaluation required the collection of related to telecommuting. In addition to identifying repeated measurements before and after the pilot. In this problems, the evaluation sought to provide feedback on paper, we compare results from the pre- and the design of the pilot, the amount of interest by post-telecommuting periods to determine what changes employees, and the effect of telecommuting on factors occurred as a result of the telecommuting pilot. such as morale, productivity, performance, and job Furthermore, we use logistic regression to measure satisfaction. associations between participation in the telecommuting pilot and work performance. The telecommuting pilot evaluation marks the first large scale assessment of an administrative program initiative 1. INTRODUCTION at the Census Bureau. The Census Bureau has historically introduced many administrative programs There are many reasons why the Federal government is with little to no prior evaluation of the effects on work eager to adopt telecommuting. Research shows that or morale. Therefore, we designed this evaluation to be telecommuting can improve employees’ productivity, a comprehensive, ground-breaking evaluation that would motivation, and concentration in addition to reducing detail the effects of telecommuting on the Census stress and tension among coworkers (OPM, 2001; Joice, Bureau’s work and employees. It is our intention for 1993; Allenby, 2001; Bellinger et. al., 1992). this evaluation to serve as the premier model or standard by which all forthcoming initiatives at the Census Despite the success of early telecommuting, the number Bureau are evaluated. of telecommuters has not dramatically increased (Armour, 2001; Ballard, 2001). Companies in both the 2. METHODS private and public sectors exhibit managerial reluctance to make changes that comply with telecommuting The telecommuting pilot ran from June 4, 2001 to (Joice, 1993; Bartlett, 2001). In addition, concerns November 30, 2001. The evaluation team gathered data about security risks, restraints on productivity, meeting in three stages: pre-pilot (February to May, 2001), pilot conflicts, and jobs that are unsuitable for telecommuting (June 4 to November 30, 2001), and post-pilot have seemingly stunted the expansion of telecommuting (November 30 to December 13, 2001). (Ballard, 2001; Armour, 2001). 1 This paper reports the results of research and analysis undertaken by Census Bureau staff. It has undergone a Census Bureau review more limited in scope than that given to official Census Bureau publications. This report is released to inform interested parties of ongoing research and to encourage discussion of work in progress. 1290 Joint Statistical Meetings - Section on Survey Research Methods 2.1 Design of Telecommuting Pilot Internet link. The link contained an embedded identification number so that employees would not have 1. Universe - The divisions that participated in the pilot to log into the survey. After three to five days, all were selected based upon the level of support exhibited employees received a generic reminder asking them to by division management and the major occupational complete the survey. We then sent a personalized series within each division. The intention was to have a follow-up e-mail that contained the survey link to balanced cross-section of employees who occupy the nonrespondents asking them to respond promptly. The major job series within the Census Bureau participate in surveys stayed in the field for roughly two weeks. the pilot. There were roughly 500 employees and supervisors in the pilot areas. Qualitative Data: We used focus groups to: 1) capture qualitative data that could not be measured by a survey 2. Selection procedure - Employees who wished to or were unanticipated by evaluators, and 2) collect telecommute completed an application. Applicants’ information to assist in the content development of the supervisors were authorized to select telecommuters. questionnaires. We recruited employees for the focus groups in a random fashion; however, the groups did not 3. Mandatory Training - There were three mandatory contain a random sample since participation was phases of training for telecommuters: overview training, ultimately voluntary. employee/supervisor training, and laptop training. The overview session encouraged employees to take part in In addition to focus groups, the evaluation team issued the program. Employee/supervisor training provided suggestion boxes for each participating area in order to tips for a successful telecommuting experience. The capture information that may be too sensitive or laptop training instructed telecommuters on how to undesirable to express during a focus group. access their e-mail remotely. There were nine components of the evaluation 4. Frequency of telecommuting - Telecommuters were conducted throughout the stages of the telecommuting permitted to telecommute no more than two days in a pilot, in the following chronology: pay period (two weeks). Pre-Pilot Phase: 5. Technical limitations - Due to the sensitivity of data 1. Focus Groups with Employees and Managers in the on internal Census networks, telecommuters did not Participating Areas, 2/01. have access to any servers or data that reside within the Census firewall from a remote site. Telecommuters 2. Pre-Pilot Web Survey of All Employees in the were not permitted to work with data protected under Participating Areas, 3/01. We collected quantitative Title 13 of the US Code from their remote location. measures from employees on their work performance, productivity, job satisfaction, commuting factors, and 2.2 Methods for the Evaluation opinions about telecommuting before the pilot began. We obtained an 88% response rate. The evaluation team used both quantitative and qualitative methods to collect data. We used web 3. Focus Groups with Supervisors of Selected surveys to collect quantitative data. Qualitative data Telecommuters, 5/01. were gathered through focus groups and suggestion boxes. 4. Pre-Pilot Web Survey of Supervisors of Selected Telecommuters, 5/01. We administered a short web Quantitative Data: We used WebSurveyor software to survey to all supervisors who managed telecommuters. design the web surveys. The surveys were single-screen The survey requested information regarding scrollable forms in html format. We chose this format telecommuters’ current performance and productivity. for two reasons: 1) there were no complex skip patterns Supervisors with more than one telecommuter on their in the questionnaires, and 2) to minimize the time to staff completed one survey for each telecommuter. We complete the survey. All survey instruments underwent obtained a 93% response rate. thorough testing in the form of expert review, cognitive testing, usability testing, and field testing. Pilot Phase: 1. Focus Groups after Month 1 of the Telecommuting In addition to providing the software for questionnaire Pilot with telecommuters, their coworkers and development, WebSurveyor also hosted the surveys on supervisors, 7/01. their site. Employees received a personalized e-mail inviting them to participate in a survey by clicking on an 2. Focus Groups after Month 2 of the Telecommuting 1291 Joint Statistical Meetings - Section on Survey Research Methods Pilot with telecommuters, their coworkers and Likewise, more telecommuters were female (64%) than supervisors, 8/01. non-telecommuters. Telecommuters were also more likely to be non-supervisory and, on average, higher Post-Pilot Phase: pay-graded than non-telecommuters. Telecommuters 1. Focus Groups after the Telecommuting Pilot with worked primarily as statisticians and computer telecommuters, their coworkers and supervisors, 12/01. specialists. 2. Post-Pilot Web Survey of All Employees in the 3.1.3 Most unable