Lessons Learned in Implementing Major Operational Changes for the 2006 Canadian Census"1
Total Page:16
File Type:pdf, Size:1020Kb
"Lessons learned in implementing major operational changes for the 2006 Canadian Census"1 Richard Barnabé Statistics Canada 120 Parkdale Avenue Ottawa, Ontario K1A 0T6, Canada [email protected] 1. Introduction Statistics Canada will conduct its 20th Census since confederation, in May 2006. By law, Censuses of Population and Agriculture are required every 5 years. For nearly 30 years, Canada has had a relatively robust and stable collection and processing methodology which has relied on the use of a large geographically dispersed workforce to collect the data, conduct rudimentary edits for completeness in a manual fashion and follow-up missing questionnaires. Converting hand-written responses from over 11 million questionnaires to digital form has meant months of keying and coding by staff in federal government taxation offices (the Canada Revenue Agency/CRA) where spare capacity existed after the peak tax season. While collection and processing remained relatively constant, significant gains were made in editing and imputation, in coverage measurement systems, and most recently, with Internet as the primary mode to disseminate Census information to a broad range of users. Given the enormous expenditure associated with a Census undertaking, as well as the fact that there is only one chance to “get it right”, change has traditionally been very gradual and follows careful testing and analysis. Nevertheless, it was clear that some fundamental changes were required to how we take a Census. First, census information in the hands of local enumerators, despite tight security provisions had become unacceptable to most Canadians. This was highlighted by the Office of the Federal Privacy Commissioner. Second, many Canadians expect a secure on-line option to complete their Census returns. Third, CRA now has a diminished keying capacity due to electronic tax filing, and could no longer meet our volume and timeliness requirements. The 2006 Census approach entails significant changes including mailing-out questionnaires to approximately two-thirds of dwellings (for the other, mostly rural, third an enumerator will again drop -off the questionnaires), a secure on-line response option, the return of all responses to one central data processing center for automated editing, the use of intelligent character recognition technologies and automated coding software, and follow-up for missing information using Computer-assisted telephone interviewing (CATI) operating out of three call centers housing our Census Help Lines (CHL) operations. This leaves essentially only the follow-up on non-response cases for Field staff. These changes require a significant amount of automation and centralization, and present a major systems integration challenge. As a tool of central control, it was also judged essential to develop a central address register. Statistics Canada decided early in 2000 on two significant measures to reduce the level of risk: to rely on proven private sector expertise to fast-track automation, and to conduct a comprehensive Census test (a dress-rehearsal) of the systems and processes critical for the new approach in May 2004. The Census Test was a success. We can now highlight some of the major issues and challenges Statistics Canada has faced to date in managing this methodological change and those that are anticipated over the next year and a half, as the project moves from development and testing to full scale implementation and production in 2006. Four aspects and our lessons learned for each are presented: Project Management and Governance, Contracting and Private sector involvement, the Internet as a collection medium, and Public Communications. 2. Project Management and Governance 1 This paper reflects the opinion of the author and not necessarily those of Statistics Canada Given the magnitude of change, the relatively short-timeframe to deliver tangible results, in light of the necessity for a Census test in 2004, and the involvement of outside contractors, it was clear that a strong project management structure and discipline was essential. Statistics Canada has had a dedicated census management office for several censuses. This office supports a cohesive project management structure of expertise from various parts of the agency. In addition, a set of Integrated Project Teams (IPTs) structured according to four key “authorities” (Contracting, Integration, Quality Management and Master Control System) was created for the 2006 Census. A Director General level Census Manager and several oversight or Steering Committees guide the Census Project Team in the decision-making process, issue escalation and resolution. A set of tools and committees/boards allow for the effective tracking and control of inter-project deliverables, the control and documentation of outstanding issues, risks and mitigation plans, change management (to several baselines on planning assumptions, technical architecture/design, and process), budgets, and formal decisions. The lessons learned to date, having completed a successful Census test, are: · Retain the matrix management structure along with the open communications to senior management for quick resolution of contentious issues · Retain the Integrated Project Team structure with a tight governance mechanism to quickly escalate and resolve outstanding issues · Maximize the use of existing control mechanisms and tools and not be tempted to continuously create new ones · Focus on the horizontal and cross-cutting issues to ensure clear communications (examples of issues: integration, scalability, outsourcing/contracting, testing, human resource management) · Quick, transparent escalation to and documented resolution by senior project authorities of major issues to avoid dangerous procrastination or inaction. 3. Private sector involvement (contracting) At the onset of the strategic development phase of the 2006 Census, an Outsourcing Steering Committee with broad representation from various areas of the agency and from the government department responsible for procurement (Public Works Government Services Canada\PWGSC) provided guidance on the outsourcing scope and mechanisms. Consultation was also undertaken with a number of other statistical agencies, such as the US Census Bureau, Australian Bureau of Statistics, and the UK Office of National Statistics. The initial scope of contracting activities for the 2006 Census was as follows: · Automated capture of Census responses using optical recognition technology since Statistics Canada had limited experience with such technology and then only on small-scale applications. · Printing of questionnaires, specifically because of the links between paper format, size, ink type/quality, thickness and other traits with the effectiveness (thus cost) of imaging and intelligent character recognition (automated data recognition software/scanning hardware). · Development of the Internet application, because of issues related to scalability and the need to integrate data input via this mode to the mainstream paper-based processing systems. · Development of the CHL and CATI applications, because of their interdependence on the automated editing functions in the data processing centre. · Automated coding applications to realize efficiencies with data processing operations. · Administration and staffing of the data processing center and acquisition of the physical plant/facility, to provide performance incentives and hold the contractor accountable for ensuring efficient end-to-end operations and for implementing the systesm they develop. The scope of outsourcing was unprecedented for the census. Statistics Canada communicated the need for a “prime-integrator” that could form a consortium with other sub-contractors. Working with PWGSC, the agency consulted with the private sector to define a fair tendering approach. Through an open, transparent and rigorous evaluation process, the contract was awarded in 2002 to Lockheed Martin Canada, with IBM Canada and Transcontinental Printing Canada as subcontractors. The contract was broken into three distinct phases – a four month design phase, a second phase leading to and including the Census Test, and a third to complete the full scale requirements for the Census. The contract stipulates strict security provisions consistent with the Statistics Act and Statistics Canada’s policies, and by its phased nature, provides the flexibility for either party to review and adjust the contract as warranted. In late 2003, early 2004, just prior to the Census test, a number of Canadians expressed concern about privacy and security of Census data if it was in the hands of a subsidiary of an “American company” that could be subject to the USA Patriot Act. There were also objections on moral grounds to Lockheed Martin, that operates other entities involved in the defense industry. Although strict measures to protect the security of Census data were in place, Statistics Canada felt that perceptions about the security of data could undermine the confidence of Canadians and thus negatively affect participation in the Census. Therefore, a critical decision was made in early 2004 to reduce the scope of the contract to exclude any contractor involvement with the actual processing of census questionnaires/data. This limits the contracting activities to the provision of hardware, software, printing, and support services. This midstream course correction presents a number of significant challenges in the areas of acquisition and fit-up and efficient operation