"Lessons learned in implementing major operational changes for the 2006 Canadian Census"1

Richard Barnabé Statistics 120 Parkdale Avenue Ottawa, K1A 0T6, Canada [email protected]

1. Introduction

Statistics Canada will conduct its 20th Census since confederation, in May 2006. By law, Censuses of Population and Agriculture are required every 5 years. For nearly 30 years, Canada has had a relatively robust and stable collection and processing methodology which has relied on the use of a large geographically dispersed workforce to collect the data, conduct rudimentary edits for completeness in a manual fashion and follow-up missing questionnaires. Converting hand-written responses from over 11 million questionnaires to digital form has meant months of keying and coding by staff in federal government taxation offices (the Canada Revenue Agency/CRA) where spare capacity existed after the peak tax season. While collection and processing remained relatively constant, significant gains were made in editing and imputation, in coverage measurement systems, and most recently, with Internet as the primary mode to disseminate Census information to a broad range of users. Given the enormous expenditure associated with a Census undertaking, as well as the fact that there is only one chance to “get it right”, change has traditionally been very gradual and follows careful testing and analysis.

Nevertheless, it was clear that some fundamental changes were required to how we take a Census. First, census information in the hands of local enumerators, despite tight security provisions had become unacceptable to most . This was highlighted by the Office of the Federal Privacy Commissioner. Second, many Canadians expect a secure on-line option to complete their Census returns. Third, CRA now has a diminished keying capacity due to electronic tax filing, and could no longer meet our volume and timeliness requirements.

The 2006 Census approach entails significant changes including mailing-out questionnaires to approximately two-thirds of dwellings (for the other, mostly rural, third an enumerator will again drop -off the questionnaires), a secure on-line response option, the return of all responses to one central data processing center for automated editing, the use of intelligent character recognition technologies and automated coding software, and follow-up for missing information using Computer-assisted telephone interviewing (CATI) operating out of three call centers housing our Census Help Lines (CHL) operations. This leaves essentially only the follow-up on non-response cases for Field staff. These changes require a significant amount of automation and centralization, and present a major systems integration challenge. As a tool of central control, it was also judged essential to develop a central address register.

Statistics Canada decided early in 2000 on two significant measures to reduce the level of risk: to rely on proven private sector expertise to fast-track automation, and to conduct a comprehensive Census test (a dress-rehearsal) of the systems and processes critical for the new approach in May 2004. The Census Test was a success. We can now highlight some of the major issues and challenges Statistics Canada has faced to date in managing this methodological change and those that are anticipated over the next year and a half, as the project moves from development and testing to full scale implementation and production in 2006. Four aspects and our lessons learned for each are presented: Project Management and Governance, Contracting and Private sector involvement, the Internet as a collection medium, and Public Communications.

2. Project Management and Governance

1 This paper reflects the opinion of the author and not necessarily those of Statistics Canada Given the magnitude of change, the relatively short-timeframe to deliver tangible results, in light of the necessity for a Census test in 2004, and the involvement of outside contractors, it was clear that a strong project management structure and discipline was essential. Statistics Canada has had a dedicated census management office for several censuses. This office supports a cohesive project management structure of expertise from various parts of the agency. In addition, a set of Integrated Project Teams (IPTs) structured according to four key “authorities” (Contracting, Integration, Quality Management and Master Control System) was created for the 2006 Census. A Director General level Census Manager and several oversight or Steering Committees guide the Census Project Team in the decision-making process, issue escalation and resolution. A set of tools and committees/boards allow for the effective tracking and control of inter-project deliverables, the control and documentation of outstanding issues, risks and mitigation plans, change management (to several baselines on planning assumptions, technical architecture/design, and process), budgets, and formal decisions.

The lessons learned to date, having completed a successful Census test, are: · Retain the matrix management structure along with the open communications to senior management for quick resolution of contentious issues · Retain the Integrated Project Team structure with a tight governance mechanism to quickly escalate and resolve outstanding issues · Maximize the use of existing control mechanisms and tools and not be tempted to continuously create new ones · Focus on the horizontal and cross-cutting issues to ensure clear communications (examples of issues: integration, scalability, outsourcing/contracting, testing, human resource management) · Quick, transparent escalation to and documented resolution by senior project authorities of major issues to avoid dangerous procrastination or inaction.

3. Private sector involvement (contracting)

At the onset of the strategic development phase of the 2006 Census, an Outsourcing Steering Committee with broad representation from various areas of the agency and from the government department responsible for procurement (Public Works Government Services Canada\PWGSC) provided guidance on the outsourcing scope and mechanisms. Consultation was also undertaken with a number of other statistical agencies, such as the US Census Bureau, Australian Bureau of Statistics, and the UK Office of National Statistics. The initial scope of contracting activities for the 2006 Census was as follows:

· Automated capture of Census responses using optical recognition technology since Statistics Canada had limited experience with such technology and then only on small-scale applications. · Printing of questionnaires, specifically because of the links between paper format, size, ink type/quality, thickness and other traits with the effectiveness (thus cost) of imaging and intelligent character recognition (automated data recognition software/scanning hardware). · Development of the Internet application, because of issues related to scalability and the need to integrate data input via this mode to the mainstream paper-based processing systems. · Development of the CHL and CATI applications, because of their interdependence on the automated editing functions in the data processing centre. · Automated coding applications to realize efficiencies with data processing operations. · Administration and staffing of the data processing center and acquisition of the physical plant/facility, to provide performance incentives and hold the contractor accountable for ensuring efficient end-to-end operations and for implementing the systesm they develop.

The scope of outsourcing was unprecedented for the census. Statistics Canada communicated the need for a “prime-integrator” that could form a consortium with other sub-contractors. Working with PWGSC, the agency consulted with the private sector to define a fair tendering approach. Through an open, transparent and rigorous evaluation process, the contract was awarded in 2002 to Lockheed Martin Canada, with IBM Canada and Transcontinental Printing Canada as subcontractors. The contract was broken into three distinct phases – a four month design phase, a second phase leading to and including the Census Test, and a third to complete the full scale requirements for the Census. The contract stipulates strict security provisions consistent with the Statistics Act and Statistics Canada’s policies, and by its phased nature, provides the flexibility for either party to review and adjust the contract as warranted.

In late 2003, early 2004, just prior to the Census test, a number of Canadians expressed concern about privacy and security of Census data if it was in the hands of a subsidiary of an “American company” that could be subject to the USA Patriot Act. There were also objections on moral grounds to Lockheed Martin, that operates other entities involved in the defense industry. Although strict measures to protect the security of Census data were in place, Statistics Canada felt that perceptions about the security of data could undermine the confidence of Canadians and thus negatively affect participation in the Census. Therefore, a critical decision was made in early 2004 to reduce the scope of the contract to exclude any contractor involvement with the actual processing of census questionnaires/data. This limits the contracting activities to the provision of hardware, software, printing, and support services.

This midstream course correction presents a number of significant challenges in the areas of acquisition and fit-up and efficient operation of the data processing center, staffing, management, and, possibly most challenging, the technical administration, trouble-shooting, and “fixing” of vendor developed software and vendor furnished specialized hardware and the associated “knowledge transfer”.

The lessons learned to date with respect to private sector involvement (contracting) are: · Develop the contracting scope on strategic requirements (in our case-to reduce integration risks) · Develop the contracting approach in partnership with the procurement authority of the government very early in the process, including extensive open and transparent industry consultation · Employ an independent evaluation/oversight capacity for the contracting process · Structure the contract to oblige the vendor to specific security, timeline, budgetary and other constraints, while having the necessary provisions to allow flexibility to adapt to change (example: structuring the contract in phases with a formal review and adjustment mechanism) · Ensure the contract obligates the vendor to demonstrate ongoing progress, providing incentives for on-time, quality deliverables · Have the necessary resources and a tight governance structure to integrate the contracting activities with the rest of the Census program, and to avoid scope-creep and ambiguous requirements specifications · Operate in a fully concerted, transparent fashion where true teamwork and partnership prevail

4. The Internet as a collection mode

In response to user demand and a government objective to have all services on line by 2005, Statistics Canada is offering for the 2006 Census an Internet option to virtually all Canadian residents living in private dwellings and having internet access. This will comprise a user-friendly application that is easy-to-use and convenient and requires no software installation or downloading on the respondent’s computer, can be filled out rapidly and is demonstrably protected by security measures that ensure the safety and confidentiality of the information submitted to Statistics Canada. In addition, the Internet responses will be transmitted to a central data processing centre to be integrated into the processing stream with the data captured from paper, and through the census help line. The operating assumption is that at least 20% of respondents will use the option. To address the challenge of usability, specialists in Human-Computer Interaction conducted a number of tests to compare, among other things, sequential versus matrix layouts, drop down responses, etc. These tests were followed by focus groups, face-to-face interviews and presentations to numerous committees throughout Statistics Canada during the design and development phase (which started in the summer of 2003). A follow-up satisfaction survey after the Census Test revealed that usability objectives were met – almost 90% of those surveyed rated their overall satisfaction level at 8 out of 10 (1 being very poor and 10 being excellent), with 98% indicating a willingness to complete their 2006 Census questionnaire over the Internet.

As part of its strategy to have most government services on-line by 2005, the has taken on the challenge to meet security requirements for the transfer of confidential information through the Internet. This initiative, called Secure Channel (SC) is to be used by Federal departments as they implement their individual on-line strategies. Statistics Canada has developed a state of the art approach for the transmission of Census data using the SC. It features two-way encyption and the use of network “islands” (air-gaps) to prevent any connection between the ouside and the Census data processing and storage systems.

The lessons learned to date with the Internet collection mode have shown the importance of: · Beginning with existing tools and expertise and using effective partnerships to fit the Census context · Developing effective testing and consultation strategies at the planning stage · Ensuring a robust security environment that can effectively integrate with the application · Ensuring the application can integrate with other media such as paper and computer assisted telephone interviewing at detailed levels · Ensuring that rapid application assistance is available

5. Public Communications

Building a Public Communications program that will effectively persuade Canadians to complete and return their questionnaires in a timely fashion is a huge challenge that remained essentially unaddressed in the Census Test, in which participation was voluntary and for which no formal Public Communications effort was mounted. Yet, some 60% of the 300,000 households in the Census test returned a completed a questionnaire, consistent with Census tests leading up to previous Censuses.

The 2006 Public Communications strategy will be based on three components – support from corporations and organizations (stress importance of census to staff and clients and encourage them to complete their forms), engagement of the media in delivering the benefits of participation and under scoring Statistics Canada’s commitment to security and confidentiality, and a modest and highly focused paid advertising initiative.

We have, however, had a taste of the kind of major and unexpected communications challenges that can affect a Census with the Patriot Act issue. While our decision to limit the contract to the provision of hardware, software, printing and support services seems to have assuaged many of the concerns, the privacy environment in Canada maintains a high level of visibility and scrutiny of this contract. We must continue to convey our strict security measures. We are seeking the views and, to the extent that they can do so, endorsement of our federal and provincial Privacy Commissioners, and, for added assurance, will ask three different IT Security firms to review our security measures.

RÉSUMÉ Statistique Canada utilisera une nouvelle approche pour le recensement de 2006. Depuis 30 ans, la méthodologie reposait sur une « armée » de recenseurs locaux, une saisie par clavier et du codage manuel. Des préoccupations concernant la vie privée, la nécessité d’offrir une option de réponse en ligne et une diminution de la capacité de saisie par clavier ont dicté des changements majeurs qui sont axés sur l’automatisation et l’intégration/centralisation des opérations de collecte et de dépouillement. Ce texte décrit la structure de gouvernance, le rôle de fournisseurs du secteur privé et l’utilisation de l’Internet comme méthode de collecte qui caractérisent ces changements, ainsi que les leçons tirées du Test de 2004 et certains défis de communications.