Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Summary of Research Proposal Summary of Research Background: In 2003, the Vancouver Coastal Health Authority (VCH) chose Mid-Main Community Health Centre (Mid-Main) as one of the clinics to receive primary health care renewal funds. A portion of the funding has been used to purchase and implement an electronic medical record (EMR) system that has some capacity to support chronic disease management through patient registries. In implementing their electronic medical record, Mid-Main has had ongoing contact with Elaine Grenon, Director of New Practice Design with the VCH Primary Health Care Network. In addition, Mid-Main has worked with Dr. Ellen Balka (professor and research scientist at Vancouver Coastal Health) who is the principal investigator of ACTION for Health, a 4-year, $3 million project that has focussed on the role of technology in the production, consumption, and use of health information. Since 2003, several ACTION for Health research projects have been carried out at Mid-Main, including a project which investigated the implementation and use of Wolf Medical Systems’ electronic medical record. The proposed project seeks to build on existing partnerships between Mid-Main, Wolf Medical Systems, VCH, and Balka’s research group through 3 related ‘knowledge to action’ projects of interest to all parties. Objectives: Project objectives are to support the translation of knowledge to action through: 1. Demonstration of enhanced use of data from electronic medical records in support of continuity of care in clinical settings and practice research. This project will focus on building capacity within Mid-Main to increase use of electronic record data to support practice improvement initiatives, particularly those targeted at the population of people who live with 3 or more chronic conditions within the cardiovascular and respiratory clusters, and people with persistent mental illness, who have been identified by VCH as using disproportionately high levels of health service resources. 2. Knowledge translation (KT) from research literature to local users (Mid-Main and VCH) and from local users (Mid-Main) to decision makers (VCH) and technology producers (Wolf), and subsequently potential electronic record adapters, about challenges related to enhanced use of electronic records for management of patients with multiple chronic conditions. 3. Translation of primary research results into good practice guidelines. In B.C., the Ministry of Health has developed a chronic disease management toolkit (CDM Toolkit) used by Mid-Main and other practices. Plans for management of patients with multiple co-morbidities involve increased use of the CDM Toolkit. Primary research will be undertaken in order to better understand how the CDM Toolkit is being used, and identify issues associated with data sharing between clinics and the CDM Toolkit. Results will be disseminated as a good practice guide, suitable for use by clinics. Hypothesis: That knowledge translation activities will result in evidence of practice changes at Mid- Main, evidence of utilization of research results by VCH, and reflected in software (Wolf). Research Plan • Objective 1: Develop a base-line understanding of use of EMR data at Mid-Main through observation and interviews; develop a plan for increased use of EMR data with Mid-Main; implement the plan; evaluate extent to which KT activities contributed to practice changes (through document analysis, observations, and interviews). • Objective 2: Through on-going engagement, respond to informational needs of potential research users. Collect data about appropriateness of materials produced through post-use evaluation of research materials, and interviews with potential users aimed at assessing utilization. • Objective 3: Conduct interviews with Mid-Main and other CDM Toolkit users about use of the CDM Toolkit; develop good practice guide (GPG) to support implementation of the CDM Toolkit; evaluate GPG through post-use surveys and interviews. 9 Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Summary of Progress Summary of Progress

During tenure of a 41/2-year, $3 million Social Sciences and Humanities Research Council of Canada grant (2003-2008) that has addressed a range of issues related to the production, consumption, and use of health information in varied settings (the ACTION for Health program), Dr. Balka has built excellent relationships with both the Mid-Main Community Health Centre and Vancouver Coastal Health. The Mid-Main Community Health Centre (Mid-Main) implemented an electronic medical record (EMR)1 with funds that were administered through Vancouver Coastal Health’s Primary Healthcare Transition Fund, and in this context, Mid-Main worked with Elaine Grenon at Vancouver Coastal Health during the EMR implementation. Balka’s research team has conducted a long term ethnographic study of the introduction and use of the EMR, which was developed by Wolf Medical Systems, a company located in Surrey, British Columbia. As an early adapter of the Wolf EMR (an EMR that was approved for use by practices that received Primary Healthcare Transition Funds), both Elaine Grenon and Mid-Main have had ongoing contact with Brendan Byrne (President, Wolf Medical Systems). In the last year, Balka’s research team has initiated contact with Dr. Byrne and shared information from ACTION for Health research undertaken at Mid-Main which addressed the EMR, with him. Hence, the basis for productive relationships required to foster successful knowledge translation are already in place.

Balka’s team has developed several reports at the request of Mid-Main (see http://www.sfu.ca/act4hlth/practitioners/index.html for examples) and a good knowledge translation relationship currently exists between Balka and Mid-Main. In her capacity as the Director of the Centre for Health Services and Policy Research, Dr. Black has had contact with several of the doctors at Mid- Main who engage in research. Drs. Balka and Black share many common interests, including interests in electronic medical records, data quality, and continuity of care. This project has yielded an opportunity for them to begin formally working together. Both Balka and Black have had ongoing contact with Vancouver Coastal Health. Balka holds a position as a senior research scientist there, and Black, through both her role as the director of the Centre for Health Services and Policy Research and as a researcher, has had ongoing contact with several people at VCH.

Any research referred to in this proposal that was conducted as part of ACTION for Health and was undertaken at Mid-Main, is complete.

1 Electronic medical records (EMR) are digital records that document the course of patients’ health care, while being a medium of communication among healthcare professionals for current and future care of the patients (American Medical Record Association, 1984). 10 Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Response to Previous Reviews Response to previous reviews

Not Applicable.

11 Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Research Proposal

Introduction A recent newspaper article indicated that as many as 1/3 of British Columbia’s population—1.3 million people—have chronic conditions such as diabetes, heart disease, or lung disease. This population consumes “80 per cent of the combined budgets of hospitals, the Medical Services Plan and PharmaCare.” (Feyerman, 2007, p. A-1). In the same article, the provincial Minister of Health, George Abbott, pointed out that “by concentrating on individuals who use the system most often, we can improve their overall health and well-being and reduce pressures and costs across the health system” (Feyerman, 2007, p. A-1). The article goes on to detail a variety of financial incentives that the province has put in place in efforts to entice doctors to invest more time in chronic disease management. Taking advantage of several of the recently announced incentives will require general practice offices to computerize their patient records and share data about management of chronic diseases with the Province through the province’s Chronic Disease Management (CDM ) Toolkit, which is a web-based resource that gives authorized doctors, nurses, and other care providers access to tools and information incorporating proven best practices for the treatment and management of chronic conditions.

The project outlined here will demonstrate the impact of sharing clinical information through eHealth solutions on continuity of care and practice patterns of healthcare providers, and will contribute to knowledge generation about best practices for the secondary use of EMR data to support practice and health system performance management and research. Building on existing relationships, this project focuses on research issues of interest to and generated by a community health clinic, a regional health provider, a research team and a software developer, and the translation of knowledge from primary and secondary research to research users, including an electronic medical record developer and vendor.

This project seeks to accelerate the flow of research knowledge to end-users, with the end goal of improving the health of Canadians through informational and management continuity of care, and by strengthening Canada’s healthcare system through knowledge transfer about the challenges of health sector information technology implementation and best practices for eHealth implementations. It will support and enhance existing partnerships and will support the development of research-to-practice links between academics from two institutions, a regional health authority, a community clinic that has been an early adapter of an electronic record system, and the designer and vendor of that electronic record system. Through support of these relationships, this project will facilitate communication among the health research, policy, practice and technology development communities in Vancouver. By working together to develop, implement, and evaluate knowledge translation activities, this project will significantly strengthen existing partnerships between the research, practice, and policy communities working to improve continuity of care in British Columbia.

Building on previous research projects and partnerships, this project will demonstrate the impact of sharing clinical information through eHealth solutions on continuity of care and practice patterns of healthcare providers and will contribute to the development of best practices for the secondary use of EMR data to support practice and health system performance management and research. It will also contribute to our understanding of effective models for change management and adoption of eHealth technologies both within an organization and between organizations.

Objectives of this project—which will be addressed in greater depth below—are to:

12- a Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal 1. Investigate whether enhanced use of data from electronic medical records supports continuity of care in clinical settings and practice research; 2. Engage in knowledge translation from research literature to local users and from local users to decision makers, and subsequently potential electronic record adapters, about challenges related to enhanced use of electronic records for management of patients with multiple chronic conditions; and 3. Translate primary research results into good practice guidelines suitable for use by clinics. Project objectives are best understood within the broad context of three different literatures: literature concerned with continuity of care, literature concerned with socio-technical approaches to the study of health informatics, and literature concerned with knowledge translation. These literatures provide a context for our research plans, while at the same time offering a rationale for our plans. Each of these areas is outlined briefly below in relation to our research research and knowledge translation plans.

Continuity of Care In recent years numerous factors have contributed to an increased concern with the notion of continuity of care in the provision of health services. Computer systems are being designed and implemented in efforts both to facilitate improved continuity of care and to measure the extent to which continuity of care is achieved. The knowledge translation activities described in this proposal are concerned with the use of computers to support continuity of care, and translation of knowledge about socio-technical challenges that emerge as computing and information systems are introduced to support continuity of care. As has been pointed out (Haggerty et al., 2003), “the concept—and reality—of continuity of care crosses disciplinary and organizational boundaries...patients are increasingly seen by an array of providers in a wide variety of organizations and places, raising concerns about fragmentation of care.” Efforts both to describe the problematic concept of continuity of care and to formulate solutions to address shortcomings in continuity of care (including the development and use of computer systems designed to support continuity of care) are complicated by the lack of consensus about the definition of continuity (Haggerty et al., 2003). Computer and health information systems are often seen as “enabling systems” (Vancouver Coastal Health, 2004) and are put in place to support continuity of care.

Continuity of care is an often used but infrequently defined term (Freeman, Shepperd, Robinson, Ehrich, & Richards, 2000). Freeman et al. concluded that continuity of care is a multi faceted concept used in different ways, including to refer to continuity across organizational boundaries, continuity of personnel, continuity of information, and continuity over periods of time. Out of in excess of ten definitions of ‘continuity of care’ Freeman et al. located, the three most common definitions referred to i) longitudinal or provider continuity (seeing the same care provider), ii) continuity across the primary care / secondary care interface (effective transitions from generalist to specialist providers), and iii) continuity through records—either written or electronic. Reid et al. (2003) argue that the core notion of continuity is that patients experience their care as connected and coherent over time. Three main types of continuity support this “uninterrupted succession of events:” informational, management, and relational continuity. Informational continuity is related to what Freeman et al. (2000) describe as continuity through records—it reflects the idea that details about past events ought to be available to inform current care. Increasingly, identification and implementation of strategies that support communication of patient information across the continuum of care is seen as an important patient safety issue (VCH, 2005).

Reid et al.’s (2003) term “management continuity” extends Freemen et al.’s (2000) concept of continuity across the primary/secondary care continuum. For Reid et al. (2003, p. 2), management continuity “means that care is clinically consistent over time, that someone is managing the case, or that the 12- b Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal management responsibility is efficiently and effectively transferred, and that as a result duplication of procedures and communications are kept to a minimum.” A similar point was raised by Freeman (1984), who suggested, long before electronic record systems were commonplace, that computerization of medical records was one means through which consistency of care could be introduced. To confuse matters more, relational continuity (one of the defining attributes of primary care) is at times referred to as longitudinality (Reid et al., 2003, p. 2). Reid et al.’s (2003) notion of relational continuity “means that ongoing patient provider relationships are nurtured in order to bridge healthcare events and create an expectation for future care” is similar to Freeman et al.’s (2000) concept of longitudinal or provider continuity over time.

Although the meanings associated with the term continuity of care are varied and often poorly specified, in spite of these varied meanings, a lack of continuity of care is often associated with poor health outcomes, while the presence of continuity of care is associated with more positive health outcomes. For example, Dirnfeld (2004, p. 3) has pointed out that a “lack of continuity impairs the quality of care and diminishes the health outcomes of such patients.” Often the drive to computerize medical records is implicitly associated with the delivery of more continuous care. For example, in a document that lists a compilation of health outcomes to be achieved through the use of electronic medical records (Vancouver Coastal Health, n.d.), continuity of care is both explicitly addressed (e.g., “improve continuity of care and measure by frequency with which patient sees preferred provider”), and implicitly addressed (e.g., “establish/ expand multi-disciplinary practices, maintain discipline-appropriate records and improve inter-provider communications”).

Broemeling, Watson, Black, and Reid (2006) have pointed out that in British Columbia, “very little consistent data is collected regarding organizations delivering primary health care at the local level” (Centre for Health Services and Policy Research, n.d.), one consequence of which is that it is not presently possible to compare primary healthcare models across jurisdictions. Broemeling et al. conclude that Canada lacks the most basic information required to assess and monitor the immediate outcomes of the primary healthcare system, especially those related to reduction of risk and duration of illness. They suggest that a “common, systematic evaluation framework and consensus process should guide Canadian efforts to develop a comprehensive, population-based data collection system and the next generation of PHC information systems” (Centre for Health Services and Policy Research, n.d.). They also have suggested that investment in “sophisticated new data collection strategies are required to make it possible to measure, monitor and manage” (Centre for Health Services and Policy Research, n.d.) primary health care. Information technology systems are seen as a means through which improved research about primary care in general can be supported as well as a means through which continuity of care (particularly informational and management continuity) can be supported and efforts undertaken to improve continuity of care can be evaluated.

Technology as Enabling Systems: Computer Support for Continuity of Care In recent years several computer-based initiatives have been undertaken in British Columbia in efforts to support, or enable, continuity of care. Increasingly, primary health care providers are turning to electronic record systems in efforts both to support improvements in continuity of care and to collect data that will support research about whether or not changes implemented are achieving desired outcomes. One clinic that has embarked on the implementation of an electronic record system in efforts to improve continuity of care and determine whether or not service delivery changes are having desired effects on patient health outcomes is the Mid-Main Community Health Centre, located in Vancouver.

12- c Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Mid-Main Community Health Centre: Demonstration and Research Site Mid-Maine received funding through the Primary Health Care Transition Fund (http://www.hc- sc.gc.ca/hcs-sss/prim/phctf-fassp/index_e.html) for a series of practice changes, which included implementation of an electronic medical record. Formed in 1988, Mid-Main’s approach incorporates the use of a multidisciplinary team of health care professionals working in collaboration with each other to provide accessible primary health care to clients. Mid-Main hoped to support collaboration in efforts to increase emphasis on chronic disease management, health promotion, and disease prevention activities through the use of electronic medical records, registries, group appointments, patient education, and support for patient self-management. Mid-Main’s approach includes the development of more effective and efficient relationships with other providers such as community-based health providers and agencies and with providers in primary, secondary, and tertiary levels of care.

Prior to receipt of Primary Health Care Transition Funds (PHCTF) that allowed Mid-Main to acquire an electronic medical record system (EMR), Mid-Main’s own database provided some statistical information about client’s demographics and about the numbers and types of primary health care services clients received by members of Mid-main’s clinical team. These statistics played an important role in planning and implementation of primary health care services for the PHCTF proposal. Statistics collected on primary healthcare services for chronic diseases such as diabetes, heart failure, and asthma prior to introduction of the EMR demonstrated the numbers and complexity of services currently provided by Mid-Main clinicians in supporting clients with these health care issues. Despite the efforts of the clinicians, many clients struggle with their chronic conditions and had frequent contact with Mid- Main providers and with providers in Emergency Rooms and Hospitals. Recognition of this gap in managing and educating clients with chronic disease led Mid-Main to consider implementing an EMR and registries as means of providing more effective primary health care. Through use of an EMR system and the development of registries, clinic staff hoped to increase use of health promotion, disease and injury prevention strategies with an end goal of supporting clients’ self-management and self-care. Mid- Main sought to implement and evaluate their service provision in terms of targets, outcomes, and performance measures while measuring client and caregiver satisfaction. Increased data availability resulting from EMR use has provided a means through which to assess health determinants and outcome measures, which in turn could be used to support Mid-Main’s ongoing system redesign.

In implementing an EMR and registries, Mid-Main’s goals included: • enhancing the range of primary health care services provided at/by Mid-Main’s health care providers to include more health promotion and education activities and to undertake chronic disease management initiatives for clients; • improving the health outcomes of clients with chronic illness; • completing more quality assurance mechanisms and evaluations of primary health care services; • improving communication between health care providers facilitated by electronic medical record systems, case conferencing, and regularly scheduled meetings; • initiating regular outcomes and evaluation monitoring of clinical practices; and • increasing outcomes monitoring and evaluation of services. Mid-Main hoped to develop indicators and outcome targets for outcomes evaluations and to begin to gather and monitor data and provide feedback to the care team. While work towards these goals has been significant since EMR implementation, challenges associated with implementation of the EMR and technical problems that delayed implementation of some features of the EMR (e.g., the EMR’s ability to communicate with the provincial CDM Toolkit) have slowed progress in reaching these goals.

12- d Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal With Mid-Main now through the initial period of EMR implementation, it is now time to engage in additional practice enhancements to improve continuity of care. With this application, we are seeking funds to hire a researcher who can work closely with Mid-Main’s care team in an effort to more fully realize Mid-Main’s goals outlined above. Specifically, the clinical practice researcher to be hired with will work with Mid-Main to assess data availability related to use of Mid-Main’s EMR (the Wolf system), assess the quality of existing data, work with the care team to determine what types of practice data the care team would like, assess the quality of those data, assist the care team in thinking through the implications to any changes in data capture and data collection strategies,2 and help the care team in developing the skills that will allow them to seamlessly integrate practice level research into daily practices. The researcher will work with the care team to develop indicators related to goals outlined above, such as improvement of health outcomes amongst clients with chronic illness, and review of practice data (e.g., what percentage of female patients have had their annual pap smear?; what percentage of a doctor’s patients with diabetes are taking advantage of preventative services such as foot care clinics etc.), and develop data retrieval and analysis strategies that can be integrated into clinical practices and can be sustained over time and which will allow Mid-Main clinical staff to increase their use of data from the EMR. Although members of the care team have learned an enormous amount in the less than two years that the EMR has been in use, and doctors are quite savvy in their use of electronic records and are interested in and willing to take advantage of the information made available through their electronic record system, at present, practice research (pulling of patient data at the practice level) to date has been undertaken by one doctor, who has been pulling practice data “off the side of her desk” for others in the practice. The clinical practice researcher to be hired with grant funds will be dedicated to helping doctors understand how to run practice searches, what the implications are for data capture and collection, and how to get data out of the EMR in report format and make sense of reports.

VCH as a Demonstration and Research Site VCH has engaged in a chronic disease management plan which includes increasing the number of general practices using electronic medical record systems and the CDM Toolkit and pilot testing software designed to facilitate informational and management continuity of care. By working closely with VCH, we will seek opportunities to disseminate knowledge gained from work undertaken through the ACTION for Health research program. In addition, we will seek opportunities to compile research syntheses on topics related to computer-supported chronic disease management and related topics that can assist VCH in carrying out chronic disease management. As a demonstration site, we will carry out observationally based ethnographic work prior to and during pilot implementations in a manner outlined in Balka and Kahnamoui (2004).

Wolf Medical Systems as a Research Subject Wolf’s EMR has been in use at Mid-Main for two years. As Mid-Main now works towards development of new skills required to conduct practice searches, and as the CDM Toolkit is upgraded, we will have opportunities to collect primary research data about usability issues, issues related to how work practices change in relation to the use of the EMR, and we will be able to document challenges and problems

2 As Mid-Main has begun to pull data from their EMR, they have learned that some of their existing strategies for entering data into the EMR lacked uniformity. Hence, in the interest of developing higher quality data that reflect standard practices, at times changes have been made to strategies surrounding data capture (e.g., how a particular health issue is recorded is standardized). It is anticipated that as Mid-Main continues to work with the practice search feature, that additional instances where data capture needs to be standardized will occur. Tracking these changes is important in terms of both documentation of data and data quality, and can serve as the basis for the generating knowledge of interest to other EMR implementers. 12- e Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal associated with use of Wolf’s EMR. As above, we will use ethnographic field techniques to collect data. Means of dissemination to Wolf Medical Systems are not yet known, but, for example, might consist of detailed logs of users’ experiences using the system together with use scenarios based on fieldwork. Data collection activities outlined will allow us to demonstrate and document the impact of sharing clinical information through eHealth solutions on continuity of care and practice patterns of healthcare providers and will contribute to the development of knowledge about best practices for secondary use of EMR data to support practice and health system performance management and research.

Socio-technical Approaches to EMR Implementation and Knowledge Translation In contrast to health technology assessment (HTA) activities (“a type of systematic research that identifies, synthesizes and analyses all available evidence on the clinical effectiveness and cost- effectiveness of health care technologies, including drugs, equipment and systems” [CCOHTA, 2003]), the socio-technical approach to evaluation of health information technology is rooted in the broad goals of technology assessment (TA): a class of policy studies that examines the effects of technology on society that may occur when a technology is introduced, extended, or modified, with special emphasis on unintended, indirect, or delayed consequences (Coates, 1974) (see Figure 1). Morgall (1993, p. 5) argues that in order to be an effective tool, HTA “must address the consequences of technology in the context of everyday life.” As it has historically been performed in Canada, HTA has left issues related to the implementation and use of IT for the delivery of health information and health services unexamined. In fact, the Canadian Agency for Drugs and Technologies in Health (CADTH) has clearly indicated that the assessment of information technology use in the delivery of health care is beyond the scope of their mandate (M. Gaucher, personal communication, April 23, 2007).

As infrastructure, IT is often taken for granted, receiving critical scrutiny only when it fails (Star & Ruhleder, 1996). Recent activities suggest this may be changing: still in its early stages, the assessment of IT in the health sector merits critical assessment in the health and socio-technical contexts. Lehoux (2006) and Berg (2003) argue for a reframing of problems with health technology by drawing on insights from science, technology and society studies (STS) and those from health technology assessment (HTA). Social informatics, which “examines the design, uses, and consequences of information and communication technologies in ways that take into account their interaction with institutional and organizational contexts” (Kling, 2000, p. 217) provides the tools for such an approach (see Table 1). It is based on the insight that standard views of technology as tools have led to underestimations of costs and complexities of computerization and overestimates of the generalizability of systems between settings and groups and that workable systems are “supported by strong sociotechnical infrastructure” (Kling, 2000, p. 228).

Although an ample literature exists which addresses the socio-technical challenges of implementing new technologies in healthcare organizations, such studies are typically excluded from systematic reviews because they are often case studies, rather than randomized controlled trials or case-control studies, the results from which are considered by many the gold standard of health research. One of the conclusions reached by Delpierre et al. (2004, p.407) in the conclusion of their systematic review of computer-based patient record systems and quality of care is that “alternative approaches considering social, cultural, and organizational factors may be needed to evaluate the usefulness of CBPRS” (computer based patient record systems). They argued that “future research is needed considering qualitative factors that could influence the use of CBPRS” (Delpierre et al., p. 413). Studies undertaken from a socio-technical perspective are often undertaken by interdisciplinary scholars who are as likely to be published in the computer science literature (see, for example, Balka & Kahnamoui, 2004; Proctor, Rouncefield, Balka,

12- f Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal & Berg, 2006) as they are in the health sciences literature. Consequently, although literature exists that yields insights about the challenges associated with successful implementation of EMRs, much of that literature is seldom consulted by health care practitioners.

Studies of everyday use of health information technologies in Canada were virtually non-existent until a few years ago, when the Social Sciences and Humanities Research Council of Canada (SSHRC) funded the ACTION for Health project (Ellen Balka, Ph.D., principle investigator), which has had, as one of its goals, the completion of several field-based studies of information technology implementation and use in varied clinical settings in Canada. The Mid-Main Community Health Centre has been one of the sites where field-based ethnographic research has been carried out by ACTION for Health researchers, whose project offices are located in Vancouver. Data collection at Mid-Main has focussed on the identification of issues, challenges, and unintended consequences of electronic record implementation, including those related to sharing of data with other organizations (such as the provincial Ministry of Health, through use of the provincial Chronic Disease Management Toolkit). Supported by ACTION for Health research funds, data collection began at Mid-Main prior to the introduction of Mid-Main’s electronic medical record system and will continue through December 2007, ending just prior to the end of the ACTION for Health funding period.

A second objective of our proposed research is to carry out knowledge translation activities that will bring research insights from published literature—including literature undertaken from a socio-technical perspective (which is often missed by health researchers and practitioners)—to local users, including practitioners at Mid-Main and decision makers and program designers at Vancouver Coastal Health Authority (VCH). In addition, as field work that began at Mid-Main prior to the implementation of EMRs and will continue through implementation concludes, VCH is also embarking, in cooperation with the provincial Ministry of Health on a widespread rollout of EMRs. Hence, it is an opportune time to work in close collaboration with VCH to develop materials—based on both published literature and fieldwork—that can be used to support subsequent rollout of EMRs within BC. Indeed, our second objective seeks both to bring often overlooked knowledge (research undertaken from a socio- technical perspective) and results from field based studies to a group of users (decision makers as well as other clinicians in need of information that can help guide successful EMR implementations and to Mid-Main’s EMR vendor, Wolf Medical Systems) all of whom have a concrete need for that information. Knowledge Translation In recent years there has been much comment about the limited extent to which research findings are used by decision makers and practitioners. Increasingly, scholars and practitioners are searching for ways to increase the use of research evidence by decision makers and practitioners. Activities undertaken with this end in mind have been referred to as knowledge mobilization (the Social Sciences and Humanities Research Council of Canada), knowledge exchange, knowledge transfer or knowledge translation (Canadian Institutes for Health Research [CIHR]), to name a few. CIHR has defined knowledge translation as “ the exchange, synthesis and ethically-sound application of knowledge - within a complex system of interactions among researchers and users - to accelerate the capture of the benefits of research for Canadians through improved health, more effective services and products, and a strengthened health care system” (Canadian Institutes for Health Research, 2005, ¶ 1). Although there are no easy prescriptions for successful knowledge translation, increasingly KT has become the subject of research, which has yielded several insights about successful KT practices.

Roos and Shapiro (1999), Lomas (1993), and Davis (2000) have all found evidence that in-person 12- g Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal interaction and early involvement of the stakeholders in the research process are elements of successful knowledge translation activities. Collaboration at all stages of the research process is necessary (Davis & Howden-Chapman, 1996; Golden-Biddle et al., 2003; National Centre for the Dissemination of Disability Research, 1996). It is also important to develop audience-specific messages that are in line with the decision-making environments in which they apply (Lavis, Ross, McLeod, & Gildner, 2003). Knowledge transfer is more effective when it is relevant to the role and priorities of the decision maker (Rogers, 1983; Dobbins et al., 2002a), and sustained dialog between researchers and decision makers has been shown to increase use of research in health policy decisions (Elliott & Popay, 2000). Shonkoff (2000) and others have highlighted the importance of credibility in the knowledge translation process. The delivery of information by someone perceived as credible increases the likelihood that such information will be acted upon. Active strategies such as “academic detailers” for clinicians and face-to- face briefings for policymakers are more effective at knowledge translation than other techniques (Eccles et al., 2001; Jacobsen, Butterill, & Goering, 2003; Lavis et al., 2003). Dissemination of research where discussion of findings is enabled is effective, as it develops closer links between researchers and practitioners. Interaction between practitioners and an “implementation officer” allowed emphasis to be placed on the benefits of particular research to specific service-planning activities (Stevens, Liabo, Frost, & Roberts, 2005). Delivery of information that occurs when knowledge is being transferred must be timed so that it arrives when it is needed (Stevens et al, 2005; Palmer & Fenner, 1999). At the heart of this project is a multi-faceted knowledge translation strategy aimed at improving the flow of research information from researchers to three different primary research users: • a community clinic using an electronic record system that, with project funds, will increase their capacity to conduct practice research using data from their electronic record system; • the designer and vendor of that electronic record system, who is interested in learning more about the problems and challenges users face in integrating his product into their practice environment; • a regional health authority which is embarking on a chronic disease management strategy aimed at improving informational and management continuity of care, and which will require family practice offices to use electronic record systems to upload data to the provincial CDM Toolkit and which will include pilot implementations of other software tools in varied settings, including family practices. Each of these research users will also serve as a focal point for research. We will work with Mid-Main to increase their capacity to undertake practice searches that will allow them to gain insight into the clinic’s management of it’s chronic disease populations, and during that process, we will collect data about the challenges they face in increasing their use of data from their electronic medical record system. Because Mid-Main currently uses Wolf Medical Systems electronic medical record software, we will gain insight about the design of that software suite and issues related to the challenges practices encounter when integrating EMR software into their daily routines. This information is of interest to Wolf Medical Systems, who can use the information both to improve design of their software and to help foster realistic expectations amongst practices embarking on computerization.

Insights gained from the study of Mid-Main as they increase their use of the practice search facility and as a result of our past work both observing the introduction of the EMR at Mid-Main and Mid-Main’s use of Wolf software to communicate with B.C.’s provincial Chronic Disease Management Toolkit are of interest to Vancouver Coastal Health. VCH is playing a significant role in computerization of general practices (which, once computerized, will use the provincial CDM Toolkit) and has recently embarked upon a major chronic disease management initiative which will include pilot implementations of new software designed to support improved informational continuity of care across a range of settings. We

12- h Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal will draw on insights gained from ACTION for Health research (particularly Balka & Wagner, 2006; Balka, Wagner, & Jensen, 2005) as the VCH change management team prepares for pilot implementations of software tools deigned to support enhanced chronic disease management. Using methods outlined in Balka and Kahnamoui (2004) and Balka, Wagner, and Jensen, (2005) we will collect data during the pilot phases of software implementations, and we will subsequently work with VCH to identify issues that may have organization-wide or cross-organizational implications and hence may warrant additional investment in knowledge translation.

Research Design The project described here will be undertaken within an action-research framework. Designed to bridge the gap between theory, research, and practice (Holter & Schwartz-Barcott, 1993), action research generates research about a social system while trying to change it (Hart & Bond, 1995). Action research is particularly well suited to research aimed at problem solving and improvement (Hart & Bond, 1995). In action research, researchers are experts in research methods, while practitioners are experts in the object of study (Korpela, Mursu, Abimbola Soriyan, & Eerola, 2002) — in this case, work practices in their clinical settings.

This project will follow a professionalizing approach to action research. “The main aim is the improvement of professional practice at the level of organizational and cultural change, rather than in terms of a challenge to existing power relationships, or the involvement of users” (Hart & Bond, 1995, p.45). While at Mid-Main and Wolf Medical Systems it may be possible to engage a significant proportion of computer system users and data users in our study (both are relatively small organizations), the sheer size of VCH makes such an approach impossible. Hence, in that context, the focus of research undertakings will not be determined at a grass roots level with the input of a majority of research participants. In contrast, our research questions have originated (and will continue to evolve through discussion) through discussion with research partners about which, out of a range of researchable questions, will best meet their needs.

Methods Once specific research questions have been determined through discussion with research partners, appropriate protocols for collection of data will be determined. It is anticipated that primary data collection will consist of extensive observation and informal interviews that will focus on documenting end user’s difficulties during system implementation processes. During pilot implementations at VCH and during early stages of practice searching at Mid-Main we anticipate that research staff will act as mediators between the social and the implementation team along the lines advocated by Westrup (2002). An ethnographic approach will be undertaken because it often provides “a much better means of anticipating the dynamic effects on work organization” (Bodker, Kensing, & Simonson, 2004, p.21). Research staff will focus on documenting end user’s difficulties during system implementation, communicating problems to the teams responsible, facilitating resolution of problems identified, and communicating results to front line staff.

Knowledge syntheses may be conducted to support partners in decision making, and if these are undertaken they will be conducted in a manner similar to a systematic review where search criteria, search terms, and databases checked will all be logged, and in addition to a summary of materials, we will present findings in a format that will allow decision makers to easily identify and assess what type of study was undertaken, how the study was carried out, what was measured or evaluated, and what research methods were used, how they were applied, etc. 12- i Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Social informatics uses a range of methods including workplace ethnography (Bodker et al., 2004; Simonsen & Kensing, 1997), focus groups and interviews, and user participation in design (Greenbaum & Kyng, 1991; Schuler & Namioka, 1993). Our methodological approach is augmented by methods of analysis of work activity developed at the Centre interdisciplinaire d'études sur la biologie, la santé, la société et l'environnement (CINBIOSE), which combines quantitative and qualitative data collection methods. This approach examines work activity in terms of balancing job demands and constraints workers face, with a worker’s capacities, health and well being (Messing, 1999; Messing & Seifert, 2001), and is sometimes referred to as Francophone ergonomics (Balka, Mason, & Elfring, 2005; Messing, Seifert, Vézina, Balka, & Chatingy, 2005).

Approaches to KT and Evaluation of KT Efforts Reflecting the suggestion of Roos and Shapiro (1999), Lomas (1993), and Davis (2000), who point out that in-person interaction and early involvement of the stakeholders in the research process are elements of successful knowledge translation activities, the process of engaging end users has begun with the development of research questions undertaken in preparation of this proposal. Following the suggestions of Davis and Howden-Chapman (1996), Golden-Biddle et al. (2003), National Centre for the Dissemination of Disability Research (1996), we anticipate that collaboration will occur at all stages of the research process.

Our approach is one of a high degree of embededness in the organizations we work with, which allows us to develop a strong sense of the role and priorities of the decision maker (Rogers, 1983; Dobbins et al., 2002). As we become aware of research users needs (which are often context-driven by issues arising in the dynamic environments in which my colleagues work), we will develop audience-specific messages that are in line with the decision-making environments in which they work (Lavis et al., 2003), and this approach will support sustained dialog between researchers and decision makers as advocated by Elliott and Popay (2000). Our model will support a dissemination of research where discussion of findings as advocated by Stevens et al. (2005) can take place. Finally, our proximity to research users will aid us in insuring that the information we are in a position to deliver will be delivered when it is needed (Palmer & Fenner, 1999; Stevens et al., 2005). While we anticipate trying a range of evaluation methods to determine to what extent our knowledge translation efforts are successful, we will be designing and using a novel KT evaluation approach as well. Based on the notion of a 360 evaluation (a form of performance evaluation that is an alternative to traditional performance evaluation methods, 360 evaluations are multi-source and multi-rater (Davies & Archer, 2005), we will develop an evaluation tool which can be delivered through a web link that provides access to a web form, which will solicit information from those who are the targets of our KT efforts. A series of questions will guide those we have targeted in our KT efforts to provide information about different people’s responses to the same KT activities (as well as solicit suggestions for changes and improvements). As much of our dissemination will occur on a face to face basis, we will know who has participated and how to contact them. In situations where we do not have access to e-mail addresses for our target population, we can distribute hard copy of blank surveys at the conclusion of a KT event. This approach will focus on the person who is the source of the information being transferred, the means through which it is being transferred, and how perceptions differ among and between various stakeholder groups.

Overview of Methods and Research Activities Objective 1: Demonstration of enhanced use of data from electronic health records in support of

12- j Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal continuity of care in clinical settings and practice research. Build capacity within Mid-Main to increase use of electronic record data to support practice improvement initiatives. Activities / Methods: RA to observe how staff are using EMR, with staff develop plans for completing advanced practice searches, implement those searches, explore data for anomalies signalling coding problems, develop plans for addressing coding problems if required; aid staff in developing practice search skills, conduct ethnographic observations of staff as they conduct searches, log problems and issues, observe and log how data are being used in clinical practice, communicate challenges related to Wolf EMR to Brendan Byrne; communicate problems associated with CDM Toolkit to Elaine Grenon; conduct qualitative data analysis of field data collected during observations of Mid-Main staff using EMR; analyse with NVivo qualitative data analysis software using constant comparison method. Develop good practice guides and vignettes that provide guidance to other practices and serve as an example of how data are being used. Indicators: Documentable changes in practice, # of advanced practice searches identified by doctors and # of searches conducted in general and conducted by doctors in particular; # searches integrated into everyday practice at end of project; evidence of dissemination of knowledge to Wolf and VCH (e.g., communications, meetings where specific exchange took place); completion of Good Practice Guides; data from 360 evaluation tool and paper versions of evaluation tool. Evaluation: Regular 360 style evaluations conducted of team members, specific activities, specific types of information. Objective 2a: Knowledge translation from research literature to local users (Mid-Main and VCH). Activities/ Methods: RAs to identify opportunities where research literature might come to bear on decisions under consideration by team partners; work with partners to determine what their literature review or knowledge synthesis needs are. Indicators: # of reviews or knowledge syntheses completed at request of each of the partners. Evaluation: 360 style evaluation of those documents. Objective 2b: KT from local users (Mid-Main) to decision makers (VCH) and developer, potential electronic record adapters, about challenges related to enhanced use of electronic records for management of patients with multiple chronic conditions. Activities: Varied, as necessitated by context, partner and other stakeholder needs. Indicators: RA and Co-investigator logs of dissemination activities in which information moved from one partner to another; concrete evidence of the uptake of that knowledge (e.g., change in software; documented change in how subsequent pilot implementations are carried out, evidence in related documents of spread of ideas, etc.). Evaluation: 360 style evaluations of events and artefacts (documents etc.); textual analyses of documents authored by project partners or used by related stakeholders that may reflect ideas generated through this KT project. Objective 3: Translation of primary research results into good practice guidelines concerning use of the CDM Toolkit and IT resources being piloted by VCH. Activities: Collection of ethnographic data through varied field techniques including observation, interviews; subsequent coding and analysis of those data using constant comparison methods; development of varied forms of output that may serve as basis for subsequent development of Good Practice Guides. Indicators: Existence of “intermediate” documents (e.g., academic papers, reports, high level overviews) from which Good Practice Guide content can be developed; # and variety of dissemination documents produced; # and variety of Good Practice Guides produced. Evaluation: 360 style evaluation of Good Practice Guides.

12- k Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Appendices

List of Tables and Figures

Table 1 Domains and Phases of Information System Evaluation in Health p. 12-m Care Figure 1 Field Based Assessment Activities in Relation to Traditional Health p. 12-n Technology Assessment Activities and the Technology Development Lifecycle

12- l Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Table 1: Domains and Phases of Information System Evaluation in Health Care

Pre-implementation Implementation Post-implementation Technical domain: x Compatibility with other systems? x Possible to tailor information to x How did the system perform on all x Upgradeable? specific needs of professionals? selected features? x Maintenance? x Downtime (frequency, duration)? x Were there unexpected problems x Data consistency? x Upgradeable? and were they solvable? x Speed? x Adaptability to changing requirements?

Professional domain: x What are the professionals’ needs? x Is it easy to use? x ‘Final’ impacts on x How much time does it take to learn the x What are the benefits compared content/effectiveness of work system/work with it? to the old situation? (changing tasks, responsibilities, x What are professionals’ interests to x (How) does it affect the routines, less errors, time saving, work with the system? content/effectiveness of work? less ‘lost’ records)? x User-friendliness? x Does it improve patient outcome x Improved data quality? x Content of information: understandable (compliance, morbidity, x Impact on patient outcomes? and complete? mortality)? x Overall satisfaction? x Overall use? Organizational domain: x Is the organization ready? x Does the organization have to x Impacts on work processes and x Are the different stakeholders ready? make adjustments (procedures, organization as a while x Is the objective of implementation strategy, decision-making, etc.)? (communication patterns, clear? x Are there unexpected negative responsibilities, decision-making x Does the investment ‘fit’ with other effects? procedures, interactions organizational strategies? within/between professional x What kind of preparations/ adjustments groups)? have to be made in advance? x Impacts on waiting lists, provided services, organizational strategy? x Impacts on patient satisfaction? Economic domain: x Expected costs of buying, training, x Unexpected costs (maintenance, x ‘Final’ costs? maintenance, etc.? upgrading, training)? x ‘Final’ benefits? x Expected benefits (return of investment)? x Are risks being managed? x Reliable vendor?

Ethical domain: x Data access, data security? x How are patient data being used x ‘Overall’ effects of use of x Accountability for use of patient data? (by whom, for what purposes)? electronic patient data (e.g. x Possible effects of use of electronic x Who is responsible for use of decision-making, autonomy, doctor patient data? patient data? – patient communication, etc.)? x How are patients involved in the implementation?

Legal domain: x Expected registration (quality, presence) x What role do electronic patient x Consequences of use of electronic improvements/ benefits? data play in legal matters (e.g. patient data? legal status)? x (Potential) legal (im)possibilities? Source: Stoop and Berg (2003) Cited in Berg, M. (2003) Health Information Management. Oxford UK: Routledge.

12-m Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Figure 1: Field Based Assessment Activities in Relation to Traditional Health Technology Assessment Activities and the Technology Development Lifecycle

Figure 1 situates traditional health technology assessment activities and field-based activities (the assessment of technologies in context) in relation to the technology development and use lifecycle.

The focus of proposed research is on the Assessment of Technology in Context.

12- n Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal References

American Medical Record Association. (1984). Confidentiality and security of secondary health records. Journal of the American Medical Record Association, 55(4), 34-37. Balka, E. & Kahnamoui, N. (2004). Technology trouble? Talk to us! Findings from an ethnographic field study. Proc. Participatory Design Conference 2004, Artful integration: interweaving media, materials and practices (Vol. 1) (pp. 224-234). New York: ACM Press. Balka, E., Mason, S., & Elfring, N. (2005).“You think it is turning but it is the multiple small stuff” Gender, the division of labour and back and shoulder injury among nursing staff. Canadian Woman Studies, 24, 145-152. Balka, E. & Wagner, I. (2006). Making things work: Dimensions of configurability as appropriation work. Computer Supported Cooperative Work Conference 2006 (CSCW ’06). November 4-8, 2006. (229-238). Banff, Alberta. Balka, E., Wagner, I., & Jensen, C. B. (2005). Reconfiguring critical computing in an era of configurability. In O. W. Bertelsen, N. O. Bouvin, P. G. Krough, & M. Kyng (Eds.). Proc. The Forth Decennial Aarhus Conference, Critical Computing- Between Sense and Sensibility (pp. 79-88). New York: ACM Press. Berg, M. (2003). Health Information Management. Oxford, UK: Routledge Bodker, K., Kensing, F., & Simonson, J. (2004). Participatory IT Design : Designing for Business and Workplace Realities. Cambridge, MA: MIT Press. Broemeling, A., M., Watson, D., E., Black, C., & Reid, R., J. (2006). Measuring the performance of primary health care: Existing capacity and future information needs. Retrieved May 30, 2007, from the Centre for Health Services and Policy Research Web site: http://www.chspr.ubc.ca/files/publications/2006/chspr_06-21R.pdf Canadian Institutes for Health Research. (2005, October 10). About knowledge translation. Retrieved May 17, 2007, from Canadian Institutes for Health Research Web site: http://www.cihr- irsc.gc.ca/e/29418.html CCOHTA, (2003, February 19). CCOHTA News Flash. Retrieved July 1, 2003 from http://www.ccohta.ca/up_to_minute/fed_budget2003_pressrelease_e.asp Centre for Health Services and Policy Research. (n.d.). Measuring the Performance of Primary Health Care Key Findings. Retrieved May 30, 2007, from the Centre for Health Services and Policy Research Web site: http://www.chspr.ubc.ca/research/phc/measuring Coates, J. F. (1974). Some methods and techniques for comprehensive impact assessments. Conference on Preparation of EIS, Henniker, NH. Davies, H., & Archer, J. (2005). Multisource feedback: Development and practical aspects. The Clinical Teacher, 2, 77-81. Davis, D. (2000). Clinical practice guidelines and the translation of knowledge: The science of continuing medical education. Canadian Medical Association Journal, 163, 1278-1279. Davis, P., Howden-Chapman, P. (1996). Translating research findings into health policy. Social Science & Medicine, 43, 865–872. Delpierre, C., Cuzin, L., Fillaux, J., Alvarez, M., Massip, P., & Lang, T. (2004). A systematic review of computer-based patient record systems and quality of care: More randomized clinical trials or a broader approach? International Journal for Quality in Health Care, 16, 407-416. Dirnfeld, V. (2004, November). Loss of “full service family practitioners” from hospital staffs in the Authority. Medlink: The VCH Physician Newsletter, 2, 3. Retrieved May 30, 2007, from http://www.vch.ca/medlink/pdfs/0205_medlink_nov04.pdf Dobbins, M., Ciliska, D., Cockerill, R., Barnsley, J., & Di, C. A. (2002). A framework for the

12- o Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal issemination and utilization of research for health-care policy and practice. Online Journal of Knowledge Synthesis for Nursing, 9(7). Eccles, M., Steen, N., Grimshaw, J., Thomas, L., McNamee, P., Soutter, J., et al. (2001). Effect of audit and feedback, and reminder messages on primary-care radiology referrals: A randomised trial, Lancet, 357, 1406–1409. Elliott, H., Popay, J. (2000). How are policy makers using evidence? Models of research utilisation and local NHS policy making. Journal of Epidemiology and Community Health, 54, 461–468. Feyerman, P. (2007, May 29). Victoria spends big to attract family MDs: $442 million allocated to make field more appealing. Vancouver Sun, pp. A1-A2. Freeman, G. (1984). Continuity of care in general practice: A review and critique. Family Practice, 1, 245-252. Freeman, G., Shepperd, S., Robinson, I., Ehrich, K., Richards, S. (2000). Continuity of care: Report of a scoping exercise for the National Co-ordinating Centre for NSH Service Delivery and Organisation R & D (NCCSDO). Retrieved May 30, 2007, from the Service Delivery and Organisation Web site: http://www.sdo.lshtm.ac.uk/files/project/2-final-report.pdf Golden-Biddle, K., Reay, T., Witt, C., Petz, S., Casebeer, A., Pablo, A., et al. (2003). Towards a communicative perspective of collaborating in research: The case of the researcher-decision- maker partnership. Journal of Health Services Research & Policy, 8(S2), 20–25. Greenbaum, J., & Kyng, M. (Eds.). (1991). Design at work: Cooperative design of computer systems. New Jersey: Lawrence Erlbaum. Haggerty, J. L., Reid, R. J., Freeman, G. K., Starfield, B. H., Adair, C. E., McKendry, R. (2003). Continuity of care: A multidisciplinary review. British Medical Journal, 327, 1219-1221. Hart, E., & Bond, M. (1995). Action Research for Health and Social Care: A Guide to Practice. Buckingham, UK: Open University Press. Holter, I. M., & Schwartz-Barcott, D. (1993). Action research: What is it? How has it been used and how can it be used in nursing? Journal of Advanced Nursing, 128, 298-304. Jacobson, N., Butterill, D., & Goering, P. (2003). Development of a framework for knowledge translation: Understanding user context. Journal of Health Research and Policy, 8(2), 94-99. Kling, R. (2000). Learning about information technologies and social change: The contribution of social informatics. Information Society, 16, 217-233. Korpela, M., Mursu, A., Abimbola Soriyan, H., & Eerola, A. (2002). Information systems research and information systems practice in a network of activities. In Y. Dittrich, C. Floyd, & R. Klischewski (Eds.), Social Thinking – Software Practice (pp. 287-308). Cambridge, MA: MIT Press. Lavis, J., Ross, S., McLeod, C., & Gildner, A. (2003). Measuring the impact of health research. Journal of Health Services Research & Policy, 8,165–170. Lehoux, P. (2006). The Problem of Health Technology: Policy Implications for Modern Health Care Systems. Oxford, UK: Routledge. Lomas, J. (1993). Diffusion, dissemination, and implementation: Who should do what? Annals of the New York Academy of Sciences, 703, 226-35. Messing, K. (1999). “An approach to action-oriented research.” Integrating Gender in Ergonomic Analysis: Strategies for Transforming Women’s Work. Brussels, Belgium: European Trade Union Technical Bureau for Health and Safety, 192. Messing, K., & Seifert, A. M. (2001). Listening to women: Action-oriented research in ergonomics. Arbete och Hälsa, 17, 93-104. Messing, K., Seifert, A. M., Vézina, N., Balka, E., & Chatigny, C. (2005). Qualitative research using numbers: Analysis developed in France and used to transform work in North America. New

12- p Ellen Balka, Ph.D, PI. Knowledge to Action: Supporting Continuity of Care and Practice Research through Secondary use of EMR data. Amount Requested, Year 1: $100,000 Research Proposal Solutions, 15, 245-260. Morgall, J. M. (1993). Technology assessment: A feminist perspective. Philadelphia, PA: Temple University Press. Palmer, C., & Fenner, J. (1999). Getting the message across. London: Gaskell. Proctor, R., Rouncefield, M., Balka, E., & Berg, M. (2006). Special issue: CSCW and dependable healthcare systems. Computer Supported Cooperative Work, 15, 413-418. National Centre for the Dissemination of Disability Research. (1996). Review of the Literature on Dissemination and Knowledge Utilization. Bethesda, MD: National Institute on Disability and Rehabilitation Research. Reid, R. J., Barer, B., McKendry, R., McGrail, K., Prosser, B., Green, B, et al. (July 2003). Patient- focused care over time: Issues related to measurement, prevalence, and strategies for improvement among patient populations. Retrieved May 30, 2007, from the Canadian Health Services Research Foundation Web site: http://www.chsrf.ca/final_research/ogc/pdf/barer_final.pdf Rogers, E. M. (1983). Diffusion of Innovations (3rd ed.). London: Collier McMillan. Roos, N. L., Shapiro, E. (1999). From research to policy: What have we learned? Medical Care, 37(Suppl 6), JS291–JS305. Schuler, D., & Namioka, A. (1993). Participatory design: Principles and practices. Hillsdale, NJ: Lawrence Erlbaum. Simonsen, J., & Kensing, F. (1997, July). Using ethnography in contextual design. Communications of the ACM, 40 (7), 82-88. Shonkoff, J. (2000). Science, policy and practice: Three cultures in search of a shared mission. Child Development, 71, 181-187. Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: Design and access for large information spaces, Information Systems Research, 7, 111-134. Stevens, M., Liabo, K., Frost, S., & Roberts, H. (2005). Using research in practice. A research information service for social care practitioners. Child Family Social Work, 10, 65–75. Vancouver Coastal Health. (n.d.). Primary Health Care Network: PHCTF Health Outcomes and the Electronic Medical Record. Retrieved May 30, 2007, from Vancouver Coastal Health Web site: http://www.vch.ca/professionals/docs/phcn/phctf_health_outcomes_emr.pdf Vancouver Coastal Health. (2004, April 30). Health service redesign plan 2004/05 to 2006/07. Retrieved August 20, 2005, from the Vancouver Coastal Health Website: http://www.vch.ca/about/docs/budget/service_redesign_plan.pdf Vancouver Coastal Health. (2005, May 19). Health service redesign plan 2005/06 to 2007/08. Retrieved from the Vancouver Coastal Health Web site: http://vch.ca/accountability/docs/2005- 2006_health_services_redesign.pdf Westrup, C. (2002). On retrieving skilled practices: The contribution of ethnography to software development. In Y. Dittrich, C. Floyd, & R. Klischewski (Eds.), Social thinking – software practice (pp. 91-110). Cambridge, MA: MIT Press.

12- q