Development of a Public Health Nursing Data Infrastructure Karen A

Public Health Nursing Vol. 29 No. 4, pp. 343–351 0737-1209/© 2012 Wiley Periodicals, Inc. doi: 10.1111/j.1525-1446.2011.00993.x SPECIAL FEATURES:METHODS Development of a Public Health Nursing Data Infrastructure Karen A. Monsen, Ph.D., R.N.,1 Betty Bekemeier, Ph.D., M.P.H., R.N.,2 Robin P. Newhouse, Ph.D., R.N.,3 and F. Douglas Scutchfield, M.D.4 1School of Nursing, University Of Minnesota, Minneapolis, Minnesota; 2School of Nursing, University of Washington, Seattle, Washington; 3School of Nursing, University of Maryland, Baltimore, Maryland; and 4Peter P Bosomworth Professor of Health Services Research and Policy, College of Public Health and Medicine, Lexington, Kentucky

Correspondence to: Karen A. Monsen, School of Nursing, University Of Minnesota, 5-140 Weaver-Densford Hall, 308 Harvard Street SE, Minneapolis, MN 55455. E-mail: [email protected]

ABSTRACT An invited group of national public health nursing (PHN) scholars, practitioners, policymakers, and other stakeholders met in October 2010 identifying a critical need for a national PHN data infrastructure to support PHN research. This article summarizes the strengths, limitations, and gaps specific to PHN data and proposes a research agenda for development of a PHN data infrastructure. Future implications are suggested, such as issues related to the development of the proposed PHN data infrastructure and future research possibilities enabled by the infrastructure. Such a data infrastructure has potential to improve accountability and measurement, to demonstrate the value of PHN services, and to improve population health.

Key words: data infrastructure, nursing workforce, Omaha System, public health nursing standards, public health systems, quantitative research.

The Institute of Medicine (IOM) report For the and policy development (Committee on Public Public’s Health: The Role of Measurement in Health Strategies to Improve Health, Institute of Action and Accountability (2011) emphasized the Medicine, 2011) . Public health nursing leaders are need for better data to enable evaluation of and ideally positioned to address these gaps in under- improvements to the public health system. The standing and improving PHN contributions to pop- report recommends that the Department of Health ulation health. and Human Services (DHHS) work with relevant stakeholders to address data infrastructure gaps A Public Health Nursing Research through a multifaceted approach with four key Agenda: Data Infrastructure components to: (1) facilitate the development of a coherent, efficient, and useful health information To advance the science of public health nursing system, (2) create a performance measurement sys- (PHN) practice, a national meeting of invited tem that promotes accountability among organiza- experts was held in October 2010 with grant sup- tions having responsibilities for protecting and port from the Agency for Healthcare Research and improving population health at local, state, and Quality (AHRQ). The outcome of this meeting was national levels, (3) construct data collection mecha- a PHN research agenda (Issel, Bekemeier & Kneipp, nisms at local, state, and national levels to ensure 2012). A key priority identified was the need to accountability, and (4) encourage key public health develop relevant PHN databases for evaluation of organizations to adopt and use these data for per- clinical practice and research. Addressing the formance reporting, quality improvement, planning, urgent need for a PHN data infrastructure will

343 344 Public Health Nursing Volume 29 Number 4 July/August 2012 enable public health leaders and policymakers to TABLE 1. Research Questions to Guide the Development of advance and to maximize PHN practice, and to a PHN Data Infrastructure meet national population health goals. What are the minimum data elements related to public a The meeting participants agreed that the PHN health nursing practice? a data infrastructure gap is consistent with the gap Which data elements ought to be in a minimum dataset? What are the core variables essential to measure public identified by the IOM for the broader public health a health nursing practice outcomes? system (Committee on Public Health Strategies to What are public health nursing sensitive outcome indicators a Improve Health, Institute of Medicine, 2011). at: 1) individual, 2) family, and 3) community levels? Currently, there is a great deal of variation in data What are sensitive indicators of public health collection and reporting of PHN data across public nursing–community partnerships (e.g., # of stakeholders, health departments and other organizations, regis- cohesiveness, appropriate representation) tries, and surveys. Adequate data sources are not What is the minimum common unit on which data ought to be gathered? consistently available at either the state or national Which metrics (related to public health nursing processes a level to assess programs and services, levels of ser- and outcomes) are reliable and valid and at what level(s)? vice (individual, family, community, and system), Which public health nursing metrics are similar across population health data, penetration of service deliv- settings and can be used across settings? ery within target populations, the PHN workforce, What predicts adoption of public health nursing sensitive/ relevant metrics by an organization? PHN-specific interventions, or PHN-sensitive out- How can we develop a database of databases with a comes (Scutchfield, Knight, Kelly, Bhandari & corresponding decision tree for selecting key variables Vasilescu, 2004; Mays et al., 2006; Bhandari, related to public health nursing practice and outcomes? Scutchfield, Charnigo, Riddell & Mays, 2010; Com- How can current databases (e.g., Omaha System) be used to improve public health nursing safety and quality of mittee on Public Health Strategies to Improve a Health, Institute of Medicine, 2011). Due to these practice? What gaps exist in outcomes data currently being collected deficits, it is difficult to fully identify and describe by/about public health nursing? the PHN workforce or to attribute the contribution What are limitations (gaps) of metrics needed for decision of PHN interventions to population health out- making? comes. a Using a modified Delphi process, the meeting Questions deemed priority by the participants in the Octo- ber 2010 agenda setting conference. participants generated 13 suggested research questions that were specific to guiding development of State of the Science of Public Health a proposed PHN data infrastructure (Table 1). Nursing Data (Kenney, Hasson & McKenna, 2011). The questions were validated and revised 1 month later at In scientific and health care disciplines, vast quan- a special session held at the 2010 Annual Meet- tities of data are becoming available due to ing of the American Public Health Association. advances in technology (Hey, Tansley & Tolle, Based on the 13 questions, four key research top- 2009). Multiple sources of PHN data are available ics were identified as priorities: (1) determining to support research examining the PHN workforce, the data elements in a minimum dataset that practice, and outcomes (Magee, Lee, Giuliano & capture practice and outcomes, (2) outcome indi- Munro, 2006; Zeni & Kogan, 2007). Each data cators that are PHN-sensitive at multiple levels, source contributes an important perspective to the (3) issues of validity and reliability of the essen- broader field of public health systems and services tial PHN data elements, and (4) use of existing research (PHSSR) and to PHN research specifically. databases to improve quality and safety of PHN Similarly, each data source has limitations that practice (Issel et al., in press). This article sum- must be addressed. Table 2 describes major PHN marizes the strengths, limitations, and gaps spe- data sources presented by PHN data experts at the cific to PHN data; proposes a research agenda for invited conference, including variables, and limita- the development of a PHN data infrastructure; tions for selected resources files, national and state and provides recommendations for future research surveys, registries, and clinical data. that may be enabled by the proposed PHN data The Health Resources and Services Administra- infrastructure. tion (HRSA) Bureau of Health Professions Area Monsen et al.: Development of a PHN Data Infrastructure 345

TABLE 2. Selected PHN Data Elements in Selected National Surveys

Data source Description Variables Limitations Compiled resources Area resource County-specific health resource Health professions, Race/ethnicity varies file (ARF) information system designed health facilities, depending on the data to be used by planners, hospitals, vital set. Reporting periods policymakers, researchers, and statistics, population are based on the other professionals interested and economics, availability of each in the nation’s health care utilization, data element delivery system. May be expenditures, and purchased from Quality health professions Resource Systems, Inc. (HRSA, training with 2011) geographic codes and descriptors which enable it to be linked to other files Registries Vaccine safety A collaborative effort between Population health Limited to defined datalink the CDC Immunization Safety outcomes, such as geographic areas and Office and eight managed care influenza outbreaks participating organizations to monitor organizations immunization safety, rare and serious events following immunization. (Centers for Disease Control & Prevention, 2011c) Special child A confidential record of infants Birth defects, chronic Specific to New Jersey, health services and children who have birth health conditions difficult to verify registry (New defects and special health care meaning of variables, Jersey) needs (mandated conditions) difficult to link by or who are at risk for time and location to developing such needs (State variables from other of New Jersey Department of databases Health & Human Services, 2011) National health system surveys National profile Longest running governmental Organizational Limited to the of local health public health survey with high characteristics, governmental public departments participation rate (82% in programmatic health workforce and (NACCHO) 2010) among local health activities, workforce provide little detail departments Harmonized with data (size and regarding the nature ASTHO State Public Health discipline), leadership of the workforce aside Survey since 2010 (National data (discipline and from attributes of the Association of City & County education) lead executives and Health Officials, 2011) proportions of staff in State public State level public health Public health workforce various job health survey profiles. Harmonized with and leadership classifications (ASTHO) NACCHO Profile since 2010 (Association of State & Territorial Health Officials, 2010) 346 Public Health Nursing Volume 29 Number 4 July/August 2012

TABLE 2. (Continued)

Data source Description Variables Limitations State health workforce surveys North Carolina Initially built from statewide Workforce Variation in data public health learning management systems characteristics elements across states workforce to identify areas of training (discipline and development need, but providing individual education) and local system (WDS) level health department staff health department data regarding education, performance experience, certification. (Hajat et al., 2009) National population health surveys National health The National Health Interview PHN-sensitive Difficult to verify interview Survey (NHIS) has monitored outcomes regarding meaning of variables, survey the health of the nation since health care access and difficult to link by 1957. NHIS data on a broad health status time and location to range of health topics are variables from other collected through personal databases household interviews, providing data to track health status, health care access, and progress toward achieving national health objectives. (Centers for Disease Control & Prevention, 2011a) National The NIS is a list-assisted PHN-sensitive Difficult to verify immunization random-digit-dialing telephone outcomes regarding meaning of variables, survey survey followed by a mailed immunization difficult to link by survey to children’s coverage time and location to immunization providers to variables from other monitor childhood databases immunization coverage since April 1994. (Centers for Disease Control & Prevention, 2011b) National survey National Survey of Children’s PHN-sensitive Difficult to verify of children’s Health was conducted by the outcomes regarding meaning of variables, health Centers for Disease Control children’s health difficult to link by and Prevention’s National status time and location to Center for Health Statistics in variables from other 2003 and 2007. (Data databases Resource Center, 2011) Clinical data PHN Interface Terminology (e.g., Interface terminology Health problems, PHN documentation Omaha System, (Martin, standards commonly interventions, and data 2005)) used in PHN clinical outcomes of PHN documentation services reliability and observer bias require attention to data quality, variation in number of users by state Administrative Client and provider details Provider ID, client Variables may not be data demographics and standardized address, claims data Monsen et al.: Development of a PHN Data Infrastructure 347

Resource File (ARF) is a national county level nurse leadership in local health departments and health resource information database. Data on the sets of services an agency provides (Bekemeier health professions includes nurses, but does not & Jones, 2009) and relationships between nurse- include PHN-specific data. Other data are available led health departments and health department per- regarding health care facilities and population, eco- formance (Bhandari et al., 2010) and clinician nomic, and environment factors (U.S. Department (nurse and physician) local public health leaders of Health & Human Services, Health Resources & and county-level mortality disparities (Bekemeier, Services Administration, Bureau of Health Profes- Grembowski, Yang & Herting, 2011). sions, 2011). Similarly, national and state registries State level data sources may include variables exist for numerous health conditions, and thus may regarding outcomes of PHN interventions. For exam- provide important contextual variables for PHN ple, Hajat et al. (2009) used North Carolina Public workforce and outcome studies. Although no Health Workforce Development System (WDS) data current PHN studies using the ARF or registries depicting county totals for “decline in teen pregnancy were found in the literature, these variables have over time” as one measure of a population-level out- potential to be used with PHN workforce data to come that was positively associated with services examine correlations between PHN capacity and conducted by PHNs and other “health professionals” important health conditions, such as obesity, chil- in the health departments under study. Issel, Be- dren with special health care needs, and immuniza- kemeier, and Baldwin (2011) identified a set of three tion rates (Centers for Disease Control & indicators that were sensitive to PHN interventions Prevention, 2011c; State of New Jersey Department across multiple health departments and two states. of Health & Senior Services, Family Health Ser- The indicators endorsed were rates of Chlamydia, vices, 2011; U.S. Department of Health & Human first trimester prenatal care, and early childhood Services, Health Resources & Services Administra- immunization. Data sources included birth certifi- tion, Bureau of Health Professions, 2011) cates, the Illinois statewide patient record system, National health system surveys include vari- and local health department WIC data in Illinois; ables indicating services provided by health depart- and birth certificates, STD/TB service, and immuni- ments, and number of staffs in job classifications, zation registry data in Washington State. including public health nurses (PHNs). However, Other PHN-related variables may potentially be they do not include the numbers, types, or qualifi- found in national population health survey databas- cations of staff providing a specific service. In addi- es, such as the National Health Interview Survey tion, staff numbers are reported based on their (Centers for Disease Control & Prevention, 2011a), workforce functions (not on their credentials); thus, the National Immunization Survey (Centers for PHNs serving in positions not classified, as a nurs- Disease Control & Prevention, 2011b), and the ing position would not be identifiable as nurses National Survey of Children’s Health (Data Resource (Association of State & Territorial Health Officials, Center, 2011). However, no studies using these 2010; National Association of City & County Health sources were found in the literature. Officials, 2011). In terms of public health agency Clinical databases (i.e., administrative, billing, leadership, the National Association of City and and interface terminology documentation data) County Health Official’s (NACCHO) National Pro- have potential for studying the quality of popula- file of Local Health Departments Surveys (Profile) tion-focused PHN interventions (Monsen et al., includes data regarding the licensure and education 2006). Administrative variables include PHN and of health department “top executives,” providing a agency identifiers. Interface terminology variables longitudinal record of PHN leadership in local pub- describe population health status and related popu- lic health systems (National Association of City & lation-based PHN interventions at individual, fam- County Health Officials, 2011). Outside of govern- ily, and community levels. The Omaha System is an mental agencies, PHNs provide services through interface terminology that is commonly used in organizations, such as nonprofit community-based PHN clinical documentation software (Farri, Mon- agencies, hospital systems, and health policy sen, Westra & Melton, 2011; Martin, 2005; Melton groups. Studies using NACCHO Profile data for et al., 2010). Examples of PHN research using clini- PHSSR include examining relationships between cal and administrative data include: a reliability 348 Public Health Nursing Volume 29 Number 4 July/August 2012 study (Monsen et al., in pressa); the examination of and standards (Monsen & Martin, 2002a,b; Monsen home visiting intervention effectiveness for high et al., 2006, in pressa; Minnesota Omaha System risk mothers (Monsen, Banerjee & Das, 2010a; Users Group, 2011). Linking variables from multiple Monsen, Radosevich, Kerr & Fulkerson, 2011b); databases depend on many factors, such as identify- intervention effectiveness for mothers with intellec- ing PHN variables that are similar across settings. tual disabilities (Monsen, Sanders, Yu, Radosevich Linking databases often requires significant time to & Geppert, 2011c); benchmarking PHN outcomes obtain, merge, and extract the data. Collaboration is (Monsen, Radosevich, Johnson, Farri, Kerr, & Gep- required to promote data sharing among these pert, 2011; Monsen et al., 2010b, 2011c); the devel- diverse data sources to build the data infrastructure, opment of a maternal risk index for PHN caseload as data may be owned by individual investigators, management (Monsen et al., 2011b); and the devel- government sponsored databases or public–private opment of a new interim improvement metric for partnerships that facilitate linkage of data owned by family home visiting (problem stabilization) (Mon- private organizations. These challenges can be mini- sen, Farri, McNaughton & Savik, 2011a). Commu- mized by careful selection of the sample and vari- nity level interventions were compared to ables, and subsequent management of the data individual level interventions in a study of PHN (Bradley, Penberthy, Devers & Holden, 2010). nurse manager interventions (Monsen & Newsom, 2011). Some states (e.g., Minnesota, Washington, Implications and Recommendations Arizona, Maine) have implemented PHN program evaluation protocols using interface terminology The PHN researchers should prioritize research that data (Elsbernd, Barnhart, Stock, Monsen & Prock, identifies minimum data elements relevant to PHN 2011). An Omaha System data warehouse of PHN practice, the core variables essential in measuring assessments, interventions, and outcomes exists PHN outcomes at various levels, the reliability and within the University of Minnesota School of Nurs- validity of PHN processes and outcomes, and how ing Center for Nursing Informatics. The purpose of existing databases can be used, expanded, combined, this data warehouse is to advance PHN outcomes or refined to meet these identified goals. The critical research (Omaha System Partnership, 2011). need for a uniform minimum dataset of PHN interventions, outcomes, and health system data drives Limitations of Public Health Nursing this research agenda. The PHN researchers bring Data Sources unique practical, conceptual, methodological, statis- tical, and informatics expertise that is requisite for Public health nursing data exist in diverse forms and the development of a PHN data infrastructure. The typically must be extracted, transformed, harmo- PHN researchers also bring expertise in public health nized, merged, and cleaned before analysis can begin systems to public health workforce interventions, (Table 2). Availability of PHN data from resource finance, management, and administration studies. files, surveys, registries, and clinical health records To conduct such research, further resources and varies by state, agency, and program. Thus, there are antecedents that support the broader field of PHSSR many challenges that must be overcome to build a are needed. These resources include funding to sup- database that comprises all necessary variables to port the research effort and attention to the congru- address a particular research question or to evaluate ence between the PHSSR agenda and the PHN program effectiveness. All secondary data sources research agenda (Bhandari et al., 2010; Mays et al., have limitations, such as missing data and inherent 2006; Scutchfield et al., 2004). errors; thus data cleaning is critical. Observer bias Public health practice-based research networks issues must be identified and addressed as a limita- (PBRN), formally established and funded by The tion, and alternative explanations for findings must Robert Wood Johnson Foundation (RWJF) in 12 be considered. Clinical data have limitations common states across the United States offer further oppor- to all observational datasets, such as fidelity to docu- tunities for identifying reliable and valid outcome mentation procedures and observer bias. Omaha Sys- measures and additional existing and new data tem users have addressed these limitations through sources (University of Kentucky College of Public training and shared inter-rater reliability materials Health, 2011). In addition, the Omaha System Monsen et al.: Development of a PHN Data Infrastructure 349

Partnership for Knowledge Discovery and Health ● What should the database infrastructure look like? Care Quality is an international partnership in ● What kind of electronic platform is recom- which PHN scholars and practitioners are actively mended? engaged in advancing PHN research using PHN- ● What exact data should be in this uniform data- generated clinical datasets (Martin, Monsen & set? Bowles, 2011). The Omaha System Partnership has ● How could/should local/regional/state data be scientific teams from the University of Minnesota collected and fed into such a database? and the University of Pennsylvania working with ● How frequently should this database be updated? affiliate members from practice settings in many ● How could this database be accessed? countries. These investigators have initiated over ● Who would be given access rights? 30 investigations of important clinical questions, It is incumbent upon the PHN community of with 16 scientific articles at the time of this publica- scholars, practitioners, and policy makers to tion. Capitalizing upon these partnerships between address these issues to advance the PHN data infra- practitioners and researchers will strengthen the structure agenda. rigor and advance the utility of research that con- The Institute of Medicine (2011) report on nects PHN practice and specific health outcomes, Accountability and Measurement in Public Health improving PHN accountability and measurement. identified a critical need for better data to enable A comprehensive uniform PHN dataset is pro- evaluation and improvement of the public health posed using combined elements of PHSSR data system. In alignment with this report, an invited (health services variables), population health data group of national PHN scholars, practitioners, and (surveys and registries), and clinical PHN documen- stakeholders used a Delphi process to identify gaps tation data (intervention and outcome variables for in PHN data. Public health nursing data exist in services at the individual, family, community, and multiple forms and settings, with major gaps, barri- systems levels). Development of this dataset would ers, and limitations to their use in research and enable meaningful investigation of a broad range of evaluation. Addressing these gaps will enable PHN practice, quality, evaluation, and safety questions. researchers and practitioners to advance toward a Several examples of possible studies enabled by the more comprehensive uniform PHN dataset, com- PHN data infrastructure are suggested below for bining elements of the PHSSR data and clinical future consideration by PHN researchers: PHN documentation data. Such a data infrastruc- ● Development of standardized metrics for PHN ture will improve accountability and measurement, performance. demonstrate the value of PHN services, and ● Evaluation and monitoring of population health improve population health. status across jurisdictions. ● Examination of links between public health sys- Acknowledgments tem factors, PHN interventions, environmental and socioeconomic factors, and client outcomes. Funded by AHRQ Conference Grant: 1 R13 HS01 ● Effectiveness of PHN services and programs at 8852-01. individual, family, community, and systems levels. ● Ecological models incorporating contextual health References systems data to improve understanding of PHN interventions and outcomes within the larger Association of State and Territorial Health Officials. public health and health care systems. (2010). 2010 ASTHO State and Territorial Public Health Survey. Retrieved from http:// Key questions to consider as the PHN data www.astho.org/Display/AssetDisplay.aspx? infrastructure is developed are id=4684. Bhandari, M. W., Scutchfield, F. D., Charnigo, R., ● Who or what agency might fund/finance such a Riddell, M. C., & Mays, G. P. (2010). New dataset? data, same story? Revisiting studies on the ● Who or what agency would/should maintain relationship of local public health system oversight? characteristics to public health performance. 350 Public Health Nursing Volume 29 Number 4 July/August 2012

Public Health Management Practice, 16, 110 North Carolina. Journal of Public Health –117. Management and Practice, 15(2), E22–E33. Bekemeier, B., Grembowski, D., Yang, Y., & Herting, Hey, T., Tansley, S., & Tolle, K. (2009). The fourth par- J. (2011). Leadership matters: Local health adigm: Data-intensive scientific discovery. department clinician leaders and their Redmond, Washington: Microsoft Research. relationship to reducing health disparities. Issel, M. L., Bekemeier, B., & Kneipp, S. (2012). A Maternal and Child Health Journal. E-pub public health nursing research agenda. Public ahead of print. doi: 10.1007/s10995-011-0794-9. Health Nursing. doi:10.1111/j.1525-1446.2011. Bekemeier, B., & Jones, M. (2009). Relationships 00989 between local public health agency functions Issel, M. L., Bekemeier, B., & Baldwin, K. (2011). and agency leadership and staffing: A look at Three population-patient care outcome indi- nurses. Journal of Public Health Management cators for public health nursing: Results of a and Practice, 16(2), e8–e16. consensus project. Public Health Nursing, 28 Bradley, C. J., Penberthy, L., Devers, K. J., & Holden, (1), 24–34. D. J. (2010). Health services research and data Kenney, S., Hasson, F., & McKenna, H. (2011). The linkages: Issues, methods, and directions for Delphi technique in nursing and health the future. Health Services Research, 10, 1468 research. London: Wiley-Blackwell. –1488. Magee, T., Lee, S. M., Giuliano, K. K., & Munro, B. Centers for Disease Control and Prevention. (2011a). (2006). Generating new knowledge from The national health interview survey. existing data: The use of large data sets for Retrieved from http://www.cdc.gov/nchs/ nursing research. Nursing Research, 55(Sup- nhis.htm pl. 2), S50–S56. Centers for Disease Control and Prevention. (2011b). Martin, K. S. (2005). The Omaha System: A key to The national immunization survey. Retrieved practice, documentation, and information from http://www.cdc.gov/nchs/nis.htm management, Reprinted (2nd ed.). Omaha, Centers for Disease Control and Prevention. (2011c). NE: Health Connections Press. Vaccine safety datalink. Retrieved from Martin, K. S., Monsen, K. A., & Bowles, K. H. (2011). http://www.cdc.gov/vaccinesafety/activities/ The Omaha System: Meaningful use examples vsd.html from practice, education, and research. Com- Committee on Public Health Strategies to Improve puters, Informatics, Nursing, 29(1), 52–58. Health, Institute of Medicine. (2011). For the Mays, G. P., McHugh, M. C., Shim, K., Perry, N., public’s health: The role of measurement in Lenaway, D., Halverson, P. K., et al. (2006). action and accountability. Washington, DC: Institutional and economic determinants of The National Academies Press. public health system performance. American Data Resource Center. (2011). National survey of Journal of Public Health, 96, 523–531. children’s health. Retrieved from http:// Melton, G. B., Westra, B. L., Raman, N., Monsen, www.nschdata.org/Content/Default.aspx K. A., Kerr, M. J., Hart, C. H., et al. (2010). Elsbernd, S., Barnhart, L., Stock, J., Monsen, K. A., Informing standard development and under- & Prock, C. (2011). Children with special standing user needs with Omaha System health care needs in Washington state: A signs and symptoms text entries in commu- population-based needs assessment and nity-based care settings. American Medical program evaluation. Poster presented at the Informatics Association Annual Symposium International Omaha System Conference. Proceedings, 2010, 512–516. Eagan, MN. Minnesota Omaha System Users Group. (2011). Farri, O., Monsen, K. A., Westra, B. L., & Melton, G. Omaha System care plans and practice (2011). Analysis of free text with Omaha System tools. Retrieved from http://omahasystemmn. targets in community-based care to inform org/KBS_care_plans.php practice and terminology development. Applied Monsen, K. A., Banerjee, A., & Das, P. (2010a). Dis- Clinical Informatics, 2(3), 304–316. doi: http:// covering client and intervention patterns in dx.doi.org/10.4338/ACI-2010-12-RA-0077. home visiting data. Western Journal of Hajat, A., Cilenti, D., Harrison, L.M., MacDonald, Nursing Research, 32(8), 1031–1054. P., Pavletic, D., Mays, G. P., et al. (2009). Monsen, K. A., Farri, O., McNaughton, D. M., & Savik, What Predicts Local Public Health Agency K. (2011a). Problem stabilization: A metric for Performance Improvement? A Pilot Study in improvement in high risk mothers. Applied Monsen et al.: Development of a PHN Data Infrastructure 351

Clinical Informatics, 2, 437–446. http://dx. (2010b). Comparing maternal child health doi.org/10.4338/ACI-2011-06-RA-0038. problems and outcomes across public health Monsen, K. A., Fitzsimmons, L. L., Lescenski, B. A., nursing agencies. Maternal and Child Health Lytton, A. B., Schwichtenberg, L. D., & Mar- Journal, 14, 412–421. tin, K. S. (2006). A public health nursing National Association of City and County Health informatics data-and-practice quality project. Officials. (2011). 2010 National profile of CIN: Computers, Informatics, Nursing, 24 local health departments study instrument. (3), 152–158. Retrieved from http://www.naccho.org/topics/ Monsen, K. A., Lytton, A. B., Ferrari, S., Halder, K., infrastructure/profile/upload/2010_Profile- Radosevich, D. M, Mitchell, S., et al. (in Consolidated-09-07-2010_website.pdf pressa). Assessment of outcome score reli- Omaha System Partnership. (2011). The Omaha ability in public health nurse documentation. System partnership for knowledge discovery On-line Journal of Nursing Informatics. and health care quality. Retrieved from Monsen, K. A., & Martin, K. S. (2002a). Developing http://omahasystempartnership.org an outcomes management program in a public Scutchfield, F. D., Knight, E. A., Kelly, A. V., Bhan- health department. Outcomes Management, 6, dari, M. W., & Vasilescu, I. P. (2004). Local 62–66. public health agency capacity and its rela- Monsen, K. A., & Martin, K. S. (2002b). Using an tionship to public health system perfor- outcomes management program in a public mance. J Public Health Manag Pract. 10, health department. Outcomes Management, 204–215. 6, 120–124. State of New Jersey Department of Health and Monsen, K. A., & Newsom, E. T. (2011). Feasibility Human Services. (2011). Special child health of usig the Omaha System to represent nurse services registry. Retrieved from http:// manager interventions. Public Health Nurs- www.nj.gov/health/fhs/sch/schr.shtml ing, 28(5), 421–428. State of New Jersey Department of Health and Senior Monsen, K. A., Radosevich, D. M., Johnson, S. C., , Services, Family Health Services. (2011). Spe- Farri, O., Kerr, M. J., & Geppert, J. S. (2011). cial Child Health Services Registry. Retrieved Benchmark attainment by maternal and child from http://www.state.nj.us/health/fhs/sch/ health clients across public health nursing schr.shtml agencies. Public Health Nursing. E-pub ahead U.S. Department of Health and Human Services, of print. doi: 10.1111/j.1525-1446.2011.00967.x Health Resources and Services Administra- Monsen, K. A., Radosevich, D. M., Kerr, M. J., & tion, Bureau of Health Professions. (2011). Fulkerson, J. A. (2011b). Public health nurses Area resource file. Retrieved from http:// tailor home visiting interventions. Public aspe.hhs.gov/datacncl/datadir/hrsa.htm Health Nursing, 28(2), 119–128. doi:10.1111/ University of Kentucky College of Public Health. j.1525-1446.2010.00911.x. (2011) Center for public health systems and Monsen, K. A., Sanders, A. N., Yu, F., Radosevich, services research: Practice based research D. M, & Geppert, J. S. (2011c). Family home networks. Retrieved from http://www.public- visiting outcomes for mothers with and with- healthsystems.org/pbrn out intellectual disabilities. Journal of Intel- Zeni, M. B., & Kogan, M. D. (2007). Existing popula- lectual Disabilities Research, 55(5), 484–499. tion-based health databases: Useful resources Monsen, K. A., Fulkerson, J. A., Lytton, A. B., Taft, for nursing research. Nursing Outlook, 55(1), L. L., Schwichtenberg, L. D., & Martin, K. S. 20–30.