Determination of Need Community Health Initiative Assessment Project

Ben Wood, Cathy O’Connor, Madeleine Biondolillo Office of Health Policy and Planning Goals for today’s discussion

We will present: • Summary of evaluation of DoN/CHI Program • Results from Survey*, contextualized with information from key informant interviews We will discuss: • Findings and Plans to move forward Recap of what’s been done

• Key Informant Interviews (stakeholders in MA plus best practice informants) • Document review • Literature review • Review of CHI database (historical trends) • CHI Stakeholder Survey Guiding Principles

• DPH is focused on the development of guidance and standards that aim to have measurable improvements in the public’s health. • Throughout this presentation findings from the assessment will be underlined where opportunity exists for the establishment of criteria to assist us in reaching this aim. Methods Overview: Database and Survey

• CHI Database used to obtain information for historical trend analyses and for setting parameters for the CHI Stakeholder Survey – Historical Trend Analysis • CHI’s from 2002-2013 – CHI Stakeholder Survey • CHI’s for the survey includes all PROJECTS which are ACTIVE or COMPLETE within the specified time frame we selected: July 2008- December 2013. INACTIVE PROJECTS are excluded. • Note: PROJECTS with CHIs have been approved since December 2013 but are not part of this study. • Total of 42 projects (pool of eligible respondents: 42 total hospital stakeholders, 63 total community stakeholders) CHIs and CHNAs (2002-2013) Total CHI Funding in Massachusetts to CHNAs through DoN Projects

This slide includes a map of Massachusetts separated by Community Health Network Areas, with the areas shaded in a darker purple to include more total Community Health Initiative Funding from 2002 to 2013. The darkest areas, which indicate $1.5 to $5.1 million dollars, is in Boston and just south of Boston, along with the Lowell area. There are no other patterns evident. Out in the Western part of the state, the shading is white indicating no DoN-funded CHNAs. There is also an inset of the City of Boston, displaying the entire area in dark purple. Total CHI Funding in Massachusetts from DoN Seeking Medical Facilities with Percent in Poverty per 2010 Census Tract

This slide includes a map of Massachusetts, with two layers. The background layer displays the percent of people in poverty by Census tract, with a darker orange indicating more poverty. The two highest ranges of poverty, 10-25% and 25%+ of the population in the tract, occur mostly in the Western part of the state, on Cape Cod, and in urban areas such as Boston. The other layer depicts medical facilities. Facilities that did not receive DoN funding are depicted as blue crosses. Green circles are used to indicate facilities that did receive funding, and the size of the circle shows how much funding was received from 2002 to 2013. The largest circle is $2.4 to $22.5 million. These large circles, indicating high funding, occur mostly in Metro Boston, Lowell, and Springfield. The Western part of the state has no hospitals that received DoN funding. An inset depicts the City of Boston, with most hospitals receiving some DoN funding and a few that are depicted not receiving funding. Total CHI Funding in Massachusetts from DoN Seeking Medical Facilities with 2013 County Health Rank*

This slide includes a map of Massachusetts, with two layers. The background layer displays state counties and County Health Rankings, with the darker orange indicating a lower ranking. Berkshire, Hampden, Bristol, and Suffolk counties are the darkest orange, indicating a ranking from 12 to 14. Norfolk and Middlesex counties have the highest ranking, from 1 to 4. The other layer depicts medical facilities. Facilities that did not receive DoN funding are depicted as blue crosses. Green circles are used to indicate facilities that did receive funding, and the size of the circle shows how much funding was received from 2002 to 2013. The largest circle is $2.4 to $22.5 million. These large circles, indicating high funding, occur mostly in Metro Boston, Lowell, and Springfield. The Western part of the state has no hospitals that received DoN funding. An inset depicts the City of Boston, with most hospitals receiving some DoN funding and a few that are depicted not receiving funding.

*County Health Rankings & Roadmaps (http://www.countyhealthrankings.org/) Total CHI Initiatives and Funding per EOHHS Region

This slide contains two graphs, with the x-axis on both being the 6 EOHHS regions: Western, Central, Northeast, Metrowest, Southeast, and Boston. The first graph displays “Total CHI Funds Per EOHHS Region”. Boston is the highest at around $25.5 million, more than twice the $10 million of the Western region. The other regions are at or below $5 million. The other graph displays “Total CHI Initiatives per EOHHS Region”. Boston is the highest at just over 120. Northeast is the second highest at just over 40, and the rest are under 40. Total CHI Initiatives and Funding according to DoN Approval Years over time

This slide contains two line graphs, each displaying data over the time period 2001 to 2013 (labeled “DoN Approval Year”). The first chart displays “Total CHI Projects according to DoN Approval Years”. The high is 70 in 2007, the low is 10 in 2004. There has been a recent downward trend since the high in 2007, with 2013 being just over 10. The other chart displays “Total CHI Funds according to their DoN Approval Years”. 2007 is the highest at around $30.5 million. 2004 is the low at around $0. Most other years around between $0 and $10 million. Total CHI Initiatives and Funding per Type of DoN Project

This slide contains two graphs, and on each graph the x-axis is a type of DoN project: Change, Construction, Equipment, Replacement, Service. The first graphs these categories by total DoNs. Equipment, at just over 60 projects, is the highest, followed by Service at just over 40. The lowest is Replacement at around 5. The other graph shows “Sum of CHI Funding Totals for DoN Project Type”. Construction is the highest at almost $50 million. Service is the second at $30 million. Replacement is the lowest at less than $5 million. Average Length of CHI Initiative according to DoN Approval Years over time

This slide contains a line chart with the DoN approval years 2002 to 2013 displayed. The y-axis is the number of initiative years. For most of the time period, the average length is around 5 years. This dips to 3 in 2011, before climbing back to 5 by 2013. Survey Results: Respondents

• Collected data on 42 DoN projects (pool of eligible respondents: 42 total hospital stakeholders, 63 total community stakeholders) • 82 Total responses • Response Rate: – 90% Hospitals (n=38) – 70% Community Partners (n=44) Basic Trends

This slide contains two bar graphs, with the x-axis on each listing six regions: Western, Central, Northeast, Metrowest, Southeast, Boston. The first graph is titled “Respondent Type” and charts number of responses. For reach region, the number is separated into “hospital” and “community”. Boston has the most responses, with 12 for hospitals and 10 from the community. Western is the least at 2 from hospitals and none from the community.

The other graph depicts count of responses by “Year of DoN approval”. 2009 is the highest for all regions, and is especially high as compared to the other years for Boston and the Northeast. There are not any other patterns evident in this graph. Survey Results

Divided into 3 Themes: 1. Decision-Making & Priority Setting 2. Measuring & Categorizing Impact 3. Aligning Community Health Initiatives Theme 1. Decision Making & Priority Setting

• Perception of control issues are prominent

• Opportunities exist for other stakeholders to be more involved Decision Making & Priority Setting: Who is making decisions?

This slide depicts a chart titled “Who decided where funding would go for the overall CHI? By respondent type.” The x-axis lists the responses as CHNA, DPH, Hospital, Local Health Department, Multiple Organizations, Other, and Other Community Partner. The responses are separated into counts for Hospital and Community respondents. CHNA, DPH, and Hospital and the three most common responses for both groups. However, for each of these options, more Community respondents chose them than Hospital respondents. The text below gives some context for the results:

Why might this be? • Hospitals express concern about being seen as a “blank check”, often “feel like a voice in the corner”; • Community members expressed that while a hospital has control over AGO’s community benefits they should not have control over CHI funds; that it is appropriate for a hospital to be part of the process but CHNA should have final say. DPH’s Role

• Most widely expressed suggestion among respondents: – MDPH should be a leader in setting process and outcome standards for investments – “ Developing a gold standard for how to make investments” – Ensuring compliance with those standards. Local Health

This slide contains two bar graphs. The first depicts a chart titled “Who decided where funding would go for the overall CHI? By respondent type.” The x-axis lists the responses as CHNA, DPH, Hospital, Local Health Department, Multiple Organizations, Other, and Other Community Partner. The responses are separated into counts for Hospital and Community respondents. CHNA, DPH, and Hospital and the three most common responses for both groups. However, for each of these options, more Community respondents chose them than Hospital respondents.

The section chart is titled “Local health department decided where funding would go for the overall CHI”. The x-axis lists the six regions: Western, Central, Northeast, Metrowest, Southeast, Boston. Only Northeast, Metrowest, and Boston contained any responses, with Boston having the majority. Text on the slide reads:

• Local Health Departments not typically part of the process • Local Health Departments most involved in Region 6/Boston Theme 2. Categorizing Impact

• Reporting • Evaluation • Measuring Impact • Defining Success Reporting

This slide contains two graphs. The first graph is titled “Does the hospital require that organizations receiving CHI funding report on how funds were used”. The Yes response was about 65, and the No responses about 35. The other graph is titled “Does the hospital require that organizations receiving CHI funding report on how funds were used?” The largest response category was “Other” at almost 70%, with “Annual Reports,” “Program Evaluation Reports” and “Financial Reports” all below 20%. The slide text reads:

• Most Hospitals (about 65%) request some form of reporting • Of those that do, most request “other” forms of reporting (standardization opportunity) “Other” Reporting included:

• Responses ranged widely. Some examples include: • “participation in CHNA meetings” • comments that while they currently do not require reporting they intend to do so in the future. One respondent noted that they “were not given the opportunity to require reporting/accountability and have been given no information from the CHNA” Evaluation

This slide contains two charts. The first is titled “Is the CHI in its entirety being evaluated? (hospital respondents only). The Yes response is just under 30%, the No response just under 50%, and the Other response just over 20%. The other graph is titled: “Are evaluations being conducted on the portion of the CHI that you are responsible for? (CHNA respondents only)”. Yes is just over 70%, No just over 20%. The slide text is:

• In roughly half of the cases of Hospital respondents, the CHI as an entire investment is not being evaluated • CHNAs however were very likely to report that evaluations were being conducted on the portion of the CHI funds that they were responsible for Measuring Impact

This slide contains a bar graph titled “Were any of the goals and objectives designed using the SMART (specific, measureable, attainable, relevant and time- based) method?” The x-axis reads Yes and No, with bars for both Community and Hospital respondents. Both types of respondents were close in percentage. Yes was just over 30%, and No was just over 60%. The slide text reads:

• Goals & Objectives tended NOT to be developed using SMART language Does the CHI impact health outcomes?

• Respondents of both types mostly reported that the CHI either has or will impact health outcomes; there is some correlation with when the CHI began. • What did they mean? • “measured by increasing program and municipal capacity to conduct public health programming” • “Exercise group for seniors to fight osteoporosis. Pre and post program evaluation done with participants” Discussion point – What could be appropriate SMART (specific, measurable, attainable, relevant, time-based) metrics? Other Impacts

• Respondents of both types tended to say that the CHI investments: • Have increased the number of partnerships working on shared goals • Have or will have an impact on the SDOH • Are improving primary and preventative care for vulnerable populations and reducing health disparities • Key informant and document review noted that impacts are mostly measured through reach of programs and services – There is a “need for shared measures for both outcomes and process” Other Impacts

• Regarding social determinants of health (SDOH): informants noted that there is a serious deficiency in funding streams that attempt to impact change at this level and these funds should be used accordingly Defining Success

This slide contains a bar graph titled “Would you define this CHI as successful (and if still in progress is it successful so far)? By respondent type”. There are two responses: Yes and No, which are depicted as bars for Hospital and Community respondents. Yes is the more common response for both groups, but Community respondents indicated stronger support; over 90% answering Yes as opposed to around 70% answering Yes for hospitals. The text on the slide reads:

Overall community respondents were more likely than hospital respondents to define the CHI as successful. Why… ? Defining Success

• Process and Outcome answer types • “Engagement of local residents and training them to be effective agents for change” • “new affordable housing has been built, 150 at risk students have received college scholarships and academic support, anti-racism training has been provided to 140 community leaders, two major minority health coalitions were funded at $400,000 a year for seven years” SMART measure development possible and needed for both process and outcomes. Theme 3. Aligning CHIs

• Community Health Assessments • Community Benefits, Accreditation, Local Health • Defining Service Areas • Impacting the way hospitals focus on population health Community Health Assessments

This slide contains two bar graphs. The first is titled “Whose community health assessments was used? By respondent types”. The response options were “Community Group,” “Hospital,” Local Health Department,” and “More than 1 CHA was used”. The majority of respondents chose “Community Group” or Hospital”, with both Community and Hospital respondents posting similar numbers.

The other bar graph is titled “Were community health assessments used to inform the selection of priorities and/or which organizations were funded for the CHI?” Hospital and Community respondents posted the exact same numbers. Yes was 70%, and No was 30%. The text on the slide reads:

Using Community Health Assessments to inform priorities • Community & Hospital respondents mostly used CHAs in developing priorities for CHI funds • Community Group and Hospital’s Assessments most commonly used But CHA related issues exist

• CHAs lack shared focus, too many different groups engaged in doing them – E.g. hospitals, community groups, local health • Conflicts between the priorities developed through a hospital based CHA and another CHA can create issues with CHI related investments – However this concern not borne out strongly with the survey data (~10% of responses noted a conflict) Community Benefits, Accreditation & CHIs

• While not assessed in the survey many issues arose in interviews • Key informants noted need for better alignment: – attorney general, local health through accreditation, state health through accreditation, DoN, CHA, etc • Both an opportunity & risk – Opportunity: metrics and priority settings; unpredictability of DoN/CHI could be tempered by routine community benefit investment – Risk: hospitals have control over community benefits, community groups could lose say over investments if everything aligns Community Benefits & CHIs

• Key informants also touched on how CHI investments can be useful in helping hospitals understand how to invest in community health in ways that will facilitate transitions to population health models Local Health

• There is a lot of opportunity for local health to be more engaged in CHIs. • local health generally not part of the CHI process (with exceptions) • Key Informant Interview feedback varied • In some contexts the board of health agents do not understand the connection to CHI related efforts (training) • In others the local health department has the capacity to provide infrastructure and guidance to these investments (leverageing) • DoN/CHI an opportunity to get local and state public health and health care delivery providers on the same page (accreditation) Service Area Issues

• Hospital respondents said that just over half of CHI investments matched up with the hospitals service area Representative themes across the spectrum on this issue – “won’t get a lot of buy-in unless the investments line up with the hospitals market share”; – critical that the CHNA gets funding, or decides how funding is used, regardless of issues of synergy with hospital service area; – there is a need to approach issues of geography from a parity perspective: who has needs and where resources are deficient; – “Don’t dilute funds through some sort of redistribution until health status in primary services areas improve” Discussion… What has CHIs meant for hospitals?

• Over 30% of hospital respondents reported that CHI investments were not being reported internally at the hospital – In some regions this rises to 60% or higher. Opportunity to raise awareness among Hospital Governance body of population health needs and opportunity. Internal Reporting

– For those that did internal reporting, the most common place for reports to be delivered include leadership groups involved in community benefits reporting, or also report to the board of trustees, – Those that didn’t require reporting mostly said that they “simply wrote a check” or that they did not hear back from CHNA as to how funds were used Does the CHI change the way hospitals do business?

This slide depicts a pie chart titled “Has the CHI changed the way the hospital does business? (Hospital respondents only)”. The Yes response is 37%, and the No response is 63%. The text on the slide reads:

• Most said no. • For those who said yes, typical responses mostly centered around the relationship between the hospital and the community. • “It has enhanced our community collaboration. It has also helped us better represent our community.” • For those that said no, responses were either of the nature that the CHI program functions entirely separately from the rest of the hospital or that it is already the direction in which the hospital is moving. • “The CHI complements the way the hospital does business” • “We were already working towards goals that are compatible with the CHI” • “Hospital initiatives remain largely separate from CHNA activities” Guiding Principle

• MDPH is focused on the development of guidance and standards that aim to have measurable improvements in the public’s health. Discussion Questions

We need to measure impact • What should those measures be? • What criteria should be applied? We need shared processes • How should decision-making be structured? • How should priorities be selected and what information should inform the decision-making process? • How should unequal investment across the state be addressed? Acknowledgements

• Noemie Sportiche, Peter James, Halley Reeves Brunsteter • Metropolitan Area Planning Council • www.mapc.org