Evaluation of a System Designed to Link People Living with HIV/AIDS with Mental

Evaluation and Program Planning 35 (2012) 133–138

Contents lists available at SciVerse ScienceDirect

Evaluation and Program Planning

jo urnal homepage: www.elsevier.com/locate/evalprogplan

Evaluation of a system designed to link people living with HIV/AIDS with mental

health services at an AIDS-service organization

a, a b

Matthew B. Feldman *, Gregg S. Weinberg , Elwin Wu

Gay Men’s Health Crisis, New York, NY, United States

Columbia University School of Social Work, New York, NY, United States

A R T I C L E I N F O A B S T R A C T

Article history: This study evaluated the Rapid Response System, an initiative designed to link people living with HIV/

Received 13 January 2011

AIDS with mental health services at an AIDS service organization. Data were extracted from agency

Received in revised form 18 August 2011

records for 314 clients who had contact with the Rapid Response System over a 6-month period. Of the

Accepted 25 August 2011

281 clients who scheduled an appointment for an evaluation to initiate mental health services, 64%

Available online 3 September 2011

completed the evaluation. In the multivariate analysis, Latinos were signiﬁcantly less likely than whites

to complete the mental health evaluation. Further, there was a signiﬁcant decrease in the likelihood of

Keywords:

completing the mental health evaluation as the number of days between the Rapid Response System

Mental health

contact and the date of the evaluation appointment increased.

Linkage

Intervention Strategies that reduce the period of time between the initial referral and initiation of services may

HIV facilitate linkage with mental health treatment, particularly in the context of larger multi-service

organizations.

1. Introduction waiting lists can discourage many people with triple diagnoses

(i.e., HIV-positive individuals with co-occurring mental health and

Recent studies of people living with HIV/AIDS have found that substance use disorders) from seeking mental health treatment.

32%–63% had mood disorders and 21%–26% had anxiety disorders Indeed, longer delays between a request or referral for mental

(Bing et al., 2001; Budin, Boslaugh, Beckett, & Winiarski, 2004; health services and an initial appointment with a provider are

Gaynes, Pence, Eron, & Miller, 2008). Mental health services are an associated with increased no-show rates (Grunebaum et al., 1996;

important component of HIV care, particularly because of the Greeno, Anderson, Shear, & Mike, 1999; Orme & Boswell, 1991). In

negative impact psychiatric problems can have on physical health. a sample of 5901 patients who were scheduled for an initial

In prior studies, depression was associated with greater declines in psychiatry appointment, Gallucci, Swartz, and Hackerman (2005)

CD4 counts (Ironson et al., 2005), immunological failure (Anastos found that the odds of cancellations and no-shows increased by

et al., 2005), a decreased virological response to highly active 12% for every day of delay between the initial contact and the

antiretroviral therapy (Anastos et al., 2005; Horberg et al., 2008; client’s appointment. Evaluations of strategies designed to

Parienti et al., 2004), and an increased risk of AIDS-related death decrease delays in connecting clients to treatment at mental

(Cook et al., 2004). There is evidence that participation in mental health clinics (e.g., offering appointments on the same day as the

health services increases engagement with medical care (Messeri, request for services or soon thereafter) yielded successful ﬁndings

Abramson, Aidala, Lee, & Lee, 2002; Rumptz et al., 2007) and in terms of decreasing no-show rates (Pomerantz, Cole, Watts, &

medication adherence (Ashman, Conviser, & Pounds, 2002; Cook Weeks, 2008; Williams, Latta, & Conversano, 2008).

et al., 2006) in people living with HIV/AIDS. The integrated care model, also known as ‘‘one-stop shopping,’’

Delays in initiating treatment can frequently impact whether has become an increasingly utilized model for providing medical

clients will ultimately engage in mental health services. Calsyn, and social services to people with chronic illnesses. Integrated care

Klinkenberg, Morse, Miller, and Cruthis (2004) suggested delays for people living with HIV/AIDS ‘‘combines HIV primary care with

in providing services, such as lengthy intake procedures and mental health and substance abuse services into a single

coordinated treatment program that simultaneously, rather than

in parallel or sequential fashion, addresses the clinical complexi-

ties associated with having multiple needs and conditions’’ (Soto,

* Corresponding author at: Gay Men’s Health Crisis, 446 W. 33rd Street New York,

Bell, & Pillen, 2004, p. S44). Ancillary services, including health

NY 10001, United States. Tel.: +1 212 367 1285.

E-mail address: matthewf@gmhc.org (M.B. Feldman). education, case management, transportation, and concrete support

134 M.B. Feldman et al. / Evaluation and Program Planning 35 (2012) 133–138

(e.g., food) are therefore a critical component of integrated HIV completing the mental health evaluation and; (3) to test whether

care. Studies have documented that integrated HIV care is there is a relationship between appointment delay and completing

associated with engagement and retention in HIV medical care the mental health evaluation.

(Ashman et al., 2002; Conover & Whetten-Goldstein, 2002; Lo,

MacGovern, & Bradford, 2002; Magnus et al., 2001; Messeri et al., 2. Methods

2002; Sherer et al., 2002). One of the beneﬁts of integrated HIV care

is decreasing the number of services for which clients must be 2.1. Site

referred to off-site providers, particularly because clients fre-

quently do not follow through on these referrals (Friedmann, GMHC offers a range of services to people living with HIV/AIDS,

D’Aunno, Jin, & Alexander, 2000; Umbricht-Schneiter, Ginn, Pabst, including ﬁnancial and legal advocacy, meals and nutrition, health

& Begelow, 1994). and wellness programs, case management, and educational and

Few studies, however, have addressed the challenges of vocational services. In 2005, GMHC formed Team 119, a mental

coordinating care in the context of an integrated care model. health services department that is staffed by mental health

Effective care coordination across departments that represent counselors, substance abuse counselors, social workers, and art

multiple disciplines and programs is especially critical in ensuring therapists who provide individual, couples, family, and group

that a client receives all of the services in an organization that therapy to a diverse population of people living with HIV/AIDS

would best address his needs. For example, case management staff presenting with a range of psychiatric and psychosocial issues.

might not refer clients to a mental health program in the same Mental health services are offered free of charge to GMHC clients.

organization because they do not know how to identify clients who

are eligible or appropriate for services, or they may not know any of 2.2. The Rapid Response System

the staff members in this department. Indeed, ineffective

interdepartmental and interdisciplinary interactions can signiﬁ- In 2008, the Rapid Response System was developed to facilitate

cantly impact workﬂow and quality of care (Abraham & Reddy, the linkage of GMHC clients to mental health services. The central

2010; Gittell et al., 2000; Strauss & Mino, 2011). To address this elements of this system include:

issue, departments can provide opportunities (e.g., in-services) for (1) An immediate response. Clients frequently present signiﬁcant

other staff to learn about their program and how clients can be psychosocial stressors or psychiatric issues when they meet with

linked to services. This type of inter-departmental outreach nonclinical GMHC staff members, who include case managers,

strategy can be an effective strategy in increasing referrals (Brown, lawyers, and nutritionists. GMHC staff members can contact the

Raue, Nassisi, & Bruce, 2001). Rapid Response System when a client is in crisis, or if they perceive

Interdepartmental communication can be facilitated by codi- that a client with whom they are meeting might beneﬁt from

ﬁed protocols that deﬁne how staff members should coordinate mental health services. Clients may also request mental health

care in an organization (e.g., referral procedures, program services. When a GMHC staff member contacts the Rapid Response

eligibility guidelines). In a study of the barriers to the integration System, the on-call staff member comes to meet with the client

of mental health and medical care, one of the most common within 15 min (a back up staff member is contacted if the on-call

administrative barriers reported by providers and administrators staff member cannot respond immediately) to conduct a brief

was the lack of policies and standards for integrating care assessment to determine if his needs can be met by Team 119.

(Kilbourne et al., 2008). One of the proposed solutions to address While there are no eligibility requirements to receiving mental

maladaptive interdepartmental interactions is the use of an health services, there are some situations in which clients may

‘‘integrator,’’ who is responsible for facilitating interdepartmental have needs that Team 119 does not have the capacity to address

care coordination activities (Abraham & Reddy, 2010; Gittell & (e.g., signiﬁcant developmental delays). In these situations, the

Weiss, 2004). Care coordination can therefore be challenging in Rapid Response System staff member facilitates a referral to a more

agencies that offer multiple services and programs, each of which appropriate organization. If a client is suicidal or homicidal, the

may have a unique and perhaps lengthy intake procedure that may staff member contacts emergency services to ensure that

delay the initiation of care or even deter individuals from engaging individual’s safety.

in services (Calsyn et al., 2004). Larger multi-service organizations If the client expresses interest in receiving counseling services,

need to consider strategies and systems that facilitate intra-agency he is offered the next available appointment for a mental health

care coordination. evaluation. The mental health evaluation is the point-of-entry to

The current study is an evaluation of the Rapid Response receiving counseling services at GMHC during which an in-depth

System, which is a set of operating procedures designed to assessment is conducted by a Team 119 staff member so that a

facilitate the linkage of clients to Team 119, the mental health treatment plan can be developed that best addresses the client’s

services department at Gay Men’s Health Crisis (GMHC), an AIDS needs. After the evaluation is completed, the client is connected

service organization in New York City. The Rapid Response System with the appropriate individual and/or group mental health

was designed to address the challenges of interdepartmental care services.

coordination by providing an easy and efﬁcient mechanism for (2) Increased number of mental health evaluation appointment

GMHC staff members to refer clients to mental health services. Any slots. The number of available mental health evaluation appoint-

client who is referred for mental health services can meet with a ment slots each week was increased from 10 to 20 in an effort to

Team 119 staff member for a brief consultation at the time of the decrease the amount of time between the initial referral or request

request. If appropriate, an appointment for a comprehensive for mental health services i.e., the Rapid Response System

mental health evaluation to initiate counseling services can also be encounter and the initiation of treatment. Clients were usually

scheduled at this time. The Rapid Response System was offered an appointment for a mental health evaluation within one

implemented in April 2008, and the evaluation occurred six week of the Rapid Response System contact.

months after its implementation. The aims for the evaluation were:

(1) to evaluate the proportion of clients seen by the Rapid Response 2.3. Sample

System who: (a) elected to scheduled an appointment for a mental

health evaluation and; (b) completed the mental health evalua- The study is based on a sample of 314 GMHC clients who had

tion; (2) to identify client characteristics associated with contact with the Rapid Response System between April 2008 and

M.B. Feldman et al. / Evaluation and Program Planning 35 (2012) 133–138 135

December 2008. Sociodemographic and service related data were bivariate analyses. These results are shown as adjusted odds

extracted from the agency’s client services database. ratios (AOR) with their corresponding 95% CI. The outcome variable

for these analyses was completing a mental health evaluation,

2.4. Measures which was coded dichotomously (0 = did not complete the mental

health intake, 1 = completed the mental health intake appoint-

Scheduling a mental health evaluation appointment. Scheduling a ment on a ﬁrst or subsequent attempt). Data were analyzed using

mental health evaluation appointment was coded dichotomously SPSS statistical software (Version 15.0).

(yes/no). If a client did not choose to schedule an appointment for a

mental health evaluation at the end of the Rapid Response System

contact, one of the following reasons for their decision were coded: 3. Results

(1) not interested in counseling; (2) did not want to make an

appointment for a mental health evaluation at that time; (3) The majority of clients who had contact with the Rapid

referred to a more appropriate program at GMHC; (4) referred to a Response System from April 2008 to December 2008 were black or

more appropriate program outside of GMHC or; (5) already in Latino (68%) and lesbian, gay, or bisexual (81%). The client ﬂow for

treatment somewhere else. the Rapid Response System is described in Fig. 1. Of the 314 clients

Completing the mental health evaluation. One of the following who had contact with the Rapid Response System, 90% (n = 281)

outcomes was coded for all clients who scheduled a mental health scheduled an appointment for a mental health evaluation. These

evaluation: (1) client was a ‘‘no show’’ for the mental health clients received an appointment for a mental health evaluation

evaluation; (2) client completed the mental health evaluation or; that was an average of ﬁve days (SD = 3.3) from the Rapid Response

(3) client rescheduled their mental health evaluation. Final System contact. Of the 281 clients who scheduled an appointment

outcomes were tracked for clients who rescheduled a mental for a mental health evaluation, 47% (n = 131) completed the

health evaluation appointment. Three categories were then evaluation on the ﬁrst attempt, 17% (n = 49) completed the

created based on these data: (1) completed the mental health evaluation on a subsequent attempt, and 36% did not complete

evaluation on the ﬁrst attempt; (2) completed the mental health the evaluation (n = 101). Of the 33 clients who did not schedule an

evaluation on a subsequent attempt and; (3) did not complete the appointment for a mental health evaluation, the most common

mental health evaluation. reasons were not wanting to make an appointment at the time of

Appointment delay. Appointment delay was deﬁned as the the Rapid Response System encounter (30%) and having no interest

number of days between the Rapid Response System contact and in counseling (27%).

the date of the ﬁrst scheduled appointment for a mental health In the bivariate analysis, gay/bisexual clients were signiﬁcantly

evaluation (even if the appointment did not ultimately occur on more likely than heterosexual clients to complete the mental

that day). health evaluation (68% vs. 49%). Further, black (60%) and Latino

Demographics. Sociodemographic data included the following (57%) were signiﬁcantly less likely than white clients (78%) to

client characteristics: (1) age; (2) gender; (3) race/ethnicity (white, complete the mental health evaluation. There was a signiﬁcant

black, Latino, other) and; (4) sexual orientation (gay, lesbian, decrease in the odds of completing the mental health evaluation

bisexual, straight). as the number of days between the Rapid Response System

encounter and the date of the evaluation appointment increased

2.5. Statistical analyses (Table 1).

In the multivariate model, Latinos were signiﬁcantly less likely

Frequencies were used to describe the proportion of clients who than whites to complete the mental health evaluation

had contact with the Rapid Response System who scheduled and (AOR = 0.42, CI = 0.21, 0.86). Further, more days between the

completed a mental health evaluation, in addition to the reasons Rapid Response System encounter and the date of the mental

that clients chose not to schedule an appointment for a mental health evaluation appointment was associated with a decreased

health evaluation. likelihood of completing the mental health evaluation

Bivariate analyses of the relationship between each of the (AOR = 0.84, CI = 0.78, 0.92) when controlling for the effects of

variables and completing the mental health evaluation were the other variables (Table 2).

conducted using logistic regression to estimate odds ratios. A An additional logistic regression was run to determine if the

multivariate logistic regression model was used to identify the interaction between gender and sexual orientation was signiﬁ-

variables independently associated with completing a mental cantly associated with completing a mental health evaluation. This

health evaluation. Criteria for including variables in the multivari- model did not converge, likely because of the small sample size

ate model included statistical signiﬁcance (P < 0.10) in the (results not shown).

Fig. 1. Rapid Response System client ﬂow.

136 M.B. Feldman et al. / Evaluation and Program Planning 35 (2012) 133–138

Table 1

Bivariate analysis. Association of sociodemographic and background characteristics with completing a mental health evaluation among 281 clients who scheduled an

evaluation appointment.

N Completed mental health evaluation OR 95% CI

n (%) or M (SD)

Yes No

Gender

Female 33 17 (52%) 16 (49%) 0.55 0.27–1.15

Male 248 163 (66%) 85 (34%) (Ref.)

Sexual orientation

Gay/bisexual 224 152 (68%) 72 (32%) 2.18 1.21–3.94

Heterosexual 57 28 (49%) 29 (51%) (Ref.)

Race/ethnicity

Black 106 64 (60%) 42 (40%) 0.44 0.23–0.84

Latino 85 48 (57%) 37 (44%) 0.37 0.19–0.73

Other 9 5 (56%) 4 (44%) 0.36 0.09–1.47

White 81 63 (78%) 18 (22%) (Ref.)

Age 42.1 (9.9) 42.8 (9.1) 0.99 0.97–1.01

Days until scheduled appointment 4.2 (3.0) 6.1 (3.5) 0.84 0.77–0.91

Table 2

& McGuire, 2006; Snowden, 1999; Wells, Klap, Koike, &

Multivariate analysis of completing a mental health evaluation.

Sherbourne, 2001). In a study of 231,400 people living with

AOR 95% CI HIV/AIDS, Burnam et al. (2001) found that blacks were signiﬁcantly

Gender less likely than whites to use individual or family outpatient

Female 1.32 0.47–3.71 mental services.

Male (Ref.) Heterosexuals were signiﬁcantly less likely than lesbian, gay

Sexual orientation

and bisexual individuals to engage with mental health services,

Gay/lesbian/bisexual 1.9 0.82–4.38

although these ﬁndings were only signiﬁcant at the bivariate level.

Heterosexual (Ref.)

Race/ethnicity In a population-based probability sample of 2074 adults aged

Black 0.54 0.27–1.08 18–64, Grella, Greenwell, Mays, and Cochran (2009) found that

Latino 0.42 0.21–0.86

gay/bisexual men were signiﬁcantly more likely than heterosexual

Other 0.34 0.08–1.46

women to have received mental health or substance abuse

White (Ref.)

treatment. Similarly, Burnam et al. (2001) found that HIV-infected

Days until scheduled appointment 0.84 0.78–0.92

gay individuals were signiﬁcantly more likely than HIV-infected

heterosexual individuals to use individual or family mental health

services.

4. Discussion This study’s ﬁndings suggest the need for increased outreach

efforts to engage clients who may have difﬁculty engaging in

The purpose of this study was to evaluate a system to link mental health services, particularly heterosexuals and people of

people living with HIV/AIDS with mental health services at a color. Further assessment should be conducted on the mental

community-based AIDS service organization. While integrated health needs of heterosexual male clients at GMHC. There is also a

care models increase access to care by providing medical and social larger issue in that the organization’s name, ‘‘Gay Men’s Health

services in one facility, strategies like the Rapid Response System Crisis,’’ may still give the impression that it is an agency that

are critical in addressing the challenges of coordinating client care primarily serves gay men. There are efforts being made to broaden

across departments. Of the 281 clients who scheduled an the appeal of GMHC to a more diverse range of populations.

appointment for a mental health evaluation, almost two-thirds There are several limitations to this study that should be noted.

ultimately completed the evaluation. In a study of a similar First, the evaluation lacked an equivalent control group with which

strategy to reduce wait times for mental health services Williams to compare the effectiveness of the Rapid Response System, or an

et al. (2008), reported an average no-show rate of 18% post- adequate amount of certain client subgroups (e.g., lesbian and

implementation. bisexual women) to conduct meaningful analyses—which may

Consistent with others (Gallucci et al., 2005; Grunebaum et al., indicate the need to increase outreach to these populations.

1996; Greeno et al., 1999; Orme & Boswell, 1991), a shorter delay Second, the clients in the sample were primarily seeking or at least

between the Rapid Response System contact and the date of the agreeing to treatment, so it is not completely representative of

mental health evaluation was associated with completing the people living with HIV/AIDS who may need mental health services.

evaluation. This ﬁnding suggests that it may be important to offer Third, a cross-sectional study design was used for this evaluation,

multiple slots for mental health evaluation appointments so and therefore inferences about causality cannot be made in terms

clients can connect with services as soon as possible. Implement- of the effectiveness of the Rapid Response System in linking clients

ing enhanced follow-up efforts (e.g., reminder calls) with clients to mental health services.

who have appointments scheduled for more than several days The Rapid Response System engages clients at the time of the

from the Rapid Response System contact might also help to request for services, in addition to arranging for the client to

decrease no- show rates. initiate treatment as soon as possible so that clients who could

In this study, nonwhite clients, particularly Latinos, were beneﬁt from mental health services do not become lost to follow

signiﬁcantly less likely than whites to complete the mental health up care. The ﬁndings of this evaluation study suggest that

evaluation. This is consistent with prior studies that have found decreasing the delay between the initial request for services and

blacks and Latinos are less likely than whites to use mental health the mental health evaluation (the point of entry to receiving

services (Alegria et al., 2002; Cabassa, Zayas, & Hansen, 2006; mental health services) was associated with successful linkage to

Lasser, Himmelstein, Woolhander, McCormick, & Bor, 2002; Ojeda care. This type of system could be particularly useful to multi-

M.B. Feldman et al. / Evaluation and Program Planning 35 (2012) 133–138 137

service organizations in addressing some of the challenges of both service delivery and program evaluation. We also realized we

interdepartmental care coordination by providing a clear and were never explicit about why we were asking them to document

efﬁcient protocol for referring clients to mental health services. certain events (e.g., tracking clients who rescheduled mental

health evaluation appointments) in addition to how those data

5. Lessons learned could ultimately be useful to the staff in planning services. These

efforts may have increased staff buy-in, and ultimately their

We identiﬁed two issues during the evaluation process that in participation in the evaluation of the Rapid Response System.

retrospect should have been addressed prior to implementation of

the Rapid Response System. First, we learned it is important to

Acknowledgments

establish speciﬁc aims to identify the questions that will be

answered by conducting the evaluation. These aims will also

The authors would like to thank the staff of the Mental Health

indicate the variables for which data should be collected. When we

Services Department at GMHC for their support of this project,

originally conceptualized the evaluation of the Rapid Response

particularly Daniel Blausey, Michael Benibgui, Chris Cochrane,

System, our main priority was to assess the proportion of clients

Michael Hamilton, and Larry Woodland. The authors would also

who scheduled and completed a mental health evaluation, and the

like to thank Eric Arnold and Stephen Hile for their valuable

time that elapsed between the initial referral for services and the

feedback during the revision process, and Jessica Diamond, who

date of the scheduled evaluation appointment. We therefore

was instrumental in developing the data entry and tracking

focused on documenting the dates of the Rapid Response System

procedures. Finally, they would also like to express their

contact and the mental health evaluation, and whether or not the

appreciation to Terri Jackson, Jeff Rindler, and Janet Weinberg,

client completed the evaluation.

whose support of program evaluation has made projects like this

When we began to analyze these data, we started to identify

possible.

variables for which we did not collect data that would have

enhanced our ability to report on the performance of the Rapid

Response System. There may have been other client-level variables References

such as sociodemographic characteristics (e.g., employment status,

Abraham, J., & Reddy, M. C. (2010). Challenges to inter-departmental coordination of

educational background) and the client’s presenting problem that

patient transfers: A workﬂow perspective. International Journal of Medical Infor-

impacted whether or not a client ultimately engaged in mental matics, 79, 112–122.

health services. Although Rapid Response System staff members Alegria, M., Canino, G., Rios, R., Vera, V., Calderon, J., Rusch, D., & Ortega, A. N. (2002).

Inequalities in use of specialty mental health services among Latinos, African

described the client’s presenting problems on the assessment form,

Americans, and Non-Latino Whites. Psychiatric Services, 53(12), 1547–1555.

there was no standardized protocol as to how this information

Anastos, K. A., Schneider, M. F., Gange, S. J., Minkoff, H., Greenblatt, R. M., Feldman, J.,

should be reported, and therefore there was no consistency in how et al. (2005). The association of race, sociodemographic, and behavioral character-

istics with response to highly active antiretroviral therapy in women. Journal of

the Rapid Response System staff recorded these data. In reviewing

Acquired Immune Deﬁciency Syndromes, 9, 537–544.

the Rapid Response System forms, it was impossible to code the

Ashman, J. J, Conviser, R., & Pounds, M. B. (2002). Associations between HIV-positive

client’s presenting problems in any systematic way. individuals’ receipt of ancillary services and medical care receipt and retention.

AIDS Care, 14(Suppl. 1), S109–S118.

Problem severity may have also impacted who completed the

Bing, E. G., Burnham, M. A., Longshore, D., Fleishman, J. A., Sherbourne, C. D., London, A.

mental health evaluation. For example, clients who were referred

S., Turner, B. J., Eggan, F., & Beckman, R. (2001). Psychiatric disorders and drug use

for mental health services because of a crisis might have been more among human immunodeﬁciency virus. Infected adults in the United States.

Archives of General Psychiatry, 58, 721–728.

likely to complete the evaluation because they felt a more urgent

Brown, E. L., Raue, P. J., Nassisi, P., & Bruce, M. L. (2001). Increasing recognition and

need to initiate treatment. Clients who were referred for

referral of the depressed elderly. Home Healthcare Nurse, 19(9), 558–564.

counseling for less severe issues, however, may have been less Budin, J., Boslaugh, S., Beckett, E., & Winiarski, M. G. (2004). Utilization of psychiatric

services integrated with primary care by persons of color in the inner city.

motivated to start treatment because their presenting problems

Community Mental Health Journal, 40(4), 365–378.

did not have the same level of urgency. Finally, the client’s

Burnam, M. A., Bing, E. G., Morton, S. C., Sherbourne, C., Fleishman, J. A., London, A. S.,

motivation for treatment also may have affected who followed Vitiello, B., Stein, M., Bozzette, S. A., & Shapiro, M. (2001). Use of mental health and

through with treatment. Rapid Response System staff have substance abuse treatment services among adults with HIV in the United States.

Archives of General Psychiatry, 58, 729–736.

reported that some clients disclosed that they agreed to schedule

Cabassa, L. J., Zayas, L. H., & Hansen, M. C. (2006). Latino adults’ access to mental health

a mental health evaluation to comply with the referring staff

care: A review of epidemiological studies. Administration and Policy in Mental

member rather than because they were interested in counseling Health and Mental Health Services Research, 33(3), 316–330.

services. Calsyn, R. J, Klinkenberg, W. D., Morse, G. A., Miller, J., & Cruthis, R. (2004). Recruitment,

engagement, and retention of people living with HIV and co-occurring mental

To address these gaps in data collection, a new Rapid Response

health and substance use disorders. AIDS Care, 16(Suppl. 1), S56–S70.

System assessment form was developed that includes a checklist of Conover, C. J., & Whetten-Goldstein, K. (2002). The impact of ancillary services on

primary care use and outcomes for HIV/AIDS patients with public insurance

presenting problems that are divided into categories, including

coverage. AIDS Care, 14(Suppl. 1), S59–S71.

health and mental health related issues, in addition to life

Cook, J. A., Grey, D., Burke-Miller, J., Cohen, M. H., Anastos, K., Gandhi, M., Richardson, J.,

transitional (e.g., losing a job), interpersonal (e.g., problems with & Young, M. (2006). Effects of treated and untreated depressive symptoms on

highly active antiretroviral therapy use in a US multi-site cohort of HIV-positive

family members), and environmental stressors (e.g., living in a

women. AIDS Care, 18(2), 93–100.

violent neighborhood). Rapid Response System staff members will

Cook, J. A., Grey, D., Burke, J., Cohen, M. H., Gurtman, A. C., Richardson, J. L., Wilson, T. E.,

be trained on how to complete this form, particularly so that there et al. (2004). Depressive symptoms and AIDS-related mortality a multisite cohort

of HIV-positive women. American Journal of Public Health, 94, 1133–1140.

can be more consistency around data collection.

Friedmann, P. D., D’Aunno, T. A., Jin, L., & Alexander, J. A. (2000). Medical and

Second, we realized that we had not involved all of the

psychosocial services in drug abuse treatment: Do stronger linkages promote

stakeholders—particularly the Rapid Response System staff—in all client utilization? Health Services Research, 35, 443–465.

of the phases of the program evaluation process. This may have Gallucci, G., Swartz, W., & Hackerman, F. (2005). Impact of the wait for an initial

appointment on the rate of kept appointments at a mental health center. Psychi-

impacted buy-in among the staff to the data entry protocol, which

atric Services, 56(3), 344–346.

was perceived as an additional and burdensome responsibility. In

Gaynes, B. N., Pence, B. W., Eron, J. J., & Miller, W. C. (2008). Prevalence and comorbidity

hindsight, we might have had a series of brainstorming sessions of psychiatric diagnoses on reference standard HIV+ patient population. Psychoso-

matic Medicine, 70, 505–511.

with the Rapid Response System staff to discuss the purpose of the

Gittell, J. H., Fairﬁeld, K. M., Bierbaum, B., Head, W., Jackson, R., Kelly, M., et al. (2000).

evaluation in greater depth. Data collection procedures could have

Impact of relational coordination on quality of care, postoperative pain and

then been developed that addressed the demands associated with functioning, and length of stay. Medical Care, 38(8), 807–819.

138 M.B. Feldman et al. / Evaluation and Program Planning 35 (2012) 133–138

Gittell, J. H., & Weiss, L. (2004). Coordination of networks within and across organiza- Rumptz, M. H., Tobias, C., Rajabiun, S., Bradford, J., Cabral, H., Young, R., & Cunningham,

tions: A multi-level framework. Journal of Management Studies, 41(1), 127–153. W. E. (2007). Factors associated with engaging socially marginalized HIV-positive

Greeno, C. G., Anderson, C. M., Shear, M. K., & Mike, G. (1999). Initial treatment persons in primary care. AIDS Patient Care and STDs, 21(Suppl. 1), S30–S39.

engagement in a rural community mental health center. Psychiatric Services, Sherer, R., Stieglitz, K., Narra, J., Jasek, J., Green, L., Moore, B., Schott, S., & Cohen, M.

50(12), 1634–1636. (2002). HIV multidisciplinary teams work: Support services improve access to and

Grella, C. E., Greenwell, L., Mays, V. M., & Cochran, S. D. (2009). Inﬂuence of gender, retention in HIV primary care. AIDS Care, 14(Suppl. 1), S31–S44.

sexual orientation, and need on treatment utilization for substance use and mental Snowden, L. R. (1999). African American service use for mental health problems. Journal

disorders: Findings from the California Quality of Life Survey. BMC Psychiatry, 9(1), of Community Psychology, 27(3), 303–319.

52–61. Soto, T. A., Bell, J., & Pillen, M. B. (2004). Literature on integrated HIV care: A review.

Grunebaum, M., Luber, P., Callahan, M., Leon, A. C., Olfson, M., & Portera, L. (1996). AIDS Care, 16(Suppl. 1), S43–S55.

Predictors of missed appointments for psychiatric conditions in a primary care Strauss, S. M., & Mino, M. (2011). Addressing the HIV-related needs of substance

clinic. Psychiatric Services, 47(8), 848–852. misusers in New York State: The beneﬁts and barriers to implementing a ‘‘one-stop

Horberg, M. A., Silverberg, M. J., Hurley, L. B., Towner, W. J., Klein, D. B., Bersoff-Matcha, shopping’’ model. Substance Use and Misuse, 46, 171–180.

S., et al. (2008). Effects of depression and selective serotonin reuptake inhibitor use Umbricht-Schneiter, A., Ginn, D. H., Pabst, K. M., & Begelow, G. E. (1994). Providing

on adherence to highly active antiretroviral therapy and on clinical outcomes in medical care to methadone clinic patients: Referral vs. on-site care. American

HIV-infected patients. Journal of Acquired Immune Deﬁciency Syndromes, 47, 384– Journal of Public Health, 84, 207–210.

390. Wells, K., Klap, R., Koike, A., & Sherbourne, C. (2001). Ethnic disparities in unmet need

Ironson, G., O’Cleirigh, C., Fletcher, M., Laurenceau, J., Balbin, E., Klimas, N., et al. (2005). for alcoholism, drug abuse, and mental health care. The American Journal of

Psychosocial factors predict CD4 and viral load change in men and women with Psychiatry, 158(12), 2027–2032.

human immunodeﬁciency virus in the era of highly active antiretroviral treatment. Williams, M. E, Latta, J., & Conversano, P. (2008). Eliminating the wait for mental health

Psychosomatic Medicine, 67, 1013–1021. services. The Journal of Behavioral Health Services & Research, 35(1), 107–114.

Kilbourne, A. M., Irmiter, C., Capobianco, J., Reynolds, K., Milner, K., Barry, K., et al.

(2008). Improving integrated general medical and mental health service in com-

Matthew B. Feldman Ph.D., MSW is the Associate Director of Program Services

munity-based practices. Administration and Policy in Mental Health, 35, 337–345.

Research and Evaluation at Gay Men’s Health Crisis in New York City where he

Lo, W., MacGovern, T., & Bradford, J. (2002). Association of ancillary services with

oversees the research and evaluation of program services. Dr. Feldman’s recent projects

primary care utilization and retention for patients with HIV/AIDS. AIDS Care,

have included a community needs assessment to inform the development of an HIV

14(Suppl. 1), S45–S57.

prevention program for young gay men of color and an evaluation of the mental health

Lasser, K. E., Himmelstein, D. U., Woolhander, S. J., McCormick, D., & Bor, D. H. (2002).

services program. Dr. Feldman also has over 10 years of clinical practice experience

Do minorities in the United States receive fewer mental health services than

working with a variety of populations, and he is an Adjunct Assistant Professor at the

whites? International Journal of Health Services, 32(3), 567–578.

Columbia University School of Social Work.

Magnus, M., Schmidt, N., Kirkhart, K., Schieffelin, C., Fuchs, N., Brown, B., & Kissinger, P.

J. (2001). Association between ancillary services and clinical and behavioral out-

comes among HIV-infected women. AIDS Patient Care and STDs, 15(3), 137–145. Gregg S. Weinberg M.A. is currently the Manager of Data Analysis at Health First in

Messeri, P. A., Abramson, D. M., Aidala, A. A., Lee, F., & Lee, G. (2002). The impact of New York City. He has over 15 years of professional experience in the development and

ancillary services on engagement in medical care in New York City. AIDS Care, management of outcomes monitoring and research and evaluation data systems in

14(Suppl. 1), S15–S29. medical, community based and for-proﬁt, private settings, speciﬁcally in developing

Ojeda, V. D., & McGuire, T. G. (2006). Gender and racial/ethnic differences in the use of and maintaining the technical, analytic and reporting infrastructure to support con-

outpatient mental health and substance use services by depressed adults. The tract and grant administration and outcomes monitoring activities.

Psychiatric Quarterly, 77, 211–222.

Orme, D. R., & Boswell, D. (1991). The pre-intake drop-put at a community mental

Elwin Wu Ph.D. is an Assistant Professor at the Columbia University School of Social

health center. Community Mental Health Journal, 27(5), 375–379.

Work where he also serves as the Associate Director of the Social Intervention Group

Parienti, J., Massari, V., Descamps, D., Vabret, A., Bouvet, E., Larouze´ , B., et al. (2004).

and the Co-Director of the HIV Intervention Science Training Program for Racial/Ethnic

Predictors of virologic failure and resistance in HIV-infected patients treated with

Minority New Investigators. Dr. Wu conducts prevention and intervention research on

nevirapine- or efavirenz-based antiretroviral therapy. Clinical Infectious Diseases,

offenders, deﬁned as those whose behaviors place others at risk for health and

38(9), 1311–1316.

psychosocial problems, residing at the nexus of drug abuse, partner violence, and

Pomerantz, A., Cole, B. H., Watts, B. V., & Weeks, W. W. (2008). Improving efﬁciency and

HIV. Dr. Wu’s experience also includes providing counseling to lesbian, gay, bisexual,

access to mental health care: Combining integrated care and advanced access.

and transgender communities, in addition to evaluating violence prevention programs

General Hospital Psychiatry, 30, 546–551.

and developing and evaluating programs for criminal justice-involved individuals.