APPENDIX B Syphilis Case Illustrating the Application of the Manual APPENDIX B

Appendix B

Syphilis Case Illustrating the Application of the Manual

THE SITUATION After analyzing syphilis morbidity reports and interview records, STD officials in the city of Chancri-La noticed an increase in the number of syphilis cases among men who reported having sex with men (MSM). From 1999 to 2002, the number of MSM cases had gone up, as well as the percentage of MSM cases. In 1999, there was only 1 MSM case, which represented .9% of the syphilis cases in males. By 2002, the number of MSM cases had increased to 14, and represented 29.2% of their male cases.

Further analysis revealed that the cases were not concentrated in one geographic part of the city, based on the males’ residences. However, through interviews conducted by the Disease Intervention Specialists (DIS), the STD officials learned that most of the males socialized in the same area.

ACTIONS TAKEN A DIS was already screening sporadically at a gay bar. To address this emerging problem, STD officials initiated meetings with six community-based organizations (CBOs) that work with the MSM community. Together, they designed a plan of action to implement jointly. One of the activities implemented was syphilis screening in different venues (i.e., bathhouses, gay bars, CBOs, mobile unit, and a gay parade). The STD director and program staff were interested in determining which of these screening approaches was more successful in reaching the target population. The following illustrates the steps involved in designing and implementing this evaluation.

Step 1: Engaging Stakeholders in the Evaluation.

1.) Who were the stakeholders in this scenario? • Implementers: STD staff, CBOs staff • Decision Makers: HD management and STD director • Participants: Representatives of the target population ( MSM) • Partners: Businesses (i.e., bathhouses, gay bars), parade organizers • State Laboratory

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 307 APPENDIX B

2.) How was stakeholders’ input obtained for the evaluation? The STD program staff organized a meeting to brief other stakeholders on the screening activities implemented and importance of conducting an evaluation of such initiative. Also to: • obtain stakeholder input, • determine stakeholder needs, interests, and concerns about the evaluation, • plan stakeholder involvement in the evaluation and what they hope to learn, and • plan methods of keeping stakeholders informed during evaluation.

3.) How was stakeholders’ involvement retained throughout the evaluation? Stakeholders’ roles and responsibilities were discussed and agreed upon. All stakeholders reviewed documents pertaining to the evaluation (e.g., evaluation plan, instruments, analysis, and report), and decisions were made by consensus. The STD staff agreed to send a monthly email summarizing the progress of the evaluation as well as STD-related information affecting MSM to all stakeholders. In addition, stakeholders participated in a monthly meeting through the end of the evaluation.

Step 2: Describing the Program

Considering the importance of mutual understanding and that the evaluation involves individuals who may be not be familiar with the STD program and the screening activity, the STD program staff shared some background information with the stakeholders at one of the monthly meetings. The project area had been involved in needs assessment activities when developing the Comprehensive STD Prevention System (CSPS) grant application and shared information on the syphilis outbreak among MSM as well as behavioral data. They also reiterated that the purpose of the entire STD program was to address the STD needs of the project area, with emphasis on MSM. Also, the STD director provided the following information about the screening activity to be evaluated.

1.) What was the goal of the screening activity? • Reduce syphilis in at-risk MSM living in Chancri-La.

2.) What were the screening related objectives? • By December 2002 the STD program staff will implement syphilis screening in 4 venues frequented by MSMs. • By December 2003, the number of at-risk MSM screened for syphilis will increase from X to Y.

308 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

3.) What were the resources? • HD staff • CBO staff • HD $ • CBO $ • Access to four venues • Screening equipment and supplies • Mobile Van • Laboratory services • Condoms

4.) What activities were being conducted with those resources? • Training of CBO staff to provide information on syphilis screening • Monthly screenings in 3 venues (bathhouses, gay bars, and through mobile van). • Screening at Gay Pride parade • Distribution of condoms • Request for assistance from local businesses frequented by MSM (e.g., permission to park mobile van in their parking lots).

Stakeholders wanted to better understand how these screening components were going to fit with other STD program activities and lead the way to the results they were expecting. They decided to develop a logic model of the screening activity (see Logic Model on next page).

Step 3: Focusing the Evaluation Design

Once stakeholders understood, via the logic model, the connections between the screening activity components and corresponding outputs and outcomes, they focused the evaluation by determining the uses and users of evaluation results, the questions they wanted the evaluation to answer, and the evaluation design(s) to be applied. The following shows the decision making process.

1.) What is the purpose of the evaluation? Stakeholders agreed that they wanted to gain insight on the screening activities implemented and determine which venue(s) was the most successful in reaching the target population (i.e., at-risk MSM).

2.) Who will use the evaluation results? All of the aforementioned stakeholders, most particularly the STD program staff.

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 309 APPENDIX B e c n e d d i n k c a . s g i n s i r n s s i D r l o i d d T o L e e i h S c c v p r u u a y e s d d h h e e f t e R b R o o • • m s o D s s e d d S - T e . t e t n s n S c E c a a i c i a e o v d M c a u d r g o n n n e t e O n d d o a s e i M o C e e t m v l s S s s M T i r n m l o a a S i 4 e r e U M a e e t t h v r r M n O k n e n p e i a c c s I f s r o y i n n L d I o s p c r I u - e i t • • r a c i t i n M n S i g a y f t n M i h g i o g n f n n k C i f f e n u e s s n a o i o t e t r u m r e n r - s s i m c e t o s m d d r a s a e h O c e e o k s s n s s S S c B i i g l l a e a a . i i C n r y e e e h h M o t a r r i d r S p p c c n n m w y y b n n I s a s a I M a u t • • m u m s O d l s o r y n c a n d g g : g 2 s a a a u e i o n n n n b i i i i r l i a n h G d M t e i t d i t e n n - n y o r s l i S S a s d v t e n i n v e e e s h n i a e i a c f e e T s n r n e l n l w o e e f e a M d g s u o d r r p a a r w e a e e a U l v a d n u m c c n d d a c b t e n i d n B d y P t s s e r s s s i s m s r o e e s n e l t o n l a T t t y a g S c e u y i d g h a i s s l m s a s u c c a e s t i i d i i p n i g a e o U O n h l l b l : r h o i l r i b u u i t i i e a e i t i t l t s a t i r B o G d t e e O c c r d d n h h h h a h e M n n t i a e t O C a t n d e d u n n p p p e S p i M 2 B a o a v f s a e B o r i n t y y d y d o o r y o r n s c C m M T t s a S c – S m – – a M i P t m e C d o f • • • • • • o M . s k e y s g c a n a d e t i g n e s o a g i r n n r a g i f n e b d a c f f S n t e o n p a o E m e s s n u e t I e i L a r l r e l e a s e s n T O i n r c a a v g I e d u o h : s c h o i n B i a s O i t s V s o r e t n i p l t t o I l n u B y i h P i y m c u o a s u T t i a w i o i s a C h b o u b l t r b y a i t i C h t i t o G m d d e a r c a r n a h h e t t A i a t n n u t r u s e p G 2 M B a : s t a e i o o i a a i d y r r n S m e C d D T i – – – s T – C M b b • • • • • • s e i l p p u S l s e n t a T n y a g r a n o s U s V i i n l o g o r i t P t i m e a n p n t a l i i a i o N r c e r o a f I d i r b d e f c e o t n d r p o n a u b e a c t u p o d a F S M S A m E m L C • • • • • • • •

310 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

3.) How will the evaluation results be used? It was determined that the results of the evaluation would be used to reduce/expand the screening activity locations.

4.) What questions do you want the evaluation to answer? Stakeholders submitted all possible questions they wanted the evaluation to answer which included: • Was the screening activity implemented as planned in the different venues? – What are the barriers and facilitators in carrying out syphilis screening in the different venues? • Which venue(s) is(are) more effective in reaching and screening at-risk MSM? – Which venue is more acceptable for syphilis screening among at-risk MSM? – How many MSM were screened, by venue? What was the number of new positives found, by venue? If not as expected, why? – Where should screenings be conducted, and when? • Where should condoms be distributed? – Were the condoms distributed to the establishments where cases are found? (Right number and to the right places.) – Were these the appropriate places to distribute condoms? • Was the number of cases reduced to the degree planned? • Did awareness about the syphilis outbreak increase among at-risk MSM and CBO staff? • Did awareness of prevention measures among at-risk MSM and CBO staff increase?

5.) Since results of the evaluation needed to be submitted within 3 months, and the STD program and CBOs did not have the resources to answer all these evaluation questions, stakeholders decided to focus on the following group of questions by their level of importance. The questions were also classified as either process or outcome. • Was the screening activity implemented as planned in the different venues? (process) – What are the barriers and facilitators in carrying out syphilis screening in the different venues? (process) • Which venue is more effective in reaching and screening at-risk MSM? – Which venue is more acceptable for syphilis screening among at-risk MSM? (process) – How many MSM were screened? What was the number of new positives found? If not as expected, why? (process) – Where should screenings be conducted, and when? (process)

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 311 APPENDIX B

6.) Which evaluation design is most appropriate to guide data collection for the evaluation questions given the available resources (budget, time, staffing)? Since the purpose of the evaluation was to make programmatic decisions about the screening venues as opposed to 1) determining the effects of the screening activity in the target population or 2) if these were due to the screening activity, quasi-experimental and experimental evaluation designs were out of the question. So stakeholders, along with a professional evaluator from the health department (HD), selected non-experimental and qualitative designs to guide the data collection process pertaining to the evaluation questions.

EVALUATION QUESTIONS DESIGN RATIONALE

• Was the screening activity Qualitative • Used to record (observe) screening activities as they implemented as planned in the Design occurred in the four venues and to determine if they different venues? were implemented with fidelity.

• Used to obtain in-depth understanding of perceived • What are the barriers and factors that either hindered or facilitated the facilitators in carrying out syphilis implementation of syphilis screening in the different screening in the different venues? venues among implementers and business owners.

• Which venue(s) is(are) more This question depends on the next three questions to be answered effective in reaching and screening at-risk MSM?

• Which venue is more acceptable Qualitative • Used to obtain the opinions of a sample of individuals for syphilis screening among at- Design at the screening venues regarding factors that risk MSM? motivated them to accept being screened in the venue, thoughts re the other venues, other venues that still • How many MSM were screened? need to be reached. What was the number of new positives found? If not as • Used to count how many MSM were actually screened expected, why? per venue and how many of these were active cases of syphilis. • Where should screening be Non- • Used after the screening takes place to determine conducted, and when? experimental where and which time/days of the week received the post only highest number of at-risk MSM being screened. design

312 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

Step 4: Gathering Credible Evidence

Since all the evaluation questions measured process of the screening activity, stakeholders reviewed the logic model to identify corresponding outputs. Then, they selected the indicators to measure progress of the syphilis screening activity in the different venues, where/from whom data would be obtained for each indicator, and the corresponding data collection method(s). The following reflects the decisions made accordingly.

To help maintain confidentiality of respondents it was agreed that (1) data collectors would strip all identifiers from the data gathered (observation logs, interviews, focus groups), and (2) secure it in the Evaluator-HD’s office.

Stakeholders organized all the decisions made up to that point and developed an evaluation plan consisting of a narrative component (stakeholders, rationale, purpose, goal/objectives to be addressed in the evaluation, logic model, users/uses of the evaluation, dissemination approach, timeline, and budget) and a matrix (evaluation question, design, indicators, data sources/methods, person responsible and schedule).

Then, the evaluators (HD/CBO) and STD staff drafted all evaluation instruments and protocols, gave these to other stakeholders for their input, and incorporated changes. Instruments were also pilot-tested.

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 313 APPENDIX B d . , , s s s e s e d y n n u f s n c m , n o o i S r r s i i s t n p e I a t t ) e i n u o n i u h t , n n n S r t w p p 5 o t t i i s e h o c o o o e s r r Y n t r a i e d i a p L c c i P c v g p m m m a . o s s r r u s ) w t E e s A d n n c y e u d m m m o t e n a v T t r n i r a a d n N o n o o o n v d t r r i n a g S e s n c c a c e a t t i t t a A . , , y a t t d p s y y y y a s s s i E i b f f f f a t n w w w t e s r e e e i i i i A E a s c t t t t p e e e o c n T i i i r m m m m s S d o n n n n p a v v v y ( r u A e e e e e s e e e e u e e e y c n o e h h h h d d d d d D i b i t r i t a a s R ( t R R i t Q • • • • e e r y h h b t c l g S a n a n i E e E o n r i L r r R f u y y y U o e ; d U b b b f t n D y y y y s n D d d d d o E y y y y e n i / / / / e e e e E t H o t t t t i m a m m m m C c c c c C s e u e e e e l l S a l l l l m m m m O l l l l / / / / p c a R o o o o c d d d d v m P C o e d C i d C d C d • • • • N O E I L T e e B D D I n n C S H H o o E N L m m m m L O o o o o s s P r r r r O f f f f S O O C r r r r E B B o o o o R C C t t t t A a a a a T e e N u u u u A l l l l h h t t O a a a a D S f f v v v v E E E o E o R E • • • • P N O ) - - I g s n n T o u e e l C D c ( p p o E o o O f n / / L ( l l o H i L a a t T w w w u u a O ) e e e E d d ) ) s v i i i i i C r d d v v v p v v M r r r i i e e e u s e e e A d d d d o t t t b T n n r n n i i n n n e I I g e ( O I ( A D • • • • e g e r l d n i a , p n ) ) r s E s s a y r u m ’ e r , e e C e s s s d a e h u r r r h n t s ) s R k t ( n r e e e e a t n t t t w c e U o s s s s a ) o t n n n l l l o v n i m a c O g e e e t a a a a t d s e n a n S u u u s n r i e m m m m M i v o d d d e e r n n e e e r i i i i S d l l l r A n s o e e e v v v i i e p p p f i i i T s e e M f s r c D r r d d d f b A m m m f e u e i T c c i i i n n n O ( I p s I ( S ( b d I d o s D • • • • f f e . o o e y s d e a r d t t d r s e i a e n n n m r a e n e i v i r s o e h i o a i o o u t f m e t u r t i i , e t u i S g h t t d s t s h g n o c s e g o s n r g r r t R e a a i S i f n w e r n a e M h n t t e n i e c e . e n v e t i e S t S O g . n e s s n n e n n o h g i l k e h s d r r n . t n n l T r E t e e e i h X t e t n d e s a M e e e e m w e y t n y i a a p I h r f e A t t C a e n s n e u m m t c d o o r e b n . h h c % m d r o r n n m h n r r a u e e R g C o c n n s c c O n i e d t f l l 0 c e s o I d e e e e r d s n e n e n t l t a e s s e T f f s f p p 0 i s R e n t y s e e s r r e v t f i t a o D p a m m e t r o o t i l 1 s n r i o c e e e b P v i e t A i i v m m r w p i e e i i n s o e i e i s i d a l l N m v s u l u h i i e e r h r e t o o t h n i t i i m I e e t r l s i p p c t n n n t c c p p s e e e e r p l u r c p o i M u a p u y e c c e e y y o a c y o h h h e h h o a o m t m f N i s B w a f i c b t v b f T d t t T i s F t r m a t v o t N • • • • • A L g g ? P r n n s i i i o M y N f n S r S e e y g ? N r e t O e h s N u s n i a e l M I t i t r e n v c a O b e O i T n c n I k u e e I t a h s e r n d s e v e A t n c t i i T T r e d a e r s h e p a r t U e i h - S s t n n t l v A e f c t r i c i n L f a g E c a i s s i t a o e h U c ? n d A h h t a s d n i s U p s a r i g e L a m e V l n y e t W W W e e i r Q n n e i i s e e E l l u A h e h r r o n i . . . e f t t r p n p r c o a f 1 2 3 V u l a i m e c y a n m Q v s Q i Q i b f o p d m s a E

314 PRACTICALPRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B s m g o n r i S e f d I ) n h d n t s S e e 5 e a g e t h r Y d r t n L a s a P i c e f e p t t s d E A . o c u e n f 4 T m i e s N n o g f o S Q p e s A n c r x , . v r i e y E e e s 3 e n e t s A b i E u b s n b Q T i v l h S o i l m , l m g l t ( r t A i i i u 2 u e c c D W Q N w w a a b n • • r e h d t 4 7 n h e o c a d n n S i a i i e r e E E e h h h P d t t L t e f R i i y y a r y U h o r U b w w b t o a a n D f y y f D d d d d G p o E y y i o y / / e e e e E t f f l H t t t t s e a o o h h m m C c c c c e C t t d u e e e e u s s l S l l l l m m O n n a l l l l / / y y r n a o o R o o o o d a a d v a e P C m d e C P C d d v m C d • • • • N O E I L T B D I C S H E N L m L O o P r O f S C r E o R t A a T N u A l O a S D I S v D E R E • • P N w w O e e i i I v v T e e C D R R E O s s L t t H L n n T e e O E m m C u u M c c A o o T D D A • • D , , g g E o o C l l R s s r r U e e ) ) t t s s s s O t t n n d d S n n e e r r e e o o m m c c A m m e e e e l l T u u r r p p A c c b b o o m m D i i a a l D l D ( ( • • , e s d s e g t , r b e s h y s n e t a . l t i s e e c o i s h s b . t h l n t t g u u i e S g t e M n . e f y n v n t t n d h n n S d d i o R i n e r S i o a i a a e t n p r o e a n a t i u d g S r r v M O y c o l e t e s v a m s s , e a c T E r e n M M e y o s d i b e h p r f f f e S S l A s C l b p y e n c l c i o o o h i i s g m e e s ) . C t . b M M t O d r r r h I u n s u w d s d e f o i o s i e ( m e e e R o i e g g n r e v w o D l n e t d i b b b h i h m n i P n n p m h t t e u u s c i e N v h h s c o o i o t m m m e s I d d n y t o p p o r a c s o h u u u e a a r o n n m m b y y c e e e s a a N a g b m r N N r a a T s w t p V p • • • • • ? t n ? a M e d S h f , h n N d S o d l u M W w e r O u N o t I y ? f e o d c O n T d b h n s e I e a s a A e e p m T n v , b x m e U u i S e r d t e n L i e g E e e r w s s t n A h e c i U o o c a s h V n p u H W t t Q ? e E e d o y s r . . e w n n r h e a 4 5 e o c f s Q c w Q w I w n

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 315 APPENDIX B

Step 5: Justifying Conclusions

While data collection was taking place, stakeholders met and determined how the data from the indicators were to be analyzed. The evaluation plan was revised to include the data analysis process (as presented in the last column of the previous table), and the schedule and person responsible for conducting the analyses. The following illustrates the main findings of the evaluation, organized by evaluation question and corresponding indicators.

Evaluation Question: Was the screening activity implemented as planned in the different venues?

INDICATOR SUMMARY OF FINDINGS

• Number of implementers • Observations of all 7 staff screening individuals revealed that most who followed the screening of them (i.e., 5) followed the screening procedure all the time in the procedures with 100% four venues. It was also found that the 2 staff not following the consistency in the four procedures were relatively new, not only to STD, but to the screening venues activity and protocols. Due to time constraints of the STD field supervisors, the training received had not included practice sessions.

• Type of changes made to • Interviews with implementers and STD director indicated that in the the screening activity in the past year, all the monthly screenings were held at the bath house, four venues from the time it but only for the first 6 months at one of the bars (because it closed), started. and only three times at the second bar. Monthly screenings were held every month in the mobile van, but not in the locations they hoped for. Screening was held all day at the Gay Pride parade.

• Mobile van locations had to change twice because of complaints from neighborhood residents. Two locations that had been chosen originally had no parking available for the van and were removed from the list.

316 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

Evaluation Question: What are the barriers and facilitators in carrying out syphilis screening in the different venues?

INDICATOR SUMMARY OF FINDINGS

• Barriers and facilitators • Screenings needed to be held at night, and it was hard to get staff identified by implementers, to work those hours. business owners and decision makers re the • STD program staff needed commercial driving licenses to drive the implementation of the mobile van; only one staff person had that license. screening activity. • When interviewed, the bar managers expressed a fear of revenue loss when patrons were away from their barstool, or tables, to getting tested. They also feared poor bar attendance if the screening events were advertised, since this might keep some patrons away. One bar closed halfway through the year. So, even though they had agreed to participate, “something” always seemed to come up on the night the screening was scheduled, so it had to be cancelled.

• In general, MSM claimed they were more interested being tested for HIV than for syphilis because HIV status was more important than syphilis, and they did not believe syphilis was present in their community.

• Having insufficient time to create attractive materials for the Gay Pride Parade to encourage MSM to be tested for syphilis.

• Facilitators included: (1) each facility having a room that was private, and could be used for screening, (2) having a contact from one of the CBOs work with the organizers of the Gay Pride parade to allow advertising and testing for syphilis, and (3) the bathhouse manger encouraged participation in the screening and advertised when the screenings would be held.

• Type of challenges re the • Gay bar owners feared that their clients were going to identify their implementation of syphilis locales with infections or consider it a “a dirty place” and lose screening at the different clients as a result. venues reported by implementers. • The mission of the gay bar was socialization; to introduce screening for a sexually transmitted disease was not compatible with their mission.

• Lack of knowledge and experience of half of the screening staff with the MSM community.

• Getting permission to draw blood at a public gathering (Gay Pride).

• Neighborhood complaints about the noise that the van produced resulted in much staff time being spent responding to complaints, and to relocating the van.

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 317 APPENDIX B

Evaluation Question: Which venue is more acceptable for syphilis screening among at- risk MSM?

INDICATOR SUMMARY OF FINDINGS

• Factors that motivated MSM • Results of interviews with MSM indicated more willingness to be to accept screening in a screened for syphilis at the bathhouse than at the gay bars. Since venue and their opinion on there is more sexual activity going on in the bathhouse than in the the other three venues. bars, they said they feel more at greater risk for syphilis and other STDs.

• Previous syphilis infection or knowing someone who had syphilis was another motivator.

• Ease of access/quickness of both the screening test and test results.

• Gay Pride testing was good for visibility; however, most MSM surveyed there declined testing if it involved waiting 30 minutes or more.

• Important to have a consistent schedule for mobile van so that clients could locate van easily to obtain results.

• Type of recommendations • Interviewees suggested having screening activities or arranging it provided by MSM about with those who have “circuit parties”. other venues, which still need to be reached. • Another suggestion was to include an ad in the local gay newspaper and in gay websites about the syphilis outbreak and where to be screened/treated.

318 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

Evaluation Question: How many MSM were screened? What was the number of new positives found? If not as expected, why?

INDICATOR SUMMARY OF FINDINGS

• Number of monthly syphilis • Bathhouses: 250 men approached; 150 screened screening among MSM at bathhouses, gay bars, and • Gay Bars: 500 men approached; 150 screened mobile van. • Mobile Van: 1000 men approached; 300 screened

• Number of syphilis • Gay Pride: 200 men approached; 30 screened screenings conducted among MSM at the gay pride parade.

• Number of screening tests • Bathhouses: 5 positive which turn positive. • Gay Bars: 2 positive

• Mobile Van: 1 positive

• Gay Pride: 0 positive

Evaluation Question: Where should screenings be conducted, and when?

INDICATOR SUMMARY OF FINDINGS

• Venue(s) yielding the most • Highest number of at-risk MSM tested: Mobile van number of tests and positives. • Highest percentage of active syphilis cases: Bathhouses

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 319 APPENDIX B

INTERPRETATION OF FINDINGS Stakeholders received these findings and met to interpret them. It was concluded that the implementation of the screening activity was facilitated by: • Having most of the screening staff follow the screening protocols. • Having available private rooms to conduct the screening at each venue. • Partnering with a proactive bath house manager (who agreed to advertise screening). • High self-perceived risk for syphilis among bathhouse clients. • Previous experience with syphilis among MSM. • Ease of access/quickness of both the screening test and test results. • Being visible at the Gay Pride Parade. • Increasing access of at-risk MSM to syphilis screening via a mobile van.

There were factors that affected screening implementation such as: • Pre-planning issues – The need for more training on the implementation of screening protocols for new staff. – Van locations with no parking available. – Limited number of staff with commercial driving license to drive the van. – Competing demands among screening staff, making it difficult to work after hours. – Neighborhood complaints about the noise produced by the van. – Lack of attractive advertising materials.

• Business limitations – Gay bar being closed. – Fear by gay bar managers having their business being perceived as “dirty” if STD testing was necessary. – Conflict between aim of bar (socialization) and distracting public health activity. – No time of gay bar managers to advertise screening.

• Target population’s low-perceived risk for syphilis and lack of awareness about the outbreak among gay bar clients, and having to wait more than 30 minutes to be screened at the parade.

320 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS APPENDIX B

RECOMMENDATIONS Based on the findings the following were recommended:

• To conduct booster sessions on screening protocols with all screening staff and couching from field supervisors with new staff.

• Continue using the mobile van for syphilis screening to reach at-risk MSMs with the following recommendations: – Before using the van in residential areas, obtain permits in advance to locate the van. It is important to meet with the neighborhood leaders to make them aware of the magnitude of the outbreak and the importance of conducting screening. Build a relationship with them to gain access and acceptability into the community, and request their input on where/when to place the van. – Increase the number of screening staff with commercial drivers licenses by given those interested time to obtain the training and license and incentives for doing so (e.g., acknowledgement at staff meeting). – Have a consistent schedule for mobile van so that clients could locate van easily to obtain results. – Make sure that the waiting time for screening is less than 30 minutes.

• Keep strengthening the relationship with the bathhouse manager so screening activities can continue.

• Since gay bars do not seem to be the most successful places for syphilis screening, keep providing them with prevention materials and explore other venues such as “circuit parties”.

• Develop monthly schedules in advance, including the exact times in which screening activities will be held, so that screening staff can make arrangements to work after hours, if needed.

• Consult with the communication or health education specialists within the health department and CBOs to develop attractive material to advertise screening times/places in the gay media and establishments, as well as places that MSM tend to visit.

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 321 APPENDIX B

Step 6: Sharing Lessons Learned and Ensuring Use of Findings

The evaluation findings were shared with pertinent audiences and some of the evaluation recommendations have been implemented by the STD program and other stakeholders. The following shows who received the evaluation results and in which format, how the STD program ensured that the evaluation results would be used for decision making, and which decisions have been implemented.

1.) Who received information on the evaluation results and in which format? • HD and STD director (executive summary and full evaluation report) • STD program staff (executive summary and oral presentation) • CBOs (executive summary and oral presentation) • MSM Leaders, represented on the Stakeholder group (oral presentation and fact sheet) • MSM Community (fact sheet) • CDC (oral presentation at the National STD Conference) • NCSD (executive summary, fact sheet) • Businesses (i.e., bathhouses, gay bars) and parade organizers (oral presentation, executive summary and fact sheet)

2.) How were stakeholders kept informed on the evaluation? • Regular monthly meetings • E-mail • Final report

3.) What steps were taken to ensure use of the evaluation findings? • Stakeholders helped draft recommendations based on the evaluation findings. • STD director proposed recommended changes to HD management, MSM leaders, and CBOs. • Follow-up meetings were conducted with those who can make decisions regarding the implementation of syphilis screening in different venues.

4.) How were evaluation findings used? • Day and times of mobile van screening were adjusted to meet increased demand at peak times for two venues. • One venue was discontinued as a result of the analysis of volume of positive test results (i.e., gay bars). • As a result of discovering that the mobile van driver needed a commercial license, the STD program identified several staff willing to drive the van and arranged commercial driver training for those staff. Four staff subsequently received their commercial driver’s license. • The STD program revised the plan to incorporate meetings to advise local law enforcement about the mobile van activities.

322 PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS