Report of Usaid External Review Team on the L.Apang

REPORT OF USAID EXTERNAL REVIEW TEAM ON THE L.APANG (DEIDS) PROGRAM Review Team: Frank W. Mount, M.D., Dr. PH, Epidemiologist Michael D. Maurier, Ph.D International Systems Development Specialist Maureen A. Lewis, M.A. Economist Introduction This report was prepared for the Asia Bureau's, Health/Population/Nutrition Office of in AID in response to from a request the Director of the USAID Mission in Thailand for assistance in assessi.ng the progress to date of the Lampang Health Development Project and determining technical the and funding requirements for completion of the project evaluation and documentation. The Lampang Project was designed as a demonstration/ prototype health care delivery system. From inception project the design included an extensive evaluation component to provide inter-regional guidelines for AID.'s expanding programs in integrated primary health care services extension. It was realistically understood that the most important indices to be included in the phased evaluation plan could not be expected to show measurable change the until end of 1979 when all the innovative project services were in place, had achieved functional integration within the Thai health care system, and had begun to impact on the health status of the target population. The original plan provided for a seven-year project duration (FY 1975-81). Only the first-five years, which focused on implementation, were provided for in the project agreement. An extensive evaluation was indicated in the logical framework, but was not budgeted for in the original project. implementation As the phase approached completion, AID representatives advised the Lampang staff to develop a proposal to the complete evaluation called for in the original DEIDS concept. Accordingly, in September 1978, a combined MOPH/UHSPH olan and budget proposal outlining requirements completion for the of the project evaluation and documentation was submitted to AID and the American Public Health Association (APHA). In December 1978, the Lampang Project Director was informed by AID that an external review of the project would required be to guide AID in determining the future level support. of The Project Director was also apparently informed that a decision would be reached by March 1979. (See letter - Februapy 20, 1979, Dr. Somboon Vachrotai - Project Director, to Mr. Donald Cohen - AID Mission Director, Thailand, Ref. No. DEIDS/16/79). - 2- Various factors delayed the formation and deployment of the external review team until the meantime, mid-May l9('t. In having received in their view inadequate response from AID on their proposal approaching and given the funding cutoff date of Project September 1979, the Director expressed his desire involvement to terminate AID in the Lampang Project effective 1979. He October 1, also indicated his consequent view external review would that the therefore be unnecessary. (Letter, February 20, 1979, Ref. No. DEIDS/16/79). would have Termination precluded the completion of anything the most rudimentary except evaluation and documentation for the project, and would have rendered and useless the extensive costly preparations already completed staff for by the project an in-depth evaluation. In recognition facts and, of these as various documents clearly indicate, that the original concerned intent of the Lampang Project as a guide to - to serve future AID efforts to develop integrated low cost primary health care delivery assess their effectiveness systems and to - would be completedly undermined, AID elected to continue efforts to implement an external review of the Lampang project. A three-member team was selected professional with appropriate qualifications but without prior with or extensive involvement knowledge of the Lampang Project. briefing ir.Washington, Without D.C. the team commenced work in Thailand on May 14, 1979. Although minimal knowledge of the history, structure, achievements involved and personalities in the project ensured that Thailand the teai arrived in free from prejudgement, several spent days had to be on background research which might utilized have been better in direct observation of the project. Modification in Scope of Work Considerable care was taken by the staff of the Mission Health, Population and Nutrition Office and other knowledgeable health care officials in Bangkok team to avoid prejudicing the before it had direct contact with team the project. The was gradually given some of the Lampang background of the Health Development Project and an aaditional the request for two years of support to complete and documentation. evaluation The team was also given background information to relate the project to the Ministry of Health structure and to Public the delivery of health care services -3 in Thailand. A brief introduction to the nature and current status of the proposal submitted by the Lampang Project made it evident that the team was actually being asked to review the current status of the project, and to help decide whether it should be supported for another two years in order to evaluate its effectiveness and efficiency. In addition, the team was being asked to express judgement on the specific evaluation and documentation activities proposed, the optimum staffing profile required to implement such activities and the level of funding required. These questions were considerably different from those outlined in the scope of work which had been originally provided to the team. The scope of work essentially requested the team to conduct in four weeks, extensive research on the effectiveness and efficiency of the Lampang Project. This, however, was precisely what the project was realisticaily proposing to do during the next two years of continued observation, data collection, analysis and documentation. It was, therefore, unrealistic to expect that a three-member team staying for only two weeks in Lampang and two weeks in Bangkok could obtain enough data and information to assess the efficiency of the project, or to identify the changes which might possibly be attributed to the introduction of this new system of health care services. Given the structure and logistical realities of the project it would not have been practical, as an example, for the team to have reviewed enough records kept by wechakorns and by health post volunteers to estimate the extent of change in the referral of patients to health centers and hospitals over a period of time. In addition, because the project has been in full operation in one district only (Hang Chat) for two-and-one half years at the longest, and for much shorter periods in the remain ing districts of the Province, measurable changes could hardly have been observed and their causes clearly identified for such gross measures of effectiveness as the crude death rate, crude birth rate and infant mortality rate. Concerned at the apparent disparity between what the team had originally been instructed to do, the immediate needs of both the project and the Mission, and the lack of practicality reflected in the original scope of work, the team met with the Mission Director, Mr. Donald D. Cohen and with the Chief of the O/HPN Office, Mr. Vernon R. Scott to re-examine the purposes of the external review and the way in which the team should proceed. It was agreed between the representatives of the Mission and the team members that an assessment of the proposed project evaluation plan and of the capabilities and needs of the Lampang Project staff for the proposed two year evaluation and documentation cycle was what would be of most use to AID and to the project. All parties at the meeting agreed that this approach was much more appropriate than a hasty attempt to guage prematurely the effectiveness and efficiency of the project. They agreed that the team should proceed with the kind of in vestigation which would enable them to assist AID in expeditiously deciding the future of the project. Activities of the Team During the first portion of the site visit to Lampang the team was given an overview of the history and activities of the project. Extensive discussions were held with the senior members of the staff, Dr. Wilson, Mr. Rogosch, Dr. Pien, Dr. Chumnoon, and Dr. Jumroon. In addition the team was given an enormous quantity of documents relating to the past activities of the project - including quarterly and annual reportz.. data collection forms and procedures, reports of consultants to the project, an evaluation progress report and the proposal for the completion of the project. While assimilating this information, the team visited a number of different types of facilities in several districts in Lampang Province. The team interviewed all levels of health workers and observed many of them carrying out their duties. Special care was taken to examine the kinds of records which are routinely kept by the various health workers. (See the attached itinerary). The team visited Hang Chat District while the community survey team was collecting follow-up information, and later spent a day n neighboring Lamphun Province reviewing the organization and operation of the Rural Primary Health Care Expansion Project. The remaining time in Lampang was devoted to careful examination of the documents provided the team and to -5 intensive discussions with project staff members on specific aspects of their wcrk and the proposal The under consideration. team spent an afternoon with the Evaluation, project's Director of Dr. Pien, reviewing the evaluation so far work done and plans for continuing data collection, and analysis. processing The health manpower training program thoroughly reviewed was with Dr, Chumnoon in a session the following day. A full day was spent in conference with Dr. Pien and Mr. Wilson, Dr. Rogosch to review in detail the plans two-year continuation for the of the project, including issues relating to staffing and budget. After returning to Bangkok on May the 30, the team met with four consultants employed at the for Development National Institute Administration (NIDA) who will ing be collaborat with the Lampang group on data collection, and analysis.

Load more