Research Division Population Services International 1120 Nineteenth Street, NW, Suite 600 Washington, D.C

Research Division Population Services International 1120 Nineteenth Street, NW, Suite 600 Washington, D.C. 20036

Concept Paper

TRaC-M: A Social Marketing Tracking Survey for Monitoring Exposure and Logical Framework Indicators

PSI Research Division 2005

©Population Services International 2005

Contact Information Virgile Capo-Chichi Steven Chapman Population Services International 1120 19th Street, N.W. Suite 600 Washington, DC 20036 Tel: +1 202 785 0072 Fax: +1 202 785 0120 Email: [email protected] Email: [email protected]

INTRODUCTION

This concept paper describes TRaC-M (Tracking Results Continuously – Monitoring only), a method for monitoring exposure to social marketing activities and logical framework indicators efficiently using lot quality assurance sampling. TRaC-M is not a replacement for TRaC, which produces the three population-based tables of the PSI Dashboard (Patel and Chapman, 2005). TRaC-M complements TRaC by providing a low cost mechanism for producing the monitoring table in between TRaC surveys for purposes primarily of monitoring exposure to a campaign and for measuring logical framework indicators at a sub-national level. TRaC-M sample and questionnaire sizes are smaller than those of TRaC and the analysis of TRaC-M results does not require a researcher. The cost of TRaC-M is not currently known, but preliminary estimates are that it is likely to cost not much more than a MAP Phase 1 survey – approximately $5,000.

PSI Research elaborates concept papers from time to time with two purposes in mind. One is to present for discussion and review a new concept or method to members and stakeholders of the Research Division. Two is to describe the concept for purposes of conducting pilot initiatives. This concept paper begins by identifying the role of TRaC-M within an initiative to monitor a social marketing intervention and then describes two elements of a TraC-M study design (questionnaire development and sampling strategy) and presentation of results. Comments and questions about the concept are welcome by the authors.

TraC-M vs TRaC: WHICH ONE IS USED WHEN?

TRaC-M is not a replacement for a TRaC survey (Chapman and Coombes, 2003). The objective of TRaC survey is to segment, monitor and evaluate social marketing interventions as frequently as annually for purposes of marketing planning and annual stakeholder reporting or, if funding is inadequate for that, at the beginning and end of a project.

TRaC surveys have sample sizes of at least 400 and usually between 1,000 and 2,500 depending on definition of risk groups, whether the survey is stratified, for example between males and females, or urban and rural, and differentials and expected changes over time in key indicators. There are two primary advantages to these large sample sizes.

2 1. They permit the analysis of correlations that are the basis of the segmentation and evaluation tables of the PSI Dashboard (Patel and Chapman, 2005).

2. They permit the measurement of statistically significant trends in the monitoring table.

These properties are highly valuable to PSI; TRaC-M is not a substitute for them.

However, TRaC surveys have two disadvantages. The first is that segmentation and evaluation analyses are not needed as frequently as monitoring information. Segmentation and evaluation analyses at frequencies greater than annually are of little to no value in decision making terms. Monitoring information is often however more frequently needed, particularly after a campaign has been conducted to measure exposure. TRaC’s problem is that it is too expensive for frequent monitoring due its large sample sizes and longer questionnaires.

The second disadvantage of TRaC surveys is that monitoring exposure and logical framework indicators on a sub-national basis, for example by region or province, can be valuable, but highly costly to obtain through TRaC, since getting these estimates inflates by a factor of at least two the sample sizes of TRaC.

TRaC-M is designed to overcome these two disadvantages. TRaC-M produces primarily monitoring information, particularly indicators relating to the reach, frequency, intensity and duration of exposure to social marketing campaigns close to the time of campaign implementation when a TRaC survey is not otherwise scheduled and when segmentation and evaluation information is not needed (Patel, 2004). For example, say that you have started a project and conducted a TRaC survey. Four months later, you have an expensive communications campaign underway and you are curious what its reach, and frequency, intensity and duration of exposure is to make decisions about your media mix. TRaC-M would be the appropriate monitoring tool, since it would measure those indicators of primary interest and not indicators that have no immediate value in decision making terms.

TRaC-M overcomes the second limitation to TRaC by being designed to monitor logical framework indicators at purpose and output level on a sub-national basis. Say that your logical framework indicators state that you seek to increase availability as measured by the proportion of respondents that say that they can find a condom within 10 minutes of their house. Your baseline TRaC survey and segmentation dashboard produce evidence that perceived availability is correlated with use. You conduct a MAP (Measuring Access and Performance) survey to

3 monitor your product delivery system and find that your product coverage is higher in some provinces than others (Chapman, Capo-Chichi, Longfield and Piot, 2004). You decide to allocate resources to those provinces where coverage is low in an effort to increase it. TRaC-M would then monitor whether perceived availability in underperforming provinces is increasing to levels achieved elsewhere. Managers in the private sector refer to this type monitoring analysis as benchmarking: the setting of standards of performance excellence and systematic comparison of underperformers to excellent performers.

TRaC-M’s efficiency and limits relative to TRaC are derived from the sampling strategy that it uses. This strategy, which is described below, is the same strategy used in MAP. TRaC-M can and should be implemented simultaneous to MAP and at low additional cost.

SAMPLE AND QUESTIONNAIRE SIZE

TRaC and TRaC-M differ in terms of sample and questionnaire size. TRaC sample sizes are calculated based on planning assumptions made by PSI social marketers. For example, if a PSI social marketer promises the donor in the logical framework that behavior will change by a large amount, say 10 percent over the course of the project, then sample sizes are relatively smaller than they would be if the social marketer promised only a small amount of change, say less than five percent. The reason for this is the formula used to calculate sample size. The key element to remember is that the smaller the difference to be measured, the larger the sample size.

TRaC questionnaires measure behavior, “bubbles” or determinants related to opportunity, ability, and motivation, population characteristics, other logical framework indicators that may be of project, national program or international relevance, and exposure to the intervention (Chapman and Patel, 2004). At a minimum, this requires approximately 100 questions per risk group and behavior to be changed. As the number of risk groups and behaviors increases, the number of questions in the questionnaire increases too.

TRaC-M, like Project MAP, uses lot quality assurance sampling to determine the sample size of persons required to be interviewed for monitoring purposes. In Project MAP, as few as 19 geographical areas can be randomly sampled and audited to determine coverage – the proportion of geographic area in which a PSI product or service is present (Valadez, Weiss, Leburg and Davis, 2001a & 2001b). TRaC-M, if implemented at a national level only, need only interview 19 persons, randomly selected, to determine whether exposure – the proportion of the population that has seen, heard, etc, the social marketing campaign is above a certain level, say 25 percent.

4 Such information when compared to the exposure objectives of a social marketing campaign provides direct and actionable evidence for decision making relating to the marketing mix.

The small sample of TRaC means that collecting data on a sub-national scale, for example by province or region, can also be done cost-effectively. For example, if the social marketer believes that exposure may differ by region, and the country has four regions, then the sample size of 19 is multiplied by 4; by interviewing 96 persons, exposure by region can be reported.

Lastly, TRaC-M does not measure “bubbles” or determinants of opportunity, ability and motivation or produce tables disaggregated by population characteristics. This results in smaller questionnaire sizes focused exclusively on exposure and logical framework indicators.

EASE AND SPEED OF ANALYSIS

The use by TRaC-M of lot quality assurance sampling results in the same ease and speed of analysis benefit as MAP: analysis and production of results is immediate and does not require a researcher with statistical training. A researcher however is required for creating the system by which the sample is drawn and for elaborating the questionnaire.

METHODOLOGY OF TRaC-M

Designing a questionnaire

The TRaC-M questionnaire can be as short as a single question: Have you heard or seen advertisement X? However, given the time and expense required to travel to the randomly selected individual (who could be anywhere in a given country) and given the objectives of a communication campaign, additional questions should be asked to the respondents.

Those questions must be, for purposes of TRaC-M,

1) phrased to elicit a yes/no response or phrased so that the result can be translated into a yes/no response,

2) be applicable to all respondents (skip patterns are not allowed in TRaC-M surveys), and

3) be consistent with best practice in measuring exposure and logical framework indicators.

5 Historically, many PSI questionnaires have had questions that ask, for example, “why did you not use a condom” to which relevant respondents were offered a list of reasons, such as 1) condoms are too expensive, 2) I trust my partner, etc. These type questions have two fundamental problems for TRaC-M. The first is that lot quality assurance sampling requires yes/no responses in order to produce valid results. The second is that a yes/no response is required from each of the 19 persons randomly selected. As such, a question such as “why did you not use a condom”, since multiple responses are possible and because it is only asked to the sub-group of the 19 that did not use a condom, must be avoided.

This limitation to lot quality assurance sampling is not however significant with regard to exposure questions or to monitoring most logical framework indicators. For example, exposure questions can be as simple as:

Did you hear that advertisement about X on the radio?

This question is used to measure reach – the percentage of the population that, for example, heard or saw an advertisement, or participated in interpersonal communication session. This question can also be used as one of several to measure intensity of exposure, or the number of channels through which an advertisement was heard or seen. Here, if the advertisement has been heard on the radio, that then means that the intensity is one channel. If the next question is whether it was seen on television, and the respondent says yes to this too, then that means that the intensity would be two channels, etcetera.

Exposure questions can also be stated in ways that later can be turned into a yes/no question. For example, frequency of exposure is measured often in terms

How many times did you hear/see the advertisement?

If the responses are:

1. Six of 19 never saw the advertisement

2. Five of 19 saw it once

3. Four of 19 saw it twice

4. Three of 19 saw it three times

6 5. One of 19 saw it four times

Then, a variety of yes/no responses can be formulated. For example, did you see the advertisement at least once? Answer: 5+4+3+1=13. Did you see it at least twice? Answer: 4+3+1=8. Duration of exposure questions can be formatted similarly.

Logical framework indicators are more often than not easy to put into yes or no questions. For example:

Did you use a condom the last time you had sex?

Do you use a condom every time you have sex?

However, these logical framework indicators are sometimes made conditional upon another behavior, such as having a casual partner, in which case measurement of the indicator is not per se able to be done among the 19 persons interviewed. Remember that an answer is required from each of the 19 respondents; if one or more of those for example has not had a casual partner, then TRaC-M can not produce a measure for this indicator by simply asking 19 persons the question. Instead, 19 persons would first need to be identified that have had a casual partner, a possibly significant complication and source of increased cost for TRaC-M. Your research backstop can brief you on your options if this is the case with your logical framework indicators.

Sampling

TRaC-M requires random samples of 19 persons, either nationally, or within each sub-national area. Therefore, the key to successful implementation of TRaC-M is how to obtain truly random samples. The main steps required for successful implementation of a TRaC-M sampling are

Step 1: Define the supervision area

The first key step in sampling is the definition of the supervision area. A supervision area can be defined as a whole country, each region or province in the country, each district, an area covered by a sales agent, an area covered by an interpersonal communication team, or other sub-national definition. Think how your program is organized. If you only do activities at the national level, then your supervision area is the whole country, and you only need to select randomly and interview 19 people. Examples of a program only working at the national level would be one in which there arre no sales regions, no regional offices, and no

7 regional wholesalers. Everything the program does is centralized. If that is the case for your program, then your supervision area is the whole country.

Most programs have more than one supervision area. The most common sub-national division is the sales region. If your program has divided the country up into, say, 4 sales regions, then you have four supervision areas, and your final sample size will be 19 x 4 sales regions or 96. Work with your research backstop to determine what the appropriate supervision area is.

Step 2: Identify and list your primary sampling units

The simple idea of random sampling is that each person in a population has an equal opportunity to be selected to be interviewed. It is for this basic reason that the size of a country’s population does not matter in calculating sample size. As long as a person in India (population: one billion) has an equal opportunity or chance to be interviewed as anyone else there or a person in Nigeria (population: 120 million) has an equal opportunity or chance to be interviewed as anyone else there, then random sampling has occurred.

Ideally, a country would have a numbered list with the names and addresses of all its residents on it. We would then pick a random number between one and the total number of persons in the country; that number would correspond to one of the persons in the population. We would then systematically select the other 18 persons by skipping 1/19tth of the fraction of the total population and interviewing each person we chose that way.

But only a couple of countries in the world maintain such a list. Otherwise, we must use another method, called ‘multi-stage cluster sampling’ to identify randomly first what is known as primary sampling unit and then the person whom we should interview. In simple terms, instead of making a list of individuals, we make a list of geographic areas that contain a known number of persons. These areas are generally called enumeration areas and are the smallest organizational units used by the national census (usually about 100 households or 400 to 500 persons). We then randomly select 19 of these “primary sampling units” or enumeration areas per supervision area. All the households are then listed and one is randomly selected. In that household then, the person to be interviewed is selected, again at random.

In many countries, lists of enumeration areas are no readily available and even when they are available, maps that show the limits of selected enumeration areas are often difficult to obtain and interpret. The best alterative to this is to use administrative units that are geographically easy to identify such as villages, communities, quarters etc to serve as the primary sampling unit. In any case, the primary sampling unit should be the lowest administrative area that can be accurately listed with current population size (or at

8 least approximate population size). For example, in Madagascar, the lowest administrative area that can be listed with population data is called the FOKONTANY (the equivalent of village in many other countries). Table 1 presents a list of FOKONTANY from five communes (or FIRAISANA). This list will be used as an example throughout this document to illustrate some of the steps1.

1 The same list will be used but with different population values. The population sizes presented here are purposefully modified to simplify the process. True population values will be subsequently used to provide an example on how to account for large or variations in population sizes.

9 Table 1: Basic layout of a list of FOKONTANY from Madagascar (Fictitious population size)

FIRAISANA NOM_FOKONTANY POPULATION Fir ALASORA Fok ALASORA 1241 Fir ALASORA Fok AMBATOMALAZA 1242 Fir ALASORA Fok AMBOAROY 2890 Fir ALASORA Fok AMBODIVOANJO 1289 Fok AMBODIVONDAVA- Fir ALASORA FIRAISAN 1295 Fir ALASORA Fok AMBOHIDRAZAKA 1295 Fok AMBOHITROMBY- Fir ALASORA FIRAISANA 1468 Fir ALASORA Fok AMPAHIBATO 1242 Fir ALASORA Fok ANKADINDRATOMBO 1788 Fir ALASORA Fok ANKADIEVO-FIRAISANA 1108 Fir ALASORA Fok ANKAZOBE 1833 Fir ALASORA Fok ATSIMON'AMBOHIPO 2677 Fir ALASORA Fok EST MAHAZOARIVO 2413 Fir ALASORA Fok AVARATRAMBONY 1581 Fir ALASORA Fok MAHATSINJO 1250 Fir ALASORA Fok MAHITSY 1770 Fir ALASORA Fok MANDIKANAMANA 2402 Fir ALASORA Fok MENDRIKOLOVANA 2963 Fir ALASORA Fok MIADANA 1934 Fir ALASORA Fok AMBOHIMARINA 1793 Fir ANKADIKELY-ILAFY Fok AMBOHIBE 2375 Fir ANKADIKELY-ILAFY Fok AMBOHIPANJA 2801 Fir ANKADIKELY-ILAFY Fok AMBOHITRARAHABA 1089 Fir ANKADIKELY-ILAFY Fok ANDAFIAVARATRA 2284 Fir ANKADIKELY-ILAFY Fok ANDRONONOBE 2739 Fir ANKADIKELY-ILAFY Fok ANKADIKELY 1505 Fir ANKADIKELY-ILAFY Fok ANTANANDRANO 1432 Fir ANKADIKELY-ILAFY Fok BELANITRA 2532 Fir ANKADIKELY-ILAFY Fok MANAZARY 1674 Fir ANKADIKELY-ILAFY Fok MANDROSOA ILAFY 3387 Fir ANKADIKELY-ILAFY Fok MANJAKA 2814 Fir ANKADIKELY-ILAFY Fok MASINANDRIANA 2446 Fir AMBOHIMANAMBOLA Fok AMBOHIBATO 1997 Fir AMBOHIMANAMBOLA Fok AMBOHIMAHATSINJO 1347 Fok AMBOHIMANAMBOLA- Fir AMBOHIMANAMBOLA FIRAIS 1288 Fok AMBOHIMANAMBOLA- Fir AMBOHIMANAMBOLA GARA 1681 Fir AMBOHIMANAMBOLA Fok AMBOHIPENO 1461 Fir AMBOHIMANAMBOLA Fok AMPAHIMANGA 1709 Fir AMBOHIMANAMBOLA Fok ANDRAMANONGA 1105 Fir AMBOHIMANAMBOLA Fok ANTANETIBE 2449 Fir AMBOHIMANAMBOLA Fok IHARAMY 1533 Fir AMBOHIMANAMBOLA Fok TANJONANDRIANA 2551 Fir ANTSINANANTSENA- SABOTSY Fok AMBATOFOTSY 1047 Fir ANTSINANANTSENA- SABOTSY Fok AMBODIVONA 2342 Fir ANTSINANANTSENA- SABOTSY Fok AMBOHIDRANO 1604

10 Fir ANTSINANANTSENA- SABOTSY Fok ANDREFANTSENA 2977 Fir ANTSINANANTSENA- SABOTSY Fok ANTSAHATSIRESY 1262 Fir ANTSINANANTSENA- SABOTSY Fok ANTSINANATSENA 2118 Fir ANTSINANANTSENA- SABOTSY Fok ANTSOFINONDRY 1878 Fir ANTSINANANTSENA- SABOTSY Fok BOTONA 2871 Fir ANTSINANANTSENA- SABOTSY Fok TSARAFARA 2038 Fir AMBOHIMANGAKELY Fok AMBOHIMAHITSY 2674 Fir AMBOHIMANGAKELY Fok AMBOHIMANGAKELY 1464 Fir AMBOHIMANGAKELY Fok AMORONAKONA 1904 Fir AMBOHIMANGAKELY Fok ANDRANOVAO 1207 Fir AMBOHIMANGAKELY Fok ANKADINDAMBO 1087 Fir AMBOHIMANGAKELY Fok ANTANAMBAO 2747 Fir AMBOHIMANGAKELY Fok BEHITSY 1809 Fir AMBOHIMANGAKELY Fok BETAFO 1020 Fir AMBOHIMANGAKELY Fok IKIANJA 3902 Fir AMBOHIMANGAKELY Fok SOAMANANDRARINY 3859 Fir AMBOHIMANGAKELY Fok TSARAHASINA 1285

This table lays out the basic format for listing primary sampling units. In the first column (or first two columns if necessary) is the name of the immediately higher administrative area (s). This is especially useful when some primary sampling units have the same name but come from different higher degree areas. In the second column (or column before last), the exact name of the FOKONTANY is presented. The last column presents the total population of the FOKONTANY. When working with specific target groups such as children under five or pregnant women, the total population should be related to those population. But when the required data is not available, total population data can be used.

Step 3: Select 19 primary sampling units (locations) in each supervisory area

The strategy for selecting the 19 primary sampling units is called systematic sampling with probability proportional to unit size or PPS. This PPS approach gives sampling units of different population size a proportional chance of being selected systematically for interviewing to take place.

Here is how it works.

Using the list of primary sampling units as described above, compute the cumulative size of locations from top to bottom. This is shown in Table 2, in the fourth column from the left. The last row of the column should give a cumulative population that is equal to the total population of the supervisory area. This is easy to do when the list is entered in a spreadsheet such as Microsoft

11 Excel. In the Madagascar data, the fourth column presents cumulative population sizes with a total population of 120,766.

Use the cumulative to compute probability intervals for each location. The start of the interval for the first location in the list is 1 and the end of the interval is the population of the first location (here, 1241). For subsequent locations, the start of the probability interval is the end of the previous line plus 1. The end of the probability interval is the cumulative population for that line. For example, for the second location, the start is 1242 (1241+1). The end is 2483.

Divide the total population (120,766) by 19 to obtain the sampling interval. Here, the result is 6356.

Select a random number between 1 and the sampling interval. This is easily done using the random selection function in Microsoft Excel in the following way. Make sure your cursor is in an empty cell (we suggest you put is in the top cell in a column that is outside of your list of location. Type the following =rand()*(6356-1)+1 and press F9. What you have just done is to generate a random number between 1 and 6356. The exact formula is rand()*(b-a)+a when you want to generate a random number between a and b. By pressing F9, your random number stays the same even you generate new ones2.

Round this number to 0 decimal points. In this example, the number generated is 5038.

Identify the first location selected by looking for the probability interval that contains the random number you just generated. In our example, our random number falls in the sample interval between 2484 and 5373 that is the third location. The first location to be selected is therefore Fok AMBOAROY in ALASORA.

Add the sampling interval to the generated random number to obtain the second number and hence the sample interval (and location) to be selected (this is done in the last column of Table 2). We find 11389 which corresponds to Fok AMPAHIBATO. Subsequent selected locations are identified in the same way by adding to the previous number the sampling interval.

2 For FTrench version of Microsoft Excel, the random number generating function is Alea()Table 2

12 Table 2: Selecting 19 primary sampling units CUMULATIVE START END FIRAISANA NOM_FOKONTANY POPULATION POPULATION INTERVAL INTERVAL Fir ALASORA Fok ALASORA 1241 1241 1 1241 1 5038 Fir ALASORA Fok AMBATOMALAZA 1242 2483 1242 2483 2 11394 Fir ALASORA Fok AMBOAROY 2890 5373 2484 5373 3 17750 Fir ALASORA Fok AMBODIVOANJO 1289 6662 5374 6662 4 24106 Fok AMBODIVONDAVA- Fir ALASORA FIRAISAN 1295 7957 6663 7957 5 30462 Fir ALASORA Fok AMBOHIDRAZAKA 1295 9251 7958 9251 6 36818 Fok AMBOHITROMBY- Fir ALASORA FIRAISANA 1468 10720 9252 10720 7 43174 Fir ALASORA Fok AMPAHIBATO 1242 11962 10721 11962 8 49530 Fir ALASORA Fok ANKADINDRATOMBO 1788 13750 11963 13750 9 55886 Fir ALASORA Fok ANKADIEVO-FIRAISANA 1108 14859 13751 14859 10 62242 Fir ALASORA Fok ANKAZOBE 1833 16692 14860 16692 11 68598 Fir ALASORA Fok ATSIMON'AMBOHIPO 2677 19369 16693 19369 12 74954 Fir ALASORA Fok EST MAHAZOARIVO 2413 21782 19370 21782 13 81310 Fir ALASORA Fok AVARATRAMBONY 1581 23363 21783 23363 14 87666 Fir ALASORA Fok MAHATSINJO 1250 24613 23364 24613 15 94022 Fir ALASORA Fok MAHITSY 1770 26383 24614 26383 16 100378 Fir ALASORA Fok MANDIKANAMANA 2402 28785 26384 28785 17 106734 Fir ALASORA Fok MENDRIKOLOVANA 2963 31748 28786 31748 18 113090 Fir ALASORA Fok MIADANA 1934 33681 31749 33681 19 119446 Fir ALASORA Fok AMBOHIMARINA 1793 35474 33682 35474 Fir ANKADIKELY-ILAFY Fok AMBOHIBE 2375 37849 35475 37849 Fir ANKADIKELY-ILAFY Fok AMBOHIPANJA 2801 40650 37850 40650 Fir ANKADIKELY-ILAFY Fok AMBOHITRARAHABA 1089 41738 40651 41738 Fir ANKADIKELY-ILAFY Fok ANDAFIAVARATRA 2284 44022 41739 44022 Fir ANKADIKELY-ILAFY Fok ANDRONONOBE 2739 46760 44023 46760 Fir ANKADIKELY-ILAFY Fok ANKADIKELY 1505 48266 46761 48266 Fir ANKADIKELY-ILAFY Fok ANTANANDRANO 1432 49697 48267 49697 Fir ANKADIKELY-ILAFY Fok BELANITRA 2532 52229 49698 52229 Fir ANKADIKELY-ILAFY Fok MANAZARY 1674 53903 52230 53903 Fir ANKADIKELY-ILAFY Fok MANDROSOA ILAFY 3387 57289 53904 57289 Fir ANKADIKELY-ILAFY Fok MANJAKA 2814 60103 57290 60103 Fir ANKADIKELY-ILAFY Fok MASINANDRIANA 2446 62550 60104 62550 Fir AMBOHIMANAMBOLA Fok AMBOHIBATO 1997 64547 62551 64547 Fir AMBOHIMANAMBOLA Fok AMBOHIMAHATSINJO 1347 65893 64548 65893 Fir Fok AMBOHIMANAMBOLA- AMBOHIMANAMBOLA FIRAIS 1288 67181 65894 67181

13 CUMULATIVE START END FIRAISANA NOM_FOKONTANY POPULATION POPULATION INTERVAL INTERVAL Fir Fok AMBOHIMANAMBOLA- AMBOHIMANAMBOLA GARA 1681 68862 67182 68862 Fir AMBOHIMANAMBOLA Fok AMBOHIPENO 1461 70324 68863 70324 Fir AMBOHIMANAMBOLA Fok AMPAHIMANGA 1709 72033 70325 72033 Fir AMBOHIMANAMBOLA Fok ANDRAMANONGA 1105 73138 72034 73138 Fir AMBOHIMANAMBOLA Fok ANTANETIBE 2449 75587 73139 75587 Fir AMBOHIMANAMBOLA Fok IHARAMY 1533 77119 75588 77119 Fir AMBOHIMANAMBOLA Fok TANJONANDRIANA 2551 79670 77120 79670 Fir ANTSINANANTSENA- SABOTSY Fok AMBATOFOTSY 1047 80718 79671 80718 Fir ANTSINANANTSENA- SABOTSY Fok AMBODIVONA 2342 83060 80719 83060 Fir ANTSINANANTSENA- SABOTSY Fok AMBOHIDRANO 1604 84664 83061 84664 Fir ANTSINANANTSENA- SABOTSY Fok ANDREFANTSENA 2977 87641 84665 87641 Fir ANTSINANANTSENA- SABOTSY Fok ANTSAHATSIRESY 1262 88902 87642 88902 Fir ANTSINANANTSENA- SABOTSY Fok ANTSINANATSENA 2118 91020 88903 91020 Fir ANTSINANANTSENA- SABOTSY Fok ANTSOFINONDRY 1878 92899 91021 92899 Fir ANTSINANANTSENA- SABOTSY Fok BOTONA 2871 95770 92900 95770 Fir ANTSINANANTSENA- SABOTSY Fok TSARAFARA 2038 97808 95771 97808 Fir AMBOHIMANGAKELY Fok AMBOHIMAHITSY 2674 100482 97809 100482 Fir AMBOHIMANGAKELY Fok AMBOHIMANGAKELY 1464 101946 100483 101946 Fir AMBOHIMANGAKELY Fok AMORONAKONA 1904 103850 101947 103850 Fir AMBOHIMANGAKELY Fok ANDRANOVAO 1207 105058 103851 105058 Fir AMBOHIMANGAKELY Fok ANKADINDAMBO 1087 106144 105059 106144 Fir AMBOHIMANGAKELY Fok ANTANAMBAO 2747 108891 106145 108891 Fir AMBOHIMANGAKELY Fok BEHITSY 1809 110700 108892 110700 Fir AMBOHIMANGAKELY Fok BETAFO 1020 111720 110701 111720 Fir AMBOHIMANGAKELY Fok IKIANJA 3902 115622 111721 115622 Fir AMBOHIMANGAKELY Fok SOAMANANDRARINY 3859 119481 115623 119481 Fir AMBOHIMANGAKELY Fok TSARAHASINA 1285 120766 119482 120766 Sampling tinterval 6356

14 In the example above, there is not large variability in the population sizes of the primary sampling units. In reality, the size of primary sampling units can vary hugely as is the case with the true Madagascar list below. FOKONTANY size vary between 342 (Fok AMBODIVONA) and 7859 (Fok SOAMANANDRARINY). This means that Fok SOAMANANDRARINY has a size that is 23 times that of Fok SOAMANANDRARINY. In sampling terms, neither a simple random sample nor a sampling with probability proportional to size, PPS, is adequate in this situation. Simple random sampling unduly favors the small units while sampling with PPS creates too great a chance that very large units are included in the sample.

Table 3: Example of true list of FOKONTANY from Madagascar FIRAISANA NOM_FOKONTANY POPULATION Fir ALASORA Fok ALASORA 1241 Fir ALASORA Fok AMBATOMALAZA 1242 Fir ALASORA Fok AMBOAROY 2890 Fir ALASORA Fok AMBODIVOANJO 1289 Fok AMBODIVONDAVA- Fir ALASORA FIRAISAN 1295 Fir ALASORA Fok AMBOHIDRAZAKA 1295 Fok AMBOHITROMBY- Fir ALASORA FIRAISANA 468 Fir ALASORA Fok AMPAHIBATO 1242 Fir ALASORA Fok ANKADINDRATOMBO 1788 Fir ALASORA Fok ANKADIEVO-FIRAISANA 1108 Fir ALASORA Fok ANKAZOBE 833 Fir ALASORA Fok ATSIMON'AMBOHIPO 677 Fir ALASORA Fok EST MAHAZOARIVO 2413 Fir ALASORA Fok AVARATRAMBONY 581 Fir ALASORA Fok MAHATSINJO 1250 Fir ALASORA Fok MAHITSY 770 Fir ALASORA Fok MANDIKANAMANA 402 Fir ALASORA Fok MENDRIKOLOVANA 963 Fir ALASORA Fok MIADANA 934 Fir ALASORA Fok AMBOHIMARINA 793 Fir ANKADIKELY-ILAFY Fok AMBOHIBE 375 Fir ANKADIKELY-ILAFY Fok AMBOHIPANJA 2801 Fir ANKADIKELY-ILAFY Fok AMBOHITRARAHABA 6089 Fir ANKADIKELY-ILAFY Fok ANDAFIAVARATRA 4284 Fir ANKADIKELY-ILAFY Fok ANDRONONOBE 5739 Fir ANKADIKELY-ILAFY Fok ANKADIKELY 4505 Fir ANKADIKELY-ILAFY Fok ANTANANDRANO 1432 Fir ANKADIKELY-ILAFY Fok BELANITRA 5532 Fir ANKADIKELY-ILAFY Fok MANAZARY 674 Fir ANKADIKELY-ILAFY Fok MANDROSOA ILAFY 3387 Fir ANKADIKELY-ILAFY Fok MANJAKA 2814 Fir ANKADIKELY-ILAFY Fok MASINANDRIANA 446 Fir AMBOHIMANAMBOLA Fok AMBOHIBATO 997 Fir AMBOHIMANAMBOLA Fok AMBOHIMAHATSINJO 1347

15 Fok AMBOHIMANAMBOLA- Fir AMBOHIMANAMBOLA FIRAIS 1288 Fok AMBOHIMANAMBOLA- Fir AMBOHIMANAMBOLA GARA 681 Fir AMBOHIMANAMBOLA Fok AMBOHIPENO 1461 Fir AMBOHIMANAMBOLA Fok AMPAHIMANGA 1709 Fir AMBOHIMANAMBOLA Fok ANDRAMANONGA 1105 Fir AMBOHIMANAMBOLA Fok ANTANETIBE 449 Fir AMBOHIMANAMBOLA Fok IHARAMY 533 Fir AMBOHIMANAMBOLA Fok TANJONANDRIANA 551 Fir ANTSINANANTSENA- SABOTSY Fok AMBATOFOTSY 1047 Fir ANTSINANANTSENA- SABOTSY Fok AMBODIVONA 342 Fir ANTSINANANTSENA- SABOTSY Fok AMBOHIDRANO 604 Fir ANTSINANANTSENA- SABOTSY Fok ANDREFANTSENA 2977 Fir ANTSINANANTSENA- SABOTSY Fok ANTSAHATSIRESY 1262 Fir ANTSINANANTSENA- SABOTSY Fok ANTSINANATSENA 2118 Fir ANTSINANANTSENA- SABOTSY Fok ANTSOFINONDRY 878 Fir ANTSINANANTSENA- SABOTSY Fok BOTONA 871 Fir ANTSINANANTSENA- SABOTSY Fok TSARAFARA 5038 Fir AMBOHIMANGAKELY Fok AMBOHIMAHITSY *6674 Fir AMBOHIMANGAKELY Fok AMBOHIMANGAKELY 1464 Fir AMBOHIMANGAKELY Fok AMORONAKONA 904 Fir AMBOHIMANGAKELY Fok ANDRANOVAO 1207 Fir AMBOHIMANGAKELY Fok ANKADINDAMBO 1087 Fir AMBOHIMANGAKELY Fok ANTANAMBAO 2747 Fir AMBOHIMANGAKELY Fok BEHITSY 809 Fir AMBOHIMANGAKELY Fok BETAFO 1020 Fir AMBOHIMANGAKELY Fok IKIANJA 3902 Fir AMBOHIMANGAKELY Fok SOAMANANDRARINY *7859 Fir AMBOHIMANGAKELY Fok TSARAHASINA 1285

In this case, large units should be broken down to two or more sub-units. A practical approach is to decide that any greater than sampling interval should be divided into a two or more units such that each sub-unit size is smaller than the sampling interval. In the present case, all FOKONTANYs larger than 6093 (the calculated sampling interval) should be divided by two sub-fokontanys. We have two Fokontany in this situation (SOAMANANDRARINY with 7859 and AMBOHIMAHITSY with 6674). So we will have SOAMANANDRARINY 1 and SOAMANANDRARINY 2 respectively with sizes of 3930.

Once this is done, the selection of locations follows the same steps as described above.

Step 4: Select one household from each location

16 This step takes place in the field during the interviewing process; all the previous steps are done in the office prior to going to the field. As such, this step is implemented by interviewers and it is essential that clear guidelines are created for them to avoid confusion. Errors at this step may jeopardize the validity of the whole sampling process. The primary principle at this stage is to create a list of approximately 30 households. Because household lists are often not readily available in those locations, we will need to create one. The simple approach is to follow the algorithm below.

Divide the location size by average household size to obtain the approximate number of households in the location

Divide that number by 30 to see how many sub-locations you need to create

Draw a rough map of the location and cut it into the required number of sub-locations

Number these sub-locations and randomly select one of them.

Go to the selected sub-location and list all households. If you have done it well, you should have a number of households that is not too far from 30 (likely between 25 and 35).

Write down on small pieces of paper the number of each household and put these in a bucket.

Shuffle the pieces of paper and pick one at random

Step 5: Select one eligible person from the household

Often, you will find one eligible person, for example in a survey of women 15-49, there will be one such woman. But sometimes, there may be more than one or even none. Here is what to do in each case.

If you find one eligible person in the household, then interview him/her if he/she consents

17 If the eligible person does not live in the household or is absent and more than 30 minutes away, then go to the next household on the list until you find an eligible person that consents3

If the type of informant you are looking for is absent but nearby (within 30minutes) then go and find the informant and interview him/her if he/she consents. If he she does not consent go to the nearest household as above

Step 6: Conducting the interview

Conducting an interview in the context of TRaC-M should be a relatively easy process. Just ask the questions on the questionnaire.

Step 7: Analyzing the data and presenting findings

The analysis of TRaC-M data and presentation of findings is a straight forward exercise. This could be done in a one page table as follows and as presented in Table 4.

In the first column, write down the name of each supervision area

In the second column, write down the number of individuals interviewed in each supervisory area, usually nineteen.

In subsequent columns, put each key indicator as in the header. Then, subdivide the column into two sub-columns (one for the number who are positive on your indicator and the second for the corresponding LQAS percentage).

You may be able to calculate an average level for the indicator by using a weighted average procedure. The weight for each supervisory area is its population size. Ask your research backstop for assistance.

Put subsequent survey round information in the rows immediately following.

3 It is often useful to keep track of this movement for verification purposes

18 Table 4: Sample Analysis Table

Indicator Date Supervisory Areas # of Total Average Supervisory Sample Correct = Areas Size Total 19 x # of Correct/ Supervisory Total Areas Sample Szie 1 2 3 4 5 6 7 8 9 10 Purpose Behavior 1 May 05 Nov 05 Apr 06

Risk 1 May 05 Nov 05 Apr 06 Output Motivation 1 May 05 Nov 05 Apr 06

Ability 1 May 05 Nov 05 Apr 06

Opportunity 1 May 05 Nov 05 Apr 06 Exposure Reach May 05 Nov 05 Apr 06

Intensity May 05 Nov 05 Apr 06

Frequency May 05 Nov 05 Apr 06

Duration May 05 Nov 05 Apr 06

19 CONCLUSIONS

TRaC-M complements TRaC by producing a new type of monitoring table for the Dashboard. This monitoring table collects exposure and sub-national level logical framework indicators efficiently at times related either to the timing of intervention campaigns or the conduct of MAP related monitoring. Application of the principles described here is straightforward and merits piloting in one or more PSI programs in 2005. .

REFERENCES

Chapman, S, V. Capo-Chichi, K. Longfield and B.Piot (2005). Project MAP: Lessons Learned and Recommendations. Population Services International: Washington, DC.

Chapman, S and Y. Coombes (2003). Project TRaC: Concept Paper. Population Services International: Washington, DC.

Chapman, S. and D. Patel (2004). PSI Behavior Change Framework (Bubbles): Proposed Revision. Concept Paper. Population Services International: Washington, DC.

Patel, D (2004). Project Exposure: Improving the Measurement of Exposure to Social Marketing Activities. Concept Paper. Population Services International: Washington, DC.

Patel, D and S. Chapman (2005). The Dashboard: A Tool for Social Marketing Decision Making. Concept Paper. Population Services International: Washington, DC.

Valadez, J. W. Weiss, C. Leburg, and R. Davis (2001a). A Participant’s Manual for Baseline Surveys and Regular Monitoring: Using LQAS for Assessing Field Programs in Community Health in Developing Countries. NGO Networks for Health: Washington, DC.

Valadez, J. W. Weiss, C. Leburg, and R. Davis (2001b). A Trainer’s Manual for Baseline Surveys and Regular Monitoring: Using LQAS for Assessing Field Programs in Community Health in Developing Countries. NGO Networks for Health: Washington, DC.