Appendix C – Pre-analysis Plan

Introduction A critical constraint to food production and food security in is low soil fertility. Evidence from past projects indicates that measures to increase soil fertility and crop yields are viable and cost-effective when farmers have access to information on good management practices, site-specific input-use recommendations, and input and output markets (Africare 2015). One particular solution along these lines is integrated soil fertility management (ISFM), or a flexible set of economically and socially acceptable uses of existing resources in conjunction with organic and mineral inputs to increase productivity (Vanaluawe 2004). Prior agronomic research on ISFM has demonstrated its efficacy under controlled scientific conditions and under farmers’ conditions (Vanluawe et al. 2005; Place et al. 2003). While the dissemination of information on ISFM is often considered the mandate of public extension systems, there is evidence suggesting that capacity among agricultural extension agents to provide advisory services on ISFM is generally low (Nkonya et al. 2015). This is consistent with wider concerns about the limited capacity of public extension systems to deliver advisory services on a wide variety of technologies and practices to small-scale, resource-poor farmers (Haug 1999; Anderson 2007; Davis 2008; Birner et al. 2009). In effect, traditional extension services have been relatively more successful—or simply more focused on—the provision of advisory services on basic input use such as improved cultivars, inorganic fertilizer, and agrochemicals, and are less equipped to provide information on relatively complex technologies and practices such as ISFM. There is a growing body of evidence on the impact of alternative approaches to promoting such complex technologies and practices, including various extension approaches such as farmer field schools (FFS; Davis et al. 2012), enhanced training and visit (T&V) systems (Kondylis et al. 2014), and other demand-driven extension services (Klerkx and Leeuwis 2008). A similarly rich body of evidence has emerged around the role of social networks and individual learning dynamics in complex technology adoption processes (Hanna et al 2014; Conley and Udry 2001). However, conclusions are almost always specific to context – to the farmers, crops, agro-ecological conditions, infrastructure, institutions, markets, and policies that influence the target population. The evaluation described here aims to expand this body of evidence by exploring the impact of Africare’s efforts to promote ISFM practices and marketing strategies through a training of trainers (ToT) approach on input use, management practices, and productivity enhancement in the of Ghana. AFricare’s project, funded by the Alliance for a Green Revolution in , is being implemented in close collaboration with the Ghanaian Ministry of Food and Agriculture (MoFA), district-level extension services and other key stakeholders in the Volta region. For this reason, there is scope for this evaluation to influence the ISFM scaling-up process. By directly engaging decision-makers as they explore their options around ISFM promotion and scaling-up, there is an opportunity to improve evidence-based decision- making on the investment of public and private resources in strengthening, training and empowering farmers and farmer-based organizations not only in the immediate project area, but also throughout Ghana. This evaluation specifically assesses the extent to which Africare’s project results in changes in awareness, learning, uptake, adoption, productivity, and welfare among targeted beneficiaries. This study will fill a knowledge gap in the evidence on extension by rigorously evaluating the extent to which Africare’s use of ToT to disseminate a suite of ISFM technologies and practices will effect improvements at the farm and household level for a range of outcome variables that are of interest to Africare and its stakeholders. Project overview The Intervention The Volta region’s predominantly agricultural sector is characterized by low productivity, limited input use, low soil fertility, land degradation from shifting cultivation (“slash-and-burn”) practices, and low yields for most food staple crops. To address these issues, Africare’s intervention promotes the adoption of a suite of ISFM technologies and practices that ultimately aim to increase on-farm yields and the farm incomes of smallholders in the Volta region. Africare’s project specifically promotes the sustainable intensification of maize, rice, cowpea, and cassava cultivation with the provision of information on (a) production inputs and their use, (b) integrated soil management practices, and (c) marketing strategies and services. This information intervention is provided by Africare-trained, MoFA-employed agricultural extension agents (AEAs) and targeted to members of farmer-based organizations (FBOs) in selected districts of the region. The Africare project, now in its second phase,1 targets 20,000 farmers over a three-year period (2015 to 2018). The project’s goal is to reach out to roughly 7,000 farmers per year in six target districts of the Volta Region – Jasikan, Kajebi, , , Afajato South and North Dayi. Its main intervention point and modality is the training of trainers through which it plans to train 30 AEAs in year one, each of whom are capable of providing training to between 90 and 150 farmers per semester. Evaluation Design The evaluation study proposed here will use a difference-in-difference (DID) approach and propensity score matching (PSM) techniques to evaluate the impact of Africare’s intervention among smallholders in Ghana’s Volta region. The DID approach and PSM techniques aim to develop a credible counterfactual with which to compare treatment effects on “treated” participants in Africare’s project. The study will base its analysis on comparisons between two groups: 1. Treatment group: FBO members who reside in one of the six districts where Africare-trained AEAs will conduct training sessions 2. Control group: FBO members who reside in one of six districts where Africare-trained AEAs will NOT conduct training sessions The DID approach and PSM techniques will be used to identify and quantify the causal relationship between participation in training sessions and outcome variables related to in increases in (1) awareness, learning, uptake, and adoption of selected ISFM-related technologies, (2) land and labor productivity, (3) returns to and incomes from farming, and (4) household food security and welfare. The DID approach and PSM techniques will be used to identify and quantify the causal relationship between participation in ISFM training and the outcome variables of interest. The DID approach and PSM techniques allow for comparison of treated households against a set of similar but untreated households that are identified based on observable characteristics, with comparisons being made both before and after the intervention. This approach offers a fairly straightforward means of evaluating the impact of Africare’s project when compared against similar households who were not part of the project (with vs. without), and

1 Africare’s project is the second phase of activities in the Volta Region, and began in September 2015. Phase I of the Africare project, also funded by AGRA, concluded in July 2014. The highlight of Phase I was the establishment of three “one stop centers” (OSCs) in three districts of the Volta region namely, Jasikan, and Hohoe (after the change in administrative boundaries the OSC that was intended for Hohoe is now in Afadjato South). By design, OSCs are structures that encompass a suite of agricultural services including an agro-input shop, warehouse, drying floor, training center, and farmers’ library. Phase II of the project shifts focus away from OSCs and places greater emphasis on the ToT approach described here.

1 by controlling for exogenous changes affecting both project participants and non-participants (before vs. after). This comparison will be made between treated group farmers and control group farmers. The study will use DID to estimate the intention to treat (ITT) impact of ISFM training, as well as the average treatment effect on the treated (ATT) using kernel matching, nearest neighbor matching, and other appropriate matching estimators. Sample The sample consists of smallholder farmers who cultivate at least one of the three target crops – maize, cassava, and cowpea—and belong to a FBO. The treatment districts where Africare’s intervention is active are Jasikan, Kadjebi, Hohoe, Afadjato South, North Dayi, and Kpando. The control districts—Biakoye, South Dayi, Ho Municipal, Ho West, Kranchi East, and South—were purposefully based on proximity and similarities in agro-ecological and socio-economic characteristics to the treatment districts.

A sample of treatment communities was drawn using probability proportional to size sampling. All communities in the control districts were included in the sample to accommodate the need for a significantly large set of control households to draw from for the purposes of finding an area of common support in the propensity score matching exercise. A random sample of households was drawn from a list of FBO members in each community. During survey implementation, enumeration teams were able to conduct surveys in 59 treatment communities and 75 control communities. In total, data were collected from 1,333 households: 629 in the treatment group and 704 in the control group.

Data sources Baseline data were collected from an individual survey of 1,333 households and a community survey of 134 communities from March to April 2016. This was prior to the intervention being launched in most treatment communities.2 Endline data will be collected following the major agricultural season in October 2017 and will gather recall data for the major season in 2016 and 2017.

Hypotheses to be tested This study will test the following hypotheses along the causal impact chain:

1. H0: No impact of ISFM training on awareness and understanding of ISFM concepts and components.

Ha: Positive impact of ISFM training on awareness and understanding of ISFM concepts and components.

2. H0: No impact of ISFM training on adoption of some or all ISFM practices. Ha: Positive impact of ISFM training on adoption of some or all ISFM practices.

3. H0: No impact of ISFM training on labor use. Ha: Positive impact of ISFM training on labor use.

4. H0: No impact of ISFM training on maize, cassava, and cowpea yields. Ha: Positive impact of ISFM training on maize, cassava, and cowpea yields.

2 A few communities that were exposed to the treatment prior to the baseline were excluded from the sample.

2

5. H0: No impact of ISFM training on farm incomes and profitability. Ha: Positive impact of ISFM training on farm incomes and profitability.

6. H0: No impact of ISFM training on farm and household assets. Ha: Positive impact of ISFM training on farm and household assets.

7. H0: No impact of ISFM training on food security. Ha: Positive impact of ISFM training on food security.

Variable construction Outcome variables for the hypotheses listed above will be constructed as follows:

1. Awareness and understanding of ISFM: The level of awareness and understanding of ISFM concepts will be measured using a content knowledge test developed by the evaluation team in consultation with Africare. Tests are scored against 25 questions where respondents get 1 point for every correct answer and no points for a wrong answer or a “don’t know” response. The 25 questions which comprise the knowledge test correspond to variables s1-s18, s20, s21, s24, s25a, s25b, s25c collected as part of the household questionnaire.

2. Adoption of ISFM: ISFM is a suite of technologies which, when adopted together, have been shown to have beneficial effects on farm productivity and soil fertility. These practices include: use of green manure or farmyard manure; use of chemical/synthetic fertilizer in its prescribed quantities; composting; growing cover crops; growing legumes; crop rotations with maize, cassava, and legumes; retaining and mulching of crop residues; doing away with shifting agriculture or slash- and-burn practices; encouraging the presence of specific insects, worms and other organisms in the soil to help with decomposition and drainage; careful weed management; use of improved seeds; use of herbicides and weedicides; maize line planting; contour plowing; and timely and accurate implementation of these recommended practices.

The following variables collected as part of the household questionnaire measure the different aspects of ISFM that are of interest to this evaluation:

a. Manure and other organic input: f7-f10 b. Chemical fertilizer use: f11-f17 c. Growing legumes and intercropping cowpea: f18a-f18e d. Doing away with slash-and-burn practices: e15-e19 e. Seeds: g2_2-g2_4 f. Herbicides, weedicides, pesticides: g2_5, g2_6 g. Ploughing: f4 h. Crop residue: g2_7, h6a-c, h7a-c Each of these variables will be used individually on the left hand side of the treatment effect equation to measure the impact of the ISFM training on adoption of the different components of ISFM. In addition a total score will be created by aggregating the variables listed above to measure adoption of the entire suite of ISFM technologies.

3

3. Labor use: Because ISFM relies on more intensive crop and resource management practices, it is hypothesized that ISFM adoption will increase the demand for household and/or hired labor. The following variables collected as part of the household questionnaire measure the various on-farm labor use requirements that are of interest to this evaluation: g3_m1-g3_m10, g3_f1-g3_f10, g3_3- g3_7b.

4. Yields: Yields are difficult to measure under any circumstances due to recall, measurement, and other types of errors. But because budgetary resources prevent us from conducting crop cuts prior to harvest as a means of estimating yields more accurately, we will be relying on farmer-reported yields.

The following variables collected as part of the household questionnaire. Harvest quantities will be measured using h2a and h2b and land area will be measured using e1_area measured in e1_units times the percentage of the plot with the crop (g1_7).

5. Farm profits: There is ample economic theory and empirical evidence to suggest that farmers seek to maximize profits—not yields—from their farming activities, and that yields are not always correlated with profits (measured in terms of revenues minus costs using both cash outlays and shadow prices).

Crop revenue will be measured using h3a, h3b, and h3c for the three major crop plots and d7 and d9 for other plots. Cost of inputs will be measured using g2c for non-labor inputs and g3_3-g3_7b for labor inputs, and further imputed using data from the community survey or, if necessary, from secondary data sources for prices and costs available with MOFA. Farm profits will be calculated by subtracting input costs from farm revenue.

6. Farm assets: An index of farm assets will be created variables with n16_1 – n16_17 using principle component analysis. Similarly a non-farm asset index will be created using n13_1 – n13_21. In addition a total asset index will also be created using the entire set of non-farm and farm assets. Total value of assets will also be constructed using n15 and n18.

7. Food security: An aggregate measure of food security will be generated using variables o1 – o11_12.

Propensity Score Matching This evaluation will employ propensity score matching (PSM) to identify a valid counterfactual for the treatment group and difference-in-difference (DID) estimators to measure the impact of Africare’s ISFM intervention. PSM will be used to select a group of farmers in the treatment and control groups who are similar on observable characteristics. Farmers in the treatment group and control group are matched using propensity scores that are generated using a probit model where a binary treatment variable is regressed on a set of observable characteristics that are expected to affect treatment and the outcome variables of interest. This model is described in Equation 1

= + + (1)

𝑇𝑇𝑖𝑖 𝛼𝛼0 𝛼𝛼1𝑿𝑿𝑖𝑖 𝜗𝜗𝑖𝑖

4

Where i indexes the household, T is a binary treatment variable, ϑ is the error term and X is a vector of characteristics that are expected to affect participation and the outcome variable without predicting treatment perfectly.

The propensity score is essentially the conditional probability of participation in the treatment intervention. Farmers that are matched on propensity score are said to be in the “common support” and the analysis is restricted to this matched group of farmers.

For the purpose of the baseline means comparison tests a preliminary estimate of the propensity score was generated and common support was defined using the following X variables: a binary variable fo whether the household’s main language is Ewe, household size, dependency ratio, age of household head, gender of household head, education level of household head, amount of land under maize cultivation, distance to nearest market, indicator variable for whether the household applied for a loan in the last 12 months, indicator variable for whether the household has savings with a bank, non-farm asset index, and non-farm income.

The sample selected to the common support was trimmed at the 25 percent level to enable a better match. As can be seen from figure 1 below, while matching reduces the standardized bias for many of the covariates, it fails to perfectly eliminate it. More data on exogenous time-invariant variables will be collected at endline to better estimate the propensity score and improve this match.

ewe head_education loan_applied head_age head_vocational area_maize has_savings head_male nonfarm_asset_index nonfarmincome distance_market depratio Unmatched hh_size Matched

-40 -20 0 20 40 Standardized % bias across covariates

Figure 1: Standardized bias before and after matching

The common support found after trimming at the 25 percent level can be seen in figure 2 below. We can see that there is large overlap between the treatment and control groups after matching however, the

5 peaks of the curves are not entirely distinct. This implies that there may be other observable or unobservable characteristics that explain treatment that have not been captured here.

Kernel density of Propensity Score by treatment status 3 2 1 0

.2 .4 .6 .8 psmatch2: Propensity Score

Non-beneficiary Beneficiary

Figure 2: Common support graphically represented Treatment effect equation To estimate the treatment effect a DID model will be estimated in which each observation is weighted with its respective propensity score. Different matching estimators (kernel matching, nearest neighbor matching, and covariate matching) will be used to match treatment and control farmers as a robustness check. The underlying specification to be estimated is described by Equation 2 as

= + + + + + + (2) where i indexes the𝑦𝑦 household𝑖𝑖𝑖𝑖𝑖𝑖 𝛽𝛽0 , j 𝛽𝛽indexes1𝑿𝑿𝑖𝑖𝑖𝑖 the𝛽𝛽2𝑇𝑇 community,𝛽𝛽3𝐸𝐸 𝛽𝛽4 and𝑇𝑇 ∗ 𝐸𝐸t indexes𝜇𝜇𝑗𝑗 time𝜀𝜀𝑖𝑖𝑖𝑖 (Imbens and Wooldridge 2007); and where y denotes the outcome variable of interest. The vector X is a set of control variables that describe household characteristics including the age, sex, and education level of the household head, the household’s landholding size, an index of the value of its non-farm asset, and an indicator capturing any idiosyncratic shocks experienced recently by the household. The variable T is a binary variable that takes on the value 1 if household is in a treatment district and 0 if control, while E is binary variable that takes on the value 1 at endline and 0 at baseline. The variable µ captures community-level fixed effects and ε is an iid error term.

Impact is measured by the coefficient β4 on the interaction term. Estimations of the average treatment effect, average treatment effect on the treated, as well as intent to treat effect will be measured.

6

Addressing survey attrition and missing data We expect low attrition in the survey, however households that cannot be found at endline will be dropped from the analysis. Since the questionnaire was administered using CAPI, the majority of questions were made mandatory and thus missing data is not a significant problem in the baseline data set. For those few variables where we do see missing data, attempts will be made to collect them at endline.

Outcome with limited variation We will include several outcome variables with limited variation. We will drop any control variable for which more than 97% of observations carry the same value, including dummy variables.

Heterogeneous treatment effects Treatment effects will be measured across different sub-groups of interest. Heterogeneous impacts will be estimated across different subgroups, including:

- Farmers living in different districts - Households practicing different religions and speaking different languages - Farmers that cultivate different crops or allocate significantly different shares of their operational landholdings to different crops - Farmers with different sizes landholdings - Farmers that use different inputs at baseline - Farmers with different levels of interaction with extension services, and knowledge of AGRA and Africare. - Farmers in different wealth terciles, as measured by the index of the value of non-farm assets described above.

Given the demand-driven nature of this intervention and the fact that Africare is currently behind schedule on achieving their project targets (see Brief on project implementation) this plan will be revisited and revised over the course of the coming year.

7

References Africare. 2015. Grant Proposal to the Alliance for a Green Revolution in Africa (AGRA): Scaling-out of Integrated Soil Fertility Management Technologies in the Volta Region, Ghana. Washington, D.C./: Africare. Anderson, J. R. 2007. Agricultural advisory services. Background paper for World Development Report 2008, Agriculture for Development. Washington, DC: World Bank. Birner, R. K. Davis, J. Pender, E. Nkonya, P. Anandajayasekeram, J. Ekboir, A. Mbabu, D. Spielman, D. Horna, and S. . 2009. From Best Practice to Best Fit: A Framework for Analyzing Agricultural Advisory Services Worldwide. Journal of Agricultural Extension and Education 15(4): 341-¬355. Conley, T. and Udry, C., 2001. Social learning through networks: The adoption of new agricultural technologies in Ghana. American Journal of Agricultural Economics, 83(3), pp.668-673. Davis, K., 2008. Extension in sub-Saharan Africa: Overview and assessment of past and current models and future prospects. Journal of International Agricultural and Extension Education, 15(3), pp.15-28. Davis, K., Nkonya, E., Kato, E., Mekonnen, D.A., Odendo, M., Miiro, R. and Nkuba, J., 2012. Impact of farmer field schools on agricultural productivity and poverty in East Africa. World Development, 40(2), pp.402-413. Garrido, M.M., Kelley, A.S., Paris, J., Roza, K., Meier, D.E., Morrison, R.S. and Aldridge, M.D., 2014. Methods for constructing and assessing propensity scores. Health services research, 49(5), pp.1701-1720. Hanna, R., Mullainathan, S. and Schwartzstein, J., 2014. Learning through noticing: Theory and evidence from a field experiment. The Quarterly Journal of Economics, 129(3), pp.1311-1353. Haug, R., 1999, January. From integrated rural development to sustainable livelihoods: what is the role of food and agriculture?. In Forum for Development Studies (Vol. 26, No. 2, pp. 181-201). Taylor & Francis Group. Imbens & Wooldridge. 2007. Lecture Notes: Difference-in-Differences Estimation. NBER http://www.nber.org/WNE/lect_10_diffindiffs.pdf Klerkx, L., & Leeuwis, C. (2008). Matching demand and supply in the agricultural knowledge infrastructure: Experiences with innovation intermediaries. Food policy, 33(3), 260-276. Kondylis, F., Mueller, V. and Zhu, S.J., 2014. Seeing is believing? evidence from an extension network experiment. Evidence from an Extension Network Experiment (August 1, 2014). World Bank Policy Research Working Paper, (7000). Place, F., Barrett, C. B., Freeman, H. A., Ramisch, J. J., & Vanlauwe, B. (2003). Prospects for integrated soil fertility management using organic and inorganic inputs: evidence from smallholder African agricultural systems. Food Policy, 28(4), 365-378. Nkonya E., F. Place, E. Kato, and M. Mwanjololo. 2015. Climate Risk Management Through Sustainable Land Management in Sub-Saharan Africa. In R. Lal B. Singh, D. Mwaseba, D. Kraybill, D. Hansen and L. Eik (eds.), Sustainable Intensification to Advance Food Security and Enhance Climate Resilience in Africa, Springer International Publishing Switzerland. Page 75112. DOI 10.1007/978-3-319-09360-4_5. pp 665 Social Science Computing Cooperative. 2015. Propensity Score Matching in Stata using teffects https://www.ssc.wisc.edu/sscc/pubs/stata_psmatch.htm

Vanlauwe, B. 2004. Integrated soil fertility management research at TSBF: the framework, the principles, and their application. Nairobi: Academy Science Publishers.

8

Vanlauwe, B., J. Diels, N. Sanginga, and R. Merckx. 2005. Long-term integrated soil fertility management in South-western : crop performance and impact on the soil fertility status. Plant and Soil 273(1-2): 337-354.

9