Livelihood interventions: overview, evaluation, and cost-effectiveness

Editorial note

This report was produced by Rethink Priorities during March and April 2022, and any updates to interventions or organizational structures since then are not included. The project was supported by the Livelihood Impact Fund, which does not necessarily endorse our conclusions.

The report focuses on the effectiveness of several initiatives that aim to increase the incomes of adults in poverty, with a focus on sub-Saharan Africa. We review three main types of intervention in the databases of the Abdul Latif Jameel Poverty Action Lab (J-PAL) and Innovations for Poverty Action (IPA) only: agricultural interventions; technical, vocational, and job search training; and entrepreneurial support for small and medium enterprises. Given that we only search these two databases and do not search for additional literature in these domains, our takeaways should not be considered comprehensive. The report also spotlights two organizations identified as promising by the Livelihood Impact Fund — Spark Microgrants and AkiraChix — for which we conducted cost-effectiveness analyses and interviews with representatives.

We have tried to flag major sources of uncertainty in the report and are open to revising our views as more information becomes available.

Key takeaways

• Livelihood interventions are often overlooked by EA organizations. This research is a first step in the direction of understanding the promise and cost-effectiveness of interventions that aim to improve incomes among impoverished adults. In line with the goals of the organization that supported this project (the Livelihood Impact Fund; LIF), we focused on three intervention types — those seeking to improve income through bolstering agricultural productivity, providing vocational training, and supporting entrepreneurs — in sub-Saharan Africa (SSA). It is possible that other intervention types or geographies are more promising.
• In conducting this research, we interviewed representatives from two nonprofits selected by LIF — Spark Microgrants and AkiraChix — and relied heavily on academic work from J-PAL and IPA to identify potentially promising interventions. We are far from identifying all promising organizations and interventions in this space, but some of the ones we review in this study seem cost-effective and are likely a good starting point.
• Agricultural interventions have high potential for impact on consumption due to the centrality of subsistence agriculture among the world’s poorest people. SSA is a particularly critical region. In some regions, up to 98% of rural households are engaged in on-farm agriculture, comprising about two-thirds of their income.
  • Some potentially promising agricultural interventions include subsidized watchmen, incentives to use fertilizers, “extension” (training) programs, and accessible or subsidized crop insurance, but their cost-effectiveness is either low or unknown.
  • Spark Microgrants is a nonprofit which offers $8,000 grants and training to impoverished rural villages to identify and implement a village-level investment. Eighty percent of “Sparked” villages purchase livestock. The purchase of livestock appears to be a potentially impactful method of improving agricultural productivity. We estimate with medium-high confidence that the Spark intervention generates about $28 in village-level income per dollar spent over the first 10 years of the program when villages purchase livestock.
    • The advantage of livestock purchases derives from the potency of animal manure as a fertilizer for crops, as well as the profitability of animal products.
    • Importantly, this report does not consider the effects on the welfare of livestock.
• Technical, vocational, and job search training also appear promising. Only about 40% of workers in developing countries are employed in the formal sector.
  • Subsidized training appears to be able to improve income by as much as 50%-55% over the following years. Workers see the largest effects when they receive formal certification of skills, so that they can attest to their capabilities and maintain job mobility.
• Life skills and job search skills training also appears to improve outcomes. For young women, community life skills training can increase their likelihood of generating their own income, and also contributes to indicators of gender equity.
• AkiraChix, a nonprofit that trains women for careers in technology such as software development, appears to significantly alter career trajectories for their graduates. We estimate with medium-high confidence that AkiraChix generates about $11 in income per dollar spent on the program.
• Over three-quarters of non-agricultural employment in “lower-income countries” comes from small and medium enterprises (SMEs). Therefore, it seems that entrepreneurial support for such businesses could have an outsized effect on workers and business owners alike.
• Standardized women’s entrepreneurship programs such as Uganda’s WINGS and ILO’s GET Ahead appear to show that business training, combined with even very small cash grants or mentorship, can seriously impact business performance. In addition, in-kind grants rather than cash transfers may reduce the likelihood that the money is funneled into a different usage.
• Access to standard financial services appears to have a large impact on a business’s bottom line.

Introduction

In this project, we set out to investigate various interventions that may help very poor people to increase their income and earning potential in the short and medium term. Our goal in this report is to provide a foundational overview of the potential for particular income-improving livelihood interventions with a focus on sub-Saharan Africa (SSA). To do so, we searched the intervention databases of the Abdul Latif Jameel Poverty Action Lab (J-PAL) and Innovations for Poverty Action (IPA) for relevant interventions with income generation potential — which we itemized and summarized in the Livelihoods Interventions and Organizations spreadsheet linked below^[1] — and additionally conducted cost-effectiveness analyses (CEAs) for specific organizations of interest to the Livelihood Impact Fund (AkiraChix and Spark Microgrants).

We restrict our assessment to the areas of particular interest to the client, each of which is discussed in detail in the following sections:

1. Agricultural interventions
2. Training and labor market interventions
3. Entrepreneurship and small business support

We do not discuss livelihood interventions related to education, infrastructure, policy, or health, as they are out of scope for the current investigation. We also recognize that there is an extensive academic literature on many of the areas discussed. Our coverage is not exhaustive, but synthesizes important elements of this literature alongside the findings of studies from reputable sources (J-PAL and IPA) conducted in relevant areas.

Agricultural interventions

While agriculture is extremely important for livelihoods in sub-Saharan Africa, we did not find any interventions in J-PAL’s and IPA’s databases that both explicitly measure impacts on income and measure program cost-effectiveness, leaving us highly uncertain about the interventions’ impact. Offering subsidies to farms to hire watchmen improves income per acre by 15%, but the program is not cost-effective; offering similar subsidies at the village level — as the authors suggest — seems like a promising way to improve cost-effectiveness by spreading costs, but such an intervention has not yet been tested to our knowledge. Nudges and incentives for fertilizer adoption, in-person and video-based extension services, and rainfall insurance offers to informal groups encourage the uptake of potentially important technologies and practices, but these studies do not measure income generation as an outcome variable. We encourage researchers to explicitly measure the costs of interventions as well as their impact on household-level income in future studies. Finally, our CEA of Spark’s village-level microgrants indicates that village-level decisions to invest in livestock generate about $28 in income per program dollar (medium-high confidence).

The importance of agriculture for income in sub-Saharan Africa

According to the World Bank, “agricultural development is one of the most powerful tools to end extreme poverty, boost shared prosperity, and feed a projected 9.7 billion people by 2050.” The sector accounts for over 25% of many low- and middle-income countries’ GDP (4% of global GDP) and more than a billion livelihoods, while an estimated 65% of poor working adults make a living in the agriculture sector (World Bank, 2022). By some accounts, growth in the agricultural sector is two to four times more effective at increasing the incomes of the poor than growth in other sectors (Townsend, 2015).

In SSA, up to 98% of rural households are engaged in on-farm agriculture, which constitutes about two-thirds of their income. SSA has the highest concentration of economies where agricultural employment accounts for 40% or more of the labor force (Figure 1). As such, SSA is a particularly critical region in which to consider income-improving livelihood interventions focused on agriculture.

Figure 1: Global share of labor force employed in the agriculture sector in 2019. From “Employment in agriculture,” by M. Roser, 2013, Our World in Data (https://ourworldindata.org/employment-in-agriculture, archived at https://perma.cc/TU49-FC9A). CC BY 4.0.

Several important considerations are necessary for the interpretation of our findings on agricultural interventions, including the potential variation in intervention impacts based on geography, the distinction between agricultural productivity and income, and the interconnectedness of the food system with other well-being indicators. We briefly discuss each of these considerations below.

First, countries and subnational regions in SSA may vary both in their potential for gains from agricultural improvements as well as in their suitability for particular agricultural interventions. For instance, a recent study on food security in SSA suggests that farm size is a primary determinant of a household’s prospects for rising above a living income and becoming food-secure, and that farms tend to be larger in regions with drier climates. In Ethiopia, Ghana, Malawi, and Tanzania, land constraints mean that only about half of households achieve food sufficiency and only a small percentage of agriculture-reliant households reach a living income, suggesting an important role for non-farm income supplementation (Giller et al., 2021). Nevertheless, average income from non-farm wages is just 8%, compared to 21% in 13 non-SSA comparator countries (World Bank, 2015).^[2]

Such geographical considerations may therefore be crucial to the success of agricultural interventions across countries and regions within SSA. We caution against assuming generalizability of interventions, and encourage prospective donors to consider the importance of contextual factors in determining the success (or failure) of the interventions we discuss.

Second, for this report we focus only on identifying livelihood interventions that increase effective incomes for the poor. However, agricultural productivity has a somewhat tenuous relationship with income and consumption. As stated above, even subsistence farmers often rely on food- and income-generating opportunities outside of the household farm, so agricultural productivity is not a 1:1 proxy for consumption. In addition, some farm workers are paid in wages, which may not reflect productivity (McCullough, 2016, p. 13). Furthermore, crops that are sold at market are vulnerable to price shocks, and even to reductions in prices if a boost in productivity floods the market with crops (Kalkuhl et al., 2016, p. 3). Thus, it is not necessarily accurate to map income or consumption onto agricultural productivity.

Third, it is important to note the interconnectedness of agricultural livelihoods and the food system more broadly with other factors related to development, health, and well-being. For example, the negative impacts of climate change on crop yields could particularly harm food-insecure regions, while the agriculture, forestry, and land use change sector contributes 25% of global greenhouse gas emissions. Current food systems are highly polluting and wasteful, with impacts on health and food security. Poor diets represent one of the primary contributors to global health problems, with 14 diet-related risk factors contributing to 21% of deaths globally in 2013 (Forouzanfar et al., 2015). Furthermore, animal agriculture contributes to the use of animals for human gain, and may cause suffering in many cases. Depending on one’s moral framework, this could be an important consideration for agricultural improvements. Thus, improving agricultural practices may not only have important implications for income but also for global health, climate change, and development on a broader scale.

Promising interventions

Of the 15 studies identified through J-PAL’s and IPA’s databases, four appeared to improve near-term income generation: subsidized watchmen, fertilizer nudges, video-based and in-person extension services, and offering rainfall insurance to informal insurance groups.

Subsidized watchmen improved income per acre by 15% and reduced neighbor disputes, but costs to farmers exceeded the intervention’s benefits

In a study of 585 farmers across 76 villages in Kenya in 2018-19, Dyer (2020) found that offering farmers the opportunity to hire subsidized watchmen in villages randomly selected to receive the intervention (i.e., “treated” villages) led to high uptake (87%) and significant treatment effects. Farmers in treated villages were 13.9 percentage points (77%) more likely to have reported planting new crops or expanding the land used to grow an existing crop, and 11.5 percentage points (61%) more likely to have purchased farm assets, leading to a $50 (15%) increase in income per acre. Additionally, positive spillovers included a 60% reduction in neighbor disputes and lower reported theft in nearby farms.

However, despite subsidization, the costs to farmers exceed income generation benefits. It is unclear whether a full cost-benefit analysis that includes positive spillovers would show this intervention to be cost-effective, and conclusions would depend on the value placed on reduced disputes and assumptions around the impact on and value of resulting social capital generation. Thus, the results weakly suggest that interventions to improve farm security could increase farm incomes in the near term if costs of security are low — perhaps through purchasing watchmen’s services on a collective basis, as suggested by the author.

Fertilizer adoption nudges and incentives can improve income by around 11%-18%, though the cost-effectiveness of the interventions is unclear

Between 2003 and 2005, Duflo et al. (2011) randomized various nudges^[3] and incentives across 1,634 farmers with children in grades five or six. Partnering with International Child Support, they designed a set of interventions to understand the role of farmers’ time preferences (i.e., patience) in a program to encourage fertilizer uptake among farmers in Kenya, which they claim will increase yield by $25.22 per acre at the cost of $19.83 per acre (ignoring labor costs; p. 2354). In their sample, 40% reported having ever used fertilizer and 29% reported using fertilizer in either or both of the last two growing seasons.

The most basic version of the intervention, called the “Savings and Fertilizer Initiative” (SAFI), offered farmers free fertilizer delivery at the end of a harvest. Experimental participants in Treatment 1 received this basic offer. Further treatments provided variations on SAFI: choice of when the fertilizer would be delivered (Treatment 2), delivery two to four months after the harvest (Treatment 3), and delivery two to four months after the harvest with a 50% subsidy for the fertilizer (Treatment 4).

Of farmers who were offered the SAFI program, 31%-41% purchased fertilizer depending on the season and treatment. All of the interventions led to a substantial increase in fertilizer uptake relative to a control group, as follows^[4] (see pp. 2373-2374, Table 4):

• T1 (basic SAFI): 11-14 percentage point increase from a 24% baseline in year one; 16-19 percentage point increase from a 26% baseline in year two (~52%-67% increase)
• T2 (SAFI with choice of delivery timing): 16-18 percentage point increase from 26% baseline (~65% increase)
• T3 (SAFI delivered later): 10 percentage point increase from 26% baseline (~38% increase, not statistically significant)
• T4 (SAFI delivered later with 50% subsidy): 13-14 percentage point increase from 26% baseline (~52% increase)

In other words, all treatments increased fertilizer use, though interventions that only offer delayed delivery appear somewhat less effective at inducing adoption.^[5] A back-of-the-envelope calculation from the paper suggests that the use of fertilizer increases income from maize farming by about $10-$16 from a baseline of $89 (i.e., about an 11%-18% income increase; p. 2351).

When searching for the cost of fertilizer to understand the cost-effectiveness of the program (which we did not explicitly find in the paper), we came across the authors’ caveat that their welfare calculations omit program administration costs, which are quite important to assessing cost-effectiveness:

[T]he SAFI program itself, with its delivery of small quantities of fertilizer to farmers by field officers, is too expensive (in terms of staff costs) to be cost effective and therefore could not be directly adopted as policy. Preliminary results from a pilot program designed to mimic key elements of SAFI without individual free delivery (and thus expensive visits to farms) suggest, however, that time-limited coupons for small discounts on fertilizer could cost effectively increase take-up. In 2009, during school meetings or after church service, coupons for a reduction of 15 percent in the price of up to 25 kilograms of fertilizer were distributed to 329 individuals at several schools and churches.… Overall, 30 percent of farmers who received the coupon purchased fertilizer.… The average quantity purchased was about 10 kilograms, corresponding to a subsidy of about $1 per farmer. (Duflo et al., 2011, pp. 2388-2389)

This pilot study suggests that there may be a role for small subsidies in encouraging fertilizer adoption. However, it is unclear how cost-effective this type of intervention is likely to be when the full costs are taken into account. Furthermore, in the two seasons following the SAFI intervention, fertilizer use dropped back down to levels similar to those in the comparison group, indicating that the treatment effects do not persist once the SAFI offer is removed.

In-person and video-based extension services improved uptake of helpful agricultural technologies among Ethiopian farmers, though further research is needed to understand the impact on incomes as well as the interventions’ cost-effectiveness

Hörner et al. (2019) evaluate the impact of in-person and video training on Ethiopian farmers’ adoption of a package of integrated soil fertility management (ISFM) technologies and practices.^[6] Randomization takes place at the level of the microwatershed.^[7] In groups of about 50 farmers, treated participants received either an extension training led by nine model farmers where information was expected to spread through the community (Treatment 1, n=36 microwatersheds), or the same extension training supplemented with a video-based informational training, with the latter available to all members of the community (Treatment 2, n=36). An additional 89 microwatersheds serve as a comparison group.

Farmers in microwatersheds that received the extension training alone (Treatment 1) adopted an additional 0.68 (31%) practices relative to those in the control group, which adopted 2.2 practices. Farmers in microwatersheds that additionally received the video-based training adopted 0.84 (38%) more practices than the control group. Treatment 1 and Treatment 2 farmers were 8.4 percentage points (55%) and 10.9 percentage points (72%) more likely to adopt all of the ISFM practices than farmers in the control group, respectively.

The authors do not measure changes to income, but a subsequent study using observational data to understand the implications of ISFM adoption on household income found that ISFM practices improve income:

Our results show that the adoption of ISFM core components on at least one maize, wheat or teff plot is associated with higher incomes achieved from these crops as well as higher labor demand for their production in both agroecological zones. However, only in Amhara and Oromia, higher crop income seems to translate into significantly higher household income, while in Tigray [the drier region], we find no significant association between the adoption of ISFM core components and changes in total household income. (Hörner & Wollni, 2021, p. 9)

The authors speculate that ISFM adoption may have different implications for household income depending on regional and household-level characteristics of the intervention site (Hörner & Wollni, 2021, p. 10). We have not conducted further literature scans to identify studies that investigate heterogeneity of ISFM’s impact on income, though we think this would be important to understand prior to implementing this intervention in a given country or region. Further research would be needed to assess cost-effectiveness.

Again, the results provide some indication that provision of extension services to train farmers in ISFM techniques may lead to higher agricultural incomes.

Offering crop insurance to informal insurance groups improved demand for rainfall insurance in Ethiopia, though impact on income and cost-effectiveness are not measured

Dercon et al. (2014) test for the impact of marketing weather insurance to informal risk-sharing groups, randomizing the content of training to group leaders on risk management and insurance across 117 informal groups with at least 100 members. In the first treatment, the training framed insurance as an individual choice with individual-level benefits, showing how the farmer could choose the right policies for herself. In the second treatment, the training framed insurance as providing benefits to members of a group with the sharing of payouts as a means to overcome basis risk,^[8] showing how a group of farmers can select the right insurance option to maximize group benefits. Leaders of these informal groups could then nominate up to some number (randomly determined from zero to 18) of other group members to take the training.

A first finding is that leaders in the second treatment nominated more group members to take the training, despite the ex-post level of understanding of the insurance products having been the same across the two treatments. Differences in post-training discussions were “particularly consistent with a story in which training B increased social learning, enthusiasm or increased risk-sharing among members” (p. 139).

More importantly, treatment also had an impact on demand. The intent-to-treat estimate suggests that average demand in the group went up by 15 percentage points, while the “social” treatment increased the probability that someone in the informal group would purchase insurance at all by 29% (see p. 140, Table 4). The authors find that demand increased only among those selected for training and not among other members of the group (p. 138).

In a 2009 survey in Ethiopia, 44% of farmers reported sizable drought-induced losses in wealth and consumption in the prior four years, and 22% reported losses due to excessive rain and floods (p. 132). Dercon and Christiaensen (2011) also identify a negative association between willingness to adopt fertilizer and weather risk due to consumption smoothing concerns (e.g., in the event of harvest failure) in the Ethiopian context.

Thus, while Dercon et al. (2014) do not explicitly consider income generation as an outcome variable, training in the social benefits of rainfall insurance to increase its uptake may have significant implications for household income and consumption. However, Amolo Ng’weno, a contact at LIF, shared the following caveat:

​​Despite many commercial attempts it has proved very difficult to offer a commercially viable weather insurance product for farmers because either (a) if you live in a disaster-prone area the true cost of the insurance is too high (e.g. 20% likelihood in any given year), or (b) if you live in a low-disaster area you don't see the need for it — always confounded by that the government will (must) step in if there is a catastrophic disaster.

Organization spotlight on Spark Microgrants

Background on Spark Microgrants

Spark Microgrants is a not-for-profit organization founded in 2010 to address low buy-in to aid spending — and therefore low impact of aid — within recipient communities. Their intervention includes a community activation process coupled with a microgrant of $8,000 over two years to implement the community’s desired project.^[9] The process is designed to empower and train community members to undertake their own community development projects and gain a stronger voice in local politics.

According to Spark’s 2020 Annual Report, it has partnered with 325 communities, with approximately 85% of implemented projects generating profits and remaining sustainable one year after the Spark intervention (Spark Microgrants, 2020, p. 4), though on its current website Spark claims to have partnered with 500 communities and to have achieved a 98% project sustainability rate (Spark Microgrants, 2022). It also claims that 85% of communities continue to meet, and that, for every Spark-backed project, communities implement an additional project. Participants are 56% female, and 71% of female leaders are leading for the first time (with 42% female leaders in their Rwandan communities).^[10]

Given that 80% of villages elect to purchase livestock, we modeled the returns from a village’s decision to purchase pigs, primarily for cropland fertilization. To gain some useful background on subsistence pig farming in southern Africa, we read and derived several inputs from Madzimure et al. (2012). To gain similar insights into the productivity impacts of pig manure application on crops, we relied primarily on Rurangwa et al. (2020).

We initially attempted to model our CEA closely on GiveWell’s (2021) GiveDirectly CEA, given that both interventions involve giving funds directly to impoverished people and allowing them to allocate those funds as they choose. Two major differences are that Spark (i) gives unconditional cash to communities instead of individuals, potentially leading to different types and scales of investments and lowering coordination costs, and (ii) includes training and support toward pursuing investment returns and community savings, potentially making it more likely that the recipients will see investment returns as well as providing opportunities for skills and social capital development.^[11]

Our efforts here were ultimately somewhat fruitless, given the major sensitivity to highly uncertain inputs as well as the difficulty of modeling an approximate doubling of consumption in a regular discounted cash flow framework. After we shared our experience, David Kitt from Spark Microgrants mentioned that he had also previously attempted to apply GiveWell’s GiveDirectly CEA framework to the context of Spark and ultimately found the exercise difficult, noting that the framework was not designed to measure intangible and subjective benefits like social cohesion and civic engagement.

In addition to the incompatibility of the GiveWell GiveDirectly framework with the Spark context, we brainstormed a (non-exhaustive) list of several plausible reasons to believe that Spark Microgrants may have a higher or lower impact per dollar spent than GiveDirectly (even if GiveDirectly recipients use their funds to invest in pig farming). For instance, we would expect the impact to be greater if:

• Community-level coordination leads to opportunities with greater returns, but coordination costs are prohibitively high without the Spark intervention.
• Community-level coordination allows for specialization (e.g., in pig farming, crop cultivation, transportation to market, provision of basic veterinary services) that would not otherwise be possible.
• There are benefits from group learning and knowledge sharing.
• Resources (e.g., veterinary services or government vocational training) become more readily available or accessible with a greater number of community members dependent on a given livelihood.
• The investments made through Spark and/or the set-up of a community savings account act as a savings commitment device; for example:
  • It appears households are more likely to sell pigs when it comes time to pay for school fees or health insurance.
  • It is possible that reputational and altruistic motivations — as well-stated community-level goals, like building a school — may make community members more inclined to commit community-level earnings to a community savings account.
• Land or labor constraints preclude individual households from investing in pig farming.

In addition, Kitt mentioned that the Spark intervention may also be more effective since it encourages integration with local government. He mentioned that sector/district governments are increasingly providing technical guidance to encourage pig farming, and technical advisers to the community can remove some of the barriers to pig farming (e.g., identifying veterinary services). He also mentioned that pig farming can be slow to turn a profit, so that individuals on their own may forgo the opportunity and seek alternative investments that pay off more quickly.

Finally, there may be several intangible benefits of Spark’s interventions, which we mention below.

Our modeling suggests that village-level investments in pig farming lead to an expected cost-effectiveness of $28 in income generation per dollar spent on the program over 10 years

Please refer to our Spark Microgrants CEA here. The CEA attempts to roughly calculate the impact of pig purchases on villagers’ consumption and income. Based on our modeling, we find that village-level planning and investment in pig farming leads to an expected program cost-effectiveness of $28.23 in income generation per program dollar over 10 years.

In the livestock investment context, increases in consumption may derive from increased crop production from the application of pig manure and, to a much smaller extent, from pork consumption.^[12] In the event of excess crop production, manure, or pigs and piglets (i.e., via reproduction), village members may sell their surplus for a profit in local markets. In researching model inputs, we focused our attention on the Rwandan context (e.g., using local Rwandan market prices for the commodities we consider), given the highest number of “sparked” villages have been in Rwanda. According to David Kitt, in the Spark context, villages sell piglets and pigs but do not sell manure and may or may not sell crops given that they tend to be far from markets and paying for middlemen and transportation is costly.

We start by using internet-researched information on pigs’ manure production, desired manure application per hectare, baseline crop yields per hectare, and yield increases from manure application to understand the impact of pigs on crop production. We can then subtract the amount of feed per pig, assuming that some percentage of their feed comes from the crops grown, to understand how much of the crop production is left for villagers’ consumption and, given excess, for sale in local markets.

Spark claims that the villagers that benefit from their microgrants more or less double their meals fairly immediately, from somewhere between one to two meals per day to about two or more meals per day. Our CEA with preferred specifications suggests that meals increase by 30% in the first year (from 1.8 to 2.3 meals a day per person), and by the second year that meals have increased by 60% (to 2.9 meals a day per person). Assuming a value of $0.20 per meal — in accordance with the recommendations of Spark’s village stakeholders — implies a consumption benefit of $78 per villager, or $46,700 for a village of 600 people from a 60% increase in meals. From our preferred specification, we calculate an undiscounted consumption benefit of about $25,100 in year one, $61,700 in year two, and $59,600 in subsequent years (during which Spark support has ended).

Finally, we consider the opportunity for villagers to sell excess crops or pigs (products) for a profit.^[13] The maximum number of pigs owned by villages is somewhat constrained by the size of their cropland since there are diminishing returns to manure application. Our calculations indicate that one pig can provide manure for about 0.6 ha each year, such that a village would want to purchase and maintain approximately five-thirds the number of pigs as they have hectares of cropland and sell additional pigs for profit. In our model, the village produces half of its desired number of pigs in year one and breeds those pigs to reach its desired number in the second year. We also assume that a majority of learning takes place in the first year and continues into the second, which we incorporate into the model in the form of higher risk of pig disease/death in these years than in subsequent years. Farmers keep enough piglets to replace dead or diseased livestock year on year.

Having accounted for villagers’ consumption of crops, manure application, and the need to replace dead livestock, we can then calculate the excess crops and pigs that can be sold in local markets, and multiply by the corresponding prices to estimate the cash revenue, which in the first year comes out to about $1,800 and by the second year stabilizes at between $15,000 and $15,100, or about $125 to $126 per household (costs are covered by the microgrant, so households can pocket the full revenue). Discounting the combined benefits from increased meal consumption and profits from market sales by 4% leads us to a program cost-effectiveness of $28 in income generation per program dollar over 10 years,^[14] which does not consider any intangible benefits (e.g., from social and human capital development, reduced violence toward women, and increased political engagement), nor the effects on the welfare of livestock.

Training and labor market interventions

Vocational training, job-search support, and computer skills training combined with job referrals are all effective ways to improve formal sector earnings in the developing world. We are moderately certain that these are the most effective training and labor market interventions that have been explored in the literature, given that we only reviewed two databases. Job-search support workshops have the highest cost-effectiveness of the interventions we reviewed, with Ethiopian job seekers earning an average of 20% more — $6.60 more per dollar spent on the program — in the first year, with potentially much higher cost-effectiveness on longer time horizons given the intervention’s one-time upfront cost. Evidence on the cost-effectiveness of vocational training programs is mixed, and the cost-effectiveness of computer skills training with a job referral is unknown. Our CEA of AkiraChix suggests that, over the first 10 years after graduating, alumnae of the program earn about an additional $11 per dollar spent on the program (medium-high confidence).

Another common source of income for rich and poor alike comes in the form of wages and salaries. About 40% of workers in developing countries are employed in the formal sector (International Labour Office, 2018, p. 14, Figure 5, Panel C), although many of those are still effectively informal due to a lack of benefits, and only 10%-15% of workers in Africa are employed in the formal sector (Bonnet et al., 2019, p. 5). Interventions that help participants develop useful skills can significantly improve their wages or salary, and even allow them to move from the informal to the formal sector (Escudero et al., 2018, p. 2644).

Promising interventions

Subsidized vocational training reduced unemployment by 23% and increased wages by 55% among low-skilled Ugandan workers, but the cost-effectiveness is less than $1 in income generation per program dollar

Alfonsi et al. (2020) assessed the potential of vocational training to improve productivity and earnings in Uganda. Partnering with local small and medium enterprises (SMEs), the J-PAL researchers evaluated the impact of on-the-job training on low-skilled workers’ employment, hours, and wages over time.

Applicants to a vocational training program were randomly assigned to one of four groups: training at a formal external vocational training institute (VT treatment), matching to a firm that would train them as an apprentice at a subsidized rate (FT treatment), matching only with no apprenticeship subsidy (Matching treatment), and a control group. The VT treatment cost about $470, while the cost of FT treatment was reimbursed to each firm at a rate of $50 per month for six months (pp. 2378-2379).

Immediately after the training program, the researchers asked participants to complete a skills test to evaluate their improvement in sector-specific skills. Both VT and FT groups improved their score relative to the control group by over 30% (p. 2386).

Over the following three years, researchers tracked applicants’ success in the job market: their ability to stay employed, number of weekly hours worked, and wages. Results were strikingly different for the VT and FT treatments. Although both groups displayed a marked improvement in sector-specific skills after training, FT trainees were much more likely to be employed shortly after the training ended, likely due to their existing relationship with an employer (p. 2392). However, this advantage quickly dissipated. VT-treated workers were much more commonly able to keep and switch jobs, likely due to their formal certifications.

The study shows the stark difference in value between in-house and external vocational training. Formal certification at a vocational training institute confers the same skills as in-house apprenticeships, but the former allows the worker substantially more lifetime earning power. This difference is probably due to in-house trained workers’ having no formal signal of skill, such that they cannot take their labor elsewhere when eventually re-entering the job market.

As a result, it seems that formal vocational training is the most cost-effective way to provide sustained income-improving skills training for unskilled workers in this context, with a reduction of 23% in unemployment and a wage about 55% higher than the control group over the studied period. With a baseline income of $6 per month, this translates to nearly $400 in additional wages alone, over the first 10 years. At a cost of about $450, the intervention increases income at slightly less than the amount spent, indicating a cost-effectiveness of less than $1 per program dollar.

Combined vocational and life skills training for young women improved annual earnings among young Ugandan women by about $50, with an estimated cost-effectiveness of $5.88 in income generation per dollar spent on the intervention over 10 years

In a three-year study in partnership with BRAC, Bandiera et al. (2020) assessed the extent to which access to life skills workshops and vocational training could improve the lives of young women in Uganda.

The Empowerment and Livelihood for Adolescents (ELA) program is one of BRAC’s flagship programs. The NGO hires a local young woman “mentor” to establish a girls’ club and lead club activities. Such activities include life skills training, where adolescent girls can learn about sexual health and hygiene, family planning, social skills, leadership skills, and how to exercise their own rights as women and as members of the community. BRAC employees and local entrepreneurs also lead vocational training for a variety of potentially income-generating skills, such as hair-dressing, tailoring, computing, agriculture, poultry rearing, and small trades operations.

Bandiera et al. (2020) randomly selected 100 out of 150 communities to receive the ELA program, while the other 50 communities served as a control group. Over the next three years, the researchers tracked a range of life outcomes for a random sample of age-appropriate girls in each community. By the end of the survey, the sample consisted of 3,522 young women. Because of the nature of the intervention, a “treated” adolescent is considered to be one with access to an ELA progam in her community, rather than someone who actually participated in activities.^[15]

The ELA program appeared to have a substantial impact on young women’s economic prospects. In the midline (endline) results, treated women were 67% (48%) more likely to be engaged in income-generating activities, albeit with a baseline of just 10.2% (Bandiera et al., 2020, p. 233). Their perception of their own entrepreneurial ability also improved by about 7.8% (2.5%) in the midline (endline) results. Overall, the annual earnings of eligible women in ELA communities improved by more than 300% (at endline), with the impact on earnings exclusively deriving from self-employment, an increase of about $50 per woman per year in 2008 USD (Online Appendix, p. 4, Table A3).^[16]

In addition to the economic effects, the ELA program also improved gender-specific outcomes for young women in the communities (p. 235). Women with access to the program reported significantly more knowledge about HIV and other STIs, as well as more control over their fertility through increased condom usage (25.3% increase from 45% baseline) and a fall in nonconsensual sex (44% decrease from 17%). Such impacts could have a pronounced effect on the well-being of women in the treated communities.

Overall, the researchers estimate that the ELA program costs about $85 per participating young woman (after initial fixed costs; p. 253), and provides about $50 of additional annual earnings (p. 252, Table 9), which translates to a cost-effectiveness of approximately a $5.88 per dollar spent on the intervention over 10 years (i.e., $50/year × 10 years ÷ $85).^[17] This calculation excludes social benefits such as gender parity, reduced violence, and improved control over family planning.

Job-search support workshops improved income significantly and cost-effectively among Ethiopian job seekers, generating $6.6 ($66) in income per dollar spent on the intervention over a one-year (10-year) time horizon, which could potentially improve with better targeting

As part of a study in Ethiopia, Abebe et al. (2020) partnered with the Addis Ababa University College of Business and Economics to offer a two-day workshop on job-search skills. Their study investigated the impact of such a workshop on employment outcomes in comparison with the effect of a transport subsidy.

The researchers worked with the university to design a job search workshop that may be able to meet the needs of job seekers. In addition to help with CV writing and application letters, students also were trained in interview preparation and received formal certifications for “hard-to-observe” skills such as cognitive, linguistic, and mathematical abilities.

The researchers randomly selected 3,052 job seekers into three treatments: 1) the aforementioned workshop, 2) a subsidy (around $1) to cover the cost of travel to an employment center up to three times weekly, or 3) the control group. They conducted interviews with each participant at the start of the study, eight months later, and four years later. They also held semiweekly short interviews to check in on job search and career progress.

Results for the first treatment were impressive. Although only 61% of the participants invited to the workshop actually attended (p. 17), just the mere offer of the workshop increased the probability of working in a permanent job after eight months by 58%, from a 12% baseline in the comparison group (p. 20). This difference existed despite the fact that the workshop did not appear to have an impact on the probability of employment, hours worked, earnings, or job satisfaction. At the four-year mark, however, the offer of the workshop led to a 20% increase in earnings, and to a 12% increase in job satisfaction. These effects were particularly pronounced among participants whom the researchers had predicted had the poorest labor-market potential (p. 28).

The transport subsidy had somewhat less promising results. Those who were offered the subsidy were 12.5% more likely to look for work, and nearly 30% more likely to visit job vacancy boards, than the control group (p. 23). However, these behaviors had no impact on employment, earnings, or satisfaction in the short term, although they did encourage job seekers to search more actively for employment. After four years, it appears that the only effect of the subsidy was to waste workers’ time and effort, as they did not seem to benefit at all from their increased search activities (p. 22).

The job application workshop generated an average wage gain of $10 per month per worker, compared to a one-time cost of $18.20 per individual (p. 6). This result indicates a potentially high cost-effectiveness of $6.60 per program dollar over a one-year period, or about $66 per program dollar over a 10-year period, assuming treatment effects on income persist.^[18] Since the impact was strongest for those with poor employment prospects, this cost-effectiveness could potentially improve with better targeting.

Computer skills training paired with a job referral increased income by 37% among Kenyan youths, though the intervention’s cost-effectiveness is unknown

In partnership with the local branch of the tech firm Sama, researchers at IPA investigated the combined effects of a computer skills training program alongside a direct path for job referrals.

In their study, Atkin et al. (2021) randomly assigned 1,115 youths in southern Kenya to receive one of three treatments: technical training, training plus a referral to work at Sama, or a control condition. The technical training consisted of mostly basic computer and working skills, while the job referral offered a direct path to a stable job with high pay and great benefits compared to most jobs in the area. The study did not include treatment for the job referral only.

The researchers found that technical skills training had no effect on earnings on its own, but that it could substantially improve earnings if coupled with a direct job referral program. Unfortunately, the study offers no insight into the impact of a job referral on its own. However, the paired intervention lowered unemployment by 10% and increased workers’ earnings by 37% (p. 9), an effect which persisted after 2.5 years (p. 10). The cost of the program was not disclosed, so it is unclear whether the intervention is cost-effective.

Organization spotlight on AkiraChix

Please refer to our AkiraChix CEA here. As part of this report, we investigated AkiraChix, an organization that runs a yearlong boot camp called codeHive to train young women in high-tech skills. The group aims to place boot camp graduates in tech-sector careers, for instance, as software engineers or UX designers.

This intervention is reasonably simple to model. The boot camp is residential and intensive, so costs are somewhat high: $5,000-$6,500 per student.^[19] According to Linda Kamau, the managing director and co-founder of AkiraChix, the program has a 97% graduation rate, and 85%-90% of alumnae go on to work in the tech field. The organization’s operational data suggests that about 65% of alumnae in tech are software engineers, while the other 35% are in roles such as UX designers and tech salespeople.^[20]

CodeHive has only nine cohorts of experience, of which the early cohorts were non-residential, so it would be difficult to draw any strong conclusions about the career paths that graduates follow. However, we can make some inferences from the operational data that AkiraChix provided. After cleaning and analyzing the data, we found that post-graduation salaries grow steadily over time (Figure 2).

Figure 2: Average monthly salary by codeHive cohort. Data from AkiraChix, personal communication, February 21, 2022. All data points calculated with a sample size less than 10 are excluded.

According to Payscale, software engineers in the Nairobi tech scene make around 11,700 USD-equivalent per year, and senior engineers make about 21,400 USD-equivalent. It is difficult to say whether codeHive graduates will follow this payscale; their lack of formal university credentials may count against them, and gender discrimination may further suppress their wages. On the other hand, it’s possible that AkiraChix recruits are qualitatively different from the average talent pool in ways that increase their chances for success. For example, they may be more intelligent or driven than average. Additionally, AkiraChix may be able to help their alumnae through job search assistance and references.

On average, it appears that codeHive alumnae get a raise of about 2,500 USD-equivalent per year in the first two years. Assuming that these raises continue at a slightly decreasing rate,^[21] we can approximate the average payscale in the Nairobi tech scene. Under this model, the average alumna is an intern and junior engineer for the first three years, gets promoted to engineer after three to four years, and becomes a senior engineer after nine to 10 years. She gets paid the same as equivalent engineers in the area, in real terms, in 2019 USD.

Although our model is quite sensitive to assumptions about the average AkiraChix alumna’s salary trajectory,^[22] it is a good fit for the existing operational data, as well as for the industry-wide trends in the local tech sector.

Our CEA finds that, using a standard 4% annual discount rate, the codeHive program would increase the average participant’s earnings by approximately $68,000 (2019 dollars) in the first 10 years. Using an average cost of $6,500 per pupil, we estimate the cost-effectiveness of AkiraChix in improving students’ income at $10.54 in income generated per dollar spent on the program — similar to LIF’s model, which estimates a cost-effectiveness of $10.30 per program dollar.

Our model does not incorporate some concerns about the codeHive program, chiefly the counterfactual, i.e., what would happen if AkiraChix did not exist.

We are reasonably convinced that the individual students would not otherwise be earning anything remotely close to Nairobi tech salaries. AkiraChix goes to great lengths to recruit by word of mouth in extremely disadvantaged regions, such as poor rural areas and refugee camps. However, the organization does recruit very selectively, choosing only those with demonstrated commitment and general aptitude. It is possible that these young women would be somewhat more prosperous than others in their community simply due to innate characteristics.

Furthermore, we are not sure how codeHive graduates compare to those who would otherwise have obtained the jobs that they take. If there are a limited number of jobs in the Nairobi tech center, it is possible that the program “takes jobs from” the counterfactual hires, which would be a concern if those other job seekers were similar in background to AkiraChix alumnae. However, we think it’s more likely that the African tech scene is talent-constrained (Adegoke, 2021), and that new codeHive graduates take jobs that would otherwise not exist, or would not be filled in Africa at all.

Another concern is scalability. Kamau envisions training 10,000 students over the next 10 years. Given that codeHive currently trains about 100 students, achieving this vision would imply an annual increase of nearly 50%, with cohorts of more than 3,000 in 10 years, which seems implausible given the current intensive residential model. However, a scaled-up version of AkiraChix may be able to reduce the per-student costs by spreading overhead over a larger group of students.

When asked about room for more funding, Kamau responded that a substantial amount of administrative time is spent on fundraising, and that they are seriously constrained by a lack of sufficient funding. Kamau mentioned a dual-use case for more money: improving the service to increase placement rates, and expanding to include more students.

We believe that it would be difficult to expand at the rate Kamau indicated, and that the majority of a new grant would go toward covering existing and improved services at the current scale. Thus, in our view, an additional grant can be expected to follow the same model as before, producing 10-12 times the size of the grant in additional income for pupils.

Entrepreneurship and small business support

Among the interventions we came across in J-PAL’s and IPA’s databases, the most effective ones involving entrepreneurship and small business support were the Women’s Income-Generating Support (WINGS) program, in-kind grants to women-owned businesses, the International Labour Organization’s GET Ahead program, and improved access to exporting opportunities (via subsidized credit access). We estimate with medium-high certainty that the cost-effectiveness is $4.70, $6.90-$23.60, and $13.60-$25.50 per program dollar spent for each of the first three interventions respectively. We are only moderately confident that these interventions are among the most effective that have been identified in the academic literature since we only explore the J-PAL and IPA databases and not the broader literature. 

Developing countries typically have nascent local entrepreneurship scenes. Firms in this context are generally small, and Dalberg Global Development Advisors (2011) estimates that as much as 78% of employment in low-income countries comes from small and medium enterprises (SMEs; p. 8, Figure 4).^[23]

Support for small businesses can come in a wide range of forms. Public support and subsidies can spur business formation and innovation (Radas et al., 2015). Organizations can support small businesses through client matchmaking, assistance with business operations, or connections to new markets.

Oyelaran-Oyeyinka and Lal (2006) found that “the contribution to knowledge accumulation through internal training is more relevant and prevalent in Small and Medium-sized Enterprises (SMEs) because they are less able to organize costly and formal external training” (p. 223). Yahya et al. (2012) affirm the positive effects of training, but highlight that innate characteristics of firms and managers may drive demand for training and thus make causal attribution difficult (p. 22).

Promising interventions

The Women’s Income-Generating Support (WINGS) program improved cash earnings by 67%-151% among Ugandan women, with an estimated cost-effectiveness of $4.70 per dollar spent on the intervention (assuming persistent effects)

In partnership with the Association of Volunteers in International Service (AVSI) Foundation, Blattman et al. (2016) studied the impact of the Women’s Income-Generating Support (WINGS) program on monthly cash earnings in northern Uganda. The WINGS program provided services to low-income individuals (primarily women), including basic business skills training, $150 cash transfers (about 30x monthly earnings) for business plan implementation, supervision, and group-dynamics training to encourage helpful group formation (e.g., self-help, savings/lending; Innovations for Poverty Action, 2020). The researchers randomized 120 villages (with 1,800 participants) in two phases, where 30 villages received all interventions apart from the group-dynamics training (Treatment 1), and 30 of those villages received all four WINGS interventions (Treatment 2).

In Treatment 1 villages, monthly cash earnings were 10,400 Ugandan shillings (i.e., 67%) higher than the 15,500 shillings average among villages in the control group at endline (Blattman et al., 2016, p. 48, Table 3).^[24] Treatment 1 villages also saw an increase in durable assets by 0.21 standard deviations relative to the control group villages (p. 48, Table 3), and increased community group membership by 12.9 percentage points relative to control (p. 53, Table 4).

Participants in Treatment 2 villages saw cash earnings increase by 23,400 shillings (significantly different from the Treatment 1 effect at the 5% level), a 151% increase relative to control (p. 48, Table 3). Overall use of loans increased by 110% in Treatment 2, as well (p. 53, Table 4).

The authors interpret the higher cash-earnings differential of Treatment 2, which included group-dynamics training, relative to Treatment 1 with caution given that durable assets and non-durable consumption are not much higher in Treatment 2 than in Treatment 1, and suggest that informal insurance and cooperative activities may contribute toward Treatment 2’s greater apparent boost to cash earnings (p. 52).

Additionally, the WINGS program increased non-farm business opportunities. Treatments 1 and 2 saw approximately 80% of participants in any non-farm self-employment at endline, whereas only 39% in the control group did (p. 48, Table 3). While the interventions did not contribute to statistically significant increases in qualitative measures of empowerment, they appear to have led to higher social support (e.g., receiving comfort when feeling sad) and community support (e.g., speaking out at community meetings) in the last month (p. 54, Table 5).^[25]

Blattman et al. (2016) include a cost-effectiveness estimate in Table 8 (p. 59), which concludes that both Treatment 1 and Treatment 2 have similar cost-effectiveness of about $4.67-$4.78 per dollar spent on the intervention if effects continue in perpetuity, respectively. Over 10 years, if treatment effects remain constant, we should expect, for the average of the two treatments, a cost-effectiveness of about $2.66 per dollar spent.^[26]

Access to exporting opportunities improved Egyptian rug manufacturers’ profits by 16%-26% with only a 5% increase in labor hours, though the authors do not measure cost-effectiveness

Global markets can provide an opportunity to export goods and services. Unlocking access to these markets may be a high-leverage way to improve the performance of local SMEs in LMICs. While there is substantial economics literature on related questions, we describe one study from J-PAL below.

In partnership with a US-based non-profit called Aid to Artisans (ATA) and an Egypt-based distributor called Hamis Carpets, Atkin et al. (2017) randomly provided small-scale rug manufacturers the opportunity to export their products to buyers in the US and Europe. Pre-arranged export orders through Hamis Carpets from foreign rug buyers allowed the researchers to randomly offer fulfillment contracts to a subset of eligible firms (i.e., five or fewer employees, had never worked with Hamis Carpets, purchased their own materials). Of 219 eligible producers, 74 received the opportunity to fulfill these orders, with Hamis Carpets managing the transactions and communications with buyers. Hamis Carpets could continue to work with the treated producers if they had a good experience.

Atkin et al.’s (2017) analysis of data collected from 2011 to 2014 from periodic surveys and master artisanal inspections demonstrates that export opportunities increased small firms’ monthly profits by 16%-26% relative to control — presumably through the 43% increase in prices for their products (p. 579, Table 6) — despite having worked just 5% more hours (pp. 575-577).^[27] The increase in profit is from a baseline of 646 Egyptian pounds, or $102 in 2012 USD, per month.

Moreover, the opportunity to export improved rug quality, with exporting firms scoring 0.79 points (27%) higher on average than control firms (2.96 points) on a five-point scale (Atkin et al., 2017, p. 583, Table 8).^[28] The treatment also led to efficiency improvements: when asked to compare similar rugs using the same equipment, those that had received the treatment produced higher quality rugs (and more quickly) than control firms. Quality and efficiency improvements were apparent after about five months. We agree with the authors’ assessment:

Taken together, the evidence indicates that learning-by-exporting is present in our data and that the learning occurs, at least in part, through information flows. Given that this learning is induced by demand for high-quality products from knowledgeable buyers in high-income countries, these changes [to SMEs in LMICs, in terms of quality, productivity, and profitability] would likely not have occurred as a result of increased market access to domestic markets. (Atkin et al., 2017, pp. 610-611)

Subsidized credit access increased small-scale industrial firms’ total bank borrowing by 20% in India, increasing export growth rates by 24%.

In 1998, the Indian banking sector expanded eligibility for priority sector^[29] lending to include small-scale industrial firms with capital (plant, machinery) investments of less than 30 million rupees ($730,000 in 1998 USD), increasing the eligibility threshold significantly from 6.5 million rupees ($160,000) prior to 1998. Just two years later, eligibility was removed for firms with investments between 10 million ($240,000) and 30 million rupees. Kapoor et al. (2017) analyze the results of this “natural experiment,” using newly eligible firms as their treatment groups in 1999 and 2000. A shortcoming of this study is that eligibility is not random, so the control group — consisting of firms that were already eligible (capital investments < 6.5 million rupees) or firms that were never eligible (capital investments > 30 million rupees) — differs from the treatment group in size of investments, and perhaps also in unobservable ways.

Through difference-in-difference analysis, the authors find that eligibility for subsidized credit leads to a significant positive impact on small firms’ growth rate of bank borrowing.^[30] Compared to firms in the control group, this growth rate increased by about 17% and total bank borrowing increased by 20% following the policy change. While the policy had no impact on the growth rate of sales, the growth rate of exports went up by about 24% relative to control firms (p. 8, Table 2 on p. 15), indicating that these firms had previously been too credit-constrained to incur the costs of entering export markets. Also, growth in credit is correlated with growth in sales to foreign markets, so firms who had entered export markets may still have been credit constrained.

Note that there are certainly other studies in the literature (i.e., outside of J-PAL’s or IPA’s databases) that one could use to check against these findings (e.g., from a very quick search, Tesfom & Lutz, 2008; Martincus & Carballo, 2010), which we could look into given more time.

This study provides some evidence for the potential of increasing access to capital to improve incomes in developing countries. Although in this context it is treated as a policy choice, it could potentially be a charitable intervention, either through dedicated organizations, extending the work of existing organizations, or policy advocacy.

In-kind grants to female-owned enterprises that were already above subsistence-level profits improved profits by about $830 ($2,828) over the first year (three years), with a cost-effectiveness of $6.90 ($23.60) per dollar spent

Fafchamps et al. (2013) analyzed the extent to which female entrepreneurs benefit from different forms of capital infusion into their existing businesses. The researchers followed a similar structure to that of de Mel et al. (2012), in order to investigate the claim that women-owned enterprises tend to be impacted by the “flypaper effect” in which money “sticks where it hits,” meaning that the form in which capital arrives impacts the extent to which it is invested in the business (p. 211). In this context, the flypaper effect for women-owned businesses would allow other members of the household to expropriate the firm’s gains at a higher rate than usual.

In their experiment, the researchers randomly allocated 792 firms to three treatment groups. Half of the firms were allocated to the control group, while one-quarter received 150 Ghanaian cedis (approximately $120 in 2008 USD) in cash, and another quarter of the participating firms were given 150 cedis worth of equipment, materials, or inventories for their business. Firms in the last group were allowed to choose any type of good or equipment that the business needed, but were never given the cash directly.

Uptake of the treatment was 98%, as only nine firms in total were unable to accept the grant once allocated to a treatment group. As such, group sorting is a very close approximation of receiving the grant itself. The researchers conducted quarterly follow-up surveys over the course of the first year, and circled back three years after the grant period for a final survey.

On average, a grant increased monthly profits for firms by about 25 cedis, or $20, per month relative to control over the course of the first year (Fafchamps et al., 2013, p. 219, Table 3). In-kind grants were particularly impactful, with an average increase of between 31 ($25) and 43 ($34) cedis in monthly profits (p. 219, Table 3).

However, this difference in profits was highly stratified by gender and by baseline business performance. Both genders benefited more from in-kind grants, but the difference between cash and in-kind grants was much bigger for women, for whom the treatment effect of cash grants was statistically insignificant (pp. 218-219). In addition, women-owned firms with “low” baseline profits seemed not to benefit at all from either form of grant, while somewhat higher-earning women-owned businesses gained as much as 77-96 cedis ($62-$77) per month in profit. Men’s firms did not experience this differentiation by baseline profits (p. 220, Table 4).

The research thus illustrates the different dynamics of male- and female-owned businesses in Ghana, which was also explored in Bernhardt et al. (2019). For men, the sudden influx of resources can be separated from the household’s finances and straightforwardly reinvested in the business. Female entrepreneurs, however, must contend with household social dynamics and the pressure, for example, to invest at least some of the influx into male relatives’ enterprises, resulting in a lower average return on capital for women-owned businesses in the developing world (Bernhardt et al., 2019, pp. 153-154).

The study points to a promising intervention of in-kind cash transfers as an effective method of increasing profitability, for female-owned enterprises in particular. However, such an approach would, according to this study, only be effective for those women who are already making above-subsistence-level profits. For female-owned enterprises with above-subsistence-level profits, in-kind grants increased profit on average 1,038 cedis ($830) in the first year after the 150 cedi transfer, for a cost-effectiveness of about $6.90 per dollar spent on the intervention. The study did a follow-up survey to assess the impact three years post-treatment, and found that the effect was not significant for most treatments, except for in-kind grants to female-owned enterprises with above subsistence-level profits. In that group, monthly profits had increased by 110 cedis ($88), indicating that the treatment might have led to a total increased profit of 3,537 cedis^[31] ($2,828) in the first three years after the intervention, for a cost-effectiveness of about $23.60 per dollar spent on the intervention**.

ILO’s GET Ahead training program improved profit by 3% — and the impact more than doubled when an 18-month mentorship program was added — among Kenyan women, with the cost-effectiveness of training alone in generating profits estimated at up to $25.50 per dollar spent, and of training with mentorship at $13.60 per dollar spent

McKenzie and Puerto (2021) partnered with the ILO to test the effectiveness of the Gender and Enterprise Together (GET Ahead) business training program on women’s business outcomes.

The GET Ahead program, run by the ILO since 2004, aims to train women in both business skills and gender awareness (Bauer et al., 2004). The aim is to improve financial outcomes, while also assisting women in measures of empowerment. It consists of a five-day training program, plus an additional transport subsidy of about $6 per day. The cost of the training is estimated at $222-$333 per woman trained (McKenzie and Puerto, 2021, p. 305).

The researchers experimented with the efficacy of adding an additional mentorship program 18 months after the completion of the GET Ahead program. Women assigned to receive the mentorship were grouped six to one with a female mentor who would hold biweekly meetings over the course of five months. The researchers estimated the cost of providing the mentoring at $553 per woman (p. 306).

In this study, the researchers randomly assigned 3,537 individuals across 157 local markets in Kenya to three treatment groups. In the first group, women received just the GET Ahead training. The second group was offered both the training and the mentorship program. The third group acted as a control. Four rounds of follow-up surveys with low attrition were used to measure impacts at one and three years after training.

The results showed the potential of both training and mentorship to improve livelihoods over the medium term. Those who underwent training were 3 percentage points more likely to survive as a business after three years, relative to control. They attracted more customers, and also recorded 15% higher profits, after three years, relative to control (p. 311, Table 3). We estimate that the training provides on average $545 in additional profit per year, while the mentorship adds as much as $665 in extra profit on top of the training benefits. In addition, firms exposed to the training showed higher capitalization after three years, indicating some reinvestment of the additional profits. It appears that the effects persist reasonably well over time.

McKenzie and Puerto (2021) also pointed out that training and mentoring increased the use of managerial practices such as keeping records and separating firm and household finances. This effect was particularly pronounced for those who received mentorship (p. 329, Table 12). In addition, the authors note that neighboring non-treated firms did not appear to lose business to the treated firms, which would indicate that new business was created or brought into the market. This is a hopeful sign that the intervention does not have negative externalities on neighboring businesses and competitors (pp. 318-319).

We believe that this study shows promise for the GET Ahead program, alone or combined with mentorship. At the costs and benefits stated above, cost-effectiveness on a 10-year horizon for training alone is $16.40-$25.50, and that of the combined treatment of training and mentorship is $13.60 per program dollar spent.

Conclusion

This research represents a first step toward understanding the potential impact and cost-effectiveness of giving opportunities that target improved income through livelihood improvements. While GiveWell has not historically focused on such interventions — in part due to their preferred set of moral weights^[33] — donors with worldviews that place a higher relative weight on income increases may be inclined to give to interventions that set out to improve income-generating opportunities for some of the world’s most impoverished people.

Our research focused on the general areas of agriculture, vocational training, and entrepreneurship, and certainly does not exhaust the research and organizational network in these spaces. Within agriculture, the interventions we came across highlight the potential of subsidized community-level watchmen to protect against crop theft and improve community relations, nudges and incentives to encourage fertilizer use (including free delivery), in-person and video-based extension services to encourage integrated soil fertility management, and encouraging adoption of rainfall insurance through informal insurance groups. However, estimation of cost-effectiveness is either not feasible with the available information or indicates low cost-effectiveness (i.e., less than $1 in income generated per dollar spent on the intervention, as with subsidized watchmen at the farm level), making it difficult to identify the most promising interventions from the perspective of maximizing income generation per dollar.

Vocational training consists of interventions that help people to be more skilled at finding or completing their jobs, and includes trade schools, apprenticeships, and job search workshops. As for (largely female-targeted) entrepreneurship interventions, multifaceted programs targeting improved business skills, (in-kind) grants, supervision or mentorship, and group dynamics training seem quite promising across a number of contexts, while increasing access to high-income foreign buyers and providing credit access may further improve the income generation potential of SMEs.

We additionally undertake cost-effectiveness analyses for two organizations of interest to the Livelihood Impact Fund. We roughly estimate – with uncertainties largely attributable to uncertainties in model inputs – that each dollar Spark Microgrants and AkiraChix have spent on their interventions has led to $28 and $11 in income generation, respectively.

How we would spend more time

• Academic and NGO expert interviews: Given the scope of the research, we spent quite a bit of time liaising with contacts at AkiraChix and Spark Microgrants, which did not leave much time for engagement with academics and practitioners who may have alerted us to additional interventions and organizations. For instance, Spark has an emerging academic research partnership with Dr. James Habyarimana, and we would consider further outreach to J-PAL and IPA personnel or academic principal investigators that publish extensively in this space. Additionally, we would love to speak with personnel from Heifer International, whom we have not contacted, to see if they have a CEA that may shed light on the robustness of our own.
• Investigation of additional promising organizations: Several additional organizations of which we are aware may merit further investigation (e.g., Tridge for entrepreneurship and small business support — and several others listed here). The focus of our search was on interventions rather than organizations, and given more time, we may be able to identify a number of other promising organizations, or do deeper dives into organizations that seem particularly promising. 
• Explanation of less promising ideas: In our Livelihoods Interventions and Organizations spreadsheet, we came across many ideas which did not seem to increase income at all. It would be useful to explain what these are and why they are not worth looking further into.
• CEA model input research: While we undertook a fair amount of research to inform our model inputs, we would spend additional time searching for causal research to improve them. Additionally, conversations with intervention recipients and further data analysis of Spark Microgrants’ interventions^[34] could help to further narrow our uncertainty regarding several of our model inputs.
• Analysis of impacts on other actors: Our research so far has only focused on the earnings of individual workers, farmers, and business owners. We would be interested to look further into impacts on other actors such as animals and non-earning family members.
• Scope expansion: Finally, further research could expand the scope of the research to build a comprehensive database of interventions either geographically (e.g., globally rather than focusing on SSA only) or in terms of the outcome of interest (e.g., considering interventions that improve the adoption of technologies or practices that may improve prospects for income generation). One could also expand the scope beyond agriculture, training, and entrepreneurship interventions; for instance, the Livelihood Impact Fund also mentioned interest in interventions including tree planting, access to solar productive assets, migration facilitation, and poverty graduation models. We recently came across this resource from the International Initiative for Impact Evaluation, and if we had more time we would likely start by exploring the literature it contains.

Acknowledgments

This research is a project of Rethink Priorities. Ruby Dickson and Greer Gosnell were the lead researchers and main authors of this report. James Hu edited the client-facing version of the report, including supplementing the literature review and reviewing each cost-effectiveness analysis, to transform it into a public-facing report. Jason Schukraft supervised the original research report. Tom Hird reviewed the original report. Melanie Basnak reviewed and supervised the transition from a client-facing to a public-facing report. Thanks to Jeremy Hockenstein and Amolo Ng’weno for helpful comments on drafts. Thanks to Adam Papineau for copyediting. Further thanks to Julie Faller and Alex Cohen (GiveWell), Linda Kamau (AkiraChix), and Lalit Kumar and David Kitt (Spark Microgrants) for taking the time to speak with us. The Livelihood Impact Fund provided funding for this project, but it does not necessarily endorse our conclusions.

If you’re interested in Rethink Priorities' work, you can learn more by visiting our research database. For regular updates, please consider subscribing to our newsletter.

References

Abebe, G., Stefano Caria, A., Fafchamps, M., Falco, P., Franklin, S., & Quinn, S. (2021). Anonymity or distance? Job search and labour market exclusion in a growing African city. The Review of Economic Studies, _88_(3), 1279–1310. https://doi.org/10.1093/restud/rdaa057
   Abdul Latif Jameel Poverty Action Lab. (2013, June 8). Nudging farmers to use fertilizer: Experimental evidence from Kenya. Evaluations. https://www.povertyactionlab.org/evaluation/nudging-farmers-use-fertilizer-experimental-evidence-kenya, archived at https://perma.cc/DJT6-USSD
   Abdul Latif Jameel Poverty Action Lab. (2020, January 16). Video-based and in-person extension services to increase agricultural technology adoption in Ethiopia. Evaluations. https://www.povertyactionlab.org/evaluation/video-based-and-person-extension-services-increase-agricultural-technology-adoption, archived at https://perma.cc/M6TJ-HSYF
   Adegoke, Y. (2021, October 8). The truth about Africa’s tech talent shortage. Rest of World. https://restofworld.org/2021/the-truth-about-africas-tech-talent-shortage/, archived at https://perma.cc/9PD6-E6MP
   Alfonsi, L., Bandiera, O., Bassi, V., Burgess, R., Rasul, I., Sulaiman, M., & Vitali, A. (2020). Tackling youth unemployment: Evidence from a labor market experiment in Uganda. Econometrica, _88_(6), 2369–2414. https://doi.org/10.3982/ECTA15959
   Atkin, D., Khandelwal, A. K., & Osman, A. (2017). Exporting and firm performance: Evidence from a randomized experiment. The Quarterly Journal of Economics, 132(2), 551–615. https://doi.org/10.1093/qje/qjx002
   Atkin, D., Schoar, A., & Wahnschafft, K. (2021). Evaluating Sama’s training and job programs in Nairobi, Kenya. http://www.sama.com/wp-content/uploads/2022/04/Samasource-Evaluation-Final.pdf, archived at https://perma.cc/VJ7Q-TY5Y
   Bandiera, O., Buehren, N., Burgess, R., Goldstein, M., Gulesci, S., Rasul, I., & Sulaiman, M. (2020). Women’s empowerment in action: Evidence from a randomized control trial in Africa. American Economic Journal: Applied Economics, _12_(1), 210–259. https://doi.org/10.1257/app.20170416
   Bauer, S., Finnegan, G., Haspels, N., Deelen, L., Seltik, H., & Majurin, E. (2008). GET Ahead for Women in Enterprise training package and resource kit (2nd ed.). International Labour Office. https://www.ilo.org/wcmsp5/groups/public/---ed_emp/---emp_ent/documents/publication/wcms_116100.pdf, archived at https://perma.cc/PJG9-4LLX
   Bernhardt, A., Field, E., Pande, R., & Rigol, N. (2019). Households matters: Revisiting the returns to capital among female microentrepreneurs. American Economic Review: Insights, _1_(2), 141–160. https://doi.org/10.1257/aeri.20180444
   Blattman, C., Green, E. P., Jamison, J., Lehmann, M. C., & Annan, J. (2016). The returns to microenterprise support among the ultrapoor: A field experiment in postwar Uganda. American Economic Journal: Applied Economics, _8_(2), 35–64. https://doi.org/10.1257/app.20150023
   Bonnet, F., Leung, V., & Chacaltana, J. (2019). Women and men in the informal economy: A statistical picture. https://www.ilo.org/wcmsp5/groups/public/---ed_emp/documents/presentation/wcms_629280.pdf, archived at https://perma.cc/5P6K-BVTE
   Building Markets. (2021, December 18). Model & impact. https://buildingmarkets.org/our-model-impact/, archived at https://web.archive.org/web/20220117164005/https://buildingmarkets.org/our-model-impact/
   Carnahan, M., Durch, W., & Gilmore, S. (2006). Economic impact of peacekeeping [Discussion Paper]. United Nations Peacekeeping.
   Dalberg Global Development Advisors. (2011). Report on support to SMEs in developing countries through financial intermediaries. https://www.eib.org/attachments/dalberg_sme-briefing-paper.pdf, archived at https://perma.cc/L8BU-T2TK
   Dasgupta, M. (2016, September 20). Moving from informal to formal sector and what it means for policymakers. Jobs and Development, World Bank Blogs. https://blogs.worldbank.org/jobs/moving-informal-formal-sector-and-what-it-means-policymakers, archived at https://perma.cc/ENJ8-GCSR
   de Mel, S., McKenzie, D., & Woodruff, C. (2012). One-time transfers of cash or capital have long-lasting effects on microenterprises in Sri Lanka. Science, 335(6071), 962–966. https://doi.org/10.1126/science.1212973
   de Mel, S., McKenzie, D., & Woodruff, C. (2014). Business training and female enterprise start-up, growth, and dynamics: Experimental evidence from Sri Lanka. Journal of Development Economics, 106, 199–210. https://doi.org/10.1016/j.jdeveco.2013.09.005
   Dercon, S., & Christiaensen, L. (2011). Consumption risk, technology adoption and poverty traps: Evidence from Ethiopia. Journal of Development Economics, _96_(2), 159–173. https://doi.org/10.1016/j.jdeveco.2010.08.003
   Dercon, S., Hill, R. V., Clarke, D., Outes-Leon, I., & Seyoum Taffesse, A. (2014). Offering rainfall insurance to informal insurance groups: Evidence from a field experiment in Ethiopia. Journal of Development Economics, 106, 132–143. https://doi.org/10.1016/j.jdeveco.2013.09.006
   Duflo, E., Kremer, M., & Robinson, J. (2011). Nudging farmers to use fertilizer: Theory and experimental evidence from Kenya. American Economic Review, 101(6), 2350–2390. https://doi.org/10.1257/aer.101.6.2350
   Dyer, J. A. G. (2020). Culture, institutions and development in Africa (28093007) [Doctoral dissertation, University of Toronto]. ProQuest Dissertations Publishing. https://www.proquest.com/pqdtglobal/pagepdf/2468405305/Record/3BB63831635B4096PQ/2?accountid=14657, archived at https://perma.cc/FC95-N7GH
   Escudero, V., Kluve, J., López Mourelo, E., & Pignatti, C. (2019). Active labour market programmes in Latin America and the Caribbean: Evidence from a meta-analysis. The Journal of Development Studies, _55_(12), 2644–2661. https://doi.org/10.1080/00220388.2018.1546843
   Fafchamps, M., McKenzie, D., Quinn, S., & Woodruff, C. (2014). Microenterprise growth and the flypaper effect: Evidence from a randomized experiment in Ghana. Journal of Development Economics, 106, 211–226. https://doi.org/10.1016/j.jdeveco.2013.09.010
   Forouzanfar, M. H., Alexander, L., Anderson, H. R., Bachman, V. F., Biryukov, S., Brauer, M., Burnett, R., Casey, D., Coates, M. M., Cohen, A., Delwiche, K., Estep, K., Frostad, J. J., Astha, K. C., Kyu, H. H., Moradi-Lakeh, M., Ng, M., Slepak, E. L., Thomas, B. A., … Murray, C. J. (2015). Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990-2013: A systematic analysis for the Global Burden of Disease Study 2013. The Lancet, 386(10010), 2287–2323. https://doi.org/10.1016/S0140-6736(15)00128-200128-2)
   Giller, K. E., Delaune, T., Silva, J. V., van Wijk, M., Hammond, J., Descheemaeker, K., van de Ven, G., Schut, A. G. T., Taulya, G., Chikowo, R., & Andersson, J. A. (2021). Small farms and development in sub-Saharan Africa: Farming for food, for income or for lack of better options? Food Security, _13_(6), 1431–1454. https://doi.org/10.1007/s12571-021-01209-0
   GiveWell. (2017, November 7). Approaches to moral weights: How GiveWell compares to other actors. The GiveWell Blog. https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/comparing-moral-weights#Valuing_income_versus_health, archived at https://perma.cc/9GSM-YYS5
   GiveWell. (2021). Cost-effectiveness analysis (version 3). https://docs.google.com/spreadsheets/d/1B1fODKVbnGP4fejsZCVNvBm5zvI1jC7DhkaJpFk6zfo/edit?usp=sharing, archived at https://docs.google.com/spreadsheets/d/18Qj-aVge84Fp2fcRKD81m7XO2oij4arufwbySk-4I4M/edit?usp=sharing
   Hecht, A. (2021, October 31). Basis risk: The spread between futures and physical prices. The Balance. https://www.thebalance.com/futures-prices-versus-physical-prices-808962, archived at https://perma.cc/CM96-RU3Y
   Hörner, D., Bouguen, A., Frölich, M., & Wollni, M. (2019). The effects of decentralized and video-based extension on the adoption of integrated soil fertility management – experimental evidence from Ethiopia (Working Paper No. 26052; NBER Working Paper Series). National Bureau of Economic Research. https://doi.org/10.3386/w26052
   Hörner, D., & Wollni, M. (2021). Integrated soil fertility management and household welfare in Ethiopia. Food Policy, 100, 102022. https://doi.org/10.1016/j.foodpol.2020.102022
   Innovations for Poverty Action. (2020, October 3). Cash and microenterprise support for the ultra-poor in Uganda. https://www.poverty-action.org/study/enterprises-ultra-poor-women-after-war-wings-program-northern-uganda, archived at https://perma.cc/84JZ-53GA
   International Labour Office. (2018). Women and men in the informal economy: A statistical picture. https://www.ilo.org/wcmsp5/groups/public/---dgreports/---dcomm/documents/publication/wcms_626831.pdf, archived at https://perma.cc/XF8K-A95Z
   International Training Centre of the International Labour Organization. (2015). Transition to the formal economy: A global knowledge sharing forum—GKSF (p. 2). https://www.ilo.org/wcmsp5/groups/public/---ed_emp/documents/meetingdocument/wcms_456553.pdf, archived at https://perma.cc/JF7J-XLPY
   Kalkuhl, M., von Braun, J., & Torero, M. (2016). Volatile and extreme food prices, food security, and policy: An overview. In M. Kalkuhl, J. von Braun, & M. Torero (Eds.), Food Price Volatility and Its Implications for Food Security and Policy (pp. 3–31). Springer International Publishing. https://doi.org/10.1007/978-3-319-28201-5_1
   Kapoor, M., Ranjan, P., & Raychaudhuri, J. (2017). The impact of credit constraints on exporting firms: Evidence from the provision and subsequent removal of subsidised credit. The World Economy, _40_(12), 2854–2874. https://doi.org/10.1111/twec.12498
   Madzimure, J., Chimonyo, M., Zander, K. K., & Dzama, K. (2012). Potential for using indigenous pigs in subsistence-oriented and market-oriented small-scale farming systems of Southern Africa. Tropical Animal Health and Production, _45_(1), 135–142. https://doi.org/10.1007/s11250-012-0184-3
   McCullough, E. B. (2017). Labor productivity and employment gaps in Sub-Saharan Africa. Food Policy, _67_, 133–152. https://doi.org/10.1016/j.foodpol.2016.09.013
   McKenzie, D., & Puerto, S. (2021). Growing markets through business training for female entrepreneurs: A market-level randomized experiment in Kenya. American Economic Journal: Applied Economics, _13_(2), 297–332. https://doi.org/10.1257/app.20180340
   Oyelaran-Oyeyinka, B., & Lal, K. (2006). Learning new technologies by small and medium enterprises in developing countries. Technovation, _26_(2), 220–231. https://doi.org/10.1016/j.technovation.2004.07.015
   Radas, S., Anić, I.-D., Tafro, A., & Wagner, V. (2015). The effects of public support schemes on small and medium enterprises. Technovation, _38_, 15–30. https://doi.org/10.1016/j.technovation.2014.08.002
   Roser, M. (2013). Employment in agriculture. Our World in Data. https://ourworldindata.org/employment-in-agriculture, archived at https://perma.cc/TU49-FC9A
   Rurangwa, E., Vanlauwe, B., & Giller, K. E. (2020). The response of climbing bean to fertilizer and organic manure in the Northern Province of Rwanda. Experimental Agriculture, _56_(5), 722–737. https://doi.org/10.1017/S0014479720000277
   Spark Microgrants. (2019, October 21). About Spark. https://www.sparkmicrogrants.org/about-spark, archived at https://perma.cc/CP6C-MPMF
   Spark Microgrants. (2020). Annual report 2020 (p. 4). https://www.sparkmicrogrants.org/s/Spark-Microgrants-Annual-Report-2020.pdf, archived at https://perma.cc/5HFA-5X2N
   Townsend, R. F. (2015). Ending poverty and hunger by 2020 (2nd ed.). World Bank Group. https://documents1.worldbank.org/curated/en/700061468334490682/pdf/95768-REVISED-WP-PUBLIC-Box391467B-Ending-Poverty-and-Hunger-by-2030-FINAL.pdf, archived at https://perma.cc/ZRW5-LMMT
   Volpe Martincus, C., & Carballo, J. (2010). Beyond the average effects: The distributional impacts of export promotion programs in developing countries. Journal of Development Economics, _92_(2), 201–214. https://doi.org/10.1016/j.jdeveco.2009.02.007
   World Bank. (2015, June 9). Are African farmers increasingly diversifying their incomes? Agriculture in Africa – Telling Facts from Myths. https://www.worldbank.org/en/programs/africa-myths-and-facts/publication/are-african-farmers-increasingly-diversifying-their-incomes, archived at https://perma.cc/HEF9-7AV4
   World Bank. (2022, April 1). Agriculture and food: Overview. https://www.worldbank.org/en/topic/agriculture/overview, archived at https://perma.cc/8MXD-X8KP
Yahya, A. Z., Othman, M. S., & Shamsuri, A. L. S. (2012). The impact of training on small and medium enterprises (SMEs) performance. Journal of Professional Management, _2_(1), 15–25. Universiti Malaya Research Repository. https://eprints.um.edu.my/9485/1/00009752_89824.pdf, archived at https://perma.cc/GJF9-2XXW

Notes

1. ^{^}

    In the process of collecting information to identify promising interventions, we began by reviewing a wide range of agricultural and vocational training interventions in Ethiopia, the Democratic Republic of the Congo, and Tanzania due to their large populations and relatively low income levels. As we narrowed the project scope, we quickly pivoted to consider only the J-PAL/IPA interventions that specifically targeted income and earnings as an outcome of interest and included all studies in sub-Saharan African countries, and gathered information only on those studies that were completed and had reported results.

2. ^{^}

    Wealthier rural households are more likely to be engaged in non-farm activities and non-agricultural wage labor.

3. ^{^}

    A “nudge” is an intervention intended to change behavior without changing actors’ choice sets or significantly changing their economic incentives to do so.

4. ^{^}

    Note that these are the effects reported in the J-PAL (2013) summary, though the study design over two seasons — with Treatment 1 only provided in the first year, and the rest provided in the second year with a smaller comparison group — leads the authors to present variations on these estimates for T1. The comparison group is much smaller in the second year, which may contribute to the lack of significance for T3.

5. ^{^}

    Somewhat surprisingly, it appears that the authors did not measure actual fertilizer usage (as opposed to adoption), so it is unclear whether the timing of fertilizer receipt would have implications for actual usage (e.g., due to the ability to plan, the salience of the delivery, or seasonal variations in available labor or weather).

6. ^{^}

    ISFM practices include compost, blended fertilizer, improved seeds, line seeding, and lime application to soil, and “is viewed as a strategy to sustainably intensify agricultural productivity and combat land degradation caused by excessive deforestation and inappropriate land-use practices by agriculture” (J-PAL, 2020). 

7. ^{^}

    A microwatershed is an agglomeration of rural communities that share a water outlet.

8. ^{^}

    Basis risk is the risk of imperfect hedging that arises due to differences in the cash price and futures price of a commodity (Hecht, 2021).

9. ^{^}

    Spark provides $4,200 in year one and $3,800 in year two to encourage learning.

10. ^{^}

     To calculate their social return on investment, Spark uses a “convenience sample of 19 communities from Rwanda,” though we do not understand — and have not asked — why it restricts its sample to just these communities.

11. ^{^}

     Other intangible benefits suggested by quotes on their website are the development of a savings culture, reduction in domestic violence, relationship development between communities and local governments, and increases in civic action.

12. ^{^}

     Given that pigs’ natural lifespan is about 15-20 years, i.e. longer than the 10-year period we consider in assessing cost-effectiveness, we do not separately model pork consumption but instead assume that households consume pork relatively infrequently. 

13. ^{^}

     Given some idea about the diets of villagers from Spark’s 2021 end-of-year survey that it shared with us, we adjusted the village’s crop diversification in the “Crop inputs” sheet of our CEA to closely match their consumption, which ultimately eliminates revenues from crop sales and costs from food purchases. Therefore, all of the revenue comes from the sale of piglets, and households do not supplement their food consumption with off-farm crop purchases, in line with anecdotal evidence from Spark.

14. ^{^}

     We use a discount rate of 4% on future returns; using a discount rate of 0% increases the cost-effectiveness to $34 in income generation per program dollar.

15. ^{^}

     As such, the results we discuss below are intent-to-treat effects (i.e., the effect on women who were offered the opportunity to participate in the program), and we would expect treatment-on-treated effects (i.e., the effects on women who actually participated in the program) to be higher.

16. ^{^}

     We use the conversion of 1,700 Ugandan shillings (UGX) per dollar based on the January 2008 exchange rate reported in the paper. Per Table A3 of the online appendix, baseline earnings and the increase in earnings at endline translate to $16 and $50, respectively, such that earnings are $76 at endline.

17. ^{^}

     The researchers do not include effects past the four-year mark. It is possible that impacts persist or drop off after this period; we don’t have a sense of which assumption is more realistic.

18. ^{^}

     This number results from a simple back-of-the-envelope calculation, i.e., (12 months/year × 10 years × $10/month) ÷ $18.20 for the 10-year time horizon. One could also imagine income gains becoming even higher, as any proportional wage increases would be made relative to the higher baseline income induced by the treatment effect. Thus, it seems plausible that the effect on income would be even greater in the long run, while the cost of the intervention remains constant, leading to an even higher long-run cost-effectiveness.

19. ^{^}

     Kamau stated the cost was around $5,000, which varies based on the economic situation in the country. However, operational data suggests that the total cost, including recruitment, administration, and tuition, may be closer to $6,500.

20. ^{^}

     In the US market, software engineers typically make 5%-10% more than UX designers (Coren, 2016). Our very limited data from AkiraChix indicates this gap may be somewhat smaller in the Kenyan tech scene.

21. ^{^}

     In our model, we use a 5% reduction in raise year-on-year, that is, each year’s raise is 95% the size of the previous year, in real terms. Because the initial raises, from intern to junior engineer, and from first- to second-year junior engineer, are so large (122.55% and 52.31% respectively), this still translates to substantial raises after 10 years. We think this is a reasonable assumption given the high initial benefits, but the model’s output does depend on it.

22. ^{^}

     We assume that graduates get large initial annual raises which taper off at 5% per year. If, for example, we change this assumption to allow for increasing raises, or a less smooth salary trajectory, the results can change substantially.

23. ^{^}

     We aren’t sure how this squares with the assertion that around 40% of employment in LMICs comes from the formal sector. A large proportion of SMEs in Dalberg Global Development Advisors (2011) are informal or “non-employer” firms (p. 7, Figure 2).

24. ^{^}

     Blattman et al. (2016) also measure a consumption treatment effect in their paper, which is three times the earnings treatment effect ($15.50 vs. $5.19 per month). They therefore suggest that the consumption effect act as an upper bound and the effect on cash earnings effect act as a lower bound, leading to a $13-$39 treatment effect in purchasing power parity terms (pp. 49-50).

25. ^{^}

     The group dynamics intervention appears to have additionally insulated against the increased neighbor disputes and verbal abuse identified in Treatment 1 villages, with qualitative research suggesting these were acts of jealousy or resentment from a handful of households (Blattman et al., 2016, p. 55).

26. ^{^}

     To obtain this estimate, we used the NPV formula the authors specify in Table 8, with a time horizon t of 10 years and a discount rate of 5% (in line with the authors’).

27. ^{^}

     The authors report the intent-to-treat results as their headline findings, but also report the somewhat higher treatment-on-treatment result of 28%-42%.

28. ^{^}

     The treatment-on-treatment results show exporting firms score 1.35 points, or 46% higher, on rug quality than control.

29. ^{^}

     Priority sectors included agriculture, small scale industries, and “the weaker sections of society” (e.g., self-employed and microcredit; p. 3). The Government of India mandate required that 40% of net bank credit be allocated to the priority sector.

30. ^{^}

     Note that the authors chose to use the growth rate instead of growth, which strikes us as unconventional. We did not have time to investigate why they did so, and if it makes more sense to analyze this variable in this specific study.

31. ^{^}

     To obtain the total increased profit over three years, we assumed that the increase in monthly profits between years one and three was linear, implying an intermediate profit value in year two.

32. ^{^}

     This cost might include overhead, rental facilities, and physical inputs such as lumber and screws.

33. ^{^}

     GiveWell values an additional year of healthy life approximately 2x-3x as much as a doubling of income, which it does not view as particularly robust given the limitations of the value-of-statistical-life and disability-adjusted-life-year literature it examined (GiveWell, 2017).

34. ^{^}

     We received data for both of these organizations the day before our deadline for finishing the report, so we could only undertake minimal analysis. More sophisticated analysis and the opportunity to ask questions and seek additional data could help to refine our CEA inputs.