We expect that reading the full report will be helpful for:

• Those who want to dive deeply into the evidence base for cost-effectiveness
  estimates of interventions aimed at reducing malaria cases
• Those who are interested in quantitative estimates of how getting malaria at
  an early age affects the future income of a child

Context

In July 2023, GiveWell commissioned Rethink
Priorities to conduct research on the long term income effects of childhood
malaria (“developmental effects”). GiveWell had based its estimate for the
income effects from childhood cases of malaria on two studies: Cutler et al. (2010) and Bleakley (2010). They used the
results of these studies to calculate that each malaria case in childhood
reduces future income by 2.3%. They then adjusted this estimate downwards to
account for concerns about how accurately these studies estimate the long-run
income effects of childhood malaria cases (GiveWell, 2023, “Key uncertainties and
future research”). At the time of our research, GiveWell’s assumption after
these adjustments was that income in adulthood decreases by 1% per childhood
malaria case. It asked us to update its previous literature review to determine
whether any further evidence should be considered and whether the assumption
should be adjusted.

Research process

• A literature search of about four hours to check whether a list of studies
  previously found by GiveWell was complete
• A review of eight studies:
  • A detailed review of two studies that GiveWell uses to calculate long term
    income effects from childhood malaria
  • A detailed review of four other studies on developmental effects from
    malaria
  • A shallow review of two other studies (one about malaria, and one about
    childhood vaccinations)
• A sense check of the income effect GiveWell assumes per childhood case of
  malaria. We made a simple model to calculate the income effect per child that
  gets severe malaria, under the assumption that the income effect is fully caused
  by children that get severe malaria.

Final report and key takeaways

All of the studies we reviewed were considerably limited by study design and
other issues, including both studies used by GiveWell to calculate developmental
effects (Cutler et al., 2010
and Bleakley, 2010), making us
skeptical about the accuracy of the effect sizes found. In particular, both
Cutler et al. and Bleakley are non-randomized and use very imprecise data for
malaria prevalence, and we also found issues with the ways the studies measured
income. GiveWell was aware that there were some issues, and asked us to research
whether there was better evidence available (GiveWell, 2023, “Long-run income effect
from eradicating malaria in childhood”).

Of six additional studies that we reviewed, we think Mora-García (2018) used somewhat more
accurate data on malaria prevalence than Cutler et al. and Bleakley. However,
our confidence in their estimates was reduced by other methodological issues. Shih and Lin (2018) initially looked
promising, but after a more detailed assessment, we advised against using their
results because they found very different effects for two separate definitions
of income, and we could not find a reasonable explanation for this.

All in all, we recommended GiveWell include the results from the study by
Mora-García in its calculations. This incorporation would revise the estimated
income reduction per malaria case from 2.2% to 2.5%. This corresponds to a
change from 1% to 1.1% after applying GiveWell’s adjustments for concerns about
the accuracy of the study results. GiveWell did not update its pre-adjustment
effect size, but our methodological critiques of the various supporting studies
re-affirmed its impression that the evidence is medium-low quality, and that
it’s appropriate to put substantial weight on its priors. After the resulting
changes, as part of a wider range of discussions, GiveWell has reduced its
(post-adjustment) estimate from 1% to a 0.6% income reduction per childhood
malaria case (GiveWell, 2023, “Increase
in annual income per child malaria case averted”).

Additionally, we developed a simple model to estimate the impact on income for
each child who suffers from severe malaria, under the assumption that this
impact is entirely due to severe malaria in children. The model can be found here.
This model serves as a method to check if the average income effect used by
GiveWell is reasonable. Using this model, we conclude that to achieve a 1%
average increase in income per malaria case averted, there needs to be a 37%
increase in income for each averted case of severe malaria. It’s important to
note that it relies on uncertain assumptions about the prevalence of severe
malaria, which we could not find any data for. We believe this income effect
sounds reasonable, but we would have more confidence in this conclusion if we
had consulted with experts on the consequences of malaria.