Few studies in development economics have travelled as far as a school-based deworming trial in western Kenya. The Primary School Deworming Project began in 1998 across 75 primary schools in Busia district. Its estimates reshaped aid budgets, inspired a global initiative, and became a staple of effective altruism. Pills costing well under a dollar per child per year appeared to keep children in school and, years later, to raise their earnings.

Then, in 2015, the evidence went on trial. Independent replication teams, working through the International Initiative for Impact Evaluation (3ie), reanalysed the original data and reported coding errors and fragile results. Headlines announced that the “worm wars” had begun. The original authors pushed back. Economists and epidemiologists found themselves disagreeing about more than deworming. They were disagreeing about what counts as credible inference.

The dispute is worth revisiting precisely because it was never fully resolved. It shows how specification choices, outcome measurement, design features, and disciplinary conventions interact to produce the numbers we cite. For anyone designing or funding interventions, the deworming saga offers an unusually transparent view of how evidence is built, stress-tested, and weighed under uncertainty. The question is not simply whether deworming works. It is how practitioners should act when careful researchers, examining the same data, reach different conclusions.

Miguel and Kremer's 2004 Econometrica paper tackled a problem that had quietly undermined earlier deworming trials. Intestinal helminths spread through contaminated soil and water, so treating one child reduces transmission to others. Individually randomised trials, the standard in medical research, therefore compare treated children against controls who also partly benefit. The resulting estimates are biased toward zero, potentially understating the true effect of treatment.

Their response was to assign treatment at the school level. Seventy-five schools were divided into three groups, phased into treatment in 1998, 1999 and 2001, creating a stepped rollout. Treatment schools received albendazole and, where schistosomiasis prevalence was high, praziquantel. The authors compared schools and exploited variation in how many treated pupils lived nearby. This allowed them to estimate both direct effects and cross-school externalities.

The headline findings were striking. School absenteeism fell by roughly one quarter in treatment schools. Spillovers were sizeable: untreated children in treatment schools benefited, and so did pupils in neighbouring schools. Test scores did not move significantly. Because the intervention was so cheap, the estimated cost per additional year of schooling was only a few dollars. That was far below alternatives such as conditional cash transfers or free uniforms.

Long-run follow-ups strengthened the case. The “Worms at Work” study, published in 2016, tracked participants roughly a decade later. Those who received more years of treatment worked more hours and were more likely to hold manufacturing jobs. A later twenty-year follow-up reported gains in consumption and earnings. Taken together, the implied returns placed deworming among the most cost-effective interventions ever evaluated.

Policy followed quickly. The Deworm the World Initiative launched in 2007. Governments in Kenya and India scaled school-based programmes to tens of millions of children. GiveWell placed deworming charities among its top recommendations. Yet a single trial, in one high-prevalence district, was carrying an enormous evidentiary load. That concentration of influence would matter when the scrutiny arrived.

Takeaway

Designs that capture externalities can reveal effects other designs miss. But the more an intervention's case rests on one study, the more that study's fragilities become policy fragilities.

In 2015, the International Journal of Epidemiology published two papers by researchers at the London School of Hygiene and Tropical Medicine. The first was a pure replication, rerunning the original analysis on the original data. It found errors, most consequentially in the variables measuring how many treated pupils lived near each school. Correcting them removed the statistically significant cross-school spillovers at greater distances.

The second paper was a statistical replication, reanalysing the data using conventions standard in epidemiology. Here the disagreements became methodological rather than mechanical. The reanalysts examined the two trial years separately instead of pooling them, and they applied different weighting. They also stressed risks of bias. Schools had been allocated from an alphabetically ordered list rather than by formal randomisation. Attendance was recorded through unannounced visits whose frequency differed across arms. Under these choices, the attendance effect weakened and, in some specifications, lost significance.

Hicks, Kremer and Miguel responded that the core findings survived. They argued that splitting the sample by year discarded statistical power and ignored the logic of the stepped design. The corrected data, they noted, still showed significant effects on attendance and within-school spillovers. And they argued that the reanalysis made several choices that each seemed defensible but together pushed estimates toward null. Independent commentators, including Michael Clemens and Justin Sandefur, largely concluded that the original story was weakened at the margins rather than overturned.

Meanwhile, systematic reviews pulled in a different direction. The Cochrane review of mass deworming, updated in 2015, found little or no effect on weight, haemoglobin, cognition or attendance. It graded much of the evidence as low quality. A Campbell Collaboration review reached broadly similar conclusions. Critics countered that these reviews pooled settings with low infection intensity and short follow-ups. That pooling would mechanically dilute effects concentrated among heavily infected children.

What made the episode instructive was that nobody was accused of fabrication. Miguel and Kremer had shared their data, which is precisely why replication was possible. The dispute turned on three questions. How much did researcher degrees of freedom matter? What was the right estimand? And which tradition better captured credibility: economics' acceptance of pooled designs with clustered standard errors, or epidemiology's risk-of-bias scoring? Two disciplines looked at the same dataset and saw different studies.

Takeaway

A replication can reveal a coding error, but a reanalysis mostly reveals a set of choices. Telling the two apart is the first step in deciding whether a finding has actually been weakened.

The first lesson is that robustness is a distribution, not a verdict. Any empirical result sits within a space of reasonable specifications. Evaluators should not stop at asking whether an estimate survives one alternative. They should ask how it behaves across the full set of defensible choices. Specification curves and multiverse analyses make this explicit, showing whether significance depends on a narrow corridor of decisions or holds broadly.

Second, pre-analysis plans would have narrowed the dispute considerably. Much of the conflict concerned choices that could have been fixed in advance: how to pool years, which attendance measures to prioritise, how to define spillover radii. Registered trials do not eliminate disagreement. They do establish which analysis carries the burden of proof and which analyses are exploratory.

Third, external validity matters as much as internal validity. Busia had exceptionally high infection rates, and plausible mechanisms suggest effects scale with worm burden. A pooled null across low-prevalence settings and a strong effect in one high-prevalence district may both be true. The relevant policy question is less whether deworming works than where, for whom, and at what infection intensity.

Fourth, contested evidence can still justify action. GiveWell continued recommending deworming but applied a steep replicability discount to the estimated long-run effects. Its argument was that low cost meant even modest true impacts would justify the spending. This is expected-value reasoning under uncertainty. It is defensible only when the discount is explicit, the downside is limited, and new evidence is actively sought.

Finally, replication should be institutionalised rather than treated as an attack. The deworming debate was bruising partly because it arrived as an event, amplified by headlines framing it as a debunking. Routine reanalysis, funded and expected, would lower the stakes for original authors and replicators alike. Fields where scrutiny is normal produce more trustworthy evidence. Their studies are not flawless, but their flaws are found early.

Takeaway

Strong evidence is not evidence that never gets challenged. It is evidence whose conclusions you can still state, with honest confidence bounds, after the challenges are done.

The deworming wars did not settle whether mass deworming is a development bargain. They clarified something more durable: influential evidence can be fragile, and much of that fragility lives in analytical choices rather than errors.

For program designers, the implications are concrete. Pre-register analyses, especially for spillover designs. Report results across specifications. Fund long-run follow-ups and multi-site replications before scaling on a single trial. Build explicit discounts for replicability and context into cost-effectiveness models, so decision-makers see the uncertainty rather than inherit hidden confidence.

Above all, treat replication as part of producing evidence, not a referendum on researchers. Miguel and Kremer's willingness to share data turned a dispute into a public good. The field learned more from the argument than it would have from silent acceptance. Evidence earns its authority by surviving scrutiny, and by showing exactly what survived.