One of Evidence Action’s main programs is our Dispensers for Safe Water initiative. Chlorine dispensers currently serve over 1.8 million people in East Africa and are poised to scale to serve four million people by the end of this year. Dispensers represent an important innovation in the rural water sector because they solve several challenges that have hindered sustainable, quality services in the sector traditionally:
- High rates of adoption and evidence of health impact: Chlorine dispensers are supported by a rigorous evaluation that demonstrates their long-term sustained adoption and cost effectiveness relative to alternative strategies.
- Sustainable service delivery: The traditional focus of water programs has been on infrastructure provision and community management, which often leads to underinvestment and disuse over time. As the sector shifts to a service-delivery approach, dispensers represent an entrepreneurial element of the “everyone, forever” philosophy. Chlorine dispensers are explicitly provided as a public service with a business model that plans for on-going supply and maintenance to be covered through a sustainable revenue stream.
- Sustainable financing in a low carbon price environment: Interest in using carbon credits to support water projects has faded as carbon prices have fallen. Chlorine dispensers are cost-effective enough to survive and thrive in a low-price environment. Given the low cost of this program, which will cost less than $0.50 per person per year at scale, even at a continued low price of carbon we are able to cover program costs with carbon revenue when we are serving 25 million people in 2018.
In a series of upcoming blog posts, we will provide more information about our commitment to a service delivery model, as opposed to an infrastructure provision model, and carbon finance. We will also provide an update on adoption rates at chlorine dispensers, now that we are long past the research stage of the program.
This blog post summarizes our take on the evidence supporting chlorine dispensers. We discuss the evidence from the epidemiology and economics literature for the health impacts of chlorine and other approaches to water quality improvements, as well as the important role of compliance or adoption in realizing health benefits.
Chlorine
Using chlorine dispensers improves the microbiologic quality of drinking water in the household by dosing water with dilute sodium hypochlorite, or calcium hypochlorite. Chlorine is very cheap, and is very effective at killing most bacterial and viral pathogens. It is less effective against parasites like cryptosporidium and when there is a large amount of organic matter in water. Safety concerns with dilute chlorine are minimal, and it is widely used as a disinfectant in water treatment plants around the world.
Chlorine can provide residual protection for up to three days (depending on storage conditions), which means that it not only disinfects water, but can also prevent recontamination. This is important in places where people store collected water in their homes, as safe handling may be hard to guarantee. The potential to keep water safe during storage in everyday containers makes chlorine different than other technologies like filters that are advocated for use in homes.
You can learn more about chlorine and tools for considering risk-benefit trade-offs that go with different water treatment technologies that have different cost effectiveness profiles in the WHO Guidelines for Drinking Water Quality (see Chapter 7, especially).
Epidemiology Evidence of Impacts
The epidemiology literature contains three systematic review articles that we know of that survey and assess the evidence for drinking water quality interventions in developing countries. These reviews include a Cochrane Review, a systematic review by the International Initiative for Impact Evaluations (3ie), and a review of the literature on chlorine, specifically, by Arnold and Colford. Each in turn concludes that the health benefits of water quality interventions like chlorination are clear, but dependent on consistent use, which can be a challenge with some technologies. Similarly, the concern that the evidence comes mostly from relatively short studies is also a consistent theme.
Note that in the text below there are references to “point-of-use” treatment, to distinguish treatment in the home from treatment at a water treatment plant. Dispensers may be thought of as “point-of-collection” treatment, rather than point-of-use; but epidemiologically this should not matter. As we discuss later, this matters for compliance and sustained adoption, instead.
The Cochrane Review summary states:
This review examined trials of interventions to improve the microbiological quality of drinking water. These include conventional improvements at the water source (e.g. protected wells, bore holes, and stand posts) and point-of-use interventions at the household level (e.g. chlorination, filtration, solar disinfection, and combined flocculation and disinfection). The review covered 38 independent comparisons from 30 trials that involved more than 53,000 people. In general, such interventions were effective in reducing episodes of diarrhea. Household interventions were more effective in preventing diarrhea than those at the source.
The 3ie review covers 65 rigorous impact evaluations of water, sanitation and/or hygiene interventions on diarrhea morbidity, covering 71 distinct interventions assessed across 130,000 children in 35 developing countries during the past three decades. It concludes that point-of-use water quality interventions appear to be highly effective – and indeed, more effective than water supply or source treatment in reducing diarrhea – but that this is very sensitive to the ability of the program to sustain high rates of product adoption. A point of use product cannot provide health benefits if people don’t use it.
Arnold and Colford conducted a systematic review of all studies that measured diarrheal health impacts in children and the impact on water quality of point-of-use chlorine drinking water treatment. Twenty-one relevant studies were identified and combined using meta-analysis to provide summary estimates of the intervention effect. The intervention reduced the risk of child diarrhea by 29%, and up to 40% if you look at effects only among people actually using the product, as opposed to only being offered it.
There are shortcomings to the existing epidemiology literature. The reviews all document the short duration of the studies that are available for their analysis, and the need for more trials that are blinded or rely on objective measures of health outcomes instead of self-reported diarrhea.
The Economics Literature: Evidence of Impacts
There are a series of papers in the economics literature that complement the epidemiology evidence on water quality and add to our understanding of the health impacts of water quality interventions. Through the lens of history, it’s quite clear that providing improved water quality can drastically reduce child mortality. Cutler and Miller (2005) [ungated version] use historical variation in the timing and location of water filtration and chlorination technology adoption across U.S. cities to identify the contribution of improved water quality to improved public health in U.S. cities. They find that clean water was responsible for about half the observed decline in mortality and nearly two-thirds of the reduction in child mortality in cities.
In a rural setting Watson (2006) [ungated version] analyzes the introduction of various water and sanitation interventions on Native American reservations in the United States during 1960–98. She uses statistical techniques to capture the impacts of this intervention alone, distinct from other factors affecting infant health. This research suggests that a 10% increase in the fraction of homes with improved water and sanitation services reduced infant mortality by 4%. Infant mortality rates fell among local residents not living on the reservation, and thus without access to the new services, as well—which means that there were significant positive externalities associated with the program.
Galiani, Gertler, and Schargrodsky (2005) study a privatization reform in 30% of municipal water companies in Argentina in the 1990s to identify the impact of the expanded coverage with safe water. They estimate that child mortality overall fell 5–7% in areas that privatized their water services because the reform did succeed in expanding coverage with safe water. The effect was largest in the poorest areas, at around 24%.
Some may question whether these service changes are comparable to what happens when point of use, or collection, technologies are introduced in developing countries today. Of course, we cannot know exactly how disease environments differ over time and place, but the balance of evidence is that efforts to reduce exposure to fecal-oral diseases via contaminated water has important health benefits.
There is an evaluation in the economics literature in a present-day rural setting that is most closely comparable to the setting where dispensers are used: Kremer and colleagues study a “spring protection” program that improves water quality at the source in rural areas, just as dispensers do. Spring infrastructure investments reduce fecal contamination partially, but not completely, just as one might think of chlorine doing (though of course without the benefits of residual protection). Nonetheless, child diarrhea, as reported by mothers, falls by one quarter. There are long lags between data collection rounds in this study, which take place over the course of several years. This is important, as it is in contrast to the shorter term epidemiological studies that were a concern to the reviewers above. It’s a useful data point that this approach results in estimates of health benefits that are pretty close to those in the epidemiology literature.
The Question of Reporting Bias
Despite the apparent consensus in the epidemiology literature summarized above, the question of quality of the evidence for water quality questions has been re-opened with a series of papers that demonstrate that bias due to the use of self-reported, subjective data may drive estimates of impact reported in the epidemiology literature. Schmidt and Cairncross note that there appear to be differences in treatment effects between blinded and unblinded studies, suggesting that when respondents know that they have been given a water treatment technology, they may change how much illness they report. Moreover, a double-blind study done in Orissa, India found no impact on childhood diarrhea from a point-of-use water treatment, though it is hard to know what to make of this result as there were low adoption rates of the intervention combined with low diarrheal prevalence rates from the start. In any event, these concerns led Schmidt and Cairncross to call for slowing down efforts to scale up household water treatment, lest it distract from efforts to improve access or other community-level interventions. Not surprisingly, this call proved controversial, as you can see from the later dialogue with the author of the Cochrane Review, Tom Clasen, and others, and questions recently raised again at the 3ie blog.
The economics literature has also documented how the traditional epidemiology approach to measuring impacts – frequent surveys about a subjective measure of health – can generate misleading results. In three field studies of health interventions, subjects were randomly assigned to be surveyed about health and then researchers measured subsequent use of a related product with data that does not rely on subjects’ self-reports. In each case, being surveyed changes behavior, and in particular, asking about diarrhea increases use of water treatment products (with associated health impacts). This means that surveying itself can complicate our efforts to understand the impacts of water interventions in traditional randomized controlled trials.
It seems clear that more studies with objective outcomes, infrequently measured, would be valuable. However, we think this concern can be overstated. Not only does the historical record suggest that improving access to safe water can save lives, but the extent of reporting bias in treatment groups would have to be very large to explain the reported reductions in diarrhea associated with cleaner water that have been documented multiple times in epidemiology trials. In fact, to the extent that reporting bias lowers estimates of diarrhea in both the treatment and the comparison groups, such bias would make it harder to statistically detect reductions in diarrhea. If the reductions in diarrhea were even a fraction as large as those estimated in the epidemiology literature and the evaluation in rural Kenya, water treatment would still be very cost-effective.
Acceptability Concerns and Adoption
The case for cost effective impacts for dispensers rests not only on the potential for health impact. People actually have to use water treatment products, after all, if they are to be effective. The high and sustained take-up of dispensers relative to point-of-use chlorination has been documented via a rigorous randomized controlled trial in the region where we work, in which adoption was sustained at about 50% for three years in the treatment group assigned to dispenser access. In the case of dispensers, the concerns raised by Schmidt and Cairncross about acceptability have been addressed. Even at scale, we continue to see average adoption rates (measured using an objective and conservative water testing approach) over 40% in areas where baseline water treatment with chlorine was less than 10%. We believe that the point-of-collection approach to service provision drives up adoption of chlorine by making it cheap, convenient, salient, and public.
WASH-Benefits and Scaling Up in Parallel
We do need to know more about how water, sanitation, and hygiene interventions interact with each other, and it would be better to have objective measures of outcomes, rather than relying on self-reported diarrhea. That’s why we are excited about the large WASH Benefits evaluation, which includes chlorine dispensers as a water treatment method, and the SHINE evaluation, both of which measure health impacts of water, sanitation, and hygiene using objective measures of child morbidity. , we welcome the iterative cycle of learning and scale-up in the case of safe drinking water.
