Overview
This blog is part one of a two-part series about our incubator, Beta’s, collaboration with Young 1ove on the “No Sugar” intervention. Check out part two in this series here.
What happens when you tell middle-school and teenage girls in Africa about the dangers of sexually engaging with older men who offer them financial favors? Does it affect their choice of sexual partner? When Stanford Professor Pascaline Dupas tried it in Kenya over ten years ago, it seemed to. Fast forward to 2014 and a Botswana-based non-profit, Young 1ove, brought together a group of partners, including the Government of Botswana, global research center the Abdul Latif Jameel Poverty Action Lab (J-PAL), Botswana-Baylor Children’s Clinical Centre of Excellence, and Evidence Action (through our incubator, Beta), to evaluate the idea again. This time, through a similar program, “No Sugar,” designed with the potential to scale up across Southern Africa. The evaluation of the “No Sugar” program yielded more mixed results than the original study in Kenya. As a result, the Government of Botswana, Young 1ove, and other partners are not scaling up No Sugar as it was originally designed. Instead, Young 1ove is redesigning the program for further evaluation of impact, before potentially scaling it up in the future.
The evidence-base: Why telling girls about the risks of older sugar daddies seemed like a good idea
In the Kenya-based randomized controlled trial, girls were informed that middle-aged “sugar daddies” were more likely to be infected with HIV than younger men. The study was designed to test whether empowering girls to make more informed choices about their sexual partners might be more effective than trying to convince them not to have sex at all. (The latter was the goal of sex education in much of Africa at the time.)
At the study’s onset, the researchers hypothesized that telling girls about the dangers of sugar daddies would change girls’ risk-reward assessments and incline girls away from sugar daddies. Why? Because girls tend to (mistakenly) think that older men are actually safer partners, and see the financial rewards of sugar daddies as outweighing the risks of contracting HIV. If girls could be encouraged to choose younger partners, they would then benefit twice over: they would avoid high-risk older partners and feel more confident to negotiate for safe sex practices with their lower-age partners. As a result, girls’ risk of getting pregnant would decrease, as would their risk of contracting HIV.
The results of the RCT in Kenya were impressive: girls who received the “sugar daddy” messaging were 28 percent less likely to be pregnant one year later than girls who were simply told to abstain from sex. In the study, pregnancy was used as a proxy for unprotected sex and exposure to risk of HIV infection.
If applicable to other contexts, the Kenya-based research offered a compelling and potentially cost-effective solution to the transactional sex problem that is a key contributor to the disproportionately high HIV infection rates among young women and girls in many parts of Southern Africa.
Turning evidence into action: Developing and testing “No Sugar”
Seeing its potential, social entrepreneurs Noam Angrist, Moitshepi Matsheng, Brenda Duverce, and Unami Moatswi took interest in the study. In 2014, they co-founded a non-profit, Young 1ove, to turn this promising evidence into programming. Later that year, they launched No Sugar, a youth-targeted awareness campaign built on Dupas’ 2005 research that told girls about the risks of having older sexual partners. Since the original research took place over a decade prior, and in a completely different context, Young 1ove smartly decided to re-evaluate No Sugar before scaling it across Botswana and, eventually, other countries in Southern Africa to better understand the intervention’s external validity across space and time. They brought together like-minded organizations, including the Global Innovation Fund, Evidence Action, J-PAL, and the Botswana-Baylor Children’s Clinical Centre of Excellence, to support the design, evaluation and potential scale-up of the program. More impressively, they secured an incredible mandate from the Government of Botswana: if No Sugar proved effective, the Government would back a nationwide scale-up of the program. As those of us in the field of scaling evidence-based development interventions know, this type of government-led scale-up is often the only way to reach real scale.
With financial support from the Global Innovation Fund and other donors, we partnered with Young 1ove to provide strategic and technical assistance, to support the development of systems, structures, and capabilities that would enable the growth of their organization, and to help position them to scale up No Sugar if the results of the evaluation were positive. Leveraging our experience with evidence-driven program design, scaling proven solutions, and partnering with governments, we worked closely with the Young 1ove team to help them continue to build and maintain a strong relationship with the Government of Botswana and other key partners; provided in-depth technical support and thought partnership on program design and delivery strategy; and weighed in on the evaluation design as led by Young 1ove, J-PAL and Baylor University. Another critical role was to help conceptualize and motivate a path to scale (which would be executed by Young 1ove and perhaps others) if the results of the evaluation were positive.
“We see Evidence Action Beta as a pivotal partner. Their expertise in thinking through systems and structures for scale are unique and took the project a giant step forward. This included late nights ensuring budgets were iron-right, Gantt charts were policy friendly, a Project Review Committee met quarterly, and an MOU was signed. These structures and systems enabled us to nurture a commitment to evidence among all of the partners as well as forge strong and lasting partnerships. As one example: today Young 1ove has a seat on the National AIDS Council, the Global Fund Country Coordinating Mechanism, and are part of a team designing the next National Strategic Framework (2018-2023) for HIV prevention for adolescent and young girls. Our partnership with Evidence Action has fundamentally shaped our organization’s approach to scale and partnership.”
— Noam Angrist, Executive Director and Co-Founder, Young 1ove
Young 1ove led the development of “No-Sugar-in-a-box”: a detailed but simple and standardized curriculum that could easily be delivered through a scaled-up program if the evaluation proved successful. The curriculum included a one-hour class in which a program facilitator 1) revealed the statistical risk of HIV infection for different age groups of men, 2) screened a short video about sugar daddies, and 3) led an interactive discussion about the dangers of sugar daddies. In designing the evaluation, the partners agreed to test two kinds of messengers for No Sugar — teachers and youth volunteers.
This lesson plan was designed to guide teachers and youth who were facilitating No Sugar sessions. Lessons began with a knowledge assessment to collect baseline data on students’ HIV knowledge. After assessment, information on the risk profile of older partners was revealed to students, followed by a video providing a relatable example of intergenerational relationships and featuring a young girl turning down overtures of a sugar daddy. Lessons ended with group discussions and a post-lesson assessment to measure changes in students’ knowledge of HIV risk by age group.
The big question: what if the results weren’t positive?
The partners made a critical ex ante commitment to evidence-based decision-making. Facilitated by Young 1ove, all the partners, including the Government of Botswana and the Global Innovation Fund, agreed on a ‘pre-policy plan’: a breakdown of the potential outcomes of the evaluation and what the policy responses would be for each (something akin to the ‘pre-analysis plan’ often developed by researchers). Jointly, the partners agreed that No Sugar would not be scaled unless the evaluation results showed clear evidence of positive impact, namely a clear and statistically significant reduction in pregnancy rates, a clear indication that girls learned and retained knowledge about the HIV prevalence of different age groups, and a downward shift in the age of girls’ sexual partners. Without evidence of impact on these fronts, No Sugar would not be scaled.
Critically, we had this discussion early—well before the evaluation results were in—guaranteeing that it was sober-minded, reflective of what we believed to be the most appropriate response to ambiguous or negative results, and unclouded by genuine but unfounded enthusiasm for a program that might ‘feel right’ but not be grounded in evidence.
Ambiguous results
After one year, Young 1ove had delivered No Sugar to over 27,030 girls in 343 schools in a randomized controlled trial that collectively included more than 40,000 girls in both the treatment and control groups. Covering over a third of Botswana, the experiment was among the largest in the country’s history.
The evaluation measured several key outcomes, including girls’ knowledge of HIV prevalence by age group, attitudes towards intergenerational sex, condom knowledge, self-reported sexual activity, and pregnancy. Self-reported activity included number of sexual partners and use of safe-sex practices. Pregnancy was measured by tracking school drop-out rates, interviewing girls at the household and community level, and through “tummy-spotting” (checking if girls were visibly pregnant). In August 2016, preliminary results were available, but they did not offer the clarity we had hoped for.
On one hand, most girls could clearly identify high-risk age groups immediately after the No Sugar class. In fact, on discovering that 45 percent of 40-year-old men in Botswana are infected with HIV, girls would sometimes gasp in surprise . Unfortunately, the data also showed that once the initial shock wore off, girls struggled to recall the information. This knowledge decay over time raised a critical question: could sharing information alone generate long-term shifts in girls’ thinking about older partners?
Similarly, even though one measure of pregnancy showed a decrease in pregnancy following the intervention, another measure, tummy spotting (a visual check for pregnancy), showed that pregnancy slightly increased in one treatment group. In addition, the data showed that many of the ‘sugar daddies’ were in their 20s, rather than in their 40s, which the original messaging was geared towards. In other words, there was no clear, unambiguous impact on pregnancy rates—the chosen proxy for unprotected sex and exposure to HIV infection, and there were important unanswered questions about the mechanisms and messaging underlying the intervention.
Young 1ove shared these ambiguous results with their partners, and since all partners had made an earlier commitment to forge ahead to scale only in the event of clear positive results, the consensus was clear: No Sugar, as it had been designed and tested, would not be scaled.
“Information about the HIV risk of older partners changed some measures of sexual behaviour…there is more to learn before scaling a program to youth” – Young 1ove (2016 Annual Report)
A working paper analyzing the full set of results is in progress and is anticipated to be available in late 2018, and we will be sharing some of our lessons gained from participating in the design and evaluation of the program. Meanwhile, Young 1ove has re-designed the relative risk campaign model, and is piloting and testing a new iteration called “Zones” that builds on lessons learned from their experience with No Sugar. We admire Young 1ove’s commitment to building evidence-based programs and look forward to learning from their ongoing journey.
