A megastudy—a massive field experiment—targeting more than 60,000 members of an American fitness chain showed that some four-week digital programs encouraging exercise boosted gym visits by 9–27%. The study, published this week in Nature, demonstrates how the megastudy experimental design rigorously tested 54 different behavioral interventions in a large population to determine their efficacy.
Policymakers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. For behavioural interventions to be comprehensively assessed, they must be both tested in the field and comparable to other interventions. Typically, individual interventions are tested in independent groups, but this approach makes it difficult to compare results in a like-for-like way. It can be hard to determine whether diverging results from these studies are due to the difference in the population studied or the differential efficacy of the interventions.
To address these challenges, Katherine Milkman, Angela Duckworth and colleagues introduce the concept of a megastudy, in which different interventions are tested in the same population. A total of 30 scientists from 15 different universities in the US worked in small, independent teams to design 54 different four-week digital programs to encourage exercise among 61,293 members of an American fitness chain. The authors found that 45% of these interventions—which consisted of digital experiences, text reminders, weekly emails and reward incentives—significantly increased weekly gym visits (by 9–27%) during the four-week plan. The top-performing intervention offered small cash rewards for returning to the gym after a missed workout. Only 8% of interventions created behaviour change that was measurable after the four-week intervention.
This megastudy model allows the comparison of dozens of different behaviour change interventions, each designed by an independent scientific team. The authors conclude that the model can accelerate the development and testing of new insights about human behaviour and ensure the relevance of these insights for public policy.