States and Counties Pioneer Model of Human Services Innovation Using Data, Evidence and Iteration

From Linn County, Oregon, to Philadelphia, Pennsylvania, a growing number of local governments across the country are turning to data and evidence to improve programs serving residents in poverty. In doing so, they’re rewriting the playbook on how research directly informs policy and practice by developing a model in which the scientific literature informs changes that are then tested and refined in the field; the improvement process is dynamic and continuous, and residents ultimately benefit by receiving better services. We know because we’re witnessing it firsthand.

Mathematica is working with human services and workforce agencies in Colorado, Massachusetts, Michigan, Ohio, Vermont and Pennsylvania, among others, to introduce a change management framework called LI2. The name refers to a three-stage process called Learn, Innovate, Improve. (Yes, we know, leave it to an organization named Mathematica to use exponents in our branded nicknames.) In the Learn stage of LI2, agency staff examine the problem and assess the program’s readiness for change. In the Innovate stage, we partner with agency staff to design promising interventions, drawing upon the best available science, existing evidence and practice wisdom. And in the Improve stage, we work with the agencies to use small-scale analytic piloting and other research methods to test the evidence-informed idea.

LI2 is deliberately designed to be collaborative, actively engaging staff who run the programs in all aspects of the process, from idea to impact. It starts with small improvements. In public policy, it’s tempting to dream big. But with LI2, we focus on the adjacent possible. In other words, in the context of where you’re at, what’s just beyond your current reality? What can you do? What’s feasible? We’re looking for realistic changes that can be achieved in the near term. Everyone in this field would like to end poverty, but how do we get from here to there? We know from experience that the journey to better outcomes involves taking many concrete steps to improve program quality, efficiency and effectiveness.

In Ramsey County, Minnesota, for example, Mathematica has partnered with the county’s Workforce Solutions department and used LI2 to co-develop a science-informed coaching model for delivering employment services, known as the Lifelong Learning Initiative (LLI). Ramsey County was one of the first communities to take the science of executive functioning and goal achievement and seek to integrate it into a service delivery model in a public agency for serving families with low-income. During the Learn phase (2015–2016), we worked with staff at all levels of the department to pinpoint the specific motivations for changing service delivery and carefully examined the program environment to understand how best to integrate the new model into existing structures and culture. During the Innovate phase (2016), we leveraged the experience of nationally recognized experts to design and iteratively refine prototypes of an executive function-informed coaching model. Between 2016 and 2018, we carried out the Improve phase by road testing a variety of different tactics and tools, all anchored in the best available evidence, in service of smooth and well-integrated implementation.

Having learned from recent efforts in the Improve phase, Ramsey County continues through the LI2 cycle, with a focus on replicating and scaling the LLI across other teams and programs in the department. Most of the evidence generated to date through this partnership has been formative—that is, Ramsey County has used these insights in real time to adapt the implementation, refinement and scaling of the LLI. But we won’t be stopping there. LI2 explicitly focuses on building the capacity of local staff to be better users and producers of high-quality research. Ramsey County’s internal evaluator, in partnership with Mathematica, will use research methods over the next few years to answer questions about how and whether the LLI is having its desired impact and whether racial disparities in outcomes are improving. These evidence-building efforts will continue to shape continuous quality improvement for the department.

Since beginning the LI2 work, we’ve been inspired by the ingenuity and determination of our partners in county government and have seen how LI2 can lead to a newfound capacity for improving outcomes for children and families. As more agencies adopt an inquisitive and experimental mindset to improve their work, our hope is that LI2 offers a refreshingly accessible approach to more analytically and systematically pursuing change.