A Principled Approach to Impact Measurement and Evaluation – Social Innovation Canada

The Adopting Common Measures program is highlighting key social purpose organizations across Canada and showcasing their impact and the progress they are making towards a more sustainable future for Canadians in keeping with Canada’s 2030 Agenda for Sustainable Development.

“There is no one way to do social impact evaluation,” says Carrie Tanasichuk, Director, Impact Measurement and Evaluation at the Greater Saint John Community Foundation. She adds, “that’s why we focus on coaching staff teams of community organizations, sharing tools and resources so they develop the skills needed to think in an evaluative way.”

Carrie Tanasichuk, Director, Impact Measurement and Evaluation at the Greater Saint John Community Foundation

Thinking creatively about how to better serve their community

Back in 2019, the Greater Saint John Community Foundation was known primarily as a funder. However, Kelly Evans, President & CEO, knew that local charities had other needs as well. She was sure they could provide additional services that would be of benefit to local organizations, the people those groups served, and ultimately, the community as a whole.

Through a community connection, she and Carrie met and began talking about the increasing need for charities to be able to determine, measure and report on the social impact of their programs and services.

With specialized expertise in evaluation and a Ph.D. in Applied Social Psychology, Carrie brought a unique perspective to this conversation. Together, Kelly and Carrie sketched out an idea for an impact measurement and evaluation service and presented it to the Community Foundation’s Board of Directors. The Board was supportive of the concept however without available funds to allocate to the project, they stipulated it would have to operate on a cost-recovery basis.

That was all the encouragement Kelly and Carrie needed: in March 2020, the Impact Measurement and Evaluation branch of the Greater Saint John Community Foundation was created. Three years later, the program has been welcomed by the community and is meeting both its programmatic and cost-recovery goals.

Developing the approach

Carrie quickly got to work and was fortunate that her colleague, Harry Daley, Director of Community Investment & Learning, was able to commit some time to the initiative.

“One week after we launched, the COVID pandemic arrived and our plans to quickly start working in-person with local charities had to be revised,” says Carrie. “Instead, Harry and I spent time sharing our thoughts and ideas about what made for efficient and effective impact measurement evaluation factors.”

They then reviewed all of their notes and started grouping items into like ideas. They were excited to discover that what emerged was, in effect, a set of principles for the approach they wanted to take. “We couldn’t believe it but it was right there on the paper in front of us,” she says, with a laugh.

These principles form the foundation of the approach with respect to coaching, teaching, or otherwise working with people on the topic of impact evaluation.

Six principles guide their approach

Every element of the work they’ve taken on since that time has been guided by the framework’s six principles.:

“I think that’s why our process has been so well-received by the community,” shares Carrie. “We have a solid, foundational grounding about our intentions, what we want to do, and how we want to do it.”

“We are helping to fill a gap, where groups needed to track and report impact but didn’t have the skillset or capacity to do it. Our approach is a coaching model rather than a consultant model because we want to help charities develop the skills and overall capacity they need to take on this process themselves, now and into the future,” explains Carrie.

Coaching to develop increased capacity

When working with community groups, Carrie is careful to explain she is not providing a step-by-step process that they can use every single time in exactly the same way. Since every evaluation process begins by trying to answer a different question (or set of them), a one-size-fits-all approach doesn’t work.  

“From a coaching point of view, I introduce evaluation as a concept, we discuss why it is important, how to come up with effective questions and determine good indicators,” says Carrie. “My goal is to fill up their evaluation-skills toolbox so they can grab whatever tool they need at a given time, know how to collect the information they need, and how to responsibly use the data they gather.”

Collaboration in action: Working with five priority neighbourhoods

As part of this work, Carrie and her colleagues created an impactful community initiative where they worked collaboratively with community-serving organizations in Saint John’s five priority neighborhoods. 

The initiative’s report describes the partnership being established to “offer tools, coaching, mentoring,and direct support that would enable them [community organizations] to measure effectiveness, clearly demonstrate the difference they are making, and support on-going learning so that their work can be sustained and evolve further.”

The year-long project was funded by the Business Community Anti-Poverty Initiative with a $100,000 investment that supported both the Impact Measurement and Evaluation branch, as well as provided funding for each of the groups in the five priority neighbourhoods so they would have the ability to participate from a funding perspective.

A phased approach

The report details that each of the community groups involved are experts in resident engagement and deliver a wide variety of programs and services in response to the complex needs of their communities. “While they have become experts in front-line service delivery, they identified a gap in their capacity to articulate their approaches and to measure the impact that they create.” (p.4)

The six established principles of impact measurement and evaluation guided the project’s approach throughout its three phases: (1) discovery, (2) evaluate and (3) learn. 

The phases included work such as:

The initiative was a success, with participants sharing feedback such as “”We have always made a difference. Now we are able to articulate it.”

The report’s conclusion highlights that “the Discovery – Evaluate – Learn process was designed and tailored for the neighbourhood organizations.This process led to an increase in their ability to articulate their work to a broad audience, independently evaluate the impact of their approaches on residents and the community, and apply their learnings to create meaningful changes in they way they work as individual organizations and as a collective.” (p. 20)

Next steps for the Impact Measurement and Evaluation branch 

Although their “six principles” approach was independently developed, Carrie has been wondering how it aligns to other impact measurement approaches, for example the Common Approach, the UN’s Sustainable Development Goals and the United Nations Declaration on the Rights of Indigenous Peoples. It’s something she’s looking at reviewing and mapping out at some point in the future, when time allows.

Carrie, Harry and Kelly are motivated to continue playing a role in building the impact measurement and evaluation capacity of the community organizations they serve in Saint John.

Carrie says, “we are continuously adapting what we do based on the needs of the groups in the community. We’re focused on being responsive, helping to address both current and emerging needs.”