Developing A Theory Of Change, Part 1: A Logical Process

This is the first in a two-part series about the Santa Cruz Museum of Art & History’s new theory of change. This week, Ian David Moss and I are each writing blog posts about our collaborative process to develop a theory of change at the Santa Cruz Museum of Art & History. Check out engagement goals before. I’ll write about the handbook soon. This blog post focuses on the theory of change and the process by which we developed it.

A theory of change is a model that connects: the activities we do, to the outcomes they effect, to the impact we seek to create in the world.

We wanted to build a theory of change for two reasons:

1. Externally, we need strong, data-driven arguments for support. We can’t just say,” fund this exhibition and the community will grow stronger.” We have to prove it. Donors want to understand the logic of how their dollars will translate into impact. A strong theory of change can make that case. 
2. Internally, everyone needs to know what “good” looks like and how their work helps contribute to the overall goals of the organization. A clear theory of change helps staff make strategic choices at every level.

We didn’t know how to develop a theory of change. But we knew we wanted to be rigorous about it. So we contracted with Ian David Moss, of Fractured Atlas and Createquity fame, to shepherd us through the process. Besides being brilliant and skilled in this area, Ian came in with an outsider’s perspective, which really helped us get out of the mindset of what we THINK we do and shift into what is actually observable and track-able.

Here’s what we did:

• Ian came to Santa Cruz, interviewed a bunch of staff, and drafted a very rough theory of change based on what he learned about our programming.
• We did a board/staff retreat where we built theories of change in two directions: UP from our activities to our intended impact, and DOWN from the intended impact to the activities that fuel it. The “DOWN” side was the most interesting, because it helped us understand the role we could play in our desired impact—and the community partners we would need to engage and support to see the impact realized to its fullest.
• We worked with Ian to revise the theory based on the retreat.
• Ian did a social psychology literature review to understand the research grounding the connections we made from activities to outcomes to impacts. We identified areas where the connections were weak and where we have to do more research to ensure that the logic is sound.
• We developed a simpulan version of the theory in a wonky powerpoint flowchart model.
• We worked with a fabulous illustrator (Crista Alejandre) to transform the flowchart into an inspiring graphic.
• We started using the theory of change to focus our programs and partnerships, evaluate our work, and change the way we talk about what we do.

Here are some questions for Ian about his side of the experience working with us on this project. Next week, I’ll write about the actual content of the theory of change and how it is starting to impact our organization.

When you are working with an organization on a theory of change, how do you sort out the organizations' aspirations from the reality of their current activities?

Ian: This is always one of the most challenging (and interesting) parts of the engagement. One of the reasons why I find theories of change valuable as a tool for this kind of conversation is that they are really good at making the chain of logic – or lack thereof – between an organization’s activities and goals really clear. That sets up a process where I map out what I perceive the connections to be, and then I run it by the organization to make sure that I’m understanding their thinking correctly. If I spot a place in the logic chain that doesn’t make sense to me, all I need to do is ask some probing questions about it. It could well be that I’ve overlooked something important, in which case I’ll add in whatever’s missing, or it could be that I’ve uncovered something the organization hasn’t thought of, which could spark a much-needed reassessment of what the strategy is or even what the real goal is. (This is exactly what happened to us at Createquity: our theory of change precipitated a global rethinking of our entire content and engagement strategy because we discovered a gap between what we were doing and what our aspirations were.) Either way, the theory of change makes the assumptions embedded in a strategy transparent to everyone and provides a way to put those assumptions to the test.

In our work together, we ended up looking primarily to social psychology research to develop a strong logical basis for the MAH’s theory of change. Do you often find that these projects take you outside of the “arts” field in terms of defining the logic that connects activities to outcomes to impacts?

It depends. I would definitely say that you guys are unusual in how you see non-arts and non-humanities research and practices as not just relevant but central to your work at the museum. But I’d venture to say that it’s a rare arts organization that can’t learn something from how things are done in the wider world, whether that means understanding how and why potential audience members are motivated to make the choices they do, or understanding the policy context for the community-level changes you’re hoping to see, or whatever. I think a very common mistake people make is to draw the frame too narrowly, to say “well, we don’t have any data on this exact thing that we’re looking for, so there’s no point in trying to answer that question.” The reality is that we have many tools to understand and to estimate the way the world works around us, and there are a lot of parallels and inferences to be drawn either from examples in analogous fields or from initiatives that have a general focus that includes the arts but isn’t specific to them.

What do you think is the most challenging part of developing a theory of change?

Different projects present different challenges, but one thing I’ve found to be consistent is that the theory of change process can end up drawing out major differences in thinking styles. There’s a certain type of person who’s really comfortable breaking ideas down into orderly, modular components and analyzing the connections between them. Then there are other folks who are not at all accustomed to thinking that way – they’re much more at home in an open-ended, anything-goes brainstorming session that encourages divergent thinking and untethered creativity. For those people, the process of creating a theory of change or logic model can very easily feel confining if you don’t set it up carefully. What I’ve found is that things go better if I make sure that nobody has unrealistic expectations placed upon them. A lot of people find it easier to have a conversation and then react to a model presented to them than be tasked with having to figure everything out themselves. On the other hand, other folks want to be super involved and that’s great too.

Any words of wisdom about how to build buy-in and encourage use once a theory of change is developed?

A really good way to do this is to include it in pelatihan materials for both current and new staff. The more that the theory of change gets talked about, the more likely it is to be used. You can also use it as a reference point for other institutional capacity-building things your organization is doing. So the MAH used it as the basis for a measurement framework for the organization. At Fractured Atlas, it was a key input for a new merk book we developed to guide our internal and external communications. It can be an attachment to grant applications or included in annual reports to donors. And it’s important that the theory of change be periodically revisited to make sure that it doesn’t reflect stale thinking. That all being said, I would emphasize that going into the process with the intention for a theory of change to be useful is the number one predictor of whether it will actually be useful. Furthermore, the best way to build buy-in for a theory of change is by giving people a voice in creating it. That’s why as much as possible I try to involve front-line staff as well as leadership in the process, so that it will feel resonant at all levels of the organization.

Thanks to Ian for collaborating on this process with us. If you are reading this via email and would like to share a comment or question, you can join the conversation here.