How can we get the most impact for our collective buck?
This is the big question facing the state, and no more so than during an election. We’re seeing this with the upcoming British elections on May 6: What should society look like? Who is responsible for achieving those outcomes? How? Who should carry the risk when those outcomes are not met? The state has a surprisingly limited track record of holding itself or its service providers accountable for outcomes. It has been increasingly good at keeping track of the numbers (the inputs and outputs), but not at what those numbers add up to (the outcomes): do people feel fulfilled, useful, valued, connected, and resilient? There are lots of reasons for overlooking outcomes: the methodologies are underdeveloped, the baseline evidence base is sparse, the political risks are high, and our knowledge of how to change and maintain outcomes is limited. Three recent publications add to our methodological know-how:
-The 2020 Public Services Trust’s Better Outcomes
-Paul Brest’s The Power of Theories of Change in the Stanford Social Innovation Review (Spring 2010)
-The New Economic Foundation’s Public Services Inside Out
All three publications work backwards from outcomes, presenting logic models as a tool for figuring out which programs, services, and initiatives to fund. Programs, services, and initiatives are conceptualized as a means to a bigger end; they matter insomuch as they enable people to live the kinds of lives they want in the kinds of societies they respect. The authors of all three pieces argue funders and commissioners should hold programs, services, and initiatives accountable for proving (or, just as usefully, disproving) their hypothesized intervention logics: their hunches for why investing in ‘x’ activity will have a measurable impact on people’s lives. Brest makes a case for funding different, even competing intervention logics in the early stages of solving a complex social issue. Since part of what makes a problem complex is the lack of a single ‘best’ solution, taking what Brest calls a developmental approach to funding can prompt systematic experimentation across a range of problems and contexts.
Of course not all logic models are equal. In our work, we’ve come up with some pretty bad logic models. A bad logic model isn’t a wrong logic model. Most of our logic models will initially be wrong, or at least incomplete, that’s just the nature of the challenges we are all working on. A bad logic model is one that fails to link the ‘what’ with the ‘how’: to explain the mechanisms by which a set of activities might influence outcomes in a particular context (i.e. how will a social marketing campaign actually change what young people in a suburban community eat?)
nef’s Public Services Inside Out is the only one of the three publications to focus on the ‘how.’ While they offer some really powerful and conceptually cogent principles for how change happens, these principles are generic, divorced from particular social problems or social contexts. They talk about good solutions as co-produced; meaning ones which build on what people can do, spark mutuality and reciprocity, enable peer support networks, blur distinctions between professionals and people, and facilitate rather than deliver. But, what these principles don’t tell us is what needs to happen within the peer network to tackle certain kinds of problems: how does the peer network need to function to help people overcome addictions versus become more energy efficient?
All three publications miss this layer of critical questioning. The templates they present for logic models leave out behaviors: the things people have to do or not do to reach their desired outcomes. Programs, services, and initiatives should work to address the factors (aka determinants) which influence those behaviors. Our first prototypes of Loops—the platform for youth & community development we worked on whilst at Participle—left out the behavioral level, and consequently, weren’t very effective. Here’s the difference:
Our first logic model—without behaviors
A later logic model—with behaviors
Without the behavioral level, our inputs didn’t do the right things; they didn’t train people to build the right kinds of capabilities in young people. Yet the outputs themselves didn’t change between logic models—developmental experiences and reflection were the major activities of both versions—it was how these activities were performed that mattered. It was the quality of the activities. This is the biggest challenge we face with our public sector work—not moving towards outcomes—but moving towards interventions that have the qualities necessary for actually changing outcomes. The words may be the same (relationships, networks, etc.) but the substance can and should differ by problem and place.
In our experience, we don’t need to just help funders and commissioners use logic models to more systematically resource programs and services, but to understand what about the programs and services will affect the outcomes in question. In this way, the logic model isn’t just a tool for validating a problem-solving approach but for building effective problem-solving approaches.
Tags
PLEASE SHARE You're free to take what's ours, use it, improve it and make derivative works under a Creative Commons Attribution 3.0 License© Inwithfor 2009-2011 Worldwide
SAY HELLO We love to get your comments and hear about new things. Send us a mail.
Tweets