Does it work?
It’s an easy question to answer when talking about the blender you picked up at a neighbor’s garage sale.
The question’s just a bit tougher to answer when asked by a funding agency trying to save the world through a comprehensive intervention.
“Does it Work?” is the kind of question that fuels the field of evaluation. Everyone involved in an intervention wants to know that they accomplished something and who wants to fund something that doesn’t work?
So how do you approach the discussion?
Below you will find 5 cartoons I’ve designed that speak to the issue of attribution. These are discussion tools, not answers.
About the illustrations
This post was inspired by Kylie Hutchinson, who gave me the idea and provided me with resources I could use to dive into the issue. The primary resource I used for this post was [PDF] Addressing the Question of Attribution in Evaluation by the Evaluation Unit of Canada’s The International Development Research Centre.
A few notes:
- If you like the post, write a comment and let me know.
- Share it with colleagues. Seeing people sharing my cartoons inspires me to create more cartoons.
- If you think I’m missing critical pieces to the overall discussion, let me know in the comments.
- Please feel free to use my cartoons in presentations, training materials, etc.
Separating from the crowd
One reason attribution is so hard is that there are often so many additional factors at play.
Impact at the micro level
Did my contribution accomplish anything?
Modeling a comprehensive intervention
The more comprehensive the intervention, the harder it is to gauge its contribution. It’s also much harder to explain and model.
Simple and complex
I tried to get at the difference between simple systems and complex systems. I’m ok with this cartoon but it still doesn’t feel quite right. What analogies do you use? Let me know in the comments.
Doing the best with uncertainty
What’s missing from this post? Let me know in the comments and I might just add more cartoons.
Susan Kistler was kind enough to offer some feedback on the simple/complex cartoon. Here is a new version based on her comments.
According to Matt Keene two trees and three whales will be saved at the 2013 EEN Pacific Forum. Since Matt didn’t specify in his post which whales, I thought I would help out. Print it out, circle your choice, then deliver to Matt.
— Matt Keene (@mwkeene) April 4, 2013