Evaluation is a systematic process to determine merit, worth, value or significance…
But then what?
Let’s say you followed a systematic process and came to some conclusions about the value of a program. Your evidence shows that this particular program *doesn’t work so well.
What comes next? It’s pretty unlikely that your work is completed once you come to your determination. Once you know.
Do you just write it up in a complicated precise report that few will ever read and call it a day?
Perhaps.
I think for some evaluators, that’s the job. To make a determination. For these evaluators, deciding what to do with that determination is for somebody else.
But if that’s not you, do you spend additional time breaking down your findings so that the evidence and rationale can be explained to your primary audience of likely decision makers?
Perhaps.
It is usually the likely decision makers who pay your bills. They are your boss or your client. As long as they understand your determination, and are happy with your work, maybe that’s when you call the job done.
But let’s say you don’t stop there, do you spend time adapting your evidence and rationale to meet the specific needs of other audience members? The people who are directly or indirectly connected to the program, the funding stream, or the outcomes.
Perhaps.
As long as we include everyone as an audience for our reporting, and create a string of reporting products to meet a variety of needs. And these reports will be shared by the organization’s communications department. That should be enough, right?
But maybe you want to know if it actually reached all these different audiences, or at least some of the major ones. You also want to know if the audiences actually understand the evaluation findings and their implications. Do you then evaluate your sharing efforts and adapt your reporting strategy based on the findings?
Perhaps.
At this point, you’ve put in a lot of effort to see that your findings reach all your different audiences. Could anyone ask for anything more?
But what if the findings are contrary to popular belief? Your determination means a program that people believe in, one that looks good on paper, doesn’t actually lead to desired positive outcomes. It does not work as it is designed to work.
Do you put in the effort to build a level trust with all of these different audiences, so that they believe you when you share your results? Do you put in the work to convince these different audience members that your determination is right, even if it’s hard to believe at first.
Perhaps.
At some point in this blog post did you start to feel overwhelmed?
Did you say to yourself, “okay, but we just don’t have the budget or time to do all of this…”?
When I see evaluators describing evaluation to others, they often focus on what it means to do an evaluation. The process they follow, the methods they use, and the people they involve. The “then what” piece is always a little bit on the fuzzy side.
So my question for you, when does it end?
*This is all pretend, of course, because rarely does any program you are called to evaluate show little to no merit. Nor does any program ever seem like it works flawlessly. In other words, most determinations are “complicated.”
Mary Davis says
Hi Chris,
Interesting blog post. The refresh of the CDC Framework for Evaluation intentionally includes action steps as the final step rather than sharing results. You’re right some funders want that while others don’t. I believe Patton recommends working with interest holders for 3 months after an evaluation is complete. I try to look for opportunities where there’s action at the end, including funding, so results don’t just sit on a shelf (or an email).
Cheers, Mary
Celestyna Galicki says
If the evaluation shows a need for change then the next step is to prepare a plan/business case outlining what is to be done, by whom, when, how much it will cost, risks, interdependencies etc. often with a range of options, and implement that plan. As an in-house evaluator, I have been increasingly involved in that next phase. It can be more challenging than the evaluation and requires a different skillset which is probably why it is not common to see evaluators offering the whole evaluation-to-implementation process.