Summary: A formative evaluation is about improving your program, activity, or intervention. A summative evaluation is about showing proof that it works.
Let’s say you run a non-profit food kitchen. You see your job as feeding the hungry people in your community. You have a pretty basic strategy that involves collecting food from the community and distributing that food through a storefront you rent in a strip mall.
Your local chamber of commerce decides to raise some money to help you. With some of that money you decide to evaluate your work and see how you could improve your strategy.
This is what we would call a formative evaluation.
A little while later you stumbled on some potential grant funding from a local philanthropy. This group wants an evaluation that shows whether or not your program works. That the funds they give you are directly leading to fewer hungry people.
This is what we would call a summative evaluation.
A formative evaluation is about improving your program, activity, or intervention. A summative evaluation is about showing proof that it works.
The soup metaphor and what formative and summative have in common.
When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative
Bob Stake
I like this metaphor because it’s simple and useful. It shows us the difference between formative and summative but it also gives us another important element.
What do both of these examples have in common?
Soup.
I know it sounds kind of silly, but it’s important. The cook is making soup. The customer is eating soup. Soup is the “intervention.”
Both formative and summative evaluations generally require an intervention (i.e. something specific to evaluate). An intervention is a potential solution to a problem.
Back to our food kitchen example. Let’s say you didn’t have a strategy. You want to do something for people in your community who might not have enough food to eat. You don’t quite know what you’re going to do, you just know there is a problem.
There are certainly evaluation methods that could help, but without a potential solution you’re not ready for either a summative or a formative evaluation.
Quick Activity: Formative, Summative, or Something Else?
Let’s end with a quick activity to help you determine next steps. I find that almost everything we do can be broken down into a few basic elements. See if you can break down one of your own activities.
What is your problem, challenge, or opportunity?
In my food pantry example above, the problem could have been a lack of affordable food options in the community. In the soup metaphor, the basic challenge was about delivering a tasty meal to a customer. An opportunity could be that somebody wrote you a big check to help you do something, but they’re letting you decide what that something is.
When we develop programs, activities, and interventions it’s usually in response to something. So what is that something for you?
What is your solution, activity, or intervention?
In order to show that what you’re doing is working or not, you have to be able to identify the thing that you’re doing (your “soup”). If you just have a problem, and no solution, then you’re not ready for a formative or summative evaluation.
And if you’re in a testing mode, trying out a variety of different solutions, you’re probably better off with a different type of evaluation approach.
What are your desired outcomes?
Actions have consequences, positive and negative. An activity, intervention, or solution is a type of action.
A consequence of an action is an outcome. Ideally, this is the positive change we hope to bring about. But remember, we could also make things worse.
Pulling it all together.
Can you fill this out?
We are facing _____ (problem). Our solution is to ________ (action). Ultimately that should lead to _______ (outcome).
A formative evaluation will attempt to show how you can improve the action in order to better lead to the desired outcome. This can help you make the stuff you do better.
A summative evaluation will attempt to show whether or not the action did lead to the desired outcome. This will show that you have a working solution and help you go and get more money to do more of it.
Learners, what questions do you have? Evaluators, what am I missing?
These guides are meant to be conversational and easy to read. But that means I leave out a bunch of potentially useful stuff. If you have any lingering questions, or suggestions as to what should be added to this guide, please leave a comment.