So what is evaluation?
Do you have a favorite definition? Is it the same definition used by your colleagues? How about your client’s definition?
Definitions can be fun topics of discussion, especially when the definitions we hold close don’t match the definitions of those around us.
About the illustrations
So I’m trying a discussion-focused approach to cartooning. I start by choosing an important but common topic of conversation. Then, with expert help, I research the topic to identify critical pieces of the overall discussion.
Once I understand some of the critical elements of the discussion, I cartoon. The cartoons are then posted on freshspectrum. This is not the end of the process. What I hope is that you, yes you, will also provide some input on missing elements.
A few notes:
- If you like the post, write a comment and let me know.
- Share it with colleagues. Seeing people sharing my cartoons inspires me to create more cartoons.
- If you think I’m missing critical pieces to the overall discussion, let me know in the comments.
- Please feel free to use my cartoons in presentations, training materials, etc.
Defining in Code
Sometimes it’s easier to define using metaphor, especially when you’re trying to paint a picture of every part of the field. Roger Miranda’s children’s book Eva the Evaluator gave me the inspiration for this cartoon.
Research and Evaluation
Jane Davidson wrote a nice post on Genuine Evaluation about the difference between evaluation and other pursuits. A few lines stood out and inspired the following cartoon.
The fundamental difference is that evaluation asks and answers questions about the quality, value, and/or importance of things (design, implementation, outputs, outcomes, impacts, the project/program/policy/etc as a whole, and so on).
If we’re not doing that, we’re not actually doing evaluation.
I reached out to colleagues to get their favorite definitions. If you have one, let me know in the comments. This definition was provided to me by David Bernstein.
“Program evaluation is the systematic assessment of program results and, to the extent feasible, systematic assessment of the extent to which the program caused those results. Evaluation includes ongoing monitoring of programs, as well as one shot studies of program processes or program impact. The approaches used are based on social science research methodologies and professional standards.”
You can find the definition here Newcomer, Hatry, and Wholey, 2010, p. xxxiii. It inspired this cartoon.
Merit, Worth, and Value
Stephanie Evergreen was also kind enough to give me a favorite definition. I’m going to go out on a limb and guess that Stephanie is not alone in liking this one by Michael Scriven.
@clysy Scriven: merit, worth, and significance
— Stephanie Evergreen (@evalu8r) April 9, 2013
So what am I missing? Do you have a definition that you love but don’t see reflected in the above illustrations. Let me know, it may just lead to a new cartoon.
Here is a new tweet inspired cartoon…
Hi @clysy love the metaphor one especially!’What is it we said we were going to do in that grant application?’
— Whitebarn Consulting (@SandraVelthuis) April 23, 2013
Here’s a heavily cited evaluation definition by the late Carol Weiss. Thanks for the prompting Sheila and David.
I always liked this definition of evaluation by Carol Weiss: •Evaluation is the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of he program or policy. »Weiss (1998) p. 4
@SheilaBRobinson Thanks for sharing, Shelia
Sheila, I like that one too. Quite possible that the Newcomer, Hatry, and Wholey definition was based on, or at least influenced by, the Weiss definition. Chris, it would be great if you could create a cartoon with that definition in honor of Carol, who passed away recently.
@DJBernstein @SheilaBRobinson Thanks David, cartoon added .
I love the grant evaluation lifecycle one!
I’m glad you like it, I think it may hit close to home for some 🙂
Your cartoons are great! The Independent Evaluation Department of the Asian Development Bank is also exploring the power of cartoons and illustrations in sending development-related messages to a wider audience. May we share some of your cartoons (with full credit and a link back of course) in our facebook page (www.facebook.com/adbevalution)?
@ThereseNgAdb thank you 🙂 All of my cartoons are open for whatever use you have in mind. I would love to stay updated on the results of your exploration. Also, let me know of you ever want to collaborate.
Maria Consuelo says
Your cartoons are great!.. Can I use it in a presentation? How can I recognize your work?
For me the best evaluation definition is fom Michel Patton, because the intention of the evaluation is improve programs and no just judge the programs.
Program evaluation is… “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programing.” – Patton, 2008
Chris Lysy says
Please feel free to use in presentations. I consider the “freshspectrum.com” in each cartoon the attribution.
Tumblr is another well-known blogging platform that is also regarded a micro blogging platform.