What is evaluation? 5 illustrations

So what is evaluation?

Do you have a favorite definition? Is it the same definition used by your colleagues? How about your client’s definition?

Definitions can be fun topics of discussion, especially when the definitions we hold close don’t match the definitions of those around us.

 

About the illustrations

So I’m trying a discussion-focused approach to cartooning. I start by choosing an important but common topic of conversation. Then, with expert help, I research the topic to identify critical pieces of the overall discussion.

Once I understand some of the critical elements of the discussion, I cartoon. The cartoons are then posted on freshspectrum. This is not the end of the process. What I hope is that you, yes you, will also provide some input on missing elements.

A few notes:

  • If you like the post, write a comment and let me know.
  • Share it with colleagues. Seeing people sharing my cartoons inspires me to create more cartoons.
  • If you think I’m missing critical pieces to the overall discussion, let me know in the comments.
  • Please feel free to use my cartoons in presentations, training materials, etc.

 

Defining in Code

Sometimes it’s easier to define using metaphor, especially when you’re trying to paint a picture of every part of the field. Roger Miranda’s children’s book Eva the Evaluator gave me the inspiration for this cartoon.

The evaluation talk

Metaphorically speaking

I’ve seen a lot of metaphoric definitions.  What’s your favorite?  Here’s a simple sports-based one.
Sport with no rules


Research and Evaluation

Jane Davidson wrote a nice post on Genuine Evaluation about the difference between evaluation and other pursuits.  A few lines stood out and inspired the following cartoon.

The fundamental difference is that evaluation asks and answers questions about the quality, value, and/or importance of things (design, implementation, outputs, outcomes, impacts, the project/program/policy/etc as a whole, and so on).

If we’re not doing that, we’re not actually doing evaluation.

Researcher and Evaluator

 

Systematic Assessment

I reached out to colleagues to get their favorite definitions.  If you have one, let me know in the comments.  This definition was provided to me by David Bernstein.

“Program evaluation is the systematic assessment of program results and, to the extent feasible, systematic assessment of the extent to which the program caused those results. Evaluation includes ongoing monitoring of programs, as well as one shot studies of program processes or program impact. The approaches used are based on social science research methodologies and professional standards.”

You can find the definition here Newcomer, Hatry, and Wholey, 2010, p. xxxiii.  It inspired this cartoon.

Systematic Assessment

 

Merit, Worth, and Value

Stephanie Evergreen was also kind enough to give me a favorite definition.  I’m going to go out on a limb and guess that Stephanie is not alone in liking this one by Michael Scriven.

 

Scriven Merit, Worth, Value

 

Additional Cartoons

So what am I missing?  Do you have a definition that you love but don’t see reflected in the above illustrations.  Let me know, it may just lead to a new cartoon.

 

Update 1

Here is a new tweet inspired cartoon…

Grant Evaluation Lifecycle

Update 2

Here’s a heavily cited evaluation definition by the late Carol Weiss.  Thanks for the prompting Sheila and David.

11 Comments

  1. SheilaBRobinson on April 23, 2013 at 9:22 pm

    I always liked this definition of evaluation by Carol Weiss: •Evaluation is the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of he program or policy. »Weiss (1998) p. 4



    • cplysy on April 24, 2013 at 8:38 am

      @SheilaBRobinson Thanks for sharing, Shelia



    • DJBernstein on May 2, 2013 at 11:55 am

      @SheilaBRobinson
       Sheila, I like that one too. Quite possible that the Newcomer, Hatry, and Wholey definition was based on, or at least influenced by, the Weiss definition. Chris, it would be great if you could create a cartoon with that definition in honor of Carol, who passed away recently.



      • cplysy on May 2, 2013 at 12:53 pm

        @DJBernstein  @SheilaBRobinson Thanks David, cartoon added .



  2. jensulewski on April 25, 2013 at 10:04 am

    I love the grant evaluation lifecycle one!



    • cplysy on April 25, 2013 at 11:42 am

      I’m glad you like it, I think it may hit close to home for some 🙂



  3. ThereseNgAdb on May 6, 2013 at 9:59 pm

    Your cartoons are great! The Independent Evaluation Department of the Asian Development Bank is also exploring the power of cartoons and illustrations in sending development-related messages to a wider audience. May we share some of your cartoons (with full credit and a link back of course) in our facebook page (www.facebook.com/adbevalution)?



    • cplysy on May 7, 2013 at 5:48 am

      @ThereseNgAdb thank you 🙂 All of my cartoons are open for whatever use you have in mind. I would love to stay updated on the results of your exploration. Also, let me know of you ever want to collaborate.



  4. Maria Consuelo on August 1, 2013 at 11:50 am

    Your cartoons are great!.. Can I use it in a presentation? How can I recognize your work?
    For me the best evaluation definition is fom Michel Patton, because the intention of the evaluation is improve programs and no just judge the programs.

    Program evaluation is… “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programing.” – Patton, 2008



    • Chris Lysy on August 1, 2013 at 11:58 am

      Thanks 🙂
      Please feel free to use in presentations. I consider the “freshspectrum.com” in each cartoon the attribution.



  5. www.tumblr.com/ on May 31, 2015 at 4:22 am

    Tumblr is another well-known blogging platform that is also regarded a micro blogging platform.