Randomized Controlled Trials, 5 illustration collaboration with Jen Hamilton

You know what’s great about collaborating with experts? After each post I know a little bit more about the evaluation world.

How about you? Have any of my cartoons helped you learn something new? I hope so.

About the Illustrations

This week’s collaboration is with friend and colleague Jen Hamilton. Jen’s leading a three day workshop on RCTs in Education.  Sadly, I didn’t get this post turned around in time enough for you to register.

These cartoons were created based on an exchange I had with Jen prior to her workshop.  The quotes below come from our conversation and sparked each of the individual cartoons.

If you’re interested in the overall RCT debate I stumbled across this nice set of videos from a 2006 Claremont event What Constitutes Credible Evidence in Evaluation and Applied Research?  (Found via Gene Shackman’s resource site)

Looking for more of a straightforward definition, here’s what the Institute of Education Sciences has to offer.

A few notes:

  • If you like the post, write a comment and let me know.
  • Share it with colleagues. Seeing people sharing my cartoons inspires me to create more cartoons.
  • Please feel free to use my cartoons in presentations, training materials, etc.

RCT Pitfalls

Getting active parental consent is hard. It often sounds like we are using kids as lab rats. Or the treatment changes halfway through the evaluation. Those are just a few of the headaches….

Treatment group was 50% less likely to picket


Even with kids randomly assigned into groups, if your attrition (esp. differential attrition) is high, you just lost the benefits of random assignment. Buh-bye. The sample has changed too much – now you have dropped down to a Quasi Experimental Design from the “gold standard” RCT.

Opted for a QED


The thing is, the curriculum developers always want the TOT but it’s the ITT that has the rigor.

TOT is fantastic

Wondering what the heck TOT and ITT means, here is a resource… Understanding Intent-to-Treat and Treatment-on-the-Treated Estimates

RCT appropriateness

Can’t assign pregnant women to treatment and control groups, when one group is drinking and smoking, while other is not. Or assign unemployed people to durations of unemployment.

In the control group

RCT ready

Also, some interventions are just not ready for RCT-land. They are still in the process of being fleshed out and developed.

Heard RCT was the gold standard


What else?

What other points about RCTs should I cartoon?


  1. Eduardo on October 15, 2013 at 1:41 pm

    How to defined the treatment group and the control group for an demand driven intervention?

  2. Karen Ball on February 18, 2014 at 1:05 pm

    These are great cartoons to use to make Evidence Based Practice lecture more interesting and reach another domain of learning. Humor.
    thank you

    • Chris Lysy on February 18, 2014 at 1:09 pm

      Thanks Karen 🙂

  3. Jacklyn on September 11, 2014 at 10:06 am

    my blog post … website (Jacklyn)