Today’s cartoons were inspired by a silent collaborator who sent me four ethics related stories. Since the stories are based in reality, the evaluator asked for their name to be left off to protect the identities of those involved.
So when you read the stories picture a silhouette with a garbled voice like you might see in the news.
Each of the stories is paired with the cartoon it inspired.
Low Response Rate
Evaluator: “Here are the survey results for your program. The results suggest that _____, but here’s a caveat – there was a low response rate, so we need to take these results with a grain of salt.”
Client: “No problem. Send me the link to the survey. I’ll take the survey a dozen times, and then the response rate will be higher.”
Evaluator: “Sorry, that’s not how it works.”
Client: “Then give me a paper copy of the survey and I’ll make photocopies until we have enough responses.”
Evaluator: “Here are the results from your program.”
Client: “Uh oh, the results don’t look good. The graphs aren’t going up. Can you re-format the graph to make sure all the bars are going upwards over time?”
Evaluators: “Here are the results from your program.”
Client: “Those results aren’t accurate.”
Evaluators: “How so?”
Client: “The bad results are obviously outliers. You need to remove those people from the sample.”
Evaluators: “We define an outlier as (1.5 x the interquartile range) below quartile 1 or above quartile 3. Other evaluators define outliers as 3 standard deviations above or below the mean. We checked, and those people are not outliers. In fact, their experience in the program was pretty typical.”
Client: “Sorry, I didn’t realize it was such a hassle to fix. I didn’t mean to create more work for you. Just send me the Word version of your report and I’ll delete that section myself.”
Evaluator: “Here are the results.”
Client: “I know our program only had a 38% success rate, but can you type 83% in the report to our funders? 83% sounds better than 38%. If anybody notices, just say you accidentally made a typo.”
I’m curious, what ethics fails have you witnessed? Anything you can share or would you need the blurry picture and garbled voice treatment too?
Update 1: Cherry-Picked Sample
Thanks to Maria Gajewski for the comment that inspired this cartoon!
My favorite is the cherry-picked sample. I was involved in a project where directors only wanted to survey students whose families were still involved in the program. It’s not too difficult to figure out families who had a bad experience would not remain with the program, but the directors just couldn’t seem to grasp this when I pointed it out!
Next time, send this cartoon to the directors
Update 2: Consequences
Ok, here is another comment inspired cartoon (thanks bridgetjones52). And a big thank you to everyone else for all the comments and shares!
When presenting results of evaluation, very very senior manager ‘informed’ us that there would be consequences!!!