Additional menu

FreshSpectrum

Chris Lysy's Evaluation Blog

Articles

My latest book, now Free-to-Read

Leave a Comment

Want to read my book without paying for it?

Now you can do that. There is a title page, introduction, and five chapters. Each is on its own page with navigation to help you go through. It’s digital, not a pdf, so that means it is auto-translatable and mobile-responsive.

To access just follow this link: https://freshspectrum.com/thereportingrevolution/

I don’t even force you to put in your email this time, although if you don’t already get my newsletter I do suggest you join us 🙂

Why free-to-read?

Why not?

I would rather people read it for free than not read it. And anytime you put a price tag on something and get Amazon involved, the number of people with access goes way down.

If you want the $10 paper version or $5 kindle version, both are still available. But if you just want to casually stroll through the chapters on your laptop, tablet, or smart phone, go for it.

I also included all the cartoons in full-color, ready to be downloaded and added to your next presentation.

Happy reading, let me know what you think.

My General Theory of Dissemination

Looking pretty is good and feeling cool is fun, but being accessible is better.

When it comes to sharing our work, we create barriers. Most of the time it’s not on purpose.

1. Format barriers.

Such as using PDFs with a smart phone audience.

2. Physical access barriers.

Only presenting at conferences where live attendance/fees are required.

3. Network barriers.

Only sharing with people inside your existing network.

4. Platform barriers.

Work is not tailored to meet the technical or cultural requirements of a platform.

5. Structural barriers.

Relying on web tools that look cool but ultimately limit reach for technical reasons.

6. Language barriers.

Presenting in your own language in a format that’s not easily auto-translatable.

7. Jargon barriers.

Using a lot of insider language.

8. Big word barriers.

Writing at a postgrad level when your stated audience is not just PhDs.

9. Big block of text barriers.

Writing in long paragraphs and failing to illustrate.

10. Frankenstein’s audience barriers.

Trying to serve everybody, and ultimately not serving anybody.

11. Graphic design barriers.

Lacking visual structure and white space.

12. Skim-ability barriers.

Not using descriptive headers or considering skim readers.

13. Visual barriers.

Contrast issues, color theory, and not considering visually impaired.

14. Screen reader barriers.

Not structuring your work properly or using alt text.

15. Legal excuse barriers.

Treating accessibility like a checklist or using tools that obscure information to limit lawsuits (not actually increase access)

16. Cultural barriers.

The team behind your work lacks diversity or input from different subsets of your audience.

If your work is important, and useful, our number one dissemination goal as writers and designers should be reducing the barriers. You can lead a horse to water, but if there is a fence between you and the river, that horse will never get a drink.

Watching and waiting.

I’ve been a bit pessimistic lately. Or perhaps I’ve just been a bit a realistic, which can be far more depressing.

In the past several weeks I’ve written and deleted a few different LinkedIn posts. I know that many of you appreciate the honesty in some of my comics, even the ones that feel a bit dark. But there is a point where it’s just a bummer.

I’ll try to do better, just maybe not today.

I didn’t attend this year’s American Evaluation Association conference. I haven’t since before COVID, but I do miss it. The other day I was looking at the conference schedule online and comparing it to the last couple of years. I pulled the full schedule for 2023, 2024, and 2025 then stuck into AI to compare.

  • Unsurprisingly, the number of conference sessions are down…by a lot (a little over half the sessions). I’ll let those of you who were there comment on the actual attendance and feel.
  • The USAID & international funding cuts are definitely reflected. This also seems to mean less of the sessions that talk more about big experimental evaluation approaches.
  • There are lot fewer sessions on storytelling. But if you look at past years you’ll start to notice that a lot of the past storytelling sessions were also talking about equity and inclusion initiatives. You know, all the DEI stuff that this US administration hates.
  • With all the government cuts and so many layoffs at big social science research organizations, the government program related sessions also seemed down a bit.
  • One thing that has certainly grown, is the number of sessions about AI. Can’t say that’s surprising.

Note: I got all of this is just by looking at the schedules. It might not be the reality on the conference floor. I’m sure there are some quirks that might make this method not wholly reliable.

I wonder how much the field of evaluation is really being hit right now. A lot of the people who present at conferences and contribute to journals, work in universities, government agencies, and big social research firms. All of these have been hit pretty bad by funding cuts.

But I also know that there are a lot of people who do evaluation work who are not part of the evaluation association. Many have probably never heard of the association. State & local governments, health departments, schools, non-profits, community foundations, and business social responsibility departments all employ evaluators. I wonder how it is in all of those worlds.

The Report Design Multitask Problem

One of the biggest report design frustrations is something that’s not on most people’s radar.

I talk about this in my book The Reporting Revolution (it‘s only $5, you should buy it!), but I’ve also talked about the concept in workshops. Here is a clip from my Everyday Report Design workshop.

Most evaluators and researchers try to write, design, and illustrate their reports all at the same time – usually the night before it’s due. This multitasking approach is why your reports feel like a struggle and often don’t look as good as you’d hoped.

There’s a better way: Split the work into three parallel tracks – writing, design, and illustration.

In this video, I’ll show you how to:

  • Separate your writing from your design work
  • Use placeholder text (lorem ipsum) to design before you write
  • Get approval on layouts early, before investing time in content
  • Reduce the late-night stress of making reports “look pretty”

This simple process shift will save you time and create better-looking reports.

Prefer the transcript?

Transcript

The Cassandra Curse

I kick off my latest book with the story about Cassandra.

I feel like being an evaluator or researcher or data analyst puts you in this position where you learn things. It then becomes your responsibility to share what you’ve learned with others. But if we fail to share it effectively, what’s the difference between us and Cassandra.

Here’s the intro.

Girl meets a god who gives her a divine power to see the future. The girl then turns down the god’s romantic advances. The god, who is clearly a creepy jerk, feels scorned and curses the girl. The girl can still see the future (her gift) but nobody will believe her prophecies (her curse).

We researchers and evaluators also have a gift. We can’t see the future, but through our methods we can measure the past and monitor the present. Then we can use that knowledge to project the future.

And like Cassandra we also have a curse. There is no guarantee that anyone will hear our words or believe what we share.

But unlike Cassandra, our curse is at least partially self-inflicted. We use the web to share our work in much the same way as we did in the late 90s. And we write reports almost as if the last 20 years never happened.

I believe we need a change, a monumental change. Not necessarily in our tools but in our mindsets.

Because the status quo is a path towards irrelevance.

The beginning pages of The Reporting Revolution.

The book is only $5 on Kindle ($10 in print), it’s a quick read and is packed with a lot of my best ideas around reporting. It also has a bunch of cartoons and reads more like a blog than a textbook. I’m proud of it and want to share it as widely as possible.

Buying the book also gets you an invite to a free webinar next month. Check out my post from the other day to learn more.

The PDF Must Die

For the last ten years or so I’ve taken a pretty measured stance on PDFs. Saying that they’re okay in moderation, perhaps even useful.

That’s the stance I take in The Reporting Revolution. I share three reporting strategies and the first is a modern PDF strategy. I kept it in because the book is really about making a transition and I know that most of you are not really ready to give up on the PDF.

But let me be completely honest. I think the PDF must die because it’s holding us back.

Back in 2014, Michael Quinn Patton reached out to me about creating a set of comics he could use in the fourth edition of his Qualitative Research & Evaluation Methods textbook. He added a “rumination” before each chapter and wanted a comic to go alongside each.

Number 5 was “Convenience Sampling is NOT Purposeful Sampling.”

He talks about how in a past edition he listed convenience sampling at the very end of a table of sampling strategies as the approach with the poorest rationale and lowest credibility. But when reviewing research publications and reports he would see methods sections that would state convenience sampling as “Patton’s purposeful sampling approach #15.”

I think PDFs are a bad habit we need to break. Like smoking and fast food. A single cigarette will probably not kill you and neither will a single big mac. And while a PDF isn’t really a bad secondary reporting tool, it’s way overused.

But I’m going to do more than just make the statement, in the next few months I’m going to systematically show you the reasons why I think…THE PDF MUST DIE.