Dissemination Systems Change

How to shift your organization away from Report Factory and towards a strong Insight Ecosystem.

I. Moving Beyond Performative Dissemination

I believe that most evaluation teams write reports with the best of intentions. They work late nights cleaning data, discovering insights, drafting narrative, and obsessing over clarity. Most people don’t get into research or evaluation for the money, they do all of this because they care.

I also believe that most evaluation managers and organization leaders direct reporting efforts with the best of intentions. They know that design and simplicity matters when it comes to report design. So they actively encourage creative professional development, testing alternative approaches, and bringing in outside help when the budget allows.

But here’s the big problem.

You can design beautiful reports. They can feature great data storytelling, be short and to the point, include engaging charts, graphs, maps, and other illustrations. They can be works of art that garner internal praise and win design awards.

And they can still sit on the proverbial dusty shelf, barely read and hardly ever used.

The simple truth is a beautiful report is just a performance. It’s only effective for the people who attend. And in most organizations, attendance is exclusive by default.

II. Placebo Reports

When a report looks good, it can act like a placebo. The evaluator feels successful because they have something beautiful to share. Leadership feels successful because when they open that beautiful report, it feels like something a leading organization would create.

It’s not a coincidence that an organization’s communications team, the one most associated with that organization’s brand, often takes on the responsibility for sharing the report. Beautiful reports become show pieces, ugly reports barely get shared.

But is that really the goal of dissemination? To make an organization look good when set against its peers?

If not, have you attempted to understand HOW someone might use the insights you share? Do you know if all the audiences you wrote the report to serve have a chance of actually experiencing your work? Do you have the evidence in hand to show examples of use?

My guess is probably not. Not because it’s impossible, but because it’s just not how traditional reporting systems were designed.

III. The Report Factory

What’s the most environmentally friendly car you can buy? A luxury EV, small battery EV, plug-in hybrid, or a cheap gas car?

That question has a fundamental flaw. While the most environmentally friendly car might be a small battery EV, it would be better for the environment if you simply walked, biked, carpooled, took public transportation, or even worked from home.

So what is the best report format for delivering information to your audience? An infographic, 1-pager, slidedoc, visual report, executive summary, technical report, or web page?

That question has the same flaw. Because while any of those might be true, your audience might be better served with a presentation, a webinar, an email, or even a conversation over a cup of tea.

Most dissemination frameworks and advice answer the first question. They assume that the goal of an evaluation or research project is to produce a report. And the goal of dissemination is to share that report. It’s a factory model, built before the internet was a thing, and it’s holding us back.

IV. Theory of Use and Insight Ecosystems

Rakesh Mohan is a former Director of Idaho’s Legislative Office of Performance Evaluations. When he held the role he would regularly take visits to the state capitol, roaming the halls and connecting directly with policymakers. Like all the best evaluators, for which there are many, Rakesh did tons that just would never show up on a traditional dissemination plan.

I know a lot of evaluators who are deeply connected to the communities who might have an interest in their work or find it useful. Connected in the kind of way that information can get shared via text message or during coffee breaks, not just through formal reporting.

When I talk about dissemination systems change, I’m not suggesting that we need to create something completely brand new. Because the best insights already spread organically through word of mouth. But if we’re willing, we can use our dissemination budgets to help strengthen this process and make it more effective. To develop and grow our insight ecosystems.

Recently, working through a dissemination challenge with one of my big federal clients, I developed a framework that I call Theory of Use. Instead of starting with a report, we identify specific audiences and key insights. Then we examine how (or if) the two connect. The result is practical insights on how we can strengthen the insight ecosystem.

V. The Path Forward: The Dissemination Audit

If you really want to improve your dissemination systems and make your work more useful, resist the temptation to start with training. You also don’t need to hire a hotshot graphic designer or illustrator.

Start with a dissemination audit.

• Step 1. Look at real resources you have shared in the past (or resources you are about to share).
• Step 2. Identify your audiences (I like to use my five audience framework) and determine whether or not all of those are actually reached by your current dissemination system. My experience is that we usually have mechanisms to share insights with clients and peers. We are less likely to have good mechanisms that reach our community, influencers, and the public.
• Step 3. Identify some insights. You don’t have to identify ALL of your insights for an audit, start with a few. Think about the desired outcomes of sharing each of these insights with your different audiences.
• Step 4. Based on your dissemination products and general actions, how likely is it that you will actually connect your insights with each audience? If dissemination has already started, do you have any concrete evidence that anyone has connected to and used your work?
• Step 5. Pull all your findings together and use it to spark conversation on how to improve your dissemination systems.

And finally, if you have a budget, just hire me to do it for you. I offer a dissemination audit for $3K and if you hire me for any additional transformation services, I take the cost of the dissemination audit off the cost of the service.