Systemic report failure is when structures, formats, and habits consistently prevent reports and other resources from reaching the right people in ways they can actually use. In evaluation and research we treat this systemic failure as if it’s a skills issue. It’s not. Better charts, interesting stories, and well-designed PDFs are not enough to make a real difference, and in this series of posts I’ll show you why.
I remember several years ago talking to a friend in the data visualization world. A few months earlier they had created a set of visuals for a new report. And they shared that report with me so I could give it a look.
It was a beautiful pdf. Well written with clear charts and well-made arguments. The kind of PDF you don’t mind reading.
I wanted to share it forward with a link to the publication so I went to Google, searched for it, and nothing came up. Eventually I was able to track it down to a single download page in a bigger resource library. You couldn’t see any of the interesting stuff on this page, it just looked any other PDF.
That’s the thing about systemic report failure. You can create beautiful reports with well-written stories, but if the system required to get that report into your audience’s hands does not exist, the report will never reach its full potential.
In this post I’ll be talking about The Content Gap. In short, it’s the unfilled void between the technical report written by the research or evaluation team and the stuff the communications team shares on social media. Because ultimately it’s the medium, not the message, that determines whether or not your work will reach an audience.
What’s the job? Evaluation, Design, and Communications
The Job for the Evaluators, Researchers, and Subject Matter Experts.
Most evaluations and research projects end with a technical report. This technical report is a reflection of the thought process that carries through the full project and it documents the work.
Even resources like toolkits and training guides end up in a technical format because they are often initially created by subject matter experts. As such, these resources are often technical even if there purpose is to reach a broader audience.
The Job for the Graphic Design Team.
If a project has a large enough budget or internal design capacity, these resources might go to a design team. This team will usually not touch the actual written content, but will likely redesign the structure and sometimes illustrate.
The Job for the Communications Team.
The communications team is the connection point between an organization and the wider world. The organization’s blog, social media channels, and email newsletter are the responsibility of the comms team. They may create featured images but they also rarely touch the content of a resource.
Sharing the Link.
At the most basic level a technical report pdf will often live on a page where it can be downloaded. This page usually has an abstract and also includes pdf downloads for an executive summary, any translations, and other related downloadable resources.
The link for this page gets shared with the communications team. The communications team share the link to the pdf as part of their overall social media strategy. It probably gets shared once on each channel (X, LinkedIn, Instagram, etc.) the organization uses. And that’s usually it.
Individual members of the research and evaluation team might also share the link through their own networks. They might also deliver internal presentations, conference presentations, and write papers where the resource link will be shared.
Evaluation: Downloads, Pageviews, and Social Stats
Evaluation for these kinds of resources is usually limited to superficial metrics like downloads, page views, and social media engagement.
The comms team might look at the social media stats. The organization’s web team (which in bigger orgs is often a different group) might look at the pageviews. The research and evaluation team will likely only ask for download stats.
The Content Gap
If you’re not a regular content creator, this probably all seems fine. It is the standard way technical and training resources have been shared since the late 90s. The basic assumption is that the people who want to read it will find it.
But I’ve been blogging, sharing comics, and creating resources for over 15 years. And I see how we attempt to disseminate our work as being totally inadequate. And one of the big reasons for that is the content gap.
The gap is in the amount of easy-to-share material that should exist between the technical report and the comms team. This includes the HTML version of the report or resource. Detachable resources like infographics, charts, or comics. And any other related media like videos, podcasts, and webinars.
Feeding the Comms Team
Unless it’s already worked into the budget, the comms team doesn’t usually have the bandwidth (or subject-matter expertise) to adapt technical reports and resources into web-friendly/audience-friendly content.
But if someone takes the time to create web-friendly resources, this is often appreciated by a team always looking for interesting and relevant things to share. It’s a lot less work to share something cool than it is to share a link to a boring technical pdf download.
Content Orbits
When you create web-friendly materials they can also develop a orbit of their own. Take my comics for example. Almost all of them were created to match with ideas I’ve shared in blog posts. But each comic can potentially have its own orbit or audience.
Videos, infographics, podcasts, webinars, and websites can draw their own direct audiences. Discovered through search engines, AI, social media, and through sharing by other creators. These people are not necessarily connected to the organization and could be completely separate from the audiences reached by the comms team.
Feedback Loops
Sharing social web-friendly content creates feedback loops. This happens through comments, shares, participation in webinars, replies to email newsletters, etc. If you let it, this feedback can influence the design of new content, drawing more of an audience, and improving the usefulness of the resources.
Rich Evaluation Data
With a content strategy evaluation can become a lot more rich. There are tons of data available through feedback loops, page visits, video plays, time on web page, reshares, social media engagement, polls, Q&A, search engine stats, and even traditional downloads. Because now that you have all the other stuff, the PDF can serve it’s true function as being the technical basis for everything that follows.
So Where is the Systemic Report Failure?
The responsibility for content creation, strategy, and evaluation is currently nobody’s job.
The subject matter team often checks out at the technical report because this is usually when the funding ends. Also they don’t really see creating websites, infographics, podcasts, webinars, and videos as being their job.
The comms team is more versed in creating content like this, but they have a different responsibility. Remember, they are the face of the organization not the caretaker of any one particular resource. They also don’t often have the subject matter expertise necessary to adapt the content properly. Same goes with the org’s graphic design team if it has one.
Content strategy is a specialty that requires a unique skillset. And this kind of position rarely exists even in the largest research and evaluation organizations.