Public Sector Content Now Has Two Audiences

Aug 6

Written By The Jumping Elephants' Marketing Team

Why optimizing government content for people is no longer enough

In short. AI has changed the consumption side of government search. The production side, how we create, curate and monitor public sector content, has not caught up. This post introduces the research we have been doing since December 2024, and AeoGov, the application we built over the last six months while working with several prominent Government of Canada departments.

Where this started

Detecting whether an AI overview is based on the authoritative government source, and then improving that source so it becomes the one being cited, is what a colleague of ours likes to call a “wicked problem”.

We started from observations rather than a thesis:

AI - based search is very likely here for the foreseeable future. Despite a rocky start (early Google AI overviews famously suggested glue as a pizza ingredient), Google and the others have been pushing hard to make AI-enabled search the default experience rather than an option beside it.
There is a genuine benefit here. A good summary lowers the cognitive load of finding and understanding government rules, which is not a small thing for people who find that content difficult.
There are real downsides too, which is most of what follows.

What we were watching was not a product launch. It was a change in how constituents engage with public content, and the engines (Google, ChatGPT, Perplexity and others) have settled in as a third-party intermediary between public sector publishers and the people they serve.

Before: the user reads, and interprets

For twenty years we have designed on the assumption that users would read the page.

A person started at a Canada.ca page, or used Google to find some candidate starting points.
It was on them to decide which blue links to click, and then to read enough of the page to work out which rules applied to their situation.
We laboured over information architecture, information scent, and plain language wording so the content would be understandable and actionable, and we tested that work with real users, because watching someone try to complete a task is still the only reliable way to know whether a page works.

The onus was on the user to understand our chosen words.

Now: an AI engine rewords and summarizes

Users ask questions in plain language, sometimes not in English or French.
The engines ingest public sector content and return a summary customized to the specific question and circumstance.
Most people never see the source page, because they do not click through even when the links are provided.

The onus has moved to the engine, and the engine is not accountable to your organization.

We think AI-enabled search is here for the foreseeable future , because Google has made it the default. Users aren’t actively choosing AI search, they are just doing what they have always done - using Google to search for answers.

What should concern us about AI search

The wrong content gets surfaced. Redundant, outdated and trivial content (ROT) is still prevalent on government sites, and the AI engines do not always pick the current page.
Authoritative content gets bypassed. The engines sometimes reach for social media or non-government sources that are easier to parse than the authoritative official government source.
Some content is effectively invisible to AI engines. Decision tools in particular, because of how they are built.

This is not primarily a misinformation problem. Often the answer provided by the AI engines is broadly right. The issue for public sector publishers is that the organization may no longer be the source on topics it is responsible for, and until recently there has been no straightforward way of knowing.

What we built

AeoGov measures two things that ordinary web reporting does not:

Is the answer grounded in your page? Not whether the answer is objectively true, but whether it reflects what your content actually says.
Is your page the one being cited? A page can rank first in ordinary search and still not be the source behind the answer someone acts on.

We recommend remediation strategies. We suggest experiments to determine what will improve grounding and citation, then periodically re-run our analysis with the same queries, so you can see whether the change worked. The measurements run at scale across a department's content, and the scoring is risk-based, which gives content teams a prioritized list of pages rather than a general recommendation to write more clearly.

What it does not do. It does not rewrite your content for you. It cannot guarantee a citation, because no one can. It is not a traffic dashboard, and it will not tell you how many people visited.

Three things that surprised us

The engines look elsewhere. They reach for social media or non-government sources even when the supporting facts are sitting on the government page.
Almost nobody checks the source. In our own research, and in what other researchers are finding.
User behaviour is changing very quickly. Most of it came from Google changing the default, not from people seeking AI tools out.

There are exceptions, although in general what is good for the AI engines is also good for human readers.

Two audiences now, not one

Public sector content creators and curators now have two audiences, the humans and the machines, and we need to create content that serves both.

The core work has not changed much. We are still trying to make sure everyone, including people in marginalized groups, has the information and the tools to complete the tasks they need to, and we are still doing it through usability research with real users and evidence-backed recommendations. What has changed is that there is now a third-party intermediary sitting between the content and the person reading it.

Where we go next

There is a lot to unpack here. We are coming off a departmental engagement with more learnings than we expected, and we are now organizing and communicating what we have found so we can start these discussions with more of you. Please reach out or follow along if this is a topic you have been thinking about.