What is a control question used for in AI visibility tracking?

It's a prompt about an unrelated topic, included so it should never move. If it does move, that points to a change in the measurement instrument rather than the market.

How do I pick a good control question?

Choose a topic with no plausible connection to your brand or category, phrased with the same structure and length as your tracked prompts.

What does it mean if my control questions start moving?

It's a signal that something upstream changed, like a model update or sampling noise, rather than anything real happening in your market.

How many control questions should a tracking program include?

Enough to notice a pattern rather than a single coincidence, run consistently alongside the same tracked prompts every time.

What is a control question? A practical definition

Glossary · Continuous Intelligence · 4 min read · last verified 2026-07-21

Reviewed before publication Editorial board Independent commercial review

In shortA control question is a prompt with no connection to your brand, included specifically because it shouldn't move. When it does, the shift likely came from the model, not the market.

A control question is a prompt included in an AI visibility tracking set specifically because it shouldn't change — it has no connection to the brand or market being watched. If a control question moves, the shift likely came from the measurement instrument, not the market.

The idea borrowed from experimental design

The control question isn't a new idea invented for AI visibility tracking — it's the same logic used in any experiment that needs a baseline. A drug trial has a placebo group specifically so researchers can separate the treatment's effect from everything else that might change a patient's condition on its own. AI visibility measurement needs the same separation: a way to tell whether a change in the numbers came from the thing being tracked, or from something else entirely — the model, the sampling, the day of the week.

A control question does that job by being deliberately irrelevant. It's a prompt about a topic, industry, or brand that has no plausible connection to the company running the measurement. Its expected behavior is to stay flat, run after run, because nothing about the actual subject of the tracking program should ever touch it.

Without something playing this role, a team has no honest way to answer the question "did the measurement instrument change, or did the market?" They're left interpreting every number in isolation, with no fixed point to compare it against. A control question turns that guesswork into a check: a stable reference reading that either holds, and quietly confirms the instrument is behaving normally, or moves, and tells the team to look upstream before drawing any conclusion about the brand itself.

What makes a good control question

A useful control question shares the same structure as the prompts being tracked — similar length, similar phrasing style, similar level of specificity — but points at an unrelated subject. If the tracked prompts ask about project management software, a control question might ask about an entirely different category, like commercial kitchen equipment or veterinary scheduling tools, phrased with the same kind of buyer-intent language.

The point of matching structure is that a control question needs to be exposed to the same sources of noise as the real prompts — the same model, the same time of day, the same sampling process — so that when it does or doesn't move, that tells you something about the instrument itself rather than about the specific unrelated topic it happens to cover.

What a moving control question tells you

Because a control question has no reason to change, any movement in it is diagnostic. This connects directly to how to tell a model update from real market movement: if a batch of control questions shifts on the same day as the tracked prompts, that's strong evidence the shift came from something upstream of the market — a model update, a retrieval change, or ordinary sampling error — rather than from anything happening in the brand's actual competitive position.

A control set that never moves is doing exactly what it's supposed to do. It only becomes useful the day something in the tracked prompts does shift, because at that point it answers the question that a single number never can on its own: is this real, or is this the instrument?

Worked example: catching a false positive with controls

Take a hypothetical tracking program running 20 prompts about a brand's category, alongside 5 unrelated control questions about a completely different industry. On a given sampling pass, the tracked prompts show a jump — mentions rise from 8 of 20 to 15 of 20, a shift that would normally get flagged as a meaningful gain.

Before reporting that as a market win, the team checks the 5 control questions from the same pass. Two of them also shifted, moving from a combined 1 mention to 4 mentions across topics with zero connection to the tracked brand. Because the controls moved too, the team treats the tracked-prompt jump as likely instrument noise rather than a real gain, and holds off on reporting it until a follow-up pass confirms whether the shift persists.

Without the control questions, that same jump from 8 of 20 to 15 of 20 would have looked like unambiguous good news.

Building a control set that actually works

A working control set needs a few things: enough questions to notice a pattern rather than a single coincidence, topics genuinely unrelated to the tracked brand and its competitors, and consistency — the same control questions run alongside the same tracked prompts, every time, so their baseline behavior is known well enough to notice when it breaks.

Control questions pair naturally with the kind of unbranded, category-level prompts discussed in why branded queries are the wrong benchmark — both exist to make the measurement more honest than a single flattering number ever could be on its own. Together, they turn a raw score into something closer to an actual signal, which is what makes it worth comparing against a broader read on AI share of voice in the first place.

A control set also needs light maintenance over time. A topic that seems safely unrelated today can drift into relevance later — a company expands into an adjacent category, or a control subject unexpectedly becomes newsworthy in a way that spills into how models answer nearby questions. Reviewing the control set periodically, and swapping out anything that's stopped being genuinely neutral, keeps the baseline trustworthy instead of becoming a second thing that quietly needs explaining.