What should procurement ask a market intelligence vendor?

Ask twelve questions in three families. Evidence: can you open the vendor's citations yourself before contract, what happens when data does not exist, and who reviews claims independently. Measurement: is the benchmark locked, are declines reported, and when does the vendor refuse to compare reports. Access: what is the minimum grant, what is never touched, and what you keep at exit.

How do you catch a rehearsed answer in a vendor evaluation?

Never accept an answer where you could ask for an artifact. Instead of asking whether citations resolve, pick one and ask the vendor to open it on screen. Instead of asking whether declines are reported, ask to see a report whose finding was bad news for the customer. Rehearsal survives questions; it rarely survives a live artifact request.

Is a vendor that admits gaps riskier than one that claims none?

Usually the opposite. Every vendor has gaps; the variable is whether they surface under questioning or under incident. A vendor that names its gaps — certifications still in progress, a single operating region — has demonstrated how it behaves when news is bad, which is the property a buyer depends on most. A vendor claiming no gaps is either not looking or not telling.

The procurement question set for market intelligence vendors

Guide · Market Growth · 5 min read · last verified 2026-07-22

Reviewed before publication Editorial board Independent commercial review

In shortTwelve procurement questions that separate measurement vendors from narrative vendors: open citations before contract, locked benchmarks, honest refusals to compare, and follow-ups that defeat rehearsal.

Procurement should ask a market intelligence vendor twelve questions, split across three families: evidence questions (can any claim in a report be traced to a source you can open yourself), measurement questions (is the benchmark locked, are declines reported, and when does the vendor refuse to compare), and access questions (what data the vendor needs, and what it will never touch). The questions that separate measurement vendors from narrative vendors share one property: they cannot be answered with a slide, only with a live artifact shown in the meeting. This set is written so a procurement lead can paste it into an RFP unchanged.

The twelve questions

Pick any three citations in your sample report — can we open them right now, before contract?
What happens when the data needed to answer our question does not exist?
Who reviews claims before publication, and are the reviewers independent of the author?
For any claim we point at, can you show the full chain from that sentence back to its sources?
Is the benchmark locked before measurement begins, and can you prove when it was locked?
Are declines reported with the same prominence as gains?
Under what conditions do you refuse to compare two reports?
When your methodology changes, what happens to trend lines that cross the change?
What is the minimum access you need to deliver the contracted work?
What data do you never touch, even if we offered it?
Which certifications do you hold today, and which are in progress?
If we leave, what do we keep — reports, evidence, and the ability to audit past claims?

Evidence questions: one through four

The first four test whether the vendor practices evidence-first AI or merely claims it. Question one moves the demo out of the vendor's hands: you choose the citations, and they must resolve on the spot. Question two tests refusal behaviour — the right answer is that the vendor says so and declines to fabricate; the right proof is a delivered report that says exactly that. Question three tests independence: review only means something when the reviewer's incentives do not depend on the finding surviving. Question four is the traceability question. A vendor that cannot walk backwards from a sentence in its own report cannot support decision traceability on your side either — and eventually an auditor will ask you to walk that chain.

Measurement questions: five through eight

An unlocked benchmark can be re-fitted after the fact until any story becomes true, so question five asks for the lock and its timestamp, not a description of rigour. Question six separates instruments from marketing channels: an instrument reports whatever it measures, and a vendor whose reports only ever contain good news is describing its incentives, not your market. Question seven is decisive — willingness to refuse comparison is the mark of a real measurement practice, because reports built on different question sets, windows, or methodology versions cannot be honestly compared, and a vendor who never refuses is comparing them anyway. Question eight accepts that methodologies change; the honest treatment is a visible break marker on the trend, never a silent restatement of history.

Deep dive: the follow-up discipline

Headline questions end up in RFP templates, and competent sales teams rehearse them. The discipline that defeats rehearsal is simple: never accept an answer where you could ask for an artifact. For each headline question, the follow-up — and the red-flag answer, verbatim.

"Open the third citation on this page, now, on your screen." Red flag: "We can arrange source access after signature."
"Show me a delivered report that says the data was insufficient." Red flag: "Our coverage means that almost never happens."
"Name the reviewer role, and show one claim a reviewer blocked." Red flag: "The whole team reviews everything" — when everyone reviews, no one is accountable.
"We will pick the sentence. Walk backwards from it." Red flag: any need to check with an analyst offline.
"Show the lock timestamp next to the first measurement date." Red flag: "We recalibrate baselines to stay current."
"Show me a report where the finding was bad news for the customer." Red flag: "Our method tends to surface opportunities."
"Give one example where you told a customer two reports could not be compared." Red flag: "Comparison is always possible with the right context."
"Show a trend line that crosses a methodology change." Red flag: a smooth line with no break marker.
"List the scopes in the narrowest grant that still delivers the work." Red flag: a request for administrator access to simplify setup.
"Put the never-touch list in the contract schedule." Red flag: reluctance to write down what was just said aloud.
"Show the audit dates, not the logo wall." Red flag: certification badges with no report dates behind them.
"Export one report with its evidence now, and open the export." Red flag: "Full export is on the roadmap."

Rehearsal survives questions. It does not survive artifacts.

Data-access and security questions: nine through twelve

These four exist because an intelligence vendor sits close to strategy, and access follows closeness. The narrowest-grant question has a right shape: the vendor states exactly what it reads, why each scope is required, and how you revoke it without a support ticket. The never-touch question has a right shape too: a short written list — credentials, customer records, anything the work does not require — that survives into the contract schedule. The exit question is the one most RFPs omit: evidence you paid for should remain auditable after you leave; every decision built on those reports loses its provenance the day an export fails. A vendor confident in its posture publishes these answers before being asked; we publish ours at magrios.com/trust.

The questions we would answer imperfectly

If you put this set to us today, two answers would be imperfect. On question eleven, some of our certifications are in progress rather than held — the honest answer is the roadmap, the audit dates, and the compensating controls in writing, not a logo. On residency, we operate from a single region today; if your policy requires in-region processing elsewhere, we are the wrong vendor this quarter, and we will say so in the first meeting rather than the last. We think that trade cuts in the buyer's favour. Every vendor has gaps; the only variable is whether they surface under questioning or under incident. A vendor that admits gaps has shown you how it behaves when the news is bad — which is exactly the behaviour question six was designed to test. A vendor that claims none has told you it is either not looking or not telling, and neither trait improves after signature.