What provenance should I demand from an AI research vendor?

Four guarantees: every claim links to a source you can open; the collection method is disclosed, including whether data is public, licensed, or yours; confidence is labeled as measured, derived, or hypothesis; and missing evidence is reported as missing. All four are checkable in a sample output before contract, and belong in the statement of work as acceptance criteria.

How do I verify where an AI tool's data comes from?

Take a finished output on a real subject and trace ten claims to the pages behind them, choosing against the vendor's interest — precise figures, direct quotes, obscure companies. Working links to the claimed pages verify the method; broken links, homepages, or a proprietary database no one may inspect falsify it. No questionnaire is involved, and the whole exercise fits inside a short meeting.

Do provenance requirements cover model training data?

Treat them separately. Output provenance — where each finding came from — is contractible and verifiable, because the tool gathered sources at scan time. Training-data provenance for foundation models is largely outside an application vendor's power to warrant, so sweeping warranties there deserve suspicion. Contract hard for the first; expect regulation, not vendors, to move the second.

Data provenance requirements when procuring AI research tools

Guide · enterprise · 4 min read · last verified 2026-07-22

Reviewed before publication Editorial board Independent commercial review

In shortFour provenance guarantees every AI research tool should meet before contract, phrased ready for the statement of work, with a source walk-back any buyer can run on a sample output before signing.

When procuring an AI research tool, require four provenance guarantees before contract: every claim links to an identifiable source you can open; the collection method is disclosed; confidence is labeled with its basis — measured, derived, or hypothesis; and absence of evidence is reported as absence rather than filled by a model. All four are verifiable in a live output before you sign. If a vendor cannot demonstrate them pre-contract, no clause will conjure them afterward.

What does provenance mean for research outputs?

Provenance is the answerable question “where did this come from?” — asked of every claim, not of the product in general. For an AI research tool, that means each finding traces to a source document, a retrieval date, and a stated method. The unit of provenance is the claim. A vendor who answers with “our proprietary database” has answered for zero claims. The practical form this takes inside a deliverable is an evidence trail: the stored path from statement back to source.

What are the four requirements, in contract-ready form?

| Requirement | Contract-ready phrasing | Pre-purchase verification |

|---|---|---|

| Source-linked claims | Each factual claim in deliverables identifies the source document it derives from, with a working link | Open a sample output; follow ten claims to their sources |

| Method disclosure | Vendor discloses what is read — public web, licensed data, customer data — and how collection honors access controls and opt-outs | Compare the disclosure against what the outputs actually contain |

| Confidence with a basis | Each finding carries a label distinguishing measurement from derivation from hypothesis | Check that labels state their basis, not just a score |

| Honest nulls | Where no evidence exists, deliverables say so rather than estimating | Ask about something no public page documents; watch what comes back |

Put these in the statement of work as acceptance criteria for deliverables, not in a side letter. Acceptance then re-runs the same verification on real output, which turns provenance into a recurring check instead of a procurement-day ceremony.

How do you verify provenance before buying?

Demand a finished output on a real subject and walk it backwards. Ten claims, chosen by you, followed to their sources — and choose against the vendor's interest: precise figures over round ones, direct quotes over paraphrases, obscure companies over famous ones. Magrios structures for this test deliberately: sample reports are open before any payment, the evidence explorer lists the source page behind every claim, and where the public web has no answer the report says “no public evidence found.” The point is not the vendor; the point is that the test is possible, so a vendor who will not enable it has answered your provenance question already. Evidence-first AI describes the passing state in more depth, and how AI assistants choose their sources explains why the selection of sources — not merely their existence — is worth reading closely.

Does this cover the model's training data?

No, and the two questions should not be merged. Provenance of research outputs is contractible and checkable: the tool gathered its sources at scan time and can show them. Provenance of a foundation model's training data is a different problem, largely outside any application vendor's power to warrant. A reasonable hypothesis is that training-data transparency will improve under regulatory pressure — the EU AI Act's documentation obligations push in that direction — but a procurement running today should contract hard for output provenance and treat sweeping training-data warranties from application vendors with suspicion.

Which red flags end the conversation?

A proprietary database no one may inspect, offered as a feature rather than admitted as a limitation.
Confidence scores with no stated basis — a number that answers “how sure?” without answering “based on what?” Confidence with a basis draws that line precisely.
Outputs that never say “we don't know.” Every honest research method has gaps; outputs without gaps are outputs with inventions.
Method claims that contradict behavior — “public web only” alongside findings no public page supports.

Any one of these is fixable in principle. A vendor showing all of them at the sales stage — the stage of best behavior — is showing you the product.

What survives into the contract?

Three durable artifacts: the four requirements as acceptance criteria in the statement of work, the sampled walk-back as the acceptance procedure, and a disclosure schedule listing what the tool reads and which subprocessors touch it. None of this is exotic drafting. It is the ordinary discipline of buying research: the deliverable must show its work, and the contract must say so before the first invoice does.