Grounding and Citation

A customer asks their RAG chatbot a question. The answer looks right. They ship it to a reviewer. The reviewer asks "where did this come from?" There is no answer, because the chatbot was never built to produce one.

This lesson is about building one that is.

What grounding actually means

Grounding is the step that binds the generated answer back to the passages it was generated over. Not "mentions sources somewhere" — a specific, checkable property: every substantive claim in the answer points at a specific retrieved passage by id, and a reader can click through to see the original sentence.

Three things fall out of this. The user can verify the answer. The compliance team has an audit trail. And you — the FDE debugging a failure — can see which passage the wrong answer came from, which is the difference between "the model is wrong sometimes" and "the retriever is returning the wrong chunk for this class of query."

The pattern that makes it hold

A free-text instruction to "cite sources" is weak. The model complies most of the time and quietly stops when the prompt gets long. The pattern that actually holds is a schema.

prompt

1. WhyTag every retrieved passage with a short id before handing it to the model.

2. >You typed: You will answer only from the passages below. Cite the passage id for every claim. If the passages do not contain the answer, respond with { "answer": null, "reason": "not_in_context" }. [S1] (policy.md#refunds-enterprise) Enterprise annual contracts have a 30-day refund window from the first charge. [S2] (policy.md#refunds-starter) Starter plan refunds are available within 7 days. Query: what is the refund window on an Enterprise annual contract?

3. Claude replied: { "answer": "30 days, measured from the first charge.", "citations": ["S1"], "confidence": "high" }

4. WhyThe schema requires a non-empty citations array. A response without one fails validation and never reaches the user. The model cannot produce a passing answer without citing — the instruction has teeth.

Two choices in that example are not cosmetic. The passage ids are short and unique — a longer id leaks tokens and the model gets them wrong on long contexts. And the schema includes an explicit "not in context" branch, so the model has somewhere to go when the retrieval was bad, rather than confabulating.

Measuring groundedness

A property you don't measure is one you don't have. Groundedness is one of the cheapest metrics to add to a RAG eval suite, and one of the most useful.

The question the judge asks, per claim, is simple: is this claim verifiable by one of the provided passages? That question is specifically answerable from the context, which is why an LLM-as-judge is reliable at it — unlike "is this true in general?", which asks the judge to know the world.

1. Step 1

   Decompose the answer

   Split the generated answer into atomic claims. One claim per sentence is the usual approximation; a small decomposition prompt does this reliably.

2. Step 2

   Verify each claim

   For each claim, ask the judge: can this be verified by one of the retrieved passages? If yes, which one? The judge's answer is yes/no plus a passage id.

3. Step 3

   Aggregate

   Groundedness = fraction of claims marked verifiable. An answer with four claims where three are grounded scores 0.75, and the ungrounded claim is the one to go look at.

The payoff is debugging. A groundedness drop from 0.95 to 0.80 after a chunking change is a precise signal: a claim class is now unsupported, and the per-claim trace tells you which. Answer-level accuracy would have moved by less and told you nothing about where.

The compliance story

In general-knowledge use cases, citations are a nice-to-have. In regulated verticals, they are the product.

An FDE on a deal in one of these verticals leads with the grounding story, not with the model benchmark. "The answer comes with a click-through to the specific clause and version it was drawn from" is the sentence that closes the room. Benchmarks change; the audit trail is the thing procurement is actually buying.

Failure modes to watch for

Three ways grounding looks fine and is quietly broken.

Citations that don't support the claim. The model cites S1 but S1 does not actually say what the answer says. The citation is a fig leaf. The fix is a verifier pass: for each (claim, cited passage) pair, ask the judge whether the citation supports the claim. Any "no" in that trace is a real failure, often the only one.

Over-grounding. The model cites every passage, including irrelevant ones, to look thorough. Precision of the citation set matters — a schema requiring "cite only the passages used" and a judge that penalises spurious citations keeps this honest.

Grounded but wrong. The retrieved passages were wrong, and the model dutifully grounded its answer in the wrong source. Groundedness is 1.0, accuracy is 0. This is why groundedness is a retrieval diagnostic, not a substitute for retrieval quality — it tells you the generation stage obeyed, not that the earlier stages were right.

What to take away

Grounding is the step that binds generated claims to specific retrieved passages by id, and it is what turns a RAG system into something a compliance team can sign off on. The pattern that holds is structured output: unique passage ids, a schema that requires a non-empty citations array, and a "not in context" branch so the model has somewhere honest to go on a bad retrieval. Measure groundedness per claim with an LLM-as-judge, because the per-claim trace tells you which sentence invented something. In finserv, healthcare and legal, lead with grounding in every customer conversation — it is the deliverable, not the extra. And remember that grounded does not mean correct: a grounded wrong answer is a retrieval problem the groundedness metric alone cannot see.

Next: the agentic loop as an FDE deploys it, which is where the LLM stops answering questions and starts doing work.