Make every answer verifiable
A RAG answer without citations is a claim. One with checkable citations is evidence.
Citations do two jobs. They let a user verify the answer in seconds, and they force the system to stay close to its sources. But they only help if they're real, so make them checkable.
How to build it:
- 1Give every chunk a stable ID, plus a title, location and date. Pass them to the model with the text.
- 2Ask the model to cite by ID next to each claim, not at the end of the paragraph.
- 3Check the citations in code. Does every cited ID exist in the retrieved set? Does the cited chunk contain the claim? A second pass can test this.
- 4Show the user the passage itself, along with the link, so verifying takes seconds.
- 5If nothing supports an answer, say so. 'I couldn't find this in the documents' is a good output.
Watch for the classic failure: a citation that points to a real document that doesn't say what the answer claims. It looks credible and it's wrong.
To measure it, sample 50 answers and score two things: is each cited passage relevant, and does it support the claim? Track both over time.
Do your users ever click your citations, and do the passages hold up when they do?
#RAG#GenerativeAI#LLM#AI
New notes go out on LinkedIn first. Building something with AI? We can help.
