Five silent ways a RAG system fails

A RAG system rarely crashes when it's wrong. It answers confidently anyway. Here are five silent failures, each with a quick check.

1. 1Retrieval miss. The right chunk was never found. Check whether the answer's source is in the top 20 results.
2. 2Wrong chunk wins. A similar but irrelevant chunk ranks first. Read the top 3 for 20 real queries.
3. 3Cut context. The answer sits one paragraph past the chunk boundary. Look for chunks that start with 'it' or 'this' and have no subject.
4. 4Stale data. The index still holds last quarter's policy. Store a last-updated date per chunk and show it in answers.
5. 5Faithfulness gap. The right chunk was retrieved, and the model still said something the text doesn't. Ask it to quote the supporting sentence for each claim.

Only the fifth is the model's fault. The first four happen before the model sees anything, which is why a bigger model so often doesn't fix a RAG app.

This week, pick 20 real questions, log where each one fails, and fix the most common failure first.

Which of the five hurts you most right now?