Corrective RAG with self-reflective retrieval

A standard RAG pipeline is a straight line: embed the question, pull the top-k chunks, paste them into a prompt, generate. The line has no opinion about whether the chunks it pulled were any good. When retrieval misses — the question was phrased differently from the source, the answer spans two documents, the knowledge base simply does not contain it — the generator still receives a prompt and still produces fluent, confident, well-formatted text. Nothing in the pipeline is watching.

That is the worst kind of failure, because it is invisible from the outside. A system that returns an error is annoying. A system that returns a plausible wrong answer with the house style intact is dangerous, and it is what you get by default.

The whole design here follows from treating retrieval quality as something the system must measure about itself at run time, not something you hope for at build time.

The pipeline is a cyclic state machine built with LangGraph rather than a linear chain. The distinction matters: a chain runs every step once in a fixed order, while a graph has conditional edges and can route work backwards. That is what makes correction possible at all.

Retrieval pulls candidates from Qdrant. A grading node then scores each retrieved document for actual relevance to the question — not vector proximity, which already got it into the candidate set, but whether it contains material that answers what was asked. Documents that fail grading are dropped.

If too little survives grading, the graph does not proceed to generation. It routes to a query-rewriting node, which reformulates the question and re-enters retrieval — the failure is usually the phrasing, not the corpus. Up to three of these self-correction loops run before the graph gives up on the internal knowledge base and falls back to live web search.

Only then does generation run, and the answer passes a grounding check before it is returned: is every claim traceable to the material actually retrieved? If it is not, the system degrades gracefully and says it does not know instead of filling the gap.

Vector search uses a bi-encoder: the query and each document are embedded separately, then compared by distance. That is what makes it fast enough to search millions of chunks — the document vectors are computed once, ahead of time. It is also what limits it, because the model never sees the query and the document together, and so cannot reason about the relationship between them.

A cross-encoder does see them together. It reads the query and a candidate document as a single input and scores the pair directly. That is far too slow to run across a whole corpus, and exactly the right tool for reordering a shortlist. So retrieval stays two-stage: Qdrant produces a broad candidate set cheaply, and the cross-encoder reranks it into the handful of chunks the generator actually gets.

Adding that reranking stage moved top-5 retrieval precision from 0.72 to 0.89 on the evaluation set. Two things are worth saying about that number. It is precision@5, not an aggregate quality score, so it means something specific: of the five chunks handed to the generator, how many genuinely bear on the question. And it is a measured before-and-after on a fixed question set, not an impression formed by trying a few queries by hand.

Every claim on this page is a number from the evaluation harness I run alongside the system: a golden question set per knowledge base, retrieval scored on hit-rate, precision@k and MRR, generation scored on groundedness, citation coverage and refusal correctness, all of it wired as a regression gate.

That harness is the reason this project has numbers instead of adjectives, and it is documented separately.