The author demonstrates that relying solely on retrieval scores to gate Retrieval-Augmented Generation (RAG) systems is an unreliable safety mechanism. By testing a Romanian question-answering system built on a 2B open-weight model via Ollama, the researcher found that retrieval scores fail to distinguish between legitimate queries and "impostors" from adjacent bureaucratic domains. Specifically, a question regarding driver's licenses scored 0.773, surpassing the highest legitimate customs question at 0.697, resulting in a negative separation margin of -0.076. This overlap occurs because embedding models capture semantic and domain similarity rather than specific factual presence. To address this, the author proposes a four-stage post-generation gating architecture: Gate A (weak retrieval threshold), Gate B (verifying citations resolve to retrieved documents), Gate C (verifying numbers appear verbatim in sources), and Gate D (verifying substantive sentences are attributable to cited documents through lexical prefix matching). This approach shifts the validation from the model's adherence to instructions to a code-enforced check of the final output against the source text. While this method increases latency—running at approximately 2 tokens per second on a four-core CPU—it ensures that the system refuses hallucinations, such as the model incorrectly defining EORI as a "Chamber of Commerce" number, by prioritizing traceability over mere similarity.

Read original