Retrieval-augmented generation
Building a RAG pipeline that can show its work
Tactis answers questions about federal nursing-home regulation for the people responsible for complying with it. In that setting a confident wrong answer is worse than no answer, which changes almost every decision in the pipeline.
The problem
A skilled nursing facility is governed by a large body of federal regulation and interpretive guidance. It is public, it is searchable, and it is nearly unusable under time pressure. The text is long, heavily cross-referenced, and written in a vocabulary that does not match how staff describe what actually happened on the floor.
The naive version of this product is a vector database and a prompt. It demos well and fails in the specific way that matters: it retrieves something adjacent, writes a fluent paragraph, and attaches a citation that does not support the claim. A surveyor does not grade you on fluency.
So the design target was never “answer everything.” It was: retrieve the right passage, keep it whole, cite it honestly, and decline when the corpus does not contain the answer.
Constraints that shaped it
Citations are the product
An answer a user cannot verify against the source is worthless here, because they are the one who has to defend it. Every retrieval decision is downstream of keeping the citation trustworthy.
Exact tokens matter
Tag numbers and section anchors are the load-bearing tokens in this corpus, and pure dense retrieval treats them as noise. That single fact forces hybrid retrieval rather than making it a nice-to-have.
Two corpora, two trust levels
Public regulation and a facility’s own documents cannot live in the same store, because they do not carry the same sensitivity. That split runs through the whole retrieval design.
The pipeline
Every stage below exists because the version without it produced a specific, reproducible failure. None of them are there for symmetry.
Intent validation and rewrite
The raw question is checked for whether it is actually a regulatory question, then rewritten into retrieval-friendly language. Staff ask things like "what happens if we miss a bath" — the corpus says "activities of daily living" and cites an F-tag. Skip this step and the embedding never lands near the right text.
Expansion into several queries
One question becomes several differently-phrased queries. A regulation can be reachable through its clinical vocabulary, its procedural vocabulary, or its tag number, and no single phrasing finds all three.
Hybrid retrieval, dense and sparse
Every query runs against both a dense embedding index and a learned sparse index. Dense catches paraphrase; sparse catches the exact tokens that matter enormously here — tag numbers, section anchors, statutory citations — and that dense retrieval is famously casual about.
Reciprocal rank fusion, authority weighted
The ranked lists are fused by reciprocal rank, so a chunk that several phrasings independently surface rises. Fusion also applies a clamped authority factor from chunk metadata: the regulation itself outranks interpretive guidance, which outranks commentary. Not all correct sources deserve equal weight.
Two passes of deduplication
Exact duplicates go first, then near-duplicates by cosine similarity, keeping the best-scoring member of each cluster. Regulatory corpora repeat themselves heavily across appendices. Without this, the context window fills with six phrasings of one rule and the actual answer never makes it in.
Diversity and reranking
Maximal marginal relevance trades a little relevance for coverage, so the context spans the question instead of stacking one facet. A cross-encoder reranker then scores what survives — expensive per pair, which is exactly why it runs last, on a short list.
Section completeness
Retrieval returns chunks; regulation is written in sections. A chunk that ends mid-requirement produces a confidently truncated answer. This step pulls sibling chunks back in so a cited section arrives whole.
Answerability check
Before drafting, the assembled context is tested against the question: is the answer actually in here? If it is not, the system says so and cites what it did find. This is the step most pipelines skip, and it is the one that decides whether the tool is trustworthy.
Retrieved text is data, never instructions
Anything a retrieval system pulls into a prompt is untrusted input. If a facility uploads a document containing ignore previous instructions, a naive pipeline hands that straight to the model in the same channel as its own system prompt.
Retrieved content is wrapped in an envelope marked with a random nonce generated per request. The user’s question and the system’s own scaffolding stay outside it; the citation lines stay inside and are still copied out. Because the closing marker is unguessable at the time the document was written, injected text cannot forge its way out of the envelope and back into the instruction channel.
Worth being precise about what this is: it marks a boundary, it does not inspect content. It is not a classifier trying to detect malicious text, and we do not claim it catches attacks by recognizing them. It makes the structural claim — this region is data — enforceable.
What we rejected
Fine-tuning a model on the regulation
It moves the text into weights, where it cannot be cited and cannot be updated when the rule changes. Everything that makes this product defensible depends on pointing at a passage that exists.
One giant context window instead of retrieval
Long-context models are real, and stuffing the corpus in is still the wrong trade: cost per question scales with the corpus rather than the question, and attention over thousands of near-identical passages degrades exactly where this corpus is densest.
Letting the model answer from general knowledge
Frontier models know a surprising amount about this regulation, and that is the trap. Allowing an unsupported answer through means the citation no longer certifies anything, because the user cannot tell which answers were grounded. Declining has to be a real outcome for grounding to mean something.
How we know a change helped
A pipeline with this many stages cannot be tuned by impression. Retrieval runs against a held-out set of questions with known correct passages, so a change to fusion weights or rerank depth produces a number rather than a feeling. This matters most for changes that feel obviously good: several plausible improvements measured flat or slightly negative, and the eval is the only reason they did not ship.
Where this generalises
Almost none of this is specific to nursing homes. Any corpus with precise identifiers, heavy internal repetition, and a real cost to being confidently wrong lands in the same place: contracts, building codes, insurance policy language, technical standards, internal engineering documentation.
If you are staring at a retrieval system that demos well and disappoints in practice, the cause is usually one of the stages above being absent rather than the model being wrong. We do this work as a service, and it usually starts by building the evaluation set first, so there is something to argue with.
Retrieval that has to be right
Tell us about your corpus and what a wrong answer costs you. That second part determines most of the architecture.