02 — Platform Retrieval System: RAG / GraphRAG at Scale¶

AI Foundation KB · Retrieval area Owner: Pharmacy Operations · Status: Architecture draft · Date: 2026-06-03

Scope. This document designs the platform retrieval system for the scale-grade AI foundation — the shared grounding layer underneath every engine (Balance Meter fall-risk, Smart Scheduling, Medication-Input Intelligence, refill anticipation, AI triage). We already run RAG over a pharmacist-curated drug KB behind a deterministic safety rail; this doc extends that proven pattern into a multi-engine, 6M-user retrieval platform.

Posture reminder (load-bearing). We are in the consumer-wellness lane. Member-facing surfaces emit reminders/confirmations, never dosing or diagnosis. Clinical intelligence is internal IP. Retrieval design here serves both the internal clinical-grade pipeline and the member-facing wellness pipeline — and the architecture must keep them separable. HIPAA/PHI applies to anything touching member data.

0. TL;DR — the architecture in one screen¶

Hybrid retrieval, in a fixed order of trust. Structured/deterministic lookup first (RxNorm/RxClass codes, NDC, the curated KB tables), then vector/semantic for unstructured passages (SPL label text, monographs, guidance), then knowledge-graph traversal (GraphRAG) for multi-hop clinical reasoning (drug→class→interaction→contraindication). A query router decides which legs run. [Gao 2024; Meibel; Medium-Amaro; IBM-GraphRAG]
Deterministic rule check wraps every model output — always. Best-in-class grounded RAG still hallucinates at single-digit rates; single-digit ≠ zero, and in a medication context "almost correct is absolutely wrong." The post-generation deterministic gate is non-negotiable and is the thing that keeps us in the wellness lane. [Self-RAG; Medium-Amaro; MDPI-Electronics]
Every clinical fact is traceable to a KB entry. Citation-enforced generation + self-reflection (Self-RAG) + corrective retrieval (CRAG) + a final deterministic verifier. [Self-RAG; CRAG; MDPI-AppliedSci; arXiv-FACTUM]
Scale levers: semantic caching (40%+ hit rate, ~10x cost cuts on hot paths), tiered latency budgets (p95 ≤ ~1.2 s to first useful token), on-device prefilter / server-side authority, and an embedding/index strategy sized for the corpus, not the hype. [MarkTechPost; Medium-RanaLatency; Redis; Nature-DrugSE]
Continuously evaluated: retrieval precision@k, context recall, faithfulness/groundedness, citation coverage — via RAGAS-style automated eval plus a clinician-reviewed gold set. [RAGAS; Maxim]
RAG vs fine-tune vs tool-use: RAG for the facts (they change; they must be cited), fine-tune for format/behavior/latency, tool-use for live structured systems (refill status, formulary). Most of the platform is RAG + light tool-use; fine-tuning is reserved for the structured-extraction model. [Medium-Paul; LabelYourData; ScalaCode]

1. RAG architecture patterns — naive vs advanced vs modular¶

1.1 The three paradigms (Gao et al. survey framing)¶

The canonical taxonomy comes from the Gao et al. survey, which frames RAG as three progressive paradigms over a shared retrieval → generation → augmentation tripartite foundation. [Gao 2024]

Naive RAG — embed the query, pull top-k by similarity, stuff into the prompt, generate. No query rewriting, no reranking, no filtering. Cheap and fast, but brittle: it retrieves chunks that look similar rather than chunks that are correct or connected, and it has no defense against low-quality context. [Gao 2024; devstark; Meilisearch-types]
Advanced RAG — adds pre-retrieval (query rewriting, entity extraction, query expansion) and post-retrieval (reranking, pruning, context compression) stages around the naive core. This is the minimum bar for anything clinical. [Gao 2024; devstark; ai-radar]
Modular RAG — a composable pipeline of swappable modules: a search module that can hit search engines / SQL / knowledge graphs; a memory module for prior Q&A; routing that decides the next action per query; and independently tunable retrievers. Each component can be debugged, A/B-tested, and optimized in isolation. [Gao 2024; devstark; IBM-techniques]

Rx360 position: the platform is Modular RAG. A single naive pipeline cannot serve five engines with different freshness, latency, and safety profiles. Modularity is also what lets us keep the member-facing wellness pipeline and the internal clinical pipeline as separate module graphs over a shared index + KB.

1.2 The hybrid retrieval order — structured FIRST, then vector, then graph¶

The defining design choice of our platform is a fixed order of trust for retrieval legs. A coordinator/router classifies the query and dispatches to the cheapest, most authoritative leg that can answer it, escalating only when needed. Multi-source RAG explicitly dispatches to different engines — vector search for text chunks, text-to-SQL for tabular data, graph engines for relationships. [Meibel; Medium-Amaro; arXiv-RAGRouter]

Leg 1 — Structured / deterministic lookup (highest trust, run first). When the query resolves to a known code or key, do not go to a probabilistic retriever. Resolve RxNorm RXCUI, RxClass, NDC, OpenFDA fields, or curated-KB rows by exact/deterministic lookup. In compliance-oriented environments — medical dosages being the textbook example — "almost correct is absolutely wrong," so deterministic look-ups are delegated for anything that maps cleanly to structured data, and RAG is reserved for genuinely unstructured passages. [Medium-Amaro; Meibel] Examples: "what class is atorvastatin in?" → RxClass lookup, no LLM in the retrieval path. "Does NDC X exist?" → table lookup. Medication-Input Intelligence's normalization step → deterministic RxNorm resolution, not semantic guess.

Leg 2 — Vector / semantic retrieval (for unstructured text). For free-text that has no clean key — SPL/DailyMed label prose, monograph narrative, FDA guidance, internal clinical notes — run dense + sparse hybrid search (see §3). This is where naive-RAG-style similarity actually belongs. [Gao 2024; Michael-Brenndoerfer]

Leg 3 — Knowledge-graph traversal / GraphRAG (for multi-hop reasoning). When the answer requires connected facts rather than similar text — "given this member's drug list, which pairs interact, via which mechanism, and which is class-contraindicated with their fall-risk profile?" — traverse the curated clinical knowledge graph. GraphRAG retrieves chunks that are actually connected (nodes + edges), whereas vector search retrieves chunks that merely look similar. It excels at multi-hop queries and global summarization; it is the right tool for the polypharmacy reasoning that sits at the center of our engines. [IBM-GraphRAG; TianPan; Meilisearch-graph]

1.3 When each leg is right (the routing table)¶

Query shape	Right leg	Why
Resolves to a code/key (RXCUI, NDC, class)	Structured lookup	Deterministic, auditable, zero hallucination surface. "Almost correct = wrong." [Medium-Amaro]
Single fact from prose ("what's the boxed warning text?")	Vector	No graph needed to fetch one passage; semantic match over label text. [Meilisearch-graph]
Multi-hop / relationship ("how is drug A connected to contraindication C via class B?")	GraphRAG	Connected retrieval + multi-hop reasoning is GraphRAG's home turf. [IBM-GraphRAG; TianPan]
Global/aggregate ("main interaction themes across this regimen")	GraphRAG (community summaries)	Dataset-wide pattern capture. [Meilisearch-graph]
Time-sensitive / just-changed data	Vector or live tool, not GraphRAG	GraphRAG shows a ~16.6% accuracy drop on time-sensitive queries and is expensive to re-index. [buildmvpfast/articsledge consensus]

Cost note on GraphRAG. Graph indexing is reported at 100–1000× the cost of vector indexing; query-time efficiency partially compensates, and LazyGraphRAG cuts indexing to ~0.1% of full GraphRAG. The 2025 practitioner consensus: layer graph capability on top of existing vector infrastructure — do not rebuild. That matches our reality: we already have the curated-KB RAG; GraphRAG is an addition for the reasoning-heavy queries, not a replacement. [salfati; buildmvpfast; TianPan]

Domain proof point. A 2026 Scientific Reports study on drug-side-effect retrieval with compact LLMs found a GraphRAG variant over a Neo4j knowledge graph hit 99.95–99.96% accuracy on drug→side-effect pairs at ~0.09 s latency, markedly faster than text-based RAG. This is direct evidence that graph-grounded retrieval over a curated clinical KG is both more accurate and faster for exactly the connected-fact queries our platform runs. [Nature-DrugSE]

2. Grounding + hallucination control¶

Thesis: retrieval reduces hallucination; it does not eliminate it. A grounded model still fabricates at single-digit rates. Therefore the model's output is advisory until a deterministic rule check confirms it against the KB. That gate — not the model — is what makes the system safe to ship in a wellness product.

2.1 Every clinical fact must be traceable to a KB entry¶

The non-negotiable contract: no clinical claim leaves the pipeline without a citation to a specific curated-KB record. Citation-enforced prompting transforms the LLM into a verifiable information extractor — the "StrictCitations" pattern requires the model to justify every claim by citing specific source identifiers, and citation-enforced prompting has been shown to reduce hallucinations specifically in medical AI RAG. [arXiv-FACTUM; MDPI-AppliedSci]

Implementation: chunks carry stable IDs; the generator must emit a source_id per asserted fact; a post-step verifies that each cited span actually supports the claim (attribution check — citations mapped to retrieved passages, with correct-mapping verification). [Maxim; arXiv-FACTUM]

2.2 Self-reflection and corrective retrieval¶

Two complementary 2024–2025 techniques harden the pipeline; they address different failure modes and stack:

Self-RAG — the model emits reflection tokens that decide whether to retrieve, and critique its own generation on fine-grained axes (is this supported? is it relevant?). Self-RAG 7B/13B produced only ~2% of predictions outside the provided passages, vs 15–20% for comparable non-reflective models — a ~10× reduction in out-of-context fabrication. It improves how the model reasons over evidence. [Self-RAG]
Corrective RAG (CRAG) — a lightweight evaluator grades retrieved docs as Correct / Incorrect / Ambiguous before generation and triggers correction (refine, or fall back to web/alternate source). It improves the quality of the evidence itself. [CRAG]

CRAG cleans the inputs; Self-RAG reflects on the output. We run both: CRAG-style grading on Leg 2/3 retrieval, Self-RAG-style critique on generation. [CRAG; Self-RAG]

2.3 Why a deterministic rule check must STILL wrap the model output¶

This is the heart of our posture and worth stating plainly:

Single-digit ≠ zero. Even with grounding + citations + reflection, residual fabrication is low but nonzero. At ~6M users × many queries/day, a 1% slip is tens of thousands of bad outputs. Probabilistic safety is not safety for medication content.
"Almost correct is absolutely wrong" in medication contexts — the literature is explicit that compliance-oriented domains like dosages cannot tolerate near-misses, which is exactly why deterministic look-ups (not RAG) own anything that maps to structured data. [Medium-Amaro]
Deterministic frameworks are the clinical state of the art. The D-RAG Evaluator uses mathematical linguistics to quantify alignment and reports an Unsupported Sentence Ratio (USR) so any ungrounded content is transparently flagged; clinical pipelines pair this with structured Evidence → Pathophysiology → Elimination → Final Double-Check reasoning protocols. [Q-search/MDPI-clinical guardrails]
It keeps us in the wellness lane. The deterministic gate is what enforces "reminders/confirmations only, never dosing/diagnosis." It can hard-stop or rewrite any output that drifts toward a prohibited claim, regardless of what the model generated.

The wrapping rail (what it actually does): - Re-validates every cited fact against the structured KB / RxNorm-RxClass — a row that the model paraphrased must still match the canonical row. - Runs the member-facing guardrail: blocks dosing/diagnostic language, enforces wellness phrasing, escalates anything ambiguous to a human/clinical queue. - Cross-references drug interactions against the structured interaction tables (deterministic), not the model's recollection — flagging is a table join, not a generation. [Maarga; Agentic-RAG-healthcare] - Emits USR / citation-coverage telemetry so a low-groundedness answer is suppressed rather than shown. [MDPI-clinical]

Layered grounding + citation enforcement + escalation is reported to cut hallucination risk 71–89%; semantic-caching + grounding stacks report ~90% hallucination decreases on hot paths. We treat those as reductions toward the deterministic floor, not as the floor itself. [SwiftFlutter; CustomGPT]

3. Vector databases, embeddings, chunking, indexing¶

3.1 Vector store choice for a 6M-user platform¶

There is no single winner; the choice is governed by corpus size, filtering needs, operational appetite, and co-location with app data. [vecstore; Medium-Anderson; Firecrawl]

Store	Sweet spot	Caveat for us
pgvector (Postgres)	Vectors alongside relational/app data; <10–100M vectors; HNSW matches/beats dedicated DBs at ~1M scale (Supabase benched pgvector HNSW beating Qdrant at 99% recall on equal compute)	Realistically slows past ~10–100M vectors. [vecstore; secondtalent]
Qdrant	Fastest filtered search (p50 < 5 ms at high recall), self-host, best-in-class metadata filtering	Operate it yourself. [vecstore; DataCamp]
Weaviate	Native hybrid search (BM25 + vector in one query) + managed cloud	— [Meilisearch; secondtalent]
Pinecone	Zero operational overhead, billions of vectors	Managed-cost, less control. [DataCamp; secondtalent]
Milvus/Zilliz	Billion-vector scale, most mature sharding/partitioning	Heavier to run. [secondtalent; DataCamp]

All purpose-built stores use HNSW (Hierarchical Navigable Small World): graph-based ANN whose search cost grows logarithmically, not linearly, with corpus size — which is why billion-vector scale is tractable. [Firecrawl; Medium-Anderson]

Rx360 recommendation. Our curated drug KB is not billions of vectors — the SPL/monograph/guidance corpus is on the order of low-millions of chunks. Start on pgvector + HNSW (vectors live next to the structured KB tables we already query in Leg 1, simplifying the hybrid join and the deterministic rail) and keep a migration path to Qdrant (filtered-search speed) or Milvus only if member-personalization vectors push us past the ~10–100M ceiling. Do not pay billion-scale operational cost for a low-millions corpus. [vecstore; secondtalent]

3.2 Embedding model choice¶

The MTEB leaderboard is the reference; current top tier: OpenAI text-embedding-3-large (~64.6), Cohere embed-v4 (~65.2), Voyage voyage-3-large (retrieval-focused leader), BGE-M3 (~63), with Gemini Embedding 2, Qwen3, Jina v5 now competitive or ahead of text-embedding-3 on many benchmarks. [Ailog; Modal; awesomeagents]

Two decisions that matter more than the leaderboard: - Dimensions are a cost lever. A 1,024-dim float32 vector = 4 KB; at 10M docs that's 40 GB of vector storage, and doubling dims doubles it. Pick the smallest dimension that holds recall (Matryoshka-style truncation where supported). [pecollective] - Domain fine-tuning beats generic. Generic MTEB scores don't always transfer; domain fine-tuning shows +10–30% gains. For a drug corpus (RxNorm names, SPL terminology, dosage strings) a domain-adapted embedding is worth it for the clinical pipeline. [Ailog/pecollective; Digital-Applied]

3.3 Hybrid (keyword + vector) search and reranking¶

Pure vector search misses exact-match cases (a specific NDC, an exact drug name, an acronym). Hybrid search combines sparse BM25 with dense vectors, fused with Reciprocal Rank Fusion (RRF) — RRF operates on ranks not scores, which sidesteps the score-incompatibility problem between BM25 and cosine. [Michael-Brenndoerfer; Digital-Applied]

Then rerank the fused candidate list with a cross-encoder (or instruction-following reranker like Voyage rerank-2.5, which lets you steer relevance with a natural-language instruction — useful for "prefer FDA-label sources"). Reranking over a hybrid candidate list consistently beats reranking BM25-only or dense-only. [Digital-Applied; arXiv-CheckThat]

3.4 Chunking¶

Chunking is the quietly decisive variable. For clinical text: structure-aware chunking (split on SPL LOINC sections, monograph headings) beats fixed-size windows; keep chunks small enough for precise citation but large enough to preserve a complete clinical statement; store rich metadata (drug, RXCUI, section, source, effective date) so Leg-1 filters and the deterministic rail can join on it. [abhs.in; Digital-Applied]

4. Scale — caching, latency, cost, on-device vs server-side¶

4.1 Semantic caching (the single biggest scale lever)¶

On a hot, repetitive query distribution (and senior-wellness med questions are repetitive), semantic caching is the dominant cost/latency win. A new query is embedded and compared to cached query embeddings; above a similarity threshold the cached response is returned instantly, skipping retrieval + generation. [MarkTechPost; dev.to-Sreeni]

Reported impact: - ~40% of repeated queries servable from cache; GPT Semantic Cache reports 61.6–68.8% cache-hit rates and up to 68.8% fewer API calls, with positive-hit accuracy >97%. [MarkTechPost; arXiv-GPTSemanticCache] - Production stacks report ~10× cost reduction and sub-second latency at millions of queries/day when caching is layered with grounding. [CustomGPT]

Critical safety carve-out for us: the cache must be keyed so that any answer carrying a clinical claim is re-validated by the deterministic rail even on a cache hit, or only post-rail, post-citation answers are cached. We never cache around the safety gate.

4.2 Latency budget (tiered by surface)¶

Users feel p95, not the average — tail latency is the design target. Practical budgets from production RAG: p95 ≤ ~1.2 s to first useful token for knowledge queries; p95 ≤ ~800 ms for short answers; TTFT p90 < 2 s before autoscale triggers. Simple RAG < 2 s, agentic RAG < 8 s. [Medium-RanaLatency; Redis]

Where the milliseconds go (and where we spend them): embedding → retrieval → reranking (cross-encoder ~50–200 ms/batch; p95 ~210–280 ms) → context build → generation → guardrails. Note: the guardrail/deterministic rail is a line item in the budget — it costs latency, and that cost is mandatory. [Medium-RanaLatency; dev.to-Neurolink]

Surface	Path	Budget
Member reminder/confirmation	cache → Leg-1 structured → rail	p95 ≤ 800 ms (mostly cache hits)
Member Q&A (wellness)	cache → hybrid vector → rerank → rail	p95 ≤ ~1.2 s to first token
Internal clinical reasoning (polypharmacy)	GraphRAG + CRAG + Self-RAG + rail	< 8 s acceptable (agentic)

4.3 On-device vs server-side¶

On-device: good for a fast prefilter / first-pass (the member's own med list, common reminders) and for PHI minimization — keep member-specific data local where possible. [arXiv-privacy]
Server-side (authoritative): the curated KB, the knowledge graph, the deterministic rail, and any clinical reasoning live server-side, on-premises / in-VPC to meet HIPAA — healthcare RAG requires strict on-prem processing, and a localization strategy that eliminates external data transmission by hosting the whole pipeline in secure infra. Hybrid PHI-handling approaches achieve >95% PHI detection while preserving utility. [Maarga; MDPI-Electronics; arXiv-privacy]

Rule: the deterministic safety rail and the source-of-truth KB are always server-side and authoritative; on-device is a latency/PHI optimization, never the safety boundary.

4.4 Cost at millions of queries/day¶

The cost model is dominated by (a) generation tokens, (b) embedding calls, (c) reranker compute, (d) vector-store ops. Caching attacks (a)+(b) directly. Routing attacks all four — Leg-1 structured lookups are nearly free vs a full generate-and-verify cycle, so the router that sends code-resolvable queries to deterministic lookup is also the biggest cost optimization, not just the biggest safety one. [MarkTechPost; Meibel]

5. Evaluation¶

You cannot ship clinical retrieval you cannot measure. We run automated RAG eval + a clinician-curated gold set, on every pipeline change.

5.1 Metrics (RAGAS framing)¶

RAGAS scores RAG without per-query human annotation across the retrieval and generation halves: [RAGAS; ragas-docs]

Dimension	Metric	What it catches
Retrieval	Context Precision	Are retrieved passages actually relevant (precision@k signal)?
Retrieval	Context Recall	Did we retrieve everything needed to answer?
Generation	Faithfulness / Response Groundedness	Does the answer strictly reflect retrieved context (penalize unsupported claims)?
Generation	Answer Relevancy / Factual Correctness	Is the answer on-point and correct?
Robustness	Noise Sensitivity	Does irrelevant context corrupt the answer?

Extended RAGAS metrics relevant to us: Context Entities Recall (did we retrieve the right drug entities?), Tool-Call Accuracy / F1 (for the tool-use legs), Agent Goal Accuracy (for agentic clinical reasoning). [ragas-metrics]

5.2 Beyond RAGAS — what clinical demands¶

Citation coverage / attribution: % of clinical sentences with a verified, correctly-mapped citation. Target → 100% for any clinical claim. [Maxim; arXiv-FACTUM]
Unsupported Sentence Ratio (USR): the deterministic-evaluator metric; tracked per release; ungrounded content must be flagged, not shown. [MDPI-clinical]
Rail-catch rate & escape rate: how often the deterministic rail catches a bad model output, and (the number that matters) how often a bad output escapes the rail. This is the real KPI for the wellness-lane guarantee.
Domain gold set: a pharmacist-reviewed question bank (interactions, classes, contraindications, label facts) scored on every change — the human anchor that automated LLM-judge metrics are validated against.

5.3 Eval cadence¶

RAGAS integrates with LangChain/LlamaIndex, so eval runs in CI on every retriever/embedding/prompt/index change; the clinician gold set runs pre-release and on a sampling basis in production. [RAGAS; Maxim]

6. RAG vs fine-tuning vs tool-use — when to use which¶

The decision hinges on data freshness, domain specificity, latency, cost, governance, and auditability. [Medium-Paul; Can-Demir]

	RAG	Fine-tuning	Tool-use
Changes	What the model knows at inference (injects retrieved context; weights unchanged)	Model behavior/weights (format, tone, taxonomy, latency)	What the model can do (call live systems/functions)
Best for	Dynamic, evolving, must-be-cited facts; regulated/high-stakes where auditability matters	Always-this-JSON, always-this-tone, classify-into-taxonomy; hard p95 latency targets	Live structured state: refill status, formulary, pharmacy hours
Weakness	Latency of retrieval; quality bounded by retrieval	Stale the moment facts change; costly to retrain; opaque	Needs reliable APIs; not a knowledge store

[Medium-Paul; LabelYourData; ScalaCode; dev.to-Ayinedjimi]

The practitioner consensus is hybrid: fine-tune for deterministic formatting/policy style, layer RAG for dynamic cited facts — ~60% of 2025–2026 production deployments use both. [Medium-Paul]

Rx360 mapping: - RAG (the spine): all clinical facts — interactions, classes, contraindications, label content. They change, and they must be cited to a KB entry; RAG gives us the auditability a regulated wellness product needs. [Medium-Paul] - Fine-tuning (narrow, deliberate): the Medication-Input Intelligence extractor (scan Rx → structured medication object) is a format/behavior task with a hard latency target and a fixed output schema — a textbook fine-tune. We do not fine-tune clinical facts into a model (they'd go stale and lose citations). [LabelYourData; ScalaCode] - Tool-use: refill anticipation and triage routing call live structured systems (refill status, formulary, scheduling) — tool-use, not retrieval. - Deterministic rail: sits outside all three; it is code, not a model, and it is the final arbiter. (See §2.3.)

7. Reference architecture (the platform retrieval system)¶

                          ┌─────────────────────────────────────┐
   member / engine query  │            QUERY ROUTER             │  (classify → cheapest authoritative leg)
            ──────────────▶│  intent + entity (RXCUI) detection  │
                          └───────┬───────────┬───────────┬─────┘
                                  │           │           │
              ┌───────────────────▼─┐   ┌─────▼──────┐  ┌─▼──────────────┐
   SEMANTIC   │ LEG 1: STRUCTURED   │   │ LEG 2:     │  │ LEG 3: GRAPHRAG│
   CACHE  ◀──▶│ deterministic lookup│   │ VECTOR     │  │ KG traversal   │
  (post-rail) │ RxNorm/RxClass/NDC/ │   │ hybrid     │  │ multi-hop /    │
              │ OpenFDA / KB tables │   │ BM25+dense │  │ community sum. │
              └─────────┬───────────┘   │ +RRF+rerank│  └─────┬──────────┘
                        │               │ (CRAG grade)│        │
                        │               └─────┬───────┘        │
                        └───────────┬─────────┴────────────────┘
                                    ▼
                        ┌───────────────────────┐
                        │  GENERATOR (cited)    │  StrictCitations + Self-RAG reflection
                        │  every fact → src_id  │
                        └───────────┬───────────┘
                                    ▼
                ┌───────────────────────────────────────┐
                │   DETERMINISTIC SAFETY RAIL (code)    │  ← ALWAYS. single-digit ≠ zero.
                │  • re-validate cited facts vs KB      │
                │  • interaction check = table join     │
                │  • wellness-lane guardrail (no dosing │
                │    /diagnosis); escalate ambiguous    │
                │  • USR / citation-coverage telemetry  │
                └───────────────────┬───────────────────┘
                                    ▼
                     member: reminder/confirmation only
                     internal: cited clinical output

Continuous eval (RAGAS + clinician gold set) wraps the whole pipeline; HIPAA/on-prem boundary encloses everything right of the router; PHI minimization / on-device prefilter sits left of it.

8. Decisions & open questions for the platform¶

Locked-in design positions: 1. Modular RAG, hybrid retrieval in trust order: structured → vector → GraphRAG, query-routed. [Gao 2024; Meibel] 2. Deterministic rail wraps every output, always — it is code, server-side, the final arbiter, and the wellness-lane enforcer. [Medium-Amaro; MDPI-clinical] 3. Citation to a KB entry is mandatory for any clinical claim. [arXiv-FACTUM; MDPI-AppliedSci] 4. Start vectors on pgvector+HNSW co-located with the KB; migrate only on real scale pressure. [vecstore; secondtalent] 5. Semantic caching post-rail as the primary scale lever; never cache around the gate. [MarkTechPost] 6. GraphRAG is an addition over existing vector infra (LazyGraphRAG for index cost), not a rebuild. [salfati; TianPan] 7. RAG for facts, fine-tune only the extractor, tool-use for live systems. [Medium-Paul]

Open questions: - Build the clinical KG on Neo4j (matches the cited drug-SE GraphRAG result) vs a property graph in pgvector-adjacent Postgres? Decide on multi-hop query volume. [Nature-DrugSE] - Domain-fine-tuned embedding (drug corpus) vs top MTEB generalist — run the +10–30% domain-gain claim against our own gold set before committing. [Ailog] - On-device prefilter scope — how much member med-list reasoning can safely run client-side under PHI minimization without crossing into clinical inference? [arXiv-privacy] - Reranker: instruction-following (Voyage rerank-2.5, "prefer FDA-label") vs plain cross-encoder — measure precision@k delta vs the +1–2 s latency. [Digital-Applied]

Sources¶

RAG paradigms & architecture - Gao et al., "Retrieval-Augmented Generation for Large Language Models: A Survey" (arXiv:2312.10997) - DevStark — Naive, Advanced, and Modular RAG architectures - AOE AI Radar — Advanced & Modular RAG patterns - IBM — RAG Techniques - Meilisearch — 14 types of RAG

GraphRAG & hybrid/structured retrieval - IBM — What is GraphRAG? - Meilisearch — GraphRAG vs Vector RAG - TianPan — GraphRAG vs Vector RAG architecture decision - Salfati Group — Graph RAG Guide 2025 - BuildMVPFast — GraphRAG vs Vector RAG 2026 - Medium (Amaro) — Retrieval for structured data: precision-first alternative to vector-only RAG - Meibel — Structure Augmented Generation - arXiv — Lightweight Query Routing for Adaptive RAG (RAGRouter-Bench) - Nature Scientific Reports — RAG-based architectures for drug side effect retrieval using compact LLMs

Grounding & hallucination control - Asai et al. — Self-RAG: Learning to Retrieve, Generate and Critique through Self-Reflection (arXiv:2310.11511) · project site - Kore.ai — Corrective RAG (CRAG) - arXiv — FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG - MDPI Applied Sciences — Reducing Hallucinations in Medical AI Through Citation-Enforced Prompting in RAG - MDPI Electronics — Evaluating RAG Variants for Clinical Decision Support: Hallucination Mitigation & Secure On-Premises Deployment - SwiftFlutter — Reducing AI Hallucinations: 12 Guardrails - Maarga Systems — Agentic RAG in Healthcare

Vector DBs, embeddings, hybrid search - VecStore — pgvector vs Pinecone vs Qdrant vs Weaviate - SecondTalent — Pinecone vs Weaviate vs Qdrant vs pgvector 2026 - Firecrawl — Best Vector Databases 2026 - DataCamp — Top Vector Databases 2026 - Ailog — Best Embedding Models 2025: MTEB Scores - Modal — Top embedding models on the MTEB leaderboard - PE Collective — Embedding Model Specs 2026 (dimensions, price, MTEB) - Michael Brenndoerfer — Hybrid Search: BM25 + Dense Retrieval (RRF) - Digital Applied — Hybrid Search: BM25, Vector & Reranking 2026 - arXiv — Deep Retrieval at CheckThat! 2025 (hybrid retrieval + reranking)

Scale, caching, latency, PHI - MarkTechPost — Reduce Cost & Latency with Semantic LLM Caching - arXiv — GPT Semantic Cache (arXiv:2411.05276) - CustomGPT — From Prototype to Production: Scaling RAG API Applications - Medium (Rana) — RAG Latency Budgets: Where to Spend Your Milliseconds - Redis — RAG at Scale: Production AI Systems 2026 - Abhishek Gautam — RAG in Production 2026: Chunking, Embedding Costs at Scale - arXiv — Privacy Challenges and Solutions in RAG-enhanced LLMs for Healthcare

Evaluation - Es et al. — RAGAS: Automated Evaluation of Retrieval Augmented Generation (arXiv:2309.15217) - RAGAS — List of available metrics - Maxim — Complete Guide to RAG Evaluation 2025

RAG vs fine-tuning vs tool-use - Medium (Paul) — Fine-Tuning vs RAG: A Data-Driven Guide - Label Your Data — RAG vs Fine-Tuning 2025 - ScalaCode — RAG vs Fine-Tuning 2026 Decision Guide - DEV (Ayinedjimi) — Fine-tuning vs RAG: a decision framework