Skip to content

Rx360 AI Foundation — Architecture Spec for Engineering (v1)

Date: 2026-06-04 · From: Gene (Pharmacy Ops) · For: Faraz (eng lead) + Emil (infra) What this is: the platform every AI engine plugs into — buildable, with the four 2026–27 SOTA upgrades folded in. Integrates the AI Foundation KB (AI_Foundation/KB/_KB_SYNTHESIS + _SOTA_VALIDATION_2026) and is the foundation under the Medication Intelligence Engine phased plan (the first engine).

Posture: consumer-wellness lane (not a medical device). Member-facing = reminders/confirmations/insights; the intelligence is internal IP. The clinical-line is a hard, deterministic boundary. HIPAA/PHI applies.

0 · TL;DR for engineering

  • The brain is probabilistic; the boundary is deterministic + human-gated. Build the deterministic clinical-line rail before any model can emit to a member.
  • One source of truth (lakehouse), many indexes (graph + vector), one door (MCP). Don't hand-maintain competing stores.
  • Build foundation-first: shared contract → knowledge layer → de-ID'd event backbone → deterministic rail → retrieval → guardrails → serving → eval/MLOps.
  • Engines don't reinvent the brain — Balance Meter, Smart Scheduling, Med-Input Intelligence, Refill, Triage all plug into the same layers via MCP.

1 · The layered architecture (with the four upgrades folded in)

ENGINES (plug in via MCP): Balance Meter · Smart Scheduling · Med-Input · Refill · Triage
        │  ask(grounded)                                   │  act(gated)
        ▼                                                   ▼
L2  CONTEXT ENGINEERING / RETRIEVAL            L4  GUARDIAN RAILS
    • agentic retrieval (multi-step)               • probabilistic "speed bumps"
    • hybrid: structured→vector→GraphRAG             (NeMo orch · Llama Guard · validators)
    • per-member MEMORY store          ─────►       • DETERMINISTIC CLINICAL-LINE RAIL (blast door)
    • assembles the context window                   fail-closed → pharmacist (HCP) queue
        ▲                                                   │ else emit {reminder|confirmation|insight}
        │                                                   ▼
L1  KNOWLEDGE LAYER (exposed as an MCP server)     L3  REASONING (compound system)
    • curated KB (Iceberg tables, knowledge-as-data)   • model(s) + orchestration; small-model+router
    • Knowledge Graph (property graph) — a PROJECTION  • a weak model + strong orchestration > the reverse
    • vector index — a PROJECTION
        ▲ provenance envelope on every response
        │
L0  DATA & GOVERNANCE BACKBONE  +  L5 SERVING/SCALE
    • LAKEHOUSE (Apache Iceberg) = single source of truth
    • event stream (Kafka-class) + schema registry · FHIR R4/US-Core interchange
    • de-IDENTIFICATION GATEWAY splits operational-PHI ⟂ learning planes
    • on-device (TinyML/offline) → BAA private-cloud → cloud · cache-as-cost · offline fallback
L6  EVALS & MLOps (cross-cutting): clinician gold set = the contract · learn→validate→sign-off→gated deploy · drift/holdout/MRT · GMLP+PCCP

2 · The component stack (build-vs-buy)

Layer Recommended Build vs buy Why
L0 lakehouse Apache Iceberg (Snowflake/Trino/Spark/Athena-queryable) buy/OSS single source of truth; governance once, reused
L0 event stream Kafka-class log + schema registry buy/OSS system-of-record; de-ID split; replay
L0 de-ID gateway Presidio / Comprehend Medical (18 Safe-Harbor identifiers) OSS + config operational⟂learning plane split
L0 interchange FHIR R4 / US Core (MedicationRequest, Provenance, Consent) standard interoperable; the shared contract
L1 KB curated rows as Iceberg tables (versioned, author≠approver, clinical-line-flagged) build (schema) knowledge-as-data, no-code-release updates
L1 graph Neo4j or Amazon Neptune (property graph), kept in sync from L0 via the event stream buy multi-hop reasoning; a projection, not a 2nd truth
L1 vector pgvector → Qdrant/Milvus; HNSW OSS semantic recall; a projection
L1 access MCP server (knowledge layer + tools exposed via Model Context Protocol) build on OSS SDK the one door; engines plug in via the standard
L2 retrieval agentic RAG loop + hybrid retriever (BM25+vector+RRF+rerank) + GraphRAG (LazyGraphRAG for cost) build +~42% faithfulness vs single-step
L2 memory per-member memory store (short-term session + long-term longitudinal) build the longitudinal-data moat, made architectural
L3 model small specialized models + a router; self-hosted Llama-class for PHI paths; API behind redaction otherwise mix cost/latency; PHI control
L4 rails NeMo Guardrails (orchestrator) + Llama Guard 3 + Guardrails-AI validators + Rx360 deterministic clinical-line rail buy + build the rail defense-in-depth, heterogeneous failure
L5 serving vLLM continuous batching · self-hosted gateway/router · semantic + prefix cache OSS makes 6M users affordable
L5 edge TinyML on-device for time-critical/PHI path; offline deterministic fallback build latency + privacy + reliability
L6 eval/MLOps RAGAS + clinician gold set + rail-escape-rate; feature store; holdout/MRT/drift; canary+rollback build on OSS eval-driven; GMLP/PCCP-ready

3 · The interface contracts (what to code to)

  1. MCP server — the knowledge layer. Tools/resources exposed: kb.lookup(drug|class|rule), kb.graph_query(...), kb.semantic_search(...), each returning a provenance envelope {value, source, kb_entry_id, version, clinical_line_flag}. Every engine calls these — no bespoke per-engine DB access.
  2. The shared medication object — FHIR MedicationRequest.dosageInstruction + Provenance (self ×0.5 → pharmacist ×1.0, first-class queryable) + confidence + flags. (Already Phase 0 of the Med-Intelligence Engine plan — this is its canonical home.)
  3. The retrieval contract — retrieve(query, member_ctx) → {context[], citations[], provenance[]}; agentic loop may call it iteratively until "sufficient."
  4. The rail contract — gate(candidate_output, target=member|pharmacist) → {emit | block+route}; deterministic, fail-closed; runs on the final rendered string after all model-based rails.
  5. The event schema — versioned domain.entity.action.vN on the Kafka log; de-identified at the boundary before the learning store.

4 · Upgrade detail — context engineering + agentic retrieval + per-member memory

  • Context engineering is the discipline: L2's job is to assemble exactly the right grounded context per decision — not just "retrieve top-k." Treat the context window as the product.
  • Agentic retrieval: the engine's retrieval is a loop — retrieve → judge sufficiency → find gaps → pull more (graph/vector/structured) → synthesize → adapt — not a single shot. Bound the loop (max hops, latency budget); every hop's source is provenance-tagged.
  • Per-member memory: a member store with short-term (session) + long-term (longitudinal: confirmed-dose rhythm, baselines, preferences, prior corrections). This is the substrate the engines personalize from — and the asset competitors can't backfill. PHI rules apply (operational plane; de-ID before learning).

5 · Upgrade detail — the deterministic clinical-line rail (the safety boundary)

Gate order (member-facing path): input rails (PHI redaction in-network · injection screen · topical gate) → retrieval grounding → model → model-based output rails (Llama Guard + validators + atomic-claim grounding) → ► DETERMINISTIC CLINICAL-LINE RAIL ◄ → execution rails (least-privilege tools; HITL in code). - The deterministic rail = non-ML lexicon/pattern + allow-listed output schema (reminder|confirmation|insight). Dosing? diagnosis? disease/clinical-mgmt claim? schema violation? timeout/error? → fail-closed: block + route to pharmacist queue. Default state = "do not emit." - Do NOT stream member-facing clinical-adjacent output (buffer fully, gate before first token). HITL/approval enforced in code outside the agent's reach. OCR'd Rx text = untrusted (injection vector). Target metric: clinical-line leakage = 0.

6 · Upgrade detail — storage (the question to get right)

The lakehouse is the truth; the graph and vector store are projections; MCP is the door. - KB + canonical data → Iceberg lakehouse (versioned, governed once). The curated-KB rows are Iceberg tables. - Graph → a property graph materialized from L0, synced via the event stream — never a hand-maintained second truth. - Vector index → embeddings of the same canonical data. - This avoids the #1 failure mode: two stores that drift out of sync.

7 · Serving / scale (6M users)

Three-tier by time-criticality × data-sensitivity: on-device (TinyML, offline scheduling, on-device Rx OCR) → BAA private-cloud (escalations) → cloud (training, RAG/KB, batch). Routing = sensitivity > complexity > confidence-cascade. Caching is a financial requirement (semantic + prefix cache + vLLM batching + self-host gateway). Deterministic offline fallback + quality circuit-breakers. OpenTelemetry decision-context logging (KB version + rail verdict).

8 · Evals & MLOps (the upgrade that keeps it honest)

  • Eval-driven development: the clinician gold set is the contract. Every engine change is gated by an eval suite (RAGAS retrieval metrics + groundedness/citation-coverage + rail escape-rate + clinical-correctness on the gold set).
  • Lifecycle: learn → validate (held-out temporal split + subgroup/fairness) → pharmacist/clinical sign-off (non-skippable, event-logged) → canary deploy → monitor (drift: PSI/KS) → auto-rollback. A vaulted never-touched holdout for honest lift.
  • Regulated-ready: GMLP 10 principles + a PCCP so wellness→SaMD needs no re-architecture.

9 · Build order (sprint-scopable, foundation-first)

  1. Sprint set A — the spine: the FHIR/medication-object contract + the curated-KB Iceberg schema + the event stream + de-ID gateway. Nothing emits or learns without this.
  2. Sprint set B — the rail: the deterministic clinical-line rail (lexicon + schema + fail-closed) + the pharmacist queue. Built before any model emits to a member.
  3. Sprint set C — knowledge + retrieval: the MCP server over the KB + graph(projection) + vector(projection); the hybrid retriever; then the agentic loop + memory.
  4. Sprint set D — model + probabilistic guardrails: the router + NeMo/Llama-Guard/validators around the model.
  5. Sprint set E — serving/scale: on-device-first, cache-heavy, offline fallback, observability.
  6. Sprint set F — eval/MLOps: the gold set + the gated learning loop.
  7. Then: engines plug in (Med-Input Intelligence first — it's specced). Rule: ground before you generate · rail before you emit · de-identify before you learn · cache before you scale · eval before you ship.

10 · Decisions for Faraz / the room

  1. Build vs buy: graph DB (Neo4j vs Neptune) · LLM (self-host Llama-class vs API-behind-redaction) · the orchestrator (NeMo vs lighter).
  2. The on-device ⟂ cloud line per engine + the minimum offline-deterministic behavior.
  3. Lakehouse choice (Iceberg on which engine — Snowflake/Databricks/Trino) + the schema-registry.
  4. Platform ownership — this foundation needs an owner (platform/infra lead) so the five engines don't fragment it. Who?
  5. Sequencing — stand up Sprint sets A–B (contract + rail) before Med-Input Engine Phase 1? (rec: yes — they gate everything.)

Companion docs: AI_Foundation/KB/ (the research + synthesis + SOTA validation) · Medication_Intelligence_Engine_PHASED_PLAN_2026-06-04 (the first engine) · Medication_Input_Engine_AUDIT/_Explainer (the runtime) · the existing Rx360_Medication_Autocomplete_API_Spec_v1 (the autocomplete API that sits on this foundation).