FFaurenis
Data Room
07

AI architecture

Executive summary

Every model call in Faurenis goes through one routing layer: endpoints map to capability tiers, tiers map to models, and a retry engine falls back one tier down rather than failing. Hosted Gemini models on Vertex AI in the EU serve production traffic; a self-hosted estate of open-weight clinical models, embeddings and entailment on GPUs is rolling out beside them, with Vertex as its fallback. Around every call sits a layered guardrail stack — consent, budget, PII stripping, schema validation, output rules, deterministic red-flag checks and a decision record — so no model output reaches a clinician unchecked.

Model fleet

  • Gemini 3 Flash
    Where it runs
    Vertex AI, EU jurisdictional endpoint
    Role
    Pro tier: notes, extraction, interpretation, Copilot clinical and concierge lanes
    Status
    Production
  • Gemini 3 Flash-Lite
    Where it runs
    Vertex AI, EU
    Role
    Flash and lite tiers: OCR, classification, HUD extraction, memory, documents, intent and routing classifiers
    Status
    Production
  • Gemini 3 Live
    Where it runs
    Vertex AI, EU
    Role
    Speech-to-speech sessions with tools and camera frames
    Status
    Pilot
  • Qwen3-8B (AWQ) + clinical adapters
    Where it runs
    Self-hosted, Cloud Run GPU (NVIDIA L4), EU
    Role
    Sovereign path for notes, critic, extraction and memory
    Status
    Rolling out
  • MedGemma 27B
    Where it runs
    Self-hosted, EU
    Role
    High-risk clinical critic — secondary reader only
    Status
    Rolling out
  • MedGemma 4B + radiograph adapter
    Where it runs
    Self-hosted, EU
    Role
    Imaging region-of-interest verifier
    Status
    Rolling out
  • BGE-M3
    Where it runs
    Self-hosted (text embeddings inference)
    Role
    1024-dimensional dense retrieval for the evidence base
    Status
    Rolling out
  • DeBERTa-v3-large MedNLI
    Where it runs
    Self-hosted (text embeddings inference)
    Role
    Entailment: does this passage support this claim
    Status
    Rolling out
  • Soniox, Google Cloud Speech, ElevenLabs
    Where it runs
    Provider EU endpoints
    Role
    Recognition and synthesis (Voice OS)
    Status
    Production

Every model the platform has ever billed for is registered in server/config/modelRegistry.ts with tier, status, pricing, context window and where it is served. Entries are permanent: usage rows are re-priced from the registry, so deleting a retired model would silently re-price history. The module refuses to load if a tier or an override points at an unregistered or retired model.

Routing: endpoints, tiers, fallback

Hosted endpoints declare a tier, never a model. The map below is generated from ENDPOINT_CONFIG, grouped by endpoint family; the per-endpoint table is in the Data Room edition. Pro is served by Gemini 3 Flash; flash and lite by Gemini 3 Flash-Lite.

  • chairside
    Tier
    pro
    Endpoints
    1
    Model family
    Gemini 3 Flash
  • copilot
    Tier
    pro
    Endpoints
    2
    Model family
    Gemini 3 Flash
  • decision
    Tier
    pro
    Endpoints
    1
    Model family
    Gemini 3 Flash
  • document
    Tier
    flash
    Endpoints
    2
    Model family
    Gemini 3 Flash-Lite
  • document
    Tier
    pro
    Endpoints
    2
    Model family
    Gemini 3 Flash
  • hud
    Tier
    flash
    Endpoints
    1
    Model family
    Gemini 3 Flash-Lite
  • hud
    Tier
    pro
    Endpoints
    1
    Model family
    Gemini 3 Flash
  • patient
    Tier
    pro
    Endpoints
    1
    Model family
    Gemini 3 Flash
  • scribe
    Tier
    pro
    Endpoints
    2
    Model family
    Gemini 3 Flash
  • twin
    Tier
    pro
    Endpoints
    4
    Model family
    Gemini 3 Flash
  • Temperature 0.1 for extraction, 0.2 for generation.
  • Copilot’s typed clinical, report, spoken clinical and concierge lanes use per-lane overrides onto the newest Gemini 3 Flash generation; overrides must be registered, callable pro-tier models.
  • Fallback runs pro → flash only. The lite tier is never a fallback target: a classifier answering a clinical prompt is worse than a refusal.
  • The retry engine makes three jittered attempts, then drops one tier. Each response is tagged with the model that actually served it, so cost and the decision record are exact.
  • Reasoning is set per model dialect (token budget or level), because thinking tokens bill at the output rate and an unbounded reasoning pass can consume the whole output budget.

Self-hosted inference estate

The clinical LLM worker is a gatekeeper service in front of vLLM on Cloud Run with an NVIDIA L4 GPU. It serves Qwen3-8B (AWQ-quantised, 32k context, prefix caching) with LoRA adapters for SOAP structuring, note critique, clinical information extraction and memory extraction, plus two sidecars for embeddings (BGE-M3) and entailment (DeBERTa MedNLI).

  • Model router
    tier · override · opt-in
  • Circuit breaker
    3 fails / 30 s
  • Vertex AI (EU)
    fallback and primary
  • Worker gatekeeper
    signed release · output hash
  • vLLM
    Qwen3-8B + LoRAs
  • Embeddings · NLI
    BGE-M3 · DeBERTa MedNLI
  • Model bucket (read-only)
    weights · adapters · manifest
  • MedGemma
    critic · ROI verifier
Connections (7)
  • Model router → Circuit breaker
  • Circuit breaker → Vertex AI (EU) · open
  • Model router → Worker gatekeeper
  • Worker gatekeeper → vLLM
  • vLLM → Embeddings · NLI
  • Worker gatekeeper → Model bucket (read-only)
  • vLLM → MedGemma
The server routes eligible endpoints to the worker first and falls back to Vertex. The worker has no database credentials and no egress.
  • Opt-in routing
    Behaviour
    Endpoints reach the worker only when listed; the wildcard deliberately skips the Copilot stream and spoken clinical answers, which must be named explicitly
  • Isolation
    Behaviour
    Internal ingress only; refuses to start if database credentials are present; read-only model mount; no egress
  • Release integrity
    Behaviour
    Ed25519-signed release manifest with three approvals (ML, quality, clinical); a SHA-256 hash of every output
  • Circuit breaker
    Behaviour
    Opens after 3 failures within 30 s; 30 s cool-down; half-open probes every 10 s with a 2.5 s timeout; pre-warms after 10 minutes idle
  • Residency
    Behaviour
    The worker region must be on the approved EU list
  • Fallback
    Behaviour
    On worker failure the call falls back to Vertex for the same tier

Adaptive reasoning

On the self-hosted path, a pure function decides per call whether the model reasons before answering. It is ordered so the safest answer wins:

  1. An explicit caller override.
  2. Critic and HUD-suggestion calls always reason.
  3. Risk signals in Croatian or English force reasoning: anticoagulants and INR, antiresorptives and MRONJ, anaphylaxis and allergy, uncontrolled systemic disease, pregnancy and paediatric dosing, contraindications, complex surgery.
  4. Long or wide cases force reasoning: a note of 2,400 characters or more, or five or more distinct teeth.
  5. Otherwise, answer directly.

The guardrail stack

  • Consent · compliance
    fails closed (451)
  • Budget · cost cap
    per clinic (429)
  • PII stripping
    free text · OIB checksum
  • Model call
    schema · low temperature
  • Decision record
    hashes · cost · consent
  • Critic (optional)
    withhold only
  • Red-flag interceptor
    deterministic
  • Output validation
    schema · rules · floors
Connections (7)
  • Consent · compliance → Budget · cost cap
  • Budget · cost cap → PII stripping
  • PII stripping → Model call
  • Model call → Output validation
  • Output validation → Red-flag interceptor
  • Red-flag interceptor → Critic (optional)
  • Critic (optional) → Decision record
Layers around every hosted or self-hosted call. Each layer can refuse; none can be skipped by an endpoint handler.
  • Consent
    Mechanism
    AI-processing consent attested by the signed-in user in their clinic and checked against the current form version, at the API edge and again in async workers; missing, revoked, stale or uncertain consent refuses the call
  • Budget
    Mechanism
    Plan token allowance and a monthly USD cap per clinic; overrides can only lower the cap
  • PII stripping
    Mechanism
    Deep scrub of free text before it leaves the server, including Croatian OIB numbers validated by checksum (ISO 7064 MOD 11,10); untrusted values are fenced in prompts
  • Output validation
    Mechanism
    Zod schema parse, then rules: forbidden certainty phrases, a dosage guard, confidence floors per surface (0.6–0.7), PHI leak, unsupported claims, prompt-injection echo. Any hit sets clinician review required
  • Red-flag interceptor
    Mechanism
    Non-model hard stops for emergencies and lethal contraindications
  • Critics
    Mechanism
    Optional second readers (Scribe critic, high-risk clinical critic) that can withhold, never rewrite
  • Decision record
    Mechanism
    One row per call: consent id, model actually used, prompt version, prompt and response hashes (never raw PHI), tokens, cost, latency, whether PII was detected
  • Provider safety settings
    Mechanism
    Harassment, hate and sexual content blocked at medium and above; dangerous content blocked at high

Retrieval and grounding

Grounded generation draws on 9 open sources. The full pipeline is specified in the Evidence base module; the fusion step is:

RRF(d) = Σ_{r ∈ {lexical, dense}}  1 / (k + rank_r(d)),   k = 60
Reciprocal Rank Fusion over lexical and dense rankings (ReciprocalRankFusion.ts).

Speech stack

  • Real-time recognition
    Primary
    Soniox real-time, EU endpoint
    Fallback
    Google Cloud Speech (long-form and command models), engine-supervisor circuit
  • Asynchronous recognition
    Primary
    Soniox async, EU
    Fallback
    —
  • Speech synthesis
    Primary
    Soniox TTS
    Fallback
    ElevenLabs; browser speech for fixed phrase keys only
  • Speech-to-speech
    Primary
    Gemini 3 Live on Vertex AI (EU or jurisdictional endpoint)
    Fallback
    No single-region fallback
SOURCE OF TRUTH
  • server/config/modelRegistry.ts
  • server/config/modelConfig.ts
  • server/services/retryableGeminiCall.ts
  • server/services/clinicalLlmWorkerClient.ts
  • server/services/clinicalLlmComplexityRouter.ts
  • server/services/clinicalLlmCircuitBreaker.ts
  • server/services/aiOutputValidator.ts
  • server/services/aiOutputRules.ts
  • services/clinical-llm-inference/
Every figure in this specification is generated from the repository and checked in CI.