Medical Intake Walkthrough
What You’ll Build
Section titled “What You’ll Build”A medical intake system where patient forms are submitted, an LLM extracts conditions and medications, a clinician reviews the extraction, and a separate LLM proposes triage actions. The system enforces two boundaries: LLM-generated facts never become accepted without explicit clinician approval, and LLM-generated actions never become executable without domain rules admitting them.
This walkthrough covers the full JacqOS pipeline with a focus on LLM capture, proposal containment, and healthcare review gates:
- Observations arrive as JSON events (intake submitted, LLM extraction result, clinician review, referral directory, triage proposal, intake finalized)
- Mappers extract semantic atoms, including PHI scope, source attribution, symptom age, and multi-valued condition lists
- Rules derive candidate facts from LLM output, then promote them to accepted facts only after role-bound clinician approval
- Rules stage triage proposals under
proposal.*, then derive referral, escalation, or blocked decisions from policy facts - Invariants enforce that finalization requires review and that unsafe lifecycle transitions produce replayable counterexamples
- Intents derive outbound actions (request extraction, notify a clinician, route a referral, escalate a safety issue)
- Recorded replay captures LLM interactions for deterministic re-execution
- Fixtures prove the system handles approval, re-extraction contradictions, clinician rejection, referral routing, unsafe advice, wrong authority, stale symptoms, and missing review
Project Structure
Section titled “Project Structure”jacqos-medical-intake/ jacqos.toml ontology/ schema.dh # 26 relations rules.dh # Relay gates, triage decisions, invariants intents.dh # Intent derivation mappings/ inbound.rhai # 6 observation kinds prompts/ extraction-system.md # LLM system prompt schemas/ intake-extraction.json # Structured output schema fixtures/ happy-path.jsonl # Approved intake contradiction-path.jsonl # Re-extraction retracts old candidates llm-disagreement-path.jsonl # Clinician rejects LLM output referral-routing-path.jsonl # Reviewed referral proposal becomes an intent unsafe-advice-path.jsonl # Unsafe self-care proposal is blocked/escalated wrong-authority-path.jsonl # Non-clinician approval is rejected stale-symptoms-path.jsonl # Stale symptoms block routing missing-review-finalization-path.jsonl # Invariant counterexample generated/ verification/ # Exported proof and provider-capture evidenceStep 1: Configure the App
Section titled “Step 1: Configure the App”jacqos.toml declares the app identity, capabilities, and resources:
app_id = "jacqos-medical-intake"app_version = "0.1.0"
[paths]ontology = ["ontology/*.dh"]mappings = ["mappings/*.rhai"]prompts = ["prompts/*.md"]schemas = ["schemas/*.json"]fixtures = ["fixtures/*.jsonl"]helpers = ["helpers/*.rhai"]
[capabilities]http_clients = ["notify_api"]models = ["extraction_model"]timers = falseblob_store = true
[capabilities.intents]"intent.request_extraction" = { capability = "llm.complete", resource = "extraction_model", result_kind = "llm.extraction_result" }"intent.notify_clinician" = { capability = "http.fetch", resource = "notify_api" }"intent.route_referral" = { capability = "http.fetch", resource = "notify_api" }"intent.escalate_safety" = { capability = "http.fetch", resource = "notify_api" }
[resources.http.notify_api]base_url = "https://notify.example.invalid"credential_ref = "NOTIFY_API_TOKEN"replay = "record"
[resources.model.extraction_model]provider = "openai"model = "gpt-4o-mini"credential_ref = "OPENAI_API_KEY"schema = "schemas/intake-extraction.json"replay = "record"Four intent capabilities drive the intake workflow. intent.request_extraction triggers an LLM call against the extraction_model resource, which uses the schemas/intake-extraction.json schema for structured output. intent.notify_clinician sends an HTTP request to the notify_api resource when extraction results are ready for review. intent.route_referral and intent.escalate_safety use the same HTTP resource, but only after the ontology derives admitted triage decisions.
Both resources set replay = "record", so LLM and HTTP interactions are captured as effect envelopes for deterministic replay during fixture verification.
The blob_store = true setting enables storage for large raw payloads — useful when intake forms include scanned documents or lengthy narratives.
Step 2: Declare Relations
Section titled “Step 2: Declare Relations”ontology/schema.dh declares every relation the ontology uses:
relation intake_submitted(intake_id: text, patient_name: text, dob: text, raw_text: text)relation intake_source_ref(intake_id: text, source_ref: text)relation intake_phi_scope(intake_id: text, phi_scope: text)relation intake_symptom_age_days(intake_id: text, age_days: int)relation candidate.conditions(intake_id: text, condition: text, extraction_seq: text)relation candidate.medications(intake_id: text, medication: text, extraction_seq: text)relation extraction_confidence(intake_id: text, confidence: text, extraction_seq: text)relation clinician_approved(intake_id: text)relation clinician_rejected(intake_id: text, corrections: text)relation review_wrong_authority(intake_id: text)relation accepted_conditions(intake_id: text, condition: text)relation accepted_medications(intake_id: text, medication: text)relation referral.route_allowed(route: text, min_role: text)relation proposal.triage_action(intake_id: text, action: text, route: text, advice: text, triage_seq: text)relation proposal.triage_urgency(intake_id: text, urgency: text, triage_seq: text)relation triage.current_seq(intake_id: text, triage_seq: text)relation triage.decision.referral(intake_id: text, route: text)relation triage.decision.escalate(intake_id: text, route: text, reason: text)relation triage.decision.blocked(intake_id: text, reason: text)relation intake_finalized(intake_id: text)relation intake_status(intake_id: text, status: text)relation intake.invariant.finalized_without_review(intake_id: text)relation intent.request_extraction(intake_id: text, raw_text: text)relation intent.notify_clinician(intake_id: text, patient_name: text)relation intent.route_referral(intake_id: text, route: text)relation intent.escalate_safety(intake_id: text, route: text, reason: text)The relations fall into six groups:
- Observation projections:
intake_submitted,intake_source_ref,intake_phi_scope,intake_symptom_age_days,extraction_confidence,clinician_approved,clinician_rejected,review_wrong_authority,intake_finalized— direct projections from atoms - Candidate evidence:
candidate.conditions,candidate.medications— LLM-derived facts that use thecandidate.prefix to signal they require acceptance before becoming authoritative - Accepted facts:
accepted_conditions,accepted_medications— promoted from candidates only after clinician approval - Proposal evidence:
proposal.triage_action,proposal.triage_urgency— LLM-derived action suggestions that must be admitted by domain rules - Triage decisions:
referral.route_allowed,triage.current_seq,triage.decision.referral,triage.decision.escalate,triage.decision.blocked - Derived state and intents:
intake_status,intake.invariant.finalized_without_review,intent.request_extraction,intent.notify_clinician,intent.route_referral,intent.escalate_safety
The candidate. and proposal. prefixes are the key design pattern. JacqOS enforces a mandatory rejection rule: relay-marked LLM output cannot become accepted fact or executable intent unless the ontology routes it through the appropriate acceptance or decision relation.
Step 3: Map Observations to Atoms
Section titled “Step 3: Map Observations to Atoms”mappings/inbound.rhai handles six observation kinds and declares the relay
contract for both fallible sensors and fallible deciders:
fn mapper_contract() { #{ requires_relay: [ #{ observation_class: "llm.extraction_result", predicate_prefixes: ["extraction.condition", "extraction.medication"], relay_namespace: "candidate", }, #{ observation_class: "llm.triage_proposal", predicate_prefixes: ["triage.action", "triage.route", "triage.advice"], relay_namespace: "proposal", } ], }}
fn map_observation(obs) { let body = parse_json(obs.payload);
if obs.kind == "intake.submitted" { let atoms = [ atom("intake.id", body.intake_id), atom("intake.patient_name", body.patient_name), atom("intake.dob", body.dob), atom("intake.raw_text", body.raw_text), ];
if body.contains("source_ref") { atoms.push(atom("intake.source_ref", body.source_ref)); }
if body.contains("phi_scope") { atoms.push(atom("intake.phi_scope", body.phi_scope)); }
if body.contains("symptom_age_days") { atoms.push(atom("intake.symptom_age_days", body.symptom_age_days)); }
return atoms; }
// llm.extraction_result emits extraction.* atoms that rules project // into candidate.conditions and candidate.medications.
if obs.kind == "clinician.review" { let atoms = [ atom("review.intake_id", body.intake_id), atom("review.approved", body.approved), ];
if body.contains("reviewer_role") { atoms.push(atom("review.reviewer_role", body.reviewer_role)); }
if body.contains("review_scope") { atoms.push(atom("review.scope", body.review_scope)); }
if body.contains("corrections") { atoms.push(atom("review.corrections", body.corrections)); }
return atoms; }
if obs.kind == "referral.directory" { return [ atom("referral.route", body.route), atom("referral.min_role", body.min_role), ]; }
if obs.kind == "llm.triage_proposal" { return [ atom("triage.intake_id", body.intake_id), atom("triage.action", body.action), atom("triage.route", body.route), atom("triage.advice", body.advice), atom("triage.urgency", body.urgency), atom("triage.seq", body.seq), ]; }
[]}The full mapper also handles llm.extraction_result and intake.finalized.
The extraction branch demonstrates multi-valued atom extraction: the LLM
returns arrays of conditions and medications, but atoms are flat key-value
pairs. The mapper emits one extraction.condition atom per condition and one
extraction.medication atom per medication. The ontology rules then project
these into the candidate. relations.
The clinician.review mapper records reviewer role and review scope. A review
without reviewer_role = "clinician" and review_scope = "intake" does not
promote candidates to accepted facts.
Step 4: Write Derivation Rules
Section titled “Step 4: Write Derivation Rules”ontology/rules.dh is where the candidate-evidence pattern comes to life.
Stratum 1: Observation Projections
Section titled “Stratum 1: Observation Projections”Base facts are projected from atoms:
rule intake_submitted(id, name, dob, raw) :- atom(obs, "intake.id", id), atom(obs, "intake.patient_name", name), atom(obs, "intake.dob", dob), atom(obs, "intake.raw_text", raw).
rule assert candidate.conditions(id, condition, seq) :- atom(obs, "extraction.intake_id", id), atom(obs, "extraction.condition", condition), atom(obs, "extraction.seq", seq).
rule assert candidate.medications(id, medication, seq) :- atom(obs, "extraction.intake_id", id), atom(obs, "extraction.medication", medication), atom(obs, "extraction.seq", seq).
rule assert extraction_confidence(id, confidence, seq) :- atom(obs, "extraction.intake_id", id), atom(obs, "extraction.confidence", confidence), atom(obs, "extraction.seq", seq).
rule assert clinician_approved(id) :- atom(obs, "review.intake_id", id), atom(obs, "review.approved", "true"), atom(obs, "review.reviewer_role", "clinician"), atom(obs, "review.scope", "intake").Notice that candidate.conditions and candidate.medications use assert — they are durable facts that persist across evaluations. Each carries an extraction_seq that identifies which LLM extraction produced it. This sequence number is the key to the retraction rules below.
Retraction on Re-extraction
Section titled “Retraction on Re-extraction”When a second LLM extraction arrives with a different sequence number, the previous candidates are retracted:
rule retract candidate.conditions(id, condition, old_seq) :- atom(old_obs, "extraction.intake_id", id), atom(old_obs, "extraction.condition", condition), atom(old_obs, "extraction.seq", old_seq), atom(new_obs, "extraction.intake_id", id), atom(new_obs, "extraction.seq", new_seq), new_seq > old_seq, old_seq != new_seq.
rule retract candidate.medications(id, medication, old_seq) :- atom(old_obs, "extraction.intake_id", id), atom(old_obs, "extraction.medication", medication), atom(old_obs, "extraction.seq", old_seq), atom(new_obs, "extraction.intake_id", id), atom(new_obs, "extraction.seq", new_seq), new_seq > old_seq, old_seq != new_seq.This ensures only the latest extraction’s candidates are active. The retracted candidates appear in the contradiction log — the system records that these facts were once believed and then superseded. In Studio, contradicted candidates surface as Blocked Activity rows whose drill inspector explains the supersession; the dedicated contradiction-queue surface ships in V1.1, with the same data already exported in every verification bundle.
Stratum 2: Candidate Acceptance
Section titled “Stratum 2: Candidate Acceptance”This is the mandatory rejection rule in action. Candidates are promoted to accepted facts only when clinician_approved exists:
rule accepted_conditions(id, condition) :- candidate.conditions(id, condition, _), clinician_approved(id).
rule accepted_medications(id, medication) :- candidate.medications(id, medication, _), clinician_approved(id).Without clinician_approved, the candidates remain unaccepted. The ontology will never derive accepted_conditions or accepted_medications from LLM output alone. This is how JacqOS prevents LLM hallucinations from silently becoming system truth.
Stratum 3: Triage Proposal Decisions
Section titled “Stratum 3: Triage Proposal Decisions”Triage proposals are staged under proposal.* and admitted only by domain
decision rules:
rule triage.decision.blocked(id, "wrong_review_authority") :- review_wrong_authority(id).
rule triage.decision.blocked(id, "stale_symptoms") :- intake_symptom_age_days(id, age_days), age_days > 14.
rule triage.decision.blocked(id, "unsafe_autonomous_advice") :- triage.current_seq(id, seq), proposal.triage_action(id, "self_care", _, _, seq), accepted_conditions(id, "chest pain").
rule triage.decision.escalate(id, "emergency_department", "red_flag_symptom") :- accepted_conditions(id, "chest pain").
rule triage.decision.referral(id, route) :- triage.current_seq(id, seq), proposal.triage_action(id, "refer", route, _, seq), referral.route_allowed(route, "clinician"), clinician_approved(id), intake_symptom_age_days(id, age_days), age_days <= 14.The proposal itself is not the action. It is evidence. JacqOS only produces referral or escalation intents after these decision facts exist.
Stratum 4: Status Derivation
Section titled “Stratum 4: Status Derivation”The intake_status relation computes the lifecycle position:
rule intake_status(id, "extracted") :- intake_submitted(id, _, _, _), candidate.conditions(id, _, _), not clinician_approved(id), not clinician_rejected(id, _), not intake_finalized(id).
rule intake_status(id, "approved") :- intake_submitted(id, _, _, _), clinician_approved(id), not intake_finalized(id).
rule intake_status(id, "rejected") :- intake_submitted(id, _, _, _), clinician_rejected(id, _), not clinician_approved(id), not intake_finalized(id).
rule intake_status(id, "finalized") :- intake_finalized(id).Status is derived, not stored. Replay from any point in the observation log produces the same status, because status is a function of which observations exist.
Invariants
Section titled “Invariants”Two invariants enforce structural correctness:
rule intake.invariant.finalized_without_review(id) :- intake_finalized(id), not clinician_approved(id).
invariant no_finalize_without_review() :- count intake.invariant.finalized_without_review(_) <= 0.
invariant no_double_finalization(id) :- count intake_finalized(id) <= 1.no_finalize_without_review guarantees that every finalized intake has
clinician approval — you cannot skip the review step. It also emits a witness
fact so the verification bundle can show a minimal counterexample. no_double_finalization
prevents an intake from being finalized more than once.
Step 5: Derive Intents
Section titled “Step 5: Derive Intents”ontology/intents.dh derives outbound actions from stable state:
rule intent.request_extraction(id, raw) :- intake_submitted(id, _, _, raw), not candidate.conditions(id, _, _), not intake_finalized(id).
rule intent.notify_clinician(id, name) :- intake_submitted(id, name, _, _), candidate.conditions(id, _, _), not clinician_approved(id), not clinician_rejected(id, _), not intake_finalized(id).
rule intent.route_referral(id, route) :- triage.decision.referral(id, route), not intake_finalized(id).
rule intent.escalate_safety(id, route, reason) :- triage.decision.escalate(id, route, reason), not intake_finalized(id).intent.request_extraction fires when an intake has been submitted but no LLM extraction has happened yet. The shell executes this as an llm.complete call against the extraction_model resource, using the prompt from prompts/extraction-system.md and the schema from schemas/intake-extraction.json. The LLM’s structured response comes back as an llm.extraction_result observation.
intent.notify_clinician fires when candidates exist but no clinician has
reviewed them yet. This sends an HTTP notification so the clinician knows there
is work to review. intent.route_referral and intent.escalate_safety are
downstream of admitted triage decisions, not raw LLM proposals.
Step 6: Write Golden Fixtures
Section titled “Step 6: Write Golden Fixtures”Fixtures define deterministic scenarios with expected world states.
Happy Path: Approved Intake
Section titled “Happy Path: Approved Intake”A patient’s intake is submitted, the LLM extracts conditions and medications with high confidence, the clinician approves, and the intake is finalized:
{"kind":"intake.submitted","payload":{"intake_id":"intake-1","patient_name":"Jane Doe","dob":"1985-03-15","raw_text":"Patient reports history of type 2 diabetes and hypertension. Currently taking metformin 500mg and lisinopril 10mg daily."}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-1","extracted_conditions":["type 2 diabetes","hypertension"],"extracted_medications":["metformin 500mg","lisinopril 10mg"],"confidence":"0.95","seq":"1"}}{"kind":"clinician.review","payload":{"intake_id":"intake-1","approved":"true","reviewer_role":"clinician","review_scope":"intake"}}{"kind":"intake.finalized","payload":{"intake_id":"intake-1"}}After replay: candidate.conditions and candidate.medications hold the LLM’s extractions. Because clinician_approved("intake-1") is true, accepted_conditions and accepted_medications are derived. intake_status("intake-1", "finalized") is the terminal state. Both invariants pass.
Contradiction Path: Re-extraction
Section titled “Contradiction Path: Re-extraction”The LLM is called twice for the same intake. The second extraction supersedes the first:
{"kind":"intake.submitted","payload":{"intake_id":"intake-2","patient_name":"John Smith","dob":"1972-08-22","raw_text":"Patient has GERD and takes omeprazole 20mg. Also reports mild asthma."}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-2","extracted_conditions":["GERD","mild asthma"],"extracted_medications":["omeprazole 20mg"],"confidence":"0.88","seq":"1"}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-2","extracted_conditions":["gastroesophageal reflux disease","asthma"],"extracted_medications":["omeprazole 20mg","albuterol inhaler"],"confidence":"0.92","seq":"2"}}When the second extraction arrives, the retraction rules fire. The seq 1 candidates ("GERD", "mild asthma", "omeprazole 20mg") are retracted and replaced by the seq 2 candidates ("gastroesophageal reflux disease", "asthma", "omeprazole 20mg", "albuterol inhaler"). The retractions appear as contradictions — contradicted candidates surface as Blocked Activity rows in Studio whose drill inspector explains the supersession; the dedicated contradiction-queue surface ships in V1.1, with the same data already exported in every verification bundle.
The final state is intake_status("intake-2", "extracted") with intent.notify_clinician pending. No clinician has reviewed yet, so no candidates are accepted.
LLM Disagreement Path: Clinician Rejection
Section titled “LLM Disagreement Path: Clinician Rejection”The LLM extracts with low confidence and the clinician rejects the result:
{"kind":"intake.submitted","payload":{"intake_id":"intake-3","patient_name":"Maria Garcia","dob":"1990-11-04","raw_text":"Patient mentions occasional headaches and something about blood pressure pills. Hard to read handwriting."}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-3","extracted_conditions":["chronic headaches","hypertension"],"extracted_medications":["amlodipine 5mg"],"confidence":"0.45","seq":"1"}}{"kind":"clinician.review","payload":{"intake_id":"intake-3","approved":"false","corrections":"Patient has tension headaches only, not chronic. No confirmed hypertension diagnosis. Medication is actually acetaminophen PRN, not amlodipine."}}The LLM’s confidence of 0.45 signals uncertainty. The clinician rejects the extraction and provides corrections. After replay: candidate.conditions and candidate.medications still hold the LLM’s output, but no accepted_conditions or accepted_medications are derived because clinician_approved is absent. Instead, clinician_rejected("intake-3", "Patient has tension headaches only...") records the corrections. The final state is intake_status("intake-3", "rejected").
This is the candidate-evidence pattern working exactly as designed: the LLM hallucinated “chronic headaches” and “hypertension” from ambiguous text, but those hallucinations never became accepted system facts.
Referral Routing Path
Section titled “Referral Routing Path”A reviewed intake with fresh symptoms receives an orthopedic referral proposal:
{"kind":"intake.submitted","payload":{"intake_id":"intake-4","patient_name":"Ava Patel","dob":"1978-06-01","raw_text":"Twisted ankle yesterday. Mild swelling, no numbness, can bear weight.","source_ref":"portal-form-8841","phi_scope":"direct-care","symptom_age_days":1}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-4","extracted_conditions":["ankle sprain"],"extracted_medications":[],"confidence":"0.91","seq":"1"}}{"kind":"clinician.review","payload":{"intake_id":"intake-4","approved":"true","reviewer_role":"clinician","review_scope":"intake"}}{"kind":"referral.directory","payload":{"route":"orthopedics","min_role":"clinician"}}{"kind":"llm.triage_proposal","payload":{"intake_id":"intake-4","action":"refer","route":"orthopedics","advice":"Schedule non-urgent orthopedic follow-up.","urgency":"routine","seq":"1"}}After replay, JacqOS derives triage.decision.referral("intake-4", "orthopedics")
and intent.route_referral("intake-4", "orthopedics"). The LLM did not route
the referral by itself; it proposed evidence that the ontology admitted.
Unsafe Advice Path
Section titled “Unsafe Advice Path”The LLM proposes self-care for accepted chest pain:
{"kind":"intake.submitted","payload":{"intake_id":"intake-5","patient_name":"Nolan Reed","dob":"1965-12-09","raw_text":"Crushing chest pain and shortness of breath started this morning.","source_ref":"nurse-call-4412","phi_scope":"urgent-care","symptom_age_days":0}}{"kind":"llm.extraction_result","payload":{"intake_id":"intake-5","extracted_conditions":["chest pain"],"extracted_medications":[],"confidence":"0.96","seq":"1"}}{"kind":"clinician.review","payload":{"intake_id":"intake-5","approved":"true","reviewer_role":"clinician","review_scope":"intake"}}{"kind":"llm.triage_proposal","payload":{"intake_id":"intake-5","action":"self_care","route":"home_care","advice":"Rest at home and monitor symptoms.","urgency":"routine","seq":"1"}}JacqOS records triage.decision.blocked("intake-5", "unsafe_autonomous_advice")
and derives intent.escalate_safety("intake-5", "emergency_department", "red_flag_symptom"). The fixture proves that the unsafe proposal is visible
and blocked, while the escalation path still fires.
Authority, Freshness, and Missing Review Paths
Section titled “Authority, Freshness, and Missing Review Paths”Three additional fixtures close common regulated-domain failure modes:
wrong-authority-path.jsonlproves a non-clinician approval derivesreview_wrong_authorityand blocks the triage decision.stale-symptoms-path.jsonlproves symptom evidence older than 14 days derivestriage.decision.blocked(..., "stale_symptoms").missing-review-finalization-path.jsonlproves finalization without clinician approval produces an expectedno_finalize_without_reviewcounterexample with provenance to the offending observation.
Step 7: Verify
Section titled “Step 7: Verify”Run all fixtures and invariants:
$ jacqos verifyReplaying fixtures... happy-path.jsonl PASS (4 observations, 13 facts) contradiction-path.jsonl PASS (3 observations, 8 facts, 3 contradictions) llm-disagreement-path.jsonl PASS (3 observations, 7 facts) referral-routing-path.jsonl PASS (5 observations, referral intent) unsafe-advice-path.jsonl PASS (4 observations, escalation intent) wrong-authority-path.jsonl PASS (4 observations, blocked decision) stale-symptoms-path.jsonl PASS (4 observations, blocked decision) missing-review-finalization-path.jsonl PASS (2 observations, expected counterexample)
Checking invariants... no_finalize_without_review PASS no_double_finalization PASS
All checks passed.Every fixture replays from scratch on a clean database. The contradiction count in contradiction-path.jsonl is expected — it reflects the three retracted seq 1 candidates, not an error.
LLM Recording and Replay
Section titled “LLM Recording and Replay”The replay = "record" setting on the extraction_model resource is what makes this example fully deterministic despite depending on an LLM.
During live execution (jacqos dev), the shell captures every terminal LLM attempt as an effect envelope. The envelope records the model resource, provider, provider model, prompt bundle digest, world-slice digest, structured-output schema, raw response, parsed response, validation state, refusal state, token usage, and outcome observation.
{ "request": { "model_ref": "extraction_model", "provider_ref": "openai", "provider_model": "gpt-4o-mini", "prompt_bundle_digest": "sha256:...", "world_slice_digest": "sha256:...", "structured_output_schema_ref": "schemas/intake-extraction.json" }, "response": { "validation": "valid", "refusal": "not_refused", "parsed_response": { "intake_id": "intake-1", "confidence": "0.95" } }}During fixture replay, the shell matches requests to recorded captures by replay identity. This gives you:
- Deterministic fixtures that produce identical facts regardless of LLM availability or model version changes
- No API key required for running
jacqos verifywhen the needed captures are present - Visible LLM behavior — effect attempts and outcome observations show what the model was asked to decide and what it returned
If replay = "replay" or provider_mode.mode = "replay" is set and a matching capture is missing, dispatch fails explicitly rather than silently calling the live API.
Prompt and Schema Design
Section titled “Prompt and Schema Design”The extraction prompt (prompts/extraction-system.md) and schema (schemas/intake-extraction.json) work together to constrain LLM output:
The prompt instructs the model to extract only explicitly mentioned conditions and medications, normalize to clinical terminology, and set confidence below 0.7 when the text is ambiguous. The schema enforces structured JSON output with required fields: intake_id, extracted_conditions, extracted_medications, and confidence.
This combination means the mapper receives predictable JSON rather than free-form text. But the candidate-evidence pattern exists precisely because even well-constrained LLM output can be wrong — as the LLM disagreement fixture demonstrates.
How the Pipeline Flows
Section titled “How the Pipeline Flows”Here is the complete flow for the happy path:
Observation: intake.submitted {intake_id: "intake-1", patient_name: "Jane Doe", ...} -> Atoms: intake.id="intake-1", intake.patient_name="Jane Doe", intake.dob="1985-03-15", intake.raw_text="Patient reports..." -> Fact: intake_submitted("intake-1", "Jane Doe", "1985-03-15", "Patient reports...") -> Intent: intent.request_extraction("intake-1", "Patient reports...")
Observation: llm.extraction_result {intake_id: "intake-1", confidence: "0.95", ...} -> Atoms: extraction.intake_id="intake-1", extraction.condition="type 2 diabetes", extraction.condition="hypertension", extraction.medication="metformin 500mg", extraction.medication="lisinopril 10mg", extraction.confidence="0.95", extraction.seq="1" -> Assert: candidate.conditions("intake-1", "type 2 diabetes", "1") -> Assert: candidate.conditions("intake-1", "hypertension", "1") -> Assert: candidate.medications("intake-1", "metformin 500mg", "1") -> Assert: candidate.medications("intake-1", "lisinopril 10mg", "1") -> Fact: intake_status("intake-1", "extracted") -> Intent: intent.notify_clinician("intake-1", "Jane Doe")
Observation: clinician.review {intake_id: "intake-1", approved: "true", reviewer_role: "clinician", review_scope: "intake"} -> Atoms: review.intake_id="intake-1", review.approved="true", review.reviewer_role="clinician", review.scope="intake" -> Assert: clinician_approved("intake-1") -> Fact: accepted_conditions("intake-1", "type 2 diabetes") -> Fact: accepted_conditions("intake-1", "hypertension") -> Fact: accepted_medications("intake-1", "metformin 500mg") -> Fact: accepted_medications("intake-1", "lisinopril 10mg") -> Fact: intake_status("intake-1", "approved")
Observation: intake.finalized {intake_id: "intake-1"} -> Atoms: finalized.intake_id="intake-1" -> Assert: intake_finalized("intake-1") -> Fact: intake_status("intake-1", "finalized")Every fact traces back to specific observations. Open Studio to follow any provenance edge visually.
Key Patterns
Section titled “Key Patterns”Candidate-evidence acceptance prevents LLM hallucinations from becoming system truth. LLM extractions land as candidate. relations. They are visible, queryable, and useful — but they never become accepted_ facts without explicit clinician approval. This is a structural guarantee enforced by the ontology, not a convention that can be accidentally bypassed.
Retraction with sequence numbers handles re-extraction cleanly. When a second LLM extraction arrives, the old candidates are retracted and the new ones asserted. The contradiction log records exactly what changed and why. No manual cleanup or state machine transitions needed.
Low-confidence extractions are visible, not hidden. The extraction_confidence relation preserves the LLM’s self-reported confidence. A clinician seeing a 0.45 confidence score knows to scrutinize the extraction more carefully. The system doesn’t hide or suppress uncertain results — it surfaces them through the candidate-evidence gate.
Proposal containment keeps LLM advice from becoming action. Triage output lands under proposal.*. Referral and escalation intents only appear after the ontology checks reviewer authority, route membership, symptom freshness, and red-flag symptoms.
Next Steps
Section titled “Next Steps”- Appointment Booking Walkthrough — slot reservation without LLM involvement
- Price Watch Walkthrough — HTTP effects and recorded replay
- Using Fallible Sensors Safely — the broader pattern for keeping noisy interpretations from becoming trusted facts, including the mapper contract and acceptance-rule reference
- LLM Agents Guide — deep dive into LLM integration patterns
- Observation-First Model — the mental model behind this architecture
- Golden Fixtures — how fixture verification works in detail
- Debugging with Provenance — tracing facts back to observations
- jacqos.toml Reference — configuration format including LLM resources
- CLI Reference — every CLI command