Answers an advisor can send.
A retrieval pipeline that drafts replies to teachers' 403(b) questions from public district documents, cites every claim, and says so when the documents are silent.
Every district runs its 403(b) differently.
Advisors who work with public-school teachers field the same questions all week: is this vendor approved here, can I borrow from my plan, how is this different from my pension. The answers live in district notices, vendor registries and state handbooks, and they differ from one district to the next.
A general-purpose chatbot answers confidently and generically, which is the worst outcome in a regulated field. The advisor needs a draft built only from that district's documents, with a source for every sentence, and an honest "not in the documents" when the answer isn't there.
Five steps, each visible in the demo.
- 01
Ingest
12 public documents (IRS, CalSTRS, CalPERS, FINRA, district notices and vendor lists) split into 396 page-level chunks. Scanned pages are OCR'd; re-running embeds nothing that hasn't changed.
- 02
Search by meaning
The question is embedded (Voyage) and matched with pgvector, fused with Postgres full-text ranking so exact names like a vendor still score.
- 03
Top 5 in territory
The advisor's districts are resolved on the server and applied inside the SQL query, before either ranker sees a row.
- 04
Claude answers only from sources
The five chunks go to Claude as search-result blocks. The prompt allows nothing else, and a source note marks template language as generic.
- 05
Answer + citations
Citations come back as structured references, never parsed from text. No citation means no draft; no answer means NOT IN THE DOCUMENTS.
The filter runs in the database, not the UI.
Each advisor covers specific districts. The browser sends only a persona name; the server looks up that persona's districts and passes them into the search function, which drops every other district's chunks before ranking. An out-of-territory document can't be retrieved, so it can't be cited.
-- inside match_chunks(): applied before vector or keyword ranking
scope as (
select c.id, c.embedding, c.fts
from chunks c
where c.district is null -- statewide documents
or c.district = any (allowed_districts)
)In the demo the two advisors are simulated, so anyone can pick either. In production the same filter becomes a row-level security policy keyed on the signed-in advisor, and the search runs with that user's token instead of a service key.
Measured before it shipped.
25 questions written in a teacher's voice: 20 the documents answer, each tagged with the document that holds the answer (checked against the ingested text), and 5 they don't, including a vendor's fees, a market forecast and a district outside the advisor's territory. Every model answered from the same retrieved chunks.
| Claude Opus 5.5 | Result |
|---|---|
| Expected document retrieved in the top 5 | 20/20 |
| Answerable questions answered with citations | 20/20 |
| Citations pointing at an expected document | 97% (90/93) |
| Unanswerable questions refused | 5/5 |
| Answerable questions wrongly refused | 0 |
| Retrieved chunks outside the advisor's territory | 0 |
| Median time to a full draft | 3.2 s |
| Cost per answer | $0.028 |
Claude Haiku 5.5 matched Opus on correctness at about 1/27th of the cost, but only with three times the token budget; its drafts ran longer and slower, so the demo answers with Opus.
Public documents, separate database.
The demo reads only public documents: federal and state guidance, district notices and vendor registries. It runs on its own database, separate from any client system, and holds no teacher or advisor records.
Typed questions aren't stored. Rate limiting keeps a salted hash of the visitor's IP address, never the address itself. The browser can't read the database at all; it only calls the answer function.
What it's built with.
- Claude API (Opus 5.5)drafting with structured citations and server-side refusal fallbacks
- Supabase Postgres + pgvectorhybrid vector and full-text search with reciprocal-rank fusion
- Voyage embeddings1,024-dimension document and query vectors
- Supabase Edge Functionsstreaming answers over server-sent events
- TypeScript / Reactthe demo UI, with a live pipeline view and citation badges
- Cloudflare Turnstilebot check, plus a per-IP daily limit on a salted hash and a global cap