Ask anything about the employee handbook and leave policy; get an answer with the exact clause cited, or an honest "not in the policy"
Plays authored answers for the suggested questions. No network, no keys. Retrieval still runs live. Switching the retrieval toggle re-ranks every answer's citations on the page.
Try one
Eval
pass200 questions, 150 answerable with the clause they must cite and 50 that must be refused. Scored by the deterministic citation policy, no model; run 2026-09-14.
| metric | vector | hybrid | bar |
|---|---|---|---|
| retrieval hit @5 | 78.7% | 95.3% | ≥ 85.0% |
| citation precision | 66.3% | 83.2% | ≥ 75.0% |
| abstained correctly | 70.0% | 84.0% | ≥ 80.0% |
| abstained wrongly | 32.7% | 12.7% |
Hybrid is what the desk uses; vector only is shown for the toggle. Thresholds fail the CI run. golden set · scorer
Governance
- Abstention policy
- Retrieval scores every candidate section. If the best reranked score is under 0.42 (hybrid) or 0.36 (vector only), the desk answers "Not in the policy", cites nothing, and in hosted mode calls no model. Above the threshold it cites one section, the best one; the golden set showed a second citation was wrong every time. A hosted answer is instructed to use only the retrieved sections and to say "Not in the policy" when they do not answer, and any citation outside those sections is dropped.
- PII-free logging
- Replay mode makes no network request. Hosted and BYOK requests log one JSON line: request id, mode, retrieval mode, token counts, cost, latency, status. Never the question, the answer, or a key. The gateway logs the same fields and never message contents. A BYOK key lives in this tab's memory, goes out once as a header, and is not stored, not sent to analytics, and forgotten when the tab closes.
- What is synthetic
- The handbook belongs to a fictional company and was written for this demo. The two law extracts are plain-language summaries from general knowledge, not the text of any Act, and not legal advice. Replay answers were authored in the shape the gateway returns, with token counts estimated and cost at list price; they are not recorded model output.
- Cost ceiling
- Hosted mode runs on the CNE demo gateway under a daily budget for this demo; when it is spent the route returns the reset time and the page offers BYOK or replay.
Corpus
A synthetic handbook for a fictional company, 10 chapters, plus two plain-language summaries of Indian labour rules. 189 sections indexed as 189 chunks; BM25 and a 256-dimension hashed embedding, built 2026-09-14.