Ask a document a question. Get an answer from the text, or an honest “not answerable”.
Paste any text and ask about it. Shannon QA finds the passages that matter, lets a small reading model propose answers, and a learned gate picks one: a span, a list, a number, yes or no, or none. There is no generative model, so every answer is a piece of your text, and when no candidate is good enough it says so.
The document
The question
How it answers
Split by length only
The text's own paragraphs, long ones cut into windows of whole sentences (up to 120 words), short lines such as headings carried into the next passage. No rules about content.
Overlap and meaning
Top 3 passages by word overlap, weighted by how rare each word is in this document, plus top 3 by MiniLM sentence-embedding similarity.
all-MiniLM-L6-v2 · 22.7M parameters
The reader proposes
An extractive QA model reads each passage and proposes up to 6 spans. Each becomes a span, a one-item list, a number (via a small number-word reader), and pairs become two-item lists.
minilm-uncased-squad2 · 33.2M
A cross-encoder judges
A DeBERTa-v3 model fine-tuned on public BoolQ plus development yes/no questions gives P(yes) for the top passages.
DeBERTa-v3-xsmall · 70.8M
The gate picks or refuses
A question-form model (from the question's embedding) and a 50-weight log-linear gate score every candidate, including none. The top score wins; its lead over the runner-up is the confidence.
This is a port of the Python system (treaty-graph's no-hand-code system) to JavaScript, run with onnxruntime-web. On 66 test questions over the three example texts it gives the same answer as the Python system every time (see the notes below).
Honest notes
Published results (from the project's pre-registered exam write-ups)
| un_4: two treaties, 348 questions | accuracy |
|---|---|
| Shannon QA (the same design, trained without un_4) | 73.0% |
| Qwen2.5-1.5B-Instruct, given the same passages (reference) | 60.9% |
| Baseline: minilm-squad2 with overlap retrieval | 51.1% |
- un_4 was used as development data before this design was fixed, so 73.0 is a development number, not a held-out exam result. The model on this page was also trained on un_4 and on other development sets. The 60.9 and 51.1 rows come from the pre-registered un_4 scoring.
- Much of the lead comes from refusing: Shannon QA refused 84.8% of the un_4 questions the text could not answer.
- Reliability (lower AURC is better: fewer wrong answers among the most confident ones): the earlier treaty-specific system, H4, scored AURC 0.199 against 0.310 for Qwen2.5-1.5B on un_4. That is not this system. For this system, a later check against a calibrated, 4-shot Qwen2.5-1.5B found the two level (cross-fit AURC 0.180 vs 0.172), so no reliability win is claimed.
- Results on new domains (two sealed exams) are not reported here.
This page
- Models: all-MiniLM-L6-v2 (Apache-2.0), deepset/minilm-uncased-squad2 (CC-BY-4.0, by deepset), and a yes/no cross-encoder fine-tuned from cross-encoder/nli-deberta-v3-xsmall (Apache-2.0) on BoolQ (CC-BY-SA-3.0). They are stored as fp16 weights, plus int8 for the yes/no model's word table, and computed in fp32. Plain int8 throughout moved the reader's spans and the yes/no probabilities too much, so it isn't used.
- The confidence shown is the gate's margin: how far the winning candidate's score is above the runner-up's. It is not a probability.
- Everything runs single-threaded in WebAssembly. A question takes about one second on a laptop and longer on a phone. Reading a long document takes a few seconds.