Harness run: 1 September 2026
Angaba's AI extraction is judged against a held-out fixture set of real supplier documents, not against documents it was tuned on. This page states the method and the current numbers plainly, and is updated only when the harness is re-run.
28 real supplier PDFs — declarations of conformity, manuals, spec sheets, test reports, plus deliberately hard negative cases — roughly 24 MB and 382+ pages combined, in German, English, Italian, French and bilingual Chinese-English. Fixture PDFs are not published on this site: they are third-party suppliers' own documents, not ours to republish.
A value is accepted only above Angaba's confidence threshold. Below it — or when the model isn't confident enough to answer at all — the field is marked needs_review or left absent, never guessed. Precision is measured only over accepted values: an accepted-but-wrong value counts against the gate; a low-confidence value correctly sent to review does not.
| Metric | Result |
|---|---|
| Manufacturer name/address values accepted | 31 of 31 correct (16 names, 15 addresses) |
| EU responsible-person values accepted | 0 — none cleared the gate; 2 were sent to review instead, the rest of the documents named none |
| Fields sent to review instead of guessed | 11, across manufacturer and responsible-person fields |
| Values present in a document but returned as absent | 1 — one manufacturer name; a recall miss the harness reports separately, and the reason recall is not what we promise |
| Documents with no product-safety content, correctly refused | 6 of 6 — three technical data sheets, an invoice, a safety-irrelevant document and an unreadable scan |
| Gate precision | 100% (standing target: ≥98%) |
An automated, normalised string comparison (src/scripts/harness.ts) checks every accepted value against a hand-built ground-truth manifest. Most manifest entries were verified by reading the full source document; some were verified from the document's live source page and search snippets rather than a complete read, which the manifest records per entry.
A wrong EU responsible person or a missing warning is a merchant liability, not a UX inconvenience. We would rather publish an unflattering number than an unverifiable one — see the refusal behaviour this is built around.
Angaba is a data tool, not legal advice.