Healthcare & claims
- Generative AI
- Risk detection
- Document intelligence
The problem
Claims and clinical documentation arrive in dozens of formats from dozens of sources. Reviewers spend most of their time locating information rather than judging it, and the volume grows faster than the team can.
The obvious answer — put a language model on it — collapses on the first hard question: how does a reviewer know the extracted value is right, and what happens when it is not?
The approach
We treated extraction as a pipeline stage rather than a product. Documents are parsed with layout awareness so tables and headings survive, chunked along their own structure, and processed with a model whose output is constrained to a typed schema.
Every extracted field carries a reference back to the passage it came from and a confidence score. Anything below the threshold is routed to a review queue instead of flowing silently into the record.
Risk signals are scored separately from extraction, so a model change in one does not quietly move the other.
The engineering
Ingestion runs through a durable queue with idempotent workers, so a crash mid-batch resumes rather than restarts. Each processing run is versioned against the prompt, model and schema used, which makes results reproducible and regressions attributable.
An evaluation set built from reviewed documents gates every change. A prompt or model update that lowers the score does not reach production.
Where it landed
Reviewers work from a pre-populated record with citations rather than a raw document, and spend their attention on judgement rather than search. Exceptions surface explicitly instead of hiding inside an average.
What we would tell another team
- Constrain model output to a typed schema — free text is not an interface.
- Cite the source passage for every derived value, or reviewers cannot trust any of them.
- Gate prompt and model changes behind an evaluation set, exactly like code.
- Route low confidence to a human rather than letting it flow through silently.
Questions
How is accuracy measured?
Against a held-out set of documents reviewed by domain experts, scored per field type rather than as a single headline number — different fields fail in different ways.
What happens to documents the system cannot handle?
They are routed to a review queue with the reason attached. An explicit exception is far more useful than a confident wrong answer.