A tax document classifier built with Jev.

We ingest thousands of tax documents using an LLM pipeline built last tax season. Jev classifies 100% of our tax document corpus at $0.001 per page — 34× cheaper and 6× faster than that LLM setup. This is the classifier, open sourced.

One request per page. The page's text goes to Jev, TypeSafe's decision model, which returns a probability over 261 IRS forms and 7 page kinds instead of text. No model is trained and nothing is hosted: the classifier is a JSON file describing each form, generated from the IRS's own PDFs.

import { classifyPage, jevBackend, pdfPageLines } from 'tax-doc-classifier'
import criteria from 'tax-doc-classifier/data/criteria.json' with { type: 'json' }

const lines = await pdfPageLines('return.pdf', 3)
const r = await classifyPage(lines, { backend: jevBackend(), criteria })
// r.form            'form-1040-schedule-a'
// r.kind            'form_page'
// r.formConfidence  0.99
// r.gated           true   (formConfidence >= 0.95)

No model is trained and nothing is hosted. The classifier is a JSON file of form descriptions (data/criteria.json) and a decision model that reads a page and picks from them. The decision model is TypeSafe Jev, a System One model that returns a probability over options instead of text.

1. Node 20+ and pnpm.
2. poppler — provides pdftotext and pdfinfo, used to read PDF pages. Only classifyPage on lines of text works without it.
   • macOS: brew install poppler
   • Debian/Ubuntu: apt install poppler-utils
3. A TypeSafe API key. Get one at typesafe.ai, then:

   export TYPESAFE_API_KEY=…
   

   or put it in a .env file (git-ignored) and load it with your usual tooling. The key is read from the environment by jevBackend(); it is never written to disk or logged.
4. Install and check:

   pnpm install
   pnpm typecheck
   pnpm test
   

To use it as a dependency in another project:

pnpm add github:<org>/tax-doc-classifier

The published package ships dist/ and data/criteria.json; pdftotext is still required on the host for the PDF helpers.

The score is strict: a page counts as an error if the answer is wrong or its confidence is below 0.95. Every number below is reproducible with pnpm eval.

The 38 low-confidence pages on the blank corpus are instruction pages that name no form, deep pages of corporate forms (5471, 8865, 1118), and single-schedule forms hedging against their parent. None is wrong.

Measured on the same bench pages, same machine, same hour. The previous classifier sends each page as a PDF to Claude Sonnet with the form registry in the prompt; different pages never share the prompt cache, so every page pays the full prompt.

PDF page ──pdftotext──▶ { header, body, footer }
                              │
                              ▼  one request
                     kind  : 7 options
                     form  : 230 options + not_in_this_list
                              │
                              ▼  second request only if form ∈ {5471, 8865, 8933, 1118, 5713}
                     sub   : that form and its schedules
                              │
                              ▼
        { form, kind, formConfidence, gated, probabilities }

• A blank page is answered without a call.
• The 230 first-list options are every form as the page prints it. Five corporate/foreign forms absorb their 35 schedules and get a second, small question; the hierarchy exists only where the page announces the parent more clearly than the schedule.
• formConfidence is the minimum over steps. Gate on it: above the gate, act; below, fall back to whatever you use today.

form_page · instructions · blank · cover_sheet · state_tax_form · broker_or_bank_statement · letter_or_other

The last three route a page away from the federal form list; they are defined but not yet evaluated.

Ids follow the IRS MeF naming, kebab-cased, parent first:

form-1040                 form-1040-schedule-a        form-1040-nr-schedule-nec
form-1099-int             form-w-2g                   form-5471-schedule-j
form-1065-schedule-k-1    form-8995-a-schedule-a

Form 1040 Schedule A in the IRS accepted-forms list becomes form-1040-schedule-a, mechanically and in both directions (mefNameToId). Every id matches ID_GRAMMAR, and the test suite refuses a criteria.json that contains one that does not.

A page number, when a caller asks for it, is a suffix: form-1040/p2. Every prefix of an id is a valid, less specific id. No suffix means the page was not asked, never "page 1".

data/criteria.json is generated — never edited by hand:

pnpm build-criteria

reads the IRS accepted forms and schedules XLSX, downloads each form's PDF from irs.gov, and extracts per form: the printed label, the title, the page count, the parent (for schedules), up to 8 box labels (information returns only), and the sibling forms it must not be confused with. Re-run it when the IRS publishes new revisions.

pnpm eval:download                                 # IRS PDFs → eval/corpus/blank
TYPESAFE_API_KEY=… pnpm eval --corpus=blank
TYPESAFE_API_KEY=… pnpm eval --corpus=bench --dir=/path/to/tax_calc_bench/ty25/test_data

Each run prints the strict score, lists every strict error with its confidence and runner-up, and writes the full per-page results to eval/results/.

The bench corpus is TaxCalcBench and is not redistributed here.

Backend is one method: ask(state, questions) → { answers, inputTokens } where each answer is a probability distribution over the question's options. jevBackend is the only implementation shipped. Anything that can return calibrated probabilities over a labelled option set can implement it.

• Text only. Scanned pages need OCR first; a page with no text layer is reported as blank.
• Federal forms only. State forms are detected by kind but not identified.
• English only.
• Requires pdftotext and pdfinfo (poppler) on the path for the PDF helpers; classifyPage itself takes lines of text and needs neither.
• The confidence is calibrated on the corpora above. Calibrate on your own pages before choosing a gate.

Node 20+, poppler, a TypeSafe API key in TYPESAFE_API_KEY.

Apache-2.0. See DATA-LICENSE.md for the IRS-derived data.