LlamaIndex and Kaggle Launch a Document Benchmark for Cost and Long Files

LlamaIndex and Kaggle have launched ExtractBench, an open leaderboard for systems that turn enterprise documents into structured data. The benchmark extends beyond a single accuracy measure, but its published product comparison includes results reported by LlamaIndex for its own extraction system.

A broader extraction test

ExtractBench evaluates 14 systems on 370 enterprise documents totaling 4,869 pages. The collection spans eight business domains and 67 document types, and includes frontier vision-language models, coding agents and specialized extraction APIs.

It scores accuracy, perception quality, table structure, document length, domain coverage and cost. Scoring is deterministic and rule-based rather than decided by a language-model judge, giving the benchmark a defined method for comparing systems across those measures.

The launch comparison

LlamaIndex reported that Extract Agentic Plus reached 95.6% overall accuracy at 8.1 cents per page. In the same comparison, it reported 93.5% accuracy and 27.8 cents per page for Codex GPT-5.5.

Where document length separates systems

Extract Agentic Plus reached 94.4% accuracy on long documents, according to LlamaIndex. Its published breakdown put Gemini 3.5 Flash at 87.9% on short documents and 27.9% on long documents, showing how a single overall figure can conceal length-sensitive results.

LlamaIndex says its agentic approach splits documents into sections and checks extracted values against source pages. The workflow is designed to handle and verify portions of a document rather than rely on a single pass over the full file.

Tracing fields back to the source

LlamaIndex says its Extract system was the only tested system to return a bounding box for every extracted value. The coordinates let reviewers locate fields in source documents, though the distinction remains a claim from the vendor.

The leaderboard is available on Kaggle, while the benchmark code and paper are available through GitHub and arXiv. LlamaIndex says developers can submit their own systems, creating a route for the public ranking to grow beyond its launch entries.