A deep dive into omniparsebench: reconstructing leaderboard results and ocr performance

A recent analysis of OmniParseBench, Datalab’s open-source benchmark for document parsers, reveals that the top two parsers are statistically tied, with Datalab accurate and Gemini 3.8 Flash both scoring around 92.9. The study, conducted on October 22, 2026, rebuilds the leaderboard using published verdicts and tests 27 parsers across 2,937 pages, including scanned forms, handwriting, and arXiv math. The researchers found that the leader varies by document type, with Gemini leading in handwriting and math, and Mistral OCR in Greek. Tesseract and Indra OCR were tested on a 50-page sample, with Tesseract achieving 30.4% accuracy on text tests. Indra OCR showed 92% agreement with published verdicts, though discrepancies in table tests were attributed to a single page. The study also highlights that the dataset card’s launch figures do not match released files, and the leaderboard’s order can change based on how tests are aggregated. The findings underscore the importance of slicing benchmarks to specific document types and reporting statistical intervals.