Which Configuration Produced Xiaomi-OCR-0's "#1" Rank? Reading the Footnotes Before Enterprise Invoice Processing
A forensic benchmark analysis of Xiaomi-OCR-0’s 95.24, 93.45, and 92.73 Real5 scores, evaluating two-stage pipelines, layout detector overhead, and invoice extraction realities.
Japanese Edition: Read the original Japanese article on Substack
Video Episode (Japanese Audio): Watch on YouTube
Executive Summary & Key Takeaways
- Core Finding: Xiaomi-OCR-0—an open-source compact optical character recognition model released by Xiaomi—claims the “#1 rank” on the Real5 benchmark with a composite score of 95.24. However, a forensic inspection of primary documentation reveals three conflicting overall scores for the exact same model: 95.24 in the technical report and GitHub README, 93.45 in the evaluation pipeline documentation, and 92.73 on the benchmark maintainer’s official Hugging Face leaderboard.
- The Operational Distinction: The headline score of 95.24 is not achieved by the model operating alone; it requires an unbundled two-stage pipeline preceded by an external layout detector (
PP-DocLayoutV3via PaddleX at a 0.25 threshold). The advertised “0.8B” parameter count (873,438,784 weights) excludes this external detector entirely. The 93.45 figure reflects an end-to-end full-page pipeline without a detector. On the maintainer’s official leaderboard, Xiaomi-OCR-0 ranks #2 at 92.73, trailingPaddleOCR-VL-1.6(93.19). - Extraction Weaknesses in Practice: While Xiaomi-OCR-0 posts an average VQA score of 83.2 (beating
Qwen3.5-2Bat 80.9), it falls behindQwen3.5-2Bon pure OCR text recognition in OCRBench (84.6 vs. 85.9). Furthermore, in Xiaomi’s internal corporate information extraction benchmark, the model scored only 65.00% on invoices, trailingQwen3.5-2B(77.00%) by 12.00 percentage points. Critical operational variables—including Japanese language support, per-page latency, and GPU VRAM requirements—are entirely absent from official documentation. - Audience & Purpose: Written for enterprise IT leaders, digital transformation managers, and engineering teams evaluating compact vision-language models for automated document processing, invoice ingestion, and financial workflow automation.

Methodological Framework: Documentation, Computation, and Interpretation
This audit separates three categories of statements:
- Documented Statements (Primary Sources): Claims, tables, and notes drawn directly from Xiaomi’s research report (arXiv:2609.36136), Hugging Face model card, GitHub repository, evaluation scripts, and maintainer leaderboards for Real5 and OmniDocBench.
- Author Calculations: Mathematical values computed by the author from published figures (e.g., subtracting baseline scores, calculating condition averages, and parameter memory footprints).
- Analytical Interpretation: Explicit engineering inferences regarding enterprise operational constraints and reproducibility limits.
All primary artifacts were collected on November 1, 2026, and re-verified on November 2, 2026. The author did not execute local model weights or evaluate proprietary hardware latency; findings are based strictly on primary documentation.

Five Diagnostic Questions for OCR Benchmark Tables
When evaluating published OCR accuracy tables, engineering teams should apply five diagnostic questions:
- What pipeline configuration was evaluated? Is it a two-stage pipeline with an external layout detector or an end-to-end full-page parser? Are detector weights counted in advertised model size?
- Who performed and verified the benchmark? Did figures originate from developer promotional materials, official maintainer leaderboards, or independent third-party audits?
- What is the mathematical composition of the aggregate metric? Does the composite score account for reading order, or does it measure character error rates and table tree distances?
- Were competing models evaluated under identical runtime conditions? Did baselines use the same detector, thresholds, and prompt formatting?
- What operational specifications are omitted from documentation? Does documentation address target language support, inference latency, and VRAM overhead?
Section 1: The 95.24 Score Requires an External Detector Omitted from the “0.8B” Size

The evaluation pipeline documentation defines two operational modes:
- End-to-End (e2e) Full-Page Processing: Feeds the complete page image directly into the vision-language model, transcribing text and structural Markdown in a single forward pass.
- Two-Stage Pipeline Processing: An external layout analysis model segments the page into bounding boxes (text paragraphs, headers, tables, images). Each cropped region is transcribed individually by Xiaomi-OCR-0, and outputs are stitched into a full-page document.
In Xiaomi’s technical report, Section 4 specifies that all primary document parsing results reflect the two-stage pipeline utilizing PP-DocLayoutV3 (PaddleX implementation) at a detection threshold of 0.25.
Crucially, the report notes that the advertised “0.8B” parameter count reflects only the vision-language backbone. Hugging Face repository metadata confirms a parameter count of 873,438,784 weights. The parameter count and memory footprint of the required PP-DocLayoutV3 detector are omitted from published totals.
While the technical report’s abstract explicitly conditions its 95.24 claim—stating the model achieves top rank “under the condition where PP-DocLayoutV3 handles layout detection”—the Hugging Face model card omits this qualifying clause, claiming unqualified “#1” status across three benchmarks.
Developer Guidance on Pipeline Selection
Xiaomi’s documentation provides practical architectural guidance:
- Two-Stage Pipeline: Recommended for standard printed, multi-column documents with predictable geometry.
- End-to-End Pipeline: Recommended for unconstrained camera captures, handwriting, calligraphy, and irregular layouts where bounding-box errors segment text across semantic boundaries, corrupting context.
In production, two-stage pipelines suffer from cascading errors: if the front-end layout detector misclassifies a table cell or merges separate paragraphs, the transcription model cannot recover lost layout structure.
Section 2: Reconciling Real5’s Three Conflicting Scores

Real5 evaluates document transcription robustness across 1,355 pages from OmniDocBench v1.5 photographed under five challenging environmental capture conditions (6,775 total images): Scan, Bend (page curvature), Screen (display moiré), Light (uneven illumination/glare), and Tilt (perspective skew). The overall score is the arithmetic mean across the five conditions.
Tracing Xiaomi-OCR-0’s performance across published sources reveals three distinct figures:
| Source Document | Pipeline Configuration | Overall Real5 Score | Scan | Bend | Screen | Light | Tilt |
|---|---|---|---|---|---|---|---|
| Technical Report & GitHub | Two-Stage (PP-DocLayoutV3) |
95.24 | 96.41 | 95.46 | 94.47 | 95.39 | 94.46 |
| Evaluation Pipeline README | End-to-End (Full-Page) | 93.45 | 94.52 | 93.40 | 93.28 | 93.86 | 92.17 |
| Maintainer Leaderboard | Not Specified in Table | 92.73 | 93.73 | 92.76 | 92.80 | 92.68 | 91.69 |
Mathematical Discrepancy Analysis
- Two-Stage vs. Leaderboard: The technical report’s two-stage score (95.24) exceeds the maintainer leaderboard score (92.73) by 2.51 points. Looking at individual conditions, the two-stage pipeline scores higher across all five categories: Scan (+2.68), Bend (+2.70), Screen (+1.67), Light (+2.71), and Tilt (+2.77).
- End-to-End vs. Leaderboard: The evaluation pipeline’s end-to-end score (93.45) also exceeds the maintainer leaderboard score (92.73) across all five conditions by 0.48 to 1.18 points.
- Rank Reversal: On the maintainer leaderboard,
PaddleOCR-VL-1.6holds rank #1 at 93.19, placing Xiaomi-OCR-0 in second place at 92.73 (-0.46 points). While the technical report claims a +2.05 point lead over PaddleOCR-VL-1.6, that comparison holds only when comparing Xiaomi’s two-stage pipeline against PaddleOCR’s published figure.
Primary sources do not explain whether the 92.73 leaderboard score reflects a different checkpoint, alternate pre-processing, or a third-party rerun.
Section 3: Comparative Analysis Across Published Leaderboards

Comparing candidate models requires cross-referencing figures across reporting bodies:
| Benchmark & Metric | Source Document | Xiaomi-OCR-0 | PaddleOCR-VL-1.6 | OvisOCR2 | TeleOCR | Documented Pipeline Context |
|---|---|---|---|---|---|---|
| Real5 (Composite) | Xiaomi Report Table | 95.24 | 93.19 | 92.29 | — | Xiaomi uses 2-stage; competitors unstated |
| Real5 (Composite) | Maintainer Leaderboard | 92.73 | 93.19 | 92.29 | 90.72 | Pipeline configurations unstated |
| OmniDocBench v1.6 | Xiaomi Report Table | 96.83 | 96.33 | 96.58 | 96.87 | Xiaomi uses 2-stage; TeleOCR leads by 0.04 |
| OmniDocBench v1.6 | Maintainer Leaderboard | — | 96.34 | 96.47 | 96.91 | Xiaomi omitted; OXR-1.0 leads at 97.03 |
Benchmark Composition: What OmniDocBench Measures
OmniDocBench comprises 1,651 PDF pages spanning 10 document genres, 5 layout formats, and 5 languages. Its composite score is an unweighted average of three metrics:
- Text Edit Distance: Normalized character edit distance for prose text.
- TEDS (Tree-Edit-Distance-based Similarity): Structural and content accuracy for tables.
- CDM (Character-level Dice Metric): Mathematical equation transcription accuracy.
Crucially, reading order is excluded from the composite metric. A model can transcribe text paragraphs out of sequence across multi-column layouts while retaining a high composite score.
Furthermore, OmniDocBench genres center on academic papers, financial reports, newspapers, textbooks, and handwritten notes. Standard enterprise commercial forms—such as receipts, purchase orders, and structured invoices—are unrepresented.
Section 4: What Averages Conceal: OCRBench Deficits and Invoice Realities

Xiaomi reports an average Visual Question Answering (VQA) score of 83.2, outperforming Qwen3.5-2B (80.9) and approaching Qwen3.5-4B (84.9). However, decomposing the five underlying benchmarks reveals critical trade-offs:
| VQA Benchmark Component | Xiaomi-OCR-0 | Qwen3.5-2B | Qwen3.5-4B | Primary Evaluation Focus |
|---|---|---|---|---|
| DocVQA | 93.1 | 92.4 | 94.6 | Document layout question answering |
| InfoVQA | 83.9 | 80.5 | 86.8 | Infographic comprehension |
| ChartQA | 85.3 | 81.3 | 84.8 | Statistical charts and graphs |
| TextVQA | 69.3 | 64.5 | 71.8 | Scene text in natural images |
| OCRBench | 84.6 | 85.9 | 86.6 | Broad optical character recognition |
| Average VQA Score | 83.2 | 80.9 | 84.9 | Unweighted aggregate across 5 tasks |
On OCRBench—the benchmark evaluating pure character and word recognition across complex document layouts—Xiaomi-OCR-0 scores 84.6, falling 1.3 points behind Qwen3.5-2B (85.9) and 2.0 points behind Qwen3.5-4B (86.6).
Enterprise Information Extraction Performance
In Section 4.3, Xiaomi evaluates zero-shot key-value information extraction across 1,655 internal document images (identity credentials, bank cards, forms, and business invoices):
- Aggregate Extraction Accuracy: Xiaomi-OCR-0 achieved 59.09%, surpassing Qwen3.5-2B (51.60%).
- Invoice Extraction Accuracy: On business invoices, Xiaomi-OCR-0 scored 65.00%, trailing Qwen3.5-2B (77.00%) by 12.00 percentage points.
The authors explicitly acknowledge that address and invoice field extraction represent ongoing architectural challenges.

Three Critical Gaps in Official Specifications
- Zero Documented Japanese Capability: Textual searches across the research paper, model card, and code repositories yield zero mentions of Japanese text evaluation.
- Missing Latency Benchmarks: Official documentation contains no processing latency figures per page. Claims of “lightweight edge execution” lack published throughput measurements.
- Omitted Memory Footprint: At 16-bit precision, 873M parameters consume roughly 1.75 GB of static storage. However, runtime GPU VRAM requirements—including vision encoder activations, KV-cache allocation, and the external
PP-DocLayoutV3detector—are unstated.
Practical Five-Step Evaluation Protocol for In-House Corporate Documents

Enterprise teams evaluating compact OCR models should execute an internal proof-of-concept:
- Curate 30 Representative Documents: Assemble approximately 30 internal documents, including structured invoices, purchase orders, complex multi-page tables, and angled mobile captures.
- Test Both Pipeline Configurations: Run candidate documents through both the two-stage pipeline and end-to-end full-page parser. Measure whether detector failures degrade transcription quality.
- Audit Unrepresented Formatting: Include document edge cases: vertical text, mixed Japanese/English typography, company seals, and degraded print quality.
- Benchmark Structured JSON Extraction: Prompt models to extract key fields (invoice date, total tax-inclusive amount, vendor registration number) into strict JSON schemas. Evaluate field-level exact-match accuracy.
- Profile Device Economics: Measure actual execution time per page and peak GPU memory on target infrastructure.
Conclusion & Governance Guidelines
Published benchmark tables serve to identify candidate architectures, not determine enterprise procurement. Xiaomi-OCR-0 demonstrates strong document parsing capabilities for an 873M parameter model, but its advertised “#1” rank depends on an unbundled two-stage pipeline with an external layout detector.
When evaluating OCR models, engineering leaders should:
- Require vendors to disclose whether benchmark scores reflect standalone models or multi-stage pipelines.
- Verify whether advertised parameter counts include pre-processing detectors.
- Disregard aggregate averages and evaluate accuracy on specific corporate document formats.
Limits of This Investigation
- Author findings are based on published documentation as of November 1–2, 2026.
- Model weights were not executed locally; performance on Japanese documents and local hardware latency remain unverified.
- Maintainer leaderboards and model repositories may be updated following publication.
Final Verification Checklist
- Verified whether benchmark scores reflect two-stage pipelines or end-to-end execution.
- Confirmed whether advertised parameter counts include external layout detectors.
- Reconciled conflicting scores across developer reports and maintainer leaderboards.
- Audited task-specific sub-scores (OCRBench, invoices) beneath aggregate averages.
- Identified unaddressed operational parameters (target languages, latency, VRAM).
- Formulated an empirical testing protocol for corporate document repositories.
Primary Sources & Documentation Links
- Xiaomi-OCR-0 Hugging Face Model Card: https://huggingface.co/SeerRay-Lab/Xiaomi-OCR-0
- Xiaomi-OCR-0 GitHub Repository: https://github.com/SeerRay-Lab/Xiaomi-OCR-0
- Xiaomi-OCR-0 Evaluation Pipeline README: https://github.com/SeerRay-Lab/Xiaomi-OCR-0/blob/main/evaluation_pipeline/README.md
- Xiaomi-OCR-0 Technical Report (arXiv:2609.36136): https://arxiv.org/pdf/2609.36136
- Real5-OmniDocBench Dataset & Leaderboard: https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench
- OmniDocBench Official Repository: https://github.com/opendatalab/OmniDocBench
- Xiaomi-OCR-0 Real5 Evaluation Result YAML: https://huggingface.co/SeerRay-Lab/Xiaomi-OCR-0/raw/main/.eval_results/real5_omnidocbench.yaml
- Hugging Face API Metadata for Xiaomi-OCR-0: https://huggingface.co/api/models/SeerRay-Lab/Xiaomi-OCR-0