Baidu’s Unlimited-OCR model, a 3-billion-parameter vision-language system, is reshaping how developers approach document parsing. Unlike traditional OCR pipelines that rely on separate layout analysis, text detection, and recognition stages, this model processes entire pages—including headings, tables, paragraphs, and footnotes—in a single decoding pass. The result is a unified, end-to-end workflow that handles both high-resolution single-page documents and multi-page PDFs with remarkable fidelity. In this tutorial, we walk through the complete pipeline: from GPU environment setup and dependency installation to generating sample documents, running single-image inference in Gundam and Base modes, and extending the system to parse multi-page PDFs using PyMuPDF and the model’s infer_multi()codecodecodecodecode method. The entire workflow is designed for reproducibility inside Google Colab, making it accessible to researchers, engineers, and data scientists who need accurate, long-context OCR without the overhead of traditional multi-stage systems.
Setting Up the GPU Environment for Unlimited-OCR Inference
The first step in building a reliable OCR pipeline is configuring the runtime environment to support the model’s memory and compute requirements. Unlimited-OCR, with its 3 billion parameters, demands a CUDA-enabled GPU and sufficient VRAM. In Google Colab, this means selecting a GPU runtime (T4, V100, or A100). The setup script begins by installing essential Python packages: transformerscodecodecodecodecode (version 4.57.1), Pillowcodecodecodecodecode, matplotlibcodecodecodecodecode, einopscodecodecodecodecode, addictcodecodecodecodecode, easydictcodecodecodecodecode, pymupdfcodecodecodecodecode, psutilcodecodecodecodecode, and acceleratecodecodecodecodecode. These libraries handle model loading, image processing, PDF rasterization, and performance monitoring.
After installation, the script checks for GPU availability and automatically selects the optimal data type. If the GPU supports bfloat16, the model loads in bfloat16; otherwise, it falls back to float16. This adaptive dtype selection is critical for memory efficiency—bfloat16 halves the memory footprint compared to float32 without sacrificing numerical stability. The model and tokenizer are then loaded from Hugging Face using AutoModel.from_pretrained()codecodecodecodecode and AutoTokenizer.from_pretrained()codecodecodecodecode with trust_remote_code=Truecodecodecodecodecode, because the model relies on custom code for its architecture. The model is moved to the GPU in evaluation mode, ready for inference.
Generating Realistic Sample Documents for Testing
To validate the pipeline, we create three sample pages that mimic a quarterly operations report. Each page is a 1240×1754 pixel image generated with PIL, containing a title, a multi-line paragraph, a table with regional revenue data, and a footnote. The table includes columns for Region, Q1, Q2, and Q3, with rows for North, South, East, and West. The footnotes reference cross-page content, testing the model’s ability to maintain context across pages. The images are saved as PNG files in an inputscodecodecodecodecode directory. A quick preview using Matplotlib confirms the layout before feeding the images into the OCR pipeline.
This synthetic data generation step is essential for controlled testing. It allows us to verify that the model correctly extracts structured information from tables, reads body text without hallucination, and preserves the hierarchical relationship between headings and content. The same approach can be adapted to realaa-world documents by replacing the make_sample_page()codecodecodecodecode function with a PDF-to-image conversion routine.
Single-Image OCR: Gundam Mode vs. Base Mode
Unlimited-OCR offers two inference modes for single images: Gundam (tiled) and Base (single view). Understanding the difference is key to choosing the right configuration for your use case.
Gundam Mode: Tiled Crops for Dense, Small Text
Gundam mode combines a global view of the document with a series of tiled image crops. The pipeline sets base_size=1024codecodecodecodecode and image_size=640codecodecodecodecode with crop_mode=Truecodecodecodecodecode. The smaller tile size (640 pixels) allows the model to focus on fine details, such as small font sizes, dense tables, or footnotes. The global view provides context, while the crops ensure that no text region is missed. This mode is ideal for documents with complex layouts, multiple columns, or very small print. The inference call uses max_length=32768codecodecodecodecode to accommodate long outputs, no_repeat_ngram_size=35codecodecodecodecode to reduce repetition, and ngram_window=128codecodecodecodecode to stabilize the decoding process. The results are saved to the outputs/single_gundamcodecodecodecodecode directory.
Base Mode: Single View for Clean, Clearly Printed Pages
Base mode processes the entire image at a single resolution of 1024 pixels, with crop_mode=Falsecodecodecodecodecode. This eliminates the overhead of tiling, making inference faster and more memory-efficient. For documents with clean, large fonts and simple layouts, Base mode produces equally accurate output while reducing latency. The same generation parameters (max_length, no_repeat_ngram_size, ngram_window) are retained to ensure consistency. The output is saved to outputs/single_basecodecodecodecodecode.
A direct comparison between the two modes reveals that Gundam mode excels on documents with dense, small text or irregular layouts, while Base mode is sufficient for standard printed pages. The choice depends on the quality and complexity of your input images.
Building a Multi-Page PDF Pipeline
Real-world documents often span multiple pages. Unlimited-OCR supports multi-page parsing through the infer_multi()codecodecodecodecode method, which processes a sequence of page images in a single long-horizon inference pass. This is a significant advantage over traditional OCR, which typically processes each page independently and then merges results—often losing cross-page context such as table continuations or footnotes that reference earlier pages.
To prepare the input, we construct a three-page PDF from the sample images using PyMuPDF (fitz). The PDF creation step opens each image as a separate page, then saves the combined document. Next, we rasterize the PDF back to high-resolution PNG images at 300 DPI using a custom pdf_to_images()codecodecodecodecode function. This ensures that the model receives the same visual quality as the original generated pages. The resulting list of image paths is passed to model.infer_multi()codecodecodecodecode with image_size=1024codecodecodecodecode, max_length=32768codecodecodecodecode, and a wider ngram_window=1024codecodecodecodecode to maintain coherence across the entire document.
The infer_multi()codecodecodecodecode method is designed to handle long context windows. It concatenates the tokenized representations of each page image, allowing the model to reference information from earlier pages when generating the output for later pages. This is particularly useful for documents with cross-page tables, multi-page narratives, or sequential annotations. The output artifacts—text, Markdown, MMD, and JSON—are saved to outputs/multi_pagecodecodecodecodecode.
Inspecting OCR Outputs and Understanding the Artifacts
After inference, the pipeline inspects the output directories for each run. It lists all files, their sizes, and for text-based formats (TXT, MD, MMD, JSON), it displays a preview of the first 1500 characters. This step is crucial for verifying the quality of the OCR extraction. The model produces structured output that mirrors the original document layout: headings are preserved, tables are formatted as markdown tables, and paragraphs are separated by line breaks. The JSON output includes the raw text, as well as metadata such as bounding boxes for each detected element (if saved).
In the Gundam mode output, we typically see more detailed table extraction because the tiled crops capture cell boundaries precisely. Base mode may occasionally merge adjacent table cells if the global resolution is insufficient. The multi-page output demonstrates coherence across pages: the footnote on page 3 correctly references “Note 3” and the table on page 2 appears in sequence. The model does not repeat headers or lose the narrative flow, thanks to the long-context generation settings.
How Does Unlimited-OCR Compare to Traditional OCR Pipelines?
Traditional OCR systems (e.g., Tesseract, Google Cloud Vision) typically require a separate layout analysis step to detect text regions, then perform character recognition, and finally post-process the results. This multi-stage pipeline is error-prone—misalignment in layout analysis can cascade into incorrect text recognition. Unlimited-OCR, as a vision-language model, bypasses this by directly generating text from the visual input. It uses a single transformer-based architecture that learns to attend to both global and local features. The model is pre-trained on a large corpus of document images and text, enabling it to understand structural cues like tables, headings, and lists without explicit rules.
For dense, small text, the Gundam mode’s tiling strategy mimics human reading: we scan the page in a grid, focusing on each region. The model’s ability to repeat this process for multiple crops ensures that no text is missed. In contrast, traditional OCR with fixed sliding windows may miss small fonts or fail to detect table boundaries. Additionally, the long-context generation (up to 32,768 tokens) allows Unlimited-OCR to output entire documents in one pass, reducing the need for complex stitching algorithms.
Configuring Generation Parameters for Stable Long-Output
One of the key challenges in large language model-based OCR is preventing repetitive or degenerate outputs, especially when the model generates thousands of tokens. The pipeline uses two critical parameters: no_repeat_ngram_sizecodecodecodecodecode and ngram_windowcodecodecodecodecode. no_repeat_ngram_size=35codecodecodecodecode prevents the model from generating the same 35-gram (sequence of 35 tokens) more than once. This dramatically reduces repetitive loops, which are common when the model is uncertain about the layout. The ngram_windowcodecodecodecodecode parameter limits the context over which the repetition penalty is applied—128 for single pages and 1024 for multi-page documents. A larger window allows the model to use longer-range context but may increase computational cost. These settings are adopted from the original model’s best practices and have been validated on long-form document parsing tasks.
Another important parameter is max_lengthcodecodecodecodecode. The default of 32768 tokens is sufficient for most multi-page documents. If your document is extremely long (e.g., 50 pages), you may need to increase this value, but GPU memory will become a bottleneck. The model’s architecture supports a maximum context length of 4096 tokens per image, but multi-page inputs are concatenated, so the effective limit depends on the number of pages and the tokenizer’s compression rate.
Practical Considerations for Scaling the Pipeline
The pipeline as described runs comfortably on a single Colab GPU (T4 with 16GB VRAM). For larger documents or higher-resolution images, consider the following optimizations:
- Reduce image resolution: If the document uses large fonts, lowering the DPI from 300 to 150 can speed up rasterization and inference without sacrificing accuracy.
- Batch processing: For multiple single-page documents, use
infer()codecodecodecodecode in a loop. The model is not designed for batched multi-page inference, but you can parallelize single-page jobs across multiple GPUs if available. - Mixed precision: The pipeline already uses bfloat16/float16. For further memory savings, enable gradient checkpointing (though not required for inference) or use
torch.cuda.amp.autocast()codecodecodecodecode in custom loops. - Output post-processing: The model often outputs Markdown tables with aligned columns. If you need machine-readable JSON, you can parse the Markdown output with libraries like
pandascodecodecodecodecode ormarkdowncodecodecodecodecode.
Reusing the Workflow for Your Own Documents
The pipeline is designed to be easily adapted. To process your own files, simply upload them to Colab’s file system, then point the image_filecodecodecodecodecode parameter (for single pages) or the pdf_to_images()codecodecodecodecode function (for PDFs) to your file. The model supports a wide variety of document types: scanned forms, technical reports, academic papers, invoices, and even handwritten notes (though accuracy may vary). The cheat sheet provided in the original code summarizes the best mode for each scenario:
- Dense, small text → Gundam mode (640, crop_mode=True)
- Clean print → Base mode (1024, crop_mode=False)
- Multi-page PDF → infer_multi() with image_size=1024, ngram_window=1024
- Long documents → keep max_length=32768 and repetition controls
This pipeline eliminates the need for a separate traditional OCR and layout analysis stack, offering a single, reproducible solution for high-res, multi-page document parsing. The combination of Gundam and Base modes, long-context generation, and adaptive dtype selection makes it a robust foundation for both research and production workflows.
