OCR has improved a lot. Modern AI can read PDFs, extract tables, understand formulas, and handle complex page layouts.
However, most systems continue to work on the document one page at a time. Because Scaling to longer documents can cause significant memory and speed limitations.
A neat solution is the new Unlimited OCR project launched by Baidu. Rather than having to store all the model’s output history, it is trained to store only what is important.
The result is a system that is capable of processing dozens of pages, perhaps even books, and has very stable memory space requirements.
The Challenge with Long Documents
Transformer-based OCR Models:
They produce text by looking at the whole history of previous tokens. This is great for short documents but is soon very expensive for longer documents:
- Memory growth (particularly KV cache)
- The longer the inference takes, the more attention computations are required.
- Higher GPU requirements
That is why traditional methods typically break up documents into separate pages and then join them at the end.
Learning from Human Behavior
The researchers were inspired by the way humans write or copy text.
If you are taking notes from a book with your handwriting, you don’t remember each previous sentence in your mind.
You primarily reference:
- The source material
- A small window for the recent output.
- Whatever the next part of your work is and whatever you’re doing.
As with all information, older information naturally fades.
In Unlimited OCR, a novel mechanism called Reference Sliding Window Attention (R-SWA).
That is used to mimic this selective memory approach.
How R-SWA Works in Unlimited OCR
Traditional setup:
- Current token attends to the entire generated history + input
With R-SWA:
Current token takes care of original document image + a small window of generated output from the past few seconds.
The model doesn’t have to keep all of the tokens that it has generated.
This keeps memory use almost the same, even for long documents, so performance stays stable.
Built on DeepSeek OCR
Image Source : huggingface.co/baidu
Unlimited OCR is based on DeepSeek OCR. The following are some of the most important architectural features:
1) Deep Encoder
Outputs a small number of visual tokens (e.g., 256) from high resolution document images (e.g., 1024×1024) with high compression
2) MoE Decoder
A MiE with 3B total parameters and only ~500M parameters activated during inference. All the attention layers are implemented using R-SWA to ensure efficiency.
Impressive Results of Unlimited OCR
Unlimited OCR does more than boost efficiency;
It reads text, formulas, and tables more accurately and understands the correct order of content better.
But even better is its long-horizon feature.
It has been successfully tested on multi page documents (up to 40+ pages) without having to reset memory between pages, which preserves the context throughout the document.
This can help process books, research papers, technical manuals, and reports more effectively.
As the length of the document increases, inference speed is almost constant.
And in long-output settings, they claimed about 35% higher throughput than the DeepSeek baseline.
Broader Implications of Unlimited OCR
Although it’s great for OCR, the R-SWA idea is extendable into areas beyond document processing.
The same attention, when used as a reference, can be helpful to:
- Speech recognition (keeping audio as reference while transcribing)
- Machine translation (source text always visible)
- Long-form multimodal tasks
It introduces a new idea for AI memory: instead of remembering everything, it remembers what matters and forgets what does not.
Conclusion
One of the core inefficiencies of Traditional OCR is addressed by Unlimited OCR.
With a smart architecture adjustment and human-inspired selective memory .
Baidu has developed a system that is more efficient and more accurate — and able to handle documents on scales that were previously clunky.
The model weights are open-sourced on Hugging Face (baidu/Unlimited-OCR) and can be used by the community to explore and build upon.
Ready for intelligent AI powered Commerce? Your journey starts at Webkul.
Be the first to comment.