无限OCR工作
Unlimited OCR Works
AI总结：
本文提出Unlimited OCR模型，通过Reference Sliding Window Attention机制，实现高效长文本处理，适用于OCR及ASR等任务。
摘要：
近期，端到端OCR模型，如DeepSeek OCR，再次将OCR推向聚光灯下。普遍观点认为，使用大语言模型（LLM）作为解码器可利用语言先验分布提升OCR性能。然而，其缺点同样明显：随着输出序列变长，累积的KV缓存会增加内存消耗并逐渐减慢生成速度。这与人类在长距离复制任务中效率无下降形成鲜明对比。在本技术报告中，我们提出Unlimited OCR，设计用于模拟人类解析工作记忆。以DeepSeek OCR为基准，我们用提出的Reference Sliding Window Attention（R-SWA）替换解码器中的所有注意力层，减少注意力计算成本同时保持整个解码过程中的KV缓存恒定。通过结合DeepSeek OCR编码器的高压缩率与恒定KV缓存设计，Unlimited OCR可在标准最大长度为32K的情况下，单次前向传递转录数十页文档。更重要的是，R-SWA是一种通用的解析注意力机制——除了OCR外，它同样适用于ASR、翻译等任务。代码和模型权重可在此http URL公开获取。
英文摘要：
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
