arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.01149cs.IR
收藏

ModernVBERT: 向更小的视觉文档检索器迈进

ModernVBERT: Towards Smaller Visual Document Retrievers

Illuin Technology(Illuin 技术);EPFL(洛桑联邦理工学院);CentraleSupélec, Paris-Saclay(巴黎萨克雷中央理工-高等电力学院)

Paul Teiletche, Quentin Macé, Max Conti, Antonio Loison, Gautier Viaud, Pierre Colombo, Manuel Faysse

AI总结:

ModernVBERT通过优化视觉文档检索模型,实现了在文档检索任务中性能优于更大模型的高效轻量级模型。

摘要:

从大规模文档集合中检索特定信息是现代AI在工业中的常见应用案例,尤其由于检索增强生成(RAG)系统的人气。尽管神经文档检索模型历史上一直只在文本空间中运行,但视觉文档检索(VDR)模型——大型视觉语言解码器重新利用作嵌入模型,可以直接使用页面截图作为输入——由于其在性能和索引延迟方面的提升而日益流行。在本工作中,我们表明,尽管这种重新利用生成模型的方法成本效益高,但会成为检索性能的瓶颈。通过受控实验,我们重新审视了整个训练流程,并建立了一套原则性的方法来改进视觉文档检索模型。我们特别测量了注意力遮蔽、图像分辨率、模态对齐数据制度以及后期交互中心对比目标的影响,这些因素逐渐成为性能的关键因素。基于这些见解,我们发布了ModernVBERT,一个具有2.5亿参数的视觉语言编码器,当在文档检索任务上微调时,其性能优于最近的模型,最高可达10倍,使在廉价CPU硬件上实现高效推理成为可能,并大大减少延迟和成本,同时保持强劲的性能。模型、代码和数据可在https://huggingface.co/ModernVBERT上获得。

英文摘要:

Retrieving specific information from a large corpus of documents is a prevalent industrial use case of modern AI, notably due to the popularity of Retrieval-Augmented Generation (RAG) systems. Although neural document retrieval models have historically operated exclusively in the text space, Visual Document Retrieval (VDR) models - large vision-language decoders repurposed as embedding models which directly work with page screenshots as inputs - are increasingly popular due to the performance and indexing latency gains they offer. In this work, we show that, while cost-efficient, this approach of repurposing generative models bottlenecks retrieval performance. Through controlled experiments, we revisit the entire training pipeline, and establish a principled recipe for improving visual document retrieval models. We notably measure the impact of attention masking, image resolution, modality alignment data regimes, and late interaction centered contrastive objectives which emerge as central performance factors. Building on these insights, we release ModernVBERT, a compact 250M-parameter vision-language encoder that outperforms recent models up to 10 times larger when fine-tuned on document retrieval tasks, enabling efficient inference on cheap CPU hardware and greatly reducing latency and costs while maintaining strong performance. Models, code and data are available at https://huggingface.co/ModernVBERT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑