arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.03440cs.IR
收藏

利用基于实时检索增强生成(RAG)的系统改善对历史档案的访问

Improving Access to Historical Archives with Real-time RAG-based Systems

University of Lausanne(洛桑大学);Swiss Federal Institute of Technology (EPFL)(瑞士联邦理工学院(苏黎世联邦理工学院));National University of Singapore (NUS)(新加坡国立大学)

Stergios Konstantinidis, Hayman Lotfy, Alexis Erne, Faruk Zahiragic, Min-Yen Kan, Michalis Vlachos

AI总结:

研究针对数字化历史档案访问难题,提出集成大语言模型的端到端框架,含OCR优化模块和语义检索重排管道,实验表明能减少OCR错误并提升检索效果,使档案系统更具交互性和语义可搜索性。

摘要:

数字化历史档案是大型异构文化遗产库,但访问方法面临如光学字符识别(OCR)输出有噪声和基于关键词检索僵化等挑战,限制检索质量。本文提出将大语言模型集成到档案处理管道中的端到端档案处理和检索框架,介绍两个核心组件,评估显示能减少OCR错误并改善下游信息检索。

英文摘要:

Digitized historical archives are large, heterogeneous cultural heritage repositories, but access methods for such archives face challenges such as noisy optical character recognition (OCR) output and rigid keyword-based retrieval, which limit retrieval quality. In this work, we present an end-to-end archival processing and retrieval framework that integrates large language models (LLMs) into the archival pipeline. Our system introduces two core components: (i) an LLM-based OCR refinement module that improves text quality, and (ii) a semantic retrieval and cross-encoder reranking pipeline supporting natural-language question answering via retrieval-augmented generation (RAG). Our evaluations are done on a historical archival dataset of 500,000 Swiss newspaper segments spanning over three centuries (1762 to 2001). Experiments are conducted across 384 natural-language test queries. Our results highlight that LLM refinements reduce OCR errors by up to 44.52% (CER) and 60.95% (WER). More importantly, this is accompanied by downstream information retrieval improvements. Compared to traditional keyword baselines, our reranking pipeline increases NDCG@10 by 31.9% (from 65.99% to 87.05%) and achieves statistically significant gains in both answer correctness and context relevance. These results demonstrate that integrating LLMs with established document processing and retrieval pipelines can elevate digital libraries from static repositories to interactive, semantically searchable archival systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑