Traditional Retrieval-Augmented Generation (RAG) approaches generally assume that retrieval and generation occur on powerful servers removed from the end user. While this reduces local hardware constraints, it introduces significant drawbacks: privacy concerns regarding data access, recurring maintenance and storage costs, increased latency, and the necessity of an internet connection. On-device RAG addresses these challenges by executing the entire pipeline locally, making it ideal for querying sensitive personal information such as financial documents, contact details, and medical history. However, on-device deployment necessitates a delicate balance between limited memory and disk space. Specifically, the context size provided to the generative model must be restricted to manage KV cache and attention memory usage, while the size of stored embeddings must be minimized to preserve disk space. In this work, we propose a unified model that compresses the RAG context and utilizes the same representations for retrieval. This approach minimizes disk utilization compared to using separate representations, while significantly reducing the context size required for generation. With an average of 1/10 of the context, our model matches the performance of a traditional RAG reader without increasing storage requirements compared to a multi-vector retrieval model. This approach represents the first model to unify retrieval and context compression using a shared model and representation. We believe this work will inspire further consolidation of distinct models to optimize on-device performance.
Structural Segmentation for Brazilian Legal Document Retrieval
Câmara dos Deputados(众议院);Senado Federal(联邦参议院);Faculdade de Engenharia Elétrica e da Computação, Universidade Estadual de Campinas (UNICAMP)(坎皮纳斯州立大学电气与计算机工程学院);Centro de Ciências e Tecnologia, Universidade Federal do Cariri (UFCA)(卡拉里联邦大学科学与技术中心)
Tools are under-documented: Simple Document Expansion Boosts Tool Retrieval
Shanghai Jiao Tong University(上海交通大学);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative(宁波空间智能与数字孪生重点实验室);Institute of Digital Twin, Eastern Institute of Technology, Ningbo(宁波东方理工大学数字孪生研究所)
Dual Skipping Guidance for Document Retrieval with Learned Sparse Representations
本文提出了一种双跳转指导计划与混合评分,以加速文档检索,使用学习稀疏表示,同时仍然提供了良好的相关性。该方案使用词汇BM 25和学习的神经项权重来分别绑定和组合候选文档的排名得分,以用于跳过和最终排名,并在倒排索引遍历期间保持两个top-k阈值。本文评估的时间效率和排名的相关性所提出的计划在搜索MS MARCO TREC数据集。
The 3rd Place Solution of CCIR CUP 2025: A Framework for Retrieval-Augmented Generation in Multi-Turn Legal Conversation
CAS Key Lab of Network Data Science(中国科学院网络数据重点实验室);Technology, ICT, CAS State Key Laboratory of AI Safety University of Chinese Academy of Sciences Beijing China(中国科学院计算技术研究所人工智能安全国家重点实验室中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing China(中国科学院计算技术研究所中国科学院大学);CAS Key Lab of Network Data(中国科学院网络数据重点实验室);State Key Laboratory of AI Safety(人工智能安全国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
本文提出一个用于多轮法律对话的检索增强生成框架,结合信息检索与大型语言模型,基于法律法规回答用户问题,并在CCIR CUP 2025中取得第三名。
Simplified Longitudinal Retrieval Experiments: A Case Study on Query Expansion and Document Boosting
TH Köln - University of Applied Sciences, Cologne, Germany(科隆应用技术大学);Friedrich-Schiller-Universität Jena(耶拿弗里德里希·席勒大学);Radboud Universiteit Nijmegen(奈梅亨拉德堡德大学);University of Kassel(卡塞尔大学)
A Survey of Long-Document Retrieval in the PLM and LLM Era
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Center for Information and Language Processing (CIS), LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)
A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation
School of Information Management Sun Yat-sen University Guangzhou, China;School of Information Management Sun Yat-sen University Guangzhou, China corresponding author
Cross-Lingual Training with Dense Retrieval for Document Retrieval
David R. Cheriton School of Computer Science, University of Waterloo(滑铁卢大学戴维·R·切里顿计算机科学学院);Department of Computer Science and Engineering, Penn State University(宾夕法尼亚州立大学计算机科学与工程系)