arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.04700cs.LGcs.AIcs.DCcs.IR
收藏

在边缘存内计算架构上稳健实现检索增强生成

Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures

University of Notre Dame(圣母大学);University at Buffalo–SUNY(纽约州立大学布法罗分校)

Ruiyang Qin, Zheyu Yan, Dewen Zeng, Zhenge Jia, Dancheng Liu, Jianbo Liu, Zhi Zheng, Ningyuan Cao, Kai Ni, Jinjun Xiong, Yiyu Shi

AI总结:

针对边缘设备上 RAG 面临的重复搜索延迟与用户数据可扩展性矛盾,本文提出 RoCR 框架,利用 CiM 原位矩阵计算、对比学习和噪声感知训练加速并增强 RAG。

摘要:

部署在边缘设备上的大语言模型(LLM)通过微调和更新部分参数进行学习。尽管这类学习方法可以经过优化以降低资源占用,但总体所需资源对边缘设备而言仍是沉重负担。相比之下,检索增强生成(RAG)是一种资源高效的 LLM 学习方法,能够在不更新模型参数的情况下提升 LLM 生成内容的质量。然而,基于 RAG 的 LLM 在每次用户与 LLM 交互时都可能对画像数据进行重复搜索。随着用户数据不断积累,这种搜索会导致显著延迟。传统降低延迟的做法会限制所保存用户数据的规模,从而在用户数据持续增长时降低 RAG 的可扩展性。如何使 RAG 在边缘设备上摆脱延迟和可扩展性约束,仍是一个开放问题。本文提出一种通过存内计算(CiM)架构加速 RAG 的新颖框架。该框架通过在存储器内部执行原位计算来加速矩阵乘法,同时避免计算单元与存储器之间代价高昂的数据传输。我们的框架 Robust CiM-backed RAG(RoCR)采用新颖的基于对比学习的训练方法和噪声感知训练,使 RAG 能够利用 CiM 高效搜索画像数据。据我们所知,这是首个利用 CiM 加速 RAG 的工作。

英文摘要:

Large Language Models (LLMs) deployed on edge devices learn through fine-tuning and updating a certain portion of their parameters. Although such learning methods can be optimized to reduce resource utilization, the overall required resources remain a heavy burden on edge devices. Instead, Retrieval-Augmented Generation (RAG), a resource-efficient LLM learning method, can improve the quality of the LLM-generated content without updating model parameters. However, the RAG-based LLM may involve repetitive searches on the profile data in every user-LLM interaction. This search can lead to significant latency along with the accumulation of user data. Conventional efforts to decrease latency result in restricting the size of saved user data, thus reducing the scalability of RAG as user data continuously grows. It remains an open question: how to free RAG from the constraints of latency and scalability on edge devices? In this paper, we propose a novel framework to accelerate RAG via Computing-in-Memory (CiM) architectures. It accelerates matrix multiplications by performing in-situ computation inside the memory while avoiding the expensive data transfer between the computing unit and memory. Our framework, Robust CiM-backed RAG (RoCR), utilizing a novel contrastive learning-based training method and noise-aware training, can enable RAG to efficiently search profile data with CiM. To the best of our knowledge, this is the first work utilizing CiM to accelerate RAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑