arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.20083cs.ETcs.AR
收藏

CQ-CiM: 面向鲁棒CiM基于检索的嵌入形状化

CQ-CiM: Hardware-Aware Embedding Shaping for Robust CiM-Based Retrieval

Villanova University(维拉诺瓦大学);Fraunhofer IPMS(弗劳恩霍夫IPMS研究所);NDSU(北达科他州立大学);University of Notre Dame(圣母大学);TU Braunschweig(布伦瑞克工业大学)

Xinzhao Li, Alptekin Vardar, Franz Müller, Navya Goli, Umamaheswara Rao Tida, Kai Ni, Xiaobo Sharon Hu, Thomas Kämpfe, Ruiyang Qin

AI总结:

本文提出CQ-CiM框架,通过联合学习压缩与量化生成兼容多种CiM设计的低比特嵌入,解决RAG中高精度高维嵌入与CiM低精度低维约束的矛盾。

摘要:

在边缘设备上部署检索增强生成(RAG)的需求很高,但受限于传统架构中大规模数据移动和计算的延迟。计算存内(CiM)架构通过在交叉bar结构内直接执行向量搜索来解决这一瓶颈。然而,CiM用于RAG受限于一个根本的『表示差距』,因为高精度、高维嵌入与CiM的低精度、低维数组约束不兼容。这种差距被CiM实现的多样性(例如SRAM、ReRAM、FeFET)进一步加剧,每种实现都有独特的设计(例如2位单元、512x512数组)。因此,RAG数据必须粗暴地重新塑造以适应目标实现。当前的数据塑造方法分别处理维度和精度,这会降低数据保真度。这不仅抵消了CiM为RAG带来的优势,还使硬件设计者困惑,不清楚失败是由于电路设计还是降质的输入数据。因此,CiM的采用仍然有限。在本文中,我们介绍了CQ-CiM,一种统一的、面向硬件的数据塑造框架,通过联合学习压缩和量化生成兼容多种CiM设计的低比特嵌入。据我们所知,这是首次针对RAG的全面CiM使用进行数据塑造的工作。

英文摘要:

Deploying Retrieval-Augmented Generation (RAG) on edge devices is in high demand, but is hindered by the latency of massive data movement and computation on traditional architectures. Compute-in-Memory (CiM) architectures address this bottleneck by performing vector search directly within their crossbar structure. However, CiM's adoption for RAG is limited by a fundamental ``representation gap,'' as high-precision, high-dimension embeddings are incompatible with CiM's low-precision, low-dimension array constraints. This gap is compounded by the diversity of CiM implementations (e.g., SRAM, ReRAM, FeFET), each with unique designs (e.g., 2-bit cells, 512x512 arrays). Consequently, RAG data must be naively reshaped to fit each target implementation. Current data shaping methods handle dimension and precision disjointly, which degrades data fidelity. This not only negates the advantages of CiM for RAG but also confuses hardware designers, making it unclear if a failure is due to the circuit design or the degraded input data. As a result, CiM adoption remains limited. In this paper, we introduce CQ-CiM, a unified, hardware-aware data shaping framework that jointly learns Compression and Quantization to produce CiM-compatible low-bit embeddings for diverse CiM designs. To the best of our knowledge, this is the first work to shape data for comprehensive CiM usage on RAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑