arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03215cs.CLcs.LG
收藏

缓存到缓存:大型语言模型之间的直接语义通信

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Tsinghua University(清华大学);Infinigence AI;The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);SLAI;Shanghai AI Laboratory(上海人工智能实验室)

Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang

AI总结:

缓存到缓存通过直接语义通信提升大型语言模型的性能和效率,实现比文本通信更高的准确率和更低的延迟。

摘要:

多LLM系统利用不同大型语言模型的互补优势,实现单个模型无法达到的性能和效率提升。在现有设计中,LLM通过文本进行交流,迫使内部表示转换为输出标记序列。这一过程不仅丢失了丰富的语义信息,还导致逐个标记生成的延迟。受这些限制的启发,我们提出问题:LLM能否超越文本进行交流?Oracle实验显示,丰富KV缓存的语义可以提高响应质量而不增加缓存大小,支持KV缓存作为模型间通信的有效介质。因此,我们提出了缓存到缓存(C2C),一种新的直接语义通信范式。C2C使用神经网络将源模型的KV缓存与目标模型的缓存进行投影和融合,以实现直接的语义转移。一个可学习的门控机制选择受益于缓存通信的目标层。与文本通信相比,C2C利用了两个模型的深度、专业语义,同时避免了显式的中间文本生成。实验表明,C2C的平均准确率比单个模型高出6.4-14.2%。它进一步比文本通信范式高出约3.1-5.4%,同时在延迟上平均提升2.5倍。我们的代码可在https://github.com/thu-nics/C2C获取。

英文摘要:

Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑