arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.02043cs.CL
收藏

面向工具使用型语言模型的简洁且精确的上下文压缩

Concise and Precise Context Compression for Tool-Using Language Models

Harbin Institute of Technology(哈尔滨工业大学);Huawei Technologies Co., Ltd(华为技术有限公司)

Yang Xu, Yunlong Feng, Honglin Mu, Yutai Hou, Yitong Li, Xinghao Wang, Wanjun Zhong, Zhongyang Li, Dandan Tu, Qingfu Zhu, Min Zhang, Wanxiang Che

AI总结:

针对工具文档冗长导致的上下文占用问题,提出选择性压缩与块压缩策略,在 API-Bank 和 APIBench 上最高 16 倍压缩时仍达到接近上界基线的性能。

摘要:

通过阅读上下文中的文档,工具使用型语言模型能够利用外部工具动态扩展其能力。代价是,每当模型需要使用工具时,我们都必须输入冗长的文档,这既占用输入窗口,又会拖慢解码过程。鉴于通用压缩方面的进展,软上下文压缩是缓解该问题的一种合适方法。然而,在压缩工具文档时,现有方法存在关键信息丢失(具体表现为工具/参数名称错误)以及难以根据文档长度调整压缩序列长度的弱点。为解决这些问题,我们提出两种策略,将工具文档压缩为简洁且精确的摘要序列,以供工具使用型语言模型使用。1)选择性压缩策略通过有意将关键信息保留为原始文本 token,缓解关键信息丢失问题。2)块压缩策略将工具文档划分为短片段,随后采用定长压缩模型实现变长压缩。该策略有助于灵活调整压缩率。在 API-Bank 和 APIBench 上的结果表明,在最高 16 倍压缩率下,我们的方法取得了与上界基线相当的性能。

英文摘要:

Through reading the documentation in the context, tool-using language models can dynamically extend their capability using external tools. The cost is that we have to input lengthy documentation every time the model needs to use the tool, occupying the input window as well as slowing down the decoding process. Given the progress in general-purpose compression, soft context compression is a suitable approach to alleviate the problem. However, when compressing tool documentation, existing methods suffer from the weaknesses of key information loss (specifically, tool/parameter name errors) and difficulty in adjusting the length of compressed sequences based on documentation lengths. To address these problems, we propose two strategies for compressing tool documentation into concise and precise summary sequences for tool-using language models. 1) Selective compression strategy mitigates key information loss by deliberately retaining key information as raw text tokens. 2) Block compression strategy involves dividing tool documentation into short chunks and then employing a fixed-length compression model to achieve variable-length compression. This strategy facilitates the flexible adjustment of the compression ratio. Results on API-Bank and APIBench show that our approach reaches a performance comparable to the upper-bound baseline under up to 16x compression ratio.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑