arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.22993cs.LGcs.CL
收藏

语言模型能否在上下文中组合技能?

Can Language Models Compose Skills In-Context?

The University of Hong Kong(香港大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Zidong Liu, Zhuoyan Xu, Zhenmei Shi, Yingyu Liang

AI总结:

本研究探究语言模型在上下文中组合基本技能以完成复合任务的能力,发现简单示例可能损害性能,理论分析强调示例与组合步骤对齐的重要性,并提出改进方法。

摘要:

从简单任务中组合基本技能以完成复合任务,对于现代智能系统至关重要。我们研究了语言模型在上下文中组合技能的能力,以执行结合了上下文示例中所展示的基本技能的复合任务。这比标准设置更具挑战性,因为在标准设置中,技能及其组合可以在训练中学习。我们利用旨在探测组合能力的语言和逻辑任务,对各种具有代表性的开源语言模型进行了系统性实验。结果表明,简单任务示例可能对性能产生令人惊讶的负面影响,因为模型通常难以正确识别和组装技能,即使有思维链示例也是如此。理论分析进一步表明,将示例与组合中的相应步骤对齐至关重要。这为探测任务启发了一种方法,其改进的性能为我们的见解提供了积极支持。

英文摘要:

Composing basic skills from simple tasks to accomplish composite tasks is crucial for modern intelligent systems. We investigate the in-context composition ability of language models to perform composite tasks that combine basic skills demonstrated in in-context examples. This is more challenging than the standard setting, where skills and their composition can be learned in training. We conduct systematic experiments on various representative open-source language models, utilizing linguistic and logical tasks designed to probe composition abilities. The results reveal that simple task examples can have a surprising negative impact on the performance, because the models generally struggle to recognize and assemble the skills correctly, even with Chain-of-Thought examples. Theoretical analysis further shows that it is crucial to align examples with the corresponding steps in the composition. This inspires a method for the probing tasks, whose improved performance provides positive support for our insights.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑