arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.19954cs.AIcs.CLcs.LG
收藏

Transformer 在验证计划能力上的表现

On the Ability of Transformers to Verify Plans

Saarland University(萨尔兰大学);Australian National University(澳大利亚国立大学);University of Toulouse / LAAS-CNRS(图卢兹大学 / LAAS-CNRS)

Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thiébaux, Michael Hahn

AI总结:

研究分析了解码器-only模型验证计划的能力,提出C*-RASP框架以保证序列长度和词汇量增长时的泛化能力,发现某些经典规划领域中Transformer能有效学习验证长计划。

摘要:

Transformers在AI规划任务中表现出不一致的成功,对何时应期望泛化能力的理论理解有限。我们通过分析解码器-only模型验证给定计划是否正确解决给定规划实例的能力,填补了这一空白。为分析测试时对象数量增长的通用情况,我们引入C*-RASP,这是C-RASP的扩展,旨在为Transformer在序列长度和词汇量同时增长时建立长度泛化保证。我们的结果识别出一大类经典规划领域,其中Transformer能证明性地学习验证长计划,并揭示显著影响长度可泛化解学习性的结构性质。实证实验证实了我们的理论。

英文摘要:

Transformers have shown inconsistent success in AI planning tasks, and theoretical understanding of when generalization should be expected has been limited. We take important steps towards addressing this gap by analyzing the ability of decoder-only models to verify whether a given plan correctly solves a given planning instance. To analyse the general setting where the number of objects -- and thus the effective input alphabet -- grows at test time, we introduce C*-RASP, an extension of C-RASP designed to establish length generalization guarantees for transformers under the simultaneous growth in sequence length and vocabulary size. Our results identify a large class of classical planning domains for which transformers can provably learn to verify long plans, and structural properties that significantly affects the learnability of length generalizable solutions. Empirical experiments corroborate our theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑