arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.05486cs.CL
收藏

语言模型作为规划器和形式化工具的约束下

Language Model as Planner and Formalizer under Constraints

Drexel University(德雷塞尔大学);Brown University(布朗大学)

Cassie Huang, Stuti Mohan, Ziyi Yang, Stefanie Tellex, Li Zhang

AI总结:

研究通过引入细粒度自然语言约束,揭示语言模型在规划任务中的性能下降,指出其鲁棒性不足及未来研究方向。

摘要:

语言模型已被广泛应用于规划,要么作为规划器生成动作序列,要么作为形式化工具表示规划领域和问题。然而,这两种方法都依赖于包含仅通用和简单环境规范的标准基准,导致对语言模型规划能力的高估和下游任务的安全隐患。我们通过在广泛使用的规划基准中增加人工标注、细粒度和丰富的自然语言约束,涵盖四个正式定义的类别,发现引入单句约束可使性能减半,表明当前语言模型的鲁棒性不足,并为未来研究提供了方向。

英文摘要:

LLMs have been widely used in planning, either as planners to generate action sequences end-to-end, or as formalizers to represent the planning domain and problem in a formal language that can derive plans deterministically. However, both lines of work rely on standard benchmarks that include only generic and simplistic environmental specifications, leading to potential overestimation of the planning ability of LLMs and safety concerns in downstream tasks. We bridge this gap by augmenting widely used planning benchmarks with manually annotated, fine-grained, and rich natural language constraints spanning four formally defined categories. Over 4 state-of-the-art reasoning LLMs, 4 formal languages, and 4 datasets, we show that the introduction of one-sentence constraints consistently halves performance, indicating current LLMs' lack of robustness and an avenue for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑