通过反事实扰动将语言规划锚定在演示中
Grounding Language Plans in Demonstrations Through Counterfactual Perturbations
AI总结：
针对大语言模型常识推理难锚定物理领域的问题，提出基于反事实扰动的框架，借模式族衔接高层语言与低层轨迹，通过扰动演示训练可微网络，习得无密集标注的锚定分类器，提升模仿学习的可解释性与反应性。
摘要：
将大语言模型（LLMs）的常识推理锚定到物理领域，仍是具身人工智能领域一个关键且未解决的问题。现有研究多聚焦于直接利用LLM在符号空间中进行规划，而本研究则借助LLM引导对多步演示中隐含的任务结构与约束的搜索。具体而言，我们借鉴操作规划领域的模式族（mode families）概念——模式族是按特定运动约束对机器人构型进行的分组——将其作为LLM高层语言表征与机器人低层物理轨迹之间的抽象层。通过对少量人类演示加入合成扰动后重放，我们生成了覆盖演示状态空间的额外成功执行样本，以及导致任务失败的反事实样本。我们的基于解释的学习框架训练了一个端到端可微神经网络，用于从失败轨迹中预测成功轨迹，且作为副产物，无需密集标注就能学习到将低层状态与图像锚定到模式族的分类器。所学得的锚定分类器还可进一步用于以可解释的方式，将语言规划转化为物理领域中的反应式策略。我们通过2D导航任务、仿真及真实机器人操作任务证明，该方法提升了模仿学习的可解释性与反应性。项目主页：https://yanweiw.github.io/glide
英文摘要：
Grounding the common-sense reasoning of Large Language Models (LLMs) in physical domains remains a pivotal yet unsolved problem for embodied AI. Whereas prior works have focused on leveraging LLMs directly for planning in symbolic spaces, this work uses LLMs to guide the search of task structures and constraints implicit in multi-step demonstrations. Specifically, we borrow from manipulation planning literature the concept of mode families, which group robot configurations by specific motion constraints, to serve as an abstraction layer between the high-level language representations of an LLM and the low-level physical trajectories of a robot. By replaying a few human demonstrations with synthetic perturbations, we generate coverage over the demonstrations' state space with additional successful executions as well as counterfactuals that fail the task. Our explanation-based learning framework trains an end-to-end differentiable neural network to predict successful trajectories from failures and as a by-product learns classifiers that ground low-level states and images in mode families without dense labeling. The learned grounding classifiers can further be used to translate language plans into reactive policies in the physical domain in an interpretable manner. We show our approach improves the interpretability and reactivity of imitation learning through 2D navigation and simulated and real robot manipulation tasks. Website: https://yanweiw.github.io/glide
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
