arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.07975cs.ROcs.AI
收藏

可执行分析概念:连接视觉语言模型洞察与精确操作的缺失环节

Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation

Mingyang Sun, Jiude Wei, Qichen He, Donglin Wang, Cewu Lu, Jianhua Sun

AI总结:

GRACE框架通过可执行分析概念(EAC)将视觉语言模型的高层理解与低层物理控制相连接,实现精确且可泛化的机器人操作,并在多种铰接物体上展现出零样本泛化能力。

摘要:

在非结构化环境中使机器人能够执行精确且泛化的操作,仍然是具身智能领域的一项基本挑战。尽管视觉语言模型(VLMs)在语义推理和任务规划方面展现了卓越的能力,但其高层理解与实际操作所需的精确物理执行之间仍存在显著差距。为弥合这一“语义到物理”的鸿沟,我们提出了GRACE,一种新颖的框架,通过可执行分析概念(EAC)——即编码物体可供性、几何约束和操作语义的数学定义蓝图——来夯实基于VLM的推理。我们的方法整合了一个结构化的策略脚手架流水线,将自然语言指令和视觉信息转化为实例化的EAC,并从中推导出抓取姿态、力方向和规划物理上可行的运动轨迹以供机器人执行。因此,GRACE在高层指令理解与低层机器人控制之间提供了统一且可解释的接口,通过语义-物理接地有效实现了精确且可泛化的操作。大量实验表明,GRACE在模拟和真实环境中的多种铰接物体上均实现了强大的零样本泛化能力,且无需任务特定训练。

英文摘要:

Enabling robots to perform precise and generalized manipulation in unstructured environments remains a fundamental challenge in embodied AI. While Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning, a significant gap persists between their high-level understanding and the precise physical execution required for real-world manipulation. To bridge this "semantic-to-physical" gap, we introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts (EAC)-mathematically defined blueprints that encode object affordances, geometric constraints, and semantics of manipulation. Our approach integrates a structured policy scaffolding pipeline that turn natural language instructions and visual information into an instantiated EAC, from which we derive grasp poses, force directions and plan physically feasible motion trajectory for robot execution. GRACE thus provides a unified and interpretable interface between high-level instruction understanding and low-level robot control, effectively enabling precise and generalizable manipulation through semantic-physical grounding. Extensive experiments demonstrate that GRACE achieves strong zero-shot generalization across a variety of articulated objects in both simulated and real-world environments, without requiring task-specific training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑