ArtiBench 和 ArtiBrain：可泛化的视觉-语言操纵基准测试

中文摘要 AI 辅助

交互式可操纵操作需要长时间、多步骤与器具的交互，同时保持物理一致性。现有的视觉-语言和扩散基政策在部分、实例和类别之间难以泛化。我们首先引入ArtiBench，一个涵盖厨房、存储、办公室和工具环境的五级基准。ArtiBench能够从跨部分和跨实例变化到长周期多物体任务进行结构化评估，揭示了可操纵物体操作的核心泛化挑战。在此基准基础上，我们提出ArtiBrain，一个模块化框架，将高层推理与自适应低层控制统一起来。ArtiBrain使用基于VLM的任务推理器（GPT-4.1）分解和验证子目标，并采用结合几何感知的关键帧执行与可操作性引导的扩散的混合控制器。一个可操作性记忆库持续积累成功的执行片段，并将部分层面的可操作性可操作性传播到未见过的可操纵部分和配置。在ArtiBench上的广泛实验表明，我们的ArtiBrain在鲁棒性和泛化性上显著优于最先进的多模态和扩散基方法。代码和数据集将在接受后发布。

英文摘要

Interactive articulated manipulation requires long-horizon, multi-step interactions with appliances while maintaining physical consistency. Existing vision-language and diffusion-based policies struggle to generalize across parts, instances, and categories. We first introduce ArtiBench, a five-level benchmark covering kitchen, storage, office, and tool environments. ArtiBench enables structured evaluation from cross-part and cross-instance variation to long-horizon multi-object tasks, revealing the core generalization challenges of articulated object manipulation. Building on this benchmark, we propose ArtiBrain, a modular framework that unifies high-level reasoning with adaptive low-level control. ArtiBrain uses a VLM-based Task Reasoner (GPT-4.1) to decompose and validate subgoals, and employs a Hybrid Controller that combines geometry-aware keyframe execution with affordance-guided diffusion for precise and interpretable manipulation. An Affordance Memory Bank continually accumulates successful execution episodes and propagates part-level actionable affordances to unseen articulated parts and configurations. Extensive experiments on ArtiBench show that our ArtiBrain significantly outperforms state-of-the-art multimodal and diffusion-based methods in robustness and generalization. Code and dataset will be released upon acceptance.

发表机构

• University of Science and Technology of China（中国科学技术大学）
• Technical University of Munich（慕尼黑技术大学）

机构由 AI 辅助整理，请以论文原文为准。