基于 affordance 的粗到细探索用于开放词汇移动操控中的基座放置
Affordance-Guided Coarse-to-Fine Exploration for Base Placement in Open-Vocabulary Mobile Manipulation
AI总结：
本文提出了一种基于 affordance 的粗到细探索方法，通过结合视觉语言模型的语义理解和几何可行性，提升开放词汇移动操控中基座放置的成功率。
摘要：
在开放词汇移动操控（OVMM）中，任务的成功往往取决于为机器人选择合适的基座放置位置。现有方法通常导航到基于接近性的区域而不考虑 affordances，导致频繁的操控失败。我们提出了基于 affordance 的粗到细探索，这是一种零样本框架，用于基座放置，它结合了视觉语言模型（VLMs）的语义理解与几何可行性，通过迭代优化过程。我们的方法构建了跨模态表示，即 affordance RGB 和障碍地图+，以对齐语义与空间上下文。这使推理能够超越 RGB 感知的自体中心限制。为了确保交互由与任务相关的 affordances 引导，我们利用粗语义先验从 VLMs 中引导搜索向与任务相关的区域，并通过几何约束细化放置，从而减少收敛到局部最优的风险。在五个多样化的开放词汇移动操控任务上评估，我们的系统实现了 85% 的成功率，显著优于经典几何规划器和基于 VLM 的方法。这展示了对 affordance 意识和多模态推理在 OVMM 中通用、受指令条件的规划中的前景。
英文摘要：
In open-vocabulary mobile manipulation (OVMM), task success often hinges on the selection of an appropriate base placement for the robot. Existing approaches typically navigate to proximity-based regions without considering affordances, resulting in frequent manipulation failures. We propose Affordance-Guided Coarse-to-Fine Exploration, a zero-shot framework for base placement that integrates semantic understanding from vision-language models (VLMs) with geometric feasibility through an iterative optimization process. Our method constructs cross-modal representations, namely Affordance RGB and Obstacle Map+, to align semantics with spatial context. This enables reasoning that extends beyond the egocentric limitations of RGB perception. To ensure interaction is guided by task-relevant affordances, we leverage coarse semantic priors from VLMs to guide the search toward task-relevant regions and refine placements with geometric constraints, thereby reducing the risk of convergence to local optima. Evaluated on five diverse open-vocabulary mobile manipulation tasks, our system achieves an 85% success rate, significantly outperforming classical geometric planners and VLM-based methods. This demonstrates the promise of affordance-aware and multimodal reasoning for generalizable, instruction-conditioned planning in OVMM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
