基于VLM引导的语义-几何推理的任务感知双臂 affordance 预测
Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning
AI总结：
本文提出一种任务感知双臂 affordance 预测框架，通过VLM实现跨物体类别和任务描述的泛化，融合多视角RGB-D数据生成全局6自由度抓取候选，结合语义和几何过滤提升任务成功率。
摘要：
双臂操作需要推理与物体交互的位置及哪只手臂执行每个动作，这是一个需要几何和语义理解的联合affordance定位和手臂分配问题。现有方法要么将affordance预测视为粗粒度部分分割，要么依赖几何启发式方法分配手臂，无法联合推理任务相关的接触区域和手臂分配。我们重新将双臂操作视为联合affordance定位和手臂分配问题，并提出一种层次化框架，利用Vision-Language Model (VLM)实现跨物体类别和任务描述的泛化，无需类别特定训练。我们的方法将多视角RGB-D观测融合为一致的3D场景表示，生成全局6-DoF抓取候选，然后通过查询VLM获取任务相关的affordance区域和手臂分配，从而确保几何有效性的同时尊重任务语义。我们在双臂平台上评估了九个真实世界操作任务，涵盖四个类别：平行操作、协调稳定、工具使用和人手交接。我们的方法在任务导向抓取上比几何和语义基线方法表现更优，证明了显式语义推理affordance和手臂分配有助于在无结构环境中实现可靠的双臂操作。
英文摘要：
Bimanual manipulation requires reasoning about where to interact with an object and which arm should perform each action, a joint affordance localization and arm allocation problem that geometry-only planners cannot resolve without semantic understanding of task intent. Existing approaches either treat affordance prediction as coarse part segmentation or rely on geometric heuristics for arm assignment, failing to jointly reason about task-relevant contact regions and arm allocation. We reframe bimanual manipulation as a joint affordance localization and arm allocation problem and propose a hierarchical framework for task-aware bimanual affordance prediction that leverages a Vision-Language Model (VLM) to generalize across object categories and task descriptions without requiring category-specific training. Our approach fuses multi-view RGB-D observations into a consistent 3D scene representation and generates global 6-DoF grasp candidates, which are then spatially and semantically filtered by querying the VLM for task-relevant affordance regions on each object, as well as for arm allocation to the individual objects, thereby ensuring geometric validity while respecting task semantics. We evaluate our method on a dual-arm platform across nine real-world manipulation tasks spanning four categories: parallel manipulation, coordinated stabilization, tool use, and human handover. Our approach achieves consistently higher task success rates than geometric and semantic baselines for task-oriented grasping, demonstrating that explicit semantic reasoning over affordances and arm allocation helps enable reliable bimanual manipulation in unstructured environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
