SeqAfford：基于多模态大语言模型的序列3D可供性推理
SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model
AI总结：
针对现有3D可供性分割仅支持单任务、无法处理长程复杂意图的问题，研究者提出序列3D可供性推理任务，构建含18万样本的基准，提出SeqAfford模型实现多模态推理与稠密分割，性能更优且具备开放世界泛化能力。
摘要：
3D可供性分割旨在将人类指令与3D物体的可触碰区域关联，以支持具身操作。现有研究通常遵循单物体、单可供性范式，即每种可供性类型或明确指令严格对应特定可供性区域，无法处理长程任务，也难以主动推理往往隐含序列可供性的复杂用户意图。本文提出序列3D可供性推理任务，通过从繁杂的用户意图出发进行推理，再将其分解为一系列分割图，拓展了传统范式。为此，我们构建了首个基于指令的可供性分割基准，涵盖单可供性与序列可供性推理，包含18万条指令-点云对。基于该基准，我们提出SeqAfford模型，为3D多模态大语言模型赋予额外的可供性分割能力，在统一框架中实现结合世界知识的推理与细粒度可供性定位。我们进一步引入多粒度语言-点云集成模块以支持3D稠密预测。大量实验评估表明，该模型性能优于现有成熟方法，且具备序列推理能力与开放世界泛化性。
英文摘要：
3D affordance segmentation aims to link human instructions to touchable regions of 3D objects for embodied manipulations. Existing efforts typically adhere to single-object, single-affordance paradigms, where each affordance type or explicit instruction strictly corresponds to a specific affordance region and are unable to handle long-horizon tasks. Such a paradigm cannot actively reason about complex user intentions that often imply sequential affordances. In this paper, we introduce the Sequential 3D Affordance Reasoning task, which extends the traditional paradigm by reasoning from cumbersome user intentions and then decomposing them into a series of segmentation maps. Toward this, we construct the first instruction-based affordance segmentation benchmark that includes reasoning over both single and sequential affordances, comprising 180K instruction-point cloud pairs. Based on the benchmark, we propose our model, SeqAfford, to unlock the 3D multi-modal large language model with additional affordance segmentation abilities, which ensures reasoning with world knowledge and fine-grained affordance grounding in a cohesive framework. We further introduce a multi-granular language-point integration module to endow 3D dense prediction. Extensive experimental evaluations show that our model excels over well-established methods and exhibits open-world generalization with sequential reasoning abilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
