VoxAfford：多尺度体素-标记融合用于开放词汇3D affordance检测
VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection
AI总结：
本文提出VoxAfford，通过多尺度几何特征增强输出标记，提升3D affordance检测的定位精度，实验显示mIoU提升8%，并验证了零样本迁移能力。
摘要：
开放词汇3D affordance检测需要在点云上定位交互区域，给定新的affordance描述。最近的方法扩展了多模态大语言模型（MLLMs），通过特殊输出标记解码为分割掩码。然而，这些标记是通过自回归生成的，模型顺序依赖而非空间邻近关系，导致语义丰富但空间贫乏。本文提出Voxel-enhanced Affordance detection（VoxAfford），通过注入冻结预训练的3D VQVAE编码器的多尺度几何特征到生成的输出标记中，以绕过这一瓶颈。每个输出标记使用其affordance语义作为查询，通过交叉注意力检索其配对体素尺度的相关几何模式，通过学习的兼容性门控控制注入强度。增强的标记通过语义条件注意力聚合为空间感知的affordance提示，并与每点特征一起传播以生成最终掩码。实验表明，VoxAfford在开放词汇affordance检测任务中取得最佳性能，mIoU提升约8%，并验证了零样本迁移能力。
英文摘要：
Open-vocabulary 3D affordance detection requires localizing interaction regions on point clouds given novel affordance descriptions. Recent methods extend multimodal large language models (MLLMs) with special output tokens that are decoded into segmentation masks. However, these tokens are produced through autoregressive generation, which models sequential dependencies rather than spatial neighborhood relations, leaving them semantically rich but spatially impoverished for 3D localization. We propose Voxel-enhanced Affordance detection (VoxAfford), which bypasses this bottleneck by injecting multi-scale geometric features from a frozen pre-trained 3D VQVAE encoder into the output tokens after generation. Each output token uses its affordance semantics as a query to retrieve relevant geometric patterns from its paired voxel scale via cross-attention, with a learned compatibility gate controlling the injection strength. The enhanced tokens are then aggregated into a spatially-aware affordance prompt through semantic-conditioned attention and propagated alongside per-point features to generate the final mask. Experiments on open-vocabulary affordance detection tasks show that VoxAfford achieves state-of-the-art performance with approximately an 8% improvement in mIoU, and real robot experiments confirm zero-shot transfer to novel objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
