DEGround:一种有效的基于自身视角的3D视觉定位基线框架
DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Framework
AI总结:
本文提出DEGround框架,通过统一的框架实现检测与定位的物体级共享,引入两个任务特定模块提升细粒度指令定位性能,实验表明在多个基准上表现最佳,尤其在EmbodiedScan数据集上精度提升显著。
摘要:
基于具身智能的核心任务是自身视角的3D视觉定位。现有方法通常采用两阶段、异构管道,将检测器与独立的定位模型配对。不兼容的解码器和框头阻碍了物体级先验的转移,分割训练导致冗余重新优化。为克服这些限制,我们提出了DEGround,一种直、优雅且有效的框架,专注于检测和定位的物体级共享。它采用一组查询作为检测和定位的共同物体表示,由共享的Transformer和边界框头解码。基于此统一框架,我们进一步引入两个任务特定的插件模块以增强细粒度指令定位。区域激活定位模块通过突出与指令相关的区域来改善空间-文本对齐,而查询级调制模块则应用句子条件的仿射调制来在初始化时生成指令感知的查询。大量实验表明,DEGround在多个基准上实现了最佳性能。值得注意的是,在EmbodiedScan数据集上,其总体精度比先前方法高出7.52%。
英文摘要:
A core task in embodied intelligence is ego-centric 3D visual grounding. Existing methods typically adopt two-stage, heterogeneous pipelines that pair a detector with a separate grounding model. Incompatible decoders and box heads hinder the transfer of object-level priors, and the split training causes redundant re-optimization. To overcome these limitations, we present DEGround, a straight, elegant, and effective framework that centers on object-level sharing over detection and grounding. It employs a set of queries that serves as the common object representation for both detection and grounding, which is decoded by a shared transformer and bounding box head. Building on this homogeneous framework, we further introduce two task-specific plug-in modules to enhance fine-grained instruction grounding. The Regional Activation Grounding module improves spatial-textual alignment by highlighting instruction-relevant regions, while the Query-wise Modulation module applies sentence-conditioned affine modulation to generate instruction-aware queries at initialization. Extensive experiments demonstrate that DEGround achieves the best performance on multiple benchmarks. Remarkably, it significantly outperforms previous methods by 7.52% at overall precision on the EmbodiedScan dataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。