一种多模态深度感知方法用于具身参照理解
A Multimodal Depth-Aware Method For Embodied Reference Understanding
AI总结：
本文提出一种多模态深度感知方法，结合语言模型数据增强、深度图模态和深度感知决策模块，提升复杂环境中的参照物识别准确性。
摘要：
具身参照理解需要根据语言指令和指向线索识别场景中的目标物体。尽管先前工作在开放词汇物体检测方面取得进展，但在存在多个候选物体的模糊场景中往往失败。为解决这些问题，我们提出了一种新的ERU框架，联合利用基于LLM的数据增强、深度图模态和深度感知决策模块。这种设计使语言和具身线索的整合更加稳健，提高了复杂或拥挤环境中的歧义消除能力。在两个数据集上的实验结果表明，我们的方法显著优于现有基线，实现了更准确和可靠的参照物检测。
英文摘要：
Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While prior works have shown progress in open-vocabulary object detection, they often fail in ambiguous scenarios where multiple candidate objects exist in the scene. To address these challenges, we propose a novel ERU framework that jointly leverages LLM-based data augmentation, depth-map modality, and a depth-aware decision module. This design enables robust integration of linguistic and embodied cues, improving disambiguation in complex or cluttered environments. Experimental results on two datasets demonstrate that our approach significantly outperforms existing baselines, achieving more accurate and reliable referent detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
