See It All：面向3D密集描述的上下文感知晚期聚合方法
See It All: Contextualized Late Aggregation for 3D Dense Captioning
AI总结：
针对3D密集描述中单个查询需同时关注物体区域与上下文的目标冲突问题，提出采用晚期聚合范式的SIA方法，通过双查询解码与新型聚合器生成上下文感知描述，在主流数据集上实现显著性能提升。
摘要：
3D密集描述是指在3D场景中定位物体并为每个物体生成描述语句的任务。近期的3D密集描述方法借鉴目标检测领域的Transformer编码器-解码器框架，构建了无需手工设计组件的端到端流水线。然而，这些方法面临目标冲突的难题：单个查询的注意力机制需要同时关注精准定位的物体区域和上下文环境。为解决这一挑战，我们提出SIA（See-It-All）——一种采用晚期聚合新范式的3D密集描述Transformer流水线。SIA同时解码两组查询：上下文查询与实例查询。实例查询专注于定位和物体属性描述，上下文查询则灵活捕获多物体间或物体与全局场景的关系感兴趣区域，随后通过简单的基于距离的度量进行聚合（即晚期聚合）。为进一步提升上下文感知描述的生成质量，我们设计了一种新型聚合器，可基于周围上下文、全局环境和物体实例生成信息完备的描述。在两个最常用的3D密集描述数据集上开展的大量实验表明，所提方法相较现有方法取得了显著提升。
英文摘要：
3D dense captioning is a task to localize objects in a 3D scene and generate descriptive sentences for each object. Recent approaches in 3D dense captioning have adopted transformer encoder-decoder frameworks from object detection to build an end-to-end pipeline without hand-crafted components. However, these approaches struggle with contradicting objectives where a single query attention has to simultaneously view both the tightly localized object regions and contextual environment. To overcome this challenge, we introduce SIA (See-It-All), a transformer pipeline that engages in 3D dense captioning with a novel paradigm called late aggregation. SIA simultaneously decodes two sets of queries-context query and instance query. The instance query focuses on localization and object attribute descriptions, while the context query versatilely captures the region-of-interest of relationships between multiple objects or with the global scene, then aggregated afterwards (i.e., late aggregation) via simple distance-based measures. To further enhance the quality of contextualized caption generation, we design a novel aggregator to generate a fully informed caption based on the surrounding context, the global environment, and object instances. Extensive experiments on two of the most widely-used 3D dense captioning datasets demonstrate that our proposed method achieves a significant improvement over prior methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
