RGBT-GroundBench: 超越RGB的复杂真实世界场景中的视觉定位
RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios
AI总结：
提出首个大规模RGB-热红外视觉定位基准RGBT-GroundBench，包含4万+图像和3级细粒度标注，统一评估协议支持RGB/热红外/融合输入，揭示低照度下性能显著下降，并引入参考基线RGBT-VGNet。
摘要：
视觉定位（VG）根据自然语言表达在图像中定位目标对象。在真实世界感知中，RGB线索在低照度和恶劣天气下常会退化，使视觉定位更具挑战性。然而，现有的VG基准大多仅基于RGB，对此类条件的覆盖有限且结构化不足，阻碍了系统性的鲁棒性评估和跨光谱比较。我们提出RGBT-GroundBench，这是首个面向复杂环境中RGB-热红外（TIR）视觉定位的大规模基准。它包含超过4万张图像（21,535对RGB-TIR对）和38,760个带有指代表达、边界框以及三级细粒度标注（场景类型、环境条件（照度和天气）和物体属性（大小和遮挡））的物体实例。作为基准套件，RGBT-GroundBench不仅提供精心整理的RGB-TIR定位标注，还提供统一的评估协议，支持仅RGB、仅TIR和RGB+TIR输入。在此协议下，我们对11个代表性VG模型在不同场景和环境条件下进行了基准测试。结果表明，定位准确性与场景复杂度强相关，基于LoRA的模型在复杂场景中更鲁棒，而低照度条件导致显著的性能下降，这一点此前很少被探索。基于这些观察，我们引入了RGBT-VGNet，这是一个在统一协议下简单且可复现的参考基线，具有非对称模态适应、语言感知视觉协同和三先验融合以实现可靠性感知的RGB-TIR集成。资源、标注、代码、检查点和评估脚本已公开发布。
英文摘要：
Visual grounding (VG) localizes target objects in an image from natural-language expressions. In real-world perception, RGB cues often degrade under low illumination and adverse weather, making visual grounding substantially more challenging. However, existing VG benchmarks are largely RGB-only and provide limited, structured coverage of such conditions, hindering systematic robustness evaluation and cross-spectral comparison. We present RGBT-GroundBench, the first large-scale benchmark for RGB-Thermal (TIR) visual grounding in complex environments. It contains over 40K images (21,535 RGB-TIR pairs) and 38,760 object instances with referring expressions, bounding boxes, and fine-grained annotations at three levels: scene types, environmental conditions (illumination and weather), and object properties (size and occlusion). As a benchmark suite, RGBT-GroundBench provides not only curated RGB-TIR grounding annotations but also a unified evaluation protocol supporting RGB-only, TIR-only, and RGB+TIR inputs. Under this protocol, we benchmark 11 representative VG models across diverse scenes and environmental conditions. Our results show that grounding accuracy is strongly correlated with scene complexity, LoRA-based models are more robust in complex scenes, and low-illumination conditions cause significant performance degradation that has been rarely explored. Guided by these observations, we introduce RGBT-VGNet, a simple and reproducible reference baseline under the unified protocol, featuring Asymmetric Modality Adaptation, Language-Aware Visual Synergy, and Tri-Prior Fusion for reliability-aware RGB-TIR integration. Resources, annotations, code, checkpoints, and evaluation scripts have been publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
