发表机构
The University of Hong Kong; Commonwealth Scientific and Industrial Research Organisation (CSIRO)（香港大学; 英联邦科学与工业研究组织）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对热图像深度估计中RGB基础模型分层表示利用不足的问题，提出RGB-HS框架，通过分层监督和基于图像质量的标记加权对齐，在基准上实现了有竞争力的性能。
AI 中文摘要
在夜间、雨天等恶劣条件下，从热图像中进行深度估计对机器人应用具有重要价值。近期研究尝试将基于RGB的基础模型的知识迁移到热模态，但这些模型编码的丰富分层表示仍未得到充分利用。为解决这一局限，我们提出RGB-HS框架，这是一种用于热图像深度估计的新框架，利用来自基于RGB的基础模型的分层监督。具体而言，我们首先将基线热编码器替换为基础模型，并引入并行RGB分支，该分支也采用相同架构的基础模型作为编码器，以RGB图像为输入。随后在两个编码器的标记之间的多个层级执行对齐，使热学生分支能够从RGB教师分支中同时捕获结构精度和语义抽象。此外，我们引入验证机制，通过基于RGB图像质量对RGB分支的标记进行加权来优化对齐过程。在流行基准上的大量实验表明，RGB-HS实现了有竞争力的性能，且能更有效地利用基于RGB的基础模型的表示能力进行热图像的深度估计。
英文摘要
Depth estimation from thermal images is highly valuable for robotic applications in adverse conditions, such as nighttime and rainy weather. Recent studies have sought to transfer knowledge from RGB-based foundation models to thermal modalities, yet the rich hierarchical representations these models encode remain underutilized. To address this limitation, we propose RGB-HS, a novel framework for thermal-image depth estimation that leverages hierarchical supervision from an RGB-based foundation model. Specifically, we first replace the baseline thermal encoder with a foundational model and introduce a parallel RGB branch that also employs a foundational model as an encoder of the same architecture, taking RGB images as input. The alignment is then performed across multiple levels between the tokens of the two encoders, allowing the thermal student branch to capture both structural precision and semantic abstraction from the RGB teacher branch. Furthermore, we introduce verification to refine the alignment process by weighting tokens from the RGB branch based on RGB image quality. Extensive experiments on the popular benchmark demonstrate that RGB-HS achieves competitive performance and more effectively exploits the representational capacity of RGB-based foundation models for depth estimation on thermal images.
CommentsAccepted in IROS 2026
