动态热高斯：多模态4D高斯泼溅

中文摘要 AI 辅助

热成像在军事及更广泛的热分析应用中发挥着至关重要的作用。近年来，3D热重建的进展将温度分析从2D空间扩展到3D空间，然而，现有的大多数工作假设温度分布是静态的，忽略了真实环境中热传递的时间动态性。为解决这一局限性，我们提出了首个面向复杂场景的动态RGB-热重建框架。我们的方法联合建模随时间变化的RGB外观、热观测和场景几何。具体而言，我们引入了一种多模态动态场景表示，将颜色和热模态锚定到共享的几何基底上，确保它们在时空变形下的一致性。我们进一步设计了多模态嵌入以增强每种模态的运动表达能力，并提出了一种多模态路由机制，该机制保留一组统一的共享多模态高斯作为几何主干，同时自适应地生成特定模态的高斯，以增强每个模态细节丰富区域的表示能力。此外，我们贡献了一个具有高频温度变化的新基准数据集，以促进4D重建的评估。大量实验表明，我们的方法实现了外观和温度的高保真时空重建。我们的代码和数据集可在以下网址获取：此https URL。

英文摘要

Thermography plays a vital role in military and broader thermal analysis applications. Recent progress in 3D thermal reconstruction has extended temperature analysis from 2D to 3D space, yet most existing works assume static temperature distributions, neglecting the temporal dynamics of heat transfer in real-world environments. To address this limitation, we propose the first dynamic RGB-Thermal reconstruction framework for complex scenes. Our method jointly models RGB appearance, thermal observations, and scene geometry as they change over time. Specifically, we introduce a multimodal dynamic scene representation that anchors both the color and thermal modalities to a shared geometric substrate, ensuring their consistency under spatiotemporal deformations. We further design multimodal embeddings to enhance the motion expressiveness for each modality, and propose a multimodal routing mechanism that retains a unified set of shared multimodal Gaussians as the geometric backbone while adaptively spawning modality-specific Gaussians to strengthen the representational capacity in detail-rich regions of each individual modality. In addition, we contribute a novel benchmark dataset featuring high-frequency temperature variations to facilitate the evaluation of 4D reconstruction. Extensive experiments demonstrate that our method achieves high-fidelity spatiotemporal reconstruction of both appearance and temperature. Our code and dataset are available at: https://github.com/LinLif1869/DTG.

发表机构

• Lishui University（丽水学院）
• Lishui Key Laboratory of Low-Altitude Multimodal Sensing and Intelligent Computing（丽水市低空多模态感知与智能计算重点实验室）
• Hangzhou Dianzi University（杭州电子科技大学）
• University of Chinese Academy of Sciences（中国科学院大学）
• Jiaxing University（嘉兴大学）
• Zhejiang University（浙江大学）

机构由 AI 辅助整理，请以论文原文为准。