arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06010cs.CVcs.AI
收藏

通过蒸馏扩散模型在实时中增强视觉系统热感知

Adding Thermal Awareness to Visual Systems in Real-Time via Distilled Diffusion Models

Northwestern University(西北大学);Northeastern University(东北大学);Hong Kong Baptist University(香港 Baptist大学);Beijing Normal - Hong Kong Baptist University(北京师范大学-香港 Baptist大学)

Yuchen Guo, Junli Gong, Wenjun Dong, Yiuming Cheung, Weifeng Su

AI总结:

本文提出FusionProxy模块,通过蒸馏扩散模型实现实时热感知融合,提升静态识别和动态任务鲁棒性,适用于边缘部署。

摘要:

基于RGB的视觉模型在夜间和雾天等挑战性场景中难以提供可靠线索,导致性能下降和安全风险。红外成像能捕捉热源,但现有高保真融合方法存在延迟问题,难以实时部署。本文提出FusionProxy,一个实时图像融合模块,作为独立插件组件,利用教师样本集合的两种互补统计:原始图像空间中的像素方差用于加权像素级监督,冻结基础骨干网络中的像素方差用于空间特征对齐。训练后,FusionProxy可直接集成到任何视觉感知系统中,无需联合优化。实验表明,该方法在静态识别任务中表现优异,并显著增强了动态任务的鲁棒性,包括闭环自动驾驶。关键在于FusionProxy在多种平台(从高端GPU到商用硬件)上实现了实时推理速度,提供灵活且通用的全天候感知解决方案。

英文摘要:

Purely RGB-based vision models often fail to provide reliable cues in challenging scenarios such as nighttime and fog, leading to degraded performance and safety risks. Infrared imaging captures heat-emitting sources and provides critical complementary information, but existing high-fidelity fusion methods suffer from prohibitive latency, rendering them impractical for real-time edge deployment. To address this, we propose FusionProxy, a real-time image fusion module designed as a fully independent, plug-and-play component with diffusion level quality. FusionProxy exploits two complementary statistics of a teacher sample ensemble: per-pixel variance in raw image space, used to weight pixel-level supervision, and per-pixel variance inside frozen foundation backbones, used to route feature-level alignment spatially. Once trained, FusionProxy can be directly integrated into any visual perception system without joint optimization. Extensive experiments demonstrate that our method achieves superior performance on static recognition tasks and significantly enhances robustness in dynamic tasks, including closed-loop autonomous driving. Crucially, FusionProxy achieves real-time inference speeds on diverse platforms, from high-end GPUs to commodity hardware, providing a flexible and generalizable solution for all-day perception.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑