具有视觉对齐4D雷达的抗天气场景语义
Weather-Robust Scene Semantics with Vision-Aligned 4D Radar
AI总结：
本文提出通过冻结视觉语言模型将雷达编码器对齐于SigLIP嵌入，以提升恶劣天气下的场景语义理解，实验表明雷达配置在雾、轻雪和重雪条件下优于相机基线。
摘要：
摄像头和激光雷达在雨、雾和雪中性能下降，而毫米波雷达基本不受影响。我们通过冻结的SigLIP视觉嵌入对雷达编码器进行对齐，并通过冻结的视觉语言模型（VLM）解码结构化场景描述。在K-RADAR数据集上，所有雷达配置在雾、轻雪和重雪序列中均优于崩溃至超过90%幻觉的相机基线。我们识别出雷达到冻结VLM之间的token-norm不匹配是主要失败模式，并展示投影器输出的LayerNorm可以解决这一问题。对编码器复杂度、描述格式和池化策略的分析揭示了影响未来雷达-VLM流水线设计的权衡。
英文摘要：
Cameras and LiDAR degrade in rain, fog, and snow, while millimeter-wave radar remains largely unaffected. We align a radar encoder to frozen SigLIP vision embeddings and decode structured scene captions through a frozen vision-language model (VLM) with approximately 7M trainable parameters. On K-RADAR with held-out fog, light snow, and heavy snow sequences, all radar configurations outperform a camera baseline that collapses to over 90% hallucination. We identify a token-norm mismatch as the dominant failure mode when bridging radar to a frozen VLM and show that projector-output LayerNorm resolves it. Analysis of encoder complexity, caption format, and pooling strategy reveals tradeoffs that inform future radar-VLM pipeline design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
