AI 中文总结
本文提出轻量级频域RAW数据增强模块FreqAdapt，通过将ISP操作映射到傅里叶频域并结合自适应频域编码器实现特征增强，在多数据集上达到SOTA性能，可无缝集成到现有目标检测框架。
AI 中文摘要
现有目标检测方法主要使用sRGB输入，这类输入是由图像信号处理器（ISP）从RAW传感器数据压缩而来，而ISP最初是为可视化目的设计的。与RGB图像相比，RAW图像具有更优的噪声特性和更丰富的信息表示，这对目标检测至关重要，尤其是在恶劣天气或低光照等具有挑战性的条件下。本文提出了FreqAdapt，一个用于频域RAW数据增强的轻量级模块。与传统的空间域处理方法不同，FreqAdapt创新性地将ISP操作映射到傅里叶频域，并基于ISP操作的物理特性进行域分离，确保每个操作在最合适的域中执行。同时，通过自适应频域编码器联合分析幅度谱、相位谱和RAW图像特征，为ISP参数预测提供全局上下文，并采用可学习的融合机制实现自适应特征增强。在多个具有不同光照和天气条件的数据集上进行的大量实验表明，FreqAdapt在保持轻量级效率和良好物理可解释性的同时，达到了最先进的性能。此外，该模块可无缝集成到现有目标检测框架中，为RAW域的视觉感知任务提供了一种新的解决方案。
英文摘要
Existing object detection methods predominantly utilize sRGB inputs, which are compressed from RAW sensor data using Image Signal Processors (ISP) originally designed for visualization purposes. Compared to RGB images, RAW images possess favorable noise characteristics and richer information representation, which are crucial for object detection, particularly under challenging conditions such as adverse weather or low-light environments. In this paper, we propose FreqAdapt, a lightweight module for adaptive RAW data enhancement in the frequency domain. Unlike traditional spatial domain processing methods, FreqAdapt innovatively maps ISP operations to the Fourier frequency domain and performs domain separation based on the physical properties of ISP operations, ensuring each operation is performed in its most suitable domain. Meanwhile, through an adaptive frequency domain encoder that jointly analyzes amplitude spectrum, phase spectrum, and RAW image features, we provide global context for ISP parameter prediction and employ a learnable fusion mechanism to achieve adaptive feature enhancement. Extensive experiments on multiple datasets with diverse lighting and weather conditions demonstrate that FreqAdapt achieves state-of-the-art performance while maintaining lightweight efficiency and good physical interpretability. Furthermore, our module can be seamlessly incorporated into existing object detection frameworks, providing a novel solution for visual perception tasks in the RAW domain.
