E2E-MFD：迈向端到端同步多模态融合检测
E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection
AI总结：
本文提出端到端多模态融合检测算法E2E-MFD，通过同步联合优化和梯度矩阵策略，在单一训练阶段实现高性能，并在M3FD和DroneVehicle数据集上分别提升mAP50 3.9%和2.0%。
摘要：
多模态图像融合与目标检测对于自动驾驶至关重要。尽管现有方法已推进了纹理细节与语义信息的融合，但其复杂的训练过程阻碍了更广泛的应用。针对这一挑战，我们提出了E2E-MFD，一种新颖的端到端多模态融合检测算法。E2E-MFD简化了流程，通过单一训练阶段即可实现高性能。它采用跨组件的同步联合优化，以避免与单个任务相关的次优解。此外，它在共享参数的梯度矩阵中实施了全面的优化策略，确保收敛到最优的融合检测配置。我们在多个公共数据集上的广泛测试揭示了E2E-MFD的卓越能力，不仅展示了视觉上吸引人的图像融合，还带来了令人印象深刻的检测结果，例如，在水平目标检测数据集M3FD和定向目标检测数据集DroneVehicle上，相比最先进的方法，mAP50分别提高了3.9%和2.0%。代码已发布在https://github.com/icey-zhang/E2E-MFD。
英文摘要：
Multimodal image fusion and object detection are crucial for autonomous driving. While current methods have advanced the fusion of texture details and semantic information, their complex training processes hinder broader applications. Addressing this challenge, we introduce E2E-MFD, a novel end-to-end algorithm for multimodal fusion detection. E2E-MFD streamlines the process, achieving high performance with a single training phase. It employs synchronous joint optimization across components to avoid suboptimal solutions tied to individual tasks. Furthermore, it implements a comprehensive optimization strategy in the gradient matrix for shared parameters, ensuring convergence to an optimal fusion detection configuration. Our extensive testing on multiple public datasets reveals E2E-MFD's superior capabilities, showcasing not only visually appealing image fusion but also impressive detection outcomes, such as a 3.9% and 2.0% mAP50 increase on horizontal object detection dataset M3FD and oriented object detection dataset DroneVehicle, respectively, compared to state-of-the-art approaches. The code is released at https://github.com/icey-zhang/E2E-MFD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
