DeepIPCv3: 面向突发行人穿越避让的事件感知多模态传感器融合
DeepIPCv3: Event-Aware Multi-Modal Sensor Fusion for Sudden Pedestrian Crossing Avoidance
AI总结：
提出DeepIPCv3框架，通过Transformer交叉模态注意力融合LiDAR点云与DVS事件流，实现突发行人穿越场景下的高反应性避让，在自定义多模态数据集上达到最优轨迹与控制精度。
摘要：
当前的端到端自动驾驶系统主要依赖基于帧的传感器，这类传感器在高度动态的突发行人穿越场景中存在固有的感知延迟和运动模糊问题。为解决这一关键安全漏洞，我们提出DeepIPCv3，一种新颖的多模态自主导航框架，它将LiDAR点云的密集3D空间几何与动态视觉传感器（DVS）的微秒级异步事件流协同融合。我们引入了一种受Transformer启发的交叉模态注意力机制，以动态关联这些不同模态，使网络能够即时优先处理高速动态更新，同时不牺牲场景结构感知。融合后的潜在表示通过一个混合策略网络映射到安全的局部路径点和可执行控制命令，该网络结合了启发式轨迹跟踪与直接神经预测。由于在真实场景中测试这些突发穿越场景存在严重物理风险，该框架使用在光照良好的正午和具有挑战性的傍晚条件下收集的自定义多模态数据集进行严格离线评估。广泛的对比和消融研究表明，DeepIPCv3达到了最先进的预测性能。通过有效消除曝光失败和运动模糊，所提出的LiDAR与DVS融合实现了最低的轨迹和控制命令误差，使得无论环境光照如何，都能实现高反应性、数学上有界的规避机动。为支持未来研究，我们将代码发布到GitHub仓库：https://github.com/oskarnatan/DeepIPCv3。
英文摘要：
Current end-to-end autonomous driving systems predominantly rely on frame-based sensors, which suffer from inherent perception latency and motion blur during highly dynamic encounters, specifically sudden pedestrian crossings. To address this critical safety vulnerability, we propose DeepIPCv3, a novel multi-modal autonomous navigation framework that synergizes the dense 3D spatial geometry of LiDAR point clouds with the microsecond-level asynchronous event streams of a Dynamic Vision Sensor (DVS). We introduce a Transformer-inspired cross-modal attention mechanism to dynamically correlate these distinct modalities, allowing the network to instantaneously prioritize high-speed dynamic updates without sacrificing structural scene awareness. The fused latent representations are then mapped to safe local waypoints and executable control commands via a hybrid policy network that blends heuristic trajectory tracking with direct neural predictions. Due to the severe physical risks associated with live testing of these sudden crossing scenarios, the framework is rigorously evaluated offline using a custom multi-modal dataset collected across both well-illuminated noon and challenging evening conditions. Extensive comparative and ablation studies demonstrate that DeepIPCv3 achieves state-of-the-art predictive performance. By effectively eliminating exposure failures and motion blur, the proposed LiDAR and DVS fusion yields the lowest trajectory and control command errors, enabling highly reactive, mathematically bounded evasive maneuvers regardless of ambient illumination. To support future research, we will release the codes to our GitHub repo at https://github.com/oskarnatan/DeepIPCv3.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
