LiFR v2：用于高帧率密集预测的补全增强事件传播
LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Prediction
浏览论文内容
中文总结 AI 辅助
LiFR v2通过事件引导补全模块和历史检索模块，解决高帧率密集预测中传播失效问题，在DSEC和SHF-Emerge上提升精度并降低深度误差，速度超100 FPS。
中文摘要 AI 辅助
动态环境中的高帧率密集感知受限于RGB相机的低更新率，因为帧间可能发生快速的场景变化。事件相机提供时间上密集但空间上稀疏的测量，与空间上密集的RGB观测互补。直接融合无法充分利用这种互补性，而事件引导的传播在新出现或去遮挡区域缺乏有效RGB支持时会失效。我们提出LiFR v2，一个统一的传播-补全-记忆框架，用于从RGB关键帧和事件进行因果任意时刻和流式密集预测。LiFR v2引入了事件引导补全模块（EGCM）来恢复传播不支持的与任务相关的表示，以及历史检索模块（HRM）来在连续查询中重用已补全的表示。该框架支持语义分割、单目深度估计和多任务密集预测，我们还进一步引入SHF-Emerge来评估快速物体出现和去遮挡。LiFR v2在DSEC上达到74.37%的mIoU，在SHF-Emerge上达到56.13%，在后者上比LiFR-Seg提高了1.85个百分点，同时将SHF-Emerge深度RMSE从1.564米降低到1.118米，优于传播基线。它还在分割和深度任务上均超过100 FPS，展示了超越RGB帧率的高帧率感知的准确性和高效性。
英文摘要
High-rate dense perception in dynamic environments is limited by the low update rate of RGB cameras, as rapid scene changes can occur between frames. Event cameras offer temporally dense but spatially sparse measurements, complementary to spatially dense RGB observations. Direct fusion cannot fully exploit this complementarity, while event-guided propagation fails on newly appearing or disoccluded regions without valid RGB support. We present LiFR v2, a unified propagation-completion-memory framework for causal anytime and streaming dense prediction from an RGB keyframe and events. LiFR v2 introduces an Event-Guided Completion Module (EGCM) to recover task-relevant representations where propagation is unsupported, and a History Retrieval Module (HRM) to reuse completed representations across successive queries. The framework supports semantic segmentation, monocular depth estimation, and multi-task dense prediction, and we further introduce SHF-Emerge to evaluate rapid object emergence and disocclusion. LiFR v2 achieves 74.37% mIoU on DSEC and 56.13% on SHF-Emerge, improving LiFR-Seg by 1.85 percentage points on the latter, while reducing SHF-Emerge depth RMSE from 1.564 m to 1.118 m over the propagation baseline. It also exceeds 100 FPS for both segmentation and depth, demonstrating accurate and efficient high-rate perception beyond RGB frame rates.
发表机构
- Southern University of Science and Technology（南方科技大学）
- The University of Hong Kong（香港大学）
机构由 AI 辅助整理，请以论文原文为准。
