发表机构
School of Computer Science, Peking University; Peng Cheng Laboratory; Department of Automation, Tsinghua University; Department of Computer Science and Technology, Tsinghua University（北京大学计算机科学学院; 鹏城实验室; 清华大学自动化系; 清华大学计算机科学与技术系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对传统主动立体系统在快速运动场景的问题，提出基于主动事件的立体视觉，构建数据集，创新设计ActiveEventNet+网络，性能优于现有方法，降低计算复杂度，能实现高速实时处理，为高速深度感知相机系统设计提供新思路。
AI 中文摘要
传统基于帧的主动立体系统成像在快速运动场景中面临重大挑战，如何设计超快速深度感知新范式仍是开放问题。本文提出基于主动事件的立体视觉问题设置，集成双目事件相机和红外2D图案投影仪用于高速密集深度感知。构建了立体相机原型系统及真实和合成数据集，提出轻量级且有效的ActiveEventNet+网络，有三项创新。结果表明其性能优于现有方法，降低计算复杂度，在高速场景中深度感知性能优越，能使原型系统达150FPS实时处理速度，为未来高速深度感知相机系统设计提供新思路。
英文摘要
Conventional frame-based imaging for active stereo systems has encountered major challenges in fast-motion scenarios. However, how to design a novel paradigm for ultrafast depth sensing remains an open issue. In this paper, we propose a novel problem setting, namely active event-based stereo vision, which attempts to integrate binocular event cameras and an infrared 2D pattern projector for high-speed dense depth sensing. Technically, we first build a stereo camera prototype system and present a real-world dataset with over 21.5k spatiotemporal synchronized labels at 15 Hz, while also establishing a realistic synthetic dataset with stereo event streams and 23.8k synchronized labels at 20 Hz. Then, we propose ActiveEventNet+, a lightweight yet effective event-based stereo matching neural network that learns to generate high-quality dense disparity maps from stereo event streams with low latency. Our ActiveEventNet+ mainly involves three innovations: incorporating lightweight blocks into event-based stereo matching frameworks, designing a novel cost volume with dynamic interactions between stereo pairs, and presenting an effective temporal consistency architecture to fully use rich temporal cues in event streams. The results show that our ActiveEventNet+ outperforms state-of-the-art methods while significantly reducing computational complexity. Our solution offers superior depth sensing performance compared to conventional frame-based stereo cameras in high-speed scenes. In particular, the lightweight ActiveEventNet enables the prototype system to achieve real-time processing at speeds up to 150 FPS. We believe that this novel active event-based stereo vision paradigm can provide new insights into the design of future high-speed depth sensing camera systems. Our dataset and code can be available at https://github.com/jianing-li/active_event_based_stereo.
CommentsAccepted by TPAMI
