arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.11684cs.CV

SpikeCV:开启连续计算机视觉时代

SpikeCV: Open a Continuous Computer Vision Era

School of Computer Science, Peking University(北京大学计算机科学学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院)

Yajing Zheng, Jiyuan Zhang, Rui Zhao, Jianhao Ding, Shiyan Chen, Ruiqin Xiong, Zhaofei Yu, Tiejun Huang

AI总结:

本文提出开源脉冲相机视觉平台 SpikeCV,通过数据集、硬件接口和模块化 Python 工具链支持超高速成像、检测与实时应用。

摘要:

SpikeCV 是一个面向脉冲相机的新型开源计算机视觉平台,脉冲相机是近年来快速发展的一种神经形态视觉传感器。在脉冲相机中,每个像素位置直接累积光强并异步发射脉冲。其输出的二进制脉冲频率可达 40,000 Hz。作为一种新型视觉表达,脉冲序列具有很高的时空完备性,并保留了外部世界的连续视觉信息。借助脉冲相机的低延迟和高动态范围优势,许多基于脉冲的算法已经取得显著进展,例如高质量成像和超高速目标检测。为构建脉冲视觉的社区生态,帮助更多用户利用脉冲相机,SpikeCV 提供了多种超高速场景数据集、硬件接口以及易用的模块库。SpikeCV 聚焦于脉冲数据封装、数据集接口标准化、视觉任务模块化以及挑战性场景的实时应用。随着开源 Python 生态的出现,SpikeCV 的模块可作为 Python 库使用,以满足研究人员的大多数数值分析需求。我们展示了 SpikeCV 在离线推理和实时应用方面的效率。项目仓库地址为 https://openi.pcl.ac.cn/Cordium/SpikeCV 和 https://github.com/Zyj061/SpikeCV。

英文摘要:

SpikeCV is a new open-source computer vision platform for the spike camera, which is a neuromorphic visual sensor that has developed rapidly in recent years. In the spike camera, each pixel position directly accumulates the light intensity and asynchronously fires spikes. The output binary spikes can reach a frequency of 40,000 Hz. As a new type of visual expression, spike sequence has high spatiotemporal completeness and preserves the continuous visual information of the external world. Taking advantage of the low latency and high dynamic range of the spike camera, many spike-based algorithms have made significant progress, such as high-quality imaging and ultra-high-speed target detection. To build up a community ecology for the spike vision to facilitate more users to take advantage of the spike camera, SpikeCV provides a variety of ultra-high-speed scene datasets, hardware interfaces, and an easy-to-use modules library. SpikeCV focuses on encapsulation for spike data, standardization for dataset interfaces, modularization for vision tasks, and real-time applications for challenging scenes. With the advent of the open-source Python ecosystem, modules of SpikeCV can be used as a Python library to fulfilled most of the numerical analysis needs of researchers. We demonstrate the efficiency of the SpikeCV on offline inference and real-time applications. The project repository address are \url{https://openi.pcl.ac.cn/Cordium/SpikeCV} and \url{https://github.com/Zyj061/SpikeCV

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑