EPRBench: 一种高质量的基于事件流的视觉位置识别基准数据集
EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition
AI总结:
EPRBench提出了一种高质量的基于事件流的视觉位置识别基准数据集,结合LLM生成场景描述与多模态融合方法,提升位置识别的准确性和模型透明度。
摘要:
基于事件流的视觉位置识别(VPR)是一种新兴的研究方向,为在低光照、过曝和高速运动等挑战性条件下传统可见光相机的不稳定性提供了一个有吸引力的解决方案。鉴于当前该领域专用数据集的匮乏,我们引入了EPRBench,这是一个专门针对基于事件流的VPR设计的高质量基准数据集。EPRBench包含10,000个事件序列和65,000个事件帧,通过手持和车载设备采集,以全面捕捉不同视角、天气条件和光照场景下的现实挑战。为了支持语义感知和语言集成的VPR研究,我们提供了由大型语言模型(LLM)生成的场景描述,随后通过人工标注进行优化,为将LLM整合到基于事件的感知管道中奠定了坚实基础。为了促进系统评估,我们在EPRBench上实现了并基准测试了15种最先进的VPR算法,为未来算法比较提供了有力的基线。此外,我们提出了一种新的多模态融合范式用于VPR:利用LLM从原始事件流中生成文本场景描述,然后指导空间关注的token选择、跨模态特征融合和多尺度表示学习。该框架不仅实现了高准确率的位置识别,还产生了可解释的推理过程,显著增强了模型的透明度和可解释性。数据集和源代码将在https://github.com/Event-AHU/Neuromorphic_ReID上发布。
英文摘要:
Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion. Recognizing the current scarcity of dedicated datasets in this domain, we introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR. EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios. To support semantic-aware and language-integrated VPR research, we provide LLM-generated scene descriptions, subsequently refined through human annotation, establishing a solid foundation for integrating LLMs into event-based perception pipelines. To facilitate systematic evaluation, we implement and benchmark 15 state-of-the-art VPR algorithms on EPRBench, offering a strong baseline for future algorithmic comparisons. Furthermore, we propose a novel multi-modal fusion paradigm for VPR: leveraging LLMs to generate textual scene descriptions from raw event streams, which then guide spatially attentive token selection, cross-modal feature fusion, and multi-scale representation learning. This framework not only achieves highly accurate place recognition but also produces interpretable reasoning processes alongside its predictions, significantly enhancing model transparency and explainability. The dataset and source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。