arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.10395cs.CVcs.AI
收藏

基于事件相机的图像人脸与视线跟踪评估

Evaluating Image-Based Face and Eye Tracking with Event Cameras

Dublin City University(都柏林城市大学);FotoNation-Tobii;University of Galway(戈尔韦大学);Insight SFI Research Center for Data Analytics(Insight SFI数据分析研究中心)

Khadija Iddrisu, Waseem Shariff, Noel E. OConnor, Joseph Lemley, Suzanne Little

AI总结:

本研究评估事件相机在人脸与视线跟踪中的应用,通过构建帧格式事件数据集并采用GR-YOLO算法,实现与传统算法的集成,在多个数据集上取得0.91的平均精度,验证了方法的有效性与鲁棒性。

摘要:

事件相机,也称为神经形态传感器,在像素级别捕捉局部光强度的变化,产生异步生成的数据,称为“事件”。这种独特的数据格式缓解了传统相机中常见的问题,例如在捕捉快速移动物体时的欠采样,从而保留了可能丢失的关键信息。然而,利用这些数据通常需要开发专门的、手工制作的事件表示,以便能够与传统卷积神经网络(CNNs)无缝集成,考虑到事件数据的独特属性。在本研究中,我们评估了基于事件的人脸和视线跟踪。我们研究的核心目标是展示将传统算法与基于事件的数据(转换为帧格式)集成的可行性,同时保留事件相机的独特优势。为了验证我们的方法,我们通过模拟来自公开可用的Helen数据集的RGB帧之间的事件,构建了一个基于帧的事件数据集。我们通过应用GR-YOLO(一种源自YOLOv3的开创性技术)来评估其对人脸和视线检测任务的有用性。该评估包括与使用YOLOv8训练数据集所得结果的比较分析。随后,将训练好的模型在来自Prophesee事件相机不同版本的真实事件流上进行测试,并进一步在事件流中的人脸(FES)基准数据集上进行评估。在我们数据集上训练的模型在所有用于验证的数据集上均表现出良好的预测性能,最佳平均精度得分为0.91。此外,训练后的模型在变化的照明条件下对真实事件相机数据表现出稳健的性能。

英文摘要:

Event Cameras, also known as Neuromorphic sensors, capture changes in local light intensity at the pixel level, producing asynchronously generated data termed ``events''. This distinct data format mitigates common issues observed in conventional cameras, like under-sampling when capturing fast-moving objects, thereby preserving critical information that might otherwise be lost. However, leveraging this data often necessitates the development of specialized, handcrafted event representations that can integrate seamlessly with conventional Convolutional Neural Networks (CNNs), considering the unique attributes of event data. In this study, We evaluate event-based Face and Eye tracking. The core objective of our study is to showcase the viability of integrating conventional algorithms with event-based data, transformed into a frame format while preserving the unique benefits of event cameras. To validate our approach, we constructed a frame-based event dataset by simulating events between RGB frames derived from the publicly accessible Helen Dataset. We assess its utility for face and eye detection tasks through the application of GR-YOLO -- a pioneering technique derived from YOLOv3. This evaluation includes a comparative analysis with results derived from training the dataset with YOLOv8. Subsequently, the trained models were tested on real event streams from various iterations of Prophesee's event cameras and further evaluated on the Faces in Event Stream (FES) benchmark dataset. The models trained on our dataset shows a good prediction performance across all the datasets obtained for validation with the best results of a mean Average precision score of 0.91. Additionally, The models trained demonstrated robust performance on real event camera data under varying light conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑