arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2404.18934cs.CVcs.HC

视觉体验数据集:超过200小时的集成眼动、里程计与第一人称视频记录

The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video

Barnard College, Columbia University(哥伦比亚大学巴纳德学院);Bates College(贝茨学院);North Dakota State University(北达科他州立大学);University of Nevada, Reno(内华达大学里诺分校);Magic Leap;Technical University of Munich(慕尼黑工业大学);Unmanned Ground Systems(无人地面系统公司);Smith-Kettlewell Eye Research Institute(史密斯-凯特威尔眼科研究所)
Michelle R. Greene, Benjamin J. Balas, Mark D. Lescroart, Paul R. MacNeilage, Jennifer A. Hart, Kamran Binaee, Peter A. Hausamann, Ronald Mezile, Bharath Shanka… 展开作者

Michelle R. Greene, Benjamin J. Balas, Mark D. Lescroart, Paul R. MacNeilage, Jennifer A. Hart, Kamran Binaee, Peter A. Hausamann, Ronald Mezile, Bharath Shankar, Christian B. Sinnott, Kaylie Capurro, Savannah Halow, Hunter Howe, Mariam Josyula, Annie Li, Abraham Mieses, Amina Mohamed, Ilya Nudnou, Ezra Parkhill, Peter Riley, Brett Schmidt, Matthew W. Shinkle, Wentao Si, Brian Szekely, Joaquin M. Torres, Eliana Weissmann

AI总结:

本研究推出包含240余小时多模态第一人称视觉数据的VEDB数据集,介绍其构建流程与伦理规范,支持视觉感知、 gaze追踪等多领域研究,开放社区贡献以推进相关研究发展。

摘要:

我们推出Visual Experience Dataset(VEDB,视觉体验数据集),这是一套包含超过240小时第一人称视频并结合注视点与头部追踪数据的数据集,为研究人类观察者所体验的视觉世界提供了前所未有的视角。该数据集包含717个记录会话,由58名年龄在6-49岁的受试者录制。本文概述了为确保样本代表性而采用的数据收集、处理与标注协议,并讨论了数据集中潜在的误差或偏差来源。VEDB的潜在应用十分广泛,包括改进注视点追踪方法、评估时空图像统计特性,以及优化用于场景与活动识别的深度神经网络。VEDB可通过成熟的开放科学平台获取,并且被规划为一个活态数据集,未来计划进行扩展并接受社区贡献。该数据集的发布强调伦理考量,例如受试者隐私保护与潜在偏差的缓解。通过提供基于真实世界体验、附带大量元数据与配套代码的数据集,作者邀请研究界使用并共建VEDB,以推动对自然场景下视觉感知与行为的更深入理解。

英文摘要:

We introduce the Visual Experience Dataset (VEDB), a compilation of over 240 hours of egocentric video combined with gaze- and head-tracking data that offers an unprecedented view of the visual world as experienced by human observers. The dataset consists of 717 sessions, recorded by 58 observers ranging from 6-49 years old. This paper outlines the data collection, processing, and labeling protocols undertaken to ensure a representative sample and discusses the potential sources of error or bias within the dataset. The VEDB's potential applications are vast, including improving gaze tracking methodologies, assessing spatiotemporal image statistics, and refining deep neural networks for scene and activity recognition. The VEDB is accessible through established open science platforms and is intended to be a living dataset with plans for expansion and community contributions. It is released with an emphasis on ethical considerations, such as participant privacy and the mitigation of potential biases. By providing a dataset grounded in real-world experiences and accompanied by extensive metadata and supporting code, the authors invite the research community to utilize and contribute to the VEDB, facilitating a richer understanding of visual perception and behavior in naturalistic settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑