使用多模态深度融合网络从集成HMD的传感器预测VR疾病
VR Sickness Prediction from Integrated HMD's Sensors using Multimodal Deep Fusion Network
摘要:
虚拟现实(VR)疾病通常被称为cybersickness是舒适使用VR系统的主要问题之一。研究人员已经提出了不同的方法来预测从生物生理数据(例如,心率、呼吸率、脑电图)。然而,收集生物生理数据通常需要外部传感器,限制了虚拟现实(VR)体验期间的运动和3D对象操作。已经进行了有限的研究来根据从盔装显示器(HMD)(例如,头部跟踪,眼睛跟踪,运动功能),允许自由运动和3D对象操作。这项研究提出了一种新的深度融合网络来预测cybersickness的严重程度从异构数据很容易从集成HMD传感器。我们提取了1755个立体视频,眼睛跟踪和头部跟踪数据,以及在VR游戏过程中从30名参与者中收集的相应的自我报告的晕机严重程度。我们应用了几种深度融合方法与从参与者收集的异构数据。我们的研究结果表明,电脑病可以预测的准确性为87.77%,均方根误差为0.51时,只使用眼动跟踪和头部跟踪数据。我们的结论是,眼动跟踪和头部跟踪数据非常适合于一个独立的网络病预测框架。
英文摘要:
Virtual Reality (VR) sickness commonly known as cybersickness is one of the major problems for the comfortable use of VR systems. Researchers have proposed different approaches for predicting cybersickness from bio-physiological data (e.g., heart rate, breathing rate, electroencephalogram). However, collecting bio-physiological data often requires external sensors, limiting locomotion and 3D-object manipulation during the virtual reality (VR) experience. Limited research has been done to predict cybersickness from the data readily available from the integrated sensors in head-mounted displays (HMDs) (e.g., head-tracking, eye-tracking, motion features), allowing free locomotion and 3D-object manipulation. This research proposes a novel deep fusion network to predict cybersickness severity from heterogeneous data readily available from the integrated HMD sensors. We extracted 1755 stereoscopic videos, eye-tracking, and head-tracking data along with the corresponding self-reported cybersickness severity collected from 30 participants during their VR gameplay. We applied several deep fusion approaches with the heterogeneous data collected from the participants. Our results suggest that cybersickness can be predicted with an accuracy of 87.77\% and a root-mean-square error of 0.51 when using only eye-tracking and head-tracking data. We concluded that eye-tracking and head-tracking data are well suited for a standalone cybersickness prediction framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。