arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.09050cs.CVcs.AI
收藏

使用深度学习为扩展现实生成逼真的合成头部旋转数据

Generating Realistic Synthetic Head Rotation Data for Extended Reality using Deep Learning

University of Antwerp(安特卫普大学);imec;Universitat Politècnica de Catalunya(加泰罗尼亚理工大学)

Jakob Struye, Filip Lemic, Jeroen Famaey

AI总结:

针对扩展现实中头部旋转预测模型训练数据采集成本高的问题,提出基于TimeGAN的头部旋转时间序列生成器,可生成匹配实测数据分布的合成样本以扩展数据集。

摘要:

扩展现实(Extended Reality)是一种向用户提供多媒体内容的革命性方式。其广受欢迎的一大原因在于,真实世界的动作能准确、即时地反映在虚拟体验中,从而带来沉浸感与交互性。这种主要由头部旋转引发的用户动作带来了若干技术挑战。例如,生成和传输的内容在很大程度上取决于用户的视线方向。因此,能主动考虑用户动作的无缝系统需要准确预测即将发生的旋转。训练和评估此类预测器需要大量方向输入数据,而由于需要人类受试者参与,采集这类数据成本高昂。一种更可行的方法是通过受试者采集少量数据集,然后使用合成数据生成方法将其扩展为规模更大的数据集。在本研究中,我们提出了一种基于TimeGAN的头部旋转时间序列生成器,TimeGAN是著名的生成对抗网络(Generative Adversarial Network)的扩展,专为生成时间序列而设计。该方法能够用与实测时间序列分布高度匹配的新样本扩展头部旋转数据集。

英文摘要:

Extended Reality is a revolutionary method of delivering multimedia content to users. A large contributor to its popularity is the sense of immersion and interactivity enabled by having real-world motion reflected in the virtual experience accurately and immediately. This user motion, mainly caused by head rotations, induces several technical challenges. For instance, which content is generated and transmitted depends heavily on where the user is looking. Seamless systems, taking user motion into account proactively, will therefore require accurate predictions of upcoming rotations. Training and evaluating such predictors requires vast amounts of orientational input data, which is expensive to gather, as it requires human test subjects. A more feasible approach is to gather a modest dataset through test subjects, and then extend it to a more sizeable set using synthetic data generation methods. In this work, we present a head rotation time series generator based on TimeGAN, an extension of the well-known Generative Adversarial Network, designed specifically for generating time series. This approach is able to extend a dataset of head rotations with new samples closely matching the distribution of the measured time series.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑