arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.00161cs.HCcs.AIcs.CV
收藏

混合现实临场感中的交互场景合成

Mutual Scene Synthesis for Mixed Reality Telepresence

Reality Labs Research, Meta(Meta现实实验室研究部);University of California, Berkeley(加州大学伯克利分校)

Mohammad Keshavarzi, Michael Zollhoefer, Allen Y. Yang, Patrick Peluse, Luisa Caldas

摘要:

通过下一代混合现实平台的远程临场感可以为计算机介导的通信提供更高水平的沉浸感,允许参与者参与各种活动,这在以前的基于2D屏幕的通信方法中是不可能的。然而,由于混合现实体验仅限于每个用户的本地物理环境,因此找到一个用户可以自由移动和相互交互的共同虚拟地面是具有挑战性的。在本文中,我们提出了一种新的交互式场景合成方法,以参与者的空间作为输入,并生成一个虚拟的合成场景,对应于所有参与者的局部空间的功能特征。我们的方法将交互功能优化模块与深度学习条件场景增强过程相结合,以生成混合现实远程呈现场景的所有参与者都可以相互且物理访问的场景。合成的场景具有可行走、可坐下和可工作的功能,这些功能都对应于用户真实环境中的物理对象。我们使用MatterPort3D数据集进行实验,并进行比较用户研究,以评估我们的系统的有效性。我们的研究结果表明,我们提出的方法可以是一个很有前途的研究方向,促进下一代空间计算平台的上下文临场感系统。

英文摘要:

Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wide spectrum of activities, previously not possible in 2D screen-based communication methods. However, as mixed reality experiences are limited to the local physical surrounding of each user, finding a common virtual ground where users can freely move and interact with each other is challenging. In this paper, we propose a novel mutual scene synthesis method that takes the participants' spaces as input, and generates a virtual synthetic scene that corresponds to the functional features of all participants' local spaces. Our method combines a mutual function optimization module with a deep-learning conditional scene augmentation process to generate a scene mutually and physically accessible to all participants of a mixed reality telepresence scenario. The synthesized scene can hold mutual walkable, sittable and workable functions, all corresponding to physical objects in the users' real environments. We perform experiments using the MatterPort3D dataset and conduct comparative user studies to evaluate the effectiveness of our system. Our results show that our proposed approach can be a promising research direction for facilitating contextualized telepresence systems for next-generation spatial computing platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑