arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.19988cs.MM
收藏

HeadsetOff:在经济型VR头显上实现照片级真实感视频会议

HeadsetOff: Enabling Photorealistic Video Conferencing on Economical VR Headsets

Yili Jin, Xize Duan, Fangxin Wang, Xue Liu

AI总结:

针对VR头显遮挡面部导致视频会议受限的问题,提出HeadsetOff系统,通过语音驱动面部重建及多模态预测、生成、自适应控制三大组件,在经济型VR头显上实现高质量低延迟的照片级真实感视频会议。

摘要:

虚拟现实(VR)在远程协作中日益普及,但当用户面部被头显遮挡时,视频会议面临挑战。现有解决方案在可及性方面存在局限。本文提出HeadsetOff这一新型系统,它借助语音驱动的面部重建技术,在经济型VR头显上实现照片级真实感的视频会议。HeadsetOff包含三大核心组件:多模态预测器、生成器和自适应控制器。预测器可基于不同模态有效预测用户的后续行为;生成器利用语音、头部动作和眨眼来驱动人脸动画;自适应控制器则根据视频质量与延迟之间的权衡,动态选择合适的生成器模型。实验结果表明,HeadsetOff可在经济型VR头显上有效实现高质量、低延迟的视频会议。

英文摘要:

Virtual Reality (VR) has become increasingly popular for remote collaboration, but video conferencing poses challenges when the user's face is covered by the headset. Existing solutions have limitations in terms of accessibility. In this paper, we propose HeadsetOff, a novel system that achieves photorealistic video conferencing on economical VR headsets by leveraging voice-driven face reconstruction. HeadsetOff consists of three main components: a multimodal predictor, a generator, and an adaptive controller. The predictor effectively predicts user future behavior based on different modalities. The generator employs voice, head motion, and eye blink to animate the human face. The adaptive controller dynamically selects the appropriate generator model based on the trade-off between video quality and delay. Experimental results demonstrate the effectiveness of HeadsetOff in achieving high-quality, low-latency video conferencing on economical VR headsets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑