arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.02201cs.HCcs.AIcs.CLcs.ET
收藏

你能把这些移到那边吗?一种基于大语言模型的VR移动器用于支持物体操控

Can You Move These Over There? An LLM-based VR Mover for Supporting Object Manipulation

The Hong Kong Polytechnic University(香港理工大学);University College London(伦敦大学学院);Heep Yunn School(协恩中学)

Xiangzhi Eric Wang, Zackary P. T. Sin, Ye Jia, Daniel Archer, Wynonna H. Y. Fong, Qing Li, Chen Li

AI总结:

本文提出VR Mover,一种基于大语言模型的VR物体操控方案,通过语音和指向实现自然交互,实验证明其能提升多物体操控体验与效率并降低负荷。

摘要:

在日常生活中,我们可以自然地使用语言和手势传达物体空间操控的指令。将这种交互形式移植到虚拟现实(VR)物体操控中可能是有益的。我们提出VR Mover,这是一种由大语言模型(LLM)赋能的解决方案,能够理解并解读用户的语音指令以支持物体操控。通过简单的指向和说话,LLM可以在无需结构化输入的情况下操控物体。我们的用户研究表明,VR Mover在多物体操控方面提升了用户的可用性、整体体验和表现,同时减少了工作负荷和手臂疲劳。用户偏好所提出的自然界面进行大范围移动,并可能互补性地切换到gizmos或虚拟手进行更精细的调整。这些发现被认为有助于为未来基于LLM的物体操控界面提供设计启示,凸显了在VR环境中实现更直观、更高效用户交互的潜力。

英文摘要:

In our daily lives, we can naturally convey instructions for the spatial manipulation of objects using words and gestures. Transposing this form of interaction into virtual reality (VR) object manipulation can be beneficial. We propose VR Mover, an LLM-empowered solution that can understand and interpret the user's vocal instruction to support object manipulation. By simply pointing and speaking, the LLM can manipulate objects without structured input. Our user study demonstrates that VR Mover enhances user usability, overall experience and performance on multi-object manipulation, while also reducing workload and arm fatigue. Users prefer the proposed natural interface for broad movements and may complementarily switch to gizmos or virtual hands for finer adjustments. These findings are believed to contribute to design implications for future LLM-based object manipulation interfaces, highlighting the potential for more intuitive and efficient user interactions in VR environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑