VirtuWander:通过大语言模型增强虚拟导览中的多模态交互
VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models
AI总结:
本文提出VirtuWander系统,利用面向领域的大语言模型将用户查询转化为多样化导览情境并促进多模态交互,通过用户研究验证其能增强虚拟博物馆导览的参与感与个性化体验。
摘要:
虚拟博物馆中的导览鼓励多模态交互,以提升用户在参与感、沉浸感和空间感知方面的体验。然而,由于理解多样化用户需求和满足个性化用户偏好的复杂性,实现这一目标具有挑战性。基于一项刻画导览寻求情境的形成性研究,我们建立了一个面向虚拟导览的多模态交互设计框架。随后,我们设计了VirtuWander,这是一个两阶段的创新系统,利用面向领域的大语言模型将用户查询转化为多样化的导览寻求情境,并促进多模态交互。通过涵盖多种导览场景并满足个性化偏好的虚拟导览示例,展示了VirtuWander的可行性和通用性。我们进一步在沉浸式模拟博物馆中通过用户研究对VirtuWander进行了评估。结果表明,我们的系统通过个性化沟通和知识丰富的辅助增强了沉浸式虚拟导览体验,表明其具有扩展到真实世界场景的潜力。
英文摘要:
Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。