CUIfy the XR：一个在XR中嵌入基于LLM的对话智能体的开源包
CUIfy the XR: An Open-Source Package to Embed LLM-powered Conversational Agents in XR
AI总结：
本文提出开源Unity包CUIfy，通过集成LLM、STT和TTS模型，在XR环境中实现低延迟、多NPC的语音交互，以构建更自然的对话用户界面。
摘要：
计算机图形学、机器学习和传感器技术的最新进展，为日常生活的扩展现实（XR）设置提供了从技能训练到娱乐等众多机会。随着大型公司提供经济实惠的消费级头戴式显示器（HMDs），XR很可能会变得普及，HMDs将像智能手机和平板电脑一样发展成为个人设备。然而，在XR中拥有智能空间和自然交互与技术进步同样重要，以便用户增加在虚拟和增强空间中的参与度。为此，由大型语言模型（LLM）驱动并配备语音转文本（STT）和文本转语音（TTS）模型的非玩家角色（NPCs），相较于传统或预编写脚本的NPCs，在促进XR中更自然的对话用户界面（CUIs）方面带来了显著优势。本文为社区提供了一个开源、可定制、可扩展且注重隐私的Unity包，CUIfy，它通过广泛使用的LLM、STT和TTS模型促进了基于语音的NPC与用户交互。我们的包还支持每个环境中包含多个基于LLM的NPC，并通过流式传输最小化不同计算模型之间的延迟，以实现用户与NPC之间可用的交互。我们在以下存储库发布源代码：https://gitlab.lrz.de/hctl/cuify
英文摘要：
Recent developments in computer graphics, machine learning, and sensor technologies enable numerous opportunities for extended reality (XR) setups for everyday life, from skills training to entertainment. With large corporations offering affordable consumer-grade head-mounted displays (HMDs), XR will likely become pervasive, and HMDs will develop as personal devices like smartphones and tablets. However, having intelligent spaces and naturalistic interactions in XR is as important as technological advances so that users grow their engagement in virtual and augmented spaces. To this end, large language model (LLM)--powered non-player characters (NPCs) with speech-to-text (STT) and text-to-speech (TTS) models bring significant advantages over conventional or pre-scripted NPCs for facilitating more natural conversational user interfaces (CUIs) in XR. This paper provides the community with an open-source, customizable, extendable, and privacy-aware Unity package, CUIfy, that facilitates speech-based NPC-user interaction with widely used LLMs, STT, and TTS models. Our package also supports multiple LLM-powered NPCs per environment and minimizes latency between different computational models through streaming to achieve usable interactions between users and NPCs. We publish our source code in the following repository: https://gitlab.lrz.de/hctl/cuify
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
