arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.19192cs.GR
收藏

通过LLMs和全景图像增强NPC对话的环境上下文

Empowering NPC Dialogue with Environmental Context Using LLMs and Panoramic Images

Faculty of Computer and Information Science, University of Ljubljana(卢布尔雅那大学计算机与信息科学学院)

Grega Radež, Ciril Bohak

AI总结:

本文提出利用LLMs和计算机视觉技术增强游戏NPC的环境感知能力,通过语义分割生成环境结构化数据,提升NPC对玩家行为的响应性和沉浸感。

摘要:

我们提出了一种方法,通过结合大语言模型(LLMs)和计算机视觉技术,增强游戏中的非玩家角色(NPC)的环境感知能力。传统NPC通常依赖预编写对话,缺乏空间理解,限制了对玩家行为的响应能力。本文方法通过捕捉NPC环境的全景图像,并应用语义分割来识别物体及其空间位置。提取的信息用于生成环境的结构化JSON表示,结合从分割中获得的物体位置和NPC包围球内的场景图数据,编码为方向向量。此表示作为输入提供给LLM,使NPC能够将空间知识融入玩家互动中。结果表明,NPC可以动态参考附近物体、地标和环境特征,从而产生更可信和吸引人的游戏体验。我们描述了系统的技术实现,并在两个阶段评估了其效果。首先,通过专家访谈收集反馈并确定改进点。在整合这些改进后,进行了一项用户研究,结果显示参与者更喜欢具有环境感知能力的NPC,证实了所提方法的有效性。

英文摘要:

We present an approach for enhancing non-playable characters (NPCs) in games by combining large language models (LLMs) with computer vision to provide contextual awareness of their surroundings. Conventional NPCs typically rely on pre-scripted dialogue and lack spatial understanding, which limits their responsiveness to player actions and reduces overall immersion. Our method addresses these limitations by capturing panoramic images of an NPC's environment and applying semantic segmentation to identify objects and their spatial positions. The extracted information is used to generate a structured JSON representation of the environment, combining object locations derived from segmentation with additional scene graph data within the NPC's bounding sphere, encoded as directional vectors. This representation is provided as input to the LLM, enabling NPCs to incorporate spatial knowledge into player interactions. As a result, NPCs can dynamically reference nearby objects, landmarks, and environmental features, leading to more believable and engaging gameplay. We describe the technical implementation of the system and evaluate it in two stages. First, an expert interview was conducted to gather feedback and identify areas for improvement. After integrating these refinements, a user study was performed, showing that participants preferred the context-aware NPCs over a non-context-aware baseline, confirming the effectiveness of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑