谁在与智能体对话？多用户3D虚拟环境中的LLM
Who Is Talking to the Agent? LLMs in Multi-User 3D Virtual Environments
查看机构详情
- University of Delaware（特拉华大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本研究针对多用户3D虚拟环境中LLM智能体的接收者判断与信息使用问题，构建LookAway语料库，发现说话者朝向显著提升准确率但易误导，且更多档案信息导致过度分享，需权衡空间线索与隐私控制。
中文摘要 AI 辅助
当多个人与一个LLM智能体共享一个3D虚拟房间时，该智能体不仅必须决定说什么，还必须判断某句话是否是对它说的，以及（如果可获取）可以使用在场其他人的哪些档案信息。为了研究这两个问题，我们构建了LookAway，一个包含40个会话的受控语料库，涉及80个不同人物角色和一个LLM智能体（共1200轮对话），其中包括模糊接收者轮次，在这些轮次中，说话者的朝向与预期接收者一致或冲突。在三种开放权重的大型语言模型和五种条件下（这些条件改变智能体看到的档案以及是否告知它每个人面向何处，共18,000个决策），当朝向一致时，增加说话者朝向信息将接收者识别准确率从56%提升至99.5%；但当说话者面向的人不是接收者时，其中两个模型在超过85%的这些轮次中依据说话者所面向的人来判断。警告一个模型朝向可能具有误导性，仅略微减少了这种影响。一个基于浏览器的3D演示程序实时展示了该效果：当说话者面向智能体时，同一句话会得到回答，而当他们面向另一个人时则保持沉默。提供两个人的档案改善了关于被询问者的回答，但也增加了使用未在共享对话中透露的档案属性的情况，其中一个模型达到了45.3%的回答比例。因此，更多上下文改善了多用户交互，但也导致了过度分享，所以共享LLM智能体需要机制来权衡空间线索并控制用户特定信息何时进入响应。
英文摘要
When several people share a 3D virtual room with an LLM agent, the agent must decide not only what to say, but whether an utterance was addressed to it and, if accessible, what profile information about the others present it may use. To study both problems, we construct LookAway, a controlled corpus of 40 sessions involving 80 distinct personas and an LLM agent (1,200 turns), including ambiguous-addressee turns in which speaker orientation agrees or conflicts with the intended addressee. Across three open-weight large language models and five conditions varying which profiles the agent sees and whether it is told where each person faces (18,000 decisions), adding speaker orientation increased addressee accuracy from 56% to 99.5% when orientation was congruent, but when the speaker faced someone other than the addressee, two of the models went by where the speaker faced on more than 85% of those turns. Warning one model that orientation could be misleading reduced this only modestly. A browser-based 3D demonstrator shows the effect live: the same sentence gets an answer when the speaker faces the agent and silence when they face the other person. Providing both personas' profiles improved responses about the person being asked about, but also increased the use of profile attributes not revealed in the shared conversation, reaching 45.3% of answers for one model. More context thus improves multi-user interaction but also leads to oversharing, so shared LLM agents need mechanisms for weighing spatial cues and controlling when user-specific information enters a response.
