应对人机交互中的共识裂痕：研究与基准
Navigating Rifts in Human-LLM Grounding: Study and Benchmark
AI总结：
本研究通过分析三个数据集揭示了LLM在共识建立上远逊于人类，并推出Rifts基准评估该缺陷，同时提出初步干预措施以缓解交互失败。
摘要：
语言模型擅长遵循指令，但往往难以掌握人类自然运用的协作式对话技巧。这种在共识建立（即对话参与者建立相互理解的过程）上的局限，可能导致用户沮丧，甚至在关键场景中引发严重后果。为系统研究人机交互中的共识挑战，我们分析了三个真人助手数据集的日志：WildChat、MultiWOZ 和 Bing Chat。我们构建了共识行为分类体系，并建立模型以标注和预测共识行为。研究揭示了人机共识与人人共识的显著差异：LLM 主动发起澄清的可能性比人类低三倍，提出后续请求的可能性低十六倍。此外，我们发现早期的共识失败能预测后续的交互崩溃。基于这些洞察，我们推出了 Rifts，一个源自公开 LLM 交互数据的基准，包含 LLM 未能发起共识的场景。我们注意到当前前沿模型在 Rifts 上表现不佳，凸显了重新考量如何训练和提示 LLM 以适应人类交互的必要性。为此，我们开发了一项旨在缓解共识失败的初步干预措施。
英文摘要：
Language models excel at following instructions but often struggle with the collaborative aspects of conversation that humans naturally employ. This limitation in grounding -- the process by which conversation participants establish mutual understanding -- can lead to outcomes ranging from frustrated users to serious consequences in high-stakes scenarios. To systematically study grounding challenges in human-LLM interactions, we analyze logs from three human-assistant datasets: WildChat, MultiWOZ, and Bing Chat. We develop a taxonomy of grounding acts and build models to annotate and forecast grounding behavior. Our findings reveal significant differences in human-human and human-LLM grounding: LLMs were three times less likely to initiate clarification and sixteen times less likely to provide follow-up requests than humans. Additionally, we find that early grounding failures predict later interaction breakdowns. Building on these insights, we introduce Rifts, a benchmark derived from publicly available LLM interaction data containing situations where LLMs fail to initiate grounding. We note that current frontier models perform poorly on Rifts, highlighting the need to reconsider how we train and prompt LLMs for human interaction. To this end, we develop a preliminary intervention aimed at mitigating grounding failures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
