人类与大语言模型对话中的用户反馈：理解用户的视角，但作为学习信号存在噪声
User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal
AI总结：
该研究针对直接索要用户反馈干扰性强的问题，分析WildChat、LMSYS数据集里人机对话的隐式用户反馈，探究结合反馈内容能否提升模型性能，发现其在短问题上有效、长复杂问题上无效，明确了该反馈的潜力与局限。
摘要：
语言模型（LM）部署后可与用户长期交互，理想情况下能基于用户反馈迭代进化。直接索要用户反馈会造成干扰，因此我们研究从用户-LM交互日志中挖掘隐式用户反馈。我们研究了两个用户-LM交互数据集（WildChat和LMSYS）：首先，我们分析用户-大语言模型对话日志中的用户反馈，揭示此类反馈的发生时机与原因；其次，我们研究从这类隐式用户反馈中提取学习信号，具体探究除反馈极性外，加入用户反馈内容（如用户希望得到澄清）能否提升模型性能。实验结果好坏参半：该方法在人工设计的短问题（MTBench）上有效，但在更长更复杂的问题（WildBench）上无增益。综上，我们对隐式用户反馈开展了深入研究，阐明了其潜力与局限性。
英文摘要：
Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus, we study harvesting implicit user feedback from user-LM interaction logs. We study two user-LM interaction datasets (WildChat and LMSYS). First, we analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs. Second, we study harvesting learning signals from such implicit user feedback. Specifically, we study whether incorporating the contents of user feedback (e.g., user wanted clarification), in addition to the polarity of the feedback, can improve the model performance. We observe mixed results, showing this helps in short human-designed questions (MTBench) but not on longer and more complex questions (WildBench). Together, we provide an in-depth study of implicit user feedback, showing its potential and limitations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
