利用对话部署数据中的隐式反馈
Leveraging Implicit Feedback from Deployment Data in Dialogue
AI总结：
本研究利用对话部署数据中用户回复长度、情感等隐式反馈信号改进社交对话智能体，实验基于BlenderBot公开数据，人工评估显示模型有提升，但不同代理信号存在利弊差异。
摘要：
我们研究通过学习用户与已部署模型之间的自然对话来改进社交对话智能体，无需额外标注。为隐式衡量机器生成话语的质量，我们利用收集到的对话片段中未来人类话语的用户回复长度、情感和反应等信号。我们的实验使用了BlenderBot（Xu等人，2023）公开发布的部署数据。人工评估表明，我们的新模型相比基线回复有所提升；但我们发现，部分代理信号也会导致更多带有不良属性的生成内容。例如，与基线相比，优化对话长度会产生更多有争议或不友好的生成内容，而优化积极情感或反应则能减少这类行为。
英文摘要：
We study improving social conversational agents by learning from natural dialogue between users and a deployed model, without extra annotations. To implicitly measure the quality of a machine-generated utterance, we leverage signals like user response length, sentiment and reaction of the future human utterances in the collected dialogue episodes. Our experiments use the publicly released deployment data from BlenderBot (Xu et al., 2023). Human evaluation indicates improvements in our new models over baseline responses; however, we find that some proxy signals can lead to more generations with undesirable properties as well. For example, optimizing for conversation length can lead to more controversial or unfriendly generations compared to the baseline, whereas optimizing for positive sentiment or reaction can decrease these behaviors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
