从对话者视角重新思考开放域对话系统的响应评估
Rethinking Response Evaluation from Interlocutor's Eye for Open-Domain Dialogue Systems
AI总结：
本研究从对话者视角分析开放域对话系统自动响应评估器所需特征，实验表明对话者感知对评估至关重要，且对话连续性预测可无监督训练此类评估器。
摘要：
开放域对话系统已开始与人类进行持续对话。这些对话系统需要适应人类对话者，并从其视角进行评估。然而，当前的自动评估方法是否能近似对话者的判断仍存疑。在本研究中，我们分析并检验了从对话者视角出发，自动响应评估器需要具备哪些特征。在 Hazumi 数据集上的首次实验表明，对话者感知在使自动响应评估与对话者判断相关联方面起着关键作用。使用 X（前身为 Twitter）上的海量对话进行的第二次实验证实，对话连续性预测能够在没有人类反馈的情况下训练出具备对话者感知的响应评估器，同时也揭示了与人类响应相比，评估生成响应的困难之处。
英文摘要：
Open-domain dialogue systems have started to engage in continuous conversations with humans. Those dialogue systems are required to be adjusted to the human interlocutor and evaluated in terms of their perspective. However, it is questionable whether the current automatic evaluation methods can approximate the interlocutor's judgments. In this study, we analyzed and examined what features are needed in an automatic response evaluator from the interlocutor's perspective. The first experiment on the Hazumi dataset revealed that interlocutor awareness plays a critical role in making automatic response evaluation correlate with the interlocutor's judgments. The second experiment using massive conversations on X (formerly Twitter) confirmed that dialogue continuity prediction can train an interlocutor-aware response evaluator without human feedback while revealing the difficulty in evaluating generated responses compared to human responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
