PoE：通用自动对话评估专家小组
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
摘要：
预计聊天机器人将在多个领域拥有丰富的知识，例如日常聊天，信息交换和情感情况下的接地。为了有效地衡量这样的会话代理的质量，基于模型的自动对话评估指标（ADEM），预计将在多个领域表现良好。尽管取得了重大进展，但在一个领域运作良好的ADEM不一定推广到另一个领域。这就需要一个专用的领域泛化网络架构。为了解决多域对话评估任务，我们提出了一个专家小组（PoE），一个多任务网络，由一个共享的Transformer编码器和一个轻量级适配器的集合。共享编码器捕获跨域对话的一般知识，而每个适配器专门用于一个特定的域并充当域专家。为了验证这个想法，我们构建了一个高质量的多域对话数据集，利用数据增强和伪标签。PoE网络在16个对话评估数据集上进行了全面评估，这些数据集涵盖了广泛的对话领域。它在所有评估数据集的平均斯皮尔曼相关性方面达到了最先进的性能。它表现出更好的zero-shot泛化比现有的国家的最先进的ADEM和能力，很容易适应新的领域与Few-Shot迁移学习。
英文摘要：
Chatbots are expected to be knowledgeable across multiple domains, e.g. for daily chit-chat, exchange of information, and grounding in emotional situations. To effectively measure the quality of such conversational agents, a model-based automatic dialogue evaluation metric (ADEM) is expected to perform well across multiple domains. Despite significant progress, an ADEM that works well in one domain does not necessarily generalize to another. This calls for a dedicated network architecture for domain generalization. To tackle the multi-domain dialogue evaluation task, we propose a Panel of Experts (PoE), a multitask network that consists of a shared transformer encoder and a collection of lightweight adapters. The shared encoder captures the general knowledge of dialogues across domains, while each adapter specializes in one specific domain and serves as a domain expert. To validate the idea, we construct a high-quality multi-domain dialogue dataset leveraging data augmentation and pseudo-labeling. The PoE network is comprehensively assessed on 16 dialogue evaluation datasets spanning a wide range of dialogue domains. It achieves state-of-the-art performance in terms of mean Spearman correlation over all the evaluation datasets. It exhibits better zero-shot generalization than existing state-of-the-art ADEMs and the ability to easily adapt to new domains with few-shot transfer learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
