意义增强表示对对话任务中语言生成的影响：对任务、语料和指标相关性的全面探索
Impact of enriched meaning representations for language generation in dialogue tasks: A comprehensive exploration of the relevance of tasks, corpora and metrics
AI总结：
本文研究了在对话任务中，通过提供任务示范器是否能提升微调模型的生成质量，分析了不同领域、语料特征和评估指标对生成质量的影响，发现意义增强输入在复杂任务和小数据集上效果显著，且语义指标比词汇指标更准确。
摘要：
对话系统应生成多样化的语言形式以与用户流畅准确地交互。在这一背景下，自然语言生成（NLG）引擎将意义表示（MRs）转换为句子，直接影响用户感知。这些MRs通常通过DAs编码沟通功能（如告知、请求、确认），并通过槽值对枚举语义内容。本文旨在分析是否为生成器提供任务示范器能增强微调模型的生成效果。该示范器是从原始数据集中提取的MR-句子对，可在训练和推理时增强输入。分析涉及五个关注不同语言方面的指标，以及四个在多个特征上不同的数据集，如领域、大小、词典、MR变化性和获取过程。据我们所知，这是首次对对话NLG进行研究，比较了MR对生成质量在不同领域、语料特征和评估指标上的影响。我们的关键发现是，所提出的增强输入对复杂任务和具有高MR和句子变化性的小数据集有效，且在零样本设置下对任何领域都有益处。此外，指标分析显示，语义指标比词汇指标更能准确捕捉生成质量。此外，在这些语义指标中，那些用人类评分训练的指标能检测到遗漏和其他细微的语义问题，而基于嵌入的指标常会遗漏。最后，指标分数的演变和Slot Accuracy和Dialogue Act Accuracy的优秀结果表明，生成模型在不同任务中表现出快速适应性和在语义和沟通意图层面的鲁棒性。
英文摘要：
Conversational systems should generate diverse language forms to interact fluently and accurately with users. In this context, Natural Language Generation (NLG) engines convert Meaning Representations (MRs) into sentences, directly influencing user perception. These MRs usually encode the communicative function (e.g., inform, request, confirm) via DAs and enumerate the semantic content with slot-value pairs. In this work, our objective is to analyse whether providing a task demonstrator to the generator enhances the generations of a fine-tuned model. This demonstrator is an MR-sentence pair extracted from the original dataset that enriches the input at training and inference time. The analysis involves five metrics that focus on different linguistic aspects, and four datasets that differ in multiple features, such as domain, size, lexicon, MR variability, and acquisition process. To the best of our knowledge, this is the first study on dialogue NLG implementing a comparative analysis of the impact of MRs on generation quality across domains, corpus characteristics, and the metrics used to evaluate these generations. Our key insight is that the proposed enriched inputs are effective for complex tasks and small datasets with high variability in MRs and sentences. They are also beneficial in zero-shot settings for any domain. Moreover, the analysis of the metrics shows that semantic metrics capture generation quality more accurately than lexical metrics. In addition, among these semantic metrics, those trained with human ratings can detect omissions and other subtle semantic issues that embedding-based metrics often miss. Finally, the evolution of the metric scores and the excellent results for Slot Accuracy and Dialogue Act Accuracy demonstrate that the generative models present fast adaptability to different tasks and robustness at semantic and communicative intention levels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
