QUARTZ:面向任务型对话摘要的基于问答的无监督抽象式精炼
QUARTZ : QA-based Unsupervised Abstractive Refinement for Task-oriented Dialogue Summarization
AI总结:
本文提出无监督框架 QUARTZ,利用 LLM 生成任务问答并据此筛选高质量对话摘要,再微调模型,在零样本设置下达到接近完全监督 SotA 的效果。
摘要:
对话摘要旨在将对话的核心含义提炼为简洁文本。这对于降低重度对话应用中固有的复杂性和噪声至关重要。尽管近期方法通常训练语言模型模仿人工撰写的摘要,但此类监督成本高昂,且往往导致输出缺乏面向特定任务的聚焦,从而限制其在医疗等下游应用中的有效性。本文提出 QUARTZ,一个面向任务型、基于效用的对话摘要框架。QUARTZ 首先利用一组大语言模型(LLM),以零样本方式从一段对话中生成多个摘要和面向任务的问答对。随后,通过让 LLM 回答任务相关问题来评估所生成摘要的质量,再(i)选择最佳候选答案,并(ii)基于这些答案识别信息量最高的摘要。最后,我们在所选择的摘要上微调最佳 LLM。在多个数据集上验证时,QUARTZ 在多种零样本设置下取得了具有竞争力的结果,可媲美完全监督的最先进(SotA)方法,证明了其有效性。
英文摘要:
Dialogue summarization aims to distill the core meaning of a conversation into a concise text. This is crucial for reducing the complexity and noise inherent in dialogue-heavy applications. While recent approaches typically train language models to mimic human-written summaries, such supervision is costly and often results in outputs that lack task-specific focus limiting their effectiveness in downstream applications, such as medical tasks. In this paper, we propose \app, a framework for task-oriented utility-based dialogue summarization. \app starts by generating multiple summaries and task-oriented question-answer pairs from a dialogue in a zero-shot manner using a pool of large language models (LLMs). The quality of the generated summaries is evaluated by having LLMs answer task-related questions before \textit{(i)} selecting the best candidate answers and \textit{(ii)} identifying the most informative summary based on these answers. Finally, we fine-tune the best LLM on the selected summaries. When validated on multiple datasets, \app demonstrates its effectiveness by achieving competitive results in various zero-shot settings, rivaling fully-supervised State-of-the-Art (SotA) methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。